Computational Methods for Nanoscale Applications
Nanostructure Science and Technology Series Editor: David J. Lockwood, FRSC National Research Council of Canada Ottawa, Ontario, Canada Current volumes in this series: Functional Nanostructures: Processing, Characterization and Applications Edited by Sudipta Seal Light Scattering and Nanoscale Surface Roughness Edited by Alexei A. Maradudin Nanotechnology for Electronic Materials and Devices Edited by Anatoli Korkin, Evgeni Gusev, and Jan K. Labanowski Nanotechnology in Catalysis, Volume 3 Edited by Bing Zhou, Scott Han, Robert Raja, and Gabor A. Somorjai Nanostructured Coatings Edited by Albano Cavaleiro and Jeff T. De Hosson Self-Organized Nanoscale Materials Edited by Motonari Adachi and David J. Lockwood Controlled Synthesis of Nanoparticles in Microheterogeneous Systems Vincenzo Turco Liveri Nanoscale Assembly Techniques Edited by Wilhelm T.S. Huck Ordered Porous Nanostructures and Applications Edited by Ralf B. Wehrspohn Surface Effects in Magnetic Nanoparticles Dino Fiorani Interfacial Nanochemistry: Molecular Science and Engineering at Liquid-Liquid Interfaces Edited by Hitoshi Watarai Nanoscale Structure and Assembly at Solid-Fluid Interfaces Edited by Xiang Yang Liu and James J. De Yoreo Introduction to Nanoscale Science and Technology Edited by Massimiliano Di Ventra, Stephane Evoy, and James R. Heflin Jr. Alternative Lithography: Unleashing the Potentials of Nanotechnology Edited by Clivia M. Sotomayor Torres Semiconductor Nanocrystals: From Basic Principles to Applications Edited by Alexander L. Efros, David J. Lockwood, and Leonid Tsybeskov Nanotechnology in Catalysis, Volumes 1 and 2 Edited by Bing Zhou, Sophie Hermans, and Gabor A. Somorjai (Continued after index)
Igor Tsukerman
Computational Methods for Nanoscale Applications Particles, Plasmons and Waves
123
Igor Tsukerman Department of Electrical and Computer Engineering The University of Akron Akron, OH 44325-3904 USA
[email protected]
Series Editor David J. Lockwood National Research Council of Canada Ottawa, Ontario Canada
ISBN: 978-0-387-74777-4 e-ISBN: 978-0-387-74778-1 DOI: 10.1007/978-0-387-74778-1 Library of Congress Control Number: 2007935245 c 2008 Springer Science+Business Media, LLC All rights reserved. This work may not be translated or copied in whole or in part without the written permission of the publisher (Springer Science+Business Media, LLC, 233 Spring Street, New York, NY 10013, USA), except for brief excerpts in connection with reviews or scholarly analysis. Use in connection with any form of information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed is forbidden. The use in this publication of trade names, trademarks, service marks, and similar terms, even if they are not identified as such, is not to be taken as an expression of opinion as to whether or not they are subject to proprietary rights. Cover Illustration: Real part of the electric field phasor in the Fujisawa-Koshiba photonic waveguide bend. From “Electromagnetic Applications of a New Finite-Difference Calculus”, by Igor Tsukerman, IEEE Transactions on Magnetics, Vol. 41, No. 7, pp. 2206–2225, 2005. c 2005 IEEE (by permission). Printed on acid-free paper. 9 8 7 6 5 4 3 2 1 springer.com
To the memory of my mother, to my father, and to the miracle of M.
Preface
The purpose of this note . . . is to sort out my own thoughts . . . and to solicit ideas from others. Lloyd N. Trefethen Three mysteries of Gaussian elimination
Nobody reads prefaces. Therefore my preference would have been to write a short one that nobody will read rather than a long one that nobody will read. However, I ought to explain, as briefly as possible, the main motivation for writing the book and to thank – as fully and sincerely as possible – many people who have contributed to this writing in a variety of ways. My motivation has selfish and unselfish components. The unselfish part is to present the elements of computational methods and nanoscale simulation to researchers, scientists and engineers who are not necessarily experts in computer simulation. I am hopeful, though, that parts of the book will also be of interest to experts, as further discussed in the Introduction and Conclusion. The selfish part of my motivation is articulated in L. N. Trefethen’s quote above. Whether or not I have succeeded in “sorting out my own thoughts” is not quite clear at the moment, but I would definitely welcome “ideas from others,” as well as comments and constructive criticism. *
*
*
I owe an enormous debt of gratitude to my parents for their incredible kindness and selflessness, and to my wife for her equally incredible tolerance of my character quirks and for her unwavering support under all circumstances. My son (who is a business major at The Ohio State University) proofread parts of the book, replaced commas with semicolons, single quotes with double quotes, and fixed my other egregious abuses of the English language.
VIII
Preface
Overall, my work on the book would have been an utterly pleasant experience had it not been interrupted by the sudden and heartbreaking death of my mother in the summer of 2006. I do wish to dedicate this book to her memory. Acknowledgment and Thanks Collaboration with Gary Friedman and his group, especially during my sabbatical in 2002–2003 at Drexel University, has influenced my research and the material of this book greatly. Gary’s energy, enthusiasm and innovative ideas are always very stimulating. During the same sabbatical year, I was fortunate to visit several research groups working on the simulation of colloids, polyelectrolytes, macro- and biomolecules. I am very grateful to all of them for their hospitality. I would particularly like to mention Christian Holm, Markus Deserno and Vladimir Lobaskin at the Max-Planck-Institut f¨ ur Polymerforschung in Mainz, Germany; Rebecca Wade at the European Molecular Biology Laboratory in Heidelberg, and Thomas Simonson at the Laboratoire de Biologie Structurale in Strasbourg, France. Alexei Sokolov’s advanced techniques and experiments in optical sensors and microscopy with molecular-scale resolution had a strong impact on my students’ and my work over the last several years. I thank Alexei for providing a great opportunity for collaborative work with his group at the Department of Polymer Science, the University of Akron. In the course of the last two decades, I have benefited enormously from my ´ communication with Alain Bossavit (Electricit´ e de France and Laboratoire de Genie Electrique de Paris), from his very deep knowledge of all aspects of computational electromagnetism, and from his very detailed and thoughtful analysis of any difficult subject that would come up. Isaak Mayergoyz of the University of Maryland at College Park has on many occasions shared his valuable insights with me. His knowledge of many areas of electromagnetism, physics and mathematics is very profound and often unmatched. My communication with Jon Webb (McGill University, Montr´eal) has always been thought-provoking and informative. His astute observations and comments make complicated matters look clear and simple. I was very pleased that Professor Webb devoted part of his sabbatical leave to our joint research on Flexible Local Approximation MEthods (FLAME, Chapter 4). Yuri Kizimovich (Plassotech Corp., California) and I have worked jointly on a variety of projects over the last 25 years. His original thinking and elegant solutions of practical problems have always been a great asset. Yury’s help and long-term collaboration are greatly appreciated. Even though over 20 years have already passed since the untimely death of my thesis advisor, Yu.V. Rakitskii, his students still remember very warmly
Preface
IX
his relentless strive for excellence and quixotic attitude to scientific research. Rakitskii’s main contribution was to numerical methods for stiff systems of differential equations. He was guided by the idea of incorporating, to the extent possible, analytical approximations into numerical methods. This approach is manifest in FLAME that I believe Rakitskii would have liked. My sincere thanks go to • • • • • • • • • • • • •
Dmitry Golovaty (The University of Akron), for his help on many occasions and for interesting discussions. Viacheslav Dombrovski, a scientist of incomparable erudition, for many pearls of wisdom. Elena Ivanova and Sergey Voskoboynikov (Technical University of St. Petersburg, Russia), for their very, very diligent work on FLAME. Benjamin Yellen (Duke University), for many discussions, innovative ideas, and for his great contribution to the NSF-NIRT project on magnetic assembly of particles. Mark Stockman (Georgia State University), for sharing his very deep and broad knowledge and expertise in many areas of plasmonics and nanophotonics. J. Douglas Lavers (the University of Toronto), for his help, cooperation and continuing support over many years. Fritz Keilmann (the Max-Planck-Institut f¨ ur Biochemie in Martinsried, Germany), for providing an excellent opportunity for collaboration on problems in infrared microscopy. Boris Shoykhet (Rockwell Automation), an excellent engineer, mathematician and finite element analyst, for many valuable discussions. Nicolae-Alexandru Nicorovici (University of Technology, Sydney, Australia), for his deep and detailed comments on “cloaking,” metamaterials, and properties of photonic structures. H. Neal Bertram (UCSD – the University of California, San Diego), for his support. I have always admired Neal’s remarkable optimism and enthusiasm that make communication with him so stimulating. Adalbert Konrad (the University of Toronto) and Nathan Ida (the University of Akron) for their help and support. Pierre Asselin (Seagate, Pittsburgh) for very interesting insights, particularly in connection with a priori error estimates in finite element analysis. Sheldon Schultz (UCSD) and David Smith (UCSD and Duke) for familiarizing me with plasmonic effects a decade ago.
I appreciate the help, support and opportunities provided by the International Compumag Society through a series of the International Compumag Conferences and through personal communication with its Board and members: Jan K Sykulski, Arnulf Kost, Kay Hameyer, Fran¸cois Henrotte, Oszk´ar B´ır´ o, J.-P. Bastos, R.C. Mesquita, and others. A substantial portion of the book forms a basis of the graduate course “Simulation of Nanoscale Systems” that I developed and taught at the
X
Preface
University of Akron, Ohio. I thank my colleagues at the Department of Electrical & Computer Engineering and two Department Chairs, Alexis De Abreu Garcia and Nathan Ida, for their support and encouragement. My Ph.D. students have contributed immensely to the research, and their work is frequently referred to throughout the book. Alexander Plaks worked on adaptive multigrid methods and generalized finite element methods for electromagnetic applications. Leonid Proekt was instrumental in the development of generalized FEM, especially for the vectorial case, and of absorbing boundary conditions. Jianhua Dai has worked on generalized finite-difference ˇ methods. Frantisek Cajko developed schemes with flexible local approximation and carried out, with a great deal of intelligence and ingenuity, a variety of simulations in nano-photonics and nano-optics. I gratefully acknowledge financial support by the National Science Foundation and the NSF-NIRT program, Rockwell Automation, 3ga Corporation and Baker Hughes Corporation. NEC Europe (Sankt Augustin, Germany) provided not only financial support but also an excellent opportunity to work with Achim Basermann, an expert in high performance computing, on parallel implementation of the Generalized FEM. I thank Guy Lonsdale, Achim Basermann and Fabienne Cortial-Goutaudier for hosting me at the NEC on several occasions. A number of workshops and tutorials at the University of Minnesota in Minneapolis1 have been exceptionally interesting and educational for me. I sincerely thank the organizers: Douglas Arnold, Debra Lewis, Cheri Shakiban, Boris Shklovskii, Alexander Grosberg and others. I am very grateful to Serge Prudhomme, the reviewer of this book, for many insightful comments, numerous corrections and suggestions, and especially for his careful and meticulous analysis of the chapters on finite difference and finite element methods.2 The reviewer did not wish to remain anonymous, which greatly facilitated our communication and helped to improve the text. Further comments, suggestions and critique from the readers is very welcome and can be communicated to me directly or through the publisher. Finally, I thank Springer’s editors for their help, cooperation and patience.
1
2
Electrostatic Interactions and Biophysics, April–May 2004, Theoretical Physics Institute. Future Challenges in Multiscale Modeling and Simulation, November 2004; New Paradigms in Computation, March 2005; Effective Theories for Materials and Macromolecules, June 2005; New Directions Short Course: Quantum Computation, August 2005; Negative Index Materials, October 2006; Classical and Quantum Approaches in Molecular Modeling, July 2007 – all at the Institute for Mathematics and Its Applications, http://www.ima.umn.edu/ Serge Prudhomme is with the Institute for Computational Engineering and Sciences (ICES), formerly known as TICAM, at the University of Texas at Austin.
Contents
Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . VII 1
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.1 Why Deal with the Nanoscale? . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.2 Why Special Models for the Nanoscale? . . . . . . . . . . . . . . . . . . . . 1.3 How To Hone the Computational Tools . . . . . . . . . . . . . . . . . . . . 1.4 So What? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1 1 3 6 8
2
Finite-Difference Schemes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2 A Primer on Time-Stepping Schemes . . . . . . . . . . . . . . . . . . . . . . . 2.3 Exact Schemes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.4 Some Classic Schemes for Initial Value Problems . . . . . . . . . . . . 2.4.1 The Runge–Kutta Methods . . . . . . . . . . . . . . . . . . . . . . . . . 2.4.2 The Adams Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.4.3 Stability of Linear Multistep Schemes . . . . . . . . . . . . . . . . 2.4.4 Methods for Stiff Systems . . . . . . . . . . . . . . . . . . . . . . . . . . 2.5 Schemes for Hamiltonian Systems . . . . . . . . . . . . . . . . . . . . . . . . . 2.5.1 Introduction to Hamiltonian Dynamics . . . . . . . . . . . . . . . 2.5.2 Symplectic Schemes for Hamiltonian Systems . . . . . . . . . 2.6 Schemes for One-Dimensional Boundary Value Problems . . . . . 2.6.1 The Taylor Derivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.6.2 Using Constraints to Derive Difference Schemes . . . . . . . 2.6.3 Flux-Balance Schemes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.6.4 Implementation of 1D Schemes for Boundary Value Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.7 Schemes for Two-Dimensional Boundary Value Problems . . . . . 2.7.1 Schemes Based on the Taylor Expansion . . . . . . . . . . . . . 2.7.2 Flux-Balance Schemes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.7.3 Implementation of 2D Schemes . . . . . . . . . . . . . . . . . . . . . . 2.7.4 The Collatz “Mehrstellen” Schemes in 2D . . . . . . . . . . . .
11 11 12 16 18 20 24 24 27 34 34 37 39 39 40 42 46 47 47 48 50 51
XII
Contents
2.8 Schemes for Three-Dimensional Problems . . . . . . . . . . . . . . . . . . . 2.8.1 An Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.8.2 Schemes Based on the Taylor Expansion in 3D . . . . . . . . 2.8.3 Flux-Balance Schemes in 3D . . . . . . . . . . . . . . . . . . . . . . . . 2.8.4 Implementation of 3D Schemes . . . . . . . . . . . . . . . . . . . . . . 2.8.5 The Collatz “Mehrstellen” Schemes in 3D . . . . . . . . . . . . 2.9 Consistency and Convergence of Difference Schemes . . . . . . . . . 2.10 Summary and Further Reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
55 55 55 56 57 58 59 64
The Finite Element Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69 3.1 Everything is Variational . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69 3.2 The Weak Formulation and the Galerkin Method . . . . . . . . . . . . 75 3.3 Variational Methods and Minimization . . . . . . . . . . . . . . . . . . . . . 81 3.3.1 The Galerkin Solution Minimizes the Error . . . . . . . . . . . 81 3.3.2 The Galerkin Solution and the Energy Functional . . . . . 82 3.4 Essential and Natural Boundary Conditions . . . . . . . . . . . . . . . . . 83 3.5 Mathematical Notes: Convergence, Lax–Milgram and C´ea’s Theorems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86 3.6 Local Approximation in the Finite Element Method . . . . . . . . . 89 3.7 The Finite Element Method in One Dimension . . . . . . . . . . . . . . 91 3.7.1 First-Order Elements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91 3.7.2 Higher-Order Elements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102 3.8 The Finite Element Method in Two Dimensions . . . . . . . . . . . . . 105 3.8.1 First-Order Elements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105 3.8.2 Higher-Order Triangular Elements . . . . . . . . . . . . . . . . . . . 120 3.9 The Finite Element Method in Three Dimensions . . . . . . . . . . . . 122 3.10 Approximation Accuracy in FEM . . . . . . . . . . . . . . . . . . . . . . . . . . 123 3.11 An Overview of System Solvers . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129 3.12 Electromagnetic Problems and Edge Elements . . . . . . . . . . . . . . 139 3.12.1 Why Edge Elements? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139 3.12.2 The Definition and Properties of Whitney-N´ed´elec Elements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142 3.12.3 Implementation Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145 3.12.4 Historical Notes on Edge Elements . . . . . . . . . . . . . . . . . . 146 3.12.5 Appendix: Several Common Families of Tetrahedral Edge Elements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147 3.13 Adaptive Mesh Refinement and Multigrid Methods . . . . . . . . . . 148 3.13.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148 3.13.2 Hierarchical Bases and Local Refinement . . . . . . . . . . . . . 149 3.13.3 A Posteriori Error Estimates . . . . . . . . . . . . . . . . . . . . . . . 151 3.13.4 Multigrid Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154 3.14 Special Topic: Element Shape and Approximation Accuracy . . 158 3.14.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 158 3.14.2 Algebraic Sources of Shape-Dependent Errors: Eigenvalue and Singular Value Conditions . . . . . . . . . . . . 160
Contents
XIII
3.14.3 Geometric Implications of the Singular Value Condition 171 3.14.4 Condition Number and Approximation . . . . . . . . . . . . . . . 179 3.14.5 Discussion of Algebraic and Geometric a priori Estimates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 180 3.15 Special Topic: Generalized FEM . . . . . . . . . . . . . . . . . . . . . . . . . . . 181 3.15.1 Description of the Method . . . . . . . . . . . . . . . . . . . . . . . . . . 181 3.15.2 Trade-offs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183 3.16 Summary and Further Reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . 184 3.17 Appendix: Generalized Curl and Divergence . . . . . . . . . . . . . . . . 186 4
Flexible Local Approximation MEthods (FLAME) . . . . . . . . . 189 4.1 A Preview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189 4.2 Perspectives on Generalized FD Schemes . . . . . . . . . . . . . . . . . . . 191 4.2.1 Perspective #1: Basis Functions Not Limited to Polynomials . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 191 4.2.2 Perspective #2: Approximating the Solution, Not the Equation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192 4.2.3 Perspective #3: Multivalued Approximation . . . . . . . . . . 193 4.2.4 Perspective #4: Conformity vs. Flexibility . . . . . . . . . . . . 193 4.2.5 Why Flexible Approximation? . . . . . . . . . . . . . . . . . . . . . . 195 4.2.6 A Preliminary Example: the 1D Laplace Equation . . . . . 197 4.3 Trefftz Schemes with Flexible Local Approximation . . . . . . . . . . 198 4.3.1 Overlapping Patches . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 198 4.3.2 Construction of the Schemes . . . . . . . . . . . . . . . . . . . . . . . . 200 4.3.3 The Treatment of Boundary Conditions . . . . . . . . . . . . . . 202 4.3.4 Trefftz–FLAME Schemes for Inhomogeneous and Nonlinear Equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 203 4.3.5 Consistency and Convergence of the Schemes . . . . . . . . . 205 4.4 Trefftz–FLAME Schemes: Case Studies . . . . . . . . . . . . . . . . . . . . . 206 4.4.1 1D Laplace, Helmholtz and Convection-Diffusion Equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 206 4.4.2 The 1D Heat Equation with Variable Material Parameter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 207 4.4.3 The 2D and 3D Laplace Equation . . . . . . . . . . . . . . . . . . . 208 4.4.4 The Fourth Order 9-point Mehrstellen Scheme for the Laplace Equation in 2D . . . . . . . . . . . . . . . . . . . . . . . . . . . . 209 4.4.5 The Fourth Order 19-point Mehrstellen Scheme for the Laplace Equation in 3D . . . . . . . . . . . . . . . . . . . . . . . . . 210 4.4.6 The 1D Schr¨ odinger Equation. FLAME Schemes by Variation of Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . 210 4.4.7 Super-high-order FLAME Schemes for the 1D Schr¨ odinger Equation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 212 4.4.8 A Singular Equation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213 4.4.9 A Polarized Elliptic Particle . . . . . . . . . . . . . . . . . . . . . . . . 215 4.4.10 A Line Charge Near a Slanted Boundary . . . . . . . . . . . . . 216 4.4.11 Scattering from a Dielectric Cylinder . . . . . . . . . . . . . . . . 217
XIV
Contents
4.5 Existing Methods Featuring Flexible or Nonstandard Approximation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 219 4.5.1 The Treatment of Singularities in Standard FEM . . . . . . 221 4.5.2 Generalized FEM by Partition of Unity . . . . . . . . . . . . . . 221 4.5.3 Homogenization Schemes Based on Variational Principles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222 4.5.4 Discontinuous Galerkin Methods . . . . . . . . . . . . . . . . . . . . 222 4.5.5 Homogenization Schemes in FDTD . . . . . . . . . . . . . . . . . . 223 4.5.6 Meshless Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 224 4.5.7 Special Finite Element Methods . . . . . . . . . . . . . . . . . . . . . 225 4.5.8 Domain Decomposition . . . . . . . . . . . . . . . . . . . . . . . . . . . . 226 4.5.9 Pseudospectral Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . 226 4.5.10 Special FD Schemes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 227 4.6 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 228 4.7 Appendix: Variational FLAME . . . . . . . . . . . . . . . . . . . . . . . . . . . . 231 4.7.1 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 231 4.7.2 The Model Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 232 4.7.3 Construction of Variational FLAME . . . . . . . . . . . . . . . . . 232 4.7.4 Summary of the Variational-Difference Setup . . . . . . . . . 235 4.8 Appendix: Coefficients of the 9-Point Trefftz–FLAME Scheme for the Wave Equation in Free Space . . . . . . . . . . . . . . . . 236 4.9 Appendix: the Fr´echet Derivative . . . . . . . . . . . . . . . . . . . . . . . . . . 237 5
Long-Range Interactions in Free Space . . . . . . . . . . . . . . . . . . . . 239 5.1 Long-Range Particle Interactions in a Homogeneous Medium . . 239 5.2 Real and Reciprocal Lattices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 242 5.3 Introduction to Ewald Summation . . . . . . . . . . . . . . . . . . . . . . . . . 243 5.3.1 A Boundary Value Problem for Charge Interactions . . . . 246 5.3.2 A Re-formulation with “Clouds” of Charge . . . . . . . . . . . 248 5.3.3 The Potential of a Gaussian Cloud of Charge . . . . . . . . . 249 5.3.4 The Field of a Periodic System of Clouds . . . . . . . . . . . . . 251 5.3.5 The Ewald Formulas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 252 5.3.6 The Role of Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . 254 5.4 Grid-based Ewald Methods with FFT . . . . . . . . . . . . . . . . . . . . . 256 5.4.1 The Computational Work . . . . . . . . . . . . . . . . . . . . . . . . . . 256 5.4.2 On Numerical Differentiation . . . . . . . . . . . . . . . . . . . . . . . 262 5.4.3 Particle–Mesh Ewald . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 264 5.4.4 Smooth Particle–Mesh Ewald Methods . . . . . . . . . . . . . . . 267 5.4.5 Particle–Particle Particle–Mesh Ewald Methods . . . . . . . 269 5.4.6 The York–Yang Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . 271 5.4.7 Methods Without Fourier Transforms . . . . . . . . . . . . . . . . 272 5.5 Summary and Further Reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . 274 5.6 Appendix: The Fourier Transform of “Periodized” Functions . . 277 5.7 Appendix: An Infinite Sum of Complex Exponentials . . . . . . . . . 278
Contents
XV
6
Long-Range Interactions in Heterogeneous Systems . . . . . . . . 281 6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 281 6.2 FLAME Schemes for Static Fields of Polarized Particles in 2D 285 6.2.1 Computation of Fields and Forces for Cylindrical Particles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 289 6.2.2 A Numerical Example: Well-Separated Particles . . . . . . . 291 6.2.3 A Numerical Example: Small Separations . . . . . . . . . . . . . 294 6.3 Static Fields of Spherical Particles in a Homogeneous Dielectric . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 303 6.3.1 FLAME Basis and the Scheme . . . . . . . . . . . . . . . . . . . . . . 303 6.3.2 A Basic Example: Spherical Particle in Uniform Field . . 306 6.4 Introduction to the Poisson–Boltzmann Model . . . . . . . . . . . . . . 309 6.5 Limitations of the PBE Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . 313 6.6 Numerical Methods for 3D Electrostatic Fields of Colloidal Particles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 314 6.7 3D FLAME Schemes for Particles in Solvent . . . . . . . . . . . . . . . . 315 6.8 The Numerical Treatment of Nonlinearity . . . . . . . . . . . . . . . . . . 319 6.9 The DLVO Expression for Electrostatic Energy and Forces . . . . 321 6.10 Notes on Other Types of Force . . . . . . . . . . . . . . . . . . . . . . . . . . . . 324 6.11 Thermodynamic Potential, Free Energy and Forces . . . . . . . . . . 328 6.12 Comparison of FLAME and DLVO Results . . . . . . . . . . . . . . . . . 332 6.13 Summary and Further Reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . 337 6.14 Appendix: Thermodynamic Potential for Electrostatics in Solvents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 338 6.15 Appendix: Generalized Functions (Distributions) . . . . . . . . . . . . 343
7
Applications in Nano-Photonics . . . . . . . . . . . . . . . . . . . . . . . . . . . 349 7.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 349 7.2 Maxwell’s Equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 349 7.3 One-Dimensional Problems of Wave Propagation . . . . . . . . . . . . 353 7.3.1 The Wave Equation and Plane Waves . . . . . . . . . . . . . . . . 353 7.3.2 Signal Velocity and Group Velocity . . . . . . . . . . . . . . . . . . 355 7.3.3 Group Velocity and Energy Velocity . . . . . . . . . . . . . . . . . 358 7.4 Analysis of Periodic Structures in 1D . . . . . . . . . . . . . . . . . . . . . . 360 7.5 Band Structure by Fourier Analysis (Plane Wave Expansion) in 1D . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 375 7.6 Characteristics of Bloch Waves . . . . . . . . . . . . . . . . . . . . . . . . . . . . 379 7.6.1 Fourier Harmonics of Bloch Waves . . . . . . . . . . . . . . . . . . . 379 7.6.2 Fourier Harmonics and the Poynting Vector . . . . . . . . . . . 380 7.6.3 Bloch Waves and Group Velocity . . . . . . . . . . . . . . . . . . . . 380 7.6.4 Energy Velocity for Bloch Waves . . . . . . . . . . . . . . . . . . . . 382 7.7 Two-Dimensional Problems of Wave Propagation . . . . . . . . . . . . 384 7.8 Photonic Bandgap in Two Dimensions . . . . . . . . . . . . . . . . . . . . . 386 7.9 Band Structure Computation: PWE, FEM and FLAME . . . . . . 389 7.9.1 Solution by Plane Wave Expansion . . . . . . . . . . . . . . . . . . 389 7.9.2 The Role of Polarization . . . . . . . . . . . . . . . . . . . . . . . . . . . 390
XVI
Contents
7.9.3 7.9.4 7.9.5 7.9.6
7.10
7.11
7.12
7.13
7.14 7.15 8
Accuracy of the Fourier Expansion . . . . . . . . . . . . . . . . . . 391 FEM for Photonic Bandgap Problems in 2D . . . . . . . . . . 393 A Numerical Example: Band Structure Using FEM . . . . 397 Flexible Local Approximation Schemes for Waves in Photonic Crystals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 401 7.9.7 Band Structure Computation Using FLAME . . . . . . . . . . 405 Photonic Bandgap Calculation in Three Dimensions: Comparison with the 2D Case . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 411 7.10.1 Formulation of the Vector Problem . . . . . . . . . . . . . . . . . . 411 7.10.2 FEM for Photonic Bandgap Problems in 3D . . . . . . . . . . 415 7.10.3 Historical Notes on the Photonic Bandgap Problem . . . . 416 Negative Permittivity and Plasmonic Effects . . . . . . . . . . . . . . . . 417 7.11.1 Electrostatic Resonances for Spherical Particles . . . . . . . 419 7.11.2 Plasmon Resonances: Electrostatic Approximation . . . . . 421 7.11.3 Wave Analysis of Plasmonic Systems . . . . . . . . . . . . . . . . . 423 7.11.4 Some Common Methods for Plasmon Simulation . . . . . . 423 7.11.5 Trefftz–FLAME Simulation of Plasmonic Particles . . . . . 426 7.11.6 Finite Element Simulation of Plasmonic Particles . . . . . . 429 Plasmonic Enhancement in Scanning Near-Field Optical Microscopy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 433 7.12.1 Breaking the Diffraction Limit . . . . . . . . . . . . . . . . . . . . . . 434 7.12.2 Apertureless and Dark-Field Microscopy . . . . . . . . . . . . . 439 7.12.3 Simulation Examples for Apertureless SNOM . . . . . . . . . 441 Backward Waves, Negative Refraction and Superlensing . . . . . . 446 7.13.1 Introduction and Historical Notes . . . . . . . . . . . . . . . . . . . 446 7.13.2 Negative Permittivity and the “Perfect Lens” Problem . 451 7.13.3 Forward and Backward Plane Waves in a Homogeneous Isotropic Medium . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 456 7.13.4 Backward Waves in Mandelshtam’s Chain of Oscillators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 459 7.13.5 Backward Waves and Negative Refraction in Photonic Crystals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 465 7.13.6 Are There Two Species of Negative Refraction? . . . . . . . 471 Appendix: The Bloch Transform . . . . . . . . . . . . . . . . . . . . . . . . . . . 477 Appendix: Eigenvalue Solvers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 478
Conclusion: “Plenty of Room at the Bottom” for Computational Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 487
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 489 Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 523
1 Introduction
Some years ago, a colleague of mine explained to me that a good presentation should address three key questions: 1) Why? (i.e. Why do it?) 2) How? (i.e. How do we do it?) and 3) So What? The following sections answer these questions, and a few more.
1.1 Why Deal with the Nanoscale? May you live in interesting times. Eric Frank Russell, “U-Turn” (1950).
The complexity and variety of applications on the nanoscale are as great, or arguably greater, than on the macroscale. While a detailed account of nanoscale problems in a single book is impossible, one can make a general observation on the importance of the nanoscale: the properties of materials are strongly affected by their nanoscale structure. Over the last two decades, mankind has been gradually inventing and acquiring means to characterize and manipulate that structure. Many remarkable effects, physical phenomena, materials and devices have already been discovered or developed: nanocomposites, carbon nanotubes, nanowires and nanodots, nanoparticles of different types, photonic crystals, and so on. On a more fundamental level, research in nanoscale physics may provide clues to the most profound mysteries of nature. “Where is the frontier of physics?”, asks L.S. Schulman in the Preface to his book [Sch97]. “Some would say 10−33 cm, some 10−15 cm and some 10+28 cm. My vote is for 10−6 cm. Two of the greatest puzzles of our age have their origins at the interface between the macroscopic and microscopic worlds. The older mystery is the thermodynamic arrow of
2
1 Introduction
time, the way that (mostly) time-symmetric microscopic laws acquire a manifest asymmetry at larger scales. And then there’s the superposition principle of quantum mechanics, a profound revolution of the twentieth century. When this principle is extrapolated to macroscopic scales, its predictions seem widely at odds with ordinary experience.” The second “puzzle” that Professor Schulman refers to is the apparent contradiction between the quantum-mechanical representation of micro-objects in a superposition of quantum states and a single unambiguous state that all of us really observe for macro-objects. Where and how exactly is this transition from the quantum world to the macro-world effected? The boundary between particle- or atomic-size quantum objects and macro-objects is on the nanoscale; that is where the “collapse of the quantum-mechanical wavefunction” from a superposition of states to one well-defined state would have to occur. Recent remarkable double-slit experiments by M. Arndt’s Quantum Nanophysics group at the University of Vienna show no evidence of “collapse” of the wavefunction and prove the wave nature of large molecules with the mass of up to 1,632 units and size up to 2 nm (tetraphenylporphyrin C44 H30 N4 and the fluorinated buckyball C60 F48 ).1 If further experiments with nanoscale objects are carried out, they will most likely confirm that the “collapse” of the wavefunction is not a fundamental physical law but only a metaphorical tool for describing the transition to the macroworld; still, such experiments will undoubtedly be captivating. Getting back to more practical aspects of nanoscale research, I illustrate its promise with one example from Chapter 7 of this book. It is well known that visible light is electromagnetic waves with the wavelengths from approximately 400 nm (violet light) to ∼700 nm (red light); green light is in the middle of this range. Thus there are approximately 2,000 wavelengths of green light per millimeter (or about 50,000 per inch). Propagation of light through a material is governed not only by the atomic-level properties but also, in many interesting and important ways, by the nanoscale/subwavelength structure of the material (i.e. the scale from 5–10 nm to a few hundred nanometers). Consider ocean waves as an analogy. A wave will easily pass around a relatively small object, such as a buoy. However, if the wave hits a long line of buoys, interesting things will start to happen: an interference pattern may emerge behind the line. Furthermore, if the buoys are arranged in a twodimensional array, possible wave patterns are richer still. Substituting an electromagnetic wave of light (say, with wavelength λ = 500 nm) for the ocean wave and a lattice of dielectric cylindrical rods (say, 200 nm in diameter) for the two-dimensional array of buoys, we get what is known as a photonic crystal.2 It is clear that the subwavelength structure 1
2
M. Arndt et al., Wave-particle duality of C60 molecules, Nature 401, 1999, pp. 680–682; http://physicsweb.org/articles/world/18/3/5. The analogy with electromagnetic waves would be closer mathematically but less intuitive if acoustic waves in the ocean were considered instead of surface waves.
1.2 Why Special Models for the Nanoscale?
3
of the crystal may bring about very interesting and unusual behavior of the wave. Even more fascinating is the possibility of controlling the propagation of light in the material by a clever design of the subwavelength structure. “Cloaking” – making objects invisible by wrapping them in a carefully designed metamaterial – has become an area of serious research (J.B. Pendry et al. [PSS06]) and has already been demonstrated experimentally in the microwave region (D. Schurig et al. [SMJ+ 06]). Guided by such material, the rays of light would bend and pass around the object as if it were not there (G. Gbur [Gbu03], J.B. Pendry et al. [PSS06], U. Leonhardt [Leo06]). A note to the reader who wishes to hide behind this cloak: if you are invisible to the outside world, the outside world is invisible to you. This follows from the reciprocity principle in electromagnetism.3 Countless other equally fascinating nanoscale applications in numerous other areas could be given. Like it or not, we live in interesting times.
1.2 Why Special Models for the Nanoscale? A good model can advance fashion by ten years. Yves Saint Laurent
First, a general observation. A simulation model consists of a physical and mathematical formulation of the problem at hand and a computational method. The formulation tells us what to solve and the computational method tells us how to solve it. Frequently more than one formulation is possible, and almost always several computational techniques are available; hence there potentially are numerous combinations of formulations and methods. Ideally, one strives to find the best such combination(s) in terms of efficiency, accuracy, robustness, algorithmic simplicity, and so on. It is not surprising that the formulations of nanoscale problems are indeed special. The scale is often too small for continuous-level macroscopic laws to be fully applicable; yet it is too large for a first-principles atomic simulation to be feasible. Computational compromises are reached in several different ways. In some cases, continuous parameters can be used with some caution and with suitable adjustments. One example is light scattering by small particles and the related “plasmonic” effects (Chapter 7), where the dielectric constant of metals or dielectrics can be adjusted to account for the size of the scatterers. In other situations, multiscale modeling is used, where a hierarchy of problems 3
Perfect invisibility is impossible even theoretically, however. With some imperfection, the effect can theoretically be achieved only in a narrow range of wavelengths. The reason is that the special metamaterials must have dispersion – i.e. their electromagnetic properties must be frequency-dependent.
4
1 Introduction
are solved and the information obtained on a finer level is passed on to the coarser ones and back. Multiscale often goes hand-in-hand with multiphysics: for example, molecular dynamics on the finest scale is combined with continuum mechanics on the macroscale. The Society for Industrial and Applied Mathematics (SIAM) now publishes a journal devoted entirely to this subject: Multiscale Modeling and Simulation, inaugurated in 2003. The applications and problems in this book have some multiscale features but can still be dealt with on a single scale4 – primarily the nanoscale. As an example: in colloidal simulation (Chapter 6) the molecular-scale degrees of freedom corresponding to microions in the solvent are “integrated out,” the result being the Poisson–Boltzmann equation that applies on the scale of colloidal particles (approximately from 10 to 1000 nm). Still, simulation of optical tips (Section 7.12, p. 433) does have salient multiscale features. Let us now discuss the computational side of nanoscale models. Computational analysis is a mature discipline combining science, engineering and elements of art. It includes general and powerful techniques such as finite difference, finite element, spectral or pseudospectral, integral equation and other methods; it has been applied to every physical problem and device imaginable. Are these existing methods good enough for nanoscale problems? The answer can be anything from “yes” to “maybe” to “no,” depending on the problem. •
•
4
When continuum models are still applicable, traditional methods work well. A relevant example is the simulation of light scattering by plasmon nanoparticles and of plasmon-enhanced components for ultra-sensitive optical sensors and near-field microscopes (Chapter 7). Despite the nanoscale features of the problem, equivalent material parameters (dielectric permittivity and magnetic permeability) can still be used, possibly with some adjustments. Consequently, commercial finite-element software is suitable for this type of modeling. When the system size is even smaller, as in macromolecular simulation, the use of equivalent material parameters is more questionable. In electrostatic models of protein molecules in solvents – an area of extensive and intensive research due to its enormous implications for biology and medicine – two main approaches coexist. In implicit models, the solvent is characterized by equivalent continuum parameters (dielectric permittivity and the Debye length). In the layer of the solvent immediately adjacent to the surface of the molecule, these equivalent parameters are dramatically different from their values in the bulk (A. Rubinstein & S. Sherman [RS04]). In contrast, explicit models directly include molecular dynamics of the solvent. This approach is in principle more accurate, as no approximation of the solvent by an equivalent medium is made, but the computational cost is extremely The Flexible Local Approximation MEthod (FLAME) of Chapter 4 can, however, be viewed as a two-scale method: the difference scheme is formed on a relatively coarse grid but incorporates information about the solution on a finer scale.
1.2 Why Special Models for the Nanoscale?
5
high due to a very large number of degrees of freedom corresponding to the molecules of the solvent. For more information on protein simulation, see T. Schlick’s book [Sch02] and T. Simonson’s review paper [Sim03] as a starting point. • When the problem reduces to a system of ordinary differential equations, the computational analysis is on very solid ground – this is one of the most mature areas of numerical mathematics (Chapter 2). It is highly desirable to use numerical schemes that preserve the essential physical properties of the system. In Molecular Dynamics, such fundamental properties are the conservation of energy and momentum, and – more generally – symplecticness of the underlying Hamiltonian system (Section 2.5). Time-stepping schemes with analogous conservation properties are available and their advantages are now widely recognized (J.M. Sanz-Serna & M.P. Calvo [SSC94], Yu.B. Suris [Sur87, Sur96], R.D. Skeel et al. [RDS97]). • Quantum mechanical effects require special computational treatment. The models are substantially different from those of continuum media for which the traditional methods (such as finite elements or finite differences) were originally designed and used. Nevertheless these traditional methods can be very effective at certain stages of quantum mechanical analysis. For example, classical finite-difference schemes (in particular, the Collatz “Mehrstellen” schemes, Chapter 2), have been successfully applied to the Kohn–Sham equation – the central procedure in Density Functional Theory. (This is the Schr¨ odinger equation, with the potential expressed as a function of electron density.) For a detailed description, see E.L. Briggs et al. [BSB96] and T.L. Beck [Bec00]. Moreover, difference schemes can also be used to find the electrostatic potential from the Poisson equation with the electron density in the right hand side. • Colloidal simulation considered in Chapter 6 is an interesting and special computational case. As explained in that chapter, classical methods of computation are not particularly well suited for this problem. Finite element meshes become too complex and impractical to generate even for a moderate number of particles in the model; standard finite-difference schemes require unreasonably fine grids to represent the boundaries of the particles accurately; the Fast Multipole Method does not work too well for inhomogeneous and/or nonlinear problems. A new finite-difference calculus of Flexible Local Approximation MEthods (FLAME) is a promising alternative (Chapter 4). This list could easily be extended to include other examples, but the main point is clear: a vast assortment of computational methods, both traditional and new, are very helpful for the efficient simulation of nanoscale systems.
6
1 Introduction
1.3 How To Hone the Computational Tools A computer makes as many mistakes in two seconds as 20 men working 20 years make. Murphy’s Laws of Computing
Computer simulation is not an exact science. If it were, one would simply set a desired level of accuracy of the numerical solution and prove that a certain method achieves that level with the minimal number of operations Θ = Θ(). The reality is of course much more intricate. First, there are many possible measures of accuracy and many possible measures of the cost (keeping in mind that human time needed for the development of algorithms and software may be more valuable than the CPU time). Accuracy and cost both depend on the class and subclass of problems being solved. For example, numerical solution becomes substantially more complicated if discontinuities and edge or corner singularities of the field need to be represented accurately. Second, it is usually close to impossible to guarantee, at the mathematical level of rigor, that the numerical solution obtained has a certain prescribed accuracy.5 Third, in practice it is never possible to prove that any given method minimizes the number of arithmetic operations. Fourth, there are modeling errors – approximations made in the formulation of the physical problem; these errors are a particular concern on the nanoscale, where direct and accurate experimental verification of the assumptions made is very difficult. Fifth, a host of other issues – from the algorithmic implementation of the chosen method to roundoff errors – are quite difficult to take into account. Parallelization of the algorithm and the computer code is another complicated matter. With all this in mind, computer simulation turns out to be partially an art. There is always more than one way to solve a given problem numerically and, with enough time and resources, any reasonable approach is likely to produce a result eventually. Still, it is obvious that not all approaches are equal. Although the accuracy and computational cost cannot be determined exactly, some qualitative measures are certainly available and are commonly used. The main characteristic is the asymptotic behavior of the number of operations and memory required for a given method as a function of some accuracy-related parameter. In mesh-based methods (finite elements, finite differences, Ewald summation, 5
There is a notable exception in variational methods: rigorous pointwise error bounds can, for some classes of problems, be established using dual formulations (see p. 153 for more information). However, this requires numerical solution of a separate auxiliary problem for Green’s function at each point where the error bound is sought.
1.3 How To Hone the Computational Tools
7
etc.) the mesh size h or the number of nodes n usually act as such a parameter. The “big-oh” notation is standard; for example, the number of arithmetic operations θ being O(nγ ) as n → ∞ means that c1 nγ ≤ θ ≤ c2 nγ , where c1,2 and γ are some positive constants independent of n. Computational methods with the operation count and memory O(n) are considered as asymptotically optimal; the doubling of the number of nodes (or some other such parameter) leads, roughly, to the doubling of the number of operations and memory size. For several classes of problems, there exist divide-and-conquer or hierarchical strategies with either optimal O(n) or slightly suboptimal O(n log n) complexity. The most notable examples are Fast Fourier Transforms (FFT), Fast Multipole Methods, multigrid methods, and FFT-based Ewald summation. Clearly, the numerical factors c1,2 also affect the performance of the method. For real-life problems, they can be determined experimentally and their magnitude is not usually a serious concern. A notable exception is the Fast Multipole Method for multiparticle interactions; its operation count is close to optimal, O(np log np ), where np is the number of particles, but the numerical prefactors are very large, so the method outperforms the bruteforce approach (O(n2p ) pairwise particle interactions) only for a large number of particles, tens of thousands and beyond. Given that the choice of a suitable method is partially an art, what is one to do? As a practical matter, the availability of good public domain and commercial software in many cases simplifies the decision. Examples of such software are •
• •
• •
Molecular Dynamics packages AMBER (Assisted Model Building with Energy Refinement, amber.scripps.edu); CHARMM/CHARMm (Chemistry at HARvard Macromolecular Mechanics, yuri.harvard.edu, accelrys. com/products/dstudio/index.html), NAMD (www.ks.uiuc.edu/Research/ namd), GROMACS (gromacs.org), TINKER (dasher.wustl.edu/tinker), DL POLY (www.cse.scitech.ac.uk/ccg/software/DL POLY/index.shtml). A finite difference Poisson-Boltzman solver DelPhi (honiglab.cpmc.columbia.edu). Finite Element software developed by ANSYS (ansys.com – comprehensive FE modeling, with multiphysics); by ANSOFT (ansoft.com – state-of-theart FE package for electromagnetic design); by Comsol (comsol.com or femlab.com – the Comsol MultiphysicsTM package, also known as FEMLAB); and others. A software suite from Rsoft Group (rsoftdesign.com) for design of photonics components and optical networks. Electromagnetic time-domain simulation software from CST (Computer Simulation Technology, cst.com).
This list is certainly not exhaustive and, among other things, does not include software for ab initio electronic structure calculation, as this subject matter lies beyond the scope of the book.
8
1 Introduction
The obvious drawback of using somebody else’s software is that the user cannot extend its capabilities and apply it to problems for which it was not designed. Some tricks are occasionally possible (for example, equations in cylindrical coordinates can be converted to the Cartesian system by a mathematically equivalent transformation of material parameters), but by and large the user is out of luck if the code is proprietary and does not handle a given problem. For open-source software, users may in principle add their own modules to accomplish a required task, but, unless the revisions are superficial, this requires detailed knowledge of the code. Whether the reader of this book is an intelligent user of existing software or a developer of his own algorithms and codes, the book will hopefully help him/her to understand how the underlying numerical methods work.
1.4 So What? Avoid clich´es like the plague! William Safire’s Rules for Writers
Multisyllabic clich´es are probably the worst type, but I feel compelled to use one: nanoscale science and technology are interdisciplinary. The book is intended to be a bridge between two broad fields: computational methods, both traditional and new, on the one hand, and several nanoscale or molecularscale applications on the other. It is my hope that the reader who has a background in physics, physical chemistry, electrical engineering or related subjects, and who is curious about the inner workings of computational methods, will find this book helpful for crossing the bridge between the disciplines. Likewise, experts in computational methods may be interested in browsing the application-related chapters. At the same time, readers who wish to stay on their side of the “bridge” may also find some topics in the book to be of interest. An example of such a topic for numerical analysts is the FLAME schemes of Chapter 4; a novel feature of this approach is the systematic use of local approximation spaces in the FD context, with basis functions not limited to Taylor polynomials. Similarly, in the chapter on Finite Element analysis (Chapter 3), the theory of shape-related approximation errors is nonstandard and yields some interesting error estimates. Since the prospective reader will not necessarily be an expert in any given subject of the book, I have tried, to the extent possible, to make the text accessible to researchers, graduate and even senior-level undergraduate students with a good general background in physics and mathematics. While part of the material is related to mathematical physics, the style of the book can be
1.4 So What?
9
characterized as physical mathematics 6 – “physical” explanation of the underlying mathematical concepts. I hope that this style will be tolerable to the mathematicians and beneficial to the reader with a background in physical sciences and engineering. Sometimes, however, a more technical presentation is necessary. This is the case in the analysis of consistency errors and convergence of difference schemes in Chapter 2, Ewald summation in Chapter 5, and the derivation of FLAME basis functions for particle problems in Chapter 6. In many other instances, references to a rigorous mathematical treatment of the subject are provided. I cannot stress enough that this book is very far from being a comprehensive treatise on nanoscale problems and applications. The selection of subjects is strongly influenced by my research interests and experience. Topics where I felt I could contribute some new ideas, methods and results were favored. Subjects that are covered nicely and thoroughly in the existing literature were not included. For example, material on Molecular Dynamics was, for the most part, left out because of the abundance of good literature on this subject.7 However, one of the most challenging parts of Molecular Dynamics – the computation of long-range forces in a homogeneous medium – appears as a separate chapter in the book (Chapter 5). The novel features of this analysis are a rigorous treatment of “charge allocation” to grid and the application of finite-difference schemes, with the potential splitting, in real space. Chapter 2 gives the necessary background on Finite Difference (FD) schemes; familiarity with numerical methods is helpful but not required for reading and understanding this chapter. In addition to the standard material on classical methods, their consistency and convergence, this chapter includes introduction to flexible approximation schemes, Collatz “Mehrstellen” schemes, and schemes for Hamiltonian systems. Chapter 3 is a concise self-contained description of the Finite Element Method (FEM). No special prior knowledge of computational methods is required to read most of this chapter. Variational principles and their role are explained first, followed by a tutorial-style exposition of FEM in the simplest 1D case. Two- and three-dimensional scalar problems are considered in the subsequent sections of the chapter. A more advanced subject is edge elements that are crucial for vector field problems in electromagnetic analysis. Readers already familiar with FEM may be interested in the new treatment of approximation accuracy as a function of element shape; this is a special topic in Chapter 3. 6
7
Not exactly the same as “engineering mathematics,” a more utilitarian, useroriented approach. J.M. Haile, Molecular Dynamics Simulation: Elementary Methods, WileyInterscience, 1997; D. Frenkel & B. Smit, Understanding Molecular Simulation, Academic Press, 2001; D.C. Rapaport, The Art of Molecular Dynamics Simulation, Cambridge University Press, 2004; T. Schlik [Sch02], and others.
10
1 Introduction
Chapter 4 introduces the Finite Difference (FD) calculus of Flexible Local Approximation MEthods (FLAME). Local analytical solutions are incorporated into the schemes, which often leads to much higher accuracy than would be possible in classical FD. A large assortment of examples illustrating the usage of the method are presented. Chapter 6 can be viewed as an extension of Chapter 5 to multiparticle problems in heterogeneous media. The simulation of such systems, due to its complexity, has received relatively little attention, and good methods are still lacking. Yet the applications are very broad – from colloidal suspensions to polymers and polyelectrolytes; in all of these cases, the media are inhomogeneous because the dielectric permittivities of the solute and solvent are usually quite different. Ewald methods can only be used if the solvent is modeled explicitly, by including polarization on the molecular level; this requires a very large number of degrees of freedom in the simulation. An alternative is to model the solvent implicitly by continuum parameters and use the FLAME schemes of Chapter 4. Application of these schemes to the computation of the electrostatic potential, field and forces in colloidal systems is described in Chapter 6. Chapter 7 deals with applications in nano-photonics and nano-optics. It reviews the mathematical theory of Bloch modes, in connection with the propagation of electromagnetic waves in periodic structures; describes plane wave expansion, FEM and FLAME for photonic bandgap computation; provides a theoretical background for plasmon resonances and considers various numerical methods for plasmon-enhanced systems. Such systems include optical sensors with very high sensitivity, as well as scanning near-field optical microscopes with molecular-scale resolution, unprecedented in optics. Chapter 7 also touches upon negative refraction and nanolensing – areas of very intensive research and debate – and includes new material on the inhomogeneity of backward wave media.
2 Finite-Difference Schemes
2.1 Introduction Due to its relative simplicity, Finite Difference (FD) analysis was historically the first numerical technique for boundary value problems in mathematical physics. The excellent review paper by V. Thom´ee [Tho01] traces the origin of FD to a 1928 paper by R. Courant, K. Friedrichs and H. Lewy, and to a 1930 paper by S. Gerschgorin. However, the Finite Element Method (FEM) that emerged in the 1960s proved to be substantially more powerful and flexible than FD. The modern techniques of hp-adaption, parallel multilevel preconditioning, domain decomposition have made FEM ever more powerful (Chapter 3). Nevertheless, FD remains a very valuable tool, especially for problems with relatively simple geometry. This chapter starts with a gentle introduction to FD schemes and proceeds to a more detailed review. Sections 2.2–2.4 are addressed to readers with little or no background in finite-difference methods. Section 2.3, however, introduces a nontraditional perspective and may be of interest to more advanced readers as well. By approximating the solution of the problem rather than a generic smooth function, one can achieve much higher accuracy. This nontraditional perspective will be further developed in Chapter 4. Section 2.4 gives an overview of classical FD schemes for Ordinary Differential Equations (ODE) and systems of ODE; Section 2.5 – an overview of Hamiltonian systems that are particularly important in molecular dynamics. Sections 2.6–2.8 describe FD schemes for boundary value problems in one, two and three dimensions. Some ideas of this analysis, such as minimization of the consistency error for a constrained set of functions, are nonstandard. Finally, Section 2.9 summarizes the most important results on consistency and convergence of FD schemes. In addition to providing a general background on FD methods, this chapter is intended to set the stage for the generalized FD analysis with “Flexible Local Approximation” described in Chapter 4. The scope of the present chapter is limited, and for a more comprehensive treatment and analysis of
12
2 Finite-Difference Schemes
FD methods – in particular, elaborate time-stepping schemes for ordinary differential equations, schemes for gas and fluid dynamics, Finite-Difference Time-Domain (FDTD) methods in electromagnetics, etc. – I defer to many excellent more specialized monographs. Highly recommended are books by C.W. Gear [Gea71] (ODE, including stiff systems), U.M. Ascher & L.R. Petzold [AP98], K.E. Brenan et al. [KB96] (ODE, especially the treatment of differential-algebraic equations), S.K. Godunov & V.S. Ryabenkii [GR87a] (general theory of difference schemes and hyperbolic equations), J. Butcher [But87, But03] (time-stepping schemes and especially Runge–Kutta methods), T.J. Chung [Chu02] and S.V. Patankar [Pat80] (schemes for computational fluid dynamics), A. Taflove & S.C. Hagness [TH05] (FDTD).
2.2 A Primer on Time-Stepping Schemes 1
The following example is the simplest possible illustration of key principles of finite-difference analysis. Suppose we wish to solve the ordinary differential equation du = λu on [0, tmax ], u(0) = u0 , Re λ < 0 (2.1) dt numerically. The exact solution of this equation uexact = u0 exp(λt)
(2.2)
obviously has infinitely many values at infinitely many points within the interval. In contrast, numerical algorithms have to operate with finite (discrete) sets of data. We therefore introduce a set of points (grid) t0 = 0, t1 , . . . , tn−1 , tn = tmax over the given interval. For simplicity, let us assume that the grid size ∆t is the same for all pairs of neighboring points: tk+1 − tk = ∆t, so that tk = k∆t. We now consider equation (2.1) at a moment of time t = tk : du (tk ) = λu(tk ) dt
(2.3)
The first derivative du/dx can be approximated on the grid in several different ways: u(tk+1 ) − u(tk ) du (tk ) = + O(∆t) dt ∆t du u(tk ) − u(tk−1 ) (tk ) = + O(∆t) dt ∆t du u(tk+1 ) − u(tk−1 ) (tk ) = + O((∆t)2 ) dt 2∆t 1
I am grateful to Serge Prudhomme for very helpful suggestions and comments on the material of this section.
2.2 A Primer on Time-Stepping Schemes
13
These equalities – each of which can be easily justified by Taylor expansion – lead to the algorithms known as forward Euler, backward Euler and central difference schemes, respectively: uk+1 − uk − uk = 0 λ∆t
(2.4)
or, equivalently, uk+1 − (1 + λ∆t)uk = 0
(forward Euler)
uk − uk−1 = uk λ∆t
(2.5) (2.6)
or (1 − λ∆t)uk − uk−1 = 0
(backward Euler)
uk+1 − uk−1 = uk 2λ∆t
(2.7) (2.8)
or uk+1 − 2λ∆tuk − uk−1 = 0
(central difference)
(2.9)
where uk−1 , uk and uk+1 are approximations to u(t) at discrete times tk−1 , tk and tk+1 , respectively. For convenience of analysis, the schemes above are written in the form that makes the dimensionless product λ∆t explicit. The (discrete) solution for the forward Euler scheme (2.4) can be easily found by time-stepping: start with the given initial value u(0) = u0 and use the scheme to find the value of the solution at each subsequent step: uk+1 = (1 + λ∆t) uk
(2.10)
This difference scheme was obtained by approximating the original differential equation, and it is therefore natural to expect that the solution of the original equation will approximately satisfy the difference equation. This can be easily verified because in this simple example the exact solution is known. Let us substitute the exact solution (2.2) into the left hand side of the difference equation (2.4): exp (λ(k + 1)∆t) − exp (kλ∆t) − exp (kλ∆t) c = u0 λ∆t exp(λ∆t) − 1 λ∆t − 1 = u0 exp(kλ∆t) + h.o.t. (2.11) = u0 exp(kλ∆t) λ∆t 2 where the very last equality was obtained via the Taylor expansion for ∆t → 0, and “h.o.t.” are higher order terms with respect to the time step ∆t. Note that the exponential factor exp(kλ∆t) goes to unity if ∆t → 0 and the other parameters are fixed; however, if the moment of time t = tk is fixed, then this exponential is proportional to the value of the exact solution
14
2 Finite-Difference Schemes
Symbol c stands for consistency error that is, by definition, obtained by substituting the exact solution into the difference scheme. The consistency error (2.11) is indeed “small” – it tends to zero as ∆t tends to zero. More precisely, the error is of order one with respect to ∆t. In general, the consistency error c is said to be of order p with respect to ∆t if c1 ∆tp ≤ |c | ≤ c2 ∆tp
(2.12)
where c1,2 are some positive constants independent of ∆t. (In the case under consideration, p = 1.) A very common equivalent form of this statement is the “big-oh” notation: |c | = O((∆t)p ) (see also Introduction, p. 7). While consistency error is a convenient and very important intermediate quantity, the ultimate measure of accuracy is the solution error, i.e. the deviation of the numerical solution from the exact one: (2.13) k = uk − uexact (tk ) The connection between consistency and solution errors will be discussed in Section 2.9. In our current example, we can evaluate the numerical error directly. The repeated “time-stepping” by the forward Euler scheme (2.10) yields the following numerical solution: uk = (1 + λ∆t)k u0 ≡ (1 − ξ)k u0
(2.14)
where ξ = −λ∆t. (Note that Re ξ > 0, as Re λ is assumed negative.) The k-th time step corresponds to the time instant tk = k∆t, and so in terms of time the numerical solution can then be rewritten as uk = [(1 − ξ)1/ξ ]−λtk u0
(2.15)
From basic calculus, the expression in the square brackets tends to e−1 as ξ → 0, and hence uk tends to the exact solution (2.2) u0 exp(λtk ) as ∆t → 0. Thus in the limit of small time steps the forward Euler scheme works as expected. However, in practice, when equations and systems much more complex than our example are solved, very small step sizes may lead to prohibitively high computational costs due to a large number of time steps involved. It is therefore important to examine the behavior of the numerical solution for any given positive value of the time step rather than only in the limit ∆t → 0. Three qualitatively different cases emerge from (2.14): ⎧ ⎨ |1 + λ∆t| < 1 ⇔ ∆t < ∆tmin , numerical solution decays (as it should); |1 + λ∆t| > 1 ⇔ ∆t > ∆tmin , numerical solution diverges; ⎩ |1 + λ∆t| = 1 ⇔ ∆t = ∆tmin , numerical solution oscillates.
2.2 A Primer on Time-Stepping Schemes
where ∆tmin = − ∆tmin =
2Re λ , |λ|2
2 , |λ|
15
Re λ < 0
(2.16)
λ < 0 (λ real)
(2.17)
For the purposes of this introduction, we shall call a difference scheme stable if, for a given initial condition, the numerical solution remains bounded for all time steps; otherwise the scheme is unstable.2 It is clear that in the second and third case above the numerical solution is qualitatively incorrect. The forward Euler scheme is stable only for sufficiently small time steps – namely, for ∆t < ∆tmin
(stability condition for the forward Euler scheme)
(2.18)
Schemes that are stable only for a certain range of values of the time step are called conditionally stable. Schemes that are stable for any positive time step are called unconditionally stable. It is not an uncommon misconception to attribute the numerical instability to round-off errors. While round-off errors can exacerbate the situation, it is clear from (2.14) the instability will manifest itself even in exact arithmetic if the time step is not sufficiently small. The backward Euler difference scheme (2.6) is substantially different in this regard. The numerical solution for that scheme is easily found to be uk = (1 − λ∆t)−k u0
(2.19)
In contrast with the forward Euler method, for negative Re λ this solution is bounded (and decaying in time) regardless of the step size ∆t. That is, the backward Euler scheme is unconditionally stable. However, there is a price to pay for this advantage: the scheme is an equation with respect to uk+1 . In the current example, solution of this equation is trivial (just divide by 1 − λ∆t), but for nonlinear differential equations, and especially for (linear and nonlinear) systems of differential equations the computational cost of computing the solution at each time step may be high. Difference schemes that require solution of a system of equations to find uk+1 are called implicit; otherwise the scheme is explicit. The forward Euler scheme is explicit, and the backward Euler scheme is implicit. The derivation of the consistency error for the backward Euler scheme is completely analogous to that of the forward Euler scheme, and the result is essentially the same, except for a sign difference: c = − u0 exp(kλ∆t) 2
λ∆t + h.o.t. 2
(2.20)
More specialized definitions of stability can be given for various classes of schemes; see e.g. C.W. Gear [Gea71], J.C. Butcher [But03], E. Hairer et al. [HrW93] as well as the following sections of this chapter.
16
2 Finite-Difference Schemes
As in the forward Euler case, the exponential factor tends to unity as the time step goes to zero, but only if k and λ are fixed. The very popular Crank–Nicolson scheme3 can be viewed as an approximation of the original differential equation at time tk+1/2 ≡ tk + ∆t/2: uk + uk+1 uk+1 − uk − = 0, λ∆t 2
k = 0, 1, . . .
(2.21)
Indeed, the left hand side of this equation is the central-difference approximation (completely analogous to (2.8), but with a twice smaller time step), while the right hand side approximates the value of u(tk+1/2 ). The time-stepping procedure for the Crank–Nicolson scheme is λ∆t λ∆t uk+1 = uk , k = 0, 1, . . . 1+ (2.22) 1− 2 2 and the numerical solution of the model problem is uk =
1 + λ∆t/2 1 − λ∆t/2
k u0
(2.23)
Since the absolute value of the fraction here is less than one for all positive (even very large) time steps, the Crank–Nicolson scheme is unconditionally stable. Its consistency error is again found by substituting the exact solution (2.2) into the scheme (2.21). The result is c = − u0 exp(kλ∆t)
(λ∆t)2 + h.o.t. 12
(2.24)
The consistency error is seen to be of second order – as such, it is (for sufficiently small time steps) much smaller than the error of both Euler schemes.
2.3 Exact Schemes As we have seen, the consistency error can be made smaller if one switches from Euler methods to the Crank–Nicolson scheme. Can the consistency error be reduced even further? One may try to “mix” the forward and backward 3
Often misspelled as Crank-Nicholson. After John Crank (born 1916), British mathematical physicist, and Phyllis Nicolson (1917–1968), British physicist. http://www-groups.dcs.st-and.ac.uk/ history/Mathematicians/Nicolson.html http://www-groups.dcs.st-and.ac.uk/ history/Mathematicians/Crank.html The original paper is: J. Crank and P. Nicolson, A practical method for numerical evaluation of solutions of partial differential equations of the heat-conduction type, Proc. Cambridge Philos. Soc., vol. 43, pp. 50–67, 1947. [Re-published in: John Crank 80th birthday special issue of Adv. Comput. Math., vol. 6, pp. 207–226, 1997.]
2.3 Exact Schemes
17
Euler schemes in a way similar to the Crank–Nicolson scheme, but by assigning some other weights θ and (1 − θ), instead of 12 , to uk and uk+1 in (2.21). However, it would soon transpire that the Crank–Nicolson scheme in fact has the smallest consistency error in this family of schemes, so nothing substantially new is gained by introducing the alternative weighting factors. Nevertheless one can easily construct schemes whose consistency error cannot be beaten. Indeed, here is an example of such a scheme: uk+1 uk − = 0 uexact (tk ) uexact (tk+1 )
(2.25)
More specifically for the equation under consideration uk uk+1 − = 0 exp(−λtk ) exp(−λtk+1 )
(2.26)
uk − uk+1 exp(λ∆t) = 0
(2.27)
Equivalently, Obviously, by construction of the scheme, the analytical solution satisfies the difference equation exactly – that is, the consistency error of the scheme is zero. One cannot do any better than that! The first reaction may be to dismiss this construction as cheating: the scheme makes use of the exact solution that in fact needs to be found. If the exact solution is known, the problem has been solved and no difference scheme is needed. If the solution is not known, the coefficients of this “exact” scheme are not available. Yet the idea of “exact” schemes like (2.25) proves very useful. Even though the exact solution is usually not known, excellent approximations for it can frequently be found and used to construct a difference scheme. One key observation is that such approximations need not be global (i.e. valid throughout the computational domain). Since difference schemes are local, all that is needed is a good local approximation of the solution. Local approximations are much more easily obtainable than global ones. In fact, the Taylor series expansion that was implicitly used to construct the Euler and Crank–Nicolson schemes, and that will be more explicitly used in the following subsection, is just an example of a local approximation. The construction of “exact” schemes represents a shift in perspective. The objective of Taylor-based schemes is to approximate the differential operator – for example, d/dt – with a suitable finite difference, and consequently the differential equation with the respective FD scheme. The objective of the “exact” schemes is to approxim´ate the solution. Approximation of the differential operator is a very powerful tool, but it carries substantial redundancy: it is applicable to all sufficiently smooth functions to which the differential operator could be applied. By focusing on the solution only, rather than on a wide class of smooth functions, one can reduce or even eliminate this redundancy. As a result, the accuracy of the
18
2 Finite-Difference Schemes
numerical solution can be improved dramatically. This set of ideas will be explored in Chapter 4. The following figures illustrate the accuracy of different one-step schemes for our simple model problem with parameter λ = −10. Fig. 2.1 shows the analytical and numerical solutions for time step ∆t = 0.05. It is evident that the Crank–Nicolson scheme is substantially more accurate than the Euler schemes. The numerical errors are quantified in Fig. 2.2. As expected, the exact scheme gives the true solution up to the round-off error.
Fig. 2.1. Numerical solution for different one-step schemes. Time step ∆t = 0.05. λ = −10.
For a larger time step ∆t = 0.25, the forward Euler scheme exhibits instability (Fig. 2.3). The exact scheme still yields the analytical solution to machine precision. The backward Euler and Crank–Nicolson schemes are stable, but the numerical errors are higher than for the smaller time step. R.E. Mickens [Mic94] derives “exact” schemes from a different perspective and extends them to a family of “nonstandard” schemes defined by a set of heuristic rules. We shall see in Chapter 4 that the “exact” schemes are a very natural particular case of a new finite-difference calculus – “Flexible Local Approximation MEthods” (FLAME).
2.4 Some Classic Schemes for Initial Value Problems For completeness, this section presents a brief overview of a few popular timestepping schemes for Ordinary Differential Equations (ODE).
2.4 Some Classic Schemes for Initial Value Problems
19
Fig. 2.2. Numerical errors for different one-step schemes. Time step ∆t = 0.05. λ = −10.
Fig. 2.3. Numerical solution for the forward Euler scheme. Time step ∆t = 0.25. λ = −10.
20
2 Finite-Difference Schemes
Fig. 2.4. Numerical solution for different one-step schemes. Time step ∆t = 0.25. λ = −10.
2.4.1 The Runge–Kutta Methods This introduction to Runge–Kutta (R-K) methods follows the elegant exposition by E. Hairer et al. [HrW93]. The main idea dates back to C. Runge’s original paper of 1895. The goal is to construct high order difference schemes for the ODE y (t) = f (t, y),
y(t0 ) = y0
(2.28)
Our starting point is a simpler problem, with the right hand side independent of y: (2.29) y (t) = f (t), y(t0 ) = y0 This problem not only has an analytical solution t f (τ )dτ y(t) = y0 +
(2.30)
t0
but also admits accurate approximations via numerical quadratures. For example, the midpoint rule gives ∆t0 y1 ≡ y(t1 ) ≈ y0 + ∆t0 f t0 + 2 ∆t1 y2 ≡ y(t2 ) ≈ y1 + ∆t1 f t1 + 2 and so on. Here t0 , t1 , etc., are a discrete set of points in time, and the time steps ∆t0 = t1 − t0 , ∆t1 = t2 − t1 , etc., do not have to be equal.
2.4 Some Classic Schemes for Initial Value Problems
21
It is straightforward to verify that this numerical quadrature (that doubles as a time-stepping scheme) has second order accuracy with respect to the maximum time step. An analogous formula for taking the numerical solution of the original equation (2.28) from a generic point t in time to t + ∆t would be ∆t ∆t , y t+ (2.31) y(t + ∆t) ≈ y(t) + ∆tf t + 2 2 The obstacle is that the value of y at the midpoint t + ∆t 2 is not directly available. However, this value may be found approximately via the forward Euler scheme with the time step ∆t/2: ∆t ∆t ≈ y(t) + f (t, y(t)) (2.32) y t+ 2 2 A valid difference scheme can now be produced by inserting this midpoint value into the numerical quadrature (2.31). The customary way of writing the overall procedure is as the following sequence:
k2
k1 = f (t, y) ∆t ∆t , y(t) + k1 = f t+ 2 2 y(t + ∆t) = y(t) + ∆t k2
(2.33) (2.34) (2.35)
This is the simplest R-K method with two stages (k1 is computed at the first stage and k2 at the second). The generic form of an s-stage explicit R-K method is as follows [HrW93]: k1 = f (t0 , y0 ) = f (t0 + c2 ∆t, y0 + ∆ta21 k1 )
k2 k3 = f (t0 + c3 ∆t, y0 + ∆t (a31 k1 + a32 k2 )) ...
ks = f (t0 + cs ∆t, y0 + ∆t (as1 k1 + · · · + as,s−1 ks−1 )) y(t + h) = y0 + ∆t(b1 k1 + b2 k2 + · · · + bs ks ) The procedure is indeed explicit, as the computation at each subsequent stage depends only on the values computed at the previous stages. The “input data” for the R-K method at any given time step consists only of one value y0 at the beginning of this step and does not include any other previously computed values. Thus the R-K time step sizes can be chosen independently, which is very useful for adaptive algorithms. The multi-stage method should not be confused with multi-step schemes (such as e.g. the Adams methods, Section 2.4.2 below) where the input data at each discrete time point contains the values of y at several previous steps. Changing the time step in multistep methods may be cumbersome and may require “re-initialization” of the algorithm.
22
2 Finite-Difference Schemes
To write R-K schemes in a compact form, it is standard to collect all the coefficients a, b and c in J. Butcher’s tableau: 0 c2 c3 ... ... cs
a21 a31 ... ... as1 b1
a32 ... ... as2 b2
... ... ... ...
... ... ... . . . . . . as,s−1 . . . . . . bs
One further intuitive observation is that the k parameters in the R-K method are values of function f at some intermediate points. As a rule, one wants these intermediate points to be close to the actual solution y(t) of (2.28). Then, according to (2.28), the ks also approximate the time derivative of y over the current time step. Thus at the i-th stage of the procedure function f is evaluated, roughly speaking, at point (t0 + ci ∆t, y0 + (ai1 + · · · + ai,s−1 )y (t0 )∆t). From these considerations, condition ci = ai1 + · · · + ai,s−1 ,
i = 2, 3 . . . s
emerges as natural (although not, strictly speaking, necessary). The number of stages is in general different from the order of the method (i.e. from the asymptotic order of the consistency error with respect to the time step), and one wishes to find the free parameters a, b and c that would maximize the order. For s ≥ 5, no explicit s-stage R-K method of order s exists (E. Hairer et al. [HrW93], J.C. Butcher [But03]). However, a family of four-stage explicit R-K methods of fourth order are available [HrW93, But03]. The most popular of these methods are 0 1/2 1/2 1
1/2 0 1/2 0 0 1 1/6 2/6 2/6 1/6
and 0 1/3 1/3 2/3 −1/3 1 1 −1 1 1 1/8 3/8 3/8 1/8
2.4 Some Classic Schemes for Initial Value Problems
23
Stability conditions for explicit Runge–Kutta schemes can be obtained along the following lines. For the model scalar equation (2.1) dy = λy on [0, tmax ], u(0) = u0 dt
(2.36)
the exact solution changes by the factor of exp(λh) over one time step. If the R-K method is of order p, the respective factor in the growth of the numerical solution is the Taylor approximation T (ξ) =
p ξk k=0
k!
,
ξ ≡ λ∆t
to this exponential factor. Stability regions then correspond to |T (ξ)| < 1 in the complex plane ξ ≡ λ∆t (Fig. 2.5).
Fig. 2.5. Stability regions in the λ∆t-plane for explicit Runge–Kutta methods of orders one through four.
Further analysis of R-K methods can be found in monographs by J. Butcher [But03], E. Hairer et al. [HrW93], and C.W. Gear [Gea71].
24
2 Finite-Difference Schemes
2.4.2 The Adams Methods Adams methods are a popular class of multistep schemes, where the solution values from several previous time steps are utilized to find the numerical solution at the subsequent step. This is accomplished by polynomial interpolation. The following brief summary is due primarily to E. Hairer et al. [HrW93]. Consider again the general ODE (2.28) (reproduced here for easy reference): (2.37) y (t) = f (t, y), y(t0 ) = y0 Let the grid be uniform, ti = t0 + i∆t, and integrate the differential equation over one time step: tn+1 f (t, y(t)) dt (2.38) y(tn+1 ) = y(tn ) + tn
The integrand is a function of the unknown solution and obviously is not directly available; however, it can be approximated by a polynomial p(t) passing through k previous numerical solution values (ti , f (yi )). The numerical solution at time step n + 1 is then found as tn+1 yn+1 = yn + p(t) dt (2.39) tn
Coefficients of p(t) can be found explicitly (e.g. via backward differences), and the scheme is then obtained after inserting the expression for p into (2.39). This explicit calculation appears in all texts on numerical methods for ODE and is not included here. Adams methods can also be used in the Nordsieck form, where instead of the values of function f at the previous time steps approximate Taylor coefficients for the solution are stored. These approximate coefficients form 2 k (k) the Nordsieck vector (yn , ∆tyn , ∆t2 yn , . . . , ∆t k! yn ). This form makes it easier to change the time step size as needed. 2.4.3 Stability of Linear Multistep Schemes It is clear from the introduction in Section 2.2 that stability characteristics of the difference scheme are of critical importance for the numerical solution. Stability depends on the intrinsic properties of the underlying differential equation (or a system of ODE), as well as on the difference scheme itself and the mesh size. This section highlights the key points in the stability analysis of linear multistep schemes; the results and conclusions will be used, in particular, in the next section (stiff systems). Stability of linear multistep schemes is covered in all texts on FD schemes for ODE (e.g. C.W. Gear [Gea71], J. Butcher [But03], E. Hairer et al. [HrW93], U.M. Ascher & L.R. Petzold [AP98]). A comprehensive classification of types
2.4 Some Classic Schemes for Initial Value Problems
25
of stability is given in the book by J.D. Lambert [Lam91]. This section, for the most part, follows Lambert’s presentation. Consider the test system of equations y = Ay,
y ∈ Rn
(2.40)
where all eigenvalues of matrix A are for simplicity assumed to be distinct and to have strictly negative real parts, so that the system is stable. Further, let a linear k-step method be k
αj y+j = ∆t
j=0
k
βj f+j
(2.41)
j=0
where f is the right hand side of the system, h is (as usual) the mesh size, and index +j indicates values at the j-th time step (the “current” step corresponding to j = 0). In our case, the right hand side f = Ay, and the multistep scheme becomes k (αj I − ∆tβj A) y+j = 0 (2.42) j=0
Since A is assumed to have distinct eigenvalues, it is diagonalizable, i.e. Q−1 AQ = Λ ≡ diag(λ1 , . . . , λn )
(2.43)
where Q is a nonsingular matrix. The same transformation can then be applied to the whole scheme (2.42) by multiplying it with Q−1 on the left and introducing a variable change y = Qz. It is easy to see that, since the system matrix becomes diagonal upon this transformation, the system splits up into completely decoupled equations for each zi , i = 1, 2, . . . , n. With some abuse of notation now, dropping the index i for zi and the respective eigenvalue λi , we get the scalar version of the scheme k
(αj − ∆tβj λ)z+j = 0
(2.44)
j=0
From the theory of difference equations it is well known that stability is governed by the roots4 rs (s = 1,2, . . . , k) of the characteristic equation k
(αj − ∆tλβj ) rj = 0
(2.45)
j=0
Clearly, stability depends on the (dimensionless) parameter hλ. The multistep method is said to be absolutely stable for given λ∆t if all the roots rs of the characteristic polynomial for this value of λ∆t lie strictly inside the unit circle in the complex plane. 4
Lambert’s notation is used here.
26
2 Finite-Difference Schemes
The set of points λ∆t in the λ∆t-plane for which the scheme is absolutely stable is called the region of absolute stability. For illustration, let us recall the simplest case – one-step schemes for the scalar equation y = λy: y+1 − y0 = λ (θy0 + (1 − θ)y+1 ) ∆t
(2.46)
For θ = 0 and 1, this is the implicit/explicit Euler method, respectively; for θ = 0.5 it is the Crank–Nicolson (trapezoidal) scheme. The characteristic equation is obtained in a standard way, by formally substituting r1 for y+1 and r0 = 1 for y0 : r−1 = λ (θ + (1 − θ)r) (2.47) ∆t The root is 1 + λθ∆t (2.48) r = 1 − λ(1 − θ)∆t For the explicit Euler scheme (θ = 1) rexpl.Euler = 1 + λ∆t
(2.49)
and so the region of absolute stability in the λ∆t-plane is the unit circle centered at −1 (Fig. 2.6).
Fig. 2.6. Stability region of the explicit Euler method is the unit circle (shaded).
For the implicit Euler scheme (θ = 0) rimpl.Euler =
1 1 − λ∆t
(2.50)
2.4 Some Classic Schemes for Initial Value Problems
27
Fig. 2.7. Stability region of the implicit Euler method is the shaded area outside the unit circle.
the region of absolute stability is outside the unit circle centered at 1 (Fig. 2.7). This stability region includes all negative values of λ∆t – that is, for a negative λ, the scheme is stable for any (positive) time step. In addition, curiously enough, the scheme is stable in a vast area with positive λ∆t – i.e. the numerical solution may decay exponentially when the exact one grows exponentially. This latter feature is somewhat undesirable but is typically of little significance, as in most cases the underlying differential equations describe stable systems with decaying solutions. What about the Crank–Nicolson scheme? For θ = 0.5 we have rCrank−Nicolson =
1 + λ∆t/2 1 − λ∆t/2
(2.51)
and it is then straightforward to verify that the stability region is the halfplane λ∆t < 0 (Fig. 2.8). The region of stability is clearly a key consideration for choosing a suitable class of schemes and the mesh size such that hλ lies inside the region of stability. 2.4.4 Methods for Stiff Systems One can identify two principal constraints on the choice of the time step in a numerical scheme for ODE. The first constraint has to do with the desired approximation accuracy (i.e. consistency error): if the solution varies smoothly and slowly in time, it can be approximated with sufficient accuracy even if the time step is large.
28
2 Finite-Difference Schemes
Fig. 2.8. Stability region of the Crank–Nicolson scheme is the left half-plane.
The second constraint is imposed by stability of the scheme. Let us recall, for example, that the stability condition for the simplest one-step scheme – the forward Euler method – is ∆t < 2/|λ| (2.18), (2.17) for real negative λ, in reference to the test equation (2.1) dy = λy on [0, tmax ], u(0) = u0 (2.52) dt More advanced explicit methods may have broader stability regions: see e.g. Fig. 2.5 for Runge–Kutta methods in Section 2.4.1. However, the improvement is not dramatic; for example, for the four-stage fourth-order Runge–Kutta method, the step size cannot exceed ∼ 2.785/|λ|. For a single scalar equation (2.52) with λ < 0 and a decaying exponential solution, the accuracy and stability restrictions on the time step size are commensurate. Indeed, accuracy calls for the step size on the order of the relaxation time 1/λ or less, which is well within the stability limit even for the simplest forward Euler scheme. However, for systems of equations the stability constraint on the step size can be much more severe than the accuracy limit. Consider the following example: dy1 = λ1 y1 ; λ1 = −1 dt dy2 = λ2 y2 ; λ2 = −1000 dt
(2.53) (2.54)
The second component (y2 ) dies out when t 1/|λ2 | = 10−3 and can then be neglected; beyond that point, the approximation accuracy would suggest the time step commensurate with the relaxation time of the first component, 1/|λ1 | = 1. However, the stability condition ∆t ≤ c/|λ| (where c depends on the method but is not much greater than 2–3 for most practical explicit schemes) has to hold for both λ and limits the time step to approximately 1/|λ2 | = 10−3 . In other words, the time step that would provide good approximation accuracy exceeds the stability limit by a factor of about 1000. A brute force
2.4 Some Classic Schemes for Initial Value Problems
29
approach is to use a very small time step and accept the high computational cost as well as the tremendous redundancy in the numerical solution that will remain virtually unchanged over one time step. An obvious possibility for a system with decoupled components is to solve the problem separately for each component. In the example above, one could time-step y1 with ∆t1 ∼ 0.1 for about 50 steps (after which y1 will die out) and y2 with ∆t2 ∼ 10−4 also for about 50 steps. However, decoupled systems are a luxury that one seldom has in practical problems. For example, the system of ODEs 500.5 −499.5 A = (2.55) z (t) = Az; z(t) ∈ R2 ; −499.5 500.5 poses the same stability problem for explicit schemes as the previous example – simply because matrix A is obtained from the diagonal matrix D = diag(1, 1000) of the previous example by an orthogonal transformation A = Q DQ, with 1 1 Q = −1 1 The “fast” and “slow” components, with their respective time scales, are now mixed up, but this is no longer immediately obvious. Recovering the two components is equivalent to solving a full eigenvalue-eigenvector problem for the system matrix, which can be done for small systems but is inefficient or even impossible for large ones. The situation is even more complicated for nonlinear problems and systems with time-varying coefficients. A practical alternative lies in switching to implicit difference schemes. In return for excellent stability properties, one pays the price of having to solve for the unknown value of the numerical solution yn+1 at the next time step. This is in general a nonlinear equation (for a scalar ODE) or a nonlinear system of algebraic equations (for a system of ODEs, y being in that case a Euclidean vector). Recall that for the ODE y (t) = f (t, y)
(2.56)
the simplest implicit scheme – the backward Euler method – is yn+1 − yn = ∆t f (tn+1 , yn+1 )
(2.57)
A set of schemes that generalize the backward Euler algorithm to higher orders is due to C.W. Gear [Gea67, Gea71, HrW93] and is called “Backward Differentiation Formulae” (BDF). For illustration, let us derive the second order BDF scheme, the derivation of higher order schemes being analogous. The second order scheme involves three grid points: t−1 = t0 − ∆t, t0 and t+1 = t0 + ∆t; quantities related to the “current” time step t0 will be labeled with index 0, quantities related to the previous and the next step will be
30
2 Finite-Difference Schemes
labeled with −1 and +1, respectively. The starting point is almost the same as for explicit Adams methods: an interpolation polynomial p(t) (quadratic for the second order scheme) that passes through three points (t−1 , y−1 ), (t0 , y0 ) and (t+1 , y+1 ). The values y0 and y−1 of the solution at the current and previous steps are known. The value y+1 at the next step is an unknown parameter, and a suitable condition is needed to evaluate it.
Fig. 2.9. Second-order BDF involves quadratic polynomial interpolation over three points: (t−1 , y−1 ), (t0 , y0 ) and (t+1 , y+1 ).
In BDF, the following condition is imposed: the interpolating polynomial p(t) must satisfy the underlying differential equation at time t+1 , i.e. p (t+1 ) = f (t+1 , y+1 )
(2.58)
To find this interpolation polynomial and then the BDF scheme itself, let us for convenience move the origin of the coordinate system to the midpoint of the stencil and set t0 = 0. Lagrange interpolation through the three points then gives p(t) = y−1
t(t − ∆t) (t + ∆t)(t − ∆t) (t + ∆t)t + y0 + y+1 (−∆t) · (−2∆t) ∆t · (−∆t) 2∆t · ∆t
t(t − ∆t) (t + ∆t)(t − ∆t) (t + ∆t)t − y0 + y+1 (2.59) 2∆t2 ∆t2 2∆t2 The derivative of p (needed to impose condition (2.58) at the next step) is = y−1
2.4 Some Classic Schemes for Initial Value Problems
p (t) =
y−1 y0 y+1 (2t − ∆t) − 2t + (2t + ∆t) 2∆t2 ∆t2 2∆t2
31
(2.60)
Condition (2.58) is obtained by substituting t = t+1 : p (t+1 ) =
2y0 3y+1 y−1 − + = f (t+1 , y+1 ) 2∆t ∆t 2∆t
(2.61)
or equivalently 1 3 y+1 − 2y0 + y−1 = ∆t f (t+1 , y+1 ) 2 2
(2.62)
This is Gear’s second order method. The scheme is implicit – it constitutes a (generally nonlinear) equation with respect to y+1 or, in the case of a vector problem (y ∈ Rn ), a system of equations. In practice, iterative linearization by the Newton–Raphson method is used and suitable linear system solvers are applied in the Newton–Raphson loop. For reference, here is a list of BDF of orders k from one through six [HrW93]. The first order BDF scheme coincides with the implicit Euler method. BDF schemes of orders higher than six are unstable.
147 y+1 60
3 y+1 − 2y0 2 3 11 y+1 − 3y0 + y−1 6 2 4 25 y+1 − 4y0 + 3 y−1 − y−2 12 3 10 5 137 y+1 − 5y0 + 5 y−1 − y−2 + y−3 60 3 4 15 20 15 6 y−1 − y−2 + y−3 − y−4 − 6y0 + 2 3 4 5
y+1 − y0 1 + y−1 2 1 − y−2 3 1 + y−3 4 1 − y−4 5 1 + y−5 6
= ∆t f+1 = ∆t f+1 = ∆t f+1 = ∆t f+1 = ∆t f+1 = ∆t f+1
Since stability considerations are of paramount importance in the choice of difference schemes for stiff problems, an elaborate classification of schemes based on their stability properties – or more precisely, on their regions of absolute stability (see Section 2.4.3) – has been developed. The relevant material can be found in C.W. Gear’s monograph [Gea71] and, in a more complete form, in J.D. Lambert’s book [Lam91]. What follows is a brief summary of this stability classification. A hierarchy of definitions of stability classes with progressively wider regions of stability are (Lambert’s definitions are adopted): A0 -stability ⇐= A(0)-stability ⇐= A(α)-stability ⇐= stiff-stability ⇐= Astability ⇐= L-stability Definition 1. A method is said to be A0 -stable if its region of absolute stability includes the (strictly) negative real semiaxis.
32
2 Finite-Difference Schemes
Definition 2. [Gea71], [Lam91] A method is said to be A(α)-stable, 0 < α < π/2, if its region of absolute stability includes the “angular” domain | arg(λ∆t) − π| ≤ α in the λ∆t-plane (Fig. 2.10). A method is said to be A(0)-stable if it is A(α)-stable for some 0 < α < π/2.
Fig. 2.10. A(α)-stability region.
Definition 3. [Gea71], [Lam91] A method is said to be A-stable if its region of absolute stability includes the half-plane Re (λ∆t) < 0. Definition 4. A method is said to be stiffly-stable if its region of absolute stability includes the union of two domains (Fig. 2.11): (i) Re (λ∆t) < −a, and (ii) −a ≤ Re (λ∆t) < 0, |Im(λ∆t)| < c, where a, c are positive real numbers. Thus stiff stability differs from A-stability in that slowly decaying but highly oscillatory solutions are irrelevant for stiff stability. The rationale is that for such solutions the time step is governed by accuracy requirements for the oscillatory components as much, or perhaps even more, than it is governed by stability requirements – hence this is not truly a stiff case. Definition 5. [Gea71, Lam91] A method is said to be L-stable if it is A-stable and, in addition, when applied to the scalar test equation y = λy, Re λ < 0, it yields yn+1 = R(λ∆t) yn , with |R(λ∆t)| → 0 as Re λ∆t → −∞. The notion of L-stability is motivated by the following test case. Consider one more time the Crank–Nicolson scheme applied to the model scalar equation y = λy:
2.4 Some Classic Schemes for Initial Value Problems
33
Fig. 2.11. Stiff-stability region.
yn+1 + yn yn+1 − yn = λ h 2 The numerical solution is easily found to be n 1 + λ∆t/2 yn = y0 1 − λ∆t/2
(2.63)
(2.64)
As already noted, the Crank–Nicolson scheme is absolutely stable for any λ∆t with a negative real part. The solution above reflects this fact, as the expression in parentheses has the absolute value less than one for Re λ∆t < 0. Still, the numerical solution exhibits some undesirable behavior for “highly negative” values of λ, i.e. for λ < 0, |λ|∆t 1. Indeed, in this case the actual solution decays very rapidly in time as exp(λt), whereas the numerical solution decays very slowly but is highly oscillatory because the expression in parentheses in (2.64) is close to −1. This is a case where the numerical solution disagrees with the exact one not just quantitatively but qualitatively. The problem is in fact much broader. If the difference scheme is not chosen judiciously, the character of the solution may be qualitatively incorrect (such as an oscillatory numerical solution vs. a rapidly decaying exact one). Further, important physical invariants (most notably energy or momentum) may not be conserved in the numerical solution, which may render the computated results nonphysical. This is important, in particular, in Molecular Dynamics, where energy conservation and, more generally, “symplecticness” of the underlying Hamiltonian system (Section 2.5) should be preserved. With regard to stiff systems, an alternative solution strategy that does not involve difference schemes can sometimes be effective. The solution of a
34
2 Finite-Difference Schemes
linear system of ODE can be analytically expressed via matrix exponential exp(At) (see Appendix 2.10). Computing this exponential is by no means easy (many caveats are discussed in the excellent papers by C. Moler & C. Van Loan n [ML78, ML03]); nevertheless the recursion relation exp(At) = (exp(At/n)) is helpful. The idea is that for n sufficiently large matrix At/n is “small enough” for its exponential to be computed relatively easily with sufficient accuracy; n is usually chosen as an integer power of two, so that the n-th power of the matrix can be computed by repeated squaring. Two interesting motifs of this and the following section can now be noted: • difference methods that ensure a qualitative/physical agreement between the numerical solutions and the exact ones; • methods blending numerical and analytical approximations. Many years ago, my advisor Iu.V. Rakitskii [Rak72, RUC79, RSY+ 85] was an active proponent of both themes. Nowadays, the qualitative similarity between discrete and continuous models is an important trend in mathematical studies and their applications. Undoubtedly, Rakitskii would have been happy to see the contribution of Yu.B. Suris, his former student, to the development of numerical methods preserving the physical invariants of Hamiltonian systems [Sur87]–[Sur96], as well as to discrete differential geometry (A.I. Bobenko & Yu.B. Suris [BSve]). Another “Rakitskii-style” development is the generalized finite-difference calculus of Flexible Local Approximation MEthods (FLAME, Chapter 4) that seamlessly incorporates local analytical approximations into difference schemes.
2.5 Schemes for Hamiltonian Systems 2.5.1 Introduction to Hamiltonian Dynamics Note: no prior knowledge of Hamiltonian systems is necessary for reading this section. As a starting example, consider a (classical) harmonic oscillator, such as a mass on a spring, described by the ODE m¨ q = − kq
(2.65)
(mass times acceleration equals force), where mass m and the spring constant k are known parameters and q is a coordinate. The general solution to this equation is k (2.66) ω02 = q(t) = q0 cos(ω0 t + φ); m for some parameters q0 and φ. Even though the above expression in principle contains all the information about the solution, recasting the differential equation in a different form
2.5 Schemes for Hamiltonian Systems
35
brings a deeper perspective. The new insights are even more profound for multiparticle problems with multiple degrees of freedom. The Hamiltonian of the oscillator – the energy function H expressed in terms of q and q˙ – comprises the kinetic and potential terms:5 H =
1 1 mq˙2 + kq 2 2 2
(2.67)
We shall view H as a function of two variables: coordinate q and momentum p = mq; ˙ in terms of these variables, H(q, p) =
kq 2 p2 + 2m 2
(2.68)
The original second-order differential equation splits up into two first-order equations ⎧ ⎨ q˙ = m−1 p (2.69) ⎩ p˙ = − kq or in matrix-vector form w˙ = Aw, w =
q ; p
A =
0 m−1 p −k 0
(2.70)
The right hand side of differential equations (2.69) is in fact directly related to the partial derivatives of H(q, p): p ∂H(q, p) = ∂p m
(2.71)
∂H(q, p) = kq ∂q
(2.72)
We thus arrive at the equations of Hamiltonian dynamics, with their elegant symmetry: ⎧ ∂H(q,p) ⎪ = q˙ ⎨ ∂p (2.73) ⎪ ⎩ ∂H(q,p) = − p˙ ∂q Energy conservation follows directly from these Hamiltonian equations by chain-rule differentiation: ∂H ∂H ∂H = p˙ + q˙ = q˙ p˙ − p˙ q˙ = 0 ∂t ∂p ∂q In the phase plane (q, p), constant energy levels correspond to ellipses 5
More generally in mechanics, the Hamiltonian can be defined by its relationship with the Lagrangian of the system, and is indeed equal to the energy of the system if expressions for the generalized coordinates do not depend on time.
36
2 Finite-Difference Schemes
H(q, p) =
kq 2 p2 + = const 2m 2
(2.74)
For the Hamiltonian system, any particular solution (q(t), p(t)), viewed as a (moving) point in the phase plane, moves along the ellipse corresponding to the energy of the oscillator. Further insight is gained by following the evolution of the w = (q, p) points corresponding to a collection of oscillators (or the same oscillator observed repeatedly under different conditions). The initial coordinates and momenta of a family of oscillators are represented by a set of points in the phase plane. One may imagine that these points fill a certain geometric domain Ω(0) at t = 0 (shaded area in Fig. 2.12). With time, each of the points will follow its own elliptic trajectory, so that at any given moment of time t the initial domain Ω(0) will be transformed into some other domain Ω(t).
Fig. 2.12. The motion of a harmonic oscillator is represented in the (q, p) phase plane by a point moving around an ellipse. Domain Ω(0) contains a collection of such points (corresponding to an ensemble of oscillators or, equivalently, to a set of different initial conditions for one oscillator) at time t = 0. Domain Ω(t) contains the points corresponding to the same oscillators at some arbitrary moment of time t. The area of Ω(t) turns out not to depend on time.
By definition, it is the solutions of the Hamiltonian system that effect the mapping from Ω(0) to Ω(t). These solutions are given by matrix exponentials (see Appendix 2.10): q(t) q(0) w(t) = = exp(At) (2.75) p(t) p(0)
2.5 Schemes for Hamiltonian Systems
37
The Jacobian of this mapping is the determinant of exp(At); as known from linear algebra, this determinant is equal to the product of eigenvalues λ1,2 (exp(At)): det (exp(At)) = λ1 (exp(At)) λ2 (exp(At)) = exp (λ1 (At)) exp (λ2 (At)) = exp (λ1 (At) + λ2 (At)) = exp (Tr(At)) = 1 (2.76) (The eigenvalues of exp(At) are equal to the exponents of the eigenvalues of At; if this looks unfamiliar, see Appendix 2.10, p. 65). Since the determinant of the transformation is unity, the evolution operator preserves the oriented area of Ω(t), in addition to energy conservation that was demonstrated earlier. This result generalizes to higher-dimensional phase spaces in multiparticle systems. Such phase spaces comprise the generalized coordinates qi and momenta pi of N particles. If particle motion is three-dimensional, there are three degrees of freedom per particle6 and hence i = 1, 2, . . . , 3N ; the dimension of the phase space is thus 6N . The most direct analogy with area conservation is that the 6N -dimensional phase volume is conserved under the evolution map [Arn89, HrW93, SSC94]. However, there is more. For any two-dimensional surface in the phase space, take its projections onto the individual phase planes (pi , qi ) and sum up the oriented areas of these projections; this sum is conserved during the Hamiltonian evolution of the surface. Transformations that have this conservation property for the sum of the areas are called symplectic. There is a very deep and elaborate mathematical theory of Hamiltonian phase flows on symplectic manifolds. A symplectic manifold is an evendimensional differentiable manifold endowed with a closed nondegenerate differential 2-form; these notions, however, are not covered in this book. Further mathematical details are described in the monographs by V.I. Arnol’d [Arn89] and J.M. Sanz-Serna & M.P. Calvo [SSC94]. 2.5.2 Symplectic Schemes for Hamiltonian Systems This subsection gives a brief summary of FD schemes that preserve the symplectic property of Hamiltonian systems. The material comes from the paper by R.D. Skeel et al. [RDS97], from the results on Runge–Kutta schemes due to Yu.B. Suris [Sur87]–[Sur90] and J.M. Sanz-Serna [SSC94], and from the compendium of symplectic symmetric Runge–Kutta methods by W. Oevel & M. Sofroniou [OS97]. The governing system of ODEs in Newtonian mechanics and, in particular, molecular dynamics is (2.77) r¨ = f (r), r ∈ Rn 6
Disregarding the internal structure of particles and any degrees of freedom that may be associated with that.
38
2 Finite-Difference Schemes
where r is the position vector for a collection of n interacting particles and f is the normalized force vector (vector of forces divided by particle masses). It is assumed that the forces do not explicitly depend on time. The simplest, and yet effective, difference scheme for this problem is known as the St¨ ormer–Verlet method:7 rn+1 − 2rn + rn−1 = f (rn ) ∆t2
(2.78)
The left hand side of the St¨ ormer scheme is a second-order (with respect to the time step ∆t) approximation of r¨; this approximation is very common. The velocity vector can be computed from the position vector by central differencing: rn+1 − rn−1 (2.79) vn = 2∆t Time-stepping for both vectors r and v simultaneously can be arranged in a “leapfrog” manner: vn+1/2 = vn−1/2 + ∆t f (rn )
(2.80)
rn+1 = rn + ∆t v(n + 1/2)
(2.81)
The leapfrog scheme (2.80), (2.81) is theoretically equivalent to the St¨ormer scheme (2.78), (2.79). The advantage of these schemes is that they are symplectic and at the same time explicit: no systems of equations need to be solved in the process of time-stepping. Several other symplectic integrators are considered by R.D. Skeel et al. [RDS97], but they are all implicit. With regard to the Runge–Kutta methods, the Suris–Sanz-Serna condition of symplecticness is bi aij + bj aji − bi bj = 0,
i, j = 1, 2, . . . s
(2.82)
where bi , aij are the coefficients of an s-stage Runge–Kutta method defined on p. 21, except that here the scheme is no longer explicit – i.e. aij can be nonzero for any pair of indexes i, j. W. Oevel & M. Sofroniou [OS97] give the following summary of symplectic Runge–Kutta schemes. There is a unique one-stage symplectic method with the Butcher tableau 1 2
1 2
1 It represents the implicit scheme ∆t 1 , (rn + rn+1 ) rn+1 = rn + ∆t f tn + 2 2 7
(2.83)
Skeel et al. [RDS97] cite S. Toxvaerd’s statement [Tox94] that “the first known published appearance [of this method] is due to Joseph Delambre (1791)”.
2.6 Schemes for One-Dimensional Boundary Value Problems
39
The following two-stage method is also symplectic: 1 1 √ 2 ± 2 3 1 1 √ 2 ∓ 2 3
1 4
1 4
1 4
∓
1 √ 2 3
± 1 4 1 2
1 √ 2 3 1 2
W. Oevel & M. Sofroniou [OS97] list a number of other methods, up to six-stage ones; these methods were derived using symbolic algebra.
2.6 Schemes for One-Dimensional Boundary Value Problems 2.6.1 The Taylor Derivation After a brief review of time-stepping schemes, we turn our attention to FD schemes for boundary value problems. Such schemes can be applied to various physical fields and potentials in one-dimension (this section), two and three dimensions (the following sections). The most common and straightforward way of generating FD schemes is by Taylor expansion. As the simplest example, consider the Poisson equation in 1D: −
d2 u = f (x) dx2
(2.84)
where f (x) is a given function that in physical problems represents the distribution of sources. The minus sign in the right hand side is conventional in many physical problems (electrostatics, heat transfer, etc.). Let us introduce a grid, for simplicity with a uniform spacing h, and consider a three-point stencil xk−1 , xk , xk+1 , where xk±1 = xk ± h. We shall look for the difference scheme in the form s−1 uk−1 + s0 uk + s+1 uk+1 = f (xk )
(2.85)
where the coefficients s (mnemonic for “scheme”) are to be determined. These coefficients are chosen to approximate, with the highest possible order in terms of the grid size h, the Poisson equation (2.84). More specifically, let u∗ be the exact solution of this equation, and let us write out the Taylor expansions of the values of u∗ at the stencil nodes: 1 2 ∗ h u k + h.o.t. 2 u∗k = u∗k 1 + h2 u∗ k + h.o.t. 2
u∗k−1 = u∗k − hu∗ k + u∗k+1 = u∗k + hu∗ k
40
2 Finite-Difference Schemes
where the primes denote derivatives at the midpoint of the stencil, x = xk , and “h.o.t.” as before stands for “higher order terms”. Substituting these Taylor expansions into the difference scheme (2.85) and collecting the powers of h, one obtains 1 (s−1 +s+1 ) u∗ k h2 + h.o.t. = −uk 2 (2.86) where in the right hand side we took note of the fact that f (xk ) = −uk . The consistency error of the scheme is, by definition,
(s−1 +s0 +s+1 ) u∗k + (−s−1 +s+1 ) u∗ k h +
c = (s−1 + s0 + s+1 )u∗k + (−s−1 + s+1 ) u∗ k h 2 1 s−1 + s+1 + 2 u∗ k h2 + h.o.t. + 2 h
(2.87)
The consistency error tends to zero as h → 0 if and only if s−1 + s0 + s+1 = 0 −s−1 + s+1 = 0 s−1 + s+1 + 2/h2 = 0 from which the coefficients of the scheme are immediately found to be s−1 = s+1 = − 1/h2 ;
s0 = 2/h2
(2.88)
and the difference equation thus reads −uk−1 + 2uk − uk+1 = f (xk ) h2
(2.89)
It is easy to verify that this scheme is of second order with respect to h, i.e. its consistency error c = O(h2 ). The Taylor analysis leading to this scheme is general, however, and can be extended to generate higher-order schemes, provided that the grid stencil is extended as well. As an exercise, the reader may verify that on a 5-point stencil of a uniform grid the scheme with coefficients [1, −16, 30, −16, 1]/(12h2 ) is of order four. Practical implementation of FD schemes involves forming a system of equations for the nodal values of function u, imposing the boundary conditions, solving this system and processing the results. The implementation is described in Section 2.6.4. 2.6.2 Using Constraints to Derive Difference Schemes In this subsection, a slightly different way of deriving difference schemes is presented. The idea is most easily illustrated in 1D but will prove to be fruitful in 2D and 3D, particularly for the development of the so-called “Mehrstellen” schemes (see Sections 2.7.4, 2.8.5).
2.6 Schemes for One-Dimensional Boundary Value Problems
41
For the 1D Poisson equation, we are looking for a three-point FD scheme of the form (2.90) s−1 uk−1 + s0 uk + s+1 uk+1 = sf Parameter sf in the right hand side is not specified a priori and will be determined, along with s±1 and s0 , as a result of a formal procedure described below. Let us again expand the exact solution u into the Taylor series around the midpoint xk of the stencil: u(x) = c0 + c1 (x − xk ) + c2 (x − xk )2 + c3 (x − xk )3 + c4 (x − xk )4 + h.o.t. (2.91) The coefficients cα are of course directly related to the derivatives of u at xk but will initially be treated as undetermined parameters; later on, information available about them will be taken into account. Consistency error of scheme (2.90) can be evaluated by substituting the Taylor expansion (2.91) into the scheme. Upon collecting similar terms for all coefficients cα , we get c = − sf + (s−1 + s0 + s+1 )c0 + (−s−1 + s+1 )hc1 + (s−1 + s+1 )h2 c2 + (−s−1 + s+1 )h3 c3 + (s−1 + s+1 )h4 c4 + h.o.t.
(2.92)
If no information about the coefficients cα were available, the best one could do to minimize the consistency error would be to set sf = 0, s−1 + s0 + s+1 = 0, and −s−1 + s+1 = 0, which yields uk−1 − 2uk + uk+1 = 0. Not surprisingly, this scheme is not suitable for the Poisson equation with a nonzero right hand side: we have not yet made use of the fact that u satisfies this equation – that is, that the Taylor coefficients cα are not arbitrary. In particular, (2.93) u (xk ) = 2c2 = − f (xk ) This condition can be taken into account by using an idea that is, in a sense, dual to the method of Lagrange multipliers in constrained optimization. (Here we are in fact dealing with a special optimization problem – namely, minimization of the consistency error in the asymptotic sense.) In typical constrained optimization, restrictions are imposed on the optimization parameters being sought; in our case, these parameters are the coefficients s of the difference scheme. Note that constraints on optimization parameters, generally speaking, inhibit optimization. In contrast, in our case the constraint applies to the parameters of the function being minimized. This narrows down the set of target functions and facilitates optimization. To incorporate the constraint on c2 (2.93) into the minimization problem, one can introduce an analog of the Lagrange multiplier λ: c = − sf + (s−1 + s0 + s+1 )c0 + (−s−1 + s+1 )hc1 + (s−1 + s+1 )h2 c2
42
2 Finite-Difference Schemes
+ (−s−1 + s+1 )h3 c3 + (s−1 + s+1 )h4 c4 + h.o.t. − λ[2c2 + f (xk )] or equivalently c = (−sf − λf (xk )) + (s−1 + s0 + s+1 )c0 + (−s−1 + s+1 )hc1 + (s−1 h2 + s+1 h2 − 2λ)c2 + (−s−1 + s+1 )h3 c3 + (s−1 + s+1 )h4 c4 + h.o.t. (2.94) where λ is an arbitrary parameter that one is free to choose in addition to the coefficients of the scheme. As Sections 2.7.4 and 2.8.5 show, in 2D and 3D there are several such constraints and therefore several extra free parameters at our disposal. Maximization of the order of the consistency error (2.94) yields the following conditions: −sf − λf (xk ) = 0 s−1 + s0 + s+1 = 0 −s−1 + s+1 = 0 s−1 h2 + s+1 h2 − 2λ = 0 This gives, up to an arbitrary factor, λ = 1, s±1 = h−2 , s0 = −2h−2 , sf = −f (xk ), and the resultant difference scheme is −uk−1 + 2uk − uk+1 = f (xk ) h2
(2.95)
This new “Lagrange-like” derivation produces a well-known scheme in one dimension, but in 2D/3D the idea will prove to be more fruitful and will lead to “Mehrstellen” schemes introduced by L. Collatz [Col66]. 2.6.3 Flux-Balance Schemes The previous analysis was implicitly based on the assumption that the exact solution was sufficiently smooth to admit the Taylor approximation to a desired order. However, Taylor expansion typically breaks down in a number of important practical cases – particularly so in the vicinity of material interfaces. In 1D, this is exemplified by the following problem: du d λ(x) = f (x) on Ω ≡ [a, b], u(a) = ua , u(b) = ub (2.96) − dx dx where the boundary values ua , ub are given. In this equation, λ is the material parameter whose physical meaning varies depending on the problem: it is thermal conductivity in heat transfer, dielectric permittivity in electrostatics, magnetic permeability in magnetostatics (if the magnetic scalar potential is used), and so on. This parameter is usually discontinuous across interfaces of
2.6 Schemes for One-Dimensional Boundary Value Problems
43
different materials. In such cases, the solution satisfies the interface boundary conditions that in the 1D case are du(x− du(x+ 0) 0) = λ(x+ (2.97) ) 0 dx dx where x0 is the discontinuity point for λ(x), and the − and + labels correspond to the values immediately to the left and to the right of x0 , respectively. The quantities −λ(x)du/dx typically have the physical meaning of fluxes: for example, the heat flux (i.e. energy passed through point x per unit time) in heat transfer problems or the flux of charges (that is, electric current) in electric conduction, etc. The fundamental physical principle of energy or flux conservation can be employed to construct a difference scheme. For any chosen subdomain (often called “control volume” – in 1D, a segment), the outgoing energy flow (e.g. heat flux) is equal to the total capacity of sources (e.g. heat sources) within that subdomain. In electro- or magnetostatics, with the electric or magnetic scalar potential formulation, a similar principle of flux balance is used instead of energy balance. For equation (2.96) energy or flux balance can mathematically be derived by integration. Indeed, let ω = [α, β] ⊂ Ω.8 Integrating the underlying equation (2.96) over ω, we obtain β du du f (x) dx (2.98) λ(α) (α) − λ(β) (β) = dx dx α + u(x− 0 ) = u(x0 );
λ(x− 0)
which from the physical point of view is exactly the flux balance equation (outgoing flux from ω is equal to the total capacity of sources inside ω). Fig. 2.13 illustrates the construction of the flux-balance scheme; α and β are chosen as the midpoints of intervals [xk−1 , xk ] and [xk , xk+1 ], respectively. The fluxes in the left hand side of the balance equation (2.98) are approximated by finite differences to yield β uk+1 − uk uk − uk−1 −1 −1 − λ(β) = h λ(α) f (x) dx (2.99) h h h α If the central point xk of the stencil is placed at the material discontinuity (as shown in Fig. 2.13), λ(α) ≡ λ− and λ(β) ≡ λ+ . The factor h−1 is introduced to normalize the right hand side of this scheme to O(1) with respect to the mesh size (i.e. to keep the magnitude of the right hand side approximately constant as the mesh size decreases). The integral in the right hand side can be computed either analytically, if f (x) admits that, or by some numerical quadrature – the simplest one being just f (xk )(β − α). This flux-balance scheme has a solid foundation as a discrete energy conservation condition. From the mathematical viewpoint, this translates into favorable properties of the algebraic system of equations (to be considered in Section 2.6.4): matrix symmetry and, as a consequence, the discrete reciprocity principle. 8
While symbol Ω refers to the whole computational domain, ω denotes its subdomain (typically “small” in some sense).
44
2 Finite-Difference Schemes
Fig. 2.13. A three-point flux balance scheme near a material interface in one dimension.
If the middle node of the stencil is not located exactly at the material boundary, the flux-balance scheme (2.99) is still usable, with λ(α) and λ(β) being the values of λ in the material where the respective point α or β happens to lie. However, numerical accuracy deteriorates significantly. This can be shown analytically by substituting the exact solution into the flux-balance scheme and evaluating the consistency error. Rather than performing this algebraic exercise, we simply consider a numerical illustration. Problem (2.96) is solved in the interval [0, √ 1]. The material boundary point is chosen to be an irrational number a = 1/ 2, so that in the course of the numerical experiment it does not coincide with a grid node of any uniform grid. There are no sources (i.e. f = 0) and the Dirichlet conditions are u(0) = 0, u(1) = 1. The exact solution and the numerical solution with 10 grid nodes are shown in Fig. 2.14. The log-log plot of the relative error norm of the numerical solution vs. the number of grid nodes is given in Fig. 2.15. The dashed line in the figure is drawn for reference to identify the O(h) slope. Comparison with this reference line reveals that the convergence rate is only O(h). Were the discontinuity point to coincide with a grid node, the scheme could easily be shown to be exact – in practice, the numerical solution would be obtained with machine precision. The farther the discontinuity point is from the nearest grid node (relative to the grid size), the higher the numerical error tends to be. This relative distance to the nearest node is plotted in Fig. 2.16 and does indeed correlate clearly with the numerical error in Fig. 2.15. As in the case of Taylor-based schemes of the previous section, the fluxbalance schemes prove to be a very natural particular case of “Trefftz– FLAME” schemes considered in Chapter 4; see in particular Section 4.4.2. Moreover, in contrast with standard schemes, in FLAME the location of material discontinuities relative to the grid nodes is almost irrelevant.
2.6 Schemes for One-Dimensional Boundary Value Problems
45
Fig. 2.14. Solution of the 1D problem with material discontinuity. λ− = 1, λ+ = 10.
Fig. 2.15. Flux-balance scheme: errors vs. the number of grid points for the 1D problem with material discontinuity. λ− = 1, λ+ = 10.
46
2 Finite-Difference Schemes
Fig. 2.16. Relative distance (as a fraction of the grid size) between the discontinuity point and the nearest grid node.
2.6.4 Implementation of 1D Schemes for Boundary Value Problems Difference schemes like (2.89) or (2.99) constitute a local relationship between the values at the neighboring nodes of a particular stencil. Putting these local relationships together, one obtains a global system of equations. With the grid nodes numbered consecutively from 1 to n,9 the n×n matrix of this system is tridiagonal. Indeed, row k of this matrix corresponds to the difference equation – in our case, either (2.89) or (2.99) – that connects the unknown values of u at nodes k − 1, k and k + 1. For example, the flux-balance scheme (2.99) leads to a matrix L with diagonal entries Lkk = (λ+ + λ− )/h and the off-diagonal ones Lk−1,k = −λ− /h, Lk,k+1 = −λ+ /h, where as before λ− and λ+ are the values of material parameter λ at the midpoints of intervals [xk−1 , xk ] and [xk , xk+1 ], respectively. These entries are modified at the end points of the interval to reflect the Dirichlet boundary conditions.10 At the boundary nodes, the Dirichlet condition can be conveniently enforced by setting the corresponding diagonal 9
10
Numbering from 0 to n−1 is often more convenient, and is the default in languages like C/C++. However, I have adopted the default numbering of Matlab and of the classic versions of FORTRAN. The implementation of Neumann and other boundary conditions is covered in all textbooks on FD schemes: L. Collatz [Col66], A.A. Samarskii [Sam01], J.C. Strikwerda [Str04], W.E. Milne [Mil70], and many others.
2.7 Schemes for Two-Dimensional Boundary Value Problems
47
matrix entry to one, the other entries in its row to zero, and the respective entry in the right hand side to the given Dirichlet value of the solution. In addition, if j is a Dirichlet boundary node and i is its neighbor, the Lij uj term in the i-th difference equation is known and therefore gets moved (with the opposite sign) to the right hand side, while the (i, j) matrix entry is simultaneously set to zero. The same procedure is valid in two and three dimensions, except that in these cases a boundary node can have several neighbors.11 The system matrix L corresponding to this three-point scheme is tridiagonal, and the system can be easily solved by Gaussian elimination (A. George & J.W-H. Liu [GL81]) or its modifications (S.K. Godunov & V.S. Ryabenkii [GR87a]).
2.7 Schemes for Two-Dimensional Boundary Value Problems 2.7.1 Schemes Based on the Taylor Expansion For illustration, let us again turn to the Poisson equation – this time in two dimensions: 2 ∂2u ∂ u = f (x, y) (2.100) + − ∂x2 ∂y 2 We introduce a Cartesian grid with grid sizes hx , hy and the number of grid subdivisions Nx , Ny in the x- and y-directions, respectively. To keep the notation simple, we consider the grid to be uniform along each axis; more generally, hx could vary along the x-axis and hy could vary along the y-axis, but the essence of the analysis would remain the same. Each node of the grid can be characterized in a natural way by two integer indices nx and ny corresponding to the x- and y-directions; 1 ≤ nx ≤ Nx + 1, 1 ≤ ny ≤ Ny + 1. To generate a Taylor-based difference scheme for the Poisson equation (2.100), it is natural to approximate the x- and y- partial derivatives separately in exactly the same way as done in 1D. The resulting scheme for grid nodes not adjacent to the domain boundary is −unx −1,ny + 2unx ,ny − unx +1,ny h2x −unx ,ny −1 + 2unx ,ny − unx ,ny +1 + = f (xn , yn ) h2y
(2.101)
where xn , yn are the coordinates of the grid node (nx , ny ). Note that difference scheme (2.101) involves the values of u on a 5-point grid stencil (three points in each coordinate direction, with the middle node shared, Fig. 2.17). As in 11
The same is true in 1D for higher order schemes with more than three stencil nodes in the interior of the domain (more than two nodes in boundary stencils).
48
2 Finite-Difference Schemes
Fig. 2.17. A 5-point stencil for difference scheme (2.101) in 2D.
1D, scheme (2.101) is of second order, i.e. its consistency error is O(h2 ), where h = max(hx , hy ). By expanding the stencil, it is possible – again by complete analogy with the 1D case – to increase the order of the scheme. For example, on the stencil with nine nodes (five in each coordinate direction, with the middle node shared) a fourth order scheme can be obtained by combining two fourth order schemes in the x- and y-directions on their respective 5-point stencils. Other stencils can be used to construct higher-order schemes, and other ideas can be applied to this construction (see for example the Collatz “Mehrstellen” schemes on a 3 × 3 stencil in Section 2.7.4). 2.7.2 Flux-Balance Schemes Let us now turn our attention to a more general 2D problem with a varying material parameter −∇ · ((x, y)∇u) = f (x, y)
(2.102)
where may depend on coordinates but not – in the linear case under consideration – on the solution u. Moreover, will be assumed piecewise smooth, with possible discontinuities only at material boundaries.12 At any material interface boundary, the following conditions hold: −
∂u− ∂u+ = + ∂n ∂n
(2.103)
where “−” and “+” refer to the values on the two sides of the interface boundary and n is the normal to the boundary in a prescribed direction. The integral form of the differential equation (2.102) is, by Gauss’s Theorem, 12
Throughout the book, “smoothness” is not characterized in a mathematically precise way. Rather, it is tacitly assumed that the level of smoothness is sufficient to justify all mathematical operations and analysis.
2.7 Schemes for Two-Dimensional Boundary Value Problems
−
(x, y) γ
∂u dγ = ∂n
49
f (x, y) dω
(2.104)
ω
where ω is a subdomain of the computational domain Ω, γ is the boundary of ω, and n is the outward normal to that boundary. The physical meaning of this integral equation is either energy conservation or flux balance, depending on the application. For example, in heat transfer this equation expresses the fact that the net flow of heat through the surface of volume ω is equal to the total amount of heat generated inside the volume by sources f . In electrostatics, (2.104) is an expression of Gauss’s Law (the flux of the displacement vector D is equal to the total charge inside the volume). The integral conservation principle (2.104) is valid for any subdomain ω. Flux-balance difference schemes are generated by applying this principle to a discrete set of subdomains (“control volumes”) such as the shaded rectangle shown in Fig. 2.18. The grid nodes involved in the construction of the scheme are the same as in Fig. 2.17 and are not labeled to avoid overloading the picture. For this rectangular control volume, the surface flux integral in the
Fig. 2.18. Construction of the flux-balance scheme. The net flux out of the shaded control volume is equal to the total capacity of sources inside that volume.
balance equation (2.104) splits up into four fluxes through the edges of the rectangle. Each of these fluxes can be approximated by a finite difference; for example, un ,n − unx +1,ny (2.105) Flux1 ≈ 1 hy x y hx where 1 is the value of the material parameter at the edge midpoint marked with an asterisk in Fig. 2.18; the hy factor is the length of the right edge of the shaded rectangle. (If the grid were not uniform, this edge length would be the average value of the two consecutive grid sizes.) The complete difference scheme is obtained by summing up all four edge fluxes:
50
2 Finite-Difference Schemes
unx ,ny − unx +1,ny un ,n − unx ,ny +1 + 2 h x x y hx hy unx ,ny − unx −1,ny unx ,ny − unx ,ny −1 + 3 h y + 4 h x = f (xn , yn ) hx hy hx hy
1 hy
The approximation of fluxes by finite differences hinges on the assumption of smoothness of the solution. At material interfaces, this assumption is violated, and accuracy deteriorates. The reason is that the Taylor expansion fails when the solution or its derivatives are discontinuous across boundaries. One can try to remedy that by generalizing the Taylor expansion and accounting for derivative jumps (A. Wiegmann & K.P. Bube [WB00]); however, this approach leads to unwieldy expressions. Another alternative is to replace the Taylor expansion with a linear combination of suitable basis functions that satisfy the discontinuous boundary conditions and therefore approximate the solution much more accurately. This idea is taken full advantage of in FLAME (Chapter 4). 2.7.3 Implementation of 2D Schemes By applying a difference scheme on all suitable grid stencils, one obtains a system of equations relating the nodal values of the solution on the grid. To write this system in matrix form, one needs a global numbering of nodes from 1 to N , where N = (Nx + 1)(Ny + 1). The numbering scheme is in principle arbitrary, but the most natural order is either row-wise or column-wise along the grid. In particular, for row-wise numbering, node (nx , ny ) has the global number n = (Nx + 1)(ny − 1) + nx − 1,
1≤n≤N
(2.106)
With this numbering scheme, the global node numbers of the two neighbors of node n = (nx , ny ) in the same row are n − 1 and n + 1, while the two neighbors in the same column have global numbers n + (Nx + 1) and n − (Nx + 1), respectively. For nodes adjacent to the domain boundary, fictitious “neighbors” with node numbers that are nonpositive or greater than N are ignored. It is then easy to observe that the 5-point stencil of the difference scheme leads to a five-diagonal system matrix, two of the subdiagonals corresponding to node–node connections in the same row, and the other two to connections in the same column. All other matrix entries are zero. The Dirichlet boundary conditions are handled in a way similar to the 1D case. Namely, for a boundary node, the corresponding diagonal entry of the system matrix can be set to one (the other entries in the same row being zero), and the entry of the right hand side set to the required Dirichlet value. Moreover, if j is a boundary node and i is its non-boundary neighbor, the term Lij uj in the difference scheme is known and is therefore moved to the right hand side (with the respective matrix entry (i, j) reset to zero).
2.7 Schemes for Two-Dimensional Boundary Value Problems
51
There is a rich selection of computational methods for solving such linear systems of equations with large sparse matrices. Broadly speaking, these methods can be subdivided into direct and iterative solvers. Direct solvers are typically based on variants of Gaussian or Cholesky decomposition, with node renumbering and possibly block partitioning; see A. George & J.W-H. Liu [GL81, GLe] and Section 3.11 on p. 129. The second one is iterative methods – variants of conjugate gradient or more general Krylov-subspace iterations with preconditioners (R.S. Varga [Var00], Y. Saad [Saa03], D.K. Faddeev & V.N. Faddeeva [FF63], H.A. van der Vorst [vdV03a]) or, alternatively, domain decomposition and multigrid techniques (W. Hackbusch [Hac85], J. Xu [Xu92], A. Quarteroni & A. Valli [QV99]); see also Section 3.13.4. 2.7.4 The Collatz “Mehrstellen” Schemes in 2D For the Poisson equation in 2D −∇2 u = f
(2.107)
consider now a 9-point grid stencil of 3 × 3 neighboring nodes. The node numbering is shown in Fig. 2.19.
Fig. 2.19. The 9-point stencil with the local numbering of nodes as shown. The central node is numbered first, followed by the remaining nodes of the standard 5-point stencil, and then by the four corner nodes.
We set out to find a scheme 9 9 sα uα = α=1
α=1
wα fα
(2.108)
with coefficients {sα }, {wα } (α = 1,2, . . . , 9) such that the consistency error has the highest order with respect to the mesh size. For simplicity, we shall now consider schemes with only one nonzero coefficient w corresponding to the central node (node #1) of the stencil. It is clear that w1 in this case can be set to unity without any loss of generality, as the coefficients s still remain undetermined; thus
52
2 Finite-Difference Schemes
9 α=1
sα uα = f1
The consistency error of this scheme is, by definition, 9 9 c = sα u∗α − f1 = sα u∗α + ∇2 u∗1 α=1
α=1
(2.109)
(2.110)
where u∗ is the exact solution of the Poisson equation and u∗α is its value at node α. The goal is to minimize the consistency error in the asymptotic sense – i.e. to maximize its order with respect to h – by the optimal choice of the coefficients sα of the difference scheme. Suppose first that no additional information about u∗ – other than it is a smooth function – is taken into consideration while evaluating consistency error (2.110). Then, expanding u∗ into the Taylor series around the central point of the 9-point stencil, after straightforward algebra one concludes that only a second order scheme can be obtained – that is, asymptotically the same accuracy level as for the five-point stencil. However, a scheme with higher accuracy can be constructed if additional information about u∗ is taken into account. To fix ideas, let us consider the Laplace (rather than the Poisson) equation ∇2 u∗ = 0
(2.111)
Differentiation of the Laplace equation with respect to x and y yields a few additional pieces of information: ∂ 3 u∗ ∂ 3 u∗ = 0 + ∂x3 ∂x2 ∂y
(2.112)
∂ 3 u∗ ∂ 3 u∗ + = 0 (2.113) ∂x∂y 2 ∂y 3 Another three equations of the same kind can be obtained by taking second derivatives of the Laplace equation, with respect to xx, xy, and yy. As the way these equations are produced is obvious, they are not explicitly written here to save space. All these additional conditions on u∗ impose constraints on the Taylor expansion of u∗ . It is quite reasonable to seek a more accurate difference scheme if only one function (namely, u∗ ) is targeted, rather than a whole class of sufficiently smooth functions. More specifically, let u∗ (x, y) = c0 + c1 x + c2 x2 + c3 x3 + c4 x4 + c5 y + c6 xy + c7 x2 y + c8 x3 y + c9 y 2 + c10 xy 2 + c11 x2 y 2 + c12 y 3 + c13 xy 3 + c14 y 4 + h.o.t.
(2.114)
where cα (α = 1, 2, . . . , 14) are some coefficients (directly related, of course, to the partial derivatives of u∗ ). For convenience, the origin of the coordinate system has been moved to the midpoint of the 9-point stencil.
2.7 Schemes for Two-Dimensional Boundary Value Problems
53
To evaluate and minimize the consistency error (2.110) of the difference scheme, we need the nodal values of the exact solution u∗ . To this end, let us first rewrite expansion (2.114) in a more compact matrix-vector form: u∗ (x, y) = pT c
(2.115)
where pT is a row vector of 15 polynomials in x, y in the order of their appearance in expansion (2.114): pT = [1, x, x2 , . . . , xy 3 , y 4 ] ; c ∈ R15 is a column vector of expansion coefficients. The vector of nodal values of u∗ on the stencil will be denoted with N u∗ and is equal to N u∗ = N c + h.o.t.
(2.116)
The 9 × 15 matrix N comprises the 9 nodal values of the 15 polynomials on the stencil, i.e. (2.117) Nαβ = pβ (xα , yα ) Such matrices of nodal values will play a central role in the “Flexible Local Approximation MEthod” (FLAME) of Chapter 4. Consistency error (2.110) for the Laplace equation then becomes = sT N c + h.o.t. Laplace c
(2.118)
where s ∈ R9 is a Euclidean vector of coefficients. If no information about the expansion coefficients c (i.e. about the partial derivatives of the solution) were available, the consistency error would have to be minimized for all vectors c ∈ R15 . In fact, however, u∗ satisfies the Laplace equation, which imposes constraints on its second-order and higher-order derivatives. Therefore the target space for optimization is actually narrower than the full R15 . If more constraints on the c coefficients are taken into account, higher accuracy of the difference scheme can be expected. A “Lagrange-like” procedure (Section 2.6.2) for incorporating the constraints on u∗ is in some sense dual to the standard technique of Lagrange multipliers: these multipliers are applied not to the optimization parameters but rather to the parameters of the target function u∗ . Thus, we introduce five Lagrange-like multipliers λ1−5 to take into account five constraints on the c coefficients: = sT N c − λ1 (c2 + c9 ) − λ2 (3c3 + c10 ) − λ3 (c7 + 3c12 ) Laplace c − λ4 (6c4 + c11 ) − λ5 (6c14 + c11 ) − λ6 (6c8 + c13 ) + h.o.t. (2.119) For example, the constraint represented by λ1 is just the Laplace equation 2 ∗ 2 ∗ itself (since c2 = 12 ∂∂xu2 , c9 = 12 ∂∂yu2 ); the constraint represented by λ2 is the derivative of the Laplace equation with respect to x (see (2.112)), and so on. In matrix form, equation (2.119) becomes = sT N c − λT Qc + h.o.t. Laplace c
(2.120)
54
2 Finite-Difference Schemes
where matrix Q corresponds to the λ-terms in (2.119). The same relationship can be rewritten in the block-matrix form T T N c + h.o.t. (2.121) c = s λ −Q As in the regular technique of Lagrange multipliers, the problem is now treated as unconstrained. The consistency error is reduced just to the higher order terms if s (2.122) ∈ Null N T ; −QT λ assuming that this null space is nonempty. The computation of matrices N and Q, as well as the null space above, is straightforward by symbolic algebra. As a result, the following coefficients are obtained for a stencil with mesh sizes hx = qx h, hy = qy h in the x- and y-directions, respectively: s1 = 20h−2 s2,3 = − 2h−2 (5qx2 − qy2 )/(qy2 + qx2 ) s4,5 = − 2h−2 (5qy2 − qx2 )/(qy2 + qx2 ) s6−9 = − h−2 . If qx = qy (i.e. hx = hy ), the scheme simplifies: s = h−2 [20, −4, −4, −4, −4, −1, −1, −1, −1] (20 corresponds to the central node, the −4’s – to the mid-edge nodes, and the −1’s – to the corner nodes). This scheme was derived, from different considerations, by L. Collatz in the 1950’s [Col66] and called a “Mehrstellenverfahren” scheme.13 (See also A.A. Samarskii [Sam01] for yet another derivation.) It can be verified that this scheme is of order four in general but of order 6 in the special case of hx = hy . It will become clear in Sections 4.4.4 and 4.4.5 (pp. 209, 210) that the “Mehrstellen” schemes are a natural particular case of Flexible Local Approximation MEthods (FLAME) considered in Chapter 4. More details about the “Mehrstellen” schemes and their application to the Poisson equation in 2D and 3D can be found in the same monographs by Collatz and Samarskii. The 3D case is also considered in Section 2.8.5, as it has important applications to long-range electrostatic forces in molecular dynamics (e.g. C. Sagui & T. Darden [SD99]) and in electronic structure calculation (E.L. Briggs et al. [BSB96]). 13
In the English translation of the Collatz book, these methods are called “Hermitian”.
2.8 Schemes for Three-Dimensional Problems
55
2.8 Schemes for Three-Dimensional Problems 2.8.1 An Overview The structure and subject matter of this section are very similar to those of the previous section on 2D schemes. To avoid unnecessary repetition, issues that are completely analogous in 2D and 3D will be reviewed briefly, but the differences between the 3D and 2D cases will be highlighted. We again start with low-order Taylor-based schemes and then proceed to higher-order schemes, control volume/flux-balance schemes, and “Mehrstellen” schemes. 2.8.2 Schemes Based on the Taylor Expansion in 3D The Poisson equation in 3D has the form 2 ∂2u ∂2u ∂ u = f (x, y, z) + + − ∂x2 ∂y 2 ∂z 2
(2.123)
Finite difference schemes can again be constructed on a Cartesian grid with the grid sizes hx , hy , hz and the number of grid subdivisions Nx , Ny , Nz in the x-, y- and z−directions, respectively. Each node of the grid is characterized by three integer indices nx ny , nz : 1 ≤ nx ≤ Nx + 1, 1 ≤ ny ≤ Ny + 1, 1 ≤ nz ≤ Nz + 1. The simplest Taylor-based difference scheme for the Poisson equation is constructed by combining the approximations of the x-, y- and z− partial derivatives: −unx −1,ny ,nz + 2unx ,ny ,nz − unx +1,ny ,nz ) h2x −unx ,ny −1,nz + 2unx ,ny ,nz − unx ,ny +1,nz + h2y −unx ,ny ,nz −1 + 2unx ,ny ,nz − unx ,ny ,nz +1 + = f (xn , yn , zn ) (2.124) h2z where xn , yn , zn are the coordinates of the grid node (nx , ny , nz ). This difference scheme involves a 7-point grid stencil (three points in each coordinate direction, with the middle node shared between them). As in 1D and 2D, scheme (2.124) is of second order, i.e. its consistency error is O(h2 ), where h = max(hx , hy , hz ). Higher-order schemes can be constructed in a natural way by combining the approximations of each partial derivative on its extended 1D stencil; for example, a 3D stencil with 13 nodes is obtained by combining three 5-point stencils in each coordinate direction, with the middle node shared. The resultant scheme is of fourth order. Another alternative is Collatz “Mehrstellen” schemes, in particular the fourth order scheme on a 19-point stencil considered in Section 2.8.5.
56
2 Finite-Difference Schemes
2.8.3 Flux-Balance Schemes in 3D Consider now a 3D problem with a coordinate-dependent material parameter: −∇ · ((x, y, z)∇u) = f (x, y, z)
(2.125)
As before, will be assumed piecewise-smooth, with possible discontinuities only at material boundaries. The potential is continuous everywhere. The flux continuity conditions at material interfaces have the same form as in 2D: −
∂u− ∂u+ = + ∂n ∂n
(2.126)
where “−” and “+” again refer to the values on the two sides of the interface boundary. The integral form of the differential equation (2.125) is, by Gauss’s Theorem ∂u dS = f (x, y, z) dω (2.127) − (x, y, z) ∂n S ω where ω is a subdomain of the computational domain Ω, S is the boundary surface of ω, and n is the normal to that boundary. As in 2D, the physical meaning of this integral condition is energy or flux balance, depending on the application. A “control volume” ω to which the flux balance condition can be applied is (2.104) is shown in Fig. 2.20. The flux-balance scheme is completely analogous
Fig. 2.20. Construction of the flux-balance scheme in three dimensions. The net flux out of the shaded control volume is equal to the total capacity of sources inside that volume. The grid nodes are shown as circles. For flux computation, the material parameters are taken at the midpoints of the faces.
2.8 Schemes for Three-Dimensional Problems
57
to its 2D counterpart (see (2.106)): unx ,ny ,nz − unx +1,ny ,nz un ,n ,n − unx ,ny +1,nz + 2 h x h z x y z hx hy unx ,ny ,nz − unx −1,ny ,nz unx ,ny ,nz − unx ,ny −1,nz + 3 h y h z + 4 h x h z hx hy unx ,ny ,nz − unx ,ny ,nz +1 unx ,ny ,nz − unx ,ny ,nz −1 + 5 h x h y + 6 h x h y hz hz = f (xn , yn , zn ) hx hy hz (2.128)
1 hy hz
As in 2D, the accuracy of this scheme deteriorates in the vicinity of material interfaces, as the derivatives of the solution are discontinuous. Suitable basis functions satisfying the discontinuous boundary conditions are used in FLAME schemes (Chapter 4), which dramatically reduces the consistency error. 2.8.4 Implementation of 3D Schemes Assuming for simplicity that the computational domain is a rectangular parallelepiped, one introduces a Cartesian grid with Nx , Ny and Nz subdivisions in the respective coordinate directions. The total number of nodes Nm in the mesh (including the boundary nodes) is Nm = (Nx + 1)(Ny + 1)(Nz + 1). A natural node numbering is generated by letting, say, nx change first, ny second and nz third, which assigns the global number n = (Nx + 1)(Ny + 1)(nz − 1) + (Nx + 1)(ny − 1) + nx − 1,
1≤n≤N (2.129) to node (nx , ny , nz ). When, say, a 7-point scheme is applied on all grid stencils, a 7-diagonal system matrix results. Two subdiagonals correspond to the connections of the central node (nx , ny , nz ) of the stencil to the neighboring nodes (nx ± 1, ny , nz ), another two subdiagonals to neighbors (nx , ny ± 1, nz ), and the remaining two subdiagonals to nodes (nx , ny , nz ± 1). Boundary conditions are handled in a way completely analogous to the 2D case. The selection of solvers for the resulting linear system of equations is in principle the same as in 2D, with direct and iterative methods being available. However, there is a practical difference. In two dimensions, thousands or tens of thousands of grid nodes are typically needed to achieve reasonable engineering accuracy; such problems can be easily solved with direct methods that are often more straightforward and robust than iterative algorithms. In 3D, the number of unknowns can easily reach hundreds of thousands or millions, in which case iterative methods may be the only option.14
14
Even for the same number of unknowns in a 2D and a 3D problem, in the 3D case the number of nonzero entries in the system matrix is greater, the sparsity pattern of the matrix is different, and the 3D solver requires more memory and CPU time.
58
2 Finite-Difference Schemes
2.8.5 The Collatz “Mehrstellen” Schemes in 3D The derivation and construction of the “Mehrstellen” schemes in 3D are based on the same ideas as in the 2D case, Section 2.7.4. For the Laplace equation, the “Mehrstellen” scheme can also be obtained as a direct and natural particular case of FLAME schemes in Chapter 4. The 19-point stencil for a fourth order “Mehrstellen” scheme is obtained by discarding the eight corner nodes of a 3×3×3 node cluster. The coefficients of the scheme for the Laplace equation on a uniform grid with hx = hy = hz are visualized in Fig. 2.21.
Fig. 2.21. For the Laplace equation, this fourth order “Mehrstellen”-Collatz scheme on the 19-point stencil is a direct particular case of Trefftz–FLAME. The grid sizes are equal in all three directions. For visual clarity, the stencil is shown as three slices c along the y axis. (Reprinted by permission from [Tsu06] 2006 Elsevier.)
In the more general case of unequal mesh sizes in the x-, y- and z−directions, the “Mehrstellen” scheme is derived in the monographs by L. Collatz and A.A. Samarskii. E.L. Briggs et al. [BSB96] list the coefficients of the scheme in a concise table form. The end result is as follows.
The coefficient corresponding to the central node of the stencil is 4/3 αh−2 α (where α = x, y, z). The coefficients corresponding to the two immediate
−2 neighbors of the central node in the α direction are −5/6h−2 α + 1/6 β hβ (β = x, y or z). Finally, the coefficients corresponding to the nodes displaced by hα and hβ in both α- and β-coordinate directions relative to the central −2 node are −1/12h−2 α −1/12hβ .
2.9 Consistency and Convergence of Difference Schemes
59
If the Poisson equation (2.123) rather than the Laplace equation is solved, with f = f (x, y, z) a smooth function of coordinates, the right hand side of
6 1 the 19-point Mehrstellen scheme is fh = 12 f0 + 12 α=1 fα , where f0 is the value of f at the middle node of the stencil and fα are the values of f at the six immediate neighbors of that middle node. Thus the computation of the right hand side involves the same 7-point stencil as for the standard secondorder scheme for the Laplace equation, not the whole 19-point stencil. HODIE schemes by R.E. Lynch & J.R. Rice [LR80] generalize the Mehrstellen schemes and include additional points in the computation of the right hand side.
2.9 Consistency and Convergence of Difference Schemes This section presents elements of convergence and accuracy theory of FD schemes. A more comprehensive and rigorous treatment is available in many monographs (e.g. L. Collatz [Col66], A.A. Samarskii [Sam01], J.C. Strikwerda [Str04], W.E. Milne [Mil70]). Consider a differential equation in 1D, 2D or 3D Lu = f
(2.130)
that we wish to approximate by a difference scheme (i) (i)
Lh uh
= fhi
(2.131) (i)
on stencil (i) containing a given set of grid nodes. Here uh is the Euclidean vector of the nodal values of the numerical solution on the stencil. Merging the difference schemes on all stencils into a global system of equations, one obtains (2.132) Lh uh = f h where uh and f h are the numerical solution and the right hand side, respectively, viewed as Euclidean vectors of nodal values on the whole grid. Exactly in what sense does (2.132) approximate the original differential equation (2.130)? A natural requirement is that the exact solution u∗ of the differential equation should approximately satisfy the difference equation. To write this condition rigorously, we need to substitute u∗ into the difference scheme (2.132). Since this scheme operates on the nodal values of u∗ , a notation for these nodal values is in order. We shall use the calligraphic letter N for this purpose: N u∗ will mean the Euclidean vector of nodal values of u∗ on the whole grid. Similarly, N (i) u∗ is the Euclidean vector of nodal values of u∗ on a given stencil (i). The consistency error vector c ≡ {ci }ni=1 of scheme (2.132) is the residual obtained when the exact solution is substituted into the difference equation; that is, (2.133) Lh N u∗ = f h + c
60
2 Finite-Difference Schemes
where as before the underscored symbols are Euclidean vectors. The consistency error (a number) is defined as a norm of the error vector: (2.134) consistency error ≡ c (h) = c k = Lh N u∗ − f h k
where k is usually 1, 2, or ∞ (see Appendix 2.10 for definitions of these norms). There is, however, one caveat. According to definition (2.134), the meaningless scheme h100 ui = 0 has consistency error of order 100 for any differential equation with a bounded solution. It is natural to interpret such high-order consistency just as an artifact of scaling and to apply a normalization condition across the board for all schemes. Specifically, we shall assume that the difference schemes are scaled in such a way that c1 f (r) ≤ f hi ≤ c2 f (r),
∀r ∈ Ω(i)
(2.135)
where c1,2 do not depend on i and h. We shall call a scheme consistent if, with scaling (2.135), the consistency error tends to zero as h → 0: (i) (2.136) c = Lh N (i) u∗ − uhi → 0 as h → 0 k
Consistency is usually relatively easy to establish. For example, the Taylor expansions in Section 2.6.1 show that the consistency error of the three-point scheme for the Poisson equation in 1D is O(h2 ); see (2.87)–(2.89). This scheme is therefore consistent. Unfortunately, consistency by itself does not guarantee convergence. To see why, let us compare the difference equations satisfied by the numerical solution and the exact solution, respectively: Lh N u∗ = f h + c
(2.137)
Lh uh = f h
(2.138)
These are equations (2.132) and (2.133) written together for convenience. Clearly, systems of equations for the exact solution u∗ (more precisely, its nodal values N u∗ ) and for the numerical solution uh have slightly different right hand sides. Consistency error c is a measure of the residual of the difference equation, which is different from the accuracy of the numerical solution of this equation. Does the small difference c in the right hand sides of (2.137) and (2.138) translate into a comparably small difference in the solutions themselves? If yes, the scheme is called stable. A formal definition of stability is as follows: h ≡ h k ≡ uh − N u∗ k ≤ Cc k
(2.139)
where the factor C may depend on the exact solution u∗ but not on the mesh size h.
2.9 Consistency and Convergence of Difference Schemes
61
Stability constant C is linked to the properties of the inverse operator L−1 h . Indeed, subtracting (2.137) from (2.138), one obtains an expression for the error vector: (2.140) h ≡ uh − N u∗ = L−1 h c (assuming that Lh is nonsingular). Hence the numerical error can be estimated as (2.141) h ≡ h k ≡ uh − N u∗ k ≤ L−1 h k c k where the matrix norm for Lh is induced by the vector norm, i.e., for a generic square matrix A, Axk Ak = max x=0 xk (see Appendix 2.10). In summary, convergence of the scheme follows from consistency and stability. This result is known as the Lax–Richtmyer Equivalence Theorem (see e.g. J.C. Strikwerda [Str04]). To find the consistency error of a scheme, one needs to substitute the exact solution into it and evaluate the residual (e.g. using Taylor expansions). This is a relatively straightforward procedure. In contrast, stability (and, by implication, convergence) are in general much more difficult to establish. For conventional difference schemes and the Poisson equation, convergence is proved in standard texts (e.g. W.E. Milne [Mil70] or J.C. Strikwerda [Str04]). This convergence result in fact applies to a more general class of monotone schemes. Definition 6. A difference operator Lh (and the respective Nm × Nm matrix) is called monotone if Lh x ≥ 0 for vector x ∈ RNm implies x ≥ 0, where vector inequalities are understood entry-wise. In other words, if Lh is monotone and Lh x has all nonnegative entries, vector x must have all nonnegative entries as well. Algebraic conditions related to monotonicity are reviewed at the end of this subsection. To analyze convergence of monotone schemes, the following Lemma will be needed. Lemma 1. If the scheme is scaled according to (2.135) and the consistency condition (2.134) holds, there exists a reference nodal vector u1h such that u1h ≤ U1 and Lh u1h ≥ σ1 > 0,
(2.142)
with numbers U1 and σ1 independent of h. (All vector inequalities are understood entry-wise.) Remark 1. (Notation.) Subscript 1 is meant to show that, as seen from the proof below, the auxiliary potential u1h may be related to the solution of the differential equation with the unit right hand side.
62
2 Finite-Difference Schemes
Proof. The reference potential u1h can be found explicitly by considering the auxiliary problem (2.143) Lu1 = 1 with the same boundary conditions as the original problem. Condition (2.136) applied to the nodal values of u1 implies that for sufficiently small h the consistency error will fall below 12 c1 , where c1 is the parameter in (2.135): 1 (i)T (i) c1 N u1 − fhi ≤ s 2 Therefore, since f = 1 in (2.135), 1 1 |s(i)T N (i) u1 | ≥ |fhi | − fhi − s(i)T N (i) u1 ≥ c1 − c1 = c1 (2.144) 2 2 (the vector inequality is understood entry-wise). Thus one can set u1h = Lh N u1 , with σ1 = 12 c1 and U1 = u1 ∞ . Theorem 1. Let the following conditions hold for difference scheme (2.132): 1. Consistency in the sense of (2.136), (2.135). 2. Monotonicity : if Lh x ≥ 0, then x ≥ 0
(2.145)
Then the numerical solution converges in the nodal norm, and uh − N u∗ ∞ ≤ c U1 /σ1
(2.146)
where σ1 is the parameter in (2.142). Proof. Let h = uh − N u∗ . By consistency, Lh h ≤ c ≤ c Lh u1h /σ1 = Lh (c u1h /σ1 ) where (2.142) was used. Hence due to monotonicity h ≤ c u1h /σ1
(2.147)
h ≥ − c u1h /σ1
(2.148)
It then also follows that Indeed, if that were not true, one would have (−h ) ≤ c u1h /σ1 , which would contradict the error estimate (2.147) for the system with (−f ) instead of f in the right hand side. We now summarize sufficient and/or necessary algebraic conditions for monotonicity. Of particular interest is the relationship of monotonicity to diagonal dominance, as the latter is trivial to check for any given scheme. The summary is based on the monograph of R.S. Varga [Var00] and the reference book of V. Voevodin & Yu.A. Kuznetsov [VK84]. The mathematical facts are cited without proof.
2.9 Consistency and Convergence of Difference Schemes
63
Proposition 1. A square matrix A is monotone if and only if it is nonsingular and A−1 ≥ 0. [As a reminder, all matrix and vector inequalities in this section are understood entry-wise.] Definition 7. A square matrix A is called an M-matrix if it is nonsingular aij ≤ 0 for all i = j and A−1 ≥ 0. Thus an M-matrix, in addition to being monotone, has nonpositive offdiagonal entries. Proposition 2. All diagonal elements of an M-matrix are positive. Proposition 3. Let a square matrix A have nonpositive off-diagonal entries. Then the following conditions are equivalent: 1. A is an M-matrix. 2. There exists a positive vector w such that A−1 w is also positive. 3. Re λ > 0 for any eigenvalue λ of A. (See [VK84] §36.15 for additional equivalent statements.) Notably, the second condition above allows one to demonstrate monotonicity by exhibiting just one special vector satisfying this condition, which is simpler than verifying this condition for all vectors as stipulated in the definition of monotonicity. Even more practical is the connection with diagonal dominance [VK84]. Proposition 4. Let a square matrix A have nonpositive off-diagonal entries. If this matrix has strong diagonal dominance, it is an M-matrix. Proposition 5. Let an irreducible square matrix A have nonpositive offdiagonal entries. If this matrix has weak diagonal dominance, it is an Mmatrix. Moreover, all entries of A−1 are then (strictly) positive. A matrix is called irreducible if it cannot be transformed to a block-triangular form by permuting its rows and columns. The definition of weak diagonal dominance for a matrix A is |Aij | (2.149) |Aii | ≥ j
in each row i. The condition of strong diagonal dominance is obtained by changing the inequality sign to strict. Thus diagonal dominance of matrix Lh of the difference scheme is a sufficient condition for monotonicity if the off-diagonal entries of Lh are nonpositive. As a measure of the relative magnitude of the diagonal elements, one can use mini |Lh, ii | (2.150) q = j |Lh,ij | with matrix Lh being weakly diagonally dominant for q = 0.5 and diagonal for q = 1. Diagonal dominance is a strong condition that unfortunately does not hold in general.
64
2 Finite-Difference Schemes
2.10 Summary and Further Reading This chapter is an introduction to the theory and practical usage of finite difference schemes. Classical FD schemes are constructed by the Taylor expansion over grid stencils; this was illustrated in Sections 2.1–2.2 and parts of Sections 2.6–2.8. The chapter also touched upon classical schemes (Runge–Kutta, Adams and others) for ordinary differential equations and special schemes that preserve physical invariants of Hamiltonian systems. Somewhat more special are the Collatz “Mehrstellen” schemes for the Poisson equation. These schemes (9-point in 2D and 19-point in 3D) are described in Sections 2.7.4 and 2.8.5. Higher approximation accuracy is achieved, in essence, by approximating the solution of the Poisson equation rather than a generic smooth function. We shall return to this idea in Chapter 4 and will observe that the Mehrstellen schemes are, at least for the Laplace equation, a natural particular case of “Flexible Local Approximation MEthods” (FLAME) considered in that chapter. In fact, in FLAME the classic FD schemes and the Collatz Mehrstellen schemes stem from one single principle and one single definition of the scheme. Very important are the schemes based on flux or energy balance for a control volume; see Sections 2.6.3, 2.7.2, and 2.8.3. Such schemes are known to be quite robust, which is particularly important for problems with inhomogeneous media and material interfaces. The robustness can be attributed to the underlying solid physical principles (conservation laws). For further general reading on FD schemes, the interested reader may consider the monographs by L. Collatz [Col66], J.C. Strikwerda [Str04], A.A. Samarskii [Sam01]. A comprehensive source of information not just on FD schemes but also on numerical methods for ordinary and partial differential equations in general is the book by A. Iserles [Ise96]. It covers one-step and multistep schemes for ODE, Runge–Kutta methods, schemes for stiff systems, FD schemes for the Poisson equation, the Finite Element Method, algebraic system solvers, multigrid and other fast solution methods, diffusion and hyperbolic equations. For readers interested in schemes for fluid dynamics, S.V. Patankar’s text [Pat80] may serve as an introduction. A more advanced book by T.J. Chung [Chu02] covers not only finite-difference, but also finite-volume and finite element methods for fluid flow. Also well-known and highly recommended are two monographs by R.J. LeVeque: one on schemes for advection-diffusion equations, with the emphasis on conservation laws [LeV96], and another one with a comprehensive treatment of hyperbolic problems [LeV02a]. The book by H.-G. Roos et al. [HGR96], while focusing (as the title suggests) on the mathematical treatment of singularly perturbed convection-diffusion problems, is also an excellent source of information on finite-difference schemes in general. For theoretical analysis and computational methods for fluid dynamics on the microscale, see books by G. Karniadakis et al. [KBA01] and by J.A. Pelesko & D.H. Bernstein [PB02].
2.10 Summary and Further Reading
65
Several monographs and review papers are devoted to schemes for electromagnetic applications. The literature on Finite-Difference Time-Domain (FDTD) schemes for electromagnetic wave propagation is especially extensive; see http://www.fdtd.org. The most well-known FDTD monograph is by A. Taflove & S.C. Hagness [TH05]. The book by A.F. Peterson et al. [PRM98] covers, in addition to FD schemes in both time and frequency domain, integral equation techniques and the Finite Element Method for computational electromagnetics.
Appendix: Frequently Used Vector and Matrix Norms The following vector and matrix norms are used most frequently. n x1 = |xi | i=1
A1 = x2 = A2 =
n
max
i=1
1≤j≤n
n i=1
|Aij |
|xi |2
12
1
max λi2 (A∗ A)
1≤i≤n
(2.151) (2.152) (2.153) (2.154)
where A∗ is the Hermitian conjugate (= the conjugate transpose) of matrix A, and λi are the eigenvalues. x∞ = A∞ =
max |xi |
1≤i≤n
max
n
1≤i≤n
j=1
|Aij |
(2.155) (2.156)
See linear algebra textbooks, e.g. Y. Saad [Saa03], R.A. Horn & C.R. Johnson [HJ90], F.R. Gantmakher [Gan59, Gan88] for further analysis and proofs.
Appendix: Matrix Exponential It is not uncommon for an operation over some given class of objects to be defined in two (or more) different ways that for this class are equivalent. Yet one of these ways could have a broader range of applicability and can hence be used to generalize the definition of the operation. This is exactly the case for the exponential operation. One way to define exp x is via simple arithmetic operations – first for x integer via repeated multiplications, then for x rational via roots, and then for all real x.15 While 15
The rigorous mathematical theory – based on either Dedekind’s cuts or Cauchy sequences – is, however, quite involved; see e.g. W. Rudin [Rud76].
66
2 Finite-Difference Schemes
this definition works well for real numbers, its direct generalization to, say, complex numbers is not straightforward (because of the ambiguity of roots), and generalization to more complicated objects like matrices is even less clear. At the same time, the exponential function admits an alternative definition via the Taylor series ∞ xn (2.157) exp x = n! n=0 that converges absolutely for all x. This definition is directly applicable not only to complex numbers but to matrices and operators. Matrix exponential can be defined as ∞ An (2.158) exp A = n! n=0 where A is an arbitrary square matrix (real or complex). This infinite series converges for any matrix, and exp(A) defined this way can be shown to have many of the usual properties of the exponential function – most notably, exp((α + β)A) = exp(αA) + exp(βA),
∀α ∈ C, ∀β ∈ C
(2.159)
If A and B are two commuting square matrices of the same size, AB = BA, then exp(A + B) = exp A exp B, if AB = BA (2.160) Unfortunately, for non-commuting matrices this property is not generally true. For a system of ordinary differential equations written in matrix-vector form as dy(t) = Ay, y ∈ Rn (2.161) dt the solution can be expressed via matrix exponential in a very simple way: y(t) = exp(At) y0
(2.162)
Note that if matrices A and A˜ are related via a similarity transform ˜ A = S −1 AS
(2.163)
then
˜ −1 AS ˜ = S −1 A˜2 S A2 = S −1 ASS and A3 = S −1 A˜3 S, etc. – i.e. powers of A and A˜ are related via the same similarity transform. Substituting this into the Taylor series (2.158) for matrix exponential, one obtains exp A = S −1 exp A˜ S
(2.164)
This is particularly useful if matrix A is diagonalizable; then A˜ can be made ˜ is a diagonal matrix diagonal and contains the eigenvalues of A, and exp(A) containing the exponents of these eigenvalues.16 16
Matrices with distinct eigenvalues are diagonalizable; so are symmetric matrices.
2.10 Summary and Further Reading
67
Since matrix exponential is intimately connected with such difficult problems as full eigenvalue analysis and solution of general ODE systems, it is not surprising that the computation of exp(A) is itself highly complex in general. The curious reader may find it interesting to see the “nineteen dubious ways to compute the exponential of a matrix” (C. Moler & C. Van Loan, [ML78], [ML03]; see also W.A. Harris et al. [WAHFS01].
3 The Finite Element Method
3.1 Everything is Variational The Finite Element Method (FEM) belongs to the broad class of variational methods, and so it is natural to start this chapter with an introduction and overview of such methods. This section emphasizes the importance of the variational approach to computation: it can be claimed – with only a small bit of exaggeration – that all numerical methods are variational. To understand why, let us consider the Poisson equation in one, two or three dimensions as a model problem: Lu ≡ −∇2 u = ρ
in Ω
(3.1)
This equation describes, for example, the distribution of the electrostatic potential u corresponding to volume charge density ρ if the dielectric permittivity is normalized to unity. Solution u is sought in a functional space V (Ω) containing functions with a certain level of smoothness and satisfying some prescribed conditions on the boundary of domain Ω; let us assume zero Dirichlet conditions for definiteness. For purposes of this introduction, the precise mathematical details about the level of smoothness of the right hand side ρ and the boundary of the 2D or 3D domain Ω are not critical, and I mention them only as a footnote.1 It is important to appreciate that solution u has infinitely many “degrees of freedom” – in mathematical terms, it lies in an infinite-dimensional functional space. In 1
The domain is usually assumed to have a Lipschitz-continuous boundary; f ∈ L2 (Ω), u ∈ H 2 (Ω), where L2 and H 2 are the Lebesgue and Sobolev spaces standard in mathematical analysis. The requirements on the smoothness of u are relaxed in the weak formulation of the problem considered later in this chapter. Henri L´eon Lebesgue (1875–1941) – a French mathematician who developed measure and integration theory. Sergei L’vovich Sobolev (1908–1989) – a Russian mathematician, renowned for his work in mathematical analysis (Sobolev spaces, weak solutions and generalized functions).
70
3 The Finite Element Method
contrast, any numerical solution can only have a finite number of parameters. A general and natural form of such a solution is a linear combination of a finite number n of linearly independent approximating functions ψα ∈ V (Ω): unum =
n
cα ψα
(3.2)
α=1
where cα are some coefficients (in the example, real; for other problems, these coefficients could be complex). We may have in mind a set of polynomial functions as a possible example of ψα (ψ1 = 1, ψ2 = x, ψ3 = y, ψ4 = xy, ψ5 = x2 , etc., in 2D). One important constraint, however, is that these functions must satisfy the Dirichlet boundary conditions, and so only a subset of polynomials will qualify. One of the distinguishing features of finite element analysis is a special procedure for defining piecewise-polynomial approximating functions. This procedure will be discussed in more detail in subsequent sections. The key question now is: what are the “best” parameters cα that would produce the most accurate numerical solution (3.2)? Obviously, we first need to define “best”. It would be ideal to have a zero residual R ≡ Lunum − ρ
(3.3)
in which case the numerical solution would in fact be exact. That being in general impossible, the constraints on R need to be relaxed. While R may not be identically zero, let us require that there be a set of “measures of fitness” of the solution – numbers fβ (R) – that are zero: fβ (R) = 0,
β = 1, 2, . . . , n
(3.4)
It is natural to have the number of these measures, i.e. the number of conditions (3.4), equal to the number of undetermined coefficients cα in expansion (3.2). In mathematical terms, the numbers fβ are functionals: each of them acts on a function (in this case, R) and produces a number fβ (R). The functionals can be real or complex, depending on the problem. To summarize: the numerical solution is sought as a linear combination of n approximating functions, with n unknown coefficients; to determine these coefficients, one imposes n conditions (3.4). As it is difficult to deal with nonlinear constraints, the functionals fβ are almost invariably chosen as linear. Example 1. Consider the 1D Poisson equation with the right hand side ρ(x) = cos x over the interval [−π/2, π/2]: −
d2 u = cos x, dx2
π π =u =0 u − 2 2
(3.5)
The obvious exact solution is u∗ (x) = cos x. Let us find a numerical solution using the ideas outlined above.
3.1 Everything is Variational
71
Let the approximating functions ψα be polynomials in x. To keep the calculation as simple as possible, the number of approximating functions in this example will be limited to two only. Linear polynomials (except for the one identically equal to zero) do not satisfy the zero Dirichlet boundary conditions and hence are not included in the approximating set. As the solution must be an even function of x, a sensible (but certainly not unique) choice of the approximating functions is π π x+ , ψ1 = x − 2 2
π 2 π 2 ψ2 = x − x+ 2 2
(3.6)
The numerical solution is thus unum = u1 ψ1 + u2 ψ2
(3.7)
Here u is a Euclidean coefficient vector in R2 with components u1,2 . Euclidean vectors are underlined to distinguish them from functions of spatial variables. The residual (3.3) then is R = − u1 ψ1 − u2 ψ2 − cos x
(3.8)
As a possible example of “fitness measures” of the solution, consider two functionals that are defined as the values of R at points x = 0 and x = π/4:2 π (3.9) f1 (R) = R(0); f2 (R) = R 4 With this choice of the test functionals, residual R, while not zero everywhere (which would be ideal but ordinarily not achievable), is forced by conditions (3.4) to be zero at least at points x = 0 and x = π/4. Furthermore, due to the symmetry of the problem, R will automatically be zero at x = −π/4 as well; this extra point comes as a bonus in this example. Finally, the residual is zero at the boundary points because both exact and numerical solutions satisfy the same Dirichlet boundary condition by construction. The reader may recognize functionals (3.9) as Dirac delta functions δ(x) and δ(x − π/4), respectively. The use of Dirac deltas as test functionals in variational methods is known as collocation; the value of the residual is forced to be zero at a certain number of “collocation points” – in this example, two: x = 0 and x = π/4. The two functionals (3.9), applied to residual (3.8), produce a system of two equations with two unknowns u1,2 : −u1 ψ1 (0) − u2 ψ2 (0) − cos 0 = 0 2
It is clear that these functionals are linear. Indeed, to any linear combination of two different Rs there corresponds a similar linear combination of their pointwise values.
72
3 The Finite Element Method
−u1 ψ1
π
− u2 ψ2
π
4 4 In matrix-vector form, this system is ψ1 (0) ψ2 (0) ; Lu = ρ, L = − ψ1 ( π4 ) ψ2 ( π4 )
− cos ρ =
π = 0 4
cos 0 cos π4
=
1
√ 2 2
(3.10)
It is not difficult to see that for an arbitrary set of approximating functions ψ and test functionals f the entry Lαβ of this matrix is fα (ψβ ). In the present example, with the approximating functions chosen as (3.6), matrix L is easily calculated to be −2 9.869604 L ≈ −2 2.467401 with seven digits of accuracy. The vector of expansion coefficients then is −0.3047378 u ≈ 0.03956838 With these values of the coefficients, and with the approximating functions of (3.6), the numerical solution becomes π π 2 π 2 π x+ + 0.03956838 x − x+ unum ≈ − 0.3047378 x − 2 2 2 2 (3.11) The numerical error is shown in Fig. 3.2 and its absolute value is in the range of (3 ÷ 8) × 10−3 . The energy norm of this error is ∼ 0.0198. (Energy norm is defined as π 2 12 2 dw wE = dx (3.12) dx −π 2 for any differentiable function w(x) satisfying the Dirichlet boundary conditions.)3 Given that the numerical solution involves only two approximating functions with only two free parameters, the result certainly appears to be remarkably accurate.4 This example, with its more than satisfactory end result, is a good first illustration of variational techniques. Nevertheless the approach described above is difficult to turn into a systematic and robust methodology, for the following reasons: 1. The approximating functions and test functionals (more specifically, the collocation points) have been chosen in an ad hoc way; no systematic strategy is apparent from the example. 3
4
In a more rigorous mathematical context, w would be treated as a function in the Sobolev space H01 [− π2 , π2 ], but for the purposes of this introduction this is of little consequence. Still, an even better numerical solution will be obtained in the following example (Example 2 on p. 73).
3.1 Everything is Variational
73
Fig. 3.1. Solution by collocation (3.11) in Example 1 (solid line) is almost indistinguishable from the exact solution u∗ = cos x (markers). See also error plot in Fig. 3.2.
2. It is difficult to establish convergence of the numerical solution as the number of approximating functions increases, even if a reasonable way of choosing the approximating functions and collocation points is found. 3. As evident from (3.10), the approximating functions must be twice differentiable. This may be too strong a constraint. It will become apparent in the subsequent sections of this chapter that the smoothness requirements should be, from both theoretical and practical point of view, as weak as possible. The following example (Example 2) addresses the convergence issue and produces an even better numerical solution for the 1D Poisson equation considered above. The Finite Element Method covered in the remainder of this chapter provides an elegant framework for resolving all three matters on the list. Example 2. Let us consider the same Poisson equation as in the previous example and the same approximating functions ψ1,2 (3.6). However, the test functionals f1,2 are now chosen in a different way: fα (R) =
π 2
−π 2
R(x) ψα (x) dx
(3.13)
74
3 The Finite Element Method
Fig. 3.2. Error of solution by collocation (3.11) in Example 1. (Note the 10−3 scaling factor.)
In contrast with collocation, these functionals “measure” weighted averages rather than point-wise values of R.5 Note that the weights are taken to be exactly the same as the approximating functions ψ; this choice signifies the Galerkin method. Substituting R(x) (3.8) into Galerkin equations (3.13), we obtain a linear system Lu = ρ,
Lαβ = −
π 2
−π 2
ψβ (x) ψα (x) dx; ρα =
π 2
−π 2
ρ(x) ψα (x) dx (3.14)
Notably, the expression for matrix entries Lβα can be made more elegant using integration by parts and taking into account zero boundary conditions: Lαβ =
π 2
−π 2
ψα (x) ψβ (x) dx
(3.15)
This reveals the symmetry of the system matrix. The symmetry is due to two factors: (i) the operator L of the problem – in this case, Laplacian in the space of functions with zero Dirichlet conditions – is self-adjoint; this allowed 5
Loosely speaking, collocation can be viewed as a limiting case of weighted averaging, with the weight concentrated at one point as the Dirac delta.
3.2 The Weak Formulation and the Galerkin Method
75
the transformation of the integrand to the symmetric form; (ii) the Galerkin method was used. The Galerkin integrals in the expressions for the system matrix (3.15) and the right hand side (3.14) can be calculated explicitly:6 π3 35 − 7π 2 −4 ; ρ = (3.16) L = 48 − 4π 2 105 −7π 2 − 2π 4 Naturally, this matrix is different from the matrix in the collocation method of the previous example (albeit denoted with the same symbol). In particular, the Galerkin matrix is symmetric, while the collocation matrix is not. The expansion coefficients in the Galerkin method are 1 −60π 2 (3π 2 − 28) −0.3154333 ≈ u = L−1 ρ = 7 0.03626545 −840 (π 2 − 10) π The numerical values of these coefficients differ slightly from the ones obtained by collocation in the previous example. The Galerkin solution is π π 2 π 2 π x+ + 0.03626545 x − x+ unum ≈ − 0.3154333 x − 2 2 2 2 (3.17) The error of solution (3.17) is plotted in Fig. 3.3; it is seen to be substantially smaller than the error for collocation. Indeed, the energy norm of this error is ∼ 0.004916, which is almost exactly four times less than the same error measure for collocation. The higher accuracy of the Galerkin solution (at least in the energy norm) is not an accident. The following section shows that the Galerkin solution in fact minimizes the energy norm of the error; in that sense, it is the “best” of all possible numerical solutions representable as a linear combination of a given set of approximating functions ψ.
3.2 The Weak Formulation and the Galerkin Method In this section, the variational approach outlined above is cast in a more general and precise form; however, it does make sense to keep the last example (Example 2) in mind for concreteness. Let us consider a generic problem of the form Lu = ρ, u ∈ V = V (Ω) (3.18) of which the Poisson equation (3.1) on p. 69 is a simple particular case. Here operator L is assumed to be self-adjoint with respect to a given inner product (· , ·) in the functional space V under consideration: 6
In more complicated cases, numerical quadratures may be needed.
76
3 The Finite Element Method
Fig. 3.3. Error of the Galerkin solution (3.7) in Example 2. (Note the 10−3 scaling factor.)
(Lu, v) = (u, Lv),
∀u, v ∈ V
(3.19)
The reader unfamiliar with the notion of inner product may view it just as a shorthand notation for integration: wv dΩ (w, v) ≡ Ω
This definition is not general7 but sufficient in the context of this section. Note that operators defined in different functional spaces (or, more generally, in different domains) are mathematically different, even if they can be described by the same expression. For example, the Laplace operator in a functional space with zero boundary conditions is not the same as the Laplace operator in a space without such conditions. One manifestation of this difference is that the Laplace operator is self-adjoint in the first case but not so in the second. Applying to the operator equation (3.18) inner product with an arbitrary function v ∈ V (in the typical case, multiplying both sides with v and integrating), we obtain (Lu, v) = (ρ, v), ∀v ∈ V (3.20) 7
Generally, inner product is a bilinear (sesquilinear in the complex case) (conjugate-)symmetric positive definite form.
3.2 The Weak Formulation and the Galerkin Method
77
Clearly, this inner-product equation follows from the original one (3.18). At the same time, because v is arbitrary, it can be shown under fairly general mathematical assumptions that the converse is true as well: original equation (3.18) follows from (3.20); that is, these two formulations are equivalent (see also p. 84). The left hand side of (3.20) is a bilinear form in u, v; in addition, if L is self-adjoint, this form is symmetric. This bilinear form will be denoted as L(u, v) (making symbol L slightly overloaded): L(u, v) ≡ (Lu, v),
∀v ∈ V
(3.21)
To illustrate this definition: in Examples 1, 2 this bilinear form is π2 π2 u v dx = u v dx L(u, v) ≡ − π 2
(3.22)
π 2
The last integration-by-parts transformation appears innocuous but has profound consequences. It replaces the second derivative of u with the first derivative, thereby relaxing the required level of smoothness of the solution. The following illustration is simple but instructive. Let u be a function with a “sharp corner” – something like |x| in Fig. 3.4: it has a discontinuous first derivative and no second derivative (in the sense of regular calculus) at x = 0. However, this function can be approximated, with an arbitrary degree of accuracy, by a smooth one – it is enough just to “round off” the corner.
Fig. 3.4. Rounding off the corner provides a smooth approximation.
“Accuracy” here is understood in the energy-norm sense: if the smoothed function is denoted with u ˜, then the approximation error is ˜ u − uE ≡ Ω
d˜ u du − dx dx
12
2 dx
where the precise specification of domain (segment) Ω is unimportant.
(3.23)
78
3 The Finite Element Method
For the smooth function u ˜, both expressions for the bilinear form (3.21) are valid and equal. For u, the first definition, containing u in the integrand, is not valid, but the second one, with u , is. It is quite natural to extend the definition of the bilinear form to functions that, while not necessarily smooth enough themselves, can be approximated arbitrarily well – in the energy norm sense – by smooth functions: du dv dΩ, u , v ∈ H01 (Ω) (3.24) L(u, v) ≡ Ω dx dx Such functions form the Sobolev space H 1 (Ω). The subspace H01 (Ω) ⊂ H 1 (Ω) contains functions with zero Dirichlet conditions at the boundary of domain Ω.8 Similarly, for the electrostatic equation (with the dielectric permittivity normalized to unity) Lu ≡ − ∇ · ∇u = ρ (3.25) in a two- or three-dimensional domain Ω with zero Dirichlet boundary conditions,9 the weak formulation is L(u, v) ≡ (∇u, ∇v) = (ρ , v)
u , v ∈ H01 (Ω)
where the parentheses denote the inner product of vector functions v · w dΩ, v , w ∈ L32 (Ω) (v, w) ≡
(3.26)
(3.27)
Ω
The analysis leading to the weak formulation (3.26) is analogous to the 1D case: the differential equation is inner-multiplied (i.e. multiplied and integrated) with a “test” function v; then integration by parts moves one of the ∇ operators over from u to v, so that the formulation can be extended to a broader class of admissible functions, with the smoothness requirements relaxed. The weak formulation (3.20) (of which (3.26) is a typical example) provides a very natural way of approximating the problem. All that needs to be done is to restrict both the unknown function u and the test function v in (3.20) to a finite-dimensional subspace Vh ⊂ V : L(uh , vh ) = (ρ, vh ),
∀vh ∈ Vh (Ω)
(3.28)
In Examples 1 and 2 space Vh had just two dimensions; in engineering practice, the dimension of this space can be on the order of hundreds of thousands and 8
9
The rigorous mathematical characterization of “boundary values” (more precisely, traces) of functions in Sobolev spaces is quite involved. See R.A. Adams [AF03] or K. Rektorys [Rek80]. Neumann conditions on the domain boundary and interface boundary conditions between different media will be considered later.
3.2 The Weak Formulation and the Galerkin Method
79
even millions. Also in practice, construction of Vh typically involves a mesh (this was not the case in Examples 1 and 2, but will be the case in the subsequent sections in this chapter); then subscript “h” indicates the mesh size. If a mesh is not used, h can be understood as some small parameter; in fact, one usually has in mind a family of spaces Vh that can approximate the solution of the problem with arbitrarily high accuracy as h → 0. Let us assume that an approximating space Vh of dimension n has been chosen and that ψα (α = 1, . . . , n) is a known basis set in this space. Then the approximate solution is a linear combination of the basis functions: uh =
n
u α ψα
(3.29)
α=1
Here u is a Euclidean vector of coefficients in Rn (or, in the case of problems with complex solutions, in Cn ). This expansion establishes an intimate relationship between the functional space Vh to which uh belongs and the Euclidean space of coefficient vectors u. If functions ψα are linearly independent, there is a one-to-one correspondence between uh and u. Moreover, the bilinear form L(uh , uh ) induces an equivalent bilinear form over Euclidean vectors: (Lu, v) = L(uh , vh )
(3.30)
for any two functions uh , vh ∈ Vh and their corresponding Euclidean vectors u, v ∈ Rn . The left hand side of (3.30) is the usual Euclidean inner product of vectors, and L is a square matrix. From basic linear algebra, each entry Lαβ of this matrix is equal to (Leα , eβ ), where eα is column #α of the identity matrix (the only nonzero entry #α is equal to one); similarly for eβ . At the same time, (Leα , eβ ) is, by definition of L, equal to the bilinear form involving ψα , ψβ ; hence (3.31) Lαβ = (Leα , eβ ) = L(ψα , ψβ ) The equivalence of bilinear forms (3.30) is central in Galerkin methods in general and FEM in particular; it can also be viewed as an operational definition of matrix L. Explicitly the entries of L are defined by the right hand side of (3.31). Example 3 below should clarify this matter further. The Galerkin formulation (3.28) is just a restriction of the weak continuous formulation to a finite-dimensional subspace, and therefore the numerical bilinear form inherits the algebraic properties of the continuous one. In particular, if the bilinear form L is elliptic, i.e. if L(u, u) ≥ c (u, u),
∀u ∈ V (c > 0)
(3.32)
where c is a constant, then matrix L is strictly positive definite and, moreover, (Lu, u) ≥ c (M u, u),
∀u ∈ Rn
(3.33)
80
3 The Finite Element Method
Matrix M is such that the Euclidean form (M u, v) corresponds to the L2 inner product of the respective functions: (M u, v) = (uh , vh )
(3.34)
Mαβ = (ψα , ψβ )
(3.35)
so that the entries are These expressions for matrix M are analogous to expressions (3.30) and (3.31) for matrix L. In FEM, M is often called the mass matrix and L – the stiffness matrix, due to the roles they play in problems of structural mechanics where FEM originated. Example 3. To illustrate the connection between Euclidean inner products and the respective bilinear forms of functions, let us return to Example 2 on p. 73 and choose the two coefficients arbitrarily as u1 = 2, u2 = −1. The corresponding function is π π 2 π 2 π x+ − x− x+ (3.36) u h = u 1 ψ1 + u 2 ψ2 = 2 x − 2 2 2 2 This function of course lies in the two-dimensional space Vh spanned by ψ1,2 . Similarly, let v 1 = 4, v 2 = −3 (also as an arbitrary example); then π π 2 π 2 π v h = v 1 ψ1 + v 2 ψ2 = 4 x − x+ − 3 x− x+ (3.37) 2 2 2 2 In the left hand side of (3.30), matrix L was calculated to be (3.16), and the Euclidean inner product is 3 2π 5 2π 7 8π 3 π 35 − 7π 2 2 4 + + , = (Lu, v) = 2 4 −1 −3 − 2π 105 −7π 3 3 35 (3.38) The right hand side of (3.30) is
π 2 π 2
uh vh dx
where functions uh , vh are given by their expansions (3.36), (3.37). Substitution of these expansions into the integrand above yields exactly the same result as the right hand side of (3.38), namely 2π 5 2π 7 8π 3 + + 3 3 35 This illustrates that the Euclidean inner product of vectors u, v in (3.30) (of which the left hand side of (3.38) is a particular case) is equivalent to the bilinear form L(u, v) of functions u, v (of which the right hand side of (3.38) is a particular case).
3.3 Variational Methods and Minimization
81
By setting vh consecutively to ψ1 , ψ2 , . . ., ψn in (3.28), one arrives at the following matrix-vector form of the variational formulation (3.28): Lu = ρ
(3.39)
with Lαβ = L(ψα , ψβ );
ρα = (ρ, ψα )
(3.40)
This is a direct generalization of system (3.14) on p. 74.
3.3 Variational Methods and Minimization 3.3.1 The Galerkin Solution Minimizes the Error The analysis in this section is restricted to operator L that is self-adjoint in a given functional space V , and the corresponding symmetric (conjugatesymmetric in the complex case) form L(u, v). In addition, if L(u, u) ≥ c(u, u), ∀u ∈ V
(3.41)
for some positive constant c, the form is called elliptic (or, synonymously, coercive). The weak continuous problem is L(u, v) = (ρ, v),
u ∈ V ; ∀v ∈ V
(3.42)
We shall assume that this problem has a unique solution u∗ ∈ V and shall refer to u∗ as the exact solution (as opposed to a numerical one). Mathematical conditions for the existence and uniqueness are cited in Section 3.5. The numerical Galerkin problem is obtained by restricting this formulation to a finite-dimensional subspace Vh ⊂ V : L(uh , vh ) = (ρ, vh ),
uh ∈ Vh ; ∀vh ∈ Vh
(3.43)
where uh is the numerical solution. Keep in mind that uh solves the Galerkin problem in the finite-dimensional subspace Vh only; in the full space V there is, in general, a nonzero residual R(uh , v) ≡ (ρ, v) − L(uh , v) = , v ∈ V
(3.44)
In matrix-vector form, this problem is Lu = ρ
(3.45)
with matrix L and the right hand side ρ defined in (3.40). If matrix L is nonsingular, a unique numerical solution exists. For an elliptic form L – a
82
3 The Finite Element Method
particularly important case in theory and practice – matrix L is positive definite and hence nonsingular. The numerical error is (3.46) h = u h − u A remarkable property of the Galerkin solution for a symmetric form L is that it minimizes the error functional E(uh ) ≡ L(h , h ) ≡ L(uh − u, uh − u)
(3.47)
In other words, of all functions in the finite-dimensional space Vh , the Galerkin solution uhG is the best approximation of the exact solution in the sense of measure (3.47). For coercive forms L, this measure usually has the physical meaning of energy. To prove this minimization property, let us analyze the behavior of functional (3.47) in the vicinity of some uh – that is, examine E(uh + λvh ), where vh ∈ Vh is an increment and λ is an adjustable numerical factor introduced for mathematical convenience. (This factor could be absorbed into vh but, as will soon become clear, it makes sense not to do so. Also, λ can be intuitively understood as “small” but this has no bearing on the formal analysis.) Then, assuming a real form for simplicity, E(uh +λvh ) = L(h +λvh , h +λvh ) = L(h , h ) + 2λL(h , vh ) + λ2 L(vh , vh ) (3.48) At a stationary point of E – and in particular at a maximum or minimum – the term linear in λ must vanish: L(h , vh ) = 0,
∀vh ∈ Vh
This condition is nothing other than L(uh , vh ) = L(u, vh ) = (f, vh ) (The last equality follows from the fact that u is the solution of the weak problem.) This is precisely the Galerkin equation. Thus the Galerkin solution is a stationary point of functional (3.47). If the bilinear form L is elliptic, expression (3.48) for the variation of the energy functional then indicates that this stationary point is in fact a minimum: the term linear in λ vanishes and the quadratic term is positive for a nonzero vh . 3.3.2 The Galerkin Solution and the Energy Functional Error minimization (in the energy norm sense) is a significant strength of the Galerkin method. A practical limitation of the error functional (3.47), however, is that it cannot be computed explicitly: this functional depends on the exact solution that is unknown. At the same time, for self-adjoint problems there is another – and computable – functional for which both the exact
3.4 Essential and Natural Boundary Conditions
83
solution (in the original functional space V ) and the numerical solution (in the chosen finite-dimensional space Vh ) are stationary points. This functional is 1 (3.49) Fu = (ρ, u) − L(u, u), u ∈ V 2 Indeed, for an increment λv, where λ is an arbitrary number and v ∈ V , we have ∆F ≡ F(u+λv) − Fu = (ρ, λv) −
1 1 1 L(λv, u) − L(u, λv) − L(λv, λv) 2 2 2
which for a symmetric real form L is ∆F = λ[(ρ, v) − L(u, v)] −
1 2 λ L(v, v) 2
The zero linear term in λ thus corresponds precisely to the weak formulation of the problem. By a very similar argument, the Galerkin solution is a stationary point of F in Vh . Furthermore, if the bilinear form L is elliptic, the quadratic term λ2 L(v, v) is nonnegative, and the stationary point is a maximum. In electrostatics, magnetostatics and other physical applications functional F is often interpreted as energy. It is indeed equal to field energy if u is the exact solution of the underlying differential equation (or, almost equivalently, of the weak problem). Other values of u are not physically realizable, and hence F in general lacks physical significance as energy and should rather be interpreted as “action” (an integrated Lagrangian). It is not therefore paradoxical that the solution maximizes – not minimizes – the functional.10 This matter is taken up again in Section 6.11 on p. 328 and in Appendix 6.14 on p. 338 in the context of electrostatic simulation. Functional F (3.49) is part of a broader picture of complementary variational principles; see the book by A.M. Arthurs [Art80] (in particular, examples in Section 1.4 of his book11 ).
3.4 Essential and Natural Boundary Conditions So far, for brevity of exposition, only Dirichlet conditions on the exterior boundary of the domain were considered. Now let us turn our attention to 10
11
One could reverse the sign of F , in which case the stationary point would be a minimum. However, this functional would no longer have the meaning of field energy, as its value at the exact solution u would be negative, which is thermodynamically impossible for electromagnetic energy (see L.D. Landau & E.M. Lifshitz [LL84]. A note for the reader interested in the Arthurs book and examples therein. In the electrostatic case, the quantities in these examples are interpreted as follows: U ≡ D (the electrostatic displacement field), v ≡ (the permittivity), Φ = u (potential), q ≡ ρ (charge density).
84
3 The Finite Element Method
quite interesting, and in practice very helpful, circumstances that arise if conditions on part of the boundary are left unspecified in the weak formulation. We shall use the standard electrostatic equation in 1D, 2D or 3D as a model: (3.50) −∇ · ∇u = ρ in Ω; u = 0 on ∂ΩD ⊂ ∂Ω At first, the dielectric permittivity will be assumed a smooth function of coordinates; later, we shall consider the case of piecewise-smooth (e.g. dielectric bodies in a host medium). Note that u satisfies the zero Dirichlet condition only on part of the domain boundary; the condition on the remaining part is left unspecified for now, so the boundary value problem is not yet fully defined. The weak formulation is (∇u, ∇v) = (ρ, v),
u, ∀v ∈ H01 (Ω, ∂ΩD )
(3.51)
H01 (Ω, ∂ΩD ) is the Sobolev space of functions that have a generalized derivative and satisfy the zero Dirichlet condition on ∂ΩD .12 Let us now examine, a little more carefully than we did before, the relationship between the weak problem (3.51) and the differential formulation (3.50). To convert the weak problem into a strong one, one integrates the left hand side of (3.51) by parts: ∂u dS − (∇ · ∇u, v) = (ρ, v) (3.52) v ∂n ∂Ω−∂ΩD It is tacitly assumed that u is such that the differential operator ∇·∇u makes sense. Note that the surface integral is taken over the non-Dirichlet part of the boundary only, as the “test” function v vanishes on the Dirichlet part by definition. The key observation is that v is arbitrary. First, as a particular choice, let us consider test functions v vanishing on the domain boundary. In this case, the surface integral in (3.52) disappears, and we have v(∇ · ∇u + ρ) dΩ = 0 (3.53) (∇ · ∇u + ρ, v) ≡ Ω
This may hold true for arbitrary v only if the integrand I ≡ ∇ · ∇u + ρ
(3.54)
in (3.53) is identically zero. The proof, at least for continuous I, is simple. Indeed, if I were strictly positive at some point r0 inside the domain, it would, 12
These are functions that are either smooth themselves or can be approximated by smooth functions, in the H 1 -norm sense, with any degree of accuracy. Boundary values, strictly speaking, should be considered in the sense of traces (R.A. Adams & J.J.F. Fournier [AF03], K. Rektorys [Rek80]).
3.4 Essential and Natural Boundary Conditions
85
by continuity, have to be positive in some neighborhood of that point. By choosing the test function that is positive in the same neighborhood and zero elsewhere (imagine a sharp but smooth peak centered at r0 as such a test function), one arrives at a contradiction, as the integral in (3.53) is positive rather than zero. This argument shows that the Poisson equation must be satisfied for the solution u of the weak problem. Further observation can be made if we now consider a test function that is nonzero on the non-Dirichlet part of the boundary. In the integrated-by-parts weak formulation (3.52), the volume integrals, as we now know, must vanish if u is the solution, because the Poisson equation is satisfied. Then we have ∂u dS = 0 (3.55) v ∂n ∂Ω−∂ΩD Since v is arbitrary, the integrand must be identically zero – the proof is essentially the same as for the volume integrand I in (3.54). We come to the conclusion that solution u must satisfy the Neumann boundary condition ∂u = 0 ∂n
(3.56)
on the non-Dirichlet part of the domain boundary (for = 0). This is really a notable result. In the weak formulation, if no boundary condition is explicitly imposed on part of the boundary, then the solution will satisfy the Neumann condition. Such “automatic” boundary conditions that follow from the weak formulation are called natural. In contrast, conditions that have to be imposed explicitly are called essential. Dirichlet conditions are essential. For cases other than the model electrostatic problem, a similar analysis is needed to identify natural boundary conditions. As a rule of thumb, conditions containing the normal derivative at the boundary are natural. For example, Robin boundary conditions (a combination of values of u and its normal derivative) are natural. Importantly, the continuity of flux ∂u/∂n across material interfaces is also a natural condition. The analysis is similar to that of the Neumann condition. Indeed, let Γ be the boundary between materials #1,2 with their respective parameters 1,2 . Separately within each material, varies smoothly, but a jump may occur across Γ . With the weak problem (3.51) taken as a starting point, integration by parts yields ∂u ∂u dS − (∇ · ∇u, v) = (ρ, v) [. . .] + v − ∂n 1 ∂n 2 ∂Ω−∂ΩD Γ (3.57) Subscripts 1 and 2 indicate that the respective electric flux density ∂u/∂n is taken in materials 1, 2; n is the unit normal to Γ , directed into material #2
86
3 The Finite Element Method
(this choice of direction is arbitrary). The integrand on the exterior boundary is omitted for brevity, as it is the same as considered previously and leads, as we already know, to the Neumann boundary condition on Ω − ∂ΩD . Consider first the volume integrals (inner products) in (3.57). Using the fact that v is arbitrary, one can show in exactly the same way as before that the electrostatic differential equation must be satisfied throughout the domain, except possibly for the interface boundary where the differential operator may not be valid in the sense of ordinary calculus. Turning then to the surface integral over Γ and again noting that v is arbitrary on that surface, one observes that the integrand – i.e. the flux jump – across the surface must be zero if u is the solution of the weak problem. This is a great practical advantage because no special treatment of material interfaces is needed. For the model electrostatic problem, the finite element algorithm for heterogeneous media is essentially the same as for the homogeneous case. However, for more complicated problems interface conditions may need special treatment and may result in additional surface integrals.13 It is in principle possible to impose natural conditions explicitly – that is, incorporate them into the definition of the functional space and choose the approximating and test functions accordingly. However, this is usually inconvenient and redundant, and therefore is hardly ever done in practice.
3.5 Mathematical Notes: Convergence, Lax–Milgram and C´ ea’s Theorems This section summarizes some essential facts about weak formulations and convergence of Galerkin solutions. The mathematical details and proofs are omitted, one exception being a short and elegant proof of C´ea’s theorem. There are many excellent books on the mathematical theory: an elaborate exposition of variational methods by K. Rektorys [Rek80] and by S.G. Mikhlin [Mik64, Mik65], as well as the well-known text by R. Weinstock [Wei74]; classical monographs on FEM by P.G. Ciarlet [Cia80], by B. Szab´ o & I. Babuˇska [SB91], and a more recent book by S.C. Brenner & L.R. Scott [BS02], among others. Those readers who are not interested in the mathematical details may skip this section – a digest of the underlying mathematical theory – without substantial harm to their understanding of the rest of the chapter. Theorem 2. (Lax–Milgram.) [BS02, Rek80] 13
One interesting example is a hybrid formulation of eddy current problems, with the magnetic vector potential inside a conducting body and the magnetic scalar potential outside. The weak formulation contains a surface integral on the boundary of the conductor. The interested reader may see C.R.I. Emson & J. Simkin [ES83], D. Rodger [Rod83] for the formulation and [Tsu90] for a mathematical analysis.
3.5 Mathematical Notes: Convergence, Lax–Milgram and C´ea’s Theorems
87
Given a Hilbert space V , a continuous and elliptic bilinear form L(· , ·) and a continuous linear functional f ∈ V , there exists a unique u ∈ V such that L(u, v) = f (v), ∀v ∈ V (3.58) As a reminder, a bilinear form is elliptic if L(u, u) ≥ c1 (u, u),
∀u ∈ V
and continuous if L(u, v) ≤ c2 u v,
∀u, v ∈ V
for some positive constants c1,2 . Here the norm is induced by the inner product: 1 (3.59) v ≡ (v, v) 2 Finally in the formulation of the Lax–Milgram theorem, V is the space of continuous linear functionals over V . A linear functional is continuous if f (v) ≤ cv, where c is some constant. The reason why the Lax–Milgram theorem is important is that its conditions correspond to the weak formulations of many problems of mathematical physics, including the model electrostatic problem of the previous section. The Lax–Milgram theorem establishes uniqueness and existence of the (exact) solution of such problems. Under the Lax–Milgram conditions, it is clear that uniqueness and existence also hold in any subspace of V – in particular, for the approximate Galerkin solution. The Lax–Milgram theorem can be proved easily for symmetric forms. Indeed, if L is symmetric (in addition to its continuity and ellipticity required by the conditions of the theorem), this form represents an inner product in V : [u, v] ≡ L(u, v). Then f (v), being a linear continuous functional, can be by the Riesz Representation Theorem (one of the basic properties of Hilbert spaces) expressed via this new inner product as f (v) = [u, v] ≡ L(u, v), which is precisely what the Lax–Milgram theorem states. The more complicated proof for nonsymmetric forms is omitted. Theorem 3. (C´ ea) [BS02, Rek80] Let V be a subspace of a Hilbert space H and L(· , ·) be a continuous elliptic (but not necessarily symmetric) bilinear form on V . Let u ∈ V be the solution of equation (3.58) from the Lax–Milgram theorem. Further, let uh be the solution of the Galerkin problem L(uh , vh ) = f (vh ),
∀vh ∈ Vh
in some finite-dimensional subspace Vh ⊂ V . Then c2 min u − v u − uh ≤ c1 v∈Vh
(3.60)
(3.61)
where c1 and c2 are the ellipticity and continuity constants of the bilinear form L.
88
3 The Finite Element Method
C´ea’s theorem is a principal result, as it relates the error of the Galerkin solution to the approximation error. The latter is much more easily amenable to analysis: good approximation can be produced by various forms of interpolation, while the Galerkin solution emerges from solving a large system of algebraic equations. For a symmetric form L and for the norm induced by L, constants c1,2 = 1 and the Galerkin solution is best in the energy-norm sense, as we already know. Proof. The error of the Galerkin solution is h ≡ uh − u,
u h ∈ Vh
(3.62)
where u is the (exact) solution of the weak problem (3.58) and uh is the solution of the Galerkin problem (3.60). This error itself satisfies a weak problem obtained simply by subtracting the Galerkin equation from the exact one: L(h , vh ) = 0,
∀vh ∈ Vh
(3.63)
This can be interpreted as a generalized orthogonality relationship: the error is “L-orthogonal” to Vh . (If L is not symmetric, it does not induce an inner product, so the standard definition of orthogonality does not apply.) Such an interpretation has a clear geometric meaning: the Galerkin solution is a projection (in a proper sense) of the exact solution onto the chosen approximation space. Then we have L(h , h ) ≡ L(h , uh − u) = L(h , uh −vh − u), ≡ L(h , wh − u);
vh ∈ Vh
The first identity is trivial, as it reiterates the definition of the error. The second equality is crucial and is due to the generalized orthogonality (3.63). The last identity is just a variable change, wh = uh − vh . Using now the ellipticity and continuity of the bilinear form, we get c1 h 22 = c1 (h , h ) ≤ L(h , h ) = L(h , wh − u) ≤ c2 h wh − u which, after dividing through by h , yields precisely the result of C´ea’s theorem: c2 h ≤ c1 wh − u C´ea’s theorem simplifies error analysis greatly: it is in general extremely difficult to evaluate the Galerkin error directly because the Galerkin solution emerges as a result of solving a (usually large) system of equations; it is much easier to deal with some good approximation wh of the exact solution (e.g. via an interpolation procedure). C´ea’s theorem relates the Galerkin solution error to the approximation error via the stability and continuity constants of the bilinear form.
3.6 Local Approximation in the Finite Element Method
89
From a practical point of view, C´ea’s theorem is the source of robustness of the Galerkin method. In fact, the Galerkin method proves to be surprisingly reliable even for non-elliptic forms: although C´ea’s theorem is silent about that case, a more general result known as the Ladyzhenskaya–Babuˇska–Brezzi (or just LBB) condition14 is available (O.A. Ladyzhenskaya [Lad69], I. Babuˇska, [Bab58], F. Brezzi [Bre74]; see also B. Szab´o & I. Babuˇska [SB91], I. Babuˇska & T. Strouboulis [BS01] and Appendix 3.10).
3.6 Local Approximation in the Finite Element Method Remember the shortcomings of collocation – the first variational technique to be introduced in this chapter? The Galerkin method happily resolves (at least for elliptic problems) two of the three issues listed on p. 72. Indeed, the way to choose the test functions is straightforward (they are the same as the approximating functions), and C´ea’s theorem provides an error bound for the Galerkin solution. The only missing ingredient is a procedure for choosing “good” approximating functions. The Finite Element Method does provide such a procedure, and the following sections explain how it works in one, two and three dimensions. The guiding principle is local approximation of the solution. This usually makes perfect physical sense. It is true that in a limited number of cases a global approximation over the whole computational domain is effective – these cases usually involve homogeneous media with a smooth distribution of sources or no sources at all, with the field approximated by a Fourier series or a polynomial expansion. However, in practical problems, local geometric and physical features of systems and devices, with the corresponding local behavior of fields and potentials, is typical. Discontinuities at material interfaces, peaks, boundary layers, complex behavior at edges and corners, and many other features make it all but impossible to approximate the solution globally.15 Local approximation in FEM is closely associated with a mesh: the computational domain is subdivided into small subdomains – elements. A large assortment of geometric shapes of elements can be used: triangular or quadrilateral are most common in 2D, tetrahedral and hexahedral – in 3D. Note that the term “element” is overloaded: depending on the context, it may mean just the geometric figure or, in addition to that, the relevant approximating space and degrees of freedom (more about that later). For example, linear and 14 15
Occasionally used with some permutations of the names. Analytical approximations over homogeneous subdomains, with proper matching conditions at the interfaces of these subdomains, can be a viable alternative but is less general than FEM. One example is the Multiple Multipole Method popular in some areas of high frequency electromagnetic analysis and optics; see e.g. T. Wriedt (ed.), [Wri99].
90
3 The Finite Element Method
quadratic approximations over a triangle give rise to different finite elements in the sense of FEM, even though the geometric figure is the same. For illustration, Fig. 3.5 – Fig. 3.7 present FE meshes for a few particles of arbitrary shapes – the first two of these figures in 2D, and the third one in 3D. The mesh in the second figure (Fig. 3.6) was obtained by global refinement of the mesh in the first figure: each triangular element was subdivided into four. Mesh refinement can be expected to produce a more accurate numerical solution, albeit at a higher computational cost. Global refinement is not the most effective procedure: a smarter way is to make an effort to identify the areas where the numerical solution is least accurate and refine the mesh there. This idea leads to local adaptive mesh refinement (Section 3.13).
Fig. 3.5. An illustrative example of a finite element mesh in 2D.
Each approximating function in FEM is nonzero only over a small number of adjacent elements and is thus responsible for local approximation without affecting the approximation elsewhere. The following sections explain how this is done.
3.7 The Finite Element Method in One Dimension
91
Fig. 3.6. Global refinement of the mesh of Fig. 3.5, with each triangular element subdivided into four by connecting the midpoints of the edges.
3.7 The Finite Element Method in One Dimension 3.7.1 First-Order Elements In one dimension, the computational domain is a segment [a, b], the mesh is a set of nodes x0 = a, x1 , . . ., xn = b, and the elements (in the narrow geometric sense) are the segments [xi−1 , xi ], i = 1,2, . . ., n. The simplest approximating function is shown in Fig. 3.8 and is commonly called a “hat function” or, much less frequently, a “tent function”.16 The hat functions form a convenient basis of the simplest finite element vector space, as discussed in more detail below. For notational convenience only, we shall often assume that the grid is uniform, i.e. the grid size h = xi − xi−1 is the same for all nodes i. For nonuniform grids, there are no conceptual changes and only trivial differences in the algebraic expressions. A formal expression for ψi on a uniform grid is ⎧ −1 ⎨ h (x − xi−1 ), xi−1 ≤ x ≤ xi h−1 (xi+1 − x), xi ≤ x ≤ xi+1 (3.64) ψi (x) = ⎩ 0 otherwise 16
About 50 times less, according to Google. “Hut function” also makes some intuitive sense but is used very infrequently.
92
3 The Finite Element Method
Fig. 3.7. An example of a finite element mesh in 3D.
Fig. 3.8. The “hat” function for first order 1D elements.
3.7 The Finite Element Method in One Dimension
93
The hat function ψi straddles two adjacent elements (segments) and satisfies the obvious Kronecker-delta property on the grid: it is equal to one at xi and zero at all other nodes. This property is not critical in theoretical analysis but is very helpful in practice. In particular, for any smooth function u(x), piecewise-linear interpolation on the grid can be written simply as the linear combination n u(xi ) ψi uinterp (x) = i=1
Indeed, the fact that the nodal values of u and uinterp are the same follows directly from the Kronecker-delta property of the ψs. We now have all the prerequisites for solving an example problem. Example 4. −
d2 u = sin x, dx2
Ω = [0, π],
u(0) = u(π) = 0
(3.65)
The obvious theoretical solution u(x) = sin x is available for evaluating the accuracy of the finite element result. Let us use a uniform grid x0 = 0, x1 = h, . . ., xn = π with the grid size h = π/n. In numerical experiments, the number of nodes will vary, and we can expect higher accuracy (at higher computational cost) for larger values of n. The weak formulation of the problem is π π du dv dx = sin x v(x) dx, u, ∀v ∈ H01 ([0, π]) (3.66) 0 dx dx 0 The FE-Galerkin formulation is simply a restriction of the weak problem to the subspace P0h ([0, π]) of piecewise-linear functions satisfying zero Dirichlet conditions; this is precisely the subspace spanned by the hat functions ψ1 , . . . , ψn−1 :17 π π duh dvh dx = vh (x) sin x dx, uh , ∀vh ∈ P0h ([0, π]) (3.67) dx dx 0 0 As we know, this can be cast in matrix-vector form by substituting
formulation n−1 the expansion i=1 uhi ψi for uh and by setting vh , sequentially, to ψ1 , . . ., ψn−1 to obtain (n − 1) equations for (n − 1) unknown nodal values uhi : Lu = f ,
u, f ∈ Rn−1
(3.68)
where, as we also know, the entries of matrix L and the right hand side f are 17
Functions ψ0 and ψn are not included, as they do not satisfy the Dirichlet conditions. Implementation of boundary conditions will be discussed in more detail later.
94
3 The Finite Element Method
π
Lij = 0
dψi dψj dx; dx dx
fi =
π
ψi (x) sin x dx
(3.69)
0
As already noted, the discrete problem, being just a restriction of the continuous one to the finite-dimensional FE space, inherits the algebraic properties of the continuous formulation. This implies that the global stiffness matrix L is positive definite in this example (and in all cases where the biliniear form of the problem is elliptic). Equally important is the sparsity of the stiffness matrix: most of its entries are zero. Indeed, the Galerkin integrals for Lij in (3.69) are nonzero only if ψi and ψj are simultaneously nonzero over a certain finite element. This implies that either i = j or nodes i and j are immediate neighbors. In 1D, the global matrix is therefore tridiagonal. In 2D and 3D, the sparsity pattern of the FE matrix depends on the topology of the mesh and on the node numbering (see Sections 3.8 and 3.8). Algorithmically, it is convenient to compute these integrals on an elementby-element basis, gradually accumulating the contributions to the integrals as the loop over all elements progresses. Clearly, for each element the nonzero contributions will come only from functions ψi and ψj that are both nonzero over this element. For element #i – that is, for segment [xi−1 , xi ] – there are four such nonzero contributions altogether: xi xi 1 dψi−1 dψi−1 1 1 i dx = dx = = Lelem i−1,i−1 dx dx h h h xi−1 xi−1 xi xi 1 dψi−1 dψi 1 −1 i dx = dx = − Lelem i−1,i = dx dx h xi−1 xi−1 h h i elem i Lelem i,i−1 = Li−1,i
Lelem i,i f i−1 =
=
1 h
xi
ψi−1 (x) sin x dx =
fi =
i
xi−1 xi
xi−1
ψi (x) sin x dx = −
by symmetry
i (same as Lelem i−1,i−1 )
sin xi − xi cos xi + xi−1 cos xi − sin xi−1 h sin xi − xi cos xi−1 + xi cos xi − sin xi−1 h
These results can be conveniently arranged into a 2 × 2 matrix 1 1 −1 elem i L = 1 h −1
(3.70)
called, for historical reasons, the element stiffness matrix, and the element contribution to the right hand side is a vector 1 sin xi − xi cos xi + xi−1 cos xi − sin xi−1 f elem i = (3.71) h − sin xi + xi cos xi−1 − xi−1 cos xi−1 + sin xi−1
3.7 The Finite Element Method in One Dimension
95
Remark 2. A word of caution: in the engineering literature, it is not uncommon to introduce “element equations” of the form Lelem i uelem
i
= f elem
i
(!!??)
Such equations are devoid of mathematical meaning. The actual Galerkin equation involves a test function that spans a group of adjacent elements (two in 1D), and so there is no valid equation for a single element. Incidentally, triangular meshes have approximately two times more elements than nodes; so, if “element equations” were taken seriously, there would be about twice as many equations as unknowns! A sample Matlab code at the end of this subsection (p. 100) gives a “nofrills” implementation of the FE algorithm for the 1D model problem. To keep the code as simple as possible, much of the formulation is hard-coded, including the specific interval Ω, expressions for the right hand side and (for verification and error analysis) the exact solution. The only free parameter is the number of elements n. In actual computational practice, such hard-coding should of course be avoided. Commercial FE codes strive to provide maximum flexibility in setting up geometrical and physical parameters of the problem, with convenient user interface. Some numerical results are shown in the following figures. Fig. 3.9 provides a visual comparison of the FE solutions for 6 and 12 finite elements with the exact solution. Not surprisingly, the solution with 12 elements is more accurate. Fig. 3.10 displays several precise measures of the error: •
The relative nodal error defined as nodal =
u − N u∗ N u∗
where u ∈ Rn−1 is the Euclidean vector of nodal values of the FE solution, u∗ (x) is the exact solution, and N u∗ denotes the vector of nodal values of u∗ on the grid. • The L2 norm of the error L2 = uh − u∗ This error measures the discrepancy between the numerical and exact solutions as functions over [0, π] rather than Euclidean vectors of nodal values. • The L2 norm of the derivative d(uh − u∗ ) H1 = dx Due to the zero Dirichlet boundary conditions, this norm differs by no more than a constant factor from the H1 -norm; hence the notation.
96
3 The Finite Element Method
Fig. 3.9. FE solutions with 6 elements (circles) and 12 elements (squares) vs. the exact solution sin x (solid line).
Due to the simplicity of this example and of the exact solution, these measures can be computed up to the roundoff error. For more realistic problems, particularly in 2D and 3D, the errors can only be estimated. In Fig. 3.10 the three error measures are plotted vs. the number of elements. The linearity of the plots on the log-log scale implies that the errors are proportional to hγ , and the slopes of the lines correspond to γ = 2 for the nodal and L2 errors and γ = 1 for the H1 error. The derivative of the solution is computed less accurately than the solution itself. This certainly makes intuitive sense and also agrees with theoretical results quoted in Section 3.10. Example 5. How will the numerical procedure change if the boundary conditions are different? First consider inhomogeneous Dirichlet conditions. Let us assume that in the previous example the boundary values are u(0) = 1, u(π) = −1, so that the exact solution is now u∗ (x) = cos x. In the hat-function expansion of the (piecewise-linear) FE solution uh (x) =
n
uhi ψi (x)
i=0
the summation now includes boundary nodes in addition to the interior ones. However, the coefficients uh0 and uhn at these nodes are the known Dirichlet values, and hence no Galerkin equations with test functions ψ0 and ψn are necessary. In the Galerkin equation corresponding to the test function ψ1 ,
3.7 The Finite Element Method in One Dimension
97
Fig. 3.10. Several measures of error vs. the number of elements for the 1D model problem: relative nodal error (circles), L2 -error (squares), H1 -error (triangles). Note the log–log scale.
(ψ0 , ψ1 ) uh0 + (ψ1 , ψ1 ) uh1 = (f, ψ1 ) the first term is known and gets moved to the right hand side: (ψ1 , ψ1 ) uh1 = (f, ψ1 ) − (ψ0 , ψ1 ) uh0
(3.72)
As usual, parentheses in these expressions are L2 inner products and imply integration over the computational domain. The necessary algorithmic adjustments should now be clear. There is no change in the computation of element matrices. However, whenever an entry of the element matrix corresponding to a Dirichlet node is encountered,18 this entry is not added to the global system matrix. Instead, the right hand side is adjusted as prescribed by (3.72). A similar adjustment is made for the other boundary node (xn = π) as well. In 2D and 3D problems, there may be many Dirichlet nodes, and all of them are handled in a similar manner. The appropriate changes in the Matlab code are left as an exercise for the interested reader. The FE solution for a small number of elements is compared with the 18
Clearly, this may happen only for elements adjacent to the boundary.
98
3 The Finite Element Method
exact solution (cos x) in Fig. 3.11, and the error measures are shown as a function of the number of elements in Fig. 3.12.
Fig. 3.11. FE solution with 8 elements (markers) vs. the exact solution cos x (solid line).
Neumann conditions in the Galerkin formulation are natural19 and therefore do not require any algorithmic treatment: elements adjacent to the Neumann boundary are treated exactly the same as interior elements. Despite its simplicity, the one-dimensional example above contains the key ingredients of general FE algorithms: 1. Mesh generation and the choice of FE approximating functions. In the 1D example, “mesh generation” is trivial, but it becomes complicated in 2D and even more so in 3D. Only piecewise-linear approximating functions have been used here so far; higher-order functions are considered in the subsequent sections. 2. Local and global node numbering. For the computation of element matrices (see below), it is convenient to use local numbering (e.g. nodes 1, 2 for a segment in 1D, nodes 1, 2, 3 for a triangular element in 2D, etc.) At 19
We showed that Neumann conditions are natural – i.e. automatically satisfied – by the solution of the continuous weak problem. The FE solution does not, as a rule, satisfy the Neumann conditions exactly but should do so in the limit of h → 0, although this requires a separate proof.
3.7 The Finite Element Method in One Dimension
99
Fig. 3.12. The relative nodal error (circles) and the H1 -error (triangles) for the model Dirichlet problem. Note the log–log scale.
the same time, some global numbering of all mesh nodes from 1 to n is also needed. This global numbering is produced by a mesh generator that also puts local node numbers for each element in correspondence with their global numbers. In the 1D example, mesh generation is trivial, and so is the local-to-global association of node numbers: for element (segment) #i, (i = 1,2, . . . , n), local node 1 (the left node) corresponds to global node i − 1, and local node 2 corresponds to global node i. The 2D and 3D cases are considered in Section 3.8 and Section 3.9. 3. Computation of element matrices and of element-wise contributions to the right hand side. In the 1D example, these quantities were computed analytically; in more complicated cases, when analytical expressions are unavailable (this is frequently the case for curved or high order elements in 2D and 3D), Gaussian quadratures are used. 4. Assembly of the global matrix and of the right hand side. In a loop over all elements, the element contributions are added to the global matrix and to the right hand side; in the FE langauge, the matrix and the right hand side are “assembled” from element-wise contributions. The entries of each element matrix are added to the respective entries of the global matrix and right hand side. See Section 3.8 for more details in the 2D case. 5. The treatment of boundary conditions. The Neumann conditions in 1D, 2D or 3D do not require any special treatment – in other words, the
100
3 The Finite Element Method
FE algorithm may simply “ignore” these conditions and the solution will, in the limit, satisfy them automatically. The Robin condition containing a combination of the potential and its normal derivative is also natural but results in an additional boundary integral that will not be considered here. Finally, the Dirichlet conditions have to be taken into account explicitly. The following algorithmic adjustment is made in the loop over all elements. If Lij is an entry of the element matrix and j is a Dirichlet node but i isn’t, then Lij is not added to the global stiffness matrix. Instead, the quantity Lij uj , where uj is the known Dirichlet value of the solution at node j, is subtracted from the right hand side entry f i , as prescribed by equation (3.72). If both i and j are Dirichlet nodes, Lij is set to zero. 6. Solution of the FE system of equations. System solvers are reviewed in Section 3.11. 7. Postprocessing of the results. This may involve differentiation of the solution (to compute fields from potentials), integration over surfaces (to find field fluxes, etc.), and various contour, line or surface plots. Modern commercial FE packages have elaborate postprocessing capabilities and sophisticated graphical user interface; this subject is largely beyond the scope of this book, but some illustrations can be found in Chapter 7. At the same time, there are several more advanced features of FE analysis that are not evident from the 1D example and will be considered (at a varying level of detail) in the subsequent sections of this chapter: • Curved elements – used in 2D and 3D for more accurate approximation of curved boundaries. • Adaptive mesh refinement (Section 3.13). The mesh is refined locally, in the subregions where the numerical error is estimated to be highest. (In addition, the mesh may be un-refined in subregions with lower errors.) The problem is then solved again on the new grid. The key to the success of this strategy is a sensible error indicator that is computed a posteriori, i.e. after the FE solution is found. • Vector finite elements (Section 3.12). The most straightforward way of dealing with vector fields in FE analysis is to approximate each Cartesian component separately by scalar functions. While this approach is adequate in some cases, it turns out not to be the most solid one in general. One deficiency is fairly obvious from the outset: some field components are discontinuous at material interfaces, which is not a natural condition for scalar finite elements and requires special constraints. This is, however, only one manifestation of a deeper mathematical structure: fundamentally, electromagnetic fields are better understood as differential forms (Section 3.12). A Sample Matlab Code for the 1D Model Problem function FEM_1D_example1 = FEM_1D_example1 (n) % Finite element solution of the Poisson equation
3.7 The Finite Element Method in One Dimension % -u’’ = sin x on [0, pi]; % Input: % n -- number of elements
101
u(0) = u(pi) = 0
domain_length = pi; % hard-coded for simplicity of this sample code h = domain_length / n; % mesh size (uniform mesh assumed) % Initialization: system_matrix = sparse(zeros(n-1, n-1)); rhs = sparse(zeros(n-1, 1)); % Loop over all elements (segments) for elem_number = 1 : n node1 = elem_number - 1; node2 = elem_number; % Coordinates of nodes: x1 = h*node1; x2 = x1 + h; % Element stiffness matrix: elem_matrix = 1/h * [1 -1; -1 1]; elem_rhs = 1/h * [sin(x2) - x2 * cos(x2) + x1 * cos(x2) - sin(x1); ... -(sin(x2) - x2 * cos(x1) + x1 * cos(x1) - sin(x1))]; % Add element contribution to the global matrix if node1 ~= 0 % contribution for nonzero Dirichlet condition only system_matrix(node1, node1) = system_matrix(node1, node1) ... + elem_matrix(1, 1); rhs(node1) = rhs(node1) + elem_rhs(1); end if (node1 ~= 0) & (node2 ~= n) % contribution for nonzero % Dirichlet condition only system_matrix(node1, node2) = system_matrix(node1, node2) ... + elem_matrix(1, 2); system_matrix(node2, node1) = system_matrix(node2, node1) ... + elem_matrix(2, 1); end if node2 ~= n % contribution for nonzero Dirichlet condition only system_matrix(node2, node2) = system_matrix(node2, node2) ... + elem_matrix(2, 2); rhs(node2) = rhs(node2) + elem_rhs(2); end end % end element cycle u_FEM = system_matrix \ rhs;
% refrain from using % matrix inversion inv()!
102
3 The Finite Element Method FEM_1D_example1.a = 0; FEM_1D_example1.b = pi; FEM_1D_example1.n = n; FEM_1D_example1.u_FEM = u_FEM; return;
3.7.2 Higher-Order Elements There are two distinct ways to improve the numerical accuracy in FEM. One is to reduce the size h of (some or all) the elements; this approach is known as (local or global) h-refinement. Remark 3. It is very common to refer to a single parameter h as the “mesh size,” even if finite elements in the mesh have different sizes (and possibly even different shapes). With this terminology, it is tacitly assumed that the ratio of maximum/minimum element sizes is bounded and not too large; then the difference between the minimum, maximum or some average size is relatively unimportant. However, several recursive steps of local mesh refinement may result in a large disparity of the element sizes; in such cases, reference to a single mesh size would be misleading. The other way to improve the accuracy is to increase the polynomial order p of approximation within (some or all) elements; this is (local or global) p-refinement. Let us start with second-order elements in one dimension. Consider a geometric element – in 1D, a segment of length h. We are about to introduce quadratic polynomials over this element; since these polynomials have three free parameters, it makes sense to deal with their values at three nodes and to place these nodes at x = 0, h/2, h relative to a local coordinate system. The canonical approximating functions satisfy the Kronecker-delta conditions at the nodes. The first function is thus equal to one at node #1 and zero at the other two nodes; this function is easily found to be h 2 (x − h) (3.73) ψ1 = 2 x − h 2 (The factors in the parentheses are due to the roots at h/2 and h; the scaling coefficient 2/h2 normalizes the function to ψ1 (0) = 1.) Similarly, the remaining two functions are 4 x(h − x) h2 2 h = 2 x x− h 2
ψ2 = ψ3
(3.74) (3.75)
3.7 The Finite Element Method in One Dimension
103
Fig. 3.13. Three quadratic basis functions over one 1D element. h = 0.1 as an example.
Fig. 3.13 displays all three quadratic approximating functions over a single 1D element. While the “bubble” ψ2 is nonzero within one element only, functions ψ1,3 actually span two adjacent elements, as shown in Fig. 3.14. The entries of the element stiffness matrix L and mass matrix M (that is, the Gram matrix of the ψs) are h ψi ψj dx Lij = 0
where the prime sign denotes the derivative, and h Mij = ψi ψj dx 0
These matrices can be computed by straightforward integration: ⎛ ⎞ 7 −8 1 1 ⎝ −8 16 − 8⎠ L = 3h 1 −8 7 ⎛ ⎞ 4 2 −1 h ⎝ 2 16 2 ⎠ M = 30 −1 2 4
(3.76)
(3.77)
104
3 The Finite Element Method
Fig. 3.14. Quadratic basis function over two adjacent 1D elements. h = 0.1 as an example.
Naturally, both matrices are symmetric. The matrix assembly procedure for second-order elements in 1D is conceptually the same as for first-order elements. There are some minor differences: • • •
For second-order elements, the number of nodes is about double the number of elements. Consequently, the correspondence between the local node numbers (1, 2, 3) in an element and their respective global numbers in the grid is a little less simple than for first-order elements. The element matrix is 3 × 3 for second order elements vs. 2 × 2 for first order ones; the global matrices are five- and three-diagonal, respectively.
Elements of order higher than two can be introduced in a similar manner. The element of order n is, in 1D, a segment of length h with n + 1 nodes x0 , x1 , . . ., xn = x0 + h. The approximating functions are polynomials of order n. As with first- and second-order elements, it is most convenient if polynomial #i has the Kronecker-delta property: equal to one at the node xi and zero at the remaining n nodes. This is the Lagrange interpolating polynomial Λi (x) =
(x − x0 )(x − x1 ) . . . (x − xi−1 )(x − xi+1 ) . . . (x − xn ) (xi − x0 )(xi − x1 ) . . . (xi − xi−1 )(xi − xi+1 ) . . . (xi − xn )
(3.78)
3.8 The Finite Element Method in Two Dimensions
105
Indeed, the roots of this polynomial are x0 , x1 , . . ., xi−1 , xi+1 , . . ., xn , which immediately leads to the expression in the numerator. The denominator is the normalization factor needed to make Λi (x) equal to one at x = xi . The focus of this chapter is on the main ideas of finite element analysis rather than on technical details. With regard to the computation of element matrices, assembly procedures and other implementation issues for high order elements, I defer to more comprehensive FE texts cited at the end of this chapter.
3.8 The Finite Element Method in Two Dimensions 3.8.1 First-Order Elements In two dimensions, most common element shapes are triangular (by far) and quadrilateral. Fig. 3.15 gives an example of a triangular mesh, with the global node numbers displayed. Element numbering is not shown to avoid congestion in the figure. This section deals with first-order triangular elements. The approximating functions are linear over each triangle and continuous in the whole domain. Each approximating function spans a cluster of elements (Fig. 3.16) and is zero outside that cluster. Expressions for element-wise basis functions can be derived in a straightforward way. Let the element nodes be numbered 1, 2, 320 in the counterclockwise direction21 and let the coordinates of node i (i = 1,2,3) be xi , yi . As in the 1D case, it is natural to look for the basis functions satisfying the Kronecker-delta condition. More specifically, the basis function ψ1 = a1 x + b1 y + c1 , where a1 , b1 and c1 are coefficients to be determined, is equal to one at node #1 and zero at the other two nodes: a1 x1 + b1 y1 + c1 = 1 a1 x2 + b1 y2 + c1 = 0 (3.79) a1 x3 + b1 y3 + c1 = 0 or equivalently in matrix-vector ⎛ x1 y1 Xd1 = e1 , X = ⎝x2 y2 x3 y 3
form ⎞ 1 1⎠ ; 1
d1
⎛ ⎞ a1 = ⎝ b1 ⎠ ; c1
e1
⎛ ⎞ 1 = ⎝0⎠ (3.80) 0
Similar relationships hold for the other two basis functions, ψ2 and ψ3 , the only difference being the right hand side of system (3.80). It immediately 20
21
These are local numbers that have their corresponding global numbers in the mesh; for example, in the shaded element of Fig. 3.15 (bottom) global nodes 179, 284 and 285 could be numbered as 1, 2, 3, respectively. The significance of this choice of direction will become clear later.
106
3 The Finite Element Method
Fig. 3.15. An example of a triangular mesh with node numbering (top) and a fragment of the same mesh (bottom).
3.8 The Finite Element Method in Two Dimensions
107
Fig. 3.16. A piecewise-linear basis function in 2D over a cluster of triangular elements. Circles indicate mesh nodes. The basis function is represented by the surface of the pyramid.
follows from (3.80) that the coefficients a, b, c for all three basis functions can be collected together in a compact way: ⎞ ⎛ a1 a2 a3 (3.81) XD = I, D = ⎝ b1 b2 b3 ⎠ c1 c2 c3 where I is the 3×3 identity matrix. Hence the coefficients of the basis functions can be expressed succinctly as D = X −1
(3.82)
From analytical geometry, the determinant of X is equal to 2S∆ , where S∆ is the area of the triangle. (That is where the counter-clockwise numbering of nodes becomes important; for clockwise numbering, the determinant would be equal to minus 2S∆ .) This leads to simple explicit expressions for the basis functions: ψ1 =
(y2 − y3 )x + (x3 − x2 )y + (x2 y3 − x3 y2 ) 2S∆
(3.83)
with the other two functions obtained by cyclic permutation of the indexes. Since the basis functions are linear, their gradients are just constants: ∇ψ1 =
y2 − y 3 x3 − x2 x ˆ + yˆ 2S∆ 2S∆
(3.84)
108
3 The Finite Element Method
with the formulas for ψ2,3 again obtained by cyclic permutation. These expressions are central in the FE-Galerkin formulation. It would be straightforward to verify from (3.83), (3.84) that ψ 1 + ψ2 + ψ3 = 1
(3.85)
∇ψ1 + ∇ψ2 + ∇ψ3 = 0
(3.86)
However, these results can be obtained without any algebraic manipulation. Indeed, due to the Kronecker delta property of the basis, any function u(x, y) linear over the triangle can be expressed via its nodal values u1,2,3 as u(x, y) = u1 ψ1 + u2 ψ2 + u3 ψ3 Equation (3.85) follows from this simply for u(x, y) ≡ 1. Functions ψ1,2,3 are also known as barycentric coordinates and have an interesting geometric interpretation (Fig. 3.17). For any point x, y in the plane, ψ1 (x, y) is the ratio of the shaded area to the area of the whole triangle: ψ1 (x, y) = S1 (x, y)/S∆ . Similar expressions are of course valid for the other two basis functions.
Fig. 3.17. Geometric interpretation of the linear basis functions: ψ1 (x, y) = S1 (x, y)/S∆ , where S1 is the shaded area and S∆ is the area of the whole triangle. (Similar for ψ2,3 .)
Indeed, the fact that S1 /S∆ is equal to one at node #1 and zero at the other two nodes is geometrically obvious. Moreover, it is a linear function of coordinates because S1 is proportional to height l of the shaded triangle (the “elevation” of point x, y over the “base” segment 2–3), and l can be obtained by a linear transformation of coordinates (x, y). The three barycentric coordinates are commonly denoted with λ1,2,3 , so the linear FE basis functions are just ψi ≡ λi (i = 1,2,3). Higher-order FE bases can also be conveniently expressed in terms of λ (Section 3.8.2).
3.8 The Finite Element Method in Two Dimensions
109
The element stiffness matrix for first order elements is easy to compute because the gradients (3.84) of the basis functions are constant: ∇λi · ∇λj dS = ∇λi · ∇λj S∆ , i, j = 1, 2, 3 (3.87) (∇λi , ∇λj ) ≡ ∆
where the integration is over a triangular element and S∆ is the area of this element. Expressions for the gradients are available (3.84) and can be easily substituted into (3.87) if an explicit formula for the stiffness matrix in terms of the nodal coordinates is desired. Computation of the element mass matrix (the Gram matrix of the basis functions) is less simple but the result is quite elegant. The integral of, say, the product λ1 λ2 over the triangular element can be found using an affine transformation of this element to the “master” triangle with nodes 1, 2, 3 at (1, 0), (0, 1) and (0, 0), respectively. Since the area of the master triangle is 1/2, the Jacobian of this transformation is equal to 2S∆ and we have22
(λ1 , λ2 ) ≡
1
λ1 λ2 dS = 2S∆
xdx 0
∆
1−x
ydy = 0
S∆ 12
Similarly, (λ1 , λ1 ) = 2
S∆ 12
and the complete element mass matrix is ⎛ ⎞ 2 1 1 S∆ M = ⎝1 2 1⎠ 12 1 1 2
(3.88)
The expressions for the inner products of the barycentric coordinates are a particular case of a more general formula that appears in many texts on FE analysis and is quoted here without proof: i! j! k! 2S∆ λi1 λj2 λk3 dS = (3.89) (i + j + k + 2)! ∆ for any nonnegative integers i, j, k. M11 of (3.88) corresponds to i = 2, j = k = 0; M12 corresponds to i = j = 1, k = 0; etc. Remark 4. The notion of “master element” (or “reference element”) is useful and long-established in finite element analysis. Properties of FE matrices and FE approximations are usually examined via affine transformations of elements to the “master” ones. In that sense, analysis of finite element interpolation errors in Section 3.14.2 below (p. 160) is less typical.
22
The Jacobian is positive for the counter-clockwise node numbering convention.
110
3 The Finite Element Method
Example 6. Let us find the basis functions and the FE matrices for a right triangle with node #1 at the origin, node #2 on the x-axis at (hx , 0), and node #3 on the y-axis at (0, hy ) (mesh sizes hx , hy are positive numbers). The coordinate matrix is ⎞ ⎛ 0 0 1 X = ⎝h x 0 1 ⎠ 0 hy 1 which yields the coefficient matrix ⎛
D = X −1
−h−1 x ⎝ −h−1 = y 1
h−1 x 0 0
⎞ 0 ⎠ h−1 y 0
Each column of this matrix is a set of three coefficients for the respective basis function; thus the three columns translate into −1 ψ1 = 1 − h−1 x x − hy y −1 ψ2 = hx x ψ3 = h−1 y y
The sum of these functions is identically equal to one as it should be according to (3.85). Functions ψ2 and ψ3 in this case are particularly easy to visualize: ψ2 is a linear function of x equal to one at node #2 and zero at the other two nodes; ψ3 is similar. The gradients are ∇ψ1 = − h−1 ˆ − h−1 ˆ x x y y −1 ∇ψ2 = hx x ˆ ˆ ∇ψ3 = h−1 y y Computing the entries of the element stiffness matrix is easy because the gradients of λs are (vector) constants. For example, −2 ∇λ1 · ∇λ1 dS = (h−2 (∇λ1 , ∇λ1 ) = x + h y ) S∆ ∆
Since S∆ = hx hy /2, the complete stiffness matrix is ⎛ −2 ⎞ hx + h−2 − h−2 − h−2 y x y hx hy h−2 0 ⎠ L = ⎝ −h−2 x x 2 0 h−2 −h−2 y y This expression becomes particularly simple if hx = hy = h: ⎛ ⎞ 2 −1 −1 1 ⎝ −1 1 0 ⎠ L = 2 −1 0 1
(3.90)
(3.91)
3.8 The Finite Element Method in Two Dimensions
The mass matrix is, according ⎛ 2 S∆ ⎝ 1 M = 12 1
to the general expression (3.88), ⎞ ⎛ ⎞ 1 1 2 1 1 h h x y ⎝ 2 1⎠ = 1 2 1⎠ 24 1 2 1 1 2
111
(3.92)
An example of Matlab implementation of FEM for a triangular mesh is given at the end of this section; see p. 114 for the description and listing of the code. As an illustrative example, consider a dielectric particle with some nontrivial shape – say, T-shaped – in a uniform external field. The geometric setup is clear from Figs. 3.18 and 3.19.
Fig. 3.18. A finite element mesh for the electrostatic problem: a T-shaped particle in an external field. The mesh has 422 nodes and 782 triangular elements.
The potential of the applied external field is assumed to be u = x and is imposed as the Dirichlet condition on the boundary of the computational domain. Since the particle disturbs the field, this condition is not exact but becomes more accurate if the domain boundary is moved farther away from the particle; this, however, increases the number of nodes and consequently the computational cost of the simulation. Domain truncation is an intrinsic difficulty of electromagnetic FE analysis (unlike, say, analysis of stresses and strains confined to a finite mechanical part). Various ways of reducing the domain truncation error are known: radiation boundary conditions and Perfectly Matched Layers (PML) for wave problems (e.g. Z.S. Sacks [SKLL93], JoYu Wu et al. [WKLL97]), hybrid finite element/boundary element methods,
112
3 The Finite Element Method
Fig. 3.19. The potential distribution for the electrostatic example: a T-shaped particle in an external field.
infinite elements, “ballooning,” spatial mappings (A. Plaks et al. [PTPT00]) and various other techniques (see Q. Chen & A. Konrad [CK97] for a review). Since domain truncation is only tangentially related to the material of this section, it is not considered here further but will reappear in Chapter 7. For inhomogeneous Dirichlet conditions, the weak formulation of the problem has to be modified, with the corresponding minor adjustments to the FE algorithm. The underlying mathematical reason for this modification is that functions satisfying a given inhomogeneous Dirichlet condition form an affine space rather than a linear space (e.g. the sum of two such functions has a different value at the boundary). The remedy is to split the original unknown function u up as (3.93) u = u0 + u=0 where u=0 is some sufficiently smooth function satisfying the given inhomogeneous boundary condition, while the remaining part u0 satisfies the homogeneous one. The weak formulation is L(u0 , v0 ) = (f, v0 ) − L(u=0 , v0 ),
u0 ∈ H01 (Ω), ∀v0 ∈ H01 (Ω)
(3.94)
In practice, the implementation of this procedure is more straightforward than it may appear from this expression. The inhomogeneous part u=0 is spanned by the FE basis functions corresponding to the Dirichlet nodes; the homogeneous part of the solution is spanned by the basis functions for all other nodes. If j is a Dirichlet boundary node, the solution value uj at this
3.8 The Finite Element Method in Two Dimensions
113
node is given, and hence the term Lij uj in the global system of FE equations is known as well. It is therefore moved (with the opposite sign of course) to the right hand side. In the T-shaped particle example, the mesh has 422 nodes and 782 triangular elements, and the stiffness matrix has 2446 nonzero entries. The sparsity structure of this matrix (also called the adjacency structure) – the set of index pairs (i, j) for which Lij = 0 – is exhibited in Fig. 3.20. The distribution of nonzero entries in the matrix is quasi-random, which has implications for the solution procedures if direct solvers are employed. Such solvers are almost invariably based on some form of Gaussian elimination; for symmetric positive definite matrices, it is Cholesky decomposition U T U , where U is an upper triangular matrix.23 While Gaussian elimination is a very reliable24 and relatively simple procedure, for sparse matrices it unfortunately produces “fill-in”: zero entries become nonzero in the process of elimination (or Cholesky decomposition), which substantially degrades the computational efficiency and memory usage. In the present example, Cholesky decomposition applied to the original stiffness matrix with 2446 nonzero entries25 produces the Cholesky factor with 24,969 nonzeros and hence requires about 20 times more memory (if symmetry is taken advantage of); compare Figs. 3.20 and 3.21. For more realistic practical cases, where matrix sizes are much greater, the effect of fill-in is even more dramatic. It is worth noting – in passing, since this is not the main theme of this section – that several techniques are available for reducing the amount of fill-in in Cholesky factorization. The main ideas behind these techniques are clever permutations of rows and columns (equivalent to renumbering of nodes in the FE mesh), block algorithms (including divide-and-conquer type recursion), and combinations thereof. A. George & J.W.H. Liu give a detailed and lucid exposition of this subject [GL81]. In the current example, the so-called reverse Cuthill–McKee ordering reduces the number of nonzero entries in the Cholesky factor to 7230, which is more than three times better than for the original numbering of nodes (Figs. 3.22 and 3.23). The “minimum degree” ordering [GL81] is better by another factor of ∼ 2: the number of nonzeros in the Cholesky triangular matrix is equal to 3717 (Figs. 3.24 and 3.25). These permutation algorithms will be revisited in the solver section (p. 129).
23
24
25
Cholesky decomposition is usually written in the equivalent form of LLT , where L is a lower triangular matrix, but symbol L in this chapter is already used for the FE stiffness matrix. It is known to be stable for symmetric positive definite matrices but may require pivoting in general. Of which only a little more than one half need to be stored due to matrix symmetry.
114
3 The Finite Element Method
Fig. 3.20. The sparsity (adjacency) structure of the global FE matrix in the Tshaped particle example.
Fig. 3.21. The sparsity structure of the Cholesky factor of the global FE matrix in the T-shaped particle example.
Appendix: Sample Matlab Code for FEM with First-Order Triangular Elements The Matlab code below is intended to be the simplest possible illustration of the finite element procedure. As such, it uses first order elements and is optimized for algorithmic simplicity rather than performance. For example, there is some duplication of variables for the sake of clarity, and symmetry of
3.8 The Finite Element Method in Two Dimensions
115
Fig. 3.22. The sparsity structure of the global FE matrix after the reverse Cuthill– McKee reordering of nodes.
Fig. 3.23. The sparsity structure of the upper-triangular Cholesky factor of the global FE matrix after the reverse Cuthill–McKee reordering of nodes.
116
3 The Finite Element Method
Fig. 3.24. The sparsity structure of the global FE matrix after the minimum degree reordering of nodes.
Fig. 3.25. The sparsity structure of the upper-triangular Cholesky factor of the global FE matrix after the minimum degree reordering of nodes.
3.8 The Finite Element Method in Two Dimensions
117
the FE stiffness matrix is not taken advantage of. Improvements become fairly straightforward to make once the essence of the algorithm is understood. The starting point for the code is a triangular mesh generated by FEMLABTM , a commercial finite element package26 integrated with Matlab. The input data structure fem generated by FEMLAB in general contains the geometric, physical and FE mesh data relevant to the simulation. For the purposes of this section, only mesh data (the field fem.mesh) is needed. Second-order elements are the default in FEMLAB, and it is assumed that this default has been changed to produce first-order elements for the sample Matlab code. The fem.mesh structure (or simply mesh for brevity) contains several fields: •
mesh.p is a 2 × n matrix, where n is the number of nodes in the mesh. The i-th column of this matrix contains the (x, y) coordinates of node #i. • mesh.e is a 7×nbe matrix, where nbe is the number of element edges on all boundaries: the exterior boundary of the domain and material interfaces. The first and second rows contain the node numbers of the starting and end points of the respective edge. The sixth and seventh row contain the region (subdomain) numbers on the two sides of the edge. Each region is a geometric entity that usually corresponds to a particular medium, e.g. a dielectric particle or air. Each region is assigned a unique number. By convention, the region outside the computational domain is labeled as zero, which is used in the Matlab code below to identify the exterior boundary edges and nodes in mesh.e. The remaining rows of this matrix will not be relevant to us here. • mesh.t is a 4 × nelems matrix, where nelems is the number of elements in the mesh. The first three rows contain node numbers of each element in counter-clockwise order. The fourth row is the region number identifying the medium where the element resides. The second input parameter of the Matlab code, in addition to the fem structure, is an array of dielectric permittivities by region number. In the T-shaped particle example, region #1 is air, and the particle includes regions #2–#4, all with the same dielectric permittivity. The following sequence of commands could be used to call the FE solver: % Set parameters: epsilon_air = 1; epsilon_particle = 10; epsilon_array = [epsilon_air epsilon_particle*ones(1, 5)]; % Solve the FE problem FEM_solve = FEM_triangles (fem, epsilon_array) The operation of the Matlab function FEM triangles below should be clear from the comments in the code and from Section 3.8.1. 26
www.comsol.com
118
3 The Finite Element Method
function FEM_triangles = FEM_triangles (fem, epsilon_array) % Input parameters: % fem -- structure generated by FEMLAB. % (See comments in the code and text.) % epsilon_array -- material parameters by region number. mesh = fem.mesh; % duplication for simplicity n_nodes = length(mesh.p); % array p has dimension 2 x n_nodes; % contains x- and y-coordinates of the nodes. n_elems = length(mesh.t); % array t has dimension 4 x n_elements. % First three rows contain node numbers % for each element. % The fourth row contains region number % for each element. % Initialization rhs = zeros(n_nodes, 1); global_stiffness_matrix = sparse(n_nodes, n_nodes); dirichlet = zeros(1, n_nodes); % flags Dirichlet conditions % for the nodes (=1 for Dirichlet % nodes, 0 otherwise) % Use FEMLAB data on boundary edges to determine Dirichlet nodes: boundary_edge_data = mesh.e;
% mesh.e contains FEMLAB data % on element edges at the domain boundary number_of_boundary_edges = size(boundary_edge_data, 2); for boundary_edge = 1 : number_of_boundary_edges % Rows 6 and 7 in the array are region numbers % on the two sides of the edge region1 = boundary_edge_data(6, boundary_edge); region2 = boundary_edge_data(7, boundary_edge); % If one of these region numbers is zero, the edge is at the % boundary, and the respective nodes are Dirichlet nodes: if (region1 == 0) | (region2 == 0) % boundary edge node1 = boundary_edge_data(1, boundary_edge); node2 = boundary_edge_data(2, boundary_edge); dirichlet(node1) = 1; dirichlet(node2) = 1; end end % Set arrays of nodal coordinates: for elem = 1 : n_elems % loop over all elements elem_nodes = mesh.t(1:3, elem); % node numbers for the element for node_loc = 1 : 3 node = elem_nodes(node_loc); x_nodes(node) = mesh.p(1, node);
3.8 The Finite Element Method in Two Dimensions
119
y_nodes(node) = mesh.p(2, node); end end % Matrix assembly -- loop over all elements: for elem = 1 : n_elems elem_nodes = mesh.t(1:3, elem); region_number = mesh.t(4, elem); for node_loc = 1 : 3 node = elem_nodes(node_loc); x_nodes_loc(node_loc) = x_nodes(node); y_nodes_loc(node_loc) = y_nodes(node); end % Get element matrices: [stiff_mat, mass_mat] = elem_matrices_2D(x_nodes_loc, y_nodes_loc); for node_loc1 = 1 : 3 node1 = elem_nodes(node_loc1); if dirichlet(node1) ~= 0 continue; end for node_loc2 = 1 : 3 % symmetry not taken advantage of, to simplify code node2 = elem_nodes(node_loc2); if dirichlet(node2) == 0 % non-Dirichlet node global_stiffness_matrix(node1, node2) = ... global_stiffness_matrix(node1, node2) ... + epsilon_array(region_number) ... * stiff_mat(node_loc1, node_loc2); else % Dirichlet node; update rhs rhs(node1) = rhs(node1) - ... stiff_mat(node_loc1, node_loc2) * ... dirichlet_value(x_nodes(node2), y_nodes(node2)); end end end end % Equations for Dirichlet nodes are trivial: for node = 1 : n_nodes if dirichlet(node) ~= 0 % a Dirichlet node global_stiffness_matrix(node, node) = 1; rhs(node) = dirichlet_value(x_nodes(node), y_nodes(node)); end end solution = global_stiffness_matrix \ rhs; % Output fields: FEM_triangles.fem = fem;
% record the fem structure
120
3 The Finite Element Method
FEM_triangles.epsilon_array = epsilon_array;
% material parameters % by region number FEM_triangles.n_nodes = n_nodes; % number of nodes in the mesh FEM_triangles.x_nodes = x_nodes; % array of x-coordinates of the nodes FEM_triangles.y_nodes = y_nodes; % array of y-coordinates of the nodes FEM_triangles.dirichlet = dirichlet; % flags for the Dirichlet nodes FEM_triangles.global_stiffness_matrix = global_stiffness_matrix; % save matrix for testing FEM_triangles.rhs = rhs; % right hand side for testing FEM_triangles.solution = solution; % nodal values of the potential return; %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% function [stiff_mat, mass_mat] = elem_matrices_2D(x_nodes, y_nodes) % Compute element matrices for a triangle. % Input parameters: % x_nodes -- x-coordinates of the three nodes, % in counter-clockwise order % y_nodes -- the corresponding y-coordinates coord_mat = [x_nodes’ y_nodes’ ones(3, 1)]; % matrix of nodal coordinates, with an extra column of ones coeffs = inv(coord_mat); % coefficients of the linear basis functions grads = coeffs(1:2, :); % gradients of the linear basis functions area = 1/2 * abs(det(coord_mat)); % area of the element stiff_mat = area * grads’ * grads; % the FE stiffness matrix mass_mat = area / 12 * (eye(3) + ones(3, 3)); % the FE mass matrix return; %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% function dirichlet_value = dirichlet_value (x, y) % Set the Dirichlet boundary condition dirichlet_value = x;
% as a simple example
return;
3.8.2 Higher-Order Triangular Elements The discussion in Section 3.8.1 suggests that in a triangular element the barycentric variables λ (p. 108) form a natural set of coordinates (albeit not
3.8 The Finite Element Method in Two Dimensions
121
independent, as their sum is equal to unity). For first order elements, the barycentric coordinates themselves double as the basis functions. They can also be used to generate FE bases for higher order triangular elements. A second order element has three corner nodes #1–#3 and three midpoint nodes (Fig. 3.26). All six nodes can be labeled with triplets of indexes (k1 , k2 , k3 ); each index ki increases from 0 to 1 to 2 along the edges toward node i (i = 1, 2, 3).
Fig. 3.26. Second order triangular element. The six nodes can be labeled with triplets of indexes (k1 , k2 , k3 ), ki = 0, 1, 2. Each node has the corresponding basis function Λkk11 (λ1 )Λkk22 (λ2 )Λkk33 (λ3 ).
To each node, there corresponds an FE basis function that is a second order polynomial in λ with the Kronecker-delta property. The explicit expression for this polynomial is Λkk11 (λ1 )Λkk22 (λ2 )Λkk33 (λ3 ). For example, the basis function corresponding to node (0, 1, 1) – the midpoint node at the bottom – is Λ1 (λ2 )Λ1 (λ3 ). Indeed, it is the Lagrange polynomial Λ1 that is equal to one at the midpoint and to zero at the corner nodes of a given edge, and it is the barycentric coordinates λ2,3 that vary (linearly) along the bottom edge. This construction can be generalized to elements of order p. Each side of the triangle is subdivided into p segments; the nodes of the resulting triangular grid are again labeled with triplets of indexes, and the corresponding basis functions are defined in the same way as above. Details can be found in the FE monographs cited at the end of the chapter.
122
3 The Finite Element Method
3.9 The Finite Element Method in Three Dimensions Tetrahedral elements, by analogy with triangular ones in 2D, afford the greatest flexibility in representing geometric shapes and are therefore the most common type in many applications. Hexahedral elements are also frequently used. This section describes the main features of tetrahedral elements; further information about elements of other types can be found in specialized FE books (Section 3.16). Due to a direct analogy between tetrahedral and triangular elements (Section 3.8), results for tetrahedra are presented below without further ado. Let the coordinates of the four nodes be xi , yi , zi (i = 1,2,3,4). A typical linear basis function – say, ψ1 – is ψ1 = a1 x + b1 y + c1 z + d1 with some coefficients a1 , b1 , c1 , d1 . The Kronecker-delta property is desired: a1 x1 + b1 y1 + c1 z1 + d1 a1 x2 + b1 y2 + c1 z2 + d2 a1 x3 + b1 y3 + c1 z3 + d3 a1 x4 + b1 y4 + c1 z4 + d4 Equivalently in matrix-vector form ⎛ x 1 y 1 z1 ⎜ x2 y 2 z2 Xf1 = e1 , X = ⎜ ⎝ x3 y 3 z3 x4 y 4 z4
= = = =
1 0 0 0
(3.95)
⎛ ⎞ ⎛ ⎞ ⎞ 1 a1 1 ⎜ b1 ⎟ ⎜0⎟ 1⎟ ⎟ ; f1 = ⎜ ⎟ ; e1 = ⎜ ⎟ (3.96) ⎝ c1 ⎠ ⎝0⎠ 1⎠ 1 d1 0
with similar relationships for the other three basis functions. In compact notation, ⎞ ⎛ a1 a2 a3 a4 ⎜ b1 b2 b3 b4 ⎟ ⎟ (3.97) XF = I, F = ⎜ ⎝ c1 c2 c3 c4 ⎠ d1 d2 d3 d4 where I is the 4 × 4 identity matrix. The coefficients of the basis functions thus are (3.98) F = X −1 The determinant of X is equal to 6V , where V is the volume of the tetrahedron (assuming that the nodes are numbered in a way that produces a positive determinant). The basis functions can be found from (3.98), say, by Cramer’s rule. Since the basis functions are linear, their gradients are constants. The sum of the basis functions is unity, for the same reason as for triangular elements: (3.99) ψ1 + ψ2 + ψ3 + ψ4 = 1
3.10 Approximation Accuracy in FEM
123
The sum of the gradients is zero: ∇ψ1 + ∇ψ2 + ∇ψ3 + ∇ψ4 = 0
(3.100)
Functions ψ1,2,3,4 are identical with the barycentric coordinates λ1,2,3,4 of the tetrahedron. They have a geometric interpretation as ratios of tetrahedral volumes – an obvious analog of the similar property for triangles (Fig. 3.17 on p. 108). The element stiffness matrix for first order elements is (noting that the gradients are constant) ∇λi · ∇λj dV = ∇λi · ∇λj V, i, j = 1, 2, 3, 4 (3.101) (∇λi , ∇λj ) ≡ ∆
where the integration is over the tetrahedron and V is its volume. The element mass matrix (the Gram matrix of the basis functions) turns out to be ⎛ ⎞ 2 1 1 1 ⎜1 2 1 1⎟ V ⎟ (3.102) M = ⎜ ⎝1 1 2 1⎠ 20 1 1 1 2 which follows from the formula λi1 λj2 λk3 λl4 dV = ∆
i! j! k! l! 6V (i + j + k + l + 3)!
(3.103)
for any nonnegative integers i, j, k, l. Higher-order tetrahedral elements are constructed in direct analogy with the triangular ones (Section 3.8.2). The second-order tetrahedron has ten nodes (four main vertices and six edge midpoints); the cubic tetrahedral element has 20 nodes (two additional nodes per edge subdividing it into three equal segments, and four nodes at the barycenters of the faces). Detailed descriptions of tetrahedral elements, as well as first- and high-order elements of other shapes (hexahedra, triangular prisms, and others) are easy to find in FE monographs (Section 3.16).
3.10 Approximation Accuracy in FEM Theoretical considerations summarized in Section 3.5 show that the accuracy of the finite element solution is directly linked, and primarily depends on, the approximation accuracy. In particular, for symmetric elliptic forms L, the Galerkin solution is actually the best approximation of the exact solution in the sense of the L-norm (usually interpreted as an energy norm). In the case of a continuous elliptic, but not necessarily symmetric, form, the solution error depends also on the ellipticity and continuity constants, according to C´ea’s
124
3 The Finite Element Method
theorem; however, the approximation error is still key. The same is true in the general case of continuous but not necessarily symmetric or elliptic forms; then the so-called Ladyzhenskaya–Babuˇska–Brezzi (LBB) condition relates the solution error to the approximation error via the inf-sup constant (Section 3.10, p. 126). In all cases, the central role of FE approximation is clear. The main theoretical results on approximation accuracy in FEM are summarized below. But first, let us consider a simple intuitive 1D picture. The exact solution (solid line in Fig. 3.27) is approximated on a FE grid of size h; several finite elements (e) are shown in the figure. The most natural and easy to analyze form of approximation is interpolation, with the exact and approximating functions sharing the same nodal values on the grid.
Fig. 3.27. Piecewise-linear FE interpolation of the exact solution.
The FE solution of a boundary value problem in general will not interpolate the exact one, although there is a peculiar case where it does (see the Appendix on p. 127). However, due to C´ea’s theorem (or Galerkin error minimization or the LBB condition, whichever may be applicable), the smallness of the interpolation error guarantees the smallness of the solution error. It is intuitively clear from Fig. 3.27 that the interpolation error decreases as the mesh size becomes smaller. The error will also decrease if higher-order interpolation – say, piecewise-quadratic – is used. (Higher-order nodal elements have additional nodes that are not shown in the figure.) If the derivative of the exact solution is only piecewise-smooth, the approximation will not suffer as long as the points of discontinuity – typically, material interfaces – coincide with some of the grid nodes. The accuracy will degrade significantly if a material interface boundary passes through a finite element. For this reason, FE meshes in any number of dimensions are generated in such a way that each element lies entirely within one medium. For curved material boundaries, this
3.10 Approximation Accuracy in FEM
125
is strictly speaking possible only if the elements themselves are curved; nevertheless, approximation of curved boundaries by piecewise-planar element FE surfaces is often adequate in practice. P.G. Ciarlet & P.A. Raviart gave the following general and powerful mathematical characterization of interpolation accuracy [CR72]. Let Σ be a finite set in Rn and let polynomial Iu interpolate a given function u, in the Lagrange or Hermite sense, over a given set of points in Σ. Notably, the only significant assumption in the Ciarlet–Raviart theory is uniqueness of such a polynomial. Then sup{Dm u(x) − Dm Iu(x) ; x ∈ K} ≤ CMp+1
hp+1 , ρm
0 ≤ m ≤ p (3.104)
Here K is the closed convex hull of Σ; h – diameter of K; p – maximum order of the interpolating polynomial; Mp+1 = sup{Dp+1 u(x); x ∈ K}; ρ – supremum of the diameters of spheres inscribed in K. C – a constant. While the result is applicable to abstract sets, in the FE context K is a finite element (as a geometric figure). Let us examine the factors that the error depends upon. Mp+1 , being the magnitude of the (p+1)st derivative of u, characterizes the level of smoothness of u; naturally, the polynomial approximation is better for smoother functions. The geometric factor can be split up into the shape and size components: m h hp+1 = hp+1−m m ρ ρ h/ρ is dimensionless and depends only on the shape of K; we shall return to the dependence of FE errors on element shape in Section 3.14. The following observations about the second factor, hp+1−m , can be made: •
Example: the maximum interpolation error by linear polynomials is O(h2 ) (p = 1, m = 0). The error in the first derivative is asymptotically higher, O(h) (p = 1, m = 1). • The interpolation error behaves as a power function of element size h but depends exponentially on the interpolation order p, provided that the exact solution has at least p + 1 derivatives. • The interpolation accuracy is lower for higher-order derivatives (parameter m). Most of these observations make clear intuitive sense. A related result is cited in Section 4.4.4 on p. 209.
126
3 The Finite Element Method
Appendix: The Ladyzhenskaya–Babuˇ ska–Brezzi Condition For elliptic forms, the Lax–Milgram theorem guarantees well-posedness of the weak problem and C´ea’s theorem relates the error of the Galerkin solution to the approximation error (Section 3.5 on p. 86). For non-elliptic forms, the Ladyzhenskaya–Babuˇska–Brezzi (LBB) condition plays a role similar to the Lax–Milgram–C´ea results, although analysis is substantially more involved. Conditions for the well-posedness of the weak problem were derived independently by O.A. Ladyzhenskaya, I. Babuˇska & F. Brezzi [Lad69, BA72, Bre74]. In addition, the Babuˇska and Brezzi theories provide error estimates for the numerical solution. Unfortunately, the LBB condition is in many practical cases not easy to verify. As a result, less rigorous criteria are common in engineering practice; for example, the “patch test” that is not considered in this book but is easy to find in the FE literature (e.g. O.C. Zienkiewicz et al. [ZTZ05]). Non-rigorous conditions should be used with caution; I. Babuˇska & R. Narasimhan [BN97] give an example of a finite element formulation that satisfies the patch test but not the LBB condition. They also show, however, that convergence can still be established in that case, provided that the input data (and hence the solution) are sufficiently smooth. A mathematical summary of the LBB condition is given below for reference. It is taken from the paper by J. Xu & L. Zikatanov [XZ03]. Let U and V be two Hilbert spaces, with inner products (·, ·)U and (·, ·)V , respectively. Let B(·, ·): U × V → R be a continuous bilinear form (3.105) B(u, v) ≤ B uU vV Consider the following variational problem: Find u ∈ U such that B(u, v) = f, v,
∀v ∈ V
(3.106)
where f ∈ V ∗ (the space of continuous linear functionals on V and ·, · is the usual pairing between V ∗ and V . . . . problem (3.106) is well posed if and only if the following conditions hold . . .: B(u, v) > 0 (3.107) inf sup u∈U v∈V uU vV Furthermore, if (3.107) hold, then inf sup
u∈U v∈V
B(u, v) B(u, v) = inf sup ≡ α > 0 v∈V u∈U uU vV uU vV
(3.108)
and the unique solution of (3.106) satisfies uU ≤
f V ∗ α
(3.109)
3.10 Approximation Accuracy in FEM
127
. . . Let Uh ⊂ U and Vh ⊂ V be two nontrivial subspaces of U and V , respectively. We consider the following variational problem: Find uh ∈ Uh such that B(uh , vh ) = f, vh ,
∀vh ∈ Vh
(3.110)
. . . problem (3.110) is uniquely solvable if and only if the following conditions hold: inf
sup
uh ∈Uh vh ∈Vh
B(uh , vh ) = uh Uh vVh
inf
sup
vh ∈Vh uh ∈Uh
B(uh , vh ) ≡ αh > 0 uh Uh vVh (3.111)
(End of quote from J. Xu & L. Zikatanov [XZ03].) The LBB result, slightly strengthened by Xu & Zikatanov, for the Galerkin approximation is Theorem 4. Let (3.105), (3.107) and (3.111) hold. Then u − uh U ≤
B αh
inf u − wh U
wh ∈Vh
(3.112)
Appendix: A Peculiar Case of Finite Element Approximation The curious special case considered in this Appendix is well known to the expert mathematicians but much less so to applied scientists and engineers. I am grateful to B.A. Shoykhet for drawing my attention to this case many years ago and to D.N. Arnold for insightful comments and for providing a precise reference, the 1974 paper by J. Douglas & T. Dupont [DD74], p. 101. Consider the 1D Poisson equation −
d2 u = f (x), dx2
Ω = [a, b]; u(a) = u(b) = 0
(3.113)
where the zero Dirichlet conditions are imposed for simplicity only. Let us examine the finite element solution uh of this equation using first-order elements. The Galerkin problem for uh on a chosen mesh is (uh , vh ) = (f, vh ),
uh , ∀vh ∈ P0h
(3.114)
where the primes denote derivatives and P0h is the space of continuous functions that are linear within each element (segment) of the chosen grid and satisfy the zero Dirichlet conditions. The inner products are those of L2 . We know from Section 3.3.1 that the Galerkin solution is the best approximation (in P0h ) of the exact solution u∗ , in the sense of minimum “energy” (uh − u∗ , uh − u∗ ). Geometrically, it is the best (in the same energy sense) representation of the curve u∗ (x) by a broken line compatible with a given mesh.
128
3 The Finite Element Method
Surprisingly, in the case under consideration the best approximation actually interpolates the exact solution; in other words, the nodal values of the exact and numerical solutions are the same. In reference to Fig. 3.27 on p. 124, approximation of the exact solution (solid line) by the the piecewise-linear interpolant (dotted line) on a fixed grid cannot be improved by shifting the dotted line up or down a bit. Proof. Let us treat vh in the Galerkin problem (3.114) for uh as a generalized function (distribution; see Appendix 6.15 on p. 343).27 Then −uh , vh = (f, vh ),
uh , ∀vh ∈ P0h
where the angle brackets denote a linear functional acting on uh and vh is the second distributional derivative of vh . This transformation of the left hand side is simply due to the definition of distributional derivative. The right hand side is transformed in a similar way, after noting that f = −u , where u is the exact solution of the Poisson equation. We obtain uh , vh = (u, vh ) or
uh − u, vh = 0, vh
∀vh ∈ P0h
(3.115) 28
is a piecewise-constant function and hence It remains to be noted that vh is a set of Dirac delta-functions residing at the grid nodes. This makes it obvious that (3.115) is satisfied if and only if uh indeed interpolates the exact solution at the nodes of the grid. Exactness of the FE solution at the grid nodes is an extreme particular case of the more general phenomenon of superconvergence: the accuracy of the FE solution at certain points (e.g. element nodes or barycenters) is asymptotically higher than the average accuracy. The large body of research on superconvergence includes books, conference proceedings and many journal publications.29 27
28
29
The reviewer of this book noted that in a purely mathematical text the use of distributional derivatives would not be appropriate without presenting a rigorous theory first. However, distributions (Dirac delta-functions in particular) make our analysis here much more elegant and simple. I rely on the familiarity of applied scientists and engineers – the intended audience of this book – with deltafunctions, even if the usage is not backed up by full mathematical rigor. With zero mean due to the Dirichlet boundary conditions for vh , but otherwise arbitrary. M. Kˇriˇzek, P. Neittaanmaki & R. Stenberg, eds. Finite Element Methods: Superconvergence, Post-Processing, and a Posteriori Estimates, Lecture Notes in Pure and Applied Mathematics, vol. 196, Marcel Dekker: New York, 1998. L.B. Wahlbin, Superconvergence in Galerkin Finite Element Methods, Berlin; New York: Springer-Verlag, 1995. M. Kˇr´ıˇzek, Superconvergence phenomena on three-dimensional meshes, Int. J. of Num. Analysis and Modeling, vol. 2, pp. 43–56, 2005. L. Chen has assembled a reference database at http://math.ucsd.edu/˜clong/Paper/html/Superconvergence.html .
3.11 An Overview of System Solvers
129
3.11 An Overview of System Solvers The finite element method leads to systems of equations with large matrices – in practice, the dimension of the system can range from thousands to millions. When the method is applied to differential equations, the matrices are sparse because each basis function is local and spans only a few neighboring elements; nonzero entries in the FE matrices correspond to the overlapping supports of the neighboring basis functions. (The situation is different when FEM is applied to integral equations. The integral operator is nonlocal and typically all unknowns in the system of equations are coupled; the matrix is full. Integral equations are considered in this book only in passing.) The sparsity (adjacency) structure of a matrix is conveniently described as a graph. For an n × n matrix, the graph has n nodes.30 To each nonzero entry aij of the matrix there corresponds the graph edge i − j. If the structure of the matrix is not symmetric, it is natural to deal with a directed graph and distinguish between edges i → j and j → i (each of them may or may not be present in the graph, independently of the other one). Symmetric structures can be described by undirected graphs. As an example, the directed graph corresponding to the matrix ⎛ ⎞ 2 0 3 1 ⎜ 1 1 0 0⎟ ⎜ ⎟ (3.116) ⎝ 0 0 4 0⎠ −1 0 0 3 is shown in Fig. 3.28. For simplicity, the diagonal entries of the matrix are always tacitly assumed to be nonzero and are not explicitly represented in the graph. An important question in finite difference and finite element analysis is how to solve such large sparse systems effectively. One familiar approach is Gaussian elimination of the unknowns one by one. As the simplest possible illustration, consider a system of two equations of the form x1 f1 a11 a12 = (3.117) a21 a22 x2 f2 For the natural order of elimination of the unknowns (x1 eliminated from the first equation and substituted into the others, etc.) and for a nonzero a11 , we obtain x1 = (f1 − a12 x2 )/a11 and −1 (a22 − a21 a−1 11 a12 ) x2 = f2 − a11 f1
(3.118)
This simple result looks innocuous at first glance but in fact foreshadows a problem with the elimination process. Suppose that in the original system 30
For matrices arising in finite difference or finite element methods, the nodes of the graph typically correspond to mesh nodes; otherwise graph nodes are abstract mathematical entities.
130
3 The Finite Element Method
Fig. 3.28. Matrix sparsity structure as a graph: an example.
(3.117) the diagonal entry a22 is zero. In the transformed system (3.118) this is no longer so: the entry corresponding to x2 (the only entry in the remaining 1 × 1 matrix) is a22 − a21 a−1 11 a12 . Such transformation of zero matrix entries into nonzeros is called “fill-in”. For the simplistic example under consideration, this fill-in is of no practical consequence. However, for large sparse matrices, fill-in tends to accumulate in the process of Gaussian elimination and becomes a serious complication. In our 2 × 2 example with a22 = 0, the fill-in disappears if the order of equations (or equivalently the sequence of elimination steps) is changed: x1 f1 a21 0 = x2 f2 a11 a12 Obviously, x1 is now found immediately from the first equation, and x2 is computed from the second one, with no additional nonzero entries created in the process. In general, permutations of rows and columns of a sparse matrix may have a dramatic effect on the amount of fill-in, and hence on the computational cost and memory requirements, in Gaussian elimination. Gaussian elimination is directly linked to matrix factorization into lowerand upper-triangular terms. More specifically, the first factorization step can be represented in the following form: ⎞ ⎞ ⎛ ⎞ ⎛ ⎛ l11 0 . . . 0 u11 u12 . . . u1n a11 a12 . . . a1n ⎟ ⎟ ⎜ 0 ⎟ ⎜ ⎜ a21 ⎟ = ⎜ l21 ⎟ ⎜ ⎟ (3.119) ⎜ ⎠ ⎠ ⎝. . . ⎠ ⎝. . . ⎝. . . A1 L1 U1 an1 ln1 0 The fact that this factorization is possible (and even not unique) can be verified by direct multiplication of the factors in the right hand side. This
3.11 An Overview of System Solvers
131
yields, for the first diagonal element, first column and first row, respectively, the following conditions: l11 u11 = a11 l21 u11 = a21 , l31 u11 = a31 , . . . , ln1 u11 = an1 l11 u12 = a12 , l11 u13 = a13 , . . . , l11 u1n = a1n where n is the dimension of matrix A. Fixing l11 by, say, setting it equal to one defines the column vector l1 = (l11 , l21 , . . . , ln1 )T and the row vector uT1 = (u11 , u12 , . . . , u1n ) unambiguously: l11 = 1;
u11 = a11
−1 −1 l21 = u−1 11 a21 , l31 = u11 a31 , . . . , ln1 = u11 an1
u12 = a12 , u13 = a13 , . . . , u1n = a1n
(3.120) (3.121) (3.122)
Further, the condition for matrix blocks L1 and U1 follows directly from factorization (3.119): L1 U1 + l1 uT1 = A1 or equivalently L1 U1 = A˜1 where A˜1 ≡ A1 − l1 uT1 The updated matrix A˜1 is a particular case of the Schur complement (R.A. Horn & C.R. Johnson [HJ90], Y. Saad [Saa03]). Explicitly the entries of A˜1 can be written as (3.123) a ˜1,ij = aij − li1 u1j = aij − ai1 a−1 11 a1j Thus the first step of Gaussian factorization A = LU is accomplished by computing the first column of L (3.120), (3.121), the first row of U (3.120), (3.122) and the updated block A˜1 (3.123). The factorization step is then repeated for A˜1 , etc., until (at the n-th stage) the trivial case of a 1 × 1 matrix results. Theoretically, it can be shown that this algorithm succeeds as long as all leading minors of the original matrix are nonzero. In practical computation, however, care should be taken to ensure computational stability of the process (see below). Once the matrix is factorized, solution of the original system of equations reduces to forward elimination and backward substitution, i.e. to solving systems with the triangular matrices L and U , which is straightforward. An important advantage of Gaussian elimination is that, once matrix factorization has been performed, equations with the same matrix but multiple right hand sides can be solved at the very little cost of forward elimination and backward substitution only. Let us review a few computational aspects of Gaussian elimination.
132
3 The Finite Element Method
1. Fill-in. The matrix update formula (3.123) clearly shows that a zero matrix entry aij can become nonzero in the process of LU -factorization. The 2 × 2 example considered above is the simplest possible case of such fill-in. A quick look at the matrix update equation (3.123) shows how the fill-in is reflected in the directed sparsity graph. If at some step of the process node k is being eliminated, any two edges i → k and k → j produce a new edge i → j (corresponding to a new nonzero matrix entry ij). This is reminiscent of the usual “head-to-tail” rule of vector addition. Fig. 3.29 may serve as an illustration. Similar considerations apply for symmetric sparsity structures represented by undirected graphs. Methods to reduce fill-in are discussed below. 2. The computational cost. For full matrices, the number of arithmetic operations (multiplications and additions) in LU -factorization is approximately 2n3 /3. For sparse matrices, the cost depends very strongly on the adjacency structure and can be reduced dramatically by clever permutations of rows and columns of the matrix and other techniques reviewed later in this section. 31 3. Stability. Detailed analysis of LU factorization (J.H. Wilkinson [Wil94], G.H. Golub & C.F. Van Loan [GL96], G.E. Forsythe & C.B. Moler [FM67], N.J. Higham [Hig02]) shows that numerical errors (due to roundoff) can accumulate if the entries of L and U grow. Such growth can, in turn, be traced back to small diagonal elements arising in the factorization process. To rectify the problem, the leading diagonal element at each step of factorization is maximized either via complete pivoting – reshuffling of rows and columns of the remaining matrix block – or via partial pivoting – reshuffling of rows only. The existing theoretical error estimates for both types of pivoting are much more pessimistic than practical experience indicates.32 31
32
Incidentally, the O(n3 ) operation count is not asymptotically optimal for solving large systems with full matrices of size n × n. In 1969, V. Strassen discovered a trick for computing the product of two 2 × 2 block matrices with seven block multiplications instead of eight that would normally be needed [Str69]. When applied recursively, this idea leads to O(nγ ) operations, with γ = log2 7 ≈ 2.807. Theoretically, algorithms with γ as low as 2.375 now exist, but they are computationally unstable and have very large numerical prefactors that make such algorithms impractical. I. Kaporin has developed practical (i.e. stable and faster than straightforward multiplication for matrices of moderate size) algorithms with the asymptotic operation count O(N 2.7760 ) [Kap04]. Note that solution of algebraic systems with full matrices can be reduced to matrix multiplication (V. Pan [Pan84]). See also S. Robinson [Rob05] and H. Cohn et al. [CKSU05]. J.H. Wilkinson [Wil61] showed that for complete pivoting the growth factor for the numerical error does not exceed n1/2 (21 × 31/2 × 41/3 × . . . × n1/(n−1) )1/2 ∼ Cn0.25 log n (which is ∼ 3500 for n = 100 and ∼ 8.6 × 106 for n = 1000). In practice, however, there are no known matrices with this growth factor higher than n. For partial
3.11 An Overview of System Solvers
133
In fact, partial pivoting works so well in practice that it is used almost exclusively: higher stability of complete pivoting is mostly theoretical but its higher computational cost is real. Likewise, orthogonal factorizations such as QR, while theoretically more stable than LU -factorization, are hardly ever used as system solvers because their computational cost is approximately twice that of LU .33 L.N. Trefethen [Tre85] gives very interesting comments on this and related matters. Remarkably, the modern use of Gaussian elimination can be traced back to a single 1948 paper by A.M. Turing34 [Tur48, Bri92]. N.J. Higham writes ([Hig02], pp. 184–185): “ [Turing] formulated the . . . LDU factorization of a matrix, proving [that the factorization exists and is unique if all leading minors of the matrix are nonzero] and showing that Gaussian elimination computes an LDU factorization. He introduced the term “condition number” . . . He used the word “preconditioning” to mean improving the condition of a system of linear equations (a term that did not come into popular use until the 1970s). He described iterative refinement for linear systems. He exploited backward error ideas. . . . he analyzed Gaussian elimination with partial pivoting for general matrices and obtained [an error bound]. ” The case of sparse symmetric positive definite (SPD) systems has been studied particularly well, for two main reasons. First, such systems are very common and important in both theory and practice. Second, it can be shown that the factorization process for SPD matrices is always numerically stable (A. George & J.W.H. Liu [GL81], G.H. Golub & C.F. Van Loan [GL96], G.E. Forsythe & C.B. Moler [FM67]). Therefore one need not be concerned with pivoting (permutations of rows and columns in the process of factorization) and can concentrate fully on minimizing the fill-in. The general case of nonsymmetric and/or non-positive definite matrices will not be reviewed here but is considered in several monographs: books by O. Østerby & Z. Zlatev [sZZ83] and by I.S. Duff et al. [DER89], as well as a much more recent book by T.A. Davis [Dav06]. The remainder of this section deals exclusively with the SPD case and is, in a sense, a digest of the excellent treatise by A. George & J.W.H. Liu [GL81]. For SPD matrices, it is easy to show that in the LU factorization U can be
33 34
pivoting, the bound is 2n−1 , and this bound can in fact be reached in some exceptional cases. QR algorithms are central in eigenvalue solvers; see Appendix 7.15 on p. 478. Alan Mathison Turing (1912–1954), the legendary inventor of the Turing machine and the Bombe device that broke (with an improvement by Gordon Welchman) the German Enigma codes during World War II. Also well known is the Turing test that defines a “sentient” machine. Overall, Turing lay the foundation of modern computer science. See http://www.turing.org.uk/turing
134
3 The Finite Element Method
Fig. 3.29. Block arrows indicate fill-in created in a matrix after elimination of unknown #1.
taken as LT , leading to Cholesky factorization LLT already mentioned on p. 113. Cholesky decomposition has a small overhead of computing the square roots of the diagonal entries of the matrix; this overhead can be avoided by using the LDLT factorization instead (where D is a diagonal matrix). Methods for reducing fill-in are based on reordering of rows and columns of the matrix, possibly in combination with block partitioning. Let us start with the permutation algorithms. The simplest case where the sparsity structure can be exploited is that of banded matrices. The band implies part of the matrix between two subdiagonals parallel to the main diagonal or, more precisely, the set of entries with indexes i, j such that −k1 ≤ i − j ≤ k2 , where k1,2 are nonnegative integers. A matrix is banded if its entries are all zero outside a certain band (in practice, usually k1 = k2 = k). The importance of this notion for Gaussian (or Cholesky) elimination lies in the easily verifiable fact that the band structure is preserved during factorization, i.e. no additional fill is created outside the band. Cholesky decomposition for a band matrix requires approximately k(k + 3)n/2 multiplicative operations, which for k n is much smaller than the number of operations needed for the decomposition of a full matrix n × n. A very useful generalization is to allow the width of the band to vary rowby-row: k = k(i). Such a variable-width band is called an envelope. Figs. 3.22 (p. 115) and 3.23 may serve as a helpful illustration. Again, no fill is created outside the envelope. Since the minimal envelope is obviously a subset of the minimal band, the computational cost of the envelope algorithm is generally lower than that of the band method.35 The operation count for the envelope 35
I disregard the small overhead related to storage and retrieval of matrix entries in the band and envelope.
3.11 An Overview of System Solvers
135
method can be found in George & Liu’s book [GL81], along with a detailed description and implementation of the Reverse Cuthill–McKee ordering algorithm that reduces the envelope size. There is no known algorithm that would minimize the computational cost and/or memory requirements for a matrix with any given sparsity structure, even if pivoting is not involved, and whether or not the matrix is SPD. D.J. Rose & R.E. Tarjan [RT75] state (but do not include the proof) that this problem for a non-SPD matrix is NP-complete and conjecture that the same is true in the SPD case. However, powerful heuristic algorithms are available, and the underlying ideas are clear from adjacency graph considerations. Fig. 3.30 shows a small fragment of the adjacency graph; thick lines in Fig. 3.31 represent the corresponding fill-in if node #1 is eliminated first. These figures are very similar to Figs. 3.28 and 3.29, except that the graph for a symmetric structure is unordered.
Fig. 3.30. Symmetric sparsity structure as a graph: an example.
Elimination of a node couples all the nodes to which it is connected. If nodes 2, 3 and 4 were to be eliminated prior to node 1, there would be no fillin in this fragment of the graph. This simple example has several ramifications. First, a useful heuristic is to start the elimination with the graph vertices that have the fewest number of neighbors, i.e. the minimum degree. (Degree of a vertex is the number of edges incident to it.) The minimum degree algorithm, first introduced by W.F. Tinney & J.W. Walker [TW67], is quite useful and effective in practice, although there is of course no guarantee that local minimization of fill-in at each step of factorization will lead to global optimization of the whole process. George & Liu [GL81] describe the Quotient
136
3 The Finite Element Method
Fig. 3.31. Fill-in (block arrows) created in a matrix with symmetric sparsity structure after elimination of unknown #1.
Minimum Degree (QMD) method, an efficient algorithmic implementation of MD in the SPARSPAK package that they developed. Second, it is obvious from Fig. 3.31 that elimination of the root of a tree in a graph is disastrous for the fill-in. The opposite is true if one starts with the leaves of the tree. This observation may not seem practical at first glance, as adjacency graphs in FEM are very far from being trees.36 What makes the idea useful is block factorization and partitioning. Suppose that graph G (or, almost equivalently, the finite element mesh) and G by a separator S, so that G = G G2 S is split into two parts G 1 2 1 and G1 G2 = ∅; this corresponds to block partitioning of the system matrix. The partitioning has a tree structure, with the separator as the root and G1,2 as the leaves. The system matrix has the following block form: ⎛ ⎞ 0 LG1,S LG1 LG2 LG2,S ⎠ L = ⎝ 0 (3.124) T LG1,S LTG2,S LS Elimination of block LG1 leaves the zero blocks unchanged, i.e. does not – on the block level – generate any fill in the matrix. For comparison, if the “root” block LS were eliminated first (quite unwisely), zero blocks would be filled. George & Liu [GL81, GL89] describe two main partitioning strategies: One-Way Dissection (1WD) and Nested Dissection (ND). In 1WD, the graph is partitioned by several dissecting lines that are, if viewed as geometric objects 36
For first order elements in FEM, the mesh itself can be viewed as the sparsity graph of the system matrix, element nodes corresponding to graph vertices and element edges to graph edges. For a 2D triangular mesh with n nodes, the number of edges is approximately 2n, whereas for a tree it is n − 1.
3.11 An Overview of System Solvers
137
on the FE mesh, approximately “parallel”.37 Taken together, the separators form the root of a tree structure for the block matrix; the remaining disjoint blocks are the leaves of the tree. Elimination of the leaves generates fill-in in the root block, which is acceptable as long as the size of this block is moderate. To get an idea about the computational savings of 1WD as compared to the envelope method, one may consider an m × l rectangular grid (m < l) in 2D38 and optimize the number of operations or, alternatively, memory requirements with respect to the chosen number of separators, each separator being a grid line with m nodes. The end result is that the memory in 1WD can be ∼ 6/m times smaller than for the envelope method [GL81]. For example, if m = 100, the savings are by about a factor of four ( 6/100 ≈ 0.25). A typical ND separator in 2D can geometrically be pictured as two lines, horizontal and vertical, that split the graph into four approximately equal parts. The procedure is then applied recursively to each of the disjoint subgraphs. For a regular m × m grid in 2D, one can write a recursive relationship for the amount of computer memory MND (m) needed for ND; this ultimately yields [GL81] 31 2 m log2 m + O(m2 ) MND (m) = 4 Hence for 2D problems ND is asymptotically almost optimal in terms of its memory requirements: the memory is proportional to the number of nodes times a relatively mild logarithmic factor. However, the computational cost is not optimal even for 2D meshes: the number of multiplicative operations is approximately 829 3 m + O(m2 log2 m) 84 That is, the computational cost grows as the number of nodes n to the power of 1.5. Performance of direct solvers further deteriorates in three dimensions. For example, the computational cost and memory for ND scale as O(n2 ) and O(n4/3 ), respectively, when the number of nodes n is large. Some improvement has been achieved by combining the ideas of 1WD, ND and QMD, with a recursive application of multisection partitioning of the graph. These algorithms are implemented in the SPOOLES software package39 developed by C. Ashcraft, R. Grimes, J. Liu and others [AL98, AG99]. For illustration, Fig. 3.32 shows the number of nonzero entries in the Cholesky factor for several ordering algorithms as a function of the number of nodes in the finite element mesh. This data is for the scalar electrostatic equation in a cubic 37
38
39
The separators need not be straight lines, as their construction is topological (based on the sparsity graph) rather than geometric. The word “parallel” therefore should not be taken literally. A similar estimate can also be easily obtained for 3D problems, but in that case 1WD is not very efficient. SParse Object Oriented Linear Equations Solver, netlib.org/linalg/spooles/spooles.2.2.html
138
3 The Finite Element Method
domain; Nested Dissection and one of the versions of Multistage Minimum Degree from the SPOOLES package perform better than other methods in this case.40
Fig. 3.32. Comparison of memory requirements (number of nonzero entries in the Cholesky factor) as a function of the number of finite element nodes for the scalar electrostatic equation in a cubic domain. Algorithms: Quotient Minimum Degree, Nested Dissection and two versions of Multistage Minimum Degree from the SPOOLES package.
The limitations of direct solvers for 3D finite element problems are apparent, the main bottleneck being memory requirements due to the fill in the Cholesky factor (or the LU factors in the nonsymmetric case): tens of millions of nonzero entries for meshes of fairly moderate size, tens of thousands of nodes. The difficulties are exacerbated in vector problems, in particular the ones that arise in electromagnetic analysis in 3D. Therefore for many 3D problems, and for some large 2D problems, iterative solvers are indispensable, their key advantage being a very limited amount of extra memory required.41 In comparison with direct solvers, iterative ones are arguably more diverse, more dependent on the algebraic properties of matrices, and would require a more wide-ranging review and explanation. To avoid sidetracking the main line of our discussion in this chapter, I refer the reader to the excellent monographs and review papers on iterative solvers by 40
41
I thank Cleve Ashcraft for his detailed replies to my questions on the usage of SPOOLES 2.2 when I ran this and other tests in the Spring of 2000. Typically several auxiliary vectors in Krylov subspaces and sparse preconditioners need to be stored; see references below.
3.12 Electromagnetic Problems and Edge Elements
139
Y. Saad & H.A. van der Vorst [Saa03, vdV03b, SvdV00], L.A. Hageman & D.M. Young [You03, HY04], and O. Axelsson [Axe96].
3.12 Electromagnetic Problems and Edge Elements 3.12.1 Why Edge Elements? In electromagnetic analysis and a number of other areas of physics and engineering, the unknown functions are often vector rather than scalar fields. A straightforward finite element model would involve approximation of the Cartesian components of the fields. This approach was historically the first to be used and is still in use today. However, it has several flaws – some of them obvious and some hidden. An obvious drawback is that nodal element discretization of the Cartesian components of a field leads to a continuous approximation throughout the computational domain. This is inconsistent with the discontinuity of some field components – in particular, the normal components of E and H – at material boundaries. The treatment of such conditions by nodal elements is possible but rather awkward: the interface nodes are “doubled,” and each of the two coinciding nodes carries the field value on one side of the interface boundary. Constraints then need to be imposed to couple the Cartesian components of the field at the double nodes; the algorithm becomes inelegant. Although this difficulty is more of a nuisance than a serious obstacle for implementing the component-wise formulation, it is also an indication that something may be “wrong” with this formulation on a more fundamental level (more about that below). So-called “spurious modes” – the hidden flaw of the component-wise treatment – were noted in the late 1970s and provide further evidence of some fundamental limitations of Cartesian approximation. These modes are frequently branded as “notorious,” and indeed hundreds of papers have been published on this subject.42 As a representative example, consider the computation of the eigenfrequencies ω and the corresponding electromagnetic field modes in a cavity resonator. The resonator is modeled as a simply connected domain Ω with perfectly conducting walls ∂Ω. The governing equation for the electric field is ∇ × µ−1 ∇ × E − ω 2 E = 0 in Ω;
n × E = 0 on ∂Ω
(3.125)
where the standard notation for the electromagnetic material parameters µ, and for the exterior normal n to the domain boundary ∂Ω is used. The ideally 42
320 ISI database references at the end of 2006 for the term “spurious modes”. This does not include alternative relevant terminology such as spectral convergence, spurious-free approximation, “vector parasites,” etc., so the actual number of papers is much higher.
140
3 The Finite Element Method
conducting walls cause the tangential component of the electric field to vanish on the boundary. Mathematically, the proper functional space for this problem is H0 (curl, Ω) – the space of square-integrable vector functions with a square-integrable curl and a vanishing tangential component at the boundary: H0 (curl, Ω) ≡ {E : E ∈ L2 (Ω), ∇×E ∈ L2 (Ω), n×E = 0 on ∂Ω} (3.126) The weak formulation is obtained by inner-multiplying the eigenvalue equation by an arbitrary test function E ∈ H0 (curl, Ω): (∇ × µ−1 ∇ × E, E ) − ω 2 (E, E ) = 0,
∀E ∈ H0 (curl, Ω)
(3.127)
where the inner product is that of L2 (Ω), i.e. X · Y dΩ (X, Y) ≡ Ω
for vector fields X and Y in H0 (curl, Ω). Using the vector calculus identity ∇ · (X × Y) = Y · ∇ × X − X · ∇ × Y
(3.128)
with X = µ−1 ∇ × E, Y = E , equation (3.127) can be integrated by parts to yield (µ−1 ∇ × E, ∇ × E ) − ω 2 (E, E ) = 0,
∀E ∈ H0 (curl, Ω)
(3.129)
(It is straightforward to verify that the surface integral resulting from to the left hand side of (3.128) vanishes, due to the fact that n × E = 0 on the wall.) The discrete problem is obtained by restricting E and E to a finite element subspace of H0 (curl, Ω); a “good” way of constructing such a subspace is the main theme of this section. The mathematical theory of convergence for the eigenvalue problem (3.129) is quite involved and well beyond the scope of this book;43 however, some uncomplicated but instructive observations can be made. The continuous eigenproblem in its strong form (3.125) guarantees, for nonzero frequencies, zero divergence of the D vector (D = E). This immediately follows by applying the divergence operator to the equation. For the weak formulation (3.129), the zero-divergence condition is satisfied in the generalized form (see Appendix 3.17 on p. 186): (E, ∇φ ) = 0
(3.130)
This follows by using, as a particular case, an arbitrary curl-free test function E = ∇φ in (3.129).44 43
44
References: the book by P. Monk [Mon03], papers by P. Monk & L. Demkowicz [MD01], D. Boffi et al. [BFea99, Bof01] and S. Caorsi et al. [CFR00]. The equivalence between curl-free fields and gradients holds true for simply connected domains.
3.12 Electromagnetic Problems and Edge Elements
141
It is now intuitively clear that the divergence-free condition will be correctly imposed in the discrete (finite element) formulation if the FE space contains a “sufficiently dense”45 population of gradients E = ∇φ . This argument was articulated for the first time (to the best of my knowledge) by A. Bossavit in 1990 [Bos90]. From this viewpoint, a critical deficiency of component-wise nodal approximation is that the corresponding FE space does not ordinarily contain “enough” gradients. The reason for that can be inferred from Fig. 3.33 (2D illustration for simplicity). Suppose that there exists a function φ vanishing outside a small cluster of elements and such that its gradient is in P13 – i.e. continuous throughout the computational domain and linear within each element. It is clear that φ must be a piecewise-quadratic function of coordinates. Furthermore, since ∇φ vanishes on the outer side of edge 23, due to the continuity of the gradient along that edge φ can only vary in proportion to n223 within element 123, where n23 is the normal to edge 23. Similarly, φ must be proportional to n234 in element 134. However, these two quadratic functions are incompatible along the common edge 13 of these two elements, unless the normals n23 and n34 are parallel.
Fig. 3.33. A fragment of a 2D finite element mesh. A piecewise-quadratic function φ vanishes outside a cluster of elements. For ∇φ to be continuous, φ must be proportional to n223 within element 123 and to n234 within element 134. However, these quadratic functions are incompatible on the common edge 13, unless the normals n23 and n34 are parallel.
This observation illustrates very severe constraints on the construction of irrotational continuous vector fields that would be piecewise-linear on a given FE mesh. As a result, the FE space does not contain a representative set of 45
The quotation marks are used as a reminder that this analysis does not have full mathematical rigor.
142
3 The Finite Element Method
gradients for the divergence-free condition to be enforced even in weak form. Detailed mathematical analysis and practical experience indicate that this failure to impose the zero divergence condition on the D vector usually leads to nonphysical solutions. The argument presented above is insightful but from a rigorous mathematical perspective incomplete. A detailed analysis can be found in the literature cited in footnote 43 on p. 140. For our purposes, the important conclusion is that the lack of spectral convergence (i.e. the appearance of “spurious modes”) is inherent in component-wise finite element approximation of vector fields. Attempts to rectify the situation by imposing additional constraints on the divergence, penalty terms, etc., have had only limited success. A radical improvement can be achieved by using edge elements described in Section 3.12.2 below. As we shall see, the approximation provided by these elements is, in a sense, more “physical” that the component-wise representation of vector fields; the corresponding mathematical structures also prove to be quite elegant. 3.12.2 The Definition and Properties of Whitney-N´ ed´ elec Elements As became apparent in Section 3.8.1 on p. 108 and in Section 3.9 on p. 123, a natural coordinate system for triangular and tetrahedral elements is formed by the barycentric coordinates λα (α = 1, 2, 3 for triangles and α = 1, 2, 3, 4 for tetrahedra). Each function λ is linear and equal to one at one of the nodes and zero at all other nodes. Since the barycentric coordinates play a prominent role in the finite element approximation of scalar fields, it is sensible to explore how they can be used to approximate vector fields as well, and not in the component-wise sense. Remark 5. The most mathematically sound framework for the material of this section is provided by the differential-geometric treatment of physical fields as differential forms rather than vector fields. A large body of material – well written and educational – can be found on A. Bossavit’s website.46 (Bossavit is an authority in this subject area and one of the key developers and proponents of edge element analysis.) Other references are cited in Section 3.12.4 on p. 146 and in Section 3.16 on p. 184. While differential geometry is a standard tool for mathematicians and theoretical physicists, it is not so for many engineers and applied scientists. For this reason, only regular vector calculus is used in this section and in the book in general; this is sufficient for our purposes. Natural “vector offspring” of the barycentric coordinates are the gradients ∇λα . These, however, are constant within each element and can therefore represent only piecewise-constant and – even more importantly – only irro6−12 = λα ∇λβ ; it tational vector fields. Next, we may consider products ψαβ 46
http://www.lgep.supelec.fr/mocosem/perso/ab/bossavit.html
3.12 Electromagnetic Problems and Edge Elements
143
is sufficient
to restrict them to α = β because the gradients are linearly dependent, ∇λα = 0. Superscript “6-12” indicates that there are six such functions for a triangle and 12 for a tetrahedron. A little later, we shall con3−6 . sider a two times smaller set ψαβ It almost immediately transpires that these new vector functions have one of the desired properties: their tangential components are continuous across element facets (edges for triangles and faces for tetrahedra), while their normal components are in general discontinuous. The most elegant way to demonstrate the tangential continuity is by noting that the generalized curl 6−12 = ∇ × (λα ∇λβ ) = ∇λα × ∇λβ is a regular function, not only ∇ × ψαβ a distribution, because the λs are continuous.47 (A jump in the tangential component would result in a Dirac-delta term in the curl; see Appendix 3.17 on p. 186 and formula (3.215) in particular.) The tangential components can also be examined more explicitly. The 6−12 over the corresponding edge αβ is circulation of ψαβ
edge αβ
6−12 ψαβ · τˆαβ dτ =
λα ∇λβ · τˆαβ dτ edge αβ
= ∇λβ · τˆαβ
λα dτ = edge αβ
1 1 1 lαβ = lαβ 2 2
(3.131)
where τˆαβ is the unit edge vector pointing from node α to node β, and lαβ is the edge length. In the course of the transformations above, it was taken into account that (i) ∇λβ is a (vector) constant, (ii) λα is a function varying from zero to one linearly along the edge, so that the component of its gradient along the edge is 1/lαβ and the mean value of λα over the edge αβ is 1/2. 6−12 is equal to 1/2 over its respecThus the circulation of each function ψαβ tive edge αβ and (as is easy to see) zero over all other edges. One type of edge element is defined by introducing (i) the functional space 6−12 basis, and (ii) a set of degrees of freedom, two per edge: spanned by the ψαβ the tangential components Eαβ of the field (say, electric field E) at each node α along each edge αβ emanating from that node. The number of degrees of freedom and the dimension of the functional space are six for triangles and 12 for tetrahedra. It is not difficult to verify that the space in fact coincides with the space of linear vector functions within the element. A major difference, however, is that the basis functions for edge elements are only tangentially continuous, in contrast with fully continuous component-wise approximation by nodal elements. The FE representation of the field within the edge element is 6−12 Eαβ ψαβ Eh = α=β 47
Here each barycentric coordinate is viewed as a function defined in the whole domain, continuous everywhere but nonzero only over a cluster of elements sharing the same node.
144
3 The Finite Element Method
An interesting alternative is obtained by observing that each pair of functions 6−12 6−12 , ψβα have similar properties: their circulations along the respective ψαβ edge (but taken in the opposite directions) are the same, and their curls are opposite. It makes sense to combine each pair into one new function as 3−6 6−12 6−12 ≡ ψαβ − ψβα = λα ∇λβ − λβ ∇λα ψαβ
(3.132)
6−12 that the circulation of It immediately follows from the properties of ψαβ 3−6 ψαβ is one along its respective edge (in the direction from node α to node β) and zero along all other edges. The FE representation of the field is almost the same as before 3−6 cαβ ψαβ Eh = α=β
except that summation is now over a twice smaller set of basis functions, one per edge: three for triangles and six for tetrahedra; cαβ are the circulations of the field along the edges. Fig. 3.34 helps to visualize two such functions for a triangular element; for tetrahedra, the nature of these functions is similar. Their rotational character is obvious from the figure, the curls being equal to 3−6 = 2∇λα × ∇λβ ∇ × ψαβ
3−6 (left) Fig. 3.34. Two basis functions ψ 3−6 visualized for a triangular element: ψ23 3−6 and ψ12 (right).
The (generalized) divergence of these vector basis functions (see Appendix 3.17, p. 186) is also of interest: 3−6 = λα ∇2 λβ − λβ ∇2 λα ∇ · ψαβ
3.12 Electromagnetic Problems and Edge Elements
145
When viewed as regular functions within each element, the Laplacians in the right hand side are zero because the barycentric coordinates are linear functions. However, these Laplacians are nonzero in the sense of distributions and contain Dirac-delta terms on the interelement boundaries due to the jumps of the normal component of the gradients of λ. Disregard of the distributional term has in the past been the source of two misconceptions about edge elements: 1. The basis set ψ 3−6 presumably cannot be used to approximate fields with nonzero divergence. However, if this were true, linear elements, by similar considerations, could not be used to solve the Poisson equation with a nonzero right hand side because the Laplacian of the linear basis functions is zero within each element. 2. Since the basis functions have zero divergence, spurious modes are eliminated. While the conclusion is correct, the justification would only be valid if divergence were zero in the distributional sense. Furthermore, there are families of edge elements that are not divergence-free and yet do not produce spurious modes. Rigorous mathematical analysis of spectral convergence is quite involved (see footnote 43 on p. 140). 3.12.3 Implementation Issues As already noted on p. 140, the finite element formulation of the cavity resonance problem (3.129) is obtained by restricting E and E to a finite element subspace Wh ⊂ H0 (curl, Ω) (µ−1 ∇ × Eh , ∇ × Eh ) − ω 2 (Eh , Eh ) = 0,
∀E ∈ Wh
(3.133)
Subspace Wh can be spanned by either of the two basis sets introduced in the previous section for tetrahedral elements (one or two degrees of freedom per edge) or, alternatively, by higher order tetrahedral bases or bases on hexahedral elements (Section 3.12.4). In the algorithmic implementation of the procedure, the role of the edges is analogous to the role of the nodes for nodal elements. In particular, the matrix sparsity structure is determined by the edge-to-edge adjacency: for any two edges that do not belong to the same element, the corresponding matrix entry is zero. An excellent source of information on adjacency structures and related algorithms (albeit not directly in connection with edge elements) is S. Pissanetzky’s monograph [Pis84]. A new algorithmic issue, with no analogs in node elements, is the orientation of the edges, as the sign of field circulations depends on it. To make orientations consistent between several elements sharing the same edge, it is convenient to use global node numbers in the mesh. One suitable convention is to define the direction from the smaller global node number to the greater one as positive.
146
3 The Finite Element Method
3.12.4 Historical Notes on Edge Elements In 1980 and 1986, J.-C. N´ed´elec proposed two families of tetrahedral and hexahedral edge elements [N´ 80, N´ 86]. For tetrahedral elements, N´ed´elec’s sixand twelve-dimensional approximation spaces are spanned by the vector basis functions λα ∇λβ − λβ ∇λα and λα ∇λβ , respectively, as discussed in the previous section. N´ed´elec’s exposition is formally mathematical and rooted heavily in the calculus of differential forms. As a result, there was for some time a disconnect between the outstanding mathematical development and its use in the engineering community. To applied scientists and engineers, finite element analysis starts with the basis functions. This makes practical sense because one cannot actually solve an FE problem without specifying a basis. Many practitioners would be surprised to hear that a basis is not part of the standard mathematical definition of a finite element. In the mathematical literature, a finite element is defined, in addition to its geometric shape, by a (finite-dimensional) approximation space and a set of degrees of freedom – linear functionals over that approximation space (see e.g. the classical book by P.G. Ciarlet [Cia80]). Nodal values are the most typical such functionals, but there certainly are other possibilities as well. As we already know, in N´ed´elec’s elements the linear functionals are circulations of the field along the edges. N´ed´elec built upon related ideas of P.-A. Raviart & J.M. Thomas who developed special finite elements on triangles in the late 1970s [RT77]. It took almost a decade to transform edge elements from a mathematical theory into a practical tool. A. Bossavit’s contribution in that regard is exceptional. He presented, in a very lucid way, the fundamental rationale for edge elements [Bos88b, Bos88a] and developed their applications to eddy current problems [BV82, BV83], scattering [BM89], cavity resonances [Bos90], force computation [Bos92] and other areas. Stimulated by prior work of uller P.R. Kotiuga48 and the mathematical papers of J. Dodziuk [Dod76], W. M¨ [M¨ 78] and J. Komorowski [Kom75], Bossavit discovered a link between the tetrahedral edge elements with six degrees of freedom and differential forms in the 1957 theory of H. Whitney [Whi57]. N´ed´elec’s original papers did not explicitly specify any bases for the FE spaces. Since practical computation does rely on the bases, the engineering and computational electromagnetics communities in the late 1980s and in the 1990s devoted much effort to more explicit characterization of edge element spaces. A detailed description of various types of elements would lead us too far astray, as this book is not a treatise on electromagnetic finite element analysis. However, to give the reader a flavor of some developments in this area, and to provide a reference point for the experts, succinct definitions of 48
Kotiuga was apparently the first to note, in his 1985 Ph.D. thesis, the connection of finite element analysis in electromagnetics with the fundamental branches of mathematics: differential geometry and algebraic topology.
3.12 Electromagnetic Problems and Edge Elements
147
several common edge element spaces are compiled in Appendix 3.12.5 (see also [Tsu03]). Further information can be found in the monographs by P. Monk [Mon03], J. Jin [Jin02] and J.L. Volakis et al. [VCK98]. Comparative analysis of edge element spaces by symbolic algebra can be found in [Tsu03]. Families of hierarchical and adaptive elements developed independently by J.P. Webb [WF93, Web99, Web02] and by L. Vardapetyan & L. Demkowicz [VD99] deserve to be mentioned separately. In hierarchical refinement, increasingly accurate FE approximations are obtained by adding new functions to the existing basis set. This can be done both in the context of h-refinement (reducing the element size and adding functions supported by smaller elements to the existing functions on larger elements) and p-refinement (adding, say, quadratic functions to the existing linear ones). Hierarchical and adaptive refinement are further discussed in Section 3.13 for the scalar case. The vectorial case is much more complex, and I defer to the papers cited above for additional information. One more paper by Webb [Web93] gives a concise but very clear exposition of edge elements and their advantages. 3.12.5 Appendix: Several Common Families of Tetrahedral Edge Elements Several representative families of elements, with the corresponding bases, are listed below. The list is definitely not exhaustive; for example, Demkowicz– Vardapetyan elements with hp-refinement and R. Hiptmair’s general perspective on high order edge elements are not included. As before, λi is the barycentric coordinate corresponding to node i (i = 1,2,3,4) of a tetrahedral element. 1. The Ahagon–Kashimoto basis (20 functions) [AK95]. {12 “edge” functions (4λi − 1)(λi ∇λj − λj ∇λi ), i = j} {4λ1 (λ2 ∇λ3 − λ3 ∇λ2 ), 4λ2 (λ3 ∇λ1 −λ1 ∇λ3 ), 4λ1 (λ3 ∇λ4 −λ4 ∇λ3 ), 4λ4 (λ1 ∇λ3 −λ3 ∇λ1 ), 4λ1 (λ2 ∇λ4 − λ4 ∇λ2 ), 4λ2 (λ1 ∇λ4 − λ4 ∇λ1 ), 4λ2 (λ3 ∇λ4 − λ4 ∇λ3 ), 4λ4 (λ2 ∇λ3 − λ3 ∇λ2 )}. 2. The Lee–Sun–Cendes basis (20 functions) [LSC91]. {12 edge-based { λ1 λ2 ∇λ3 , λ1 λ3 ∇λ2 , λ2 λ3 ∇λ4 , λ2 λ4 ∇λ3 , functions λi ∇λj , i = j} λ3 λ4 ∇λ1 , λ3 λ1 ∇λ4 , λ4 λ1 ∇λ2 , λ4 λ2 ∇λ1 }. 3. The Kameari basis (24 functions) [Kam99]. {the Lee basis} { ∇(λ2 λ3 λ4 ), ∇(λ1 λ3 λ4 ), ∇(λ1 λ2 λ4 ), ∇(λ1 λ2 λ3 ) }. 4. The Ren–Ida basis (20 functions) [RI00]. {12 edge-based functions λi ∇λj , i = j} { λ1 λ2 ∇λ3 − λ2 λ3 ∇λ1 , λ1 λ3 ∇λ2 − λ2 λ3 ∇λ1 , λ1 λ2 ∇λ4 − λ4 λ2 ∇λ1 , λ1 λ4 ∇λ2 −λ4 λ2 ∇λ1 , λ1 λ3 ∇λ4 −λ4 λ3 ∇λ1 , λ1 λ4 ∇λ3 −λ3 λ4 ∇λ1 , λ2 λ3 ∇λ4 − λ4 λ3 ∇λ2 , λ2 λ4 ∇λ3 − λ3 λ2 ∇λ4 }. 5. The Savage–Peterson basis [SP96]. {12 edge-based functions λi ∇λj , i = j} { λi λj ∇λk − λi λk ∇λj , λi λj ∇λk − λj λk ∇λi , 1 ≤ i < j < k ≤ 4}. 6. The Yioultsis–Tsiboukis basis (20 functions) [YT97]. {(8λi 2 − 4λi )∇λj + {16λ1 λ2 ∇λ3 − 8λ2 λ3 ∇λ1 − 8λ3 λ1 ∇λ2 ; (−8λi λj + 2λj )∇λi , i = j}
148
3 The Finite Element Method
16λ1 λ3 ∇λ2 −8λ3 λ2 ∇λ1 −8λ2 λ1 ∇λ3 ; 16λ4 λ1 ∇λ2 −8λ1 λ2 ∇λ4 −8λ2 λ4 ∇λ1 ; 16λ4 λ2 ∇λ1 −8λ2 λ1 ∇λ4 −8λ1 λ4 ∇λ2 ; 16λ2 λ3 ∇λ4 −8λ3 λ4 ∇λ2 −8λ4 λ2 ∇λ3 ; 16λ2 λ4 ∇λ3 −8λ4 λ3 ∇λ2 −8λ3 λ2 ∇λ4 ; 16λ3 λ1 ∇λ4 −8λ1 λ4 ∇λ3 −8λ4 λ3 ∇λ1 ; 16λ3 λ4 ∇λ1 − 8λ4 λ1 ∇λ3 − 8λ1 λ3 ∇λ4 }. 7. The Webb–Forghani basis (20 functions) [WF93]. {6 edge-based functions {6 edge-based functions ∇(λi λj ), i = j} { λi ∇λj − λj ∇λi , i = j} λ1 λ2 ∇λ3 , λ1 λ3 ∇λ2 , λ2 λ3 ∇λ4 , λ2 λ4 ∇λ3 , λ3 λ4 ∇λ1 , λ3 λ1 ∇λ4 , λ4 λ1 ∇λ2 , λ4 λ2 ∇λ1 }. 8. The Graglia–Wilton–Peterson basis (20 functions) [GWP97]. { (3λi − 9/2 × {λ2 (λ3 ∇λ4 − λ4 ∇λ3 ), λ3 (λ4 ∇λ2 − 1)(λi ∇λj − λj ∇λi ), i = j} λ2 ∇λ4 ), λ3 (λ4 ∇λ1 − λ1 ∇λ4 ), λ4 (λ1 ∇λ3 − λ3 ∇λ1 ), λ4 (λ1 ∇λ2 − λ2 ∇λ1 ), λ1 (λ4 ∇λ2 − λ2 ∇λ4 ), λ1 (λ2 ∇λ3 − λ3 ∇λ2 ), λ2 (λ1 ∇λ3 − λ3 ∇λ1 )}.
3.13 Adaptive Mesh Refinement and Multigrid Methods 3.13.1 Introduction One of the most powerful ideas that has shaped the development of Finite Element Analysis since the 1980s is adaptive refinement. Once an FE problem has been solved on a given initial mesh, special a posteriori error estimates or indicators49 are used to identify the subregions with relatively high error. The mesh is then refined in these areas, and the problem is re-solved. It is also possible to “unrefine” the mesh in the regions where the error is perceived to be small. The procedure is then repeated recursively and is typically integrated with efficient system solvers such as multigrid cycles or multilevel preconditioners (Section 3.13.4). There are two main versions of mesh refinement. In h-refinement, the mesh size h is reduced in selected regions to improve the accuracy. In p-refinement, the element-wise order p of local approximating polynomials is increased. The two versions can be combined in an hp-refinement procedure. There are numerous ways of error estimation (Section 3.13.3 on p. 151) and numerous algorithms for effecting the refinement. To summarize, adaptive techniques are aimed at generating a quasioptimal mesh adjusted to the local behavior of the solution, while maintaining a high convergence rate of the iterative solver. Three different but related issues arise: 1. Implementation of local refinement without violating the geometric conformity of the mesh. 2. Efficient multilevel iterative solvers. 3. Local a posteriori error estimates. 49
Estimates provide an approximate numerical value of the actual error. Indicators show whether the error is relatively high or low, without necessarily predicting its numerical value.
3.13 Adaptive Mesh Refinement and Multigrid Methods
149
Fig. 3.35 shows nonconforming (“slave”) nodes appearing on a common boundary between two finite elements e1 and e2 if one of these elements (say, e1 ) is refined and the other one (e2 ) is not. The presence of such nodes is a deviation from the standard set of requirements on a FE mesh. If no restrictions are imposed, the continuity of the solution at slave nodes will generally be violated. One remedy is a transitory (so-called “green”) refinement of element e2 (W.F. Mitchell [Mit89, Mit92], F. Bornemann et al. [BEK93]) as shown in Fig. 3.35, right. However, green refinement generally results in non-nested meshes, which may affect the performance of iterative solvers.
Fig. 3.35. Local mesh refinement (2D illustration for simplicity). Left: continuity of the solution at “slave” nodes must be maintained. Right: “green refinement”. c (Reprinted by permission from [TP99a] 1999 IEEE.)
3.13.2 Hierarchical Bases and Local Refinement Alternatively, nonconforming nodes may be retained if proper continuity conditions are imposed. This can be accomplished in a natural way in the hierarchical basis (H. Yserentant [Yse86], W.F. Mitchell [Mit89, Mit92], U. R¨ ude [R¨93]). A simple 1D example (Fig. 3.36) illustrates the hierarchical basis representation of a function. In the nodal basis a piecewise-linear function has a vector of nodal values u(N ) = (u1 , u2 , u3 , u4 , u5 , u6 )T . Nodes 5 and 6 are generated by refining the coarse level elements 1-2 and 2-3. In the hierarchical basis, the degrees of freedom at nodes 5, 6 correspond to the difference between the values on the fine level and the interpolated value from the coarse level. Thus the vector in the hierarchical basis is 1 1 u(H) = (u1 , u2 , u3 , u4 , u5 − (u1 + u2 ), u6 − (u2 + u3 ))T 2 2
(3.134)
This formula effects the transformation from nodal to hierarchical values of the same piecewise-linear function. More generally, let a few levels of nested FE meshes (in one, two or three dimensions) be generated by recursively subdividing some or all elements on
150
3 The Finite Element Method
Fig. 3.36. A fragment of a two-level 1D mesh. (Reprinted by permission from c [TP99a] 1999 IEEE.)
a coarser level into several smaller elements. For simplicity, only first order nodal elements will be considered and it will be assumed that new nodes are added at the midpoints of the existing element edges. (The ideas are quite general, however, and can be carried over to high order elements and edge elements; see e.g. P.T.S. Liu & J.P. Webb [LW95], J.P. Webb & B. Forghani [WF93].) The hierarchical representation of a piecewise-linear function can be obtained from its nodal representation by a recursive application of elementary transforms similar to (3.134). Precise theory and implementation are detailed by H. Yserentant [Yse86]. An advantage of the hierarchical basis is the natural treatment of slave nodes (Fig. 3.35, left). The continuity of the solution is ensured by simply setting the hierarchical basis value at these nodes to zero. Remark 6. In the nonconforming refinement of Fig. 3.35 (left), element shapes do not deteriorate. However, this advantage is illusory. Indeed, the FE space for the “green refinement” of Fig. 3.35 (right) obviously contains the FE space of Fig. 3.35 (left), and therefore the FE solution with slave nodes cannot be more accurate than for green refinement. Thus the effective “mesh quality,” unfortunately, is not preserved with slave nodes. For tetrahedral meshes, subdividing an element into smaller ones when the mesh is refined is not trivial; careless subdivision may lead to degenerate elements. S.Y. Zhang [Zha95] proposed two schemes: “labeled edge subdivision” and “short-edge subdivision” guaranteeing that tetrahedral elements do not degenerate in the refinement process. The initial stage of both methods is the same: the edge midpoints of the tetrahedron are connected, producing four corner tetrahedra and a central octahedron. The octahedron can be further subdivided into four tetrahedra in three different ways [Zha95] by using one additional edge. The difference between Zhang’s two refinement schemes is in the way this additional edge is chosen. The “labeled edge subdivision”
3.13 Adaptive Mesh Refinement and Multigrid Methods
151
algorithm relies on a numbering convention for nodes being generated (see [Zha95] for details). In the “short edge subdivision” algorithm the shortest of the three possible interior edges is selected. For tetrahedra without obtuse planar angles between edges both refinement schemes are equivalent, provided that the initial refinement is the same – i.e. for a certain numbering of nodes of the initial element [Zha95]. Zhang points out that “in general, it is not simple to find the measure of degeneracy for a given tetrahedron” [Zha95] and uses as such a measure the ratio of the maximum edge length to the radius of the inscribed sphere. A. Plaks and I used a more precise criterion – the minimum singular value condition (Section 3.14) to compare the two refinement schemes. Short-edge subdivision in general proves to be better than labeled edge subdivision [TP99b]. 3.13.3 A Posteriori Error Estimates Adaptive hp-refinement requires some information about the distribution of numerical errors in the computational domain. The FE mesh is refined in the regions where the error is perceived to be higher and left unchanged, or even unrefined, in regions with lower errors. Numerous approaches have been developed for estimating the errors a posteriori – i.e. after the FE solution has been found. Some of these approaches are briefly reviewed below; for comprehensive treatment, see monographs by M. Ainsworth & J.T. Oden [AO00], I. Babuˇska & T. Strouboulis [BS01], R. Verf¨ urth [Ver96], and W. Bangerth & R. Rannacher [BR03]. Much information and many references for this section were provided by S. Prudhomme, the reviewer of this book; his help is greatly appreciated. The overview below follows the book chapter by Prudhomme & Oden [PO02] as well as W.F. Mitchell’s paper [Mit89]. Recovery-based error estimators These methods were proposed by O.C. Zienkiewicz & J.Z. Zhu; as of May 2007, their 1987 and 1992 papers [ZZ87, ZZ92a, ZZ92b] were cited 768, 531 and 268 times, respectively. The essence of the method, in a nutshell, is in field averaging. The computed field within an element is compared with the value obtained by double interpolation: element-to-node first and then nodeto-element. The intuitive observation behind this idea is that the field typically has jumps across element boundaries; these jumps are a numerical artifact that can serve as an error indicator. The averaging procedure captures the magnitudes of the jumps. Some versions of the Zienkiewicz–Zhu method rely on superconvergence properties of the FE solution at special points in the elements. For numerical examples and validation of gradient-recovery estimators, see e.g. I. Babuˇska et al. [BSU+ 94]. The method is easy to implement and in my experience (albeit limited mostly to magnetostatic problems) works well
152
3 The Finite Element Method
[TP99a].50 One difficulty is in handling nodes at material interfaces, where the field jump can be a valid physical property rather than a numerical artifact. In our implementation [TP99a] of the Zienkiewicz–Zhu scheme, the field values were averaged at the interface nodes separately for each of the materials involved. Ainsworth & Oden [AO00] note some drawbacks of recovery-based estimators and even present a 1D example where the recovery-based error estimate is zero, while the actual error can be arbitrarily large. Specifically, they consider a 1D Poisson equation with a rapidly oscillating sinusoidal solution. It can be shown (see Appendix 3.10, p. 127) that the FE-Galerkin solution with first-order elements actually interpolates the exact solution at the FE mesh nodes. Hence, if these nodes happen to be located at the zeros of the oscillating exact solution, the FE solution, as well as all the gradients derived from it, are identically zero! Prudhomme & Oden also point out that for problems with shock waves gradient recovery methods tend to indicate mesh refinement around the shock rather than at the shock itself. Residual-based methods While the solution error is not directly available, residual – the difference between the right and left hand sides of the equation – is. For a problem of the form Lu = ρ (3.135) and the corresponding weak formulation L(u, v) = (ρ, v)
(3.136)
Ruh ≡ ρ − Luh
(3.137)
R(uh , v) ≡ (ρ, v) − L(uh , v)
(3.138)
the residual is or in the weak form
Symbols L and R here are overloaded (with little possibility of confusion) as operators and the corresponding bilinear forms. The numerical solution uh satisfies the Galerkin equation in the finitedimensional subspace Vh . In the full space V residuals (3.137) or (3.138) are, in general, nonzero and can serve as a measure of accuracy. In principle, the error, and hence the exact solution, can be found by solving the problem with the residual in the right hand side. However, doing so is no less difficult than solving the original problem in the first place. Instead, one looks for 50
Joint work with A. Plaks.
3.13 Adaptive Mesh Refinement and Multigrid Methods
153
computationally inexpensive ways of extracting useful information about the magnitude of the error from the magnitude of the residual. One of the simplest element-wise error estimators of this kind combines, with proper weights, two residual-related terms: (Lu − ρ)2 integrated over the volume (area) of the element and the jump of the normal component of flux density, squared and integrated over the facets of the element (R.E. Bank & A.H. Sherman [BS79]). P. Morin et al. [MNS02] develop convergence theory for adaptive methods with this estimator and emphasize the importance of the volume-residual term that characterizes possible oscillations of the solution. A different type of method, proposed by I. Babuˇska & W.C. Rheinboldt in the late 1970s, makes use of auxiliary problems over small clusters (“patches”) of adjacent elements [BR78b, BR78a, BR79]. To gain any additional nontrivial information about the error, the auxiliary local problem must be solved with higher accuracy than the original global problem, i.e. the FE space has to be locally enriched (usually using h- or p-refinement). An alternative interpretation (W.F. Mitchell [Mit89]) is that such an estimator measures how strongly the FE solution would change if the mesh were to be refined locally. Yet another possibility is to solve the problem with the residual globally but approximately, using only a few iterations of the conjugate gradient method (Prudhomme & Oden [PO02]). Goal-oriented error estimation In practice, FE solution is often aimed at finding specific quantities of interest – for example, field, temperature, stress, etc. at a certain point (or points), equivalent parameters (e.g. capacitance or resistance between electrodes), and so on. Naturally, the effort should then be concentrated on obtaining these quantities of interest, rather than the overall solution, with maximum accuracy. Pointwise estimates have a long history dating back at least to the the 1940s–1950s (H.J. Greenberg [Gre48], C.B. Maple, [Map50]; K. Washizu [Was53]). The key idea can be briefly summarized as follows. One can express the value of solution u at a point r0 using the Dirac delta functional as (3.139) u(r0 ) = u, δ(r − r0 ) (Appendix 6.15 on p. 343 gives an introduction to generalized functions (distributions), with the Dirac delta among them.) Further progress can be made by using Green’s function g of the L operator:51 Lg(r, r0 ) = δ(r − r0 ). Then u(r0 ) = (u, Lg(r, r0 )) = (L∗ u, g(r, r0 )) = L∗ (u, g(r, r0 ))
(3.140)
where symbol L∗ is the adjoint operator and (again with overloading) the corresponding bilinear form L∗ (u, v) ≡ L(v, u). The role of Green’s function in 51
The functional space where this operator is defined, and hence the boundary conditions, remain fixed in the analysis.
154
3 The Finite Element Method
this analysis is to convert the delta functional (3.139) that is hard to evaluate directly into an L-form that is closely associated with the problem at hand. The right hand side of (3.140) typically has the physical meaning of the mutual energy of two fields. For example, if L is the Laplace operator (selfadjoint if the boundary conditions are homogeneous), then the right hand side is (∇u, ∇g) – the inner product (mutual energy) of fields −∇u (the solution) and −∇g (field of a point source). Importantly, due to the variational nature of the problem, lower and upper bounds can be established for u(r0 ) of (3.140) (A.M. Arthurs [Art80]). Moreover, bounds can be established for the pointwise error as well. In the finite element context (1D), this was done in 1984 by E.C. Gartland [EG84]. Also in 1984, in a series of papers [BM84a, BM84b, BM84c], I. Babuˇska & A.D. Miller applied the duality ideas to a posteriori error estimates and generalized the method to quantities of physical interest. In Babuˇska & Miller’s example of an elasticity problem of beam deformation, such quantities include the average displacement of the beam, the shear force, the bending moment, etc. For a contemporary review of the subject, including both the duality techniques and goal-oriented estimates with adaptive procedures, see R. Becker & R. Rannacher [BR01] and J.T. Oden & S. Prudhomme [OP01]. For electromagnetic applications, methods of this kind were developed by R. Albanese, R. Fresa & G. Rubinacci [AF98, AFR00], by J.P. Webb [Web05] and by P. Ingelstrom & A. Bondeson [IB]. Fully Adaptive Multigrid In this approach, developed by W.F. Mitchell [Mit89, Mit92] and U. R¨ ude [R¨ 93]), solution values in the hierarchical basis (Section 3.13.2, p. 149) characterize the difference between numerical solutions at two subsequent levels of refinement and can therefore serve as error estimators. 3.13.4 Multigrid Algorithms The presentation of multigrid methods in this book faces a dilemma. These methods are first and foremost iterative system solvers – the subject matter not in general covered in the book. On the other hand, multigrid methods, in conjunction with adaptive mesh refinement, have become a truly state-ofthe-art technique in modern FE analysis and an integral part of commercial FE packages; therefore the chapter would be incomplete without mentioning this subject. Fortunately, several excellent books exist, the most readable of them being the one by W.L. Briggs et al. [BHM00], with a clear explanation of key ideas and elements of the theory. For a comprehensive exposition of the mathematical theory, the monographs by W. Hackbusch [Hac85], S.F. McCormick
3.13 Adaptive Mesh Refinement and Multigrid Methods
155
[McC89], P. Wesseling [Wes91] and J.H. Bramble [Bra93], as well as the seminal paper by A. Brandt [Bra77], are highly recommended; see also the review paper by C.C. Douglas [Dou96]. On a historical note, the original development of multilevel algorithms is attributed to the work of the Russian mathematicians R.P. Fedorenko [Fed61, Fed64] and N.S. Bakhvalov [Bak66] in the early 1960s. There was an explosion of activity after A. Brandt further developed the ideas and put them into practice [Bra77]. As a guide for the reader unfamiliar with the essence of multigrid methods, this section gives a narrative description of the key ideas, with “hand-waving” arguments only. Consider the simplest possible model 1D equation Lu ≡ −
d2 u = f dx2
on Ω = [0, a];
u(0) = u(a) = 0
(3.141)
where f is a given function of x. FE-Galerkin discretization of this problem leads to a system of equations Lu = f
(3.142)
where u and f are Euclidean vectors and L is a square matrix; u represents the nodal values of the FE solution. For first order elements, matrix L is three-diagonal, with 2 on the main diagonal and −1 on the adjacent ones. (The modification of the matrix due to boundary conditions, as described in Section 3.7.1, will not be critical in this general overview.) Operator L has a discrete set of spatial eigenfrequencies and eigenmodes, akin to the modes of a guitar string. As Fig. 3.37 illustrates, the discrete operator L of (3.142) inherits the oscillating behavior of the eigenmodes but has only a finite number of those. There is the Nyquist limit for the highest spatial frequency that can be adequately represented on a grid of size h. Fig. 3.37 exhibits the eigenmodes with lowest and highest frequency on a uniform grid with 16 elements. Any iterative solution process for equation (3.142) – including multigrid solvers – involves an approximation v to the exact solution vector u. The error vector (3.143) e ≡ u − v is of course generally unknown in practice; however, the residual r = f − Lv is computable. It is easy to see that the residual is equal to Le : r = f − Lv = Lu − Lv = Le
(3.144)
The following sequence of observations leads to the multigrid methodology. 1. High-frequency components of the error – or, equivalently, of the residual – (similar to the bottom part of Fig. 3.37) can be easily and rapidly reduced
156
3 The Finite Element Method
Fig. 3.37. Eigenvectors with lowest (top) and highest (bottom) spatial frequency. Laplace operator discretized on a uniform grid with 16 elements.
by applying basic iterative algorithms such as Jacobi or Gauss–Seidel. In contrast, low-frequency components of the error decay very slowly. See [BHM00, Tre97, GL96] for details. 2. Once highly oscillatory components of the error have been reduced and the error and the residual have thus become sufficiently smooth, the problem can be effectively transferred to a coarser grid (typically, twice coarser). The procedure for information transfer between the grids is outlined below. The spatial frequency of the eigenmodes relative to the coarser grid is higher than on the finer grid, and the components of the error that are
3.13 Adaptive Mesh Refinement and Multigrid Methods
157
oscillatory relative to the coarse grid can be again eliminated with basic iterative solvers. This is effective not only because the relative frequency is higher, but also because the system size on the coarser grid is smaller. 3. It remains to see how the information transfer between finer and coarser grids is realized. Residuals are transferred from finer to coarser grids. Correction vectors obtained after smoothing iterations on coarser grids are transferred to finer grids. There is more than one way of defining the transfer operators. Vectors from a coarse grid can be moved to a fine one by some form of interpolation of the nodal values. The simplest fine-tocoarse transfer is injection: the values at the nodes of the coarse grids are taken to be the same as the values at the corresponding nodes of the fine grid. However, it is often desirable that the coarse-to-fine and fine-to-coarse transfer operators be adjoint to one another,52 especially for symmetric problems, to preserve the symmetry. In that case the fine-to-coarse transfer is different from injection. Multigrid utilizes these ideas recursively, on a sequence of nested grids. There are several ways of navigating these grids. V-cycle starts on the finest grid and descends gradually to the coarsest one; then moves back to the finest level. W-cycle also starts by traversing all fine-to-coarse levels; then, using the coarsest level as a base, it goes back-and forth in rounds spanning an increasing number of levels. Finally, full multigrid cycle starts at the coarsest level and moves back-and-forth, involving progressively more and more finer levels. A precise description and pictorial illustrations of these algorithms can be found in any of the multigrid books. Convergence of multigrid methods depends on the nature of the underlying problem: primarily, in mathematical terms, on whether or not the problem is elliptic and on the level of regularity of the solution, on the particular type of the multigrid algorithm employed, and to a lesser extent on other details (the norms in which the error is measured, smoothing algorithms, etc.) For elliptic problems, convergence can be close to optimal – i.e. proportional to the size of the problem, possibly with a mild logarithmic factor that in practice is not very critical. Furthermore, multigrid methods can be used as preconditioners in conjugate gradient and similar solvers; particularly powerful are the Bramble– Pasciak–Xu (BPX) preconditioners developed in J. Xu’s Ph.D. thesis [Xu89] and in [BPX90]. Since BPX preconditioners are expressed as double sums over all basis functions and over all levels, they are relatively easy to parallelize. A broad mathematical framework for multilevel preconditioners and for the analysis of convergence of multigrid methods in general is established 52
There is an interesting parallel with Ewald methods of Chapter 5, where chargeto-grid and grid-to-charge interpolation operators must be adjoint for conservation of momentum in a system of charged particles to hold numerically; see p. 262.
158
3 The Finite Element Method
in Xu’s papers [Xu92, Xu97]. Results of numerical experiments with BPX for several electromagnetic applications are reported by A. Plaks and myself in [Tsu94, TPB98, PTPT00]. Another very interesting development is algebraic multigrid (AMG) schemes, where multigrid ideas are applied in an abstract form (K. St¨ uben et al. [St¨ u83, SL86, St¨ u00]). The underlying problem may or may not involve any actual geometric grids; for example, there are applications to electric circuits and to coupled field-circuit problems (D. Lahaye et al. [LVH04]). In AMG, a hierarchical structure typical of multigrid methods is created artificially, by examining the strength of the coupling between the unknowns. The main advantage of AMG is that it can be used as a “black box” solver. For further information, the interested reader is referred to the books cited above and to the tutorials posted on the MGNet website.53
3.14 Special Topic: Element Shape and Approximation Accuracy The material of this section was inspired by my extensive discussions with Alain Bossavit and Pierre Asselin in 1996–1999. (By extending the analysis of J.L. Synge [Syn57], Asselin independently obtained a result similar to the minimum singular value condition on p. 170.) Numerical experiments were performed jointly with Alexander Plaks. I also thank Ivo Babuˇska and Randolph Bank for informative conversations in 1998–2000. 3.14.1 Introduction Common sense, backed up by rigorous error estimates (Section 3.10, p. 125) tells us that the accuracy of the finite element approximation depends on the element size and on the order of polynomial interpolation. More subtle is the dependence of the error on element shape. Anyone who has ever used FEM knows that a triangular element similar to the one depicted on the left side of Fig. 3.38 is “good” for approximation, while the element shown on the right is “bad”. The flatness of the second element should presumably lead to poor accuracy of the numerical solution. But how flat are flat elements? How can element shape in FEM be characterized precisely and how can the “source” of the approximation error be identified? Some of the answers to these questions are classical but some are not yet well known, particularly the connection between approximation accuracy and FE matrices (Section 3.14.2), as well as the minimum singular value criterion for the “edge shape matrix” (Sections 3.14.2 and 3.14.3). The reader need not be an expert in FE analysis to understand the first part of this section; the second part is more advanced. Overall, the section is 53
http://www.mgnet.org/mgnet-tuts.html
3.14 Special Topic: Element Shape and Approximation Accuracy
159
based on my papers [Tsu98b, Tsu98a, Tsu98c, TP98, TP99b, Tsu99] (joint work with A. Plaks).
Fig. 3.38. “Good” and “bad” element shape (details in the text).
For triangular elements, one intuitively obvious consideration is that small angles should be avoided. The mathematical basis for that is given by Zl´ amal’s minimum angle condition [Zl8]: if the minimum angle of elements is bounded away from zero, φmin ≥ φ0 > 0, then the FE interpolation error tends to zero for the family of meshes with decreasing mesh sizes. Geometrically equivalent to Zl´amal’s condition is the boundedness of the ratio of the element diameter (maximum element edge lmax ) to the radius ρ of the inscribed circle. Zl´ amal’s condition implies that small angles should be avoided. But must they? In mathematical terms, one may wonder if Zl´ amal’s condition is not only sufficient but in some sense necessary for accurate approximation. If Zl´ amal’s condition were necessary, a right triangle with a small acute angle would be unsuitable. However, on a regular mesh with right triangles, first-order FE discretization of the Laplace equation is easily shown to be identical with the standard 5-point finite difference scheme. But the FD scheme does not have any shape related approximation problems. (The accuracy is limited by the maximum mesh size but not by the aspect ratio.) This observation suggests that Zl´ amal’s condition could be too stringent. Indeed, a less restrictive shape condition for triangular elements exists. It is sufficient to require that the maximum angle of an element be bounded away from π. In particular, according to this condition, right triangles, even with very small acute angles, are acceptable (what matters is the maximum angle that remains equal to π/2). The maximum angle condition appeared in J.L. Synge’s monograph [Syn57] (pp. 209–213) in 1957, before the finite element era. (Synge considered piecewise-linear interpolation on triangles without calling them finite elements.) In 1976, I. Babuˇska & A.K. Aziz [BA76] published a more detailed analysis of FE interpolation on triangles and showed that the maximum angle condition was not only sufficient, but in a sense essential for the convergence of FEM. In addition, they proved the corresponding 92] generalized the maximum angle Wp1 -norm estimate. In 1992, M. Kˇriˇzek [Kˇ condition to tetrahedral elements: the maximum angle for all triangular faces
160
3 The Finite Element Method
and the maximum dihedral angle should be bounded away from π. Other estimates for tetrahedra (and, more generally, simplices in Rd ) were given by Yu.N. Subbotin [Sub90] and S. Waldron [Wal98a]. P. Jamet’s condition [Jam76] is closest to the result of this section but is more difficult to formulate and apply. On a more general theoretical level, the study of piecewise-polynomial interpolation in Sobolev spaces, with applications to spline interpolation and FEM, has a long history dating back to the fundamental works of J. Deny & J.L. Lions, J.H. Bramble & S.R. Hilbert [BH70], I. Babuˇska [Bab71], and the already cited Ciarlet & Raviart paper. Two general approaches systematically developed by Ciarlet & Raviart have now become classical. The first one is based on the multipoint Taylor formula (P.G. Ciarlet & C. Wagschal [CW71]); the second approach (e.g. Ciarlet [Cia80]) relies on the Deny-Lions and Bramble–Hilbert lemmas. In both cases, under remarkably few assumptions, error estimates for Lagrange and Hermite interpolation on a set of points in Rn are obtained. For tetrahedra, the “shape part” of Ciarlet & Raviart’s powerful result (p. 125) translates into the ratio of the element diameter (i.e. the maximum edge) to the radius of the inscribed sphere. Boundedness of this ratio ensures convergence of FE interpolation on a family of tetrahedral meshes with decreasing mesh sizes. However, as in the 2D case, such a condition is a little too restrictive. For example, “right tetrahedra” (having three mutually orthogonal edges) are rejected, even though it is intuitively felt, by analogy with right triangles, that there is in fact nothing wrong with them. A precise characterization of the shape of tetrahedral elements is one of the particular results of the general analysis that follows. An algebraic, rather than geometric, source of interpolation errors for arbitrary finite elements is identified and its geometric interpretation for triangular and tetrahedral elements is given. 3.14.2 Algebraic Sources of Shape-Dependent Errors: Eigenvalue and Singular Value Conditions First, we establish a direct connection between interpolation errors and the maximum eigenvalue (or the trace) of the appropriate FE stiffness matrices. This is different from the more standard consideration of matrices of the affine transformation to/from a reference element (as done e.g. by N. Al Shenk [She94]). As shown below, the maximum eigenvalue of the stiffness matrix has a simple geometric meaning for first and higher order triangles and tetrahedra. Even without a geometric interpretation, the eigenvalue/trace condition is useful in practical FE computation, as the matrix trace is available at virtually no additional cost. Moreover, the stiffness matrix automatically reflects the chosen energy norm, possibly for inhomogeneous and/or anisotropic parameters.
3.14 Special Topic: Element Shape and Approximation Accuracy
161
For the energy-seminorm approximation on first order tetrahedral nodal elements, or equivalently, for L2 -approximation of conservative fields on tetrahedral edge elements (Section 3.12), the maximum eigenvalue analysis leads to a new criterion in terms of the minimum singular value of the “edge shape matrix”. The columns of this matrix are the Cartesian representations of the unit edge vectors of the tetrahedron. The new singular value estimate has a clear algebraic and geometric meaning and proves to be not only sufficient, but in some strong sense necessary for the convergence of FE interpolation on a sequence of meshes. The minimum singular value criterion is a direct generalization of the Synge–Babuˇska–Aziz maximum angle condition to three (and more) dimensions. Even though the approach presented here is general, let us start with first order triangular elements to fix ideas. Let Ω ⊂ R2 be a convex polygonal domain. Following the standard definition, we shall call a set M of triangular elements n Ki , M = {K1 , K2 , . . . , Kn }, a triangulation of the domain if (a) i=1 Ki = Ω; (b) any two triangles either have no common points, or have exactly one common node, or exactly one common edge. Let hi = diam Ki ; then the mesh size h is the maximum of hi for all elements in M (i.e. the maximum edge length of all triangles). Let N be the ¯ of all triangles in M , geometric set of nodes {ri } (i = 1, 2, . . . , n, ri ∈ Ω) and let P 1 (M ) be the space of functions that are continuous in Ω and linear within each of the triangular elements Ki .54 Let P 1 (Ki ) be the restriction of P 1 (M ) to a specific element Ki . Thus P 1 (Ki ) is just the (three-dimensional) space of linear functions over the element. ¯ for simplicity, one can Considering interpolation of functions in C 2 (Ω) ¯ → P 1 (M ) by define the interpolation operator Π : C 2 (Ω) (Πu)(ri ) = u(ri ),
¯ ∀ri ∈ N , ∀u ∈ C 2 (Ω)
(3.145)
We are interested in evaluating the interpolation error Πu−u in the energy norm · E induced by an inner product (· , ·)E (“E” for “energy,” not to be confused with Euclidean spaces).55 Remark 7. In FE applications, u is normally the solution of a certain boundary value problem in Ω. The error bounds for interpolation and for the Galerkin or Ritz projection are closely related (e.g. by C´ea’s lemma or the LBB condition, Section 3.5). Although this provides an important motivation to study interpolation errors, here u need not be associated with any boundary value problem. 54
55
Elsewhere in the book, symbol N denotes the nodal values of a function. The usage of this symbol for the set of nodes is limited to this section only and should not cause confusion. The analysis is also applicable to seminorms instead of norms if the definition of energy inner product is relaxed to allow (u, u)E = 0 for a nonzero u.
162
3 The Finite Element Method
Consider a representative example where the inner product and the energy ¯ are introduced as seminorm in C 2 (Ω) ∇u · ∇v dΩ (3.146) (u, v)E,Ω = Ω 1
|u|E = (u, u)E2
(3.147)
(If Dirichlet boundary conditions on a nontrivial part of the boundary are incorporated in the definition of the functional space, the seminorm is in fact a norm.) The element stiffness matrix A(Ki ) for a given basis {ψ1 , ψ2 , ψ3 } of P 1 (Ki ) corresponds to the energy inner product (3.146) viewed as a bilinear form on P 1 (Ki ) × P 1 (Ki ): (u, v)E,Ki = (A(Ki ) u(Ki ), v(Ki )),
∀u, v ∈ P 1 (Ki )
(3.148)
where vectors of nodal values of a given function are underscored. u(Ki ) is an E 3 vector of node values on a given element and u is an E n vector of node values on the whole mesh. The standard E 3 inner product is implied in the right hand side of (3.148). Explicitly the entries of the element stiffness matrix are given by ∇ψj · ∇ψl dΩ, j, l = 1, 2, 3 (3.149) A(Ki )jl = (ψj , ψl )E,Ki = Ki
To obtain an error estimate over a particular element Ki , we shall use, as ¯ an auxiliary function, the first order Taylor approximation T u of u ∈ C 2 (Ω) around an arbitrary point r0 within that element: (T u)(r0 , r) = u(r0 ) + ∇u(r0 ) · (r − r0 ) Fig. 3.39 illustrates this in 1D. The difference between the nodal values of the Taylor approximation T u and the exact function u (or its FE interpolant Πu) is “small” (on the order of O(h2 ) for linear approximation) and shapeindependent in 2D and 3D. At the same time, the difference between T u and Πu in the energy norm is generally much greater: not only is the order of approximation lower, but also the error can be adversely affected by the element shape. Obviously, somewhere in the transition from the nodal values to the energy norm the precision is lost. Since the energy norm in the FE space is governed by the FE stiffness matrix, the large error in the energy norm indicates the presence of a large eigenvalue of the matrix. For a more precise analysis, let us write the function u as its Taylor approximation plus a (small) residual term R(r0 , r): u(r) = (T u)(r0 , r) + R(r0 , r),
r ∈ Ki ,
where R(r0 , r) can be expressed via the second derivatives of u at an interior point of the segment [r, r0 ]:
3.14 Special Topic: Element Shape and Approximation Accuracy
163
Fig. 3.39. Taylor approximation vs. FE interpolation. Function u (solid line) is approximated by its piecewise-linear node interpolant Πu (dashed line) and by element-wise Taylor approximations T u (dotted lines). The energy norm difference between Πu and T u is generally much greater than the difference in their node values.
R(r0 , r) =
Dα u(r0 + θ(r − r0 )) (r − r0 )α , α!
0≤θ<1
(3.150)
|α|=2
with the standard shorthand notation for the multi-index α = (α1 , α2 , . . . , αd ) (in the current example d = 2), |α| = α1 + α2 + . . . + αd , α! = α1 !α2 ! . . . , αd !, and partial derivatives Dα u =
∂ |α| u d . . . ∂xα d
α2 1 ∂xα 1 ∂x2
It follows from (3.150) that the residual term is indeed small, in the sense that |R(r0 , r)| = |(T u)(r0 , r) − u(r)| ≤ u2,∞,Ki |r − r0 |2
(3.151)
|∇R(r0 , r)| = |∇(T u)(r0 , r) − ∇u(r)| ≤ u2,∞,Ki |r − r0 |
(3.152)
where um,∞,K =
|Dα u|L∞ (K)
(3.153)
|α|=m
The key observations leading to the maximum eigenvalue condition can be informally summarized as follows: 1. The Taylor approximation is uniformly accurate within the element due to (3.151), (3.152) and is completely independent of the element geometry. Therefore, for the purpose of evaluating the dependence of the interpolation error on shape, T u can be used in lieu of u, i.e. one can consider the difference Πu − T u instead of Πu − u.
164
3 The Finite Element Method
2. The energy norm of the difference Πu − T u is generally much higher than the nodal values of Πu − T u: the nodal values are of the order O(h2 ) and independent of element shape due to (3.151), while the energy norm is O(h) and depends on the shape. 3. The above observations imply that in the transition from node values to the energy norm the accuracy is lost. Since within the element Ki both u and T u lie in the FE space P 1 (Ki ), and since in this space the energy norm is induced by the element stiffness matrix A(Ki ), a large energy norm can be attributed to the presence of a large eigenvalue in that stiffness matrix. The first of these statements can be made precise by writing Πu − uE,Ki ≤ Πu − T uE,Ki + T u − uE,Ki 1
≤ Πu − T uE,Ki + chi Vi 2 u2,∞,Ki
(3.154)
where the second inequality follows from estimate (3.152) of the Taylor residual, Vi = meas(Ki ), and c is an absolute constant independent of the element shape and of u. We now focus on the term Πu − T uE,Ki in (3.154). Restrictions of both u and T u to Ki lie in the FE space P 1 (Ki ), and therefore 1
Πu − T uE,Ki = (A(Ki )(u(Ki ) − T u(Ki )), u(Ki ) − T u(Ki )) 2 (3.155) The standard Euclidean inner product in E 3 is implied in the right hand side of (3.155), and we recall that the underscore denotes Euclidean vectors of nodal values. It follows immediately from (3.155) that Πu − T uE,Ki ≤
max
x=0,x∈R3
(A(Ki )x, x) (x, x)
12
u(Ki ) − T u(Ki )E 3 (3.156)
that is, 1
2 (A(Ki )) u(Ki ) − T u(Ki )E 3 Πu − T uE,Ki ≤ λmax
(3.157)
In the right hand side of (3.157), λmax is the maximum eigenvalue of the element stiffness matrix (3.148), (3.149). The difference u(Ki ) − T u(Ki ) is the error vector for the Taylor expansion at element nodes, and due to the uniformity (3.151), (3.152) of the Taylor approximation, we have u(Ki ) − T u(Ki )E 3 ≤ ch2i |u|2,∞,Ki
(3.158)
(the generic constant c is not necessarily the same in all occurrences). Combining (3.157) and (3.158), we obtain the element-wise estimate 1
2 (A(Ki ))|u|2,∞,Ki Πu − T uE,Ki ≤ ch2i λmax
(3.159)
3.14 Special Topic: Element Shape and Approximation Accuracy
or, taking into account the triangle inequality (3.154), 1 1 2 Πu − uE,Ki ≤ c h2i λmax (A(Ki )) + hi Vi 2 |u|2,∞,Ki
165
(3.160)
The corresponding global estimate is Πu − uE,Ω ≤ c|u|2,∞,Ω
h4i λmax (A(Ki )) + h2i Vi
12 (3.161)
Ki ∈M
This result can be simplified by noting that λmax (A(Ki )) ≤ trA(Ki ),
trA(K ) = trA, V = V , where A is the global stiffness matrix i i Ki Ki and V = meas(Ω): 1 1 (3.162) Πu − uE,Ω ≤ c|u|2,∞,Ω h2 (tr 2 A + hV 2 ) Alternatively, one can factor out the element area Vi in (3.160) to obtain 1 1 2 ˆ i )) + hi (3.163) Πu − uE,Ki ≤ cVi 2 |u|2,∞,Ki h2i (λmax (A(K ˆ i ) = A(Ki )/Vi . where the hat denotes the scaled element stiffness matrix A(K Then the global error estimate simplifies to 1 1 2 ˆ i )) + hi |u|2,∞,K (3.164) (A(K Πu − uE,Ω ≤ cV 2 max h2i λmax i Ki ∈M
The maximum eigenvalue can again be replaced with the (easily computable) matrix trace. Remark 8. The trace- and max-terms in estimates (3.162), (3.164) are not of ˆ i )) the order O(h2 ) as it might appear, but O(h), since both trA and λmax (A(K are O(h−2 ). The analysis above can be generalized, without any substantial changes, to elements of any geometric shape and order: Theorem 5. Let M be a finite element mesh in a bounded domain Ω ∈ Rd (d ≥ 1) and let the following assumptions hold for any (scalar or vector) ¯ Ω → Rs , with some nonnegative integers m and s. function u ∈ (C m+1 )s (Ω): (A.1) A given energy (semi)norm is bounded as |u|2E,Ki ≤
ν
c2j |u|2j,∞,Ki ,
cν > 0, Vi = meas(Ki )
(3.165)
j=0
for any element Ki , with constants cj independent of the element. (A.2) The FE approximation space over Ki contains all polynomials of degree ≤ m.
166
3 The Finite Element Method
(A.3) The FE degrees of freedom – linear functionals ψj over the FE space – are bounded as |ψj (Ki )u| ≤
µ
c˜2l |u|l,∞,Ki ,
c˜µ > 0
(3.166)
l=0
for a certain µ ≥ 0, with some absolute constants c˜l . Then 1 1 Πu − uE,Ω ≤ c|u|m+1,∞,Ω hκ tr 2 A + hτ V 2
(3.167)
where V = meas(Ω), κ = m + 1 − µ, τ = m + 1 − ν, and the global stiffness matrix A is given by (3.148), (3.149). Alternatively, 1 1 2 ˆ i )) + hτ (3.168) (A(K Πu − uE,Ω ≤ c|u|m+1,∞,Ω V 2 max hκi λmax Ki
ˆ i ) = A(Ki )/Vi . where A(K The meaning of the parameters in the theorem is as follows: m characterizes the level of smoothness of the function that is being approximated; s = 1 for scalar functions and s > 1 for vector functions with s components, approximated component-wise; ν is the highest derivative “contained” in the energy (semi)norm; µ is the highest derivative in the degrees of freedom. Example 7. First order tetrahedral node elements satisfy assumptions (A.1– A.3). Indeed, for the energy norm (3.147), condition (3.165) holds with ν = 1, c0 = 0, c1 = 1. (A.2) is satisfied with m = 1, and (A.3) is valid because of the uniformity (3.151) of the Taylor approximation within a sufficiently small circle. More generally, (A.3) is satisfied if FE degrees of freedom are represented by a linear combination of values of the function and its derivatives at some specified points of the finite element. Example 8. First order triangular nodal elements. Let the seminorm be (3.147), (3.146). Then the trace of the scaled element stiffness matrix has a simple geometric interpretation. The diagonal elements (j = 1, 2, 3), where the dj s are the altitudes of the matrix are equal to d−2 j of the triangle (Fig. 3.40). Therefore, denoting interior angles of the triangle with φj and its sides with lj , and assuming hi = diam(Ki ) = l1 ≥ l2 ≥ l3 , one obtains 2 3 3 l1 −2 ˆ i )) ≤ Tr A(K ˆ i) = d−2 = h λmax (A(K j i dj j=1 j=1 <
h−2 i
l2 + l 3 d1
2
+
l1 d2
2
+
l1 d3
2
−2 ≤ 3h−2 φ2 + sin−2 φ3 ) i (sin
(3.169)
3.14 Special Topic: Element Shape and Approximation Accuracy
167
which leads to Zl´ amal’s minimum angle condition. This result is reasonable but not optimal, which shows that the maximum eigenvalue criterion does not generally guarantee the sharpest estimates. Nevertheless the optimal condition for first order elements – the maximum angle condition – will be obtained below by applying the maximum eigenvalue criterion to the N´ed´elec–Whitney– Bossavit edge elements.
Fig. 3.40. Geometric parameters of a triangular element Ki .
Example 9. For first order tetrahedral elements, the trace of the scaled nodal stiffness matrix can also be interpreted geometrically. A simple transformation similar to (3.169) [Tsu98b] yields the minimum–maximum angle condition for angles φjl between edges j and faces l: φjl are to be bounded away from both zero and π to ensure that the interpolation error tends to zero as the element size decreases. For higher order scalar elements on triangles and tetrahedra, the matrix trace is evaluated in an analogous but lengthier way, and the estimate is similar, except for an additional factor that depends on the order of the element.56 Example 10. L2 -approximation of scalar functions on tetrahedral or triangular node elements. Suppose that Ω is a two- or three-dimensional polygonal (polyhedral) domain and that continuous and discrete spaces are taken as L2 (Ω) and P 1 (M ), respectively, for a given triangular/tetrahedral mesh. Assume that the energy inner product and norm are the standard L2 ones. This energy norm in the FE space is induced by the “mass matrix” ˆ i )jl = V −1 φi φl dΩ; A(K φi φl dΩ (3.170) A(Ki )jl = i Ki
56
Ki
Here we are discussing shape dependence only, as the factor related to the dependence of the approximation error on the element size is obvious.
168
3 The Finite Element Method
For first order tetrahedral elements, this matrix is given by (3.102) on p. 123, repeated here for convenience: ⎛ ⎞ 2 1 1 1 ⎜ ⎟ ˆ i ) = 1 ⎜1 2 1 1⎟ A(K (3.171) 20 ⎝1 1 2 1⎠ 1 1 1 2 The maximum eigenvalue of Aˆ is equal to 1/4 and does not depend on the element shape. Assumptions (A.1–A.3) of Theorem 5 hold with m = 1, µ = ν = 0, c0 = c˜0 = 1, and therefore approximation of the potential is shapeindependent due to (3.168). This known result is obtained here directly from the maximum eigenvalue condition. Analysis for first order triangular elements is completely similar, and the conclusion is the same. Example 11. (L2 )3 -approximation of conservative vector fields on tetrahedral or triangular meshes. In lieu of the piecewise-linear approximation of u on a triangular or tetrahedral mesh, one may consider the equivalent piecewiseconstant approximation of ∇u on the same mesh. Despite the equivalence of the two approximations, the corresponding error estimates are not necessarily the same, since the maximum eigenvalue criterion is not guaranteed to give optimal results in all cases. It therefore makes sense to apply the maximum eigenvalue condition to interpolation errors in L32 (Ω) for a conservative field q = ∇u on a tetrahedral mesh. To this end, a version of the first order edge element on a tetrahedron K may be defined by the Whitney–N´ed´elec–Bossavit space (see Section 3.12, p. 139) spanned by functions wjl , 1 ≤ j < k ≤ 4 wjk = ljk (λj ∇λk − λk ∇λj )
(3.172)
where the λs are the barycentric coordinates of the tetrahedron. (They also are the nodal basis functions of the first order scalar element.) The scaling factor ljk , introduced for convenience of further analysis, is the length of edge jk. As a reminder, the dimension of the Whitney–N´ed´elec–Bossavit space over one element is equal to the number of element edges, i.e. three for triangles and six for tetrahedra. There is the corresponding global FE space W (M ) (W for “Whitney”) over the whole mesh M . It is a subspace of H(curl, Ω) = {q : q ∈ L32 (Ω), ∇ × q ∈ L32 (Ω)}. The “exactness property” (see A. Bossavit [Bos88b, Bos88a]) of this space is critical for the analysis of this section: if the computational domain is simply connected, a vector field in W (M ) is conservative if and only if it is the gradient of a continuous piecewise-linear scalar field u ∈ P 1 (Ω) on the same mesh. The exactness property remains valid if the definitions of functional spaces are amended in a natural way to include Dirichlet conditions for the tangential components of the field on part of the domain boundary.
3.14 Special Topic: Element Shape and Approximation Accuracy
169
The degrees of freedom are defined as the average values of the tangential components of the field along the edges: −1 q · dτ (3.173) ψjk (q) = ljk edge jk
The maximum eigenvalue estimate could now be directly applied to interpolation in W (M ). However, a more accurate result is obtained by taking the maximum in the right hand side of the generic expression (3.157) in a subspace of R6 . This subspace corresponds to R6 -vectors q of edge d.o.f.’s for vector fields q ∈ ∇P 1 (K). Within a given element, such vector fields are in fact constant and can therefore be treated as vectors in R3 . The subspace maximization of (3.157) yields |q|2 dΩ (A(Ki ) q, q)E 6 |q|2 Ki = max = meas(K ) max (3.174) max3 i q2E 6 q2E 6 q∈R q∈R3 q∈R3 q2 E6 To evaluate the ratio in the right hand side, note that the R6 -vector q of the edge projections of q is related to the column vector of the Cartesian components q C = (qx , qy , qz )T of q as q = E T (Ki ) q C
(3.175)
Here E T (Ki ) is the 3 × 6 “edge shape matrix” whose columns are the unit vectors eα (1 ≤ α ≤ 6) directed along the tetrahedral edges (in either of the two directions): (3.176) E(K) = [e1 | e2 | e3 | e4 | e5 | e6 ] The element index i has been dropped for simplicity of notation. Singular value decomposition (G.H. Golub & C.F. Van Loan [GL96]) of this matrix is the key to further analysis: E(K) = LΣQT
(3.177)
where L is a 3 × 3 orthogonal matrix, Q is a 6 × 6 orthogonal matrix, and ⎞ ⎛ σ1 0 0 0 0 0 (3.178) Σ = ⎝ 0 σ2 0 0 0 0⎠ 0 0 σ3 0 0 0 is the matrix containing the singular values σ1 ≥ σ2 ≥ σ3 ≥ 0 of the edge shape matrix E. Hence q2E 6 = (E T (K)q C , E T (K)q C ) = (E(K) E T (K)q C , q C ) and the last maximum in (3.174) is max3
q∈R
|q|2 = q2E 6
max3
qC ∈R
(qC , qC ) (E(K) E T (K)qC , qC )
(3.179)
170
3 The Finite Element Method −2 T = λ−1 min (E(K) E (K)) = σmin (E(K))
(3.180)
where λmin is the minimum eigenvalue, and σmin is the minimum singular value (if q = 0, σmin = 0 is implied in (3.180)). The last equality of (3.180) is based on the well known fact (G.H. Golub & C.F. Van Loan [GL96]) that σj2 (E(K)) = λj (E(K)E T (K)) = λj (E T (K)E(K))
(3.181)
where for E T E only the nonzero eigenvalues are considered. The minimum singular value σmin (E(K)) is zero if and only if there exists a vector q orthogonal to all six edge vectors ej (so that q = 0), that is, if and only if all edges are coplanar (and the tetrahedron is thus degenerate). In general, the minimum singular value characterizes the “level of degeneracy,” or “flatness” of a tetrahedron. In the maximum eigenvalue condition, parameters now have the following values: m = 0 (the pertinent Taylor approximation is just a vector constant); ν = 0 (L2 -norm); µ = 0 (the d.o.f.’s are the tangential field components along the edges, with no derivatives involved). Hence κ = τ = 1 in (3.167), (3.168), and, with (3.174), (3.180) in mind, one has 12 −2 2 hi σmin (E(Ki )) + 1 Vi |u|2,∞,Ki (3.182) Πu − uE,Ω ≤ c Ki ∈M
This is a global error estimate, but each individual term in the sum represents a (squared) element-wise error. It is not hard to establish an upper bound for σmin (E(Ki )). Indeed, 3 1 2 1 tr(E T E) = 2 σj (E) = 3 j=1 3
σmin (E) ≤
(3.183)
so (3.182) can be simplified: Πu − uE,Ω ≤ c
h2i
−2 σmin (E(Ki )) Vi
12 |u|2,∞,Ki
(3.184)
Ki ∈M
Analysis for triangular elements is quite similar, and the final result is the same. In addition, for triangular elements the following proposition holds: Proposition 6. The minimum singular value criterion for the 2 × 3 edge shape matrix of a first order triangular element is equivalent to the Synge– Babuˇska–Aziz maximum angle condition.
3.14 Special Topic: Element Shape and Approximation Accuracy
171
Proof. The minimum singular value can be explicitly evaluated in this case. Letting the x-axis run along one of the edges of the element (Fig. 3.41), one has the edge shape matrix in the form 1 cos φ1 − cos φ2 (3.185) E = 0 cos φ1 − cos φ2
Fig. 3.41. Three unit edge vectors for a triangular element.
The trace of (EE T )−1 is found to be (with some help of symbolic algebra) Tr(EE T )−1 =
3 sin2 φ1 + sin2 φ2 + sin2 φ3
(3.186)
−1 T T Since tr(EE T )−1 = λ1 (EE T )−1 + λ2 (EE T )−1 = λ−1 1 (EE ) + λ2 (EE ) = −2 −2 σ1 (E) + σ2 (E), one has
2 1 2 (sin2 φ1 + sin2 φ2 + sin2 φ3 ) ≤ σmin (sin2 φ1 + sin2 φ2 + sin2 φ3 ) (E) ≤ 3 3 (3.187) It can immediately be seen from these inequalities that the minimum singular value can be arbitrarily close to zero if and only if the maximum angle approaches π (and the other two angles approach zero). 3.14.3 Geometric Implications of the Singular Value Condition The Minimum Singular Value vs. the Inscribed Sphere Criterion The most common geometric characteristic of a tetrahedral finite element K is the ratio of radius r of the inscribed sphere to the maximum edge lmax . The following inequality shows that the singular value criterion is less stringent than the r/lmax ratio.
172
3 The Finite Element Method
Proposition 7. [Tsu98a] σmin (E) ≥
r lmax
(3.188)
Proof. We appeal to a geometric interpretation of the minimum singular value. For vector q ∈ R6 of edge projections of an arbitrary nonzero Cartesian vector qC ∈ R3 , we have qE 6 σmin (E) ≤ (3.189) q C E 3 where the exact equality is achieved when (and only when) qC is an eigenvector corresponding to the minimum eigenvalue of EE T . Thus σmin (E) =
min
q C E 3 =1
qE 6
(3.190)
We can assume without loss of generality that the first node of the tetrahedron is placed at the origin and that the tetrahedron is scaled to lmax = 1 and rotated to have the unit eigenvector v corresponding to the minimum eigenvalue of EE T run along the z-axis. Let zβ (β = 1, 2, 3, 4) be the z-coordinates of the nodes. According to (3.190), 2 (E) = vE 6 = (v · eαβ )2 ≥ (v · e1β )2 σmin 1≤α<β≤4
=
2≤α<β≤4
zβ l1β
2≤α<β≤4
2 ≥
zβ2
(3.191)
2≤α<β≤4
where each edge is now labeled by its two end nodes; l1β is the length of the edge connecting nodes 1 and β, l1β ≤ lmax = 1. The first summation in (3.191) is over all six edges αβ, while the subsequent summations are over three nodes β = 2, 3, 4 and the corresponding edges 1β. It immediately follows from (3.191) that for all nodes |zβ | ≤ σmin . The scaled tetrahedron therefore lies entirely between the planes z = ±σmin ; hence r ≤ σmin Remark 9. The converse statement that σmin (E) ≤ cr/lmax is not true. Consider a sequence of tetrahedra with three mutually orthogonal edges, two of these edges being of unit length and the third one tending to zero. Then the radius of the inscribed sphere tends to zero, while the minimum singular value remains equal to one [TP98]. Jamet’s Condition P. Jamet [Jam76] obtained accurate interpolation error estimates under quite general assumptions. For tetrahedral elements, the governing factor in Jamet’s estimate is cos θ, where θ is defined as
3.14 Special Topic: Element Shape and Approximation Accuracy
θ = max min θi , i
ξ
i = 1, . . . , 6
173
(3.192)
Here θi is the angle between an arbitrary unit vector ξ ∈ R3 and the unit edge vector ei ; the minimum is taken over all edges, and the maximum is taken over all unit vectors ξ. (Jamet’s angle characterizes, geometrically, how far the edges are from being perpendicular to a certain vector ξ.) It turns out that Jamet’s measure is very closely related to the minimum singular value criterion. Indeed, one can rewrite (3.192) as cos θ = min max cos θi = min E T ξ∞,E 6 ξ
i
ξ
(3.193)
versus σmin (E) = min E T ξ2,E 6 ξ
That is, the only theoretical difference between Jamet’s cos θ and the minimum singular value of the edge shape matrix is in the matrix norm employed. This adds further credence to the analysis and results that involve eigenvalues and singular values of FE matrices. Jamet’s condition is more general than the present formulation of the minimum singular value estimate (in particular, Jamet’s analysis applies to any Sobolev norms in Wpm ). On the other hand, computational algorithms (SVD) for the minimum singular value, unlike for Jamet’s angle, are well established and readily available. The Minimum Singular Value vs. Angle Conditions The minimum singular value of the edge shape matrix can be computed and used as an a priori algebraic measure of the interpolation error; alternatively, −2 can be replaced with tr (EE T )−1 . At the same time, given that σmin σmin characterizes the level of linear independence of the element edges and the overall “flatness” of the element, geometric implications of the minimum singular value condition are worth investigating. The following proposition shows that asymptotically the singular value criterion is equally or less restrictive than criteria based on solid angles. Proposition 8. Let {Ki }∞ i=1 be a sequence of tetrahedra with their diameters hi tending to zero, and let Ei be the edge shape matrix (3.176) of Ki . Then, if the minimum singular value condition is violated, i.e. if σmin (Ei ) → 0 as i → ∞, then there exists a subsequence of {Ki } for which all solid (trihedral) angles tend to either zero or 2π. Proof. As before, without loss of generality, each tetrahedron Ki can be assumed to have one of its nodes at the origin of a Cartesian system and to be rotated to have the minimum eigenvector of EE T run along the z axis. Let S be the unit sphere in R3 . To each tetrahedron Ki in the sequence (i) (i) there corresponds a point Pi ≡ (e1 , . . . , e6 ) ∈ S 6 representing the six unit
174
3 The Finite Element Method (i)
edge vectors el of Ki . Since S 6 is compact, one can select a subsequence of {Ki }, again denoted {Ki }, with the respective points Pi converging to a point (∞) (∞) P∞ ≡ (e1 , . . . , e6 ) ∈ S 6 . Since (i) 2 el · zˆ , (3.194) σmin (Ei ) = 1≤l≤6 (∞)
all six unit vectors el must lie in the xy-plane, and consequently the trihedral angle formed by any three of these vectors is zero or 2π. Since the trihedral angles depend continuously on Pi , the proposition follows. Remark 10. If a solid angle tends to zero, it does not necessarily imply that the minimum singular value does, too. A counterexample is the same as in Remark 9. A valid asymptotic condition is for the maximum solid angle to be bounded away from 2π. Indeed, if this condition were violated, the three edges forming the largest trihedral angle would tend to three distinct coplanar vectors. Hence all six edges would in the limit be coplanar, which corresponds to a zero singular value. M. Kriˇzek [Kˇ92] introduced a sufficient convergence condition requiring that all dihedral angles, as well as all face angles, be bounded away from π. The Proposition below shows that the minimum singular value criterion is equally or less restrictive than the Kˇr´ıˇzek condition. Proposition 9. Let γdj (j = 1, 2, . . ., 6) be the dihedral angles of a tetrahedron K and γfβl (l = 1, 2, 3; 1 ≤ β ≤ 4) be the angles of each triangular face β. Let γd0 be the angle with the maximum sine of all dihedral angles: sin γd0 = max(sin γdj ). Similarly, for each face β, let sin γfβ0 be the maximum of all sin γfβl for face β. Finally, let sin γf 0 be the minimum of sin γfβ0 over all faces β; i.e. sin γf 0 = min max sin γfβl 1≤β≤4 1≤l≤3
Then σmin (E(K)) ≥
12 2 γd0 sin γf 0 sin 3 2
(3.195)
Proof. Consider the two faces forming the dihedral angle γd0 with the maximum sine of all sin γdj . Let one of these faces lie in the xz-plane and let their common edge be on the z axis, with one node at the origin as shown in Fig. 3.42. Further, consider an arbitrary unit vector v = x ˆ sin θ cos φ + yˆ sin θ sin φ + zˆ cos θ in R3 , and let v1 and v2 be its projections on faces (1) and (2), respectively (Fig. 3.42). Then
3.14 Special Topic: Element Shape and Approximation Accuracy
175
Fig. 3.42. Tetrahedral nodes and critical angles. 1, 2, 3 are the nodes of face (1); 1, 4, 2 are the nodes of face (2).
v12 = sin2 θ cos2 φ + cos2 θ,
v22 = sin2 θ cos2 (φ − γd0 ) + cos2 θ
Further projecting v1 and v2 on each of the three edges of the respective faces (1) and (2) and using expression (3.187) for the minimum singular value of the edge shape matrix of a triangle, one obtains: 2
1≤j≤5
1 (1) (1) (1) 2 vej ≥ (sin2 θ cos2 φ + cos2 θ) (sin2 γf 1 + sin2 γf 2 + sin2 γf 3 ) 3
1 (2) (2) (2) + (sin2 θ cos2 (φ − γd0 ) + cos2 θ) (sin2 γf 1 + sin2 γf 2 + sin2 γf 3 ) 3 1 1 (1) (2) ≥ (sin2 θ cos2 φ + cos2 θ) sin2 γf 0 + (sin2 θ cos2 (φ−γd0 ) + cos2 θ) sin2 γf 0 3 3 ! 1 sin2 γf 0 ≥ sin2 θ(cos2 φ + cos2 (φ − γd0 )) + 2 cos2 θ 3 1 2 γd0 γd0 + 2 cos2 θ sin2 γf 0 ≥ sin2 sin2 γf 0 ≥ sin2 θ · 2 sin2 2 3 3 2 (The factor of two in the left hand side is due to the fact that the projection on edge 1-2 is counted twice in the right hand side. Summation over 1 ≤ j ≤ 5 excludes edge 3-4.)
176
3 The Finite Element Method
Conversely, let the Kˇr´ıˇzek condition be violated for some sequence of tetrahedra. Suppose first that a dihedral angle tends to π in that sequence. Then all six edges tend to positions in one fixed plane (after a possible rotation of each tetrahedron in the sequence). The edge projections of a unit vector perpendicular to that plane will tend to zero, and so will the minimum singular value of the edge shape matrix. Similarly, if one of the face angles tends to π, then all the edges of that face tend to positions on one straight line, and consequently all six edges again tend to positions in one plane, and σmin (E(Ki )) → 0. It follows that the minimum singular value and Kˇr´ıˇzek conditions are equivalent as asymptotic criteria of convergence of piecewise-linear interpolation on a family of tetrahedral meshes. The Minimum Singular Value vs. Trihedral Volume Consider first three unit edge vectors corresponding to a common tetrahedral node. There is a 3 × 3 submatrix E3 of E associated in the obvious way with these three edges. The volume of the parallelepiped based on the three unit vectors is (3.196) V3 = |detE3 | Both σmin (E3 ) and V3 characterize the level of linear independence of the three unit vectors, suggesting a connection between these two measures. Since the product of the eigenvalues is equal to the determinant, and the sum of the eigenvalues is equal to the trace, one has 2
[σ1 (E3 ) σ2 (E3 ) σ3 (E3 )]
= λ1 (E3T E3 ) λ2 (E3T E3 ) λ3 (E3T E3 )
= det(E3T E3 ) = det2 (E3 ) = V32 that is, σ1 (E3 ) σ2 (E3 ) σ3 (E3 ) = V3
(3.197)
Similarly, σ12 (E3 ) + σ22 (E3 ) + σ32 (E3 ) = λ1 (E3T E3 ) + λ2 (E3T E3 ) + λ3 (E3T E3 ) = tr E3T E3 = 1 + 1 + 1 = 3
(3.198)
From (3.198), one immediately obtains 2 1 ≤ σmax (E3 ) ≤ 3
and therefore it follows from (3.197), with the convention σmax = σ1 ≥ σ2 ≥ σ3 = σmin , that 4 2 (E3 ) ≤ σ12 (E3 ) σ22 (E3 ) ≤ σ12 (E3 ) σ22 (E3 ) σ32 (E3 ) = V32 ≤ 9σmin (E3 ) σmin
Hence
3.14 Special Topic: Element Shape and Approximation Accuracy
177
1 V3 ≤ σmin (E3 ) ≤ V32 (3.199) 3 The right inequality indicates that σmin and V3 could be of different “orders of magnitude”. Examples given in [TP98] demonstrate that the inequalities in (3.199) cannot be asymptotically improved. The maximum “trihedral volume” V3 based on three unit edge vectors57 may serve as a sufficient convergence condition for FE interpolation. However, due to a “nonlinear” relationship (3.199) between V3 and σmin , volume V3 is expected to be a less accurate a priori error measure than σmin .
Necessity of the minimum singular value condition There are several, and not equivalent, definitions of a shape condition being “essential” for the convergence of FE approximation. These definitions can be subdivided into the following broad categories: (a): if a shape condition is violated, the interpolation error may fail to tend to zero for some families {Ki } of elements (of a given type) with hi = diam(Ki ) → 0 and for some admissible functions; (b): if a shape condition is violated for any family of elements {Ki } of a given type, the interpolation error will not tend to zero for some admissible functions; (c),(d): same as (a) and (b), respectively, but for the error of the numerical solution (the Galerkin projection) instead of the interpolation error. Clearly, (b) is stronger than (a). Categories (c)–(d) are much more difficult to establish than (a)–(b). For first order triangular elements the minimum and the maximum angle conditions are both “essential” in the sense of (a), but only the maximum angle condition (equivalent in this case to the minimum singular value criterion) is “essential” in the sense of (b). M. Kˇr´ıˇzek [Kˇ92] proved that his condition is essential in the (a)-sense. Babuka and Aziz [BA76] showed that the maximum angle condition for triangles is essential in the (c) sense. It is easy to demonstrate that the minimum singular value condition is essential in the (a) sense; in fact, either of the two examples given by Kˇr´ıˇzek [Kˇ 92] suffices for this (the minimum singular value condition is violated, and there is no convergence). Establishing the necessity of the minimum singular value condition in a stronger (b) sense is more difficult. To this end, we need a definition that allows for freedom of solid rotation and translation of tetrahedral elements.
57
Strictly speaking, the maximum should be taken over all triples of edges, not necessarily having a common node.
178
3 The Finite Element Method
Definition 8. For a given tetrahedron K, the equivalence class of tetrahedra ˆ obtained from K by rigid rotations and/or translations is denoted with K. ˆ by Any energy norm uE,K on K is extended to the equivalence class K ˆ uE,K,Ω = sup{uE,K , K ∈ K, ˆ
K ⊂ Ω}
(3.200)
The necessity of the minimum singular value condition in the (b)-sense can then be stated as follows. Proposition 10. Let {Ki }∞ i=1 be an arbitrary sequence of tetrahedral elements such that hi ≡ diam(Ki ) → 0 and σmin (E(Ki )) → 0 as i → ∞. In addition, assume that the ratio of the maximum edge hi ≡ lmax (Ki ) to the minimum edge lmin (Ki ) is uniformly bounded on {Ki }∞ i=1 . Then there exists a function ¯ for which the H 1 -error of linear interpolation tends to infinity: u ∈ C 2 (Ω) Π1 (Ki )u − uH 1 ,Kˆ i ,Ω → ∞
(3.201)
Proof. The starting point is exactly as in the proofs of Proposition 7 and Proposition 8. Since arbitrary translations and rotations are allowed by Definition 8 in the norm used in (3.200), the minimum eigenvector of EE T may be assumed to run along the z-axis. Then, for the elements in the sequence, all edges will tend to the xy-plane. Hence one can select a subsequence of elements, again denoted as {Ki }, (i) (i) (i) (i) with their nodes r1 , r2 , r3 , r4 converging to four points r1−4 in the (i) xy-plane, with r1 = r1 = 0 for all i. Due to the assumed boundedness of lmax /lmin , the four points r1−4 must be distinct. Consider first the case when no three of the points rj ≡ (xj , yj ) (j = 1, 2, 3, 4) lie on a straight line. Introduce a Cartesian system with point r1 at the origin, point r2 on the x axis at (x2 , 0), point r3 at (x3 , y3 ), point r4 at (i) (i) (i) (i) (x4 , y4 ), and points rj ≡ (xj , yj , zj ). Since by assumption points r1 , r2 , r3 do not lie on the same line, y3 = 0
(3.202)
For each Ki , there exists a quadratic function of x, y (i)
uquadr (a(i) ; x, y) =
1 (i) 2 (i) (i) a x + a2 xy + a3 y 2 2 1 (i)
(i)
(3.203)
(i)
with a coefficient vector a(i) = (a1 , a2 , a3 )T such that z (i) z (i) 2
(3.204)
1 (Q(i) )−1 z (i) z (i) 2
(3.205)
(i)
uquadr (a(i) ; x, y) = Indeed, the suitable a(i) is given by a(i) =
3.14 Special Topic: Element Shape and Approximation Accuracy
179
where the matrix ⎛ Q(i) =
1 ⎜ 12 ⎝2 1 2
(i)2
x2 (i)2 x3 (i)2 x4
(i) (i)
1 2 1 2 1 2
x2 y2 (i) (i) x3 y3 (i) (i) x4 y4
⎞ (i)2 y2 (i)2 ⎟ y3 ⎠ (i)2 y4
can easily be verified to be nonsingular if no three points r1−4 lie on one straight line. Moreover, since Q(i) is a continuous function of coordi(i) nates xj , (Q(i) )−1 exists and is uniformly bounded for the sequence58 and limi→∞ (Q(i) )−1 = Q−1 exists. Therefore the sequence of coefficient vectors a(i) defined by (3.205) is bounded, and one can select a converging subsequence (∞) a(i) → a(∞) , with the corresponding function uquadr = uquadr (a(∞) ; x, y). (i)
According to (3.204), the coefficients of uquadr are chosen in such a way that its linear interpolant over Ki is simply (i)
(i)
ulin ≡ Π1 (Ki ) uquadr =
z (i) z (i) 2
Therefore the z-derivative of the interpolation error (i)
(i)
∂z (ulin − uquadr ) =
1 z (i) 2
→ ∞
because z (i) 2 → 0 as σmin (E(Ki )) → 0. This implies that the interpola(∞) tion error for the limiting function uquadr also tends to infinity, despite the boundedness of the seminorm |uquadr |2,∞,Ki = a(∞) 1 . If three of the points r1−4 lie on one straight line, the corresponding face is degenerate, and the proof can be essentially repeated in two dimensions in the plane of this face. 3.14.4 Condition Number and Approximation Practical experience has shown (see e.g. F.-X. Zgainski et al. [ZMC+ 97]) that the condition number of the FE stiffness matrix is a useful measure of mesh quality. Since the condition number strongly affects the performance of iterative systems solvers, it is not surprising that slow convergence of the solvers and poor accuracy of the solution (due to poor quality of the FE mesh) typically go hand in hand. Based on the results of this section, it can be argued that poor approximation and poor conditioning of the system are related to each other indirectly: both of these quantities stem from the maximum eigenvalue of the global stiffness matrix. This connection is schematically illustrated in Fig. 3.43. (The 58
With a possible exception of a finite set of indices.
180
3 The Finite Element Method
minimum eigenvalue has no bearing on interpolation accuracy and can typically be viewed as a fixed parameter associated with the size of the computational domain.59 )
Fig. 3.43. A large eigenvalue of the FE stiffness matrix is a common source of both ill-conditioning of the FE system and poor accuracy of the solution.
3.14.5 Discussion of Algebraic and Geometric a priori Estimates We have explored the dual algebraic/geometric nature of finite element interpolation errors. From the algebraic perspective, the error was shown to be governed by the maximum eigenvalue of the FE stiffness matrix. When the maximum eigenvalue estimate is applied to triangular and tetrahedral elements, several known geometric conditions and several nonstandard results are obtained. For triangular elements in particular, Zl´ amal’s minimum angle condition and the Synge–Babuˇska–Aziz maximum angle condition are recovered. For tetrahedral elements, the maximum eigenvalue estimate leads to an interesting result. The shape of tetrahedral elements turns out to be accurately represented, in the FE context, by the minimum singular value of the “edge shape matrix”. This singular value characterizes, on the one hand, the “flatness” of the element and, on the other hand, the accuracy of the FE interpolation. There are several links between the minimum singular value and some geometric parameters of the tetrahedron, but the minimum singular value is, in some well-defined sense, one of the most precise measures. (Jamet’s condition is another one.) Due to its generality, the maximum eigenvalue condition can be applied in cases where no other shape criteria are immediately available. For example, 59
Strictly speaking, the ratio of maximum/minimum eigenvalues is in general a suitable measure of conditioning for symmetric positive definite matrices only. This case is implicitly assumed, to avoid further complications.
3.15 Special Topic: Generalized FEM
181
anisotropy of material parameters should result, intuitively, in some “scaling” of the coordinate axes before any geometric accuracy criteria can be considered. In contrast, the maximum eigenvalue criterion accommodates anisotropy automatically, since material parameters are built into the stiffness matrix. This criterion can be applied to elements of any shape and order but is not without limitations. First, it provides a priori estimates only; it remains to be seen whether similar ideas can be used to enhance a posteriori estimates critical for adaptive mesh refinement (Section 3.13). Second, the maximum eigenvalue criterion is a sufficient but not generally a necessary condition; it does not guarantee the best error estimate. This is well illustrated by two cases considered in this section: (a) for conservative fields on Whitney edge elements, the result (expressed via the minimum singular value of the edge shape matrix) is optimal; (b) at the same time, for triangular node elements the maximum eigenvalue criterion leads to Zl´ amal’s minimum angle condition rather than to the more accurate Synge–Babuˇska–Aziz maximum angle condition. The theoretical results provide general and easy-to-implement a priori criteria of FE accuracy. The computational overhead in the overall FE procedure is negligible. For tetrahedral elements in particular, the precise characterization of shape via the minimum singular value of the element “edge shape matrix” can be recommended for engineering practice. Experimental results reported by M. Dorica & D.D. Giannacopoulos [DG05] and by A. Plaks & myself [TP98] support this conclusion.
3.15 Special Topic: Generalized FEM 3.15.1 Description of the Method A detailed explanation and analysis of Generalized FEM proposed originally by I. Babuˇska & J.M. Melenk [MB96, BM97] is widely available (e.g. T. Strouboulis et al. [SBC00]). Of all interesting features of GFEM, the most salient one is its ability to employ a variety of special non-polynomial approximating functions. In particular, jumps of the normal derivatives of the potential at interface boundaries can be represented by special basis functions. Strouboulis et al. [SBC00] present an extensive set of application examples with special functions for material inclusions in stress analysis. Babuˇska et al. [BCO94] applied Generalized FEM (before the method was referred to as such) to problems with “rough” coefficients – discontinuities at material interfaces. A. Plaks et al. [PTFY03] implemented GFEM for problems with magnetized particles. In GFEM the computational domain Ω is covered with overlapping subdomains (“patches”) Ω(i) , and different local approximations are merged by npatches on this system of patches. More precisely, Partition of Unity (PU) {Ωi }i=1 (i) a set of PU functions {ϕ }, 1 ≤ i ≤ npatches is constructed to satisfy
182
3 The Finite Element Method
npatches
ϕ(i) ≡ 1
in Ω,
supp ϕ(i) = Ω(i)
(3.206)
i=1
That is, each function ϕ(i) is associated with the respective patch Ω(i) and vanishes outside that patch. Then the global solution u can be decomposed into its “patch components” u(i)
npatches
u = u
i=1
npatches
ϕ(i) =
npatches
uϕ(i) =
i=1
u(i) , with u(i) ≡ uϕ(i) (3.207)
i=1
Fig. 3.44 gives a simple 1D illustration of the PU principle, with just two overlapping patches. A seamless transition from the solution in the first patch to the solution in the second patch is achieved by multiplying these individual solutions by the weighting functions ϕ(1) and ϕ(2) , respectively. As a reference point moves from left to right, the weight of the first solution gradually decreases, while simultaneously the weight of the second solution increases.
Fig. 3.44. The idea of partition of unity illustrated in 1D: weighting functions ϕ(1) and ϕ(2) are used to merge two solutions in the overlapping subdomains. The sum of the weighting functions is unity everywhere. (Reprinted by permission from [Tsu06] c 2006 Elsevier.)
Decomposition (3.207) is valid for the exact solution but can equally well be used for assembling a global approximate solution from the local ones. Suppose that locally, within each patch Ω(i) , the exact solution u can be (i) approximated by a linear combination uh of some approximating functions (i) gα : (i) (i) uh = c(i) (3.208) α gα α
3.15 Special Topic: Generalized FEM
183
(i)
cα being some (real- or complex-valued) coefficients. The final system of (i) approximating functions ψα is built with ϕ(i) as weight functions: ψα(i) = gα(i) ϕ(i)
(3.209)
The global approximation error is guaranteed to be bounded by the local (patch-wise) errors [BM97], [SBC00], [BBO03], with rigorously provable estimates of the global error in terms of local errors and the norms of the PU functions φ. 3.15.2 Trade-offs The multiplication by ϕ(i) in (3.209) guarantees seamless merging of patchwise approximations, with rigorously provable estimates of the global error in terms of local errors and the norms of the PU functions ϕ [BM97]. On the negative side, however, this multiplication complicates the set of approximating functions and tends to make it more ill-conditioned (in some cases even linearly dependent, see [BM97]). For positive definite problems, the linear dependence can be tolerated because the resultant algebraic system remains consistent and positive-semidefinite and can be handled by clever linear algebra algorithms (see T. Strouboulis et al. [SBC00] for further information). The “no free lunch” clich´e applies fully to GFEM. While the rigid requirements on mesh structure and the approximating functions are greatly relaxed, the computational burden is shifted toward numerical quadratures that need to be computed in the Galerkin method over the intersections of overlapping patches. This complex task can be accomplished in general only by adaptive numerical integration. The efficiency of this integration is critical for the overall performance of the algorithm. In addition, GFEM-PU may lead to a combinatorial increase in the number of degrees of freedom. For illustration, consider a regular hexahedral mesh where a “patch” is defined as a set of eight hexahedra around a common node. In the presence of material boundaries, it is sensible to replace the usual eight trilinear basis functions with eight special functions satisfying the derivative jump condition at the interface (see also Chapter 4). In GFEMPU, each of these special functions gets multiplied by the “shape function” ϕ of the patch. As each hexahedral element of the mesh is an intersection of eight patches (centered at its eight respective nodes) and each of these patches contributes eight approximating functions, the stiffness matrix for elements close to material interfaces is 64 × 64 instead of the usual 8 × 8. For all of the above reasons, alternative approaches may be worth exploring. One such approach that generalizes finite difference, rather than finite element, analysis is discussed in Chapter 4.
184
3 The Finite Element Method
3.16 Summary and Further Reading The Finite Element Method is arguably the most powerful computational tool ever invented. Its solid variational foundation makes the method remarkably robust – often beyond the areas where a complete mathematical analysis is available. FEM is well established in traditional branches of engineering such as stress analysis, heat transfer, electromagnetic fields in machines and microwave circuits, etc. However, FEM has not yet been taken full advantage of in some areas of nanoscale simulation. Examples include nano-photonics and nanooptics – more specifically, plasmonic field enhancement by particle clusters, scattering of light by optical tips in near-field microscopy, and wave propagation in photonic crystal devices. These and other cases presented in Chapter 7 will hopefully stimulate further applications of FEM in nanoscale science and technology. The present chapter explains the fundamentals of FEM (the underlying variational principles, finite elements and spaces, FE matrices, algorithmic implementation) and provides an overview of state-of-the-art techniques of FE analysis (adaptive mesh refinement and multigrid algorithms). The chapter also covers more advanced topics: edge elements, a priori estimates of numerical accuracy as a function of element shape, and Generalized FEM. Adaptive hp-refinement aims at the most effective use of the computational resources by constructing quasi-optimal meshes: the density of elements is higher in regions where the solution is less smooth and changes more rapidly; the density is lower in regions of smooth variation of the solution. Adaptive techniques are now an integral part of the commercial FE packages. The same is true for edge elements in electromagnetic applications: the gap between the elegant mathematical theory and practical utility was bridged in the 1990s, especially after it became clear that many families of edge elements, in contrast with the nodal ones, do not produce nonphysical eigensolutions known as “spurious modes”. Generalized FEM occupies a niche in practical applications. This will most likely continue to be the case, although the niche may grow to some extent. The power of GFEM lies in its ability to use a wide selection of approximations not limited to element-wise polynomials as in the standard FEM. This could be a great advantage in many cases where particular features of the physical field or potential, such as singularities, boundary layers, dipole-like behavior, etc., are known a priori and can therefore be accurately represented by special approximating functions. However, there is a substantial price to be paid for this advantage: complex numerical quadratures, increased number of unknowns, and possible ill-conditioning or in some cases even linear dependence of the system of approximating functions. The special section on a priori error estimates in this chapter examines the links between algebraic and geometric accuracy measures. While it is well known that “flat” elements provide poor numerical approximation of the
3.16 Summary and Further Reading
185
solution, it is argued in Section 3.14 that the “true source” of the error is of algebraic nature. This source can be traced to the maximum eigenvalue of the FE stiffness matrix and, in the case of triangular and tetrahedral elements, to the minimum singular value of the “edge shape matrix”. It is shown that the latter measure is, in some sense, a precise one, and its connection with various geometric parameters is examined. The reader who would like to learn more about Finite Element analysis is in an enviable position. There are many excellent books and papers on all aspects of FEM, written from the engineering, mathematical and computational perspectives. Researchers and developers of engineering applications cannot go wrong with the books by O.C. Zienkiewicz et al. [ZTZ05, ZT05]. In engineering electromagnetics, P.P. Silvester’s group was the first to apply finite elements; his book with R.L. Ferrari [SF90] is still valuable. J. Jin’s more recent monograph [Jin02] is a very good source of information on FEM in electromagnetics and includes, in addition to standard subjects, chapters on vector finite elements, absorbing boundary conditions, finite element – boundary integral methods and on time-domain analysis. The book by J.L. Volakis et al. [VCK98] also covers vector elements, as well as applications to radiation and scattering and hybrid finite element – boundary integral methods. Several books are focused on the applications of FEM to low-frequency electromagnetic fields in electric machines and devices: J.P. A. Bastos & N. Sadowski [aPABS03], S. Salon & M. V.K. Chari [SC99], G. Meunier (ed.) [Meu07]. On the mathematical side, there are several magnificent books as well. The works by G. Strang & G.J. Fix [SF73] and I. Babuˇska, A.K. Aziz & B. Szab´ o [BA72, SB91] are classical. The main reference on the mathematical treatment of FEM in electromagnetism is P. Monk’s monograph [Mon03]. The monograph by L. Demkowicz [Dem06] bridges mathematical theory and applications, with the emphasis on hp-adaptivity. The book deals with elliptic and wave problems and includes 1D and 2D codes developed by Demkowicz & co-workers. Finally, A. Bossavit’s book [Bos98] is in a category of its own due to its unconventional approach and style. The focus of this book is on the mathematical principles and structures underlying FE methods in electromagnetism – in particular, concepts of variational analysis, differential geometry and algebraic topology. While the content is mostly mathematical, Bossavit’s style of writing makes the material accessible to non-experts (still, the reader will need enough patience and perseverance to understand the book). In the coming years, I look forward to seeing further applications of FEM in the simulation of micro- and nanoscale systems. Electromagnetic field analysis in optics and photonics seems particularly interesting, as it may well lead to the development of new devices and materials with completely unconventional properties and behavior (Chapter 7).
186
3 The Finite Element Method
3.17 Appendix: Generalized Curl and Divergence This section is an extension of Appendix 6.15 (p. 343) on generalized functions (distributions) and their derivatives. The conventional representation of the divergence and curl operators – say, in Cartesian coordinates – requires differentiability: ∇·A =
∂Ay ∂Az ∂Ax + + ; ∂x ∂y ∂z
(∇ × A)x =
∂Ay ∂Az − ; ∂y ∂z
etc.
However, derivatives in these expressions can be treated in the generalized sense of distributions (see Appendix 6.15), thereby extending the notion of divergence and curl to functions that are not differentiable in the standard sense of differential calculus. Example 12. The A field with a step-like x-component, Ax = 0 for x < 0 and Ax = 1 for x ≥ 0, and zero y- and z-components, has generalized divergence ∇ · A = δ(x). For the electric field, this Dirac-delta divergence corresponds to a surface charge. Example 13. The A field with a step-like z-component, Az = 0 for y < 0 and Az = 1 for y ≥ 0, and zero x- and y-components, has generalized curl ∇ × A = δ(y)ˆ x. For the magnetic field, this Dirac-delta curl corresponds to a surface current. Instead of appealing to the Cartesian representation of divergence and curl with generalized derivatives, one can give an equivalent but coordinate-free definition via integration-by-parts identities. For divergence, (∇ · A, φ) = − (A, ∇φ)
(3.210)
where the inner product is that of L2 . This identity, in the regular calculus sense, follows from the calculus formula ∇ · (Aφ) = φ∇ · A + A · ∇φ if fields A, φ are continuously differentiable and φ has a compact support. (The latter requirement ensures that the surface integral term in the integration by parts vanishes). One can then extend the notion of divergence to nondifferentiable fields and define generalized divergence as the linear functional ∇ · A, φ ≡ − (A, ∇φ)
(3.211)
over smooth scalar functions φ with a compact support. Equation (3.210) ensures that the extended definition is consistent with the regular calculus version of divergence as long as the vector field is smooth. For a vector field that has a jump of its normal component across a surface S, but is otherwise smooth, the generalized divergence is
3.17 Appendix: Generalized Curl and Divergence
187
∇ · A, φ ≡ − (A, ∇φ) =
[An ]φ dS + ({∇ · A}, φ) S
where integration by parts was applied. Here [An ] = An+ −An− is the jump of the normal component of the vector field across S (n+ referring to the region into which the normal to S is pointing). Thus ∇ · A = [An ] δS + {∇ · A}
(3.212)
where generalized divergence is implied in the left hand side and divergence in its regular calculus sense is specified by the curly brackets in the right hand side. This is V.S. Vladimirov’s notation; see Appendix 6.15 on p. 343 and also footnote 18 on p. 320 and 44 on p. 347. The curl operator is generalized in a similar fashion: (∇ × A, B) = (A, ∇ × B)
(3.213)
where the inner product is again that of L2 . This identity, in the regular calculus sense, follows from (3.128) if fields A, B are continuously differentiable and B has a compact support. Generalized curl is defined as ∇ × A, B ≡ (A, ∇ × B)
(3.214)
over smooth vector functions B with a compact support. For vector fields with a discontinuous tangential component across a surface S, but smooth otherwise, the generalized curl is ∇ × A = [A × n ˆ ] δS + {∇ × A}
(3.215)
This formula is analogous, and obtained in a similar way, to expression (3.212) for generalized divergence. A key observation in the context of edge elements is that a jump of the tangential component of a vector field across a surface leads to the Dirac-delta term for the generalized curl on this surface; Example 13 on p. 186 is a simple but representative illustration of this property that is not difficult to verify in general. The tangential component is continuous if and only if the generalized curl exists as a regular function, not only a distribution.
4 Flexible Local Approximation MEthods (FLAME)
This chapter is based to a large extent upon my papers [Tsu05a, Tsu05b, Tsu06].
4.1 A Preview Although the Finite Element Method (FEM) described in Chapter 3 is one of the most powerful and general analysis techniques, in some cases the complicated FE meshes, data structures and solvers can become computationally expensive or even impractical. Finite Difference (FD) algorithms (Chapter 2), on the other hand, operate on geometrically simple grids and the data structures associated with them are much simpler than those of FEM. The system solvers also tend to be more efficient. The downside, in comparison with FEM, is relatively poor numerical accuracy at material interfaces not conforming to the simple FD grid. This leads to a legitimate question: given a regular grid not geometrically conforming to material interfaces, what is – in some sense – “the best” one can do? The answer, in general, is not the classical FD schemes. This chapter argues in favor of a new FD calculus referred to by the acronym “FLAME”: Flexible Local Approximation MEthods. The word “Flexible” implies that any desired approximation of the solution (exponentials, spherical harmonics, plane waves, generic or special polynomials, etc.) can be incorporated directly into the FD scheme. This is in contrast with Taylor polynomial expansions that form the basis of standard FD. In FLAME, approximation is always treated as local, with the intention to represent local features of the solution that in many cases may qualitatively be known a priori (for example, the behavior of the potential near a material interface; see also Section 4.5 on p. 219). As a preview, consider a simple 2D test problem: a cylindrical magnetic particle, with relative permeability µp = 100, immersed in a uniform external
190
4 Flexible Local Approximation MEthods (FLAME)
field. A contour plot and a grayscale plot of the magnetic scalar potential u (the magnetic field H = −∇u) are shown in Fig. 4.1 for illustration.1
Fig. 4.1. A contour plot and a grayscale plot of the magnetostatic potential for a cylindrical particle in a uniform external field.
Fig. 4.2 compares two meshes that give about the same level of numerical accuracy for this problem. The Finite Element mesh has 31,537 nodes, 62,592 second order triangular elements and 125,665 degrees of freedom (d.o.f.); the relative error in the potential at the nodes is 2.07 × 10−8 . The FLAME grid has 900 d.o.f. (30 × 30), and the relative error in the potential at the nodes is 2.77 × 10−8 if 9-point (3 × 3) stencils are used. The high accuracy of FLAME schemes is due to the approximating functions employed in FLAME. For the particle problem, these functions are cylindrical harmonics that represent the behavior of the potential in the vicinity of the particle much better than the Taylor polynomials do in standard FD. This chapter explains how FLAME schemes are constructed. First, Section 4.2 provides an introduction to FLAME and highlights the main ideas behind it. Some of these ideas, such as Trefftz basis functions in the finite-difference context, multivalued approximation, trade-off between conformity and flexibility of approximation, are nonstandard. As a preliminary example, FLAME is developed for the (trivial) case of the 1D Laplace equation in Section 4.2.6 to fix ideas. General construction of Trefftz–FLAME is presented in Section 4.3, where case studies in 1D, 2D and 3D are provided. In Trefftz–FLAME, the approximating functions are chosen as local solutions of the underlying differential equation. In a number of practically interesting cases, the local solutions are not difficult to derive analytically; in addition to this chapter, computational examples are given 1
The electrostatic problem for a dielectric particle is completely analogous.
4.2 Perspectives on Generalized FD Schemes
191
Fig. 4.2. Two meshes yielding about the same level of accuracy for the particle problem. The FE mesh has 31,537 nodes, 62,592 second order triangular elements and 125,665 degrees of freedom. The FLAME grid has 900 degrees of freedom. c (Reprinted by permission from [Tsu06] 2006 Elsevier.)
in Chapter 6 (electrostatic interactions of colloidal particles) and Chapter 7 (electromagnetic field enhancement by plasmonic particles and waves in photonic crystals). Section 4.5 reviews existing classes of methods with nontraditional approximation: Generalized FEM (GFEM), variational homogenization, pseudospectral methods, and others. FLAME borrows some features of these methods (most notably, flexible approximation from Generalized FEM) but is not a particular case of any of them. Some existing methods turn out to be particular cases of FLAME: the exact schemes by R.E. Mickens [Mic94, Mic00]; the Hadley schemes for electromagnetic wave propagation [Had02a]; the “Measured Equation of Invariance” [MPC+ 94] by K.K. Mei et al. The chapter concludes with a discussion (Section 4.6) and appendices on the variaitonal version of FLAME, the 9-point 2D FLAME for the wave equation, and the Fr´echet derivative.
4.2 Perspectives on Generalized FD Schemes 4.2.1 Perspective #1: Basis Functions Not Limited to Polynomials Taylor polynomials are generic and may be the best option when no a priori information about the solution is available. When the local behavior of the solution is known, more effective approximations can usually be generated. For example, if the solution exhibits boundary layers, wave-like behavior, dipole components, etc., in certain regions, as schematically shown in
192
4 Flexible Local Approximation MEthods (FLAME)
Fig. 4.3, then it may be appropriate to use exponentials, sinusoids, dipole harmonics, and so on, as approximating functions in the respective regions. The subsequent sections of this chapter show how this can be accomplished in a generalized finite-difference framework.
Fig. 4.3. Physical fields or potentials often have salient local features: boundary layers, wave-like behavior, peaks (left picture), dipole components (right picture), etc. Numerical accuracy can be improved significantly if such local behavior is taken into account.
4.2.2 Perspective #2: Approximating the Solution, Not the Equation In classic Taylor-based FD schemes, one approximates the underlying differential equation – i.e. the operator and the right hand side. For instance, on a three-point stencil in 1D one can expect a second order approximation of the Poisson equation. There is, however, substantial redundancy built into this approach. Indeed, the scheme covers all sufficiently smooth functions for which the Taylor approximation is valid. Yet it is only the solution of the problem that is of direct interest; it is, in a sense, wasteful to approximate other functions. To highlight this point, imagine for a moment that the exact solution u∗ is known. It is then trivial to find a three-point scheme that is itself exact, e.g.: uk uk+1 uk−1 − 2 ∗ + ∗ = 0 u∗k−1 uk uk+1
(4.1)
It is easy to dismiss this example as frivolous, as it requires knowledge of the exact solution. The message, however, is that as more information about the
4.2 Perspectives on Generalized FD Schemes
193
solution is utilized, higher accuracy can be achieved; equation (4.1) is just an extreme example of this principle. One practical illustration is the use of harmonic polynomials to approximate harmonic functions (Sections 4.4.4, 4.4.5). More generally, the “Trefftz” version of FLAME calculus employs basis functions that satisfy the differential equation being solved. No effort is wasted on trying to approximate functions that do not satisfy the equation. This “Trefftz” approximation is purely local and therefore relatively easy to construct. 4.2.3 Perspective #3: Multivalued Approximation In FD analysis, interpolation between the nodes is usually viewed just as a postprocessing tool not inherent in the FD method itself. However, approximation between the nodes is in fact an integral part of the derivation of classical FD schemes. Indeed, this approximation involves Taylor expansions around grid nodes (Fig. 4.4). Each of these expansions “lives” in a neighborhood of its node. The disparate Taylor expansions coexist in the overlap region of two or more such neighborhoods. This is precisely the viewpoint
Fig. 4.4. Taylor approximations around two grid nodes coexist in the overlap area.
taken in FLAME, except that any desirable approximating functions are allowed rather than just the Taylor polynomials. Each of these approximations is purely local and valid in the vicinity of a given grid stencil; as in classic FD, two or more such approximations may coexist at any given point. The discrepancies between these approximations are expected to tend to zero if the method converges as the grid is refined. At the same time, these discrepancies may prove useful as an a posteriori error indicator in practical computation (J. Dai & I. Tsukerman [DT07]). 4.2.4 Perspective #4: Conformity vs. Flexibility The following schematic chart (Fig. 4.5) puts various methods into a “flexibility vs. conformity” perspective. “Conformity” is a common jargon term for
194
4 Flexible Local Approximation MEthods (FLAME)
(loosely speaking) a sufficient level of smoothness of the solution. More formally, in “fully conforming” methods the numerical solution belongs to the appropriate Sobolev space over the whole computational domain.2 Various methods shown in the chart are reviewed in Section 4.5 (p. 219).
Fig. 4.5. A schematic “conformity vs. flexibility” view of various numerical methods. One can gain flexibility of approximation by giving up conformity. This general trend is indicated by the dashed arrow. GFEM outperforms this trend, at a high computational and algorithmic cost. Classic FD schemes under perform. FLAME c schemes fill the existing void. (Reprinted by permission from [Tsu06] 2006 Elsevier.)
The dashed arrow in the figure shows the general trend: flexibility of approximation can be gained by giving up some conformity of the method. Two methods stand out of that trend: Generalized FEM (Section 3.15, p. 181) and classic FD (Chapter 2). GFEM outperforms the trend: it is fully conforming (i.e. operating in a globally defined subspace of the relevant Sobolev space) and yet allows any desirable approximating functions to be used. However, this advantage is achieved at a high computational and algorithmic cost. Classic FD schemes 2
In vector field problems, divergence-conforming and especially curl-conforming spaces H(div; Ω) and H(curl; Ω) are widely used; see A. Bossavit’s & P. Monk’s monographs [Bos98, Mon03].
4.2 Perspectives on Generalized FD Schemes
195
under perform relative to the general trend: they are fully nonconforming and yet make use only of local polynomial (i.e. Taylor) expansions. FLAME schemes fill the existing void in the upper-left corner of the chart: they are fully nonconforming and admit arbitrary approximations. Clearly, it would be somewhat simplistic to ask which side of this chart is “better”. No one would question the wonderful success of conventional FE analysis lying at the “conformal” end. However, the conformity requirements do impose significant limitations in a number of practical cases. This was understood early on in the development of FEM – hence the notion of “variational crimes” (G. Strang [Str72]), the Crouzeix–Raviart elements (M. Crouzeix & P.A. Raviart [CR73]), etc. The advantages of the nonconforming end of the spectrum are clear for problems with multiple moving particles, where finite element mesh generation may be inefficient or impractical. 4.2.5 Why Flexible Approximation? As already noted, in many physical problems some salient features of the solution are qualitatively known a priori. Such features include singularities at point sources, edge and corners; boundary layers; derivative jumps at material interfaces; strong dipole field components near polarized spherical particles; cusps of electronic wavefunctions at the nuclei; electrostatic double layers around colloidal particles – and countless other examples. Such “special” behavior of physical fields is arguably a rule rather than an exception. Clearly, taking this behavior into account in numerical simulation will tend to produce more accurate and physically meaningful results. The special features of the field are typically local, and in numerical modeling it is therefore desirable to employ various local approximations of the field. The focus of this chapter is precisely on “Flexible Local Approximation” and on methods capable of providing it – that is, employing a variety of approximating functions not limited to polynomials. One motivation for developing this class of methods is to minimize the notorious “staircase” effect at curved and slanted interface boundaries on regular Cartesian grids. In the spirit of “Flexible Local Approximation,” the behavior of the solution at the interfaces is represented algebraically, by suitable basis functions on simple grids, rather than geometrically on conforming meshes. More specifically, fields around spherical particles can be approximated by several spherical harmonics; fields scattered from cylinders by Bessel functions, and so on. Such analytical approximations are incorporated directly into the difference scheme. This approach can be contrasted with very well known, and very powerful, Finite Element (FE) methodology, where the geometric features of the problem are represented on complex conforming meshes. The flexibility of approximation in FEM is achieved through adaptive mesh refinement: changing
196
4 Flexible Local Approximation MEthods (FLAME)
the mesh size (h-refinement) or the order of approximation (p-refinement). Still, approximation remains piecewise-polynomial. FEM is indispensable in many problems where the geometries are complex and material parameters vary. In addition to mechanical, thermal and electromagnetic modeling of traditional devices and machines, FEM has recently penetrated new areas of macromolecular simulation. Molecular interface surfaces can be viewed as intersections of hundreds or thousands of spheres and consequently are geometrically extremely complex. These interfaces separate the interior of the molecule, that can be approximated by an equivalent relative dielectric constant on the order of 1 to 4, from the solvent that in “implicit” models is considered as a continuum with equivalent dielectric and Debye parameters ([BSS+ 01, GPN01, HN95, CF97, FEVM01, RAH01, Sim03, DTRS07], references therein, and Chapter 6). The computational cost of finite element macromolecular simulation can be enormous. N.A. Baker et al. [BSS+ 01] used a massively parallel supercomputer with 1152 processors to simulate cell structures with 88,000 to 1.25 million atoms; the Poisson–Boltzmann model was used (see Chapter 6). The computational overhead of mesh generation and matrix assembly in FEM is significant, and for geometrically simple problems FEM may not be competitive with Finite Difference (FD) schemes and other methods operating on simple Cartesian grids. One extreme example of geometric simplicity comes from molecular dynamics simulations, where charges or dipoles are typically considered in a cubic box with periodic boundary conditions. The Ewald algorithm (taking advantage of Fast Fourier Transforms) is then usually the method of choice (Chapter 5). Problems with multiple moving particles also call for development and application of new techniques. Generation of geometrically conforming FE meshes is obviously quite complicated or impractical when the particles move and their number is large (say, on the order of a hundred or more). Parallel adaptive Generalized FEM has been developed [GS00, GS02a, GS02b], but the procedure is quite complicated both algorithmically and computationally. Standard FD schemes would require unreasonably fine meshes to resolve the shapes of all particles. An alternative approach is to use two types of grid: spherical meshes around the particles and a global Cartesian grid [Fus92, DHM+ 04]. The electrostatic potential then has to be interpolated back and forth between the grids, which reduces the numerical accuracy. The celebrated Fast Multipole Method (FMM) has clear advantages for systems with a large number of known charges or dipoles in free space (or a homogeneous medium). For inhomogeneous media (e.g. a dielectric substrate, or finite size particles with dielectric or magnetic parameters different for those of free space) FMM can still be used as a fast matrix-vector multiplication algorithm embedded in an iterative process for the unknown distribution of volume sources. However, the benefits of FMM in this case are much less clear. An even stronger case in favor of difference schemes (as compared to FMM)
4.2 Perspectives on Generalized FD Schemes
197
can be made if the problem is nonlinear (for example, the Poisson–Boltzmann equation). FMM will remain outside the scope of this chapter. The proposed new FLAME schemes provide a practical alternative that is both uncomplicated and accurate (Section 4.3). In addition to multiparticle simulations, FLAME techniques can be applied to a variety of other problems. As a peculiar example, super high-order 3-point schemes are derived for the 1D Schr¨ odinger equation in Sections 4.4.6, 4.4.7 and for a 1D singular equation in Section 4.4.8. With the 20th -order 3-point scheme as an illustration, the solution of the harmonic oscillator problem is found almost to machine precision with 10–20 grid nodes. The system matrix remains tridiagonal. 4.2.6 A Preliminary Example: the 1D Laplace Equation The 1D Laplace equation is trivial and is used here only to provide the simplest possible example of the Trefftz–FLAME schemes. For convenience, consider a uniform grid with size h, choose a 3-point stencil and place the origin at the middle node of the stencil. The key step in Trefftz–FLAME schemes is to approximate the solution – locally, over the stencil – by a linear combination of basis functions satisfying the underlying differential equation. The 1D Laplace equation is so simple that the two independent local solutions ψ1 = 1;
ψ2 = x
also happen to be global solutions of the equation (disregarding the boundary conditions), but this circumstance is irrelevant for FLAME. The numerical solution over the stencil is uh = c1 ψ1 + c2 ψ2
(4.2)
In general, all the variables in this equation may be different for different grid stencils, although for the 1D Laplace equation c1,2 happen to be the same throughout the domain. In the future, if there is any possibility of confusion, the stencil number will be indicated with a superscript, but for now it is omitted for simplicity. We are looking for a difference scheme with some coefficient vector s ≡ (s1 , s2 , s3 )T ∈ R3 (s – mnemonic for “scheme”) that would relate the nodal values uh1,2,3 of the numerical solution on the stencil: s1 uh1 + s2 uh2 + s3 uh3 = 0
(4.3)
Since uh (4.2) contains only two independent parameters (c1,2 ), it is clear that the three nodal values must be linearly related and thus (4.3) must hold for some s. Finding a suitable coefficient vector s is easy, and we shall do so in a way that will be straightforward to generalize. The nodal values that figure in (4.3) are
198
4 Flexible Local Approximation MEthods (FLAME)
uh1 ≡ uh (x1 ) = c1 ψ1 (x1 ) + c2 ψ2 (x1 ) uh2 ≡ uh (x2 ) = c1 ψ1 (x2 ) + c2 ψ2 (x2 ) uh3 ≡ uh (x3 ) = c1 ψ1 (x3 ) + c2 ψ2 (x3 )
(4.4)
The matrix-vector form of equations (4.3) and (4.4) is sT uh = 0
(4.5)
uh = N c
(4.6)
and where uh = (uh1 , uh2 , uh3 )T is the R3 -vector of nodal values, c = (c1 , c2 )T is the R2 -vector of coefficients, and n is the 3 × 2 matrix of nodal values of the basis functions: ⎞ ⎛ ψ1 (x1 ) ψ2 (x1 ) (4.7) N = ⎝ψ1 (x2 ) ψ2 (x2 )⎠ ψ1 (x3 ) ψ2 (x3 ) Combining (4.5) and (4.6), one obtains sT N c = 0
(4.8)
For this identity to be valid for any c, we must have, from basic linear algebra, s ∈ Null N T
(4.9)
Let us spell this out for the 1D Laplace equation. With ψ1 = 1, ψ2 = x and the coordinates of the nodes (−h, 0, h), the (transposed) nodal matrix (4.7) is 1 1 1 T N = −h 0 h The Trefftz–FLAME difference scheme then is s = Null N T = (1, −2, 1)T (times an arbitrary coefficient) which coincides with the standard 3-point scheme for the Laplace equation. In the remainder of this chapter, we shall see that the definition (4.9) of the scheme has a great deal of generality and is applicable to a variety of equations (Section 4.3). First, however, we need to discuss a general setup for local, finite-difference-like, approximation.
4.3 Trefftz Schemes with Flexible Local Approximation 4.3.1 Overlapping Patches An important element of the setup, to be used in the remainder of this chapter, is a set of overlapping patches Ω(i) covering the computational domain Ω =
4.3 Trefftz Schemes with Flexible Local Approximation
199
∪Ω(i) , i = 1, 2, . . . n. This cover of the domain is the same as in Generalized FEM (see Sections 3.15, p. 181, and 4.5.2, p. 221); however, FLAME differs from GFEM in many critical respects as we shall see. The domain cover is needed to define a local, patch-wise, approximation of the solution. More precisely, within each patch Ω(i) we introduce a local approximation space Ψ(i) = span{ψα(i) , α = 1, 2, . . . , m(i)}
(4.10)
Note that no global approximation space will be considered. Instead, the following notion of multivalued approximation is introduced: For a given domain cover {∪Ω(i) } with corresponding local spaces Ψ(i) , a multivalued approximation uh {∪Ω(i) } of a given potential u is just a collection of patch-wise approximations: (i)
uh {∪Ω(i) } ≡ {uh ∈ Ψ(i) }
(4.11)
In regions where two or more patches overlap (Fig. 4.6), several local approximations coexist and do not have to be the same. This situation in fact is inherent in the FD methodology but is almost never stated explicitly.3 The second ingredient of FLAME is a set of n nodes (the number of nodes is equal to the number of patches). Although a meshless setup is possible, we shall for maximum simplicity assume a regular grid with a mesh size h. The i-th stencil is defined as a set of m(i) nodes within Ω(i) : stencil #i ≡ {nodes ∈ Ω(i) }. For any continuous potential u, N u will denote the set of its values at all grid nodes (viewed as a Euclidean vector in Rn ), and N (i) u – the set of nodal values on stencil #i. Although the FLAME solution may be multivalued between the nodes, its values at the nodes are required to be unique. (i) Within each patch, the approximate solution uh is sought as a linear (i) combination of m(i) basis functions {ψα }: (i)
uh =
m(i) α=1
(i) c(i) α ψα
(4.12)
Here we are following the same line of reasoning as in the preliminary example of Section 4.2.6 on p. 197, but in a more general setting. We need (i) to relate the coefficient vector c(i) ≡ {cα } ∈ Rm of expansion (4.12) to the (i) vector u(i) ∈RM of the nodal values of uh on stencil #i. (Both M and m can be different for different patches (i); this is understood but not explicitly indicated for simplicity of notation.) The relevant transformation matrix N (i) , 3
One might argue that in FD methods approximation between the grid nodes is not multivalued but simply undefined. This point of view is not incorrect but ignores the fact that the very derivation of FD schemes typically relies upon disparate Taylor expansions in the neighborhoods of each grid point.
200
4 Flexible Local Approximation MEthods (FLAME)
Fig. 4.6. Overlapping patches with 5-point stencils. (Reprinted by permission from c [Tsu04b] 2004 IEEE.)
u(i) = N (i) c(i) contains the nodal values of the basis functions on position vector of node k, then ⎛ (i) (i) ψ (r1 ) ψ2 (r1 ) . . . ⎜ 1(i) ⎜ ψ (r ) ψ2(i) (r2 ) . . . N (i) = ⎜ 1 2 ⎝ ... ... ... (i) (i) ψ1 (rM ) ψ2 (rM ) . . .
(4.13) the stencil; if rk is the ⎞ (i) ψm (r1 ) ⎟ (i) ψm (r2 ) ⎟ ⎟ ... ⎠ (i) ψm (rM )
(4.14)
4.3.2 Construction of the Schemes In the remainder, except for Appendix 4.7.3, the focus will be on the Trefftz version of FLAME, where the approximating functions ψ (i) satisfy the underlying differential equation (4.15) exactly. Trefftz methods are well known in the variational context (I. Herrera [Her00]); in contrast, here a purely finitedifference approach is taken and will prove to be attractive in a variety of cases.4 Trefftz–FLAME is simpler and at the same time usually more effective than the more general variational version of FLAME considered in Appendix 4.7.3 on p. 232. Since the basis functions by construction already satisfy the underlying (i) differential equation, so does the approximate solution uh , automatically. As we shall see, there will typically be fewer approximating functions than 4
The starting point for this development of Trefftz–FLAME schemes was Gary Friedman’s non-variational version of FLAME for unbounded problems [Fri05], [HFT04].
4.3 Trefftz Schemes with Flexible Local Approximation
201
nodes within the patch – most frequently, m functions for M = m + 1 stencil nodes. The nodal matrix N (i) is thus in general rectangular.5 The number of approximating functions may be different for different patches, but for brevity of notation this is not explicitly indicated. Let us initially assume that the underlying differential equation within a patch Ω(i) has a zero right hand side: Lu = 0 in Ω(i)
(4.15)
where L is a differential operator (one may want to have in mind, say, the Laplace operator as one of the simplest examples). (i) Within each patch, the approximate solution uh is sought as a linear (i) combination (4.12) of m(i) basis functions {ψα }. Identity (4.13) relates the vector of coefficients c(i) to the nodal values: N (i) c(i) = u(i)
(4.16)
In the simplest 1D example, with m = 2 basis functions ψ1,2 at three grid points xi−1 , xi , xi+1 , matrix N (i) (4.14) is ⎞ ⎛ ψ2 (xi−1 ) ψ1 (xi−1 ) ψ2 (xi ) ⎠ N (i) = ⎝ ψ1 (xi ) (4.17) ψ2 (xi+1 ) ψ1 (xi+1 ) We have already seen this for the 1D Laplace equation and the three-point stencil in Section 4.2.6. More generally for an M -point stencil, a vector of coefficients s(i) ∈ RM of the difference scheme is sought to yield s(i)T u(i) = 0
(4.18)
(i)
for the nodal values u(i) of any function uh of form (4.12). Due to (4.13) and (4.18), (4.19) s(i)T N (i) c(i) = 0 For this to hold for any set of coefficients c(i) , the null-space condition already familiar to us must hold: (4.20) s(i) ∈ Null N (i)T If the null space is of dimension one, s(i) represents the desired scheme (up to an arbitrary factor), and (4.20) is the principal expression of this Trefftz– FLAME scheme. The meaning of (4.20) is simple: each equation in the system N (i)T s(i) = 0 implies that the respective basis function satisfies the difference equation with coefficients s(i) . There is thus an elegant duality feature between the continuous and discrete problems: any linear combination of the basis 5
However, in the variational-difference formulation (Appendix 4.7.3), the number of basis functions is typically equal to the number of nodes.
202
4 Flexible Local Approximation MEthods (FLAME)
functions satisfies both the differential equation (due to the choice of the “Trefftz” basis) and the difference equation with coefficients s(i) . An alternative interpretation of (4.20) is that s(i) is orthogonal to the image of N (i) due to (4.19), hence s(i) is in the null space of N (i)T . In the complex case, though, orthogonality should not be understood in terms of the standard complex inner product which, unlike (4.19), includes conjugates. While there is no obvious way to determine the dimension of the null space a priori, for several classes of problems considered later the dimension is indeed one. If the null space is empty, the construction of the Trefftz– FLAME scheme fails, and one may want to either increase the size of the stencil or reduce the basis set. If the dimension of the null space is greater than one, there are two general options. First, the stencil and/or the basis can be changed. Second, one may use the additional freedom in the choice of the coefficients s(i) to seek an “optimal” (in some sense) scheme as a linear combination of the independent null space vectors. For example, it may be desirable to find a diagonally dominant scheme. Once the basis and the stencil are chosen, the Trefftz–FLAME scheme is generated in a very simple way: • Form matrix N (i) of the nodal values of the basis functions. • Find the null space of N (i)T . Proposition 11. The Trefftz–FLAME scheme defined by (4.20) is invariant (i) with respect to the choice of the basis in the local space Ψ(i) ≡ span{ψα }. Proof. A linear transformation of the ψ-basis replaces N T with QN T , where Q is a nonsingular matrix, which does not affect the null space. The algorithm can be sketched as a “machine” for generating Trefftz– FLAME schemes (Fig. 4.7). It should be stressed that the algorithm is heuristic and no blanket claim of convergence can be made. The schemes need to be considered on a caseby-case basis, which is done for a variety of problems in Section 4.3. However, consistency can be proven (Section 4.3.5) in general, and convergence then follows for the subclass of schemes with a monotone difference operator [Tsu05a]. As we shall see in Section 4.4, definition (4.20), despite its simplicity, is surprisingly rich. For different choices of basis functions and stencils it gives rise to a variety of difference schemes. 4.3.3 The Treatment of Boundary Conditions Note that in the FLAME framework approximations over different stencils are completely independent from one another. Therefore, if the domain boundary conditions are of standard types and no special behavior of the solution at
4.3 Trefftz Schemes with Flexible Local Approximation
203
Fig. 4.7. A “machine” for Trefftz–FLAME schemes. (Reprinted by permission from c [Tsu05a] 2005 IEEE.)
the boundaries is manifest, one can simply employ any standard FD scheme at the boundary.6 If the solution is known to exhibit some special features at the boundary, it may be possible to incorporate these features into FLAME. One example – Perfectly Matched Layers (PML) for electromagnetic and acoustic wave propagation – is considered briefly in Section 4.4.11 and in [Tsu05a]. 4.3.4 Trefftz–FLAME Schemes for Inhomogeneous and Nonlinear Equations So far we considered Trefftz–FLAME schemes only for homogeneous equations (i.e. with the zero right hand side within a given patch). For inhomogeneous equations of the form (4.21) Lu = f in Ω(i) (i)
a natural approach is to split the solution up into a particular solution uf (i)
of the inhomogeneous equation and the remainder u0 satisfying the homogeneous one: (i) (i) (4.22) u = u 0 + uf (i)
Lu0 = 0;
(i)
Luf
= f
(4.23)
Superscript (i) emphasizes that the splitting is local, i.e. needs to be introduced only within its respective patch Ω(i) containing the grid stencil around node 6
Since most Taylor-based schemes are particular cases of FLAME (with polynomial basis functions), it would be technically correct to say that the whole set of difference equations, including the treatment of boundary conditions, is based on FLAME.
204
4 Flexible Local Approximation MEthods (FLAME) (i)
i. Since uf is local (and in particular need not satisfy any exterior boundary conditions), it is usually relatively easy to construct. Let a Trefftz–FLAME scheme s(i) be generated for a given set of basis functions and assume that the consistency error for this scheme tends to zero as h → 0; that is, (i) (i) (4.24) s(i)T N (i) u0 = ≡ h, u0 → 0 as grid size h → 0 where N (i) , as before, denotes the nodal values of a function on stencil (i). Then clearly s(i)T N (i) u = s(i)T N (i) u0 + s(i)T N (i) uf = s(i)T N (i) uf + This immediately implies that the consistency error of the difference scheme s(i)T uh = s(i)T N (i) uf
(4.25)
is , i.e. exactly the same as for the homogeneous case. (The Euclidean vector uh of nodal values does not need the superscript because the nodal values are unique and do not depend on the patch.) Note that there are absolutely no (i) constraints on the smoothness of uf , provided that it has valid nodal values. (i)
The particular solution uf can even be singular as long as the singularity point does not coincide with a grid node. In [Tsu04a] difference schemes of this kind were constructed for the Coulomb potential of point charges. An electrostatic problem with a line charge source is solved in a similar way in [Tsu05a]. For nonlinear problems, the Newton–Raphson method is traditionally used for the discrete system of equations. In connection with FLAME schemes, Newton–Raphson–Kantorovich iterations are applied to the original continuous problem rather than the discrete one. Let the equation be Lu = f
(4.26)
where L is a differentiable operator. The (k + 1)-th approximation uk+1 to the exact solution is obtained from the k-th approximation uk by linearization in the following way. If u = uk + δu, Lu = L(uk + δu) = Luk + L (uk )δu + o(δu)
(4.27)
where L is the Fr´echet derivative of L (Appendix 4.9). Ignoring higher-order terms, one gets an approximation δuk for δu by solving the linear system L (uk ) δuk = f − Luk
(4.28)
and then updates the solution: uk+1 = uk + δuk
(4.29)
4.3 Trefftz Schemes with Flexible Local Approximation
Equivalently,
uk+1 = uk + (L (uk ))−1 (f − Luk )
205
(4.30)
Along with an initial guess u0 , iterative process (4.28), (4.29) – or just (4.30) – defines the Newton–Raphson–Kantorovich algorithm. Trefftz–FLAME schemes can then be applied to L (which of course is a linear operator by definition), provided that a suitable set of local approximating functions can be found. Further analysis of the N–R–K iterations for FLAME schemes in colloidal simulation (the Poisson–Boltzmann equation) can be found in Section 6.8 on p. 319. 4.3.5 Consistency and Convergence of the Schemes Let us rewrite the patch-wise difference equation (4.25) in matrix form as a global system of difference equations for the underlying differential equation Lu = f : (i) (4.31) Lh uh = f h , with f hi = s(i)T N (i) uf (i)
(if the differential equation is homogeneous within the patch, then uf = 0). Note that the i-th row of matrix Lh contains the coefficients of scheme s(i)T and, in addition, a (large) number of zero entries.7 We shall assume that the equations can be scaled in such a way that c1 f (r) ≤ f hi ≤ c2 f (r),
∀r ∈ Ω(i) ,
c1,2 > 0
(4.32)
where c1,2 do not depend on i and h. This scaling is important because otherwise e.g. the meaningless scheme h100 ui = 0 would technically be consistent (as defined below) for any differential equation. The consistency error of scheme (4.31) is, by definition, obtained by substituting the nodal values of the exact solution u∗ into the difference equation. We shall call this scheme consistent if, with scaling (4.32), the following condition holds: consistency error ≡ c (h) = max s(i)T N (i) u∗ − f hi i
= max s(i)T (N (i) u∗ − uhi ) → 0 as h → 0 i
(4.33)
For FLAME schemes, consistency follows directly from the approximation properties of the basis set as long as (4.32) holds. Indeed, let a (h) be the (i) approximation error of the “homogeneous part” u0 of the exact solution u∗ (i) in a patch Ω : 7
Our notation would perhaps be more consistent if the matrix were denoted with Lh and the scheme with l(i) or, alternatively, if the scheme were s(i) and the matrix were Sh . However, throughout the book the usual symbol L is adopted for differential and difference operators, and s is used as a mnemonic symbol for “scheme”.
206
4 Flexible Local Approximation MEthods (FLAME)
a (h) =
c
m (i) min u∗ − uf − (i) m ∈R
α=1
(i) c(i) α ψα
∞
Equivalently, there exists a coefficient vector c(i) ∈ Rm such that m (i) (i) c(i) η∞ = a (h) u ∗ = uf + α ψα + η, α=1
(4.34)
(4.35)
For the nodal values, one then has due to (4.16) N (i) u∗ = N (i) uf
(i)
+ N (i) c(i) + η
(4.36)
where η = N (i) η is the vector of nodal values of η on stencil i and N (i) is (as always) the matrix of nodal values of the basis functions. Due to (4.35), η∞ ≤ a (h) and due to (4.36), the consistency error for scheme (4.31) with coefficients (4.20) is (i) |c (h)| = max s(i)T N (i) u∗ − s(i)T N (i) uf = max s(i)T (N (i) c(i) + η) i
i
= max s(i)T N (i) c(i) + s(i)T η = max s(i)T η ≤ M a (h) i
i
(4.37)
which shows that the consistency error is bounded by the approximation error. Theoretical results relevant to the convergence of the schemes were summarized in Chapter 2. Estimate (2.148) (p. 62) of the solution error is the ratio of approximation and stability parameters. The approximation accuracy a is key. In fact, the “Trefftz” bases are effective not just because they (by definition) satisfy the underlying differential equation, but because they happen to have superior approximation properties in many cases (see e.g. Sections 4.4.4, 4.4.5).
4.4 Trefftz–FLAME Schemes: Case Studies 4.4.1 1D Laplace, Helmholtz and Convection-Diffusion Equations The 1D Laplace equation was already considered as a preliminary example in Section 4.2.6 of this chapter (p. 197). A less trivial case is the 1D Helmholtz equation d2 u − κ2 u = 0 dx2 with any complex κ. Two basis functions satisfying the Helmholtz equation are ψ1 = exp(κx); ψ2 = exp(−κx)
4.4 Trefftz–FLAME Schemes: Case Studies
207
For a three-point stencil with the coordinates of the nodes (−h, 0, h) (the middle node is placed at the origin for simplicity), the matrix of nodal values (4.14) is exp(−κh) 1 exp(κh) T N = exp(κh) 1 exp(−κh) and the resultant difference scheme is s = Null N T = (1, − 2 cosh(κh), 1)T
(4.38)
Since the theoretical solution in this 1D case is exactly representable as a linear combination of the chosen basis functions, the difference scheme yields the exact solution (in practice, up to the round-off error). This scheme is known and has been derived in a different way by R.E. Mickens [Mic94]; see also C. Farhat et al. [FHF01] and I. Harari & E. Turkel [HT95]. Quite similarly, for the 1D convection-diffusion equation D
du d2 u = 0, − b 2 dx dx
D>0
with constant coefficients D and b, one has two Trefftz basis functions: ψ1 = 1; ψ2 = exp(qx),
q = b/D
For the 3-point stencil (−h, 0, h), the (transposed) matrix of nodal values (4.14) is 1 1 1 NT = exp(−qh) 1 exp(qh) and the Trefftz–FLAME difference scheme is exp(qh) + 1 1 exp(qh) T , − , s = Null N = exp(qh) − 1 exp(qh) − 1 exp(qh) − 1
(4.39)
(up to an arbitrary factor). This coincides (in the case of the homogeneous convection-diffusion equation with constant coefficients) with the well-known exponentially fitted scheme (see e.g. D.B. Spalding [Spa72], G.D. Raithby & K.E. Torrance [RT74], S.V. Patankar [Pat80]). 4.4.2 The 1D Heat Equation with Variable Material Parameter Consider the 1D homogeneous heat conduction equation: du d λ(x) = 0 dx dx
(4.40)
where λ(x) is the material parameter. Two approximating functions for the FLAME-Trefftz scheme can be chosen as linearly independent solutions of this equation on the interval [xk−1 , xk+1 ]:
208
4 Flexible Local Approximation MEthods (FLAME)
ψ1 = 1,
x
ψ2 =
λ−1 (ξ)dξ
xk
With this basis, the transposed nodal matrix (4.14) for the stencil (xk−1 , xk , xk+1 ) is 1 1 1 T N = −Σk−1 0 Σk+1 xk x where Σk−1 = xk−1 λ−1 (ξ) dξ, Σk+1 = xkk+1 λ−1 (ξ) dξ have the physical meaning of thermal resistances of the respective segments. The difference scheme is, up to an arbitrary factor, −1 −1 s = Null N T = (−Σ−1 k−1 , Σk−1 + Σk+1 ,
T − Σ−1 k+1 )
(4.41)
which has a clear interpretation as a flux balance equation: −1 Σ−1 k−1 (uk − uk−1 ) + Σk+1 (uk − uk+1 ) = 0
Such schemes are indeed typically derived from flux balance considerations (see e.g. the “homogeneous schemes” in [Sam01]) but, as we can now see, emerge as a natural particular case of Trefftz–FLAME. If the integrals in the expressions for thermal resistances Σ can be calculated exactly, the scheme is itself exact, i.e. the consistency error is zero (the theoretical solution satisfies the FD equation). This holds even if the material parameter λ is discontinuous. A very similar analysis applies to the 1D linear electrostatic equation with a variable (and possibly discontinuous) permittivity . 4.4.3 The 2D and 3D Laplace Equation Consider a regular rectangular grid, for simplicity with spacing h the same in both directions, and the standard 5-point stencil. The origin of the coordinate system is placed for convenience at the central node of the stencil. With four basis functions [1, x, y, x2 − y 2 ] satisfying the Laplace equation, the nodal matrix (4.14) becomes ⎛ ⎞ 1 1 1 1 1 ⎜ 0 −h 0 h 0 ⎟ ⎟ NT = ⎜ ⎝ h 0 0 0 −h⎠ −h2 h2 0 h2 − h2 The difference scheme is then Null N T = (−1, −1, 4, −1, −1)T (times an arbitrary constant), which coincides with the standard 5-point scheme for the Laplace equation. A more general case with different mesh sizes in the x- and y- directions is handled similarly. The 3D case is also fully analogous. With six basis functions {1, x, y, z, x2 − y 2 , x2 − z 2 } and the standard 7-point stencil on a uniform grid, one
4.4 Trefftz–FLAME Schemes: Case Studies
209
arrives, after computing the null space of the respective 6 × 7 matrix N T , at the standard 7-point scheme with the coefficients (−1, −1, −1, 6, −1, −1, −1)T . As in 2D, the case of different mesh sizes in the x-, y- and z-directions does not present any difficulty. 4.4.4 The Fourth Order 9-point Mehrstellen Scheme for the Laplace Equation in 2D The solution is, by definition, a harmonic function. Harmonic polynomials are known to provide an excellent (in some sense, even optimal [BM97]) approximation of harmonic functions [And87, BM97, Ber66, Mel99]. Indeed, for a fixed polynomial order p, the FEM and harmonic approximation errors are similar [BM97]; however, the FEM approximation is realized in a much wider space containing all polynomials up to order p, not just the harmonic ones. For solving the Laplace equation, the standard FE basis set can thus be viewed as having substantial redundancy that is eliminated by using the harmonic basis. The following result is cited in [BM97]: Theorem 6. (Szeg¨ o). Let Ω ⊂ R2 be a simply connected bounded Lipschitz ˜ is harmonic on Ω. ˜ Then ˜ ⊃⊃ Ω and assume that u ∈ L2 (Ω) domain. Let Ω ∞ there is a sequence (up )p=0 of harmonic polynomials of degree p such that u − up L∞ (Ω) ≤ c exp(−γp) uL2 (Ω) ˜ ∇(u − up )L∞ (Ω) ≤ c exp(−γp) uL2 (Ω) ˜
(4.42)
˜ where γ, c > 0 depend only on Ω, Ω. For comparison, the H 1 -norm error estimate in the standard FEM is Theorem 7. (P.G. Ciarlet & P.A. Raviart, I. Babuska & M. Suri [CR72], [Cia80], [BS94]). For a family of quasiuniform meshes with elements of order p and maximum diameter h, the approximation error in the corresponding finite element space V n is inf u − vH 1 (Ω) = Chµ−1 p−(k−1) uH k (Ω)
v∈V n
where µ = min(p + 1, k) and c is a constant independent of h, p, and u. For a fixed polynomial order p, the FEM and harmonic polynomial estimates are similar (factor O(hp ) vs. O([exp(−γ)]p ) if the solution is sufficiently smooth. However, the FEM approximation is realized in a much wider space containing all polynomials up to order p, not just the harmonic ones. For solving the Laplace equation, the standard FE basis set can thus be viewed as having substantial redundancy that is eliminated by using the harmonic basis.
210
4 Flexible Local Approximation MEthods (FLAME)
With these observations in mind, one may choose the basis functions as harmonic polynomials in x, y up to order 4, namely, {1, x, y, xy, x2 −y 2 , x(x2 − 3y 2 ), y(3x2 − y 2 ), (x2 − y 2 )xy, (x2 − 2xy − y 2 )(x2 + 2xy − y 2 )}. Then for a 3 × 3 stencil of adjacent nodes of a uniform Cartesian grid, the computation of the nodal matrix (4.14) (transposed) and its null space is simple with any symbolic algebra package. If the mesh size is equal in both x- and y- directions, the resultant scheme has order 6. Its coefficients are 20 for the central node, −4 for the four mid-edge nodes, and −1 for the four corner nodes of the stencil. In the standard texts (L. Collatz [Col66], A.A. Samarskii [Sam01]), this scheme is derived by manipulating the Taylor expansions for the solution and its derivatives. 4.4.5 The Fourth Order 19-point Mehrstellen Scheme for the Laplace Equation in 3D Construction of the scheme is analogous to the 2D case. The 19-point stencil is obtained by considering a 3 × 3 × 3 cluster of adjacent nodes and then discarding the eight corner nodes. The basis functions are chosen as the 25 independent harmonic polynomials in x, y, z up to order 4. Computation of the matrix of nodal values (4.14) and of the null space of its transpose is straightforward by symbolic algebra. The result is the 19-point fourth-order “Mehrstellen” scheme by L. Collatz [Col66] (see also A.A. Samarskii [Sam01]) already discussed in Chapter 2 (Section 2.8.5, p. 58). In that chapter, as well as in the Collatz and Samarskii books, the scheme is derived from completely different considerations.8 We can now see, however, that in the Trefftz–FLAME framework Mehrstellen schemes and classic Taylor-based schemes for the Laplace equation stem from the same root – namely, the nullspace equation (4.20). The scheme is defined by the chosen stencil and a harmonic polynomial basis. As a side note, the 19-point Mehrstellen scheme, due to its geometrically compact stencil, reduces processor communication in parallel solvers and therefore has gained popularity in computationally intensive applications of physical chemistry and quantum chemistry: electrostatic fields of multiple charges, the Poisson–Boltzmann equation in colloidal and protein simulation, and the Kohn–Sham equation of Density Functional Theory (E.L. Briggs et al. [BSB96]). 4.4.6 The 1D Schr¨ odinger Equation. FLAME Schemes by Variation of Parameters This test problem is borrowed from the comparison study by R. Chen et al. [CXS93] of several FD schemes for the boundary value (rather than eigenvalue) problem for the 1D Schr¨ odinger equation over a given interval [a, b]: 8
A generalization of the Mehrstellen schemes, known as the HODIE schemes by R.E. Lynch & J.R. Rice [LR80], will not be considered here.
4.4 Trefftz–FLAME Schemes: Case Studies
−u + (V (x) − E)u = 0,
u(a) = ua , u(b) = ub
211
(4.43)
The specific numerical example is the 5th energy level of the harmonic oscillator, with V (x) = x2 and E = 11 (= 2 × 5 + 1). For testing and verification, boundary conditions are taken from the analytical solution, and as in [CXS93] the interval [a, b] is [−2, 2]. The exact solution is uexact = (15x − 20x3 + 4x5 ) exp(−x2 /2)
(4.44)
To construct a Trefftz–FLAME scheme for (4.43) on a stencil [xi−1 , xi , xi+1 ] (where xi±1 = xi ±h), one would need to take two independent local solutions of the Schr¨odinger equation as the FLAME basis functions. The exact solution in our example is reserved exclusively for verification and error analysis. We shall construct Trefftz–FLAME scheme pretending that the theoretical solution is not known, as would be the case in general for an arbitrary potential V (x). Thus in lieu of the exact solutions the basis set will contain their approximations. There are at least two ways to construct such approximations. This subsection uses a perturbation technique that produces a fourth-order scheme. The next subsection employs the Taylor expansion that leads to 3point schemes of arbitrarily high order. At an arbitrary point x0 let V (x) = κ2 + δV, where κ2 ≡ V (x0 )
(4.45)
u(x) = u0 (x) + δu(x)
(4.46)
u0 (x) = c+ exp(κx) + c− exp(−κx),
with arbitrary c+ , c−
(4.47)
Substituting these expressions into the Schr¨ odinger equation and ignoring the higher order term, one gets the perturbation equation δu − κ2 δu = δV u0
(4.48)
Solving this equation by variation of parameters, one obtains after some algebra x 1 u0 (ξ) exp(−κξ)δV (ξ)dξ u(x) = u0 (x) + δu(x) = u0 (x) + exp(κx) 2 x0 x 1 − exp(−κx) u0 (ξ) exp(κξ)δV (ξ)dξ (4.49) 2 x0 Two independent sets of values for c+ , c− then yield two basis functions for FLAME. Fig. 4.8 compares convergence of several schemes: the well-known Numerov scheme, the “Numerov–Mickens scheme” [CXS93], Trefftz–FLAME, and the Mickens scheme [Mic94, CXS93]. The first three schemes are all of order four, but the FLAME errors are much smaller. In the following section, the FLAME error is further reduced, in many cases to machine precision.
212
4 Flexible Local Approximation MEthods (FLAME)
Fig. 4.8. Convergence of the variation of parameters – FLAME scheme for the Schr¨ odinger equation. Comparison with other schemes described in [CXS93] is very favorable (note the logarithmic scale). As the Numerov and Numerov-Mickens schemes, the FLAME scheme is of fourth order but its error is much smaller. The Taylor version of FLAME (see below) performs much better still. (Reprinted by c permission from [Tsu06] 2006 Elsevier.)
4.4.7 Super-high-order FLAME Schemes for the 1D Schr¨ odinger Equation For sufficiently smooth potentials V (x), as in our example of the harmonic oscillator, one can expand the potential and the solution into a Taylor series around the central stencil node xi to obtain two local independent solutions with any desired order of accuracy. Consequently, the order of the FLAME scheme can also be arbitrarily high, even though the stencil still has only three points. For the 20th -order scheme as an example, the roundoff level of the numerical error is reached for the uniform grid with just 10–15 nodes (Table 4.4.7). For a fixed grid size and varying order of the scheme, the error falls off very rapidly as the order is increased and quickly saturates at the roundoff level (Fig. 4.9). Table 4.1. Errors for the 3-point FLAME scheme of order 20 Number of nodes Mean absolute error 7 2.14E-10 2.06E-14 11 1.75E-15 15
4.4 Trefftz–FLAME Schemes: Case Studies
213
Fig. 4.9. Error vs. order of the Trefftz–FLAME scheme for the model Schr¨ odinger c equation. (Reprinted by permission from [Tsu06] 2006 Elsevier.)
4.4.8 A Singular Equation G.W. Reddien & L.L. Schumaker [RS76] (RS) proposed a spline-based collocation method for 1D singular boundary value problems and use the following example:9 (x0.5 u ) − x0.5 u = 0,
0 < x < 1, u(0) = 1, u(1) = 0
(4.50)
Here we apply the non-variational FLAME method to the same example and compare the results. A 3-point stencil on a uniform grid is used for FLAME. The two basis functions for FLAME are constructed separately for stencil points in the vicinity of the singularity point x = 0 and away from zero. 1) Let the midpoint xi of the i-th stencil be sufficiently far away from zero (the singularity point of the differential equation): xi > δ, where δ is a chosen threshold. Expanding u over the i-th stencil into the Taylor series with respect to ξ = x − xi , ∞ u = ck ξ k (4.51) k=0
one obtains, by straightforward calculation, the following recursion: ck+2 =
ck xi + ck−1 − ck+1 (k + 1)(k + 12 ) , xi (k + 1)(k + 2)
k = 0, 1, . . .
(4.52)
where the coefficients with negative indices are understood to be zero. Two basis functions are obtained by choosing two independent sets of starting values for c0,1 for the recursion and by retaining a finite number of terms, k = K, in series (4.51). 9
This example is as a result of my short communication with Larry L. Schumaker and Douglas N. Arnold.
214
4 Flexible Local Approximation MEthods (FLAME)
2) For xi < δ, the approach is similar but the series expansion is different: u =
∞ k=0
bk xk/2
(4.53)
Straightforward algebra again yields ∀b0 , b1 ; bk+2 =
b 2 = b3 = 0
4bk−2 , (k + 1)(k + 2)
k = 0, 1, . . .
(4.54)
Two independent basis functions are then obtained in the same manner as above, with terms k ≤ 2K retained in (4.53). Numerical values of the solution at x = 0.5 are given in [RS76, CR72] and serve as a basis for accuracy comparison. As Tables 4.2 and 4.3 show, Trefftz–FLAME gives orders of magnitude higher accuracy than the methods of [RS76, CR72]. The price for this accuracy gain is the analytical work needed for “preprocessing,” i.e. for deriving the FLAME basis functions. This example is intended to serve as an illustration of the capabilities of FLAME and its possible applications; it does not imply that FLAME is necessarily better than all methods designed for singular equations. Many other effective techniques have been developed (e.g. M. Kumar [Kum03]). n 8 16 8192
FLAME, K = 6 0.25204513942296 0.252044597187729 0.252042091673094
FLAME, K = 12 0.252041978171219 0.252041977565477 0.252041976551393
RS [RS76] 0.25305 0.25223
Jamet [Jam70] 0.29038 0.27826 0.25310
Table 4.2. Numerical values of the solution at x = 0.5: FLAME vs. other methods. The number of grid subdivisions and the order of the scheme in FLAME varied.
n 8 16 8192
FLAME, K = 6 3.16E-06 2.62E-06 1.15E-07
FLAME, K = 12 1.68E-09 1.07E-09 5.80E-11
RS [RS76] 1.01E-03 1.88E-04
Jamet [Jam70] 3.83E-02 2.62E-02 1.06E-03
Table 4.3. Numerical errors of the solution at x = 0.5: FLAME vs. other methods. The result for the FLAME scheme of order 40 with 8192 grid subdivisions was treated as “exact” for the purposes of error evaluation.
4.4 Trefftz–FLAME Schemes: Case Studies
215
4.4.9 A Polarized Elliptic Particle This subsection gives an example of FLAME in two dimensions. A dielectric cylinder, with an elliptic cross-section, is immersed in a uniform external field. An analytical solution using complex variables is developed, for example, by W.B. Smythe [Smy89]. If lx > ly are the two semiaxes of the ellipse and the applied external field is in the x-direction, then the solution in the first quadrant of the plane can be described by the following sequence of expressions [Smy89], with z = x + iy: α2 = lx2 − ly2 α √ z − z 2 − α2
z1 = A =
(lx + ly )(lx − ly ) (lx − ly )(lx + ly )
B =
lx + l y lx + ly
Potential outside the ellipse:
α u = Re 2
A z1 + z1
Potential inside the ellipse: u = Re
1 α B z1 + 2 z1
Similar expressions hold in other quadrants and for the y-direction of the applied field. In the numerical example below, the computational domain Ω is taken to be the unit square [0, 1] × [0, 1]. To eliminate the numerical errors associated with the finite size of this domain, the analytical solution (for the x-direction of the external field) is imposed, for testing and verification purposes, as the Dirichlet condition on the exterior boundary of Ω. For the usual 5-point stencil in 2D, four basis functions would normally be needed to yield the null space of dimension one in Trefftz–FLAME. The choice of three basis functions is clear: ψ1 = 1, and ψ2,3 are the theoretical solutions for two perpendicular directions of the applied external field (along each axis of the ellipse). Deriving a fourth Trefftz function is not worth the effort. Instead, Trefftz–FLAME is applied with the three basis functions. This yields a two-dimensional null space, with two independent 5-point difference schemes s1,2 ∈ R5 . It then turns out to be possible to find a linear combination
216
4 Flexible Local Approximation MEthods (FLAME)
of these two schemes with a dominant diagonal entry, so that the convergence conditions of Section 4.3.5 are satisfied.10 The particular results below are for the material parameter in = 10 within the ellipse, for out = 1 outside the ellipse, and for the main axis of the ellipse aligned with the external field. The semiaxes are lx = 0.22 and ly = 0.12. The FLAME basis functions ψ1,2,3 are introduced for all stencils having at least one node inside the ellipse and, in addition in some experiments, in several layers around the ellipse. These additional layers are such that ξmidpoint < ξcutoff , where ξ = (x/lx )2 + (y/ly )2 − 1 (with x, y measured from the center of the ellipse), ξmidpoint is the value of ξ for the midpoint of the stencil, and ξcutoff is an adjustable threshold. For ξcutoff = 0 no additional layers with the special basis are introduced. For ξcutoff 1 the special bases are used throughout the domain, which yields the solution with machine precision.11