ADVANCES IN IMAGING AND ELECTRON PHYSICS VOLUME 125
EDITOR-IN-CHIEF
PETER W. HAWKES CEMES-CNRS Toulouse, France
ASSOCIATE EDITORS
BENJAMIN KAZAN Xerox Corporation Palo Alto Research Center Palo Alto, California
TOM MULVEY Department of Electronic Engineering and Applied Physics Aston University Birmingham, United Kingdom
Advances in
Imaging and Electron Physics Edited by
PETER W. HAWKES CEMES-CNRS Toulouse, France
VOLUME 125
Amsterdam Boston London New York Oxford Paris San Diego San Francisco Singapore Sydney Tokyo
∞ This book is printed on acid-free paper. C 2002, Elsevier Science (USA). Copyright
All Rights Reserved. No part of this publication may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopy, recording, or any information storage and retrieval system, without permission in writing from the Publisher. The appearance of the code at the bottom of the first page of a chapter in this book indicates the Publisher’s consent that copies of the chapter may be made for personal or internal use of specific clients. This consent is given on the condition, however, that the copier pay the stated per copy fee through the Copyright Clearance Center, Inc. (www.copyright.com), for copying beyond that permitted by Sections 107 or 108 of the U.S. Copyright Law. This consent does not extend to other kinds of copying, such as copying for general distribution, for advertising or promotional purposes, for creating new collective works, or for resale. Copy fees for pre-2002 chapters are as shown on the title pages. If no fee code appears on the title page, the copy fee is the same as for current chapters. 1076-5670/2002 $35.00 Explicit permission from Academic Press is not required to reproduce a maximum of two figures or tables from an Academic Press chapter in another scientific or research publication provided that the material has not been credited to another source and that full credit to the Academic Press chapter is given.
Academic Press An imprint of Elsevier Science. 525 B Street, Suite 1900, San Diego, California 92101-4495, USA http://www.academicpress.com
Academic Press 84 Theobald’s Road, London WC1X 8RR, UK http://www.academicpress.com International Standard Book Number: 0-12-014767-X PRINTED IN THE UNITED STATES OF AMERICA 02 03 04 05 06 07 9 8 7 6 5 4 3 2
1
CONTENTS
Contributors . . . . . . . . . . . . . . . . . . . . . . . . . . Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Future Contributions . . . . . . . . . . . . . . . . . . . . . .
ix xi xiii
An Algebraic Approach to Subband Signal Processing Marilena Barnabei and Laura B. Montefusco
I. II. III. IV. V. VI. VII. VIII. IX.
Introduction . . . . . . . . . . . An Overview of Recursive Matrices . Recursive Operators of Filter Theory More Algebraic Results . . . . . . Analysis and Synthesis Filter Banks . M-Channel Filter Bank Systems . . Transmultiplexers . . . . . . . . The M-Band Lifting . . . . . . . Conclusion . . . . . . . . . . . References . . . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
1 4 13 22 30 37 44 48 60 61
Background . . . . . . . . . . . . . . . . . . . . . Fundamentals . . . . . . . . . . . . . . . . . . . . ALCHEMI Results . . . . . . . . . . . . . . . . . . Predicting Sublattice Occupancies. . . . . . . . . . . . Competing (or Supplementary) Techniques . . . . . . . . Summary . . . . . . . . . . . . . . . . . . . . . . Current Challenges and Future Directions (a Personal View). References . . . . . . . . . . . . . . . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
63 64 77 100 102 106 106 107
. . . .
. . . .
. . . .
120 122 141 149
Determining the Locations of Chemical Species in Ordered Compounds: ALCHEMI I. P. Jones
I. II. III. IV. V. VI. VII.
Aspects of Mathematical Morphology K. Michielsen, H. De Raedt, and J. Th. M. De Hosson
I. II. III. IV.
Introduction . . . . . . . . Integral Geometry: Theory . . Integral Geometry in Practice. Illustrative Examples . . . .
. . . .
. . . .
. . . .
v
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
vi
CONTENTS
V. Computer Tomography Images of Metal Foams . VI. Summary . . . . . . . . . . . . . . . . . Appendix A: Algorithm . . . . . . . . . . . Appendix B: Programming Example (Fortran 90) Appendix C: Derivation of Eq. (36) . . . . . . Appendix D: Proof of Eq. (56) . . . . . . . . Appendix E: Proof of Eq. (57) . . . . . . . . References . . . . . . . . . . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
170 176 176 178 182 184 188 190
Introduction . . . . . . . . . . . . . . . . . . . . . . . History . . . . . . . . . . . . . . . . . . . . . . . . . Ultrafast Scanning Probe Microscopy . . . . . . . . . . . . Junction Mixing STM . . . . . . . . . . . . . . . . . . . Distance Modulated STM . . . . . . . . . . . . . . . . . Photo-Gated STM . . . . . . . . . . . . . . . . . . . . Ultrafast STM by Direct Optical Coupling to the Tunnel Junction Conclusions . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . .
. . . . . . . . .
195 196 199 205 216 218 225 228 228
. . . . . . . . . . .
. . . . . . . . . . .
232 237 252 270 291 317 325 327 336 340 349
I. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . II. Electron Diffraction and Diffraction Contrast . . . . . . . . . . .
356 358
Ultrafast Scanning Tunneling Microscopy G. M. Steeves and M. R. Freeman
I. II. III. IV. V. VI. VII. VIII.
Low-Density Parity-Check Codes—A Statistical Physics Perspective Renato Vicente, David Saad, and Yoshiyuki Kabashima
I. II. III. IV. V. VI. VII.
Introduction . . . . . . . . . . . . . . . Coding and Statistical Physics . . . . . . . Sourlas Codes . . . . . . . . . . . . . . Gallager Codes . . . . . . . . . . . . . MacKay–Neal Codes . . . . . . . . . . . Cascading Codes . . . . . . . . . . . . . Conclusions and Perspectives. . . . . . . . Appendix A. Sourlas Codes: Technical Details Appendix B. Gallager Codes: Technical Details Appendix C. MN Codes: Technical Details . . References . . . . . . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
Computer-Aided Crystallographic Analysis in the TEM Stefan Zaefferer
CONTENTS
vii
III. A Universal Procedure for Orientation Determination from Electron Diffraction Patterns . . . . . . . . . . . . . . . . . . . . . IV. Automation of Orientation Determination in TEM . . . . . . . . . V. Characterization of Grain Boundaries . . . . . . . . . . . . . . VI. Determination of Slip Systems . . . . . . . . . . . . . . . . . VII. Phase and Lattice Parameter Determination . . . . . . . . . . . . VIII. Calibration . . . . . . . . . . . . . . . . . . . . . . . . . IX. Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . .
369 377 391 397 403 408 411 413
Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
417
This Page Intentionally Left Blank
CONTRIBUTORS
Numbers in parentheses indicate the pages on which the author’s contributions begin.
Marilena Barnabei (1), Department of Mathematics, University of Bologna, I-40127 Bologna, Italy M. R. Freeman (195), Department of Physics, University of Alberta, Edmonton, AB T6G 2J1, Canada J. Th. M. De Hosson (119), Department of Applied Physics, Materials Science Centre and Netherlands Institute for Metals Research, University of Groningen, NL-9747 AG Groningen, The Netherlands H. De Raedt (119), Institute for Theoretical Physics, Materials Science Center, University of Groningen, NL-9747 AG Groningen, The Netherlands I. P. Jones (63), Center for Electron Microscopy, University of Birmingham, Birmingham B15 2TT, United Kingdom Yoshiyuki Kabashima (231), Department of Computational Intelligence and Systems Science, Tokyo Institute of Technology, Yokohama 2268502, Japan K. Michielsen (119), Institute for Theoretical Physics, Materials Science Center, University of Groningen, NL-9747 AG Groningen, The Netherlands Laura B. Montefusco (1), Department of Mathematics, University of Bologna, I-40127 Bologna, Italy David Saad (231), Neural Computing Research Group, University of Aston, Birmingham B4 7ET, United Kingdom G. M. Steeves (195), California NanoSystems Institute, University of California, Santa Barbara, California 931006 Renato Vicente (231), Neural Computing Research Group, University of Aston, Birmingham B4 7ET, United Kingdom Stefan Zaefferer (355), Max Plank Institute for Iron Research, D-40237 D¨usseldorf, Germany ix
This Page Intentionally Left Blank
PREFACE
The contributions to this latest volume of the Advances extend over a wide range of themes: subband signal processing, electron microscopy and diffraction, mathematical morphology, fast scanning tunneling microscopy and coding theory. The volume opens with a presentation by M. Barnabei and L. B. Montefusco of the algebraic theory, largely initiated and developed by them, that makes it possible to study filter banks systematically from the time-domain viewpoint. For this, a knowledge of recursive matrix theory is needed and this is provided in the opening paragraphs. Since banded recursive matrices can be described by two Laurent polynomials, these are likewise defined and their properties presented in the introductory sections. The authors then examine the connection between banded recursive matrices and some of the linear filters encountered in filter theory. In the remainder of the chapter, these notions are used to investigate filter bank systems in great detail. In the second chapter, I. P. Jones describes how the locations of chemical sites in ordered compounds are determined by channeling-enhanced microanalysis, the technique that is now universally referred to as ALCHEMI. In this short and complete monograph on the subject, the author first explains how the technique works, considers the accuracy to be expected and investigates related questions. In the long section that follows, results obtained by ALCHEMI are tabulated for crystals of various kinds; these tables are an invaluable source of information and include the references in which the results were first described. I. P. Jones concludes with a personal view of current challenges and future directions, to which he will, I hope, return one day in these Advances. Mathematical morphology appears frequently in these pages but the aspect of this subject considered by K. Michielsen, H. de Raedt, and J. Th. M. de Hosson has not been discussed here before. They present an approach to morphological image analysis whereby the shape and connectivity patterns of the individual pixels of an image are described by additive image functionals. It is known that the number of such functionals is three in the case of two-dimensional images and four in the three-dimensional case. The authors present the basic mathematics of these ideas in full detail, from which the newcomer to this aspect of mathematical morphology can learn all the necessary background. They then turn to a number of applications, which demonstrate just how useful the theory is; the case of metal foams is explored in full. In the fourth chapter, by G. M. Steeves and M. R. Freeman, we return to microscopy: a form of scanning probe microscopy that allows extremely rapid xi
xii
PREFACE
changes to be followed. This is very useful for the study of electron transport in transistors or nanowires, for example, or in DNA or for the observation of transitions in magnetic materials. The authors describe the various stages through which the technique has passed on its way to the remarkable performances being achieved today. Many examples are included, from which potential users of these instruments can assess the suitability of ultra-fast STM to address their own problems. Digitized images are large, commonly redundant data-sets and they frequently have to be transmitted along noisy channels. Somewhat surprisingly, despite all the effort that has been put into concatenated and Turbo codes, it is the low-density parity-check codes that have reached the best performance and these are the subject of the chapter by R. Vicente, D. Saad and Y. Kabashima. After extensive introductory material, which will enable the reader to comprehend the present situation in coding theory, the authors examine in detail four of the low-density parity-check codes, which perform particularly impressively: the Sourlas, Gallagher, MacKay–Neal and cascading codes. This chapter is an invaluable introduction to modern thinking in coding methods. This brings us to the final chapter, in which S. Zaefferer discusses automation of the interpretation of electron diffraction patterns. The existence of programs for performing this task is transforming diffraction pattern analysis, which daunted all but the experts in the past. Here, the author covers eight topics in detail: the determination of crystal orientation from single-crystal diffraction patterns; automatic evaluation of spot and Kikuchi patterns; techniques for determining orientation based on dark-field images or Debye–Scherrer ring patterns; simulation of diffraction patterns; grain boundary characterization; determination of the Burgers vector, crystallographic line direction, slip plane and nature of dislocations; phase identification and lattice-parameter fitting; and calibration of microscope operating characteristics (camera length, accelerating voltage and rotation). All this is extensively illustrated. In conclusion, I thank all the contributors for the care they have brought to their chapters and their efforts to ensure that their material, however complicated, is accessible to newcomers to the subject. Peter Hawkes
FUTURE CONTRIBUTIONS
T. Aach Lapped transforms G. Abbate New developments in liquid-crystal-based photonic devices S. Ando Gradient operators and edge and corner detection A. Arnéodo, N. Decoster, P. Kestener and S. Roux (vol. 126) A wavelet-based method for multifractal image analysis C. Beeli (vol. 127) Structure and microscopy of quasicrystals I. Bloch Fuzzy distance measures in image processing G. Borgefors Distance transforms B. L. Breton, D. McMullan and K. C. A. Smith (Eds) Sir Charles Oatley and the scanning electron microscope A. Bretto Hypergraphs and their use in image modelling Y. Cho (vol. 127) Scanning nonlinear dielectric microscopy E. R. Davies (vol. 126) Mean, median and mode filters H. Delingette Surface reconstruction based on simplex meshes A. Diaspro (vol. 126) Two-photon excitation in microscopy R. G. Forbes Liquid metal ion sources
xiii
xiv
FUTURE CONTRIBUTIONS
E. Förster and F. N. Chukhovsky X-ray optics A. Fox The critical-voltage effect L. Frank and I. Müllerová Scanning low-energy electron microscopy L. Godo & V. Torra Aggregation operators A. Hanbury Morphology on a circle P. W. Hawkes (vol. 127) Electron optics and electron microscopy: conference proceedings and abstracts as source material M. I. Herrera The development of electron microscopy in Spain J. S. Hesthaven (vol. 127) Higher-order accuracy computational methods for time-domain electromagnetics K. Ishizuka Contrast transfer and crystal images G. Kögel Positron microscopy N. Krueger The application of statistical and deterministic regularities in biological and artificial vision systems A. Lannes (vol. 126) Phase closure imaging B. Lahme Karhunen-Lo`eve decomposition B. Lencová Modern developments in electron optical calculations M. A. O’Keefe Electron image simulation
FUTURE CONTRIBUTIONS
N. Papamarkos and A. Kesidis The inverse Hough transform M. G. A. Paris and G. d’Ariano Quantum tomography T.-c. Poon (vol. 126) Scanning optical holography E. Rau Energy analysers for electron microscopes H. Rauch The wave-particle dualism D. de Ridder, R. P. W. Duin, M. Egmont-Petersen, L. J. van Vliet and P. W. Verbeek (vol. 126) Nonlinear image processing using artificial neural networks O. Scherzer Regularization techniques G. Schmahl X-ray microscopy S. Shirai CRT gun design methods T. Soma Focus-deflection systems and their applications I. Talmon Study of complex fluids by transmission electron microscopy M. Tonouchi Terahertz radiation imaging N. M. Towghi Ip norm optimal filters Y. Uchikawa Electron gun optics D. van Dyck Very high resolution electron microscopy K. Vaeth and G. Rajeswaran Organic light-emitting arrays
xv
xvi
FUTURE CONTRIBUTIONS
C. D. Wright and E.W. Hill Magnetic force microscopy F. Yang and M. Paindavoine (vol. 126) Pre-filtering for pattern recognition using wavelet transforms and neural networks M. Yeadon (vol. 127) Instrumentation for surface studies
ADVANCES IN IMAGING AND ELECTRON PHYSICS, VOL. 125
An Algebraic Approach to Subband Signal Processing MARILENA BARNABEI AND LAURA B. MONTEFUSCO Department of Mathematics, University of Bologna, I-40127 Bologna, Italy
I. Introduction . . . . . . . . . . . . . . . . . . . . . . II. An Overview of Recursive Matrices . . . . . . . . . . . . . A. Basics . . . . . . . . . . . . . . . . . . . . . . . . B. Recursive Matrices . . . . . . . . . . . . . . . . . . C. Monomial Recursive Matrices . . . . . . . . . . . . . . D. Matrix Representation of Operations on Laurent Polynomials . III. Recursive Operators of Filter Theory . . . . . . . . . . . . A. Basic Linear Operators . . . . . . . . . . . . . . . . . B. Other Decimation Operators . . . . . . . . . . . . . . C. Interchanging and Combining Basic Operators . . . . . . . IV. More Algebraic Results . . . . . . . . . . . . . . . . . . A. Block Toeplitz Matrices . . . . . . . . . . . . . . . . B. Products of Hurwitz Matrices . . . . . . . . . . . . . . V. Analysis and Synthesis Filter Banks . . . . . . . . . . . . . A. Analysis and Synthesis Operators . . . . . . . . . . . . B. Matrix Description of Analysis Filter Banks . . . . . . . . C. Matrix Description of Synthesis Filter Banks . . . . . . . VI. M-Channel Filter Bank Systems . . . . . . . . . . . . . . A. Matrix Description . . . . . . . . . . . . . . . . . . B. Alias-Free Filter Banks . . . . . . . . . . . . . . . . C. Perfect Reconstruction Filter Banks . . . . . . . . . . . VII. Transmultiplexers . . . . . . . . . . . . . . . . . . . . A. Matrix Description . . . . . . . . . . . . . . . . . . B. Perfect Reconstruction Transmultiplexers . . . . . . . . . VIII. The M-Band Lifting . . . . . . . . . . . . . . . . . . . A. Algebraic Preliminaries . . . . . . . . . . . . . . . . B. Lifting and Dual Lifting . . . . . . . . . . . . . . . . C. The Algorithms . . . . . . . . . . . . . . . . . . . . D. Factorization into Lifting Steps . . . . . . . . . . . . . IX. Conclusion . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1 4 4 7 9 12 13 13 15 19 22 23 26 30 30 33 35 37 37 40 42 44 45 46 48 49 53 56 56 60 61
I. Introduction In recent years, subband filter theory has been developed considerably, due to its increasing number of applications, mainly in telecommunications and signal processing. More specifically, this theory has been fruitfully exploited in 1 Copyright 2002, Elsevier Science (USA). All rights reserved. ISSN 1076-5670/02 $35.00
2
BARNABEI AND MONTEFUSCO
such areas as speech and image compression, coding of high-resolution video and audio, and advanced television. Moreover, the connection with multiresolution signal analysis, used, for example, in pattern analysis and computer vision, indicates the role played by filter bank theory even outside the previous application fields. The performance of application systems based on filter banks is significantly affected both by the properties of the individual analysis and synthesis filters and by the structure and quality of the overall system. This implies that the design of such systems requires imposing global conditions—such as alias or cross-talk cancelation and perfect reconstruction—as well as individual conditions on the filters involved, as, for example, passband or stopband deviation, phase characteristics, and the computational efficiency of their implementation. Hence, the design of M-band filter bank systems represents a challenging problem, due to the great number of constraints involved. In addition, the difficulties grow as soon as the number of bands taken into consideration increases, and filters are required to meet a general set of prespecified properties. Most of the early theoretical developments and design procedures of Mband filter bank systems were based on the so-called transform domain formulation (see, e.g., Vaidyanathan, 1993; Vetterli and Kovacevic, 1995; Woods, 1991; and references therein). This approach gives an explicit description of the global properties of the system, hence aliasing, magnitude, and phase distorsions can be controlled in the design. Transform domain formulation uses two different, but equivalent, analysis methods, called alias-component formulation and polyphase formulation. The first one yields simple conditions for alias-free and perfect reconstruction properties, but it does not provide a mechanism for choosing the analysis filters in order for the synthesis filters to be acceptable. Polyphase formulation partially overcomes the above-mentioned drawbacks, and permits great simplification of the theoretical results, leading at the same time to computationally efficient implementations of filter banks. However, this formulation does not provide a mechanism for choosing individual filters with suitable properties, since these constraints are not given in the polyphase domain, but on the filters themselves. Time-domain formulation was recently introduced to deal with this problem (see, e.g., Nayebi, Barnwell, and Smith, and references therein). Indeed, it has two major advantages. Firstly, many different systems can be formulated in exactly the same way, resulting in the same design procedure. Secondly, it offers the flexibility to balance the tradeoff between competing system features, such as reconstruction error and filter quality. Nevertheless, this formulation cannot provide the simplicity and elegance of the polyphase approach, since it only works in the real domain. This is due to the fact that the essence of the time-domain formulation is a set of bi-infinite matrix equations relating output samples to input samples, and that the full potential of the structure of these
SUBBAND SIGNAL PROCESSING
3
matrices and their products has not as yet been exploited. The main contribution of the present work and of the research papers which preceded it (Barnabei and Montefusco, 1998; Barnabei et al., 1998, 2000) consists in presenting a theoretical basis that allows us to study filter banks from the time-domain point of view, while having a well-structured algebraic framework at our disposal. This allows us to explicitly characterize the bi-infinite matrices describing the action of analysis and synthesis filter banks, together with their combined action. Moreover, it provides the possibility of completely recovering all the results obtained from the polyphase approach, while still working in the real domain. In addition, the purely algebraic language of this approach allows us to give full mathematical justification to several computational rules widely used in practical applications. More precisely, the two cornerstones on which our presentation is based are the recursive matrix theory and the algebra of Laurent polynomials. A banded recursive matrix is a bi-infinite matrix that can be completely described by two Laurent polynomials, and the same happens for the product of recursive matrices. This allows us to switch from the context of banded bi-infinite matrices to the algebra of Laurent polynomials, taking advantage of its abundant properties. The recursive matrix machinery can then be fruitfully used to represent and easily handle some linear operators—acting on finite or infinite sequences—that are widely used in filter theory. In fact, such linear operators are associated with particular bi-infinite Toeplitz or Hurwitz matrices, which are special cases of recursive matrices. For example, the operation of convolution followed by decimation—a typical filtering operation—can be represented by means of a Hurwitz matrix, while the upsampling operation corresponds to the transpose of a Hurwitz matrix. Hence, the behavior of these linear operators can be studied through an analysis of the properties of the products of Hurwitz and Toeplitz matrices and their transposes. On the other hand, this matrix based approach allows us to recover the elegant and simple results of polyphase formulation by exploiting the possibility of describing a recursive matrix in terms of the generating function of its rows. The basic step in this context is the bijection between a Laurent polynomial and the M-tuple of its decimated polynomials, namely, a complete set of its subsampled versions. In Barnabei et al. (1998), we reinterpreted the analysis and synthesis filter banks in the previous algebraic setting, giving an explicit and computationally efficient description of their action. Moreover, we showed that the operator describing an analysis/synthesis system is associated with a block Toeplitz matrix, and that the alias cancelation condition is equivalent to requiring that such a matrix is also a “scalar” Toeplitz matrix. In this way, we managed to obtain a simple algebraic characterization for alias-free and perfect reconstruction analysis/synthesis systems, handling the 2- and M-band cases with equal
4
BARNABEI AND MONTEFUSCO
ease. A similar characterization can be given for transmultiplexing systems, as we show in the present paper. The algebraic tools we have developed not only allow us to control the global properties of filter bank systems but also enable us to impose constraints on the filters themselves, namely, on the coefficients of the corresponding Laurent polynomials. However, it is computationally cumbersome to impose both of these conditions directly. In Barnabei et al. (2000), we show how it is possible to reduce this complexity, by constructing in various consecutive steps (lifting steps) perfect reconstruction systems whose filters satisfy prespecified requirements. This construction makes use only of linear combinations of Laurent polynomials, therefore dramatically reducing the computational complexity of designing M-band systems with good filtering properties. The present paper is organized as follows. In Sections II and IV we give a detailed description of the algebraic notions and results that constitute the theoretical foundations of the present work. In Sections III and V we describe the action of the operators that are the building blocks of filter bank theory in recursive matrix notation, and we obtain equivalent results in terms of decimated polynomials, which correspond to the polyphase components. Sections VI and VII are devoted to the study of M-band filter banks and transmultiplexers. We give characterizations of alias and cross-talk cancelation conditions and perfect reconstruction systems, both in terms of M-decimated matrices and in terms of Hurwitz matrices. Finally, in Section VIII we present the extension to the M-band case of the lifting scheme, a powerful tool, yielding an easy custom-designed construction of biorthogonal filters with preassigned features. By fully exploiting the properties of the algebra of Laurent polynomials, we succeed in describing the building aspects of the lifting scheme, and its decomposition aspects, hence obtaining a factorization procedure that leads to faster analysis–synthesis algorithms.
II. An Overview of Recursive Matrices A. Basics In this section we recall the basic definitions and properties of Laurent series which will be widely used in the rest of the paper. Let K be any field, and K Z the K -vector space of bi-infinite sequences in K , (ai ), with i ∈ Z. We associate to the sequence (ai )i∈Z its generating function, namely, the formal series α := ai t i i∈Z
where t is a formal variable.
SUBBAND SIGNAL PROCESSING
5
For every integer i, the coefficient ai of t i in α will be also denoted by i|α. In the following, we give as examples the most commonly used sequences with their respective generating functions. 1. The generating function of the infinite sequence whose elements are all equal to zero is the zero-series ζ := 0t i . i∈Z
2. The generating function of the infinite sequence (δi,0 )i∈Z , where δ is the Kronecker symbol, is the one-series, namely, the series υ := δi,0 t i . i∈Z
3. The generating function of the sequence (δi,1 )i∈Z is the identity series t, and, in general, the generating function of the sequence (δi,k )i∈Z is the series t k . The correspondence between a sequence of K Z and its generating function is a bijection. Hence, we shall often identify bi-infinite sequences in K and formal series. Moreover, if α is a formal series, we shall denote by [α] the bi-infinite column matrix whose elements are the coefficients of α. A Laurent series is a formal series α := ai t i i∈Z
in which only a finite number of coefficients ai with negative index i are nonzero, namely, an index h exists such that i < h ⇒ ai = 0. The minimum index d such that ad = 0 is the degree deg(α) of the Laurent series α. The degree of the zero-series ζ is conventionally defined to be −∞. For example, the series υ is a Laurent series of degree zero, while the series t is a Laurent series of degree 1. We recall that thesum α + β andthe (convolution) product αβ of two Laurent series α := i≥h ai t i , β := i≥k bi t i are the Laurent series defined as follows: α+β := (ai + bi )t i , i
αβ : =
i≥h+k
ci t i ,
6
BARNABEI AND MONTEFUSCO
where, for every index i, ci :=
i−k
a j bi− j .
j=h
Note that deg(α + β) = min(deg(α), deg(β)), while deg(αβ) = deg(α) + deg(β). The set L+ of all Laurent series turns out to be a field with respect to the operations previously defined. In particular, the identity elements for sum and product are the series ζ and υ, respectively, and, for every nonzero series α, its reciprocal series α −1 exists, namely, a Laurent series such that αα −1 = υ = α −1 α. Let α := i ai t i , β := j b j t j be Laurent series such that β has positive degree; the composition α ◦ β is defined as the Laurent series: i j i α(β) = α ◦ β := bjt . ai β = ai i
i
j
Note that deg(α ◦ β) = deg(α) · deg(β). The identity element with respect to composition is the series t. It is easily seen that a Laurent series α admits compositional inverse α, ˜ namely, another Laurent series such that α ◦ α˜ = t = α˜ ◦ α, whenever deg(α) = 1. In Figure 1, we give some examples of sequences in R Z whose generating functions are Laurent series. Note that the zeroth entry is boldface. The duality map is defined as the map T : K Z → K Z,
T(ai ) = (a−i ).
The map T is an involution and turns out to be a vector space automorphism. If α is the generating function of the sequence (ai ), the generating function of
Figure 1. Examples of Laurent series.
SUBBAND SIGNAL PROCESSING
7
the dual sequence T(ai ) will be denoted by the symbol α*. Hence: α := ai t i ⇐⇒ α ∗ := a−i t i = ai t −i . i
i
i
+
The maximum subspace P of L that is invariant under the action of the duality map is the subspace of Laurent series with only a finite number of nonzero coefficients. The elements of P are called Laurent polynomials. We remark that the duality map α → α ∗ , when restricted to the subspace of Laurent polynomials, can be represented as follows: T(α) = α ◦ t −1 . In fact, even if the composition of a Laurent series α with the series t −1 is not recovered under the general definition, it makes sense in the special case in which α is a Laurent polynomial, and once again yields a Laurent polynomial.
B. Recursive Matrices We will be concerned here with the K -vector space M of bi-infinite matrices over the field K , namely, matrices M : Z × Z → K . Let M = [m i j ] be a matrix in M. For every i ∈ Z, the generating functions of the ith row of M will be denoted by M(i): M(i) = mi j t j . j
We now come to the definition of a recursive matrix, which can be loosely described as a bi-infinite matrix that can be completely determined by two series, the first of which gives the recurrence rule, while the second provides the boundary conditions. The formal definition of a recursive matrix can be given in several equivalent terms, each one of which highlights a different feature pre sented by these matrices. More precisely, let α = j≥d a j t j , β = j≥h b j t j be nonzero Laurent series. The (α, β)-recursive matrix is the unique matrix M ∈ M defined by the following equivalent conditions: r r
M(i) = α i β for every integer i. For every integer j, we have:
* m0 j = b j , * for i > 0, the element m i, j can be recursively computed from the elements of the preceding row of M, namely, as m i−1, j−s , (1) m i, j = s≥d
8
BARNABEI AND MONTEFUSCO
* for i < 0, the element m i, j can be computed from the elements of the following row of M, namely, a¯ s m i+1, j−s , (2) m i, j = s≥−d
where the a¯ s are the coefficients of the series α −1 , the reciprocal series of α. r
M is the unique solution, in M, of the following linear problem: F M G = M;
A(0) = β
(3)
where F is the bi-infinite forward shift matrix, while G is the bi-infinite Toeplitz matrix with generating function α. The (α, β)-recursive matrix will be denoted by R(α, β) and the series α, β will be called the recurrence rule and boundary value of R(α, β), respectively. Example II.1 In Figure 2, we show the recursive matrix R(α, υ), where α = 1 + t. The zeroth row and column are boldface. Note that the southeast section of the matrix is simply the Pascal triangle. The main property of recursive matrices is the fact that—under suitable conditions—they can be multiplied, and the product is again a recursive matrix, whose recurrence rule and boundary value can be explicitly described. More precisely, we have:
Figure 2. The “Pascal” matrix R(1 + t, υ).
9
SUBBAND SIGNAL PROCESSING
Theorem II.1 (Product Rule) Let α, β, γ , δ be nonzero Laurent series, and let γ have positive degree. Then, R(α, β) × R(γ , δ) = R(α ◦ γ , (β ◦ γ )δ). Proof. Set R(α, β) = P = ( pi j ), R(γ , δ) = Q = (qi j ), M = P × Q. For every integer h, we have M(h) = ph j q ji t i = ph j Q( j) = ph j γ j δ i
j
j
j
h
= (P(h) ◦ γ )δ = ((α β) ◦ γ )δ = (α ◦ γ )h (β ◦ γ )δ,
which gives the assertion. The Product Rule immediately yields an explicit description of the recursive inverse of a recursive matrix (if it exists). Proposition II.1 Let α, β be nonzero Laurent series, with deg(α) = 1. Then the recursive matrix R(α, β) is invertible and its inverse is the recursive matrix R(α, ˜ (β ◦ α) ˜ −1 ). In other words, every recursive matrix whose recurrence rule has degree 1 admits a recursive inverse.
C. Monomial Recursive Matrices A notable class of recursive matrices is the class of recursive matrices whose recurrence rule is a monomial, namely, a Laurent series of the kind k α = t k , where k is any nonzero integer. If M = R(t , β) is such a matrix, with β = i bi t i , then, according to formula (1), the element m i j of M is given by m i j = b j−ki . We point out that this class of recursive matrices recovers three well-known types of bi-infinite structured matrices, namely: r
r
r
A Toeplitz matrix with generating function β is the recursive matrix R(t, β) (see Fig. 3). A Hurwitz matrix of step k and generating function β is the recursive matrix R(t k , β) (see Fig. 4). A Hankel matrix with generating function β is the recursive matrix R(t −1 , β) (see Fig. 5).
10
BARNABEI AND MONTEFUSCO
Figure 3. The Toeplitz matrix R(t, β).
These three classes of matrices are of special interest since they reveal a particularly pleasant algebraic behavior, and they are a crucial instrument in the algebraic theory of signal processing. For example, by multiplying a general recursive matrix by a Toeplitz matrix we succeed in changing the boundary value without affecting its recurrence rule. Hence, every recursive matrix can be seen as the product of a recursive matrix whose boundary value is υ, and a Toeplitz matrix. In fact, by the Product Rule, we get: Proposition II.2 Let α, β be nonzero Laurent series. Then the recursive matrix R(α, β) admits the following factorization: R(α, β) = R(α, υ) × R(t, β). As we have already remarked, Toeplitz, Hurwitz, and Hankel recursive matrices are widely used in the applications, together with their products and powers. For this reason, we summarize some important results on these matrices, which follow immediately from the Product Rule. Corollary II.1 The product of two Toeplitz recursive matrices is again a Toeplitz matrix, whose generating function is the product of the two generating
Figure 4. The Hurwitz matrix R(t 2 , β).
SUBBAND SIGNAL PROCESSING
11
Figure 5. The Hankel matrix R(t −1 , β).
functions: R(t, α) × R(t, β) = R(t, αβ). In particular, two Toeplitz recursive matrices always commute. Corollary II.2 For every positive integer p, the pth power of the Toeplitz recursive matrix R(t, α) is the Toeplitz matrix R(t, α p ). Corollary II.3 The recursive inverse of the Toeplitz recursive matrix R(t, β) is the Toeplitz matrix R(t, β −1 ). Corollary II.4 The products of Toeplitz and Hurwitz recursive matrices are Hurwitz matrices; more precisely: R(t, α) × R(t k , β) = R(t k , (α ◦ t k )β); R(t k , β) × R(t, α) = R(t k , αβ).
Corollary II.5 The product of two Hurwitz recursive matrices is again a Hurwitz matrix with a different step: R(t h , α) × R(t k , β) = R(t h·k , (α ◦ t k )β). Corollary II.6 For every positive integer p, the pth power of the Hurwitz recursive matrix R(t k , α) is a Hurwitz matrix with a different step: p
R(t k , α) p = R(t k , β) where β=α
p−1 i=1
α◦t
ki
.
To give further results concerning Hankel matrices, we now restrict ourselves to the banded case, namely, we consider monomial recursive matrices
12
BARNABEI AND MONTEFUSCO
whose boundary value is a Laurent polynomial. These matrices will play a fundamental role in the applications we are going to consider. For such matrices, the Product Rule can be extended to the case when the recurrence rule of the second matrix is a monomial of negative degree. For example, we get the following results for the products of Toeplitz and Hankel banded matrices: Corollary II.7 The products of banded Toeplitz and Hankel matrices are Hankel matrices; more precisely: R(t, α) × R(t −1 , β) = R(t −1 , α ∗ β);
R(t −1 , β) × R(t, α) = R(t −1 , αβ).
Corollary II.8 The product of two banded Hankel matrices is a Toeplitz matrix: R(t −1 , α) × R(t −1 , β) = R(t, α ∗ β). As a consequence, the pth power of a banded Hankel matrix is either a Toeplitz or a Hankel matrix, according to the parity of the integer p: Corollary II.9 For every positive integer h, the (2h)th power of the Hankel recursive matrix R(t −1 , α) is a Toeplitz matrix: R(t −1 , α)2h = R(t, α h α ∗h ), while the (2h + 1)th power of R(t −1 , α) is a Hankel matrix: R(t −1 , α)2h+1 = R(t −1 , α h+1 α ∗h ).
D. Matrix Representation of Operations on Laurent Polynomials It is worth remarking that it is possible to represent some operations on Laurent polynomials by means of suitable recursive matrices using the Product Rule. For example: 1. The product of two Laurent polynomials α, β can be seen as the row vector [αβ]T = [α]T × R(t, β) = [β]T × R(t, α).
(4)
or, equivalently, as the column vector [αβ] = R(t, β ∗ ) × [α] = R(t, α ∗ ) × [β]. In fact, since R(t, α) × R(t, β) = R(t, αβ),
(5)
SUBBAND SIGNAL PROCESSING
13
we get (4) by equating the zeroth row on both sides. Identity (5) follows from the well-known fact that the transpose of the Toeplitz matrix R(t, α) is the Toeplitz matrix R(t, α ∗ ). 2. The composition of two Laurent polynomials α, β is the row vector: [α ◦ β]T = [α]T × R(β, υ)
(6)
or, equivalently, the column vector [α ◦ β] = R(β, υ)T × [α].
(7)
In fact, again using the Product Rule, we have: R(t, α) × R(β, υ) = R(β, α ◦ β), and (6) is obtained by equating the zeroth row on both sides. 3. The dual polynomial α ∗ of a Laurent polynomial α is the column vector: [α ∗ ] = [α ◦ t −1 ] = R(t −1 , υ) × [α].
(8)
The above identity is a consequence of identity (7) and of the symmetry of Hankel matrices.
III. Recursive Operators of Filter Theory In this section, we show the connection between banded recursive matrices and some linear operators widely used in filter theory. This connection emphasizes the algebraic properties of such operators, hence leading to a simple and general description, together with a computationally efficient implementation of their combined action.
A. Basic Linear Operators This subsection is devoted to the analysis of some linear operators on the subspace P of Laurent polynomials, frequently used in practical applications, whose associated matrices reveal a recursive structure. r
Duality As we have already remarked, the duality map T : P → P can be represented as follows: T(σ ) = σ ◦ t −1 ,
(9)
where σ is any Laurent polynomial. Using identity (8), we get: [T(σ )] = [σ ∗ ] = R(t −1 , υ) × [σ ],
(10)
14
BARNABEI AND MONTEFUSCO
Figure 6. The action of the counter-identity matrix R(t −1 , υ).
that is, the duality map T is a linear operator, whose associated matrix is the Hankel matrix R(t −1 , υ) (the counter-identity matrix) (see Fig. 6). r
Filters Let α be a fixed Laurent polynomial, and Fα be the finite-length (FIR) filter corresponding to α, namely, the linear operator on the vector space P such that, for every σ ∈ P , Fα (σ ) = α ∗ σ . The action of a filter is usually represented by the following scheme:
As we have already seen (identity (5)), the matrix associated with the filter Fα is the Toeplitz matrix R(t, α), namely, [Fα (σ )] = [α ∗ σ ] = R(t, α) × [σ ].
(11)
In particular, the filter Ft h corresponding to the monomial t h is called an h-step shift and multiplies a given Laurent polynomial by t −h . When h > 0, the shift Ft h is called a delay, while when h < 0, it is called an advance. r
k-upsampler Let k be a fixed integer, k ≥ 2. The k-upsampling operator (or k-expander) Uk , represented as
is defined as follows: if σ =
i si
k−1
is a Laurent polynomial, k−1
k−1
k−1
U (σ ) = (↑ k)σ := (.. 0......0 s−1 0......0 s0 0.....0 s1 0.....0 ...). k
This is equivalent to the composition of σ with the series t k , namely, Uk (σ ) = σ ◦ t k . Hence, by identity (7), we get [Uk (σ )] = [σ ◦ t k ] = R(t k , υ)T × [σ ].
(12)
SUBBAND SIGNAL PROCESSING
15
Figure 7. The action of the matrix R(t 3 , υ)T associated with the 3-upsampling operator U3 .
In other words, Uk is a linear operator, whose associated matrix is the transposed Hurwitz matrix R(t k , υ)T (see Fig. 7). r
k-downsampler For every fixed integer k ≥ 2, the k-downsampling operator Dk0 , represented as
is defined as follows: if σ =
Dk0 (σ ) = σ0 :=
i si t
i
i
is a Laurent polynomial, ski t i = ki|σ t i . i
It is immediately clear that the matrix associated with Dk0 is the Hurwitz matrix R(t k , υ), namely, k (13) D0 (σ ) = R(t k , υ) × [σ ]. B. Other Decimation Operators The downsampling operator defined above associates to a given Laurent polynomial σ the polynomial σ0 that only partially maintains the features of the signal. In order to recover the whole information contained in σ it is necessary to associate to σ all its k-decimated polynomials, namely, the k polynomials
16
BARNABEI AND MONTEFUSCO
σ0 , σ1 , . . . , σk−1 , defined as follows: σ0 := Dk0 (σ ), σr := Dk0 (t −r σ ) for r = 1, 2, . . . , k − 1. In other words, for every r = 0, 1, . . . , k − 1, σr := ski+r t i = ki + r |σ t i . i
(14)
i
Extending the notation introduced for the downsampling operator, we denote by Drk the linear operator such that Drk (σ ) = σr for r = 0, 1, . . . , k − 1. These operators will be called the k-decimation operators. By definition, the r th k-decimation operator can be written as Drk = Dk0 Ft r
(15)
and represented by
Hence, the matrix associated with the decimation operator Drk is given by the product R(t k , υ) × R(t, t r ) that yields, using the Product Rule, the Hurwitz matrix R(t k , t r ) (see Fig. 8), namely: [Drk σ ] = [σr ] = R(t k , t r ) × [σ ].
(16)
Every k-decimated polynomial of σ represents an aliased version of the original input signal. However, by using all of this incomplete information, it is possible to recover exactly the original signal. In fact, the explicit description of every decimated polynomial in terms of the coefficients of σ given by identity (14) allows us to state the following reconstruction rule: σ = σ0 ◦ t k + · · · + t r (σr ◦ t k ) + · · · + t k−1 (σk−1 ◦ t k ),
(17)
SUBBAND SIGNAL PROCESSING
17
Figure 8. R(t 3 , t) × [σ ] = [σ1 ].
represented by the following scheme:
We have seen that the Hurwitz matrix R(t k , t r ) is associated with the operator whenever 0 ≤ r ≤ k − 1. In general, we can describe the linear operator associated with a Hurwitz matrix of the kind R(t k , t n ), without any restriction on the exponent n. More precisely, using the Product Rule, we get:
Drk
Proposition III.1 Let k, n be fixed integers, with k ≥ 2, and let q, r be the unique integers such that 0 ≤ r < k and n = kq + r . The linear operator associated with the Hurwitz matrix R(t k , t n ) is the operator Drk followed by the shift Ft q . In other words, R(t k , t n ) = R(t, t q ) × R(t k , t r ).
(18)
In Figure 8 the action of the linear operator associated with the matrix R(t 3, t) is shown. Lastly, it is useful—in view of further applications—to describe the k-decimated series of the product of two given series.
18
BARNABEI AND MONTEFUSCO
Proposition III.2 Let k ≥ 2 be a fixed integer and α, β two Laurent series. The zeroth k-decimated series of the product αβ is Dk0 (αβ) = α0 β0 + t
k−1
αi βk−i ,
i=1
and, for 0 < r < k, the r th k-decimated series of αβ is Drk (αβ) =
r i=0
αi βr −i + t
k−1
i=r +1
αi βr +k−i .
where αi , βi denote the ith k-decimated series of α, β, respectively. Proof. By identity (17), we have αβ = (α0 ◦ t k + · · · + t k−1 (αk−1 ◦ t k ))(β0 ◦ t k + · · · + t k−1 (βk−1 ◦ t k )) =
k−1
i, j=0
(t i+ j ((αi β j ) ◦ t k )).
For i, j = 0, . . . , k − 1, we have
⎧ ⎨α0 β0 if i = j = 0 Dk0 (t i+ j ((αi β j ) ◦ t k )) = tαi β j if i + j = k ⎩0 otherwise
The first identity now follows by linearity. The second identity can be proved by similar arguments. Example III.1 Let α = a−1 t −1 + a0 + a1 t,
β = b−1 t −1 + b0 + b1 t.
The 3-decimated series of α and β are, respectively: α0 = a 0 ,
α1 = a 1 ,
β0 = b0 ,
β1 = b1 ,
α2 = a−1 t −1 ,
β2 = b−1 t −1 .
We have: αβ = a−1 b−1 t −2 + (a−1 b0 + a0 b−1 )t −1 + (a0 b0 + a−1 b1 + a1 b−1 ) + (a0 b1 + a1 b0 )t + a1 b1 t 2 ;
D30 (αβ) = a0 b0 + a−1 b1 + a1 b−1 = α0 β0 + tα1 β2 + tα2 β1 ;
D31 (αβ) = a−1 b−1 t −1 + a0 b1 + a1 b0 = α0 β1 + α1 β0 + tα2 β2 .
SUBBAND SIGNAL PROCESSING
19
C. Interchanging and Combining Basic Operators The general results about recursive matrices stated in the previous sections allow us to examine the interplay between the basic operators of filter theory. First of all, we study the conditions for interchanging the duality operator T and the k-decimation operators Drk . These operators do not commute unless r = 0, as shown in the next proposition: Proposition III.3 Let k be a fixed integer, k ≥ 2. We have TDk0 = Dk0 T,
and, for every integer r, 0 < r < k, TDrk = Ft −1 Dkk−r T, that is, for every Laurent polynomial σ, (σ0 )∗ = (σ ∗ )0 ,
(19)
(σr )∗ = t(σ ∗ )k−r .
(20)
and, for r > 0,
Proof. The operator
TDk0
is associated with the matrix
R(t −1 , υ) × R(t k , υ) = R(t −k , υ),
and Dk0 T is associated with
R(t k , υ) × R(t −1 , υ), which gives the same recursive matrix. For r > 0, the operator TDrk is associated with the matrix R(t −1 υ) × R(t k , t r ) = R(t −k , t r ),
while Dkk−r T is associated with the matrix
R(t k , t k−r ) × R(t −1 , υ) = R(t −k , t r −k ). The assertion now follows by noting that R(t, t −1 ) × R(t −k , t r −k ) = R(t −k , t k t r −k ) = R(t −k , t r ) and recalling that R(t, t −1 ) is the matrix associated with the shift Ft −1 (see Eq. (11)). Example III.2 Set k = 3, and let
σ = · · · + s−2 t −2 + s−1 t −1 + s0 + s1 t + s2 t 2 + s3 t 3 + s4 t 4 + · · ·
20
BARNABEI AND MONTEFUSCO
be a Laurent polynomial; then D31 σ = σ1 = · · · + s−2 t −1 + s1 + s4 t + · · · , and TD31 σ = (σ1 )∗ = · · · + s4 t −1 + s1 + s−2 t + · · · . On the other hand, σ ∗ = · · · + s4 t −4 + s3 t −3 + s2 t −2 + s1 t −1 + s0 + s−1 t + s−2 t 2 + · · · and D32 Tσ = (σ ∗ )2 = · · · + s4 t −2 + s1 t −1 + s−2 + · · · ; hence, (σ1 )∗ = t(σ ∗ )2 .
For the sake of simplicity, from now on we will write σr∗ for (σr )∗ . The problem of interchanging the k-decimation operators Drk and the h-step shift can easily be handled by a systematic use of the Product Rule. We give below a simple example of this kind of result: Proposition III.4 Let k ≥ 2 be a fixed integer, and σ a Laurent polynomial. We have: Ft h Drk (σ ) = Dk0 Ft n (σ )
(21)
where n = hk + r . Proof. The matrix associated with the left-hand side operator is R(t, t h ) × R(t k , t r ) = R(t k , t hk+r ) = R(t k , υ) × R(t, t n ),
and the last one is precisely the matrix associated with the operator Dk0 Ft n . The preceding proposition can be illustrated by the following scheme:
We now consider two results largely used in signal manipulation, namely, the two Noble identities (Strang and Nguyen, 1996). The first Nobel identity gives a commutation rule for the product of a filter and a decimator, while the second one gives a similar rule for the product of a filter and an expander. These identities can be rewritten in terms of recursive matrices, so that their proof is an immediate consequence of the factorization property of recursive
SUBBAND SIGNAL PROCESSING
21
matrices (Proposition II.2) and Corollary II.4. More precisely, the first Noble identity, Fα Dk0 = Dk0 Fα◦t k ,
(22)
represented by the scheme
corresponds to the equality: R(t, α) × R(t k , υ) = R(t k , α ◦ t k ) = R(t k , υ) × R(t, α ◦ t k ), while the second Noble identity, Uk Fα = Fα◦t k Uk ,
(23)
represented by
corresponds to the following equality: R(t k , υ)T × R(t, α) = (R(t k , α ∗ ◦ t k ))T = R(t, α ◦ t k ) × R(t k , υ)T . We notice that the present approach, based on the correspondence between linear operators and recursive matrices, allows us to state a further identity that generalizes the results of both (21) and (22), whose proof is straightforward: Proposition III.5 Let k ≥ 2 be a fixed integer, and α a Laurent polynomial. We have: Fα Drk = Drk Fα◦t k = Dk0 Ft r (α◦t k ) .
(24)
The matrix-based presentation of the basic operators provides an effective way to describe the building blocks of a k-channel maximally decimated filter bank system. In fact, these are linear operators—analysis and synthesis operators—that can be expressed as the composition of filters with decimators or expanders. Such operators can therefore be represented in terms of banded recursive matrices. In particular, the analysis operator consists of a filter followed by a k-decimator, according to the following scheme:
22
BARNABEI AND MONTEFUSCO
This yields the operator Dk0 Fγ , whose associated matrix is the Hurwitz matrix R(t k , υ) × R(t, γ ) = R(t k , γ ).
(25)
The synthesis operator is the combined action of a k-expander followed by a filter:
which yields the operator Fα∗ Uk . Once again using the Product Rule, the synthesis operator corresponds to the transposed Hurwitz matrix R(t k , α)T . In fact, R(t, α ∗ ) × R(t k , υ)T = R(t, α)T × R(t k , υ)T = (R(t k , υ) × R(t, α))T = R(t k , α)T .
(26)
In practical applications of filter theory, the analysis of an input signal σ is followed by a synthesis that produces the output signal. It is therefore important to be able to describe, in terms of recursive matrices, the action of the analysis–synthesis operator:
namely, the operator Fα∗ Uk Dk0 Fγ . Making use of the preceding considerations, it immediately follows that this operator corresponds to the product R(t k , α)T × R(t k , γ ).
(27)
IV. More Algebraic Results In order to perform a detailed study of filter bank systems we need some further results about monomial matrices. More precisely, we introduce the notion of block Toeplitz matrix and study its main features. Furthermore, we succeed in giving an explicit expression for the product of a banded Hurwitz matrix and
SUBBAND SIGNAL PROCESSING
23
a transposed banded Hurwitz matrix and vice versa. These results will allow us to give a compact and efficient description of the action of the analysis and synthesis operators together with their combined action. A. Block Toeplitz Matrices The notion of recursive matrix can be extended to the case of matrices whose entries are square matrices of fixed order k, yielding the notion of block recursive matrix. The definition and the main properties of such matrices are formally almost the same as in the scalar case, even if in the block case the sboundary value is a block Laurent polynomial, that is, a polynomial = i=−h P (i) t i (i) whose coefficients are k × k square matrices P (i) = [ plm ]: ⎞ ⎛ (−h) (−h) p00 p01 ··· ⎟ −h ⎜ (−h) = ⎝p ··· ··· ⎠t 10 (−h) ··· ··· pk−1,k−1 ⎞ ⎞ ⎛ (s) (s) ⎛ (−h+1) (−h+1) p00 p01 ··· p00 p01 ··· ⎟ s ⎟ −h+1 ⎜ ⎜ · · · + ⎝ p (s) · · · + ⎝ p (−h+1) · · · ··· ⎠t . ··· ⎠t 10 10 (s) (−h+1) · · · · · · pk−1,k−1 ··· ··· pk−1,k−1
Sometimes it will be useful to see the block Laurent polynomial as a polynomial matrix, namely, a matrix whose elements are scalar Laurent polynomials: ⎛ ⎞ π00 π01 ··· ⎟ ⎜ = ⎝π10 · · · ··· ⎠, · · · · · · πk−1,k−1 where
−h πi j = pi(−h) + pi(−h+1) t −h+1 + · · · · · · + pi(s)j t s . j t j
A detailed study of block recursive matrices, together with some applications to multiwavelet function theory, can be found in Bacchelli (1999) and Bacchelli and Lazzaro (2001). In the present paper, for the sake of simplicity, we will restrict our discussion to the case of block Toeplitz s matrices. Given a block Laurent polynomial = i=−h P (i) t i , the block Toeplitz matrix with generating function is defined as the bi-infinite matrix A, whose elements Ai j are k × k square matrices, such that Ai j = P ( j−i) for every pair of indices i, j, or, equivalently, A is the block recursive matrix R(t, ). Every block matrix can clearly be seen also as a scalar matrix, and vice versa. However, in general, a block Toeplitz matrix does not maintain its recursive structure when viewed as a scalar matrix. Nevertheless, given a k × k block
24
BARNABEI AND MONTEFUSCO
Toeplitz matrix, it is possible to describe the scalar generating function of its nth (scalar) row as follows: Proposition IV.1 Let A be the k × k block Toeplitz whose generating s matrix function is the block Laurent polynomial = i=−h P (i) t i . For every integer n, let r, q be the unique integers such that 0 ≤ r < k and n = kq + r . Then the scalar generating function of the nth row of A is given by: k−1 j k kq A(n) = t t (πr j ◦ t ) . (28) j=0
Proof. Since n = kq + r , the elements of the nth scalar row of A can be found in the r th row of the qth block row, whose generating function is t q . Hence: k−1 k−1 s s (i) ik (i) j+ik kq j kq t A(n) = t pr j t . pr j t =t i=−h j=0
j=0
i=−h
The assertion now follows from the fact that s pr(i)j t ik = πr j ◦ t k . i=−h
The preceding result allows us to give the following necessary and sufficient condition for a block Toeplitz matrix to be also a scalar Toeplitz matrix: Theorem IV.1 Let A be a (k × k)-block Toeplitz matrix whose generating function is the polynomial matrix . The matrix A is a scalar Toeplitz matrix if and only if the polynomial matrix is a t-circulant matrix, namely, it has the form ⎡ ⎤ φ0 φ1 φ2 ... φk−1 φ0 φ1 ··· φk−2 ⎥ ⎢tφk−1 ⎢ ⎥ tφ tφ φ · · · φ ⎢ k−1 0 k−3 ⎥ . = ⎢ k−2 (29) .. ⎥ .. .. .. ⎣ ... ⎦ . . . . tφ1 tφ2 · · · tφk−1 φ0
If this is the case, the generating function of the scalar Toeplitz matrix A is given by φ = φ0 ◦ t k + tφ1 ◦ t k + · · · + tk−1 φ k−1 ◦ t k , that is, for i = 0, 1, . . . , k − 1, φi = Dik φ.
SUBBAND SIGNAL PROCESSING
25
Proof. We point out that the scalar Toeplitz matrix R(t, φ) can always be seen as the (k × k)-block Toeplitz matrix R(t, ), whose block generating function is the polynomial matrix ⎛ ⎞ π00 π01 ··· ··· ⎠, = ⎝π10 · · · · · · · · · πk−1,k−1
where the jth element of each row of is the jth k-decimated polynomial of the generating function of the corresponding row of the scalar Toeplitz matrix, that is, for every j = 0, 1, . . . , k − 1, πi j = Dkj (t i φ) = Dkj Ft −i φ
i = 0, 1, · · · , k − 1,
or equivalently, in matrix notation, [πi j ] = R(t k , t j ) × R(t, t −i ) × [φ] = R(t k , t j−i ) × [φ]. For every pair of indices i, j, let q, r be the unique integers such that 0 ≤ r ≤ k − 1 and j − i = kq + r . Recalling that 0 ≤ i, j ≤ k − 1, we get ⎧ 0 ⇒r = j −i ⎪ ⎨ for i ≤ j . q= ⇒r =k + j −i ⎪ ⎩−1 for i > j
By using relation (18), we get k r for i ≤ j R(t , t ) × [φ] , [πi j ] = R(t, t −1 ) × R(t k , t r ) × [φ] for i > j namely, πi j =
Drk φ = φr t Drk φ = t φr
for i ≤ j . for i > j
This proves the t-circulant structure of the matrix . Conversely, let A be a (k × k)-block Toeplitz matrix whose generating function is the polynomial matrix of the form (29), namely, such that, for every i, j, its (i, j)th entry πi j satisfies φr where r = j − i, for i ≤ j πi j = . t φr where r = k + j − i, for i > j Again by relation (18), we get: πi j = Dkj (t i φ).
26
BARNABEI AND MONTEFUSCO
By replacing this expression in (28) and using (17), we get the assertion. The explicit expression of the generating function φ is an immediate consequence of the preceding considerations.
B. Products of Hurwitz Matrices As shown in the previous section, the key elements in filter bank computations are banded Toeplitz and Hurwitz matrices, their transposes, and their products. Hence we devote the present subsection to a detailed study of such matrices. First of all, for the sake of simplicity, we introduce a compact notation to represent the set of the decimated polynomials of a given Laurent polynomial. More precisely, if α is a Laurent polynomial, the k-decimated vector of α will be the column vector ⎡ ⎤ α0 ⎢ α1 ⎥ ⎥ ∆k (α) := ⎢ (30) ⎣ ... ⎦ , αk−1
whose entries are the k-decimated polynomials of α. Similarly, the starred k-decimated vector of α will be the column vector ⎡ ∗ ⎤ α0 ⎢ ∗ .. ⎥ k ∗ (31) ∆ (α) := ⎣ α1 . ⎦ . ∗ αk−1
In view of further applications, we introduce a similar notation to denote the set of the k-decimated polynomials of an ordered k-tuple of Laurent polynomials α := (α (0) , α (1) , . . . , α (k−1) ). The k-decimated matrix of α is defined as the following matrix: ⎤ ⎡ (0) α0(1) · · · α0(k−1) α0 ⎥ ⎢ ⎥ ⎢ (0) ⎢ α1 α1(1) · · · α1(k−1) ⎥ ⎥ ⎢ ⎥. ∆k (α) = ⎢ (32) ⎢ . .. .. .. ⎥ ⎥ ⎢ .. . . . ⎥ ⎢ ⎦ ⎣ (0) α(k−1)
(1) α(k−1)
(k−1) · · · α(k−1)
We now show that the product of a banded Hurwitz matrix, and the transpose of another banded Hurwitz matrix with the same step, turns out to be a Toeplitz matrix, whose generating function can be described explicitly.
27
SUBBAND SIGNAL PROCESSING
Theorem IV.2 Let α, β be nonzero Laurent polynomials, and k a fixed integer, k ≥ 2. Set H = R(t k , α), K = R(t k , β). Then, the product H × K T is a Toeplitz matrix, whose generating function λ is given by ∗ λ := Dk0 (αβ ∗ ) = α0 β0∗ + · · · + αk−1 βk−1 .
(33)
In other words, λ = ∆k (α)T × ∆k (β)∗ .
(34)
Proof. out that, given two Laurent polynomials γ := n We point δ := i=−n di t i , the (canonical) scalar product γ · δ := (c−n , c−n+1 , . . . , cn ) · (d−n , d−n+1 , . . . , dn ) =
can be written as n
i=−n
n
i=−n
n
i=−n
ci t i ,
= ci di
ci di = 0|γ ∗ δ = 0γ δ ∗ .
Now, setting P = [ pi j ] = H × K T , we have
pi j = H (i) · K ( j) = 0|H (i)K ( j)∗
= 0|t ki αt −k j β ∗ = k( j − i)|αβ ∗ .
Hence, P(i) =
Setting j − i = q, we get P(i) =
j
pi j t j =
k( j − i)|αβ ∗ t j . j
kq|αβ ∗ t q+i = t i Dk0 (αβ ∗ ), q
which proves that P is a banded Toeplitz matrix with generating function λ = Dk0 (αβ ∗ ). The second expression for λ is an immediate consequence of Propositions III.2 and III.3. Our next aim is to show that the product of the transpose of a banded Hurwitz matrix and a banded Hurwitz matrix with the same step k is once again a Toeplitz matrix, although in this case it is a block Toeplitz matrix. In order to do this, we need to show that the transpose of the banded Hurwitz matrix R(t k , α) presents some kind of recursive structure. More precisely,
28
BARNABEI AND MONTEFUSCO
Figure 9. The transposed Hurwitz matrix R(t 3 , α)T .
we can describe its row-generating functions by means of the k-decimated polynomials of α as follows: Theorem IV.3 Let α be a Laurent polynomial, and k a fixed integer, k ≥ 2. Set H := R(t k , α)T . For every integer n, let r, q be the unique integers such that 0 ≤ r < k and n = kq + r . Then we get the following expression for the generating function of the nth row of H : H (n) = t q+1 (α ∗ )k−r = t q αr∗ . In particular, H (0) = α0∗ .
Proof. Set K = R(t k , α). By Proposition II.2 we have: K = R(t k , υ) × R(t, α).
This means that every column of K can be seen as the k-downsampled version of the corresponding column of R(t, α). Recalling that the generating function of the nth column of R(t, α) is t n α ∗ , by Propositions III.3 and III.4, we get H (n) = Dk0 (t n α ∗ ) = t q+1 (α ∗ )k−r = t q αr∗ . In other words, as shown in Figure 9, the transpose of the banded Hurwitz matrix R(t k , α) can be seen as the “wedge” of k Toeplitz matrices, whose ∗ . generating functions are the polynomials α0∗ , α1∗ , . . . , αk−1 These polynomials turn out to be crucial in describing the blocks of a product of the kind R(t k , α)T × R(t k , β). In fact, we have:
SUBBAND SIGNAL PROCESSING
29
Proposition IV.2 Let α, β be nonzero Laurent polynomials, and k a fixed integer, k ≥ 2. Set H := R(t k , α), K := R(t k , β). For every integer n, let r, q be the unique integers such that 0 ≤ r < k and n = kq + r . Then (H T × K )(n) = t kq (αr∗ ◦ t k )β.
(35)
Proof. Denote by Tr the Toeplitz matrix with generating function αr∗ , namely, Tr := R(t, αr∗ ). By Theorem IV.3, we have H T (n) = t q αr∗ = Tr (q). Hence, (H T × K )(n) = (Tr × K )(q), that is, the nth row of the product H T × K equals the qth row of the matrix Tr × K , which turns out to be the Hurwitz matrix R(t k , (αr∗ ◦ t k )β), and this gives the assertion. Relation (35) shows that, given two integers n 1 , n 2 such that n 1 − n 2 is a multiple of k, namely, n 1 = kq1 + r,
n 2 = kq2 + r,
then, (H T × K )(n 1 ) = t kq1 (αr∗ ◦ t k )β,
(H T × K )(n 2 ) = t kq2 (αr∗ ◦ t k )β.
Loosely speaking, the generating functions of the corresponding rows of the matrix H T × K are the same polynomial (αr∗ ◦ t k )β multiplied by different shifts. This highlights a block Toeplitz structure for the matrix H T × K , as shown by the next result. Theorem IV.4 Let α, β be nonzero Laurent polynomials, and k a fixed integer, k ≥ 2. Set H := R(t k , α), K := R(t k , β). Then the product H T × K is a (k × k)-block Toeplitz matrix, whose generating function is the block Laurent polynomial = i Pi t i , where ⎡ ⎤ i(α0 )∗ β0 · · · i(α0 )∗ βk−1 ⎦. ··· ··· ··· Pi = ⎣ i(αk−1 )∗ β0 · · · i(αk−1 )∗ βk−1
Equivalently, can be represented as the polynomial matrix: = ∆k (α)∗ × ∆k (β)T .
Proof. The block Toeplitz structure of the matrix H T × K is a consequence of the preceding considerations concerning formula (35). To give an
30
BARNABEI AND MONTEFUSCO
explicit description of these blocks, we make use of identity (17). In fact, we have: (H T × K )(n) = t kq (αr∗ ◦ t k )(β0 ◦ t k + t(β1 ◦ t k ) + · · · + t k−1 (βk−1 ◦ t k )) = t kq ((αr∗ β0 ) ◦ t k + t(αr∗ β1 ) ◦ t k + · · · + t k−1 (αr∗ βk−1 ) ◦ t k ). V. Analysis and Synthesis Filter Banks From now on, we use the tools presented in the previous sections to review several classic results from the filter bank literature, and mathematically describe their main features. Main applications of filter banks are found in subband coding schemes and transmultiplexers. Both of these applications make use of two basic devices, analysis and synthesis banks, which consist of M analysis operators and M synthesis operators, respectively. In fact, in subband coding, an analysis bank splits the input signal into M downsampled subband signals, which are then quantized and coded individually. After decoding, the reconstructed signal is obtained by means of a synthesis filter bank. Dually, the transmultiplexing scheme starts from several individual signals and, using a synthesis filter bank, combines them into a single signal from which the individual components can be recovered using an analysis filter bank. Given the importance of analysis and synthesis systems, we devote the present section to a matrix description of the corresponding operators.
A. Analysis and Synthesis Operators We start by giving an explicit description of the action of a single analysis and synthesis operator, together with their combined action. r
Analysis The analysis operator Dk0 Fγ is associated with the Hurwitz matrix R(t k , γ ). Hence, the output σˆ of its action on the input signal σ coincides with the zeroth column of the matrix R(t k , γ ) × R(t, σ ∗ ) = R(t k , γ σ ∗ ). Recalling that if β is any Laurent polynomial, the generating function of the zeroth column of the Hurwitz matrix R(t k , β) is given by the Laurent polynomial β0∗ = Dk0 (β ∗ ) (Proposition IV.3), we get σˆ = Dk0 (γ σ ∗ )∗ = Dk0 (γ ∗ σ ).
31
SUBBAND SIGNAL PROCESSING
Using Propositions III.2 and III.3, the latter polynomial can be written as σˆ = Dk0 (γ ∗ σ ) = namely, σˆ = ∆k (γ )∗T × ∆k (σ ) = [γ0∗
k−1 (γi )∗ σi , i=0
⎡
⎤ σ0 ⎢ σ1 ⎥ ∗ ⎥ ]×⎢ γ1∗ · · · γk−1 ⎣ ... ⎦ .
(36)
σk−1
This yields the computationally efficient and easily parallelizable description of the action of an FIR analysis operator depicted below:
r
Synthesis The synthesis operator Fα∗ Uk corresponds to the transposed Hurwitz matrix R(t k , α)T .
As in the previous case, the output σˆ is the zeroth column of the matrix R(t k , α)T × R(t, σ ∗ ) = R(t k , α)T × R(t, σ )T = R(t k , (σ ◦ t k )α)T , that is, σˆ = (σ ◦ t k )α. Recalling that α=
k−1 i=0
t i (αi ◦ t k ),
we get the following algebraic expression for the output of a synthesis operator: σˆ =
k−1 i=0
t i ((σ αi ) ◦ t k )
32
BARNABEI AND MONTEFUSCO
that can be schematized as follows:
Bearing in mind the bijection between a Laurent polynomial and its kdecimated vector, the action of the synthesis operator can be also described as follows: ∆k (σˆ ) = ∆k (α) σ,
(37)
that is, ⎤ ⎤ ⎡ σˆ 0 α0 σ ⎢ σˆ 1 ⎥ ⎢ α1 σ ⎥ ⎢ . ⎥ = ⎢ . ⎥. ⎣ .. ⎦ ⎣ .. ⎦ σˆ k−1 αk−1 σ ⎡
This last expression turns out to be particularly efficient for practical applications, both due to its reduced computational complexity and because of its inherent parallel structure. r
Analysis–synthesis By identities (36) and (37), the output σˆ of the analysis–synthesis operator Fα∗ Uk Dk0 Fγ is the unique polynomial whose k-decimated vector is given by ∆k (σˆ ) = ∆k (α) × ∆k (γ )∗T × ∆k (σ ), namely, ⎤ ⎤ ⎡ σˆ 0 α0 ⎢ σˆ 1 ⎥ ⎢ α1 ⎥ ⎢ . ⎥ = ⎢ . ⎥ × [γ ∗ 0 ⎣ .. ⎦ ⎣ .. ⎦ αk−1 σˆ k−1 ⎡
⎡
⎤ σ0 ⎢ σ1 ⎥ ∗ ⎥ γ1∗ · · · γk−1 ]×⎢ ⎣ ... ⎦ . σk−1
(38)
33
SUBBAND SIGNAL PROCESSING
On the other hand, the analysis–synthesis operator is associated with the matrix R(t k , α)T × R(t k , γ ). By Theorem IV.4, this matrix is a (k × k)block Toeplitz matrix, whose generating function is the polynomial matrix: = ∆k (α)∗ × ∆k (γ )T . The above highlights the following general result: Proposition V.1 Let Q : P → P be a linear operator associated with the block Toeplitz matrix R(t, Ω), where Ω is the square polynomial matrix ⎡
ω0,0 ⎢ ω1,0 Ω=⎢ ⎣ ...
ωk−1,0
··· ··· .. .
ω0,1 ω1,1 .. . ω1,k−1
⎤ ω0,k−1 ω1,k−1 ⎥ ⎥ ωi, j ∈ P . .. ⎦ .
· · · ωk−1,k−1
Then the image σˆ = Qσ of the Laurent polynomial σ under the operator Q can be described in terms of its k-decimated polynomials as follows: ∆k (σˆ ) = Ω∗ × ∆k (σ ), where ⎡
∗ ω0,0
⎢ ω∗ ⎢ Ω∗ = ⎢ 1,0 ⎣ ... ∗ ωk−1,0
∗ ω0,1 ∗ ω1,1 .. . ∗ ω1,k−1
···
··· .. .
∗ ω0,k−1 ∗ ω1,k−1 .. .
∗ · · · ωk−1,k−1
⎤
⎥ ⎥ ⎥. ⎦
B. Matrix Description of Analysis Filter Banks In a maximally decimated M-channel analysis bank the input signal is divided into M filtered versions of equal bandwith, which are then maximally decimated, namely, subsampled by the same integer M, according to the scheme shown below, where, for i = 0, 1, . . . , M − 1, we have denoted by
34
BARNABEI AND MONTEFUSCO
Fγ (i) (γ (i) ∈ P ) the analysis filters:
Hence, the size M analysis operator, which associates the M-tuple (σ (0) , σ , . . . , σ (M−1) ) of its filtered and decimated versions to the input signal σ , is uniquely represented by the M-tuple of Laurent polynomials (1)
γ = γ (0) , γ (1) , . . . , γ (M−1) , and will be denoted by the symbol Cγ , namely, Cγ (σ ) = D0M Fγ (0) (σ ), D0M Fγ (1) (σ ), . . . , D0M Fγ (M−1) (σ ) . The M-tuple γ will be called the analysis filter vector. Its zeroth component γ (0) usually denotes a low-pass filter, while the components γ (1) , γ (2) , . . . , γ (M−1) denote high-pass filters. Recalling that, by (25), each operator D0M Fγ (i) is associated with the banded Hurwitz matrix Ci := R(t M , γ (i) ), the size M analysis operator Cγ can be associated with the matrix column vector ⎡
⎤ C0 ⎢ C1 ⎥ ⎢ . ⎥. ⎣ .. ⎦ C M−1
35
SUBBAND SIGNAL PROCESSING
Its action can therefore be represented as: ⎤ ⎤ ⎡ [σ (0) ] C0 ⎢ [σ (1) ] ⎥ ⎢ C ⎥ ⎢ ⎥ ⎢ 1 ⎥ .. ⎢ ⎥ = ⎢ .. ⎥ × [σ ]. ⎣ ⎦ ⎣ . ⎦ . (M−1) C M−1 [σ ] ⎡
(39)
Making use of formula (36), we immediately get a computationally more efficient description of the action of an analysis filter bank, as follows: ⎡ (0) ∗ γ0 ⎢ (1) ⎥ ⎢ ⎢ σ ⎥ ⎢ (1) ∗ ⎢ ⎥ ⎢ γ0 ⎢ ⎥=⎢ ⎢ .. ⎥ ⎢ ⎢ . ⎥ ⎢ ··· ⎣ ⎦ ⎣ (M−1) ∗ (M−1) σ γ0 ⎡
σ (0)
⎤
(0) ∗ γ1 (1) ∗ γ1 ···
(M−1) ∗ γ1
···
(0) ∗ γ M−1 (1) ∗ γ M−1
⎤
⎡
σ0
⎤
⎥ ⎢ ⎥ ⎥ ⎢ σ1 ⎥ ⎥ ⎢ ⎥ ⎥×⎢ ⎥ ⎥ ⎢ . ⎥, ··· . . . ⎥ ⎢ .. ⎥ ⎦ ⎣ ⎦ (M−1) ∗ σ M−1 · · · γ M−1 ···
or, equivalently, ⎡
⎤ σ (0) ⎢ σ (1) ⎥ ⎢ .. ⎥ = ∆ M (γ)∗T × ∆ M (σ ), ⎣ . ⎦ σ (M−1)
(40)
where ∆ M (γ)∗ is the starred M-decimated matrix of γ.
C. Matrix Description of Synthesis Filter Banks A maximally upsampled M-channel synthesis bank upsamples by M the M input signals and filters them, before summing them to create a single output signal. This is represented in the scheme below, where, for i = 0, 1, . . . ,
36
BARNABEI AND MONTEFUSCO
M − 1, we have denoted by Fα(i)∗ (α (i) ∈ P ) the synthesis filters:
Hence, the size M synthesis operator that associates the output σˆ to the M input signals (σ (0) , σ (1) , . . . , σ (M−1) ), is uniquely represented by the M-tuple of Laurent polynomials α = α (0) , α (1) , . . . , α (M−1) , and will be denoted by the symbol Aα , namely,
M−1 Aα σ (0) , σ (1) , . . . , σ (M−1) = Fα(i)∗ U M σ (i) . i=0
The M-tuple α will be called the synthesis filter vector. As in the analysis case, the component α (0) is a low-pass filter, and the components α (1) , α (2) , . . . , α (M−1) are high-pass filters. Recalling that, by (26), each operator Fα(i)∗ U M is associated with the transpose of the banded Hurwitz matrix Ai := R(t M , α (i) ), the size M synthesis operator Aα can be associated with the matrix row vector T A0 A1T · · · A TM−1 . Its action can therefore be represented as:
[σˆ ] =
A0T
⎡
⎢ ⎢ A1T · · · A TM−1 × ⎢ ⎣
[σ (0) ] [σ (1) ] .. . [σ (M−1) ]
⎤
⎥ ⎥ ⎥. ⎦
(41)
A description of the M-decimated vector of the output signal σˆ , can be obtained
37
SUBBAND SIGNAL PROCESSING
by exploiting formula (37): ⎡ ⎤ ⎡ (0) σˆ 0 α0 ⎥ ⎢ (0) ⎢ ⎢ σˆ 1 ⎥ ⎢ α1 ⎢ ⎥ ⎢ ⎢ . ⎥=⎢ . ⎢ .. ⎥ ⎢ .. ⎣ ⎦ ⎣ (0) σˆ M−1 α(M−1)
or, equivalently,
α0(1) α1(1) .. . (1) α(M−1)
· · · α0(M−1)
⎤
⎡
σ (0)
⎥ ⎢ · · · α1(M−1) ⎥ ⎢ σ (1) ⎥ ⎢ ×⎢ . .. .. ⎥ ⎢ . . ⎥ . ⎦ ⎣ .
(M−1) · · · α(M−1)
σ (M−1)
⎤ σ (0) ⎢ σ (1) ⎥ . ⎥ ∆ M (σˆ ) = ∆ M (α) × ⎢ ⎣ .. ⎦ . σ (M−1) ⎡
⎤
⎥ ⎥ ⎥ ⎥, ⎥ ⎦
(42)
VI. M-Channel Filter Bank Systems In the present section, we take advantage of the preceding considerations to give an algebraic description of a maximally decimated M-channel filter bank FIR system. This kind of system accepts as input a signal, hence, a Laurent polynomial, and produces as output another Laurent polynomial. It can therefore be interpreted as a linear operator on the vector space P of Laurent polynomials. The theoretical results stated above allow us to give an explicit description of this operator by means of its associated matrix, which turns out to be recursive. By studying the algebraic features of this matrix we can recover the conditions for the alias cancelation and perfect reconstruction properties of the analysis–synthesis filter bank system. Such a system is usually described in the literature using the polyphase decomposition of the filters. In the present algebraic setting, this corresponds to considering the decimated vectors of the Laurent polynomials representing the analysis and synthesis filters. Hence, the polyphase approach can be recovered using the description in terms of generating functions of the recursive matrices involved, and recalling the bijection between a Laurent polynomial and its decimated vector.
A. Matrix Description A maximally decimated M-channel filter bank consists of a size M analyisis operator Cγ , followed by a size M synthesis operator Aα , according to the
38
BARNABEI AND MONTEFUSCO
following scheme:
It is therefore represented by the linear operator Lα,γ = Aα Cγ . Hence, an M-channel filter bank is uniquely identified by the two M-tuples of Laurent polynomials γ = γ (0) , γ (1) , . . . , γ (M−1) , α = α (0) , α (1) . . . , α (M−1) ,
namely, the analysis and synthesis filter vectors, and represented by the two M-tuples of Hurwitz matrices (C0 , C1 , . . . , C M−1 ), where, for i = 0, 1, . . . , M − 1, Ci = R t M , γ (i) ,
(A0 , A1 , . . . , A M−1 ), Ai = R t M , α (i) .
Making use of formulas (39) and (41), it is easily seen that the action of the linear operator Lα,γ can be represented as ⎤ ⎡ C0 ⎢ C1 ⎥ ⎥ [σˆ ] = A0T A1T · · · A TM−1 × ⎢ ⎣ ... ⎦ × [σ ]. C M−1 Hence, by (27), its associated matrix is
A0T × C0 + A1T × C1 + · · · + A TM−1 × C M−1 .
(43)
39
SUBBAND SIGNAL PROCESSING
By Theorem IV.4, every summand in (43) is a block Toeplitz matrix, hence, so is the matrix associated with the linear operator Lα,γ . We have therefore the following result, whose proof is a straightforward consequence of the same theorem: Theorem VI.1 Let Ai = R(t M , α (i) ), Ci = R(t M , γ (i) ), i = 0, 1, . . . , M − 1, be banded Hurwitz matrices. The matrix A0T × C0 + A1T × C1 + · · · + A TM−1 × C M−1 is an (M × M)-block Toeplitz matrix, whose generating function is the polynomial matrix given by ⎡ (0) ∗ α0 ⎢ ∗ ⎢ α (0) ⎢ = ⎢ 1. ⎢ .. ⎣ (0) ∗ α M−1
(1) ∗ α0 (1) ∗ α1 .. . (1) ∗ α M−1
∗ ⎤ ⎡ (0) γ0 γ1(0) · · · α0(M−1) ∗ ⎥ ⎢ ⎢ (1) γ1(1) · · · α1(M−1) ⎥ ⎥ ⎢ γ0 ⎥×⎢ . .. .. .. ⎥ ⎢ .. . . . ⎦ ⎣ (M−1) ∗ · · · α M−1 γ0(M−1) γ1(M−1)
⎤
(0) · · · γ M−1
(1) · · · γ M−1 .. .. . . (M−1) · · · γ M−1
⎥ ⎥ ⎥ ⎥, ⎥ ⎦
or, equivalently, = ∆ M (α)∗ × ∆ M (γ)T . The preceding result represents one of the most interesting achievements of the present algebraic approach, as it gives a description of a filter bank system both in terms of the associated matrix (time-domain formulation) and of generating function (polyphase formulation). In fact, using Proposition V.1, we immediately get the following description of the action of Lα,γ in terms of M-decimated vectors: ∆ M (σˆ ) = ∗ × ∆ M (σ ) = ∆ M (α) × ∆ M (γ)∗T × ∆ M (σ ).
(44)
The matrix ∗ is usually called the transmission matrix or transfer function matrix in the filter bank literature, and (44) is the well-known polyphase characterization of the action of an M-band filter bank.
40
BARNABEI AND MONTEFUSCO
B. Alias-Free Filter Banks The output σˆ of an M-channel filter bank is clearly better, the closer it is to the corresponding input signal σ . The best case is obviously when the output is a shifted and possibly scaled version of the input: in this case, the filter bank is said to have the perfect reconstruction property. This situation will be discussed in the next subsection. In this subsection we will consider the most important case of approximate reconstruction, namely, the case in which σˆ can be obtained from σ by multiplying it by a fixed Laurent polynomial. In the filter bank language this is called alias-free reconstruction, and the analysis–synthesis system is said to possess the alias-free property. Since such a system corresponds to a linear operator, we restate in algebraic language the definition of alias-free property for a linear operator on P . A linear operator Q : P → P will be called alias-free if and only if there exists a nonzero Laurent polynomial τ such that σˆ = τ σ for every Laurent polynomial σ . Equivalently, Q is alias-free if and only if it is a filter, namely, its associated matrix is a banded Toeplitz matrix. In this case, the Laurent polynomial τ will be said to be the distortion function of the operator Q. We note that, when Q is the linear operator describing the combined action of the analysis and synthesis operators of an M-channel filter bank system, the preceding definition gives the usual notion of alias-free system, as given, for example, in Vaidyanathan (1993). As we have already seen, the matrix associated with the linear operator describing the action of an M-channel filter bank system is a block Toeplitz matrix. Hence, the alias cancelation condition is equivalent to requiring that this matrix is also a scalar Toeplitz matrix, whose generating function is the distortion function of the system. The connection between block and scalar Toeplitz matrices has been analyzed in detail in Section IV.A, hence, using Theorem IV.1, we are able to characterize alias-free systems and give an explicit expression of their distortion function. Theorem VI.2 Let Lα,γ be the linear operator describing the action of an M-channel filter bank relative to the two M-tuples γ = γ (0) , γ (1) , . . . , γ (M−1) ,
α = α (0) , α (1) . . . , α (M−1) ,
SUBBAND SIGNAL PROCESSING
41
The operator Lα,γ is alias-free if and only if the matrix = ∆ M (α)∗ × ∆ M (γ)T ⎡ (0) ∗ (1) ∗ ∗⎤ ⎡ (0) (0) ⎤ γ0 γ1(0) · · · γ M−1 · · · α0(M−1) α0 α0 ⎥ ⎢ ∗ (1) ∗ ∗⎥ ⎢ (1) ⎥ ⎢ (1) ⎢ α (0) γ1(1) · · · γ M−1 · · · α1(M−1) ⎥ α1 ⎥ ⎢ γ0 ⎥ ⎢ 1 =⎢ . ⎥×⎢ . .. .. .. ⎥ .. .. .. . ⎥ ⎢ ⎢ .. . ⎥ . . . . . ⎣ ⎦ ⎣ . ⎦ (0) ∗ (1) ∗ (M−1) ∗ (M−1) (M−1) (M−1) α M−1 · · · α M−1 α M−1 γ1 · · · γ M−1 γ0
is t-circulant. In this case, the distortion function of Lα,γ is the Laurent polynomial τ=
M−1 i=0
∗ α0(i) ◦ t M γ (i) .
Proof. The first statement is an immediate consequence of Theorems IV.1 and VI.1. We point out that, in the alias-free case, by Theorem VI.1, the generating function of the scalar Toeplitz matrix is given by φ=
M−1 j=0
tj
M−1 i=0
(i) ∗ M (i) M α0 (◦t ) γ j ◦ t ,
or, equivalently, φ=
M−1 i=0
(i) (i) ∗ M−1 (i) ∗ M M−1 α0 ◦ t M γ (i) . α0 (◦t ) t j γj ◦ tM = j=0
i=0
Hence, the distortion function is τ = φ∗ =
M−1 i=0
(i) M (i) ∗ α0 ◦ t (γ ) .
An interesting question in filter bank design is whether, given an analysis filter vector, it is possible to choose an appropriate synthesis filter vector in order to cancel aliasing. The answer to this problem is given by the next result (see Vetterli and Kovacevic, 1995). Proposition VI.1 Given an analysis filter vector γ = γ (0) , γ (1) , . . . , γ (M−1) ,
42
BARNABEI AND MONTEFUSCO
alias-free reconstruction is possible if and only if the determinant of the M-decimated matrix ∆ M (γ) is different from the zero polynomial, namely, the matrix ∆ M (γ) has full rank. Obviously, the dual problem of finding an alias-canceling analysis filter vector when a synthesis filter vector is given has a similar solution.
C. Perfect Reconstruction Filter Banks We now come to the characterization of perfect reconstruction systems. Let Lα,γ be the linear operator describing the action of an M-channel filter bank system relative to the two M-tuples γ = γ (0) , γ (1) , . . . , γ (M−1) , α = α (0) , α (1) . . . , α (M−1) .
The operator Lα,γ is said to have the perfect reconstruction property whenever a nonzero constant c ∈ K and an integer h exist such that Lα,γ (σ ) = c t −h σ for every Laurent polynomial σ . This is equivalent to the fact that Lα,γ is alias-free and its associated matrix is the scalar Toeplitz matrix R(t, c t h ). Theorem VI.2 immediately yields the following algebraic characterization of perfect reconstruction systems: Theorem VI.3 Let Lα,γ be the linear operator describing the action of an M-channel filter bank system relative to the two M-tuples γ = γ (0) , γ (1) , . . . , γ (M−1) , α = α (0) , α (1) . . . , α (M−1) .
The operator Lα,γ has the perfect reconstruction property if and only if both of the following conditions are satisfied: 1. the matrix = ∆ M (α)∗ × ∆ M (γ)T is t-circulant; 2. the following identity holds: M−1 i=0
(i) ∗ M (i) α0 ◦ t γ = c t h ,
SUBBAND SIGNAL PROCESSING
43
that is, setting h = q M + r, with 0 ≤ r < M, M−1 (i) ∗ (i) 0 for j = r α0 γ j = c t q for j = r. i=0
In other words, the perfect reconstruction condition is equivalent to requiring that the matrix above is t-circulant and the elements of its first row are all zero polynomials, except one, which must be a monomial. If this is the case, we can always suitably shift and scale the filters involved in such a way that the matrix is simply the identity matrix. So, without loss of generality, we can restate the perfect reconstruction condition as follows: Proposition VI.2 The operator Lα,γ describing the action of an M-channel filter bank system has the perfect reconstruction property if and only if the following condition is satisfied: ∆ M (α)∗ × ∆ M (γ)T = I.
(45)
Due to the fact that we are working with FIR filters, namely, the filter vectors α and γ are M-tuples of Laurent polynomials, it follows from the above condition (45) that the determinants of both matrices ∆ M (α)∗ and ∆ M (γ) are nonzero monomials. As a consequence, the two matrices are the inverse one of the other, hence, condition (45) is equivalent to the following: ∆ M (γ)T × ∆ M (α)∗ = I.
(46)
The perfect reconstruction condition (46) can also be stated in terms of Hurwitz matrices: Proposition VI.3 The M-channel filter bank described by the two filter vectors γ, α corresponds to a perfect reconstruction operator if and only if the Hurwitz matrices Ai = R t M , α (i) , Ci = R t M , γ (i) , i = 0, 1, . . . , M − 1 satisfy the following conditions:
Ci × A Tj = δi, j I,
i, j = 0, 1, . . . , M − 1,
or, in matrix notation, ⎤ ⎡ ⎤ ⎡ C0 I 0 ··· 0 ⎢ C1 ⎥ T T ⎢0 I ··· 0 ⎥ ⎢ . ⎥ × A A · · · AT 0 1 M−1 = ⎣· · · · · · · · · · · ·⎦ . ⎣ .. ⎦ 0 0 ··· I C M−1
(47)
44
BARNABEI AND MONTEFUSCO
Proof. It is sufficient to remark that condition (46) can be equivalently written as T ∗ ∆ M γ (i) × ∆ M α ( j) = δi j ,
i, j = 0, 1 . . . , M − 1.
(48)
The assertion now follows from Theorem IV.2. A pair (α, γ) satisfying one of the above equivalent conditions (45), (46), or (47) will be called a pair of dual (or biorthogonal) filter vectors. The simplest pair of dual filter vectors is the “trivial” pair (δ, δ), where δ = (1, t, t 2 , . . . , t M−1 ). As in the alias-free case, a crucial problem in FIR filter bank design is finding conditions under which, given an analysis filter bank, there exists a corresponding synthesis bank yielding a perfect reconstruction system. As we have already remarked, the answer to this problem can be given in algebraic language as follows: Proposition VI.4 Given an analysis filter vector γ = γ (0) , γ (1) , . . . , γ (M−1) ,
perfect reconstruction is possible if and only the determinant of the Mdecimated matrix ∆ M (γ) is a nonzero monomial. If this is the case, the dual M-tuple α of γ is uniquely determined by condition (45). Obviously, if (α, γ) is a pair of dual filter vectors, the same holds for the pair (γ, α).
VII. Transmultiplexers In the previous section, we considered the problem of decomposing a given signal into components, from which the original signal can be recovered. We will now consider the dual problem of starting from many input signals and combining them into a single output, from which the original signals can be recovered. This problem has some important applications, for example, in digital telephone networks, where several users share a common channel to transmit information. An efficient solution to this problem can be given in terms of synthesis and analysis filter banks, so all the algebraic tools previously developed can also be used in this dual context.
45
SUBBAND SIGNAL PROCESSING
A. Matrix Description A device synthesizing a single signal from M input signals, followed by the inverse operation of recovering the initial M inputs, is called an Mtransmultiplexer. It can be algebraically seen as a size M synthesis operator Aα , followed by a size M analyisis operator Cγ according to the following scheme:
It is therefore represented by the linear operator Tα,γ = Cγ Aα . As in the subband coding case, an M-transmultiplexer is also uniquely identified by the two M-tuples of Laurent polynomials γ = γ (0) , γ (1) , . . . , γ (M−1) , α = α (0) , α (1) . . . , α (M−1) ,
namely, the synthesis and analysis filter vectors, and represented by the two M-tuples of Hurwitz matrices (A0 , A1 , . . . , A M−1 ),
where, for i = 0, 1, . . . , M − 1, Ai = R t M , α (i) ,
(C0 , C1 , . . . , C M−1 ), Ci = R t M , γ (i) .
Making use of formulas (39) and (41), the action of the linear operator Tα,γ can be represented as ⎤ ⎤ ⎡ ⎤ ⎡ ⎡ [σˆ (0) ] [σ (0) ] C0 ⎢ [σˆ (1) ] ⎥ ⎢ C1 ⎥ T T ⎢ [σ (1) ] ⎥ ⎥. ⎢ ⎥ = ⎢ . ⎥ × A A · · · AT ⎢ (49) .. . 0 1 M−1 × ⎣ ⎦ ⎣ ⎦ ⎣ .. ⎦ .. . [σˆ (M−1) ]
C M−1
[σ (M−1) ]
46
BARNABEI AND MONTEFUSCO
Its associated matrix is therefore ⎡ ⎤ C0 × A0T C0 × A1T · · · C0 × A TM−1 ⎥ ⎢ ⎢ C1 × A0T C1 × A1T · · · C1 × A TM−1 ⎥ ⎥. ⎢ ⎣ ⎦ ··· ··· ··· T T T C M−1 × A0 C M−1 × A1 · · · C M−1 × A M−1
(50)
By Theorem IV.2, every entry Ci × A Tj in (50) is a Toeplitz matrix with generating function T ∗ λi, j = ∆ M γ (i) × ∆ M α ( j) .
Hence, the matrix associated with the linear operator Tα,γ is a Toeplitz block matrix. The above matrix represents a description of the transmultiplexing system from an algebraic point of view, that corresponds to the so-called time-domain formulation. The analog of the polyphase formulation is obtained equivalently either by translating formula (49) in terms of generating functions, or by exploiting the description of synthesis and analysis filter banks given in (42) and (40), respectively. The first approach yields the following expression for the ith output signal: σˆ (i) =
M−1 j=0
λi,∗ j σ ( j) =
M−1 j=0
T ∗ ∆ M γ (i) × ∆ M α ( j) σ ( j) ,
(51)
while the second point of view gives the result stated below.
Proposition VII.1 Let α, γ be the synthesis and analysis filter vectors identifying an M-transmultiplexer. The action of the corresponding linear operator Tα,γ = Cγ Aα can be described as follows: ⎡ (0) ⎤ ⎡ (0) ⎤ σ σˆ (1) ⎥ ⎢ ⎢ σˆ (1) ⎥ ⎢ .. ⎥ = ∆ M (γ)∗T × ∆ M (α) × ⎢ σ .. ⎥ . ⎣ . ⎦ ⎣ . ⎦ (M−1) σ (M−1) σˆ B. Perfect Reconstruction Transmultiplexers The above matrix description of transmultiplexers allows us to simply characterize filter vectors that yield cross-talk-free and perfect reconstruction transmultiplexers. These two kinds of systems are the analogs of alias-free and perfect reconstruction filter banks.
SUBBAND SIGNAL PROCESSING
47
In particular, an M-transmultiplexer (and, hence, the corresponding operator) is said to be free from cross-talk when every output signal σˆ (i) only depends on the corresponding input σ (i) , namely, nonzero Laurent polynomials τ (0) , τ (1) . . . , τ (M−1) exist such that, for every i = 0, 1, . . . , M − 1, σˆ (i) = τ (i) σ (i) . This last condition is clearly satisfied whenever the Toeplitz block matrix (50) associated with the linear operator Tα,γ is block diagonal. This happens if and only if Ci × A Tj = 0 for i = j,
(52)
namely, T ∗ λi, j := ∆ M γ (i) × ∆ M α ( j) = 0 for i = j.
If this is the case, for every i = 0, 1, . . . , M − 1, ∗ σ (i) . σˆ (i) = λi,i
Similarly, by using the synthesis–analysis filter bank description, the crosstalk-free condition can be stated as follows: Proposition VII.2 The M-transmultiplexer identified by the synthesis and analysis filter vectors α, γ is free from cross-talk if and only if the matrix ∆ M (γ)∗T × ∆ M (α) is a diagonal matrix. The question whether, given a synthesis filter vector, it is possible to choose an appropriate analysis filter vector in order to cancel cross-talk, can be asked here as well. The answer to this problem is similar to the alias-free case. In fact, easy considerations lead to the following result: Proposition VII.3 Given a synthesis filter vector α = (α (0) , α (1) , . . . , α (M−1) ), cross-talk elimination is possible if and only if the determinant of the Mdecimated matrix ∆ M (α) is different from the zero polynomial, namely, the matrix ∆ M (α) has full rank. We remark that, even if the condition stated in the preceding proposition is the same as the condition given in Proposition VI.1, the analogy holds only
48
BARNABEI AND MONTEFUSCO
in the case when a filter vector (analyisis or synthesis) is given, and we are looking for a suitable filter vector allowing for alias or cross-talk cancelation. On the contrary, when a complete system is given, namely, both filter vectors are fixed, the alias-free and cross-talk-free conditions are not in general equivalent. The situation is completely different if we consider perfect reconstruction systems. In fact, the operator Tα,γ describing the action of an M-transmultiplexer is said to have the perfect reconstruction property when every output signal σˆ (i) coincides with the corresponding input σ (i) , namely, for every i = 0, 1, . . . , M − 1, σˆ (i) = σ (i) . The matrix description of the action of Tα,γ allows us to state the perfect reconstruction property as follows: Proposition VII.4 The operator Tα,γ describing the action of an M-transmultiplexer has the perfect reconstruction property if and only if one of the following equivalent conditions is satisfied: (53) Ci × A Tj = δi, j I, i, j = 0, 1, . . . , M − 1, M (i) M (i) where Ai = R t , α , Ci = R t , γ , i = 0, 1, . . . , M − 1; or ∆ M (γ)T × ∆ M (α)∗ = I.
(54)
The two equivalent conditions above are exactly the same as those given in Propositions VI.2 and VI.3, showing that, in the perfect reconstruction case, transmultiplexers and analysis–synthesis systems are dual. In fact, we have: Proposition VII.5 A perfect reconstruction M-channel filter bank system is equivalent to a perfect reconstruction M-transmultiplexer. This implies that Proposition VI.4, originally stated for an analysis–synthesis system, remains valid in the transmultiplexing case as well.
VIII. The M-Band Lifting So far, we have mostly been interested in studying conditions under which two assigned filter vectors yield a perfect reconstruction system. However, perfect reconstruction is only one of the requirements needed in practical applications, since in most cases the filters themselves have to satisfy some properties, such
SUBBAND SIGNAL PROCESSING
49
as frequency selectivity, phase characteristics, and computational efficiency. All this together makes the design of an M-band perfect reconstruction system a complex problem, due to the great number of parameters involved, and the difficulties grow as soon as the number of bands taken into consideration increases. In the two-band case, a powerful tool, yielding an easy custom-design construction of biorthogonal filters, is the well-known lifting scheme (Sweldens, 1996, 1997), which on the one hand can be seen as a simple tool for the construction of biorthogonal filters with preassigned features (building property), and on the other hand yields a factorization procedure that leads to faster analysis–synthesis algorithms (decomposition property). The aim of this section is to exploit the algebraic tools previously developed in order to carefully analyze both aspects of the lifting scheme in the M-band setting and to show and characterize its capabilities.
A. Algebraic Preliminaries The lifting scheme can be described as a procedure that starts from a simple perfect reconstruction system and builds new biorthogonal pairs of filter vectors, so that the new filter coefficients satisfy prespecified requirements. To give a mathematical description of this procedure, it is necessary, given a dual pair (α, γ) of biorthogonal filter vectors, to characterize in algebraic form all the dual pairs (α, ¯ γ) ¯ that share the same zeroth component α (0) of α (lowpass filter) and the same (M − 1)-tuple of components (γ (1) , γ (2) , . . . , γ (M−1) ) of γ (high-pass filters). To do this, we recall that a pair of filter vectors (α, γ) is a dual (or biorthogonal) pair whenever it satisfies identity (46), or, equivalently, the following identity: ∆ M (γ)∗T × ∆ M (α) = I
(55)
and we give two preliminary results, that represent the theoretical support of the lifting scheme. Theorem VIII.1 Let (α, γ) be a dual pair of filter vectors. For any fixed index i = 1, 2, . . . , M − 1, every Laurent polynomial γˆ (i) such that ∗T ∆ M γˆ (i) × ∆ M α ( j) = δi j
j = 1, 2, . . . , M − 1
(56)
50
BARNABEI AND MONTEFUSCO
is given by γˆ (i) = γ (i) + μ(i) ◦ t M γ (0)
as μ(i) ranges over the set of all Laurent polynomials. Proof. Conditions (56) yield a system of M-1 linear equations in the M (i) unknowns (γˆ0(i) )∗ , (γˆ1(i) )∗ , . . . , (γˆ M−1 )∗ , namely, ⎧ (1) (i) ∗ (i) ∗ =0 + · · · + α (1) ˆ M−1 α0 γˆ0 ⎪ M−1 γ ⎪ ⎪ ⎪ . . . ⎪ ⎨ (i) ∗ ∗ (57) α0(i) γˆ0(i) + · · · + α (i) =1 ˆ M−1 M−1 γ ⎪ ⎪ . . . ⎪ ⎪ ⎪ (i) ∗ ⎩ (M−1) (i) ∗ + · · · + α (M−1) ˆ M−1 = 0 γˆ0 α0 M−1 γ
The M-tuple ∆ M (γ (i) )∗ is a solution of the system (57), while ∆ M (γ (0) )∗ is a solution of the associated homogeneous linear system. Hence, in the quotient field L+ of Laurent series, the general solution of (57) is given by ∗ ∗ χ = ∆ M γ (i) + τ ∆ M γ (0) , (58)
where τ is a Laurent series. In other terms, the components of the vector χ are given by: ⎡ (i)∗ ⎡ ⎤ ⎤ χ0 γ0 + τ γ0(0)∗ ⎢ χ ⎥ ⎢ (i)∗ ⎥ ⎢ 1 ⎥ ⎢ γ + τ γ (0)∗ ⎥ 1 1 ⎥. ⎥ ⎢ ⎢ ⎥ ⎢ .. ⎥ = χ = ⎢ .. ⎦ ⎣ ⎣ . ⎦ . χ M−1
(i)∗ (0)∗ γ M−1 + τ γ M−1
We now need to characterize the polynomial solutions of system (57). We have assumed that the filter vector γ admits a dual vector α, which implies that det∆ M (γ)∗ is a nonzero monomial. By using the Laplace expansion formula for the determinant, we find that the g.c.d. of the M-tuple ((γ0(0) )∗ , (γ1(0) )∗ , . . . , (0) ∗ ) ) is a monomial. This implies that the solution χ given by (57) is a (γ M−1 polynomial vector if and only if τ is a Laurent polynomial. At this point we note that, given any polynomial solution χ of (57), its dual polynomial vector is ⎡ (i) ⎤ γ0 + τ ∗ γ0(0) ⎢ (i) ⎥ ⎢ γ + τ ∗ γ (0) ⎥ ⎥ 1 1 χ∗ = ⎢ ⎢ ⎥ .. ⎣ ⎦ . (0) (i) + τ ∗ γ M−1 γ M−1
SUBBAND SIGNAL PROCESSING
51
and hence, by the reconstruction rule (17), the Laurent polynomial whose M-decimated vector coincides with χ∗ is ∗ ◦ tM) γˆ (i) = χ0∗ ◦ t M + t(χ1∗ ◦ t M ) + · · · + t M−1 (χ M−1 = γ0(i) ◦ t M + (τ ∗ ◦ t M ) γ0(0) ◦ t M + t γ1(i) ◦ t M + (τ ∗ ◦ t M ) γ1(0) ◦ t M (i) (0) + t M−1 γ M−1 ◦ t M + (τ ∗ ◦ t M ) γ M−1 ◦ tM
= γ (i) + (τ ∗ ◦ t M )γ (0)
the thesis now follows by setting μ(i) = τ ∗ . Theorem VIII.2 Given a biorthogonal pair of filter vectors (α, γ), every Laurent polynomial γˆ (0) such that ∗T × ∆ M α (0) = 1 ∆ M γˆ (0)
is given by
(59)
γˆ (0) = γ (0) + λ(1) ◦ t M γ (1) + · · · + λ(M−1) ◦ t M γ (M−1) ,
as λ(1) , λ(2) , . . . , λ(M−1) range over the set of all Laurent polynomials. Proof. Condition (59) yields a linear equation in the M unknowns (γˆ0(i) )∗ , namely,
(i) )∗ , (γˆ1(i) )∗ , . . . , (γˆ M−1
∗ ∗ (0) ∗ α0(0) γˆ0(0) + α1(0) γˆ1(0) + · · · + α (0) ˆ M−1 = 1. M−1 γ
(60)
The M-tuple ∆ M (γ (0) )∗ is a solution of equation (60), while the M-tuples ∆ M (γ (1) )∗ , ∆ M (γ (2) )∗ , . . . , ∆ M (γ (M−1) )∗ are M − 1 linearly independent solutions of the associated homogeneous equation. Hence, the general solution of (60) is given by the vector ∗ ∗ ∗ (61) χ = ∆ M γ (0) + λ(1) ∆ M γ (1) + · · · + λ(M−1) ∆ M γ (M−1) ,
where λ(1) , λ(2) , . . . , λ(M−1) belong to the field of Laurent series. We now prove that the solution χ is a polynomial vector if and only if every λ(i) is a Laurent polynomial. Indeed, suppose that χ is a polynomial solution of Eq. (60). Without loss of generality, we can write every λ(i) as the ratio of two Laurent polynomials, namely, λ(i) = τ (i) /ρ, with τ (i) , ρ ∈ P , i = 1, 2, . . . , M − 1. Then, setting χ′ := ρ(χ − ∆ M γ (0)∗ ), the (M-1)-tuple (τ (1) , . . . , τ (M−1) ) is a polynomial solution of the system of M linear equations in the M − 1
52
BARNABEI AND MONTEFUSCO
unknowns φ (1) , . . . , φ (M−1) : ⎡ (1) ∗ (2) ∗ γ0 ... γ0 (2) ∗ ⎢ (1) ∗ ⎢ γ1 γ1 ... ⎢ ⎢. . . ... ... ⎣ (1) ∗ (2) ∗ ... γ M−1 γ M−1
γ0(M−1)
∗ ⎤
⎡
φ (1)
⎤
⎡
χ0′
⎤
∗ ⎥ ⎢ (2) ⎥ ⎢ ′ ⎥ ⎢ φ ⎥ ⎢ χ1 ⎥ γ1(M−1) ⎥ ⎥×⎢ ⎥ ⎢ ⎥ ⎥ ⎢ .. ⎥ = ⎢ .. ⎥ , ... ⎦ ⎣ . ⎦ ⎣ . ⎦ (M−1) ∗ ′ χ M−1 φ (M−1) γ M−1
(62)
where we have denoted by χi′ the ith component of the polynomial vector χ′ . It is known (Heger, 1858, p. 111) that a linear system whose coefficients belong to a Euclidean domain E admits a solution in E whenever its matrix of coefficients has the same rank k, say, as the augmented matrix, and the g.c.d. of the nonzero minors of order k of the augmented matrix divides the g.c.d. of the minors of the same order of the matrix of coefficients. In our case, the M minors of order M − 1 of the matrix of coefficients are the cofactors of the elements of the zeroth row of the matrix ∆ M (γ)∗ , and therefore their g.c.d. is a monomial. On the other hand, the g.c.d. of the minors of order M − 1 of the augmented matrix is divisible by ρ, and this implies that ρ is a monomial. Theorems VIII.1 and VIII.2 can be equivalently formulated in terms of Hurwitz matrices. In fact, Propositions II.2 and VI.3 allow us to state the following results: Theorem VIII.3 Let (α, γ) be a pair of dual filter vectors. Set Ai = R t M , α (i) , Ci = R t M , γ (i) , i = 0, 1, . . . , M − 1.
For any fixed index i = 1, 2, . . . , M − 1, every Hurwitz matrix Cˆ i = R(t M , γˆ (i) ) such that Cˆ i × A Tj = δi j I, j = 1, 2, . . . , M − 1 is given by Cˆ i = Ci + Ti × C0 ,
as Ti = R t, μ(i) ranges over the set of banded Toeplitz matrices.
(63)
Theorem VIII.4 Let (α, γ) be a pair of dual filter vectors. Set Ai = R t M , α (i) , Ci = R t M , γ (i) , i = 0, 1, . . . , M − 1. Every Hurwitz matrix Cˆ 0 = R t M , γˆ (0) such that Cˆ 0 × A0T = I
is given by Cˆ 0 = C0 +
M−1 j=1
Q j × Cj,
(64)
SUBBAND SIGNAL PROCESSING
53
as Q j = R t, λ( j) , j = 1, 2, . . . , M − 1
range over the set of all banded Toeplitz matrices.
B. Lifting and Dual Lifting The preceding results allow us to extend the lifting scheme to the M-band case, giving a characterization of the algebraic form of all biorthogonal pairs of finite filter vectors that can be obtained from a given pair, either by modifying the analysis low-pass filter and consequently updating the synthesis high-pass filters (lifting), or by modifying the analysis high-pass filters and consequently updating the synthesis low-pass filter (dual lifting). This characterization only makes use of linear combinations of Laurent polynomials, therefore dramatically reducing the computational complexity of designing M-band systems with good filtering properties. Theorem VIII.5 (Lifting Scheme) Let (α, γ) be a dual pair of filter vectors, with α = α (0) , α (1) , . . . , α (M−1) , γ = γ (0) , γ (1) , . . . , γ (M−1) .
Every dual pair (α, γ) of the form α = α (0) , αˆ (1) , . . . , αˆ (M−1) , can be obtained by choosing
γˆ (0) = γ (0) + αˆ (i) = α (i) −
λ(i)
∗
γ = γˆ (0) , γ (1) , . . . , γ (M−1) . M−1 j=1
( j) M ( j) λ ◦t γ ,
◦ t M α (0) ,
i = 1, 2, . . . , M − 1,
(65)
(66) (67)
where λ(1) , λ(2) , . . . , λ(M−1) are arbitrary Laurent polynomials. Proof. Set Ai = R(t M , α (i) ), Ci = R(t M , γ (i) ), i = 0, 1, . . . , M − 1, Aˆ i = R(t M , αˆ (i) ), i = 1, 2, . . . , M − 1, Cˆ 0 = R(t M , γˆ (0) ). By Proposition VI.3, the condition of (α, γ) being a dual pair is equivalent to the following three relations involving the associated Hurwitz matrices: i) Cˆ 0 × A0T = I ,
ii) C j × Aˆ iT = δi j I,
iii) Cˆ 0 × Aˆ iT = 0,
i, j = 1, 2, . . . , M − 1, and i = 1, 2, . . . , M − 1.
54
BARNABEI AND MONTEFUSCO
Making use of the symmetry of the duality relation, Theorems VIII.3 and VIII.4 imply that conditions i) and ii) are satisfied if and only if Cˆ 0 = C0 +
M−1 j=1
Qj × Cj
and Aˆ i = Ai + Ti × A0 ,
i = 1, 2, . . . , M − 1,
where Q j = R(t, λ( j) ), Ti = R(t, μ(i) ) are arbitrary banded Toeplitz matrices. We now get, for every i = 1, 2, . . . , M − 1: M−1 Cˆ 0 × Aˆ iT = C0 + Q j × C j × AiT + A0T × TiT j=1
= C0 × +
AiT
M−1 j=1
+ C0 × A0T × TiT
Q j × C j × AiT +
M−1 j=1
Q j × C j × A0T × TiT
∗ = 0 + TiT + Q i + 0 = R t, μ(i) + R t, λ(i) .
Hence, condition iii) holds if and only if ∗ −R t, μ(i) = R t, λ(i) ,
namely,
∗ μ(i) = − λ(i) .
The above result shows how, starting from a given perfect reconstruction M-channel filter bank, it is possible to modify the analysis low-pass filter, and consequently update the synthesis high-pass filters, while still maintaining the perfect reconstruction property. The polynomials λ(i) in Eqs. (66) and (67) above represent the degrees of freedom that are left after imposing the biorthogonality conditions, and hence they give us full control over all dual pairs (α, γ) of the form (65). In practical situations, especially in the context of image compression, it is sometimes more useful to modify the analysis high-pass filters by adding some desirable properties, such as vanishing moments, prefixed shape, etc., and consequently updating the low-pass synthesis filter. This second construction, known as the dual lifting scheme, is simply obtained by applying the lifting scheme to the pair (γ, α), where α now denotes the analysis filter vector, and γ the synthesis filter vector:
55
SUBBAND SIGNAL PROCESSING
Theorem VIII.6 (Dual Lifting Scheme) Let (α, γ) be a dual pair of filter vectors, with γ = γ (0) , γ (1) , . . . , γ (M−1) . α = α (0) , α (1) , . . . , α (M−1) ,
Every dual pair (α, γ) of the form α = αˆ (0) , α (1) , . . . , α (M−1) ,
γ = γ (0) , γˆ (1) , . . . , γˆ (M−1)
can be obtained by choosing γˆ (i) = γ (i) + λ(i) ◦ t M γ (0) , i = 1, 2, . . . , M − 1, αˆ (0) = α (0) −
M−1 j=1
(68)
( j)∗ M ( j) λ ◦t α ,
(69)
where λ(1) , λ(2) , . . . , λ(M−1) are arbitrary Laurent polynomials. To fully exploit the capabilities offered by the algebraic framework introduced in the previous subsection, we now give an explicit formulation of the lifting scheme in terms of Hurwitz matrices. Theorem VIII.7 Let (C0 , C1 , . . . , C M−1 ), (A0 , A1 , . . . , A M−1 ) be two Mtuples of banded Hurwitz matrices corresponding to a perfect reconstruction M-channel filter bank, namely, Ci × A Tj = δi, j I, i, j = 0, 1, . . . , M − 1. Every pair of M-tuples of banded Hurwitz matrices of the form (A0 , Aˆ 1 , . . . , Aˆ M−1 )
(Cˆ 0 , C1 , . . . , C M−1 ), satisfying ⎡
Cˆ 0 C1 .. .
⎤
⎡
I
0
...
0
⎤
⎢ ⎥ ⎢ I ... 0 ⎥ ⎢0 ⎥ ⎢ ⎥ ⎥ × A0T Aˆ 1T · · · Aˆ TM−1 = ⎢ ⎢ ⎥ ⎦ ⎣. . . . . . . . . . . .⎦ ⎣ C M−1 0 0 ... I
can be obtained by choosing
Cˆ 0 = C0 + and
M−1 i=1
Q i × Ci
Aˆ j = A j − Q Tj × A0 , j = 1, 2, . . . , M − 1, where Q 1 , Q 2 , . . . , Q M−1 are arbitrary banded Toeplitz matrices.
(70)
(71)
56
BARNABEI AND MONTEFUSCO
C. The Algorithms The matrix formulation of lifting suggests a computationally efficient strategy for the realization of the lifted filters. In fact, it is not necessary to explicitly evaluate the new filters given by formulas (66) and (67), since, using (70) and (71), we find that the output of the new filters can be computed starting from the output of the old system, simply by knowing the Toeplitz matrices Q i . More precisely, we can sketch the following optimized analysis–synthesis algorithm. Analysis: M−1 (0) σ Q i × Ci × [σ ] = Cˆ 0 × [σ ] = C0 × [σ ] + i=1
M−1 = σ (0) + Q i × σ (i) ,
(72)
i=1
where
Synthesis:
(i) σ = Ci × [σ ] i = 1, 2, . . . , M − 1.
M−1 M−1 Aˆ iT × σ (i) = A0T × σ (0) + AiT × σ (i) , [σ ] = A0T × σ (0) + i=1
i=1
where we have used (72) to get
(0) (0) M−1 Q i × σ (i) . − = σ σ i=1
We refer to Lazzaro (1999) for a detailed analysis of the computational aspects of this algorithm.
D. Factorization into Lifting Steps The lifting scheme represents a powerful tool for M-band biorthogonal filter construction, particularly due to the fact that it can be iterated. In fact, after adding suitable properties to the analysis low-pass filter, one can use the dual lifting scheme to add other properties to the analysis high-pass filters, thereby obtaining filters with any desired properties after a finite number of lifting steps.
SUBBAND SIGNAL PROCESSING
57
The flexibility of the lifting scheme prompts us to ask whether an M-channel biorthogonal filter bank can be obtained using lifting. In the two-band case, it has been shown (Daubechies and Sweldens, 1998) that every dual pair of finite filters can be obtained with a finite number of lifting steps. In the following, we show that a similar result also holds in the M-channel case, even if this case needs some additional work, due to the higher complexity of handling M × M polynomial matrices. To prove this result, we must make some preliminary considerations. First of all, recalling the bijective correspondence between a filter vector and its M-decimated matrix and using the reconstruction rule given in (17), we remark that the lifting steps (66) and (67) can be rewritten in matrix notation as follows: ∆ M (γ) = ∆ M (γ) × Λ M (λ),
∆ M (α) = ∆ M (α) × Λ M (−λ∗ )T , where λ is the vector whose components are the M − 1 polynomials λ(1) , λ(2) , . . . , λ(M−1) , and Λ M (λ) is the matrix ⎤ ⎡ 1 0 ··· 0 (1) 1. · ·. · 0. ⎥ ⎢ λ Λ M (λ) = ⎣ .. .. ⎦ , .. .. . (M−1) λ 0 ··· 1 which will be called the lifting matrix relative to λ. Consequently, ⎡ 1 ⎢0. ∗ T M Λ (−λ ) = ⎣ . . 0
−λ(1)∗ 1. .. 0
⎤ · · · −λ(M−1)∗ · ·. · 0. ⎥ ∗ −1 T M ⎦ = (Λ (λ ) ) . .. .. ··· 1
The dual lifting step (68), (69) can be similarly represented as ∆ M (γ) = ∆ M (γ) × Λ M (λ)T ,
∆ M (α) = ∆ M (α) × Λ M (−λ∗ ). Moreover, we define the notion of partial lifting matrix as follows: if N < M, let µ = (μ(1) , μ(2) , . . . , μ(N −1) ) be an (N -1)-tuple of Laurent polynomials. The N-order partial lifting matrix relative to µ will be the M × M matrix ! I 0 M−N N (µ) = ΛM . 0 Λ N (µ)
58
BARNABEI AND MONTEFUSCO
We are now able to prove that every dual pair of filter vectors can be obtained with a finite number of lifting and dual lifting steps, alternated with permutations, starting from a suitably scaled trivial pair. Theorem VIII.8 Let (α, γ) be a dual pair of filter vectors. Laurent polynoj mial vectors λk , k = 0, 1, . . . , M − 2, µh , j = 0, 1, . . . , n h , h = 2, 3, . . . , M exist, with (M − k − 1) and (h − 1) components, respectively, such that ∆ M (α) = D
M−2
Λ M−k M (λk )
M
(73)
Ωh ,
h=2
k=0
where D is a diagonal matrix whose determinant is a nonzero monomial, and, for every h = 2, 3, . . . , M, Ωh =
nh j=0
j T ΛhM µh P (h) j ,
P (h) j being suitable permutation matrices acting on the last h columns, and n h an integer bounded by the maximum degree of the nonzero elements of the hth row of ∆ M (α). Moreover, T
∆ M (γ)∗ =
2
h=M
Ω−1 h
0
−1 Λ M−k M (−λk )D .
(74)
k=M−2
Proof. The proof is essentially based on the fact that any square matrix Σ with Laurent polynomial elements whose determinant is a nonzero monomial can be diagonalized by means of right elementary operations, as stated in Gantmacher (1959, Ch.VI, §2). We recall here the main steps of the proof, to highlight the connections between the elementary operations involved and the matricial formulation of lifting and dual lifting steps. The diagonalization process consists of two phases, the first of which reduces the matrix Σ to a lower triangular form by right multiplication by a finite number of permutation matrices and by matrices, with polynomial entries, of the form ⎤ ⎡ 1 0 0 ··· 0 ⎥ ⎢0 1 0 ··· 0 0 ⎥ ⎢· · · · · · · · · · · · · · · ⎢ ⎥ ⎢0 ⎥ 0 · · · 0 1 ⎢ ⎥. ⎢ 1 ξ 1 ξ2 · · · ξ h ⎥ ⎢ ⎥ ⎢ 0 1 0 ··· 0 ⎥ ⎣ ··· ··· ··· ··· ··· ⎦ 0 0 0
···
0
1
59
SUBBAND SIGNAL PROCESSING
Note that these can be seen as dual partial lifting matrices. The second phase reduces the lower triangular matrix previously obtained to a diagonal matrix D by right multiplication by a finite number of matrices of the form ⎤ ⎡ 1 0 0 ··· 0 ⎥ ⎢0 1 0 ··· 0 0 ⎥ ⎢· · · · · · · · · · · · · · · ⎥ ⎢ ⎥ ⎢0 0 ··· 0 1 ⎥, ⎢ ⎢ 1 0 0 ··· 0 ⎥ ⎥ ⎢ ⎢ ψ1 1 0 ··· 0 ⎥ ⎣ ··· ··· ··· ··· ··· ⎦ 0 ψk 0
···
0
1
which correspond to partial lifting steps. We point out that the elementary operations involved in the whole process do not change the determinant of the original matrix, up to a sign. Therefore, the diagonal elements of the matrix D are nonzero monomials. Formula (73) can now be easily deduced from the above factorization result, while (74) is an immediate consequence of (73) and (45). We remark that the proof of Theorem VIII.8 is based upon a factorization theorem which greatly differs from the well-known Smith factorization result, since it makes use only of right multiplication by elementary matrices. In fact, in this context, these are the only operations which can be seen as lifting steps. Example VIII.1 We give now an example of the previous factorization theorem, by considering the factorization of the 4-decimated matrix ∆4 (α) relative to the 4-band filters, with two vanishing moments, constructed in Montefusco and Lazzaro (1997), namely, α (0) = .269 + .394t + .519t 2 + .644t 3 + .230t 4 + .105t 5 − .019t 6 − .144t 7 ,
α (1) = −.098 − .072t − .206t 2 − .180t 3 + .751t 4 + .343t 5 − .064t 6 − .472t 7 , α (2) = .500 − .500t − .500t 2 + .500t 3 ,
α (3) = .217 − .677t + .657t 2 − .237t 3 + .053t 4 + .024t 5 − .004t 6 − .033t 7 , where, for the sake of simplicity, we have written only the first three digits. We have ⎡ ⎤ .269 + .230t −.098 + .751t .500 .217 + .053t ⎢.394 + .105t −.072 + .343t −.500 −.677 + .024t ⎥ ⎥, ∆4 (α) = ⎢ ⎣.519 − .019t −.206 − .064t −.500 .657 − .004t ⎦ .644 − .144t −.180 − .472t .500 −.237 − .033t
60
BARNABEI AND MONTEFUSCO
and the algorithm described in the proof of Theorem VIII.8 yields ∆4 (α) := DΛ4 (λ0 )Λ3 (λ1 )Λ2 (λ2 )Ω2 Ω3 Ω4 , where ⎡ ⎤ .5 0 0 0 0 0 ⎥ ⎢0 .7169 D := ⎣ ⎦, 0 0 1.448 0 0 0 0 −1.9255t ⎡
1 ⎢0 Λ (λ1 ) := ⎣ 0
⎡
1 0 −1 ⎢ −.697t 1 Λ4 (λ0 ) := ⎢ ⎣−.345t −1 0 −.259t −1 0 ⎤ ⎡ 0 1 0 0 0⎥ 0 ⎢0 1 2 , Λ (λ2 ) := ⎣ 0 0 1 0⎦ 0 0 .102t −1 1 ⎤ ⎡ 0 0 1 0 0 0 0 ⎥ ⎢0 1 0 × 1 −.064 − .456t ⎦ ⎣0 0 0 0 1 0 0 1
0 0 1 0 .609t −1 1 0 −.207t −1 0 ⎡ 1 0 0 T (2) ⎢0 1 2 Ω2 = Λ µ2 P0 = ⎣ 0 0 0 0 T T Ω3 = Λ3 µ03 P0(3) Λ3 µ13 ⎤ ⎡ ⎤ ⎡ ⎡ 1 0 0 0 1 0 0 1 0 0 0 ⎢0 1 −.238 + 1.527t −.625⎥ ⎢0 0 1 0⎥ ⎢0 1 .306 × × =⎣ 0 0 1 0 ⎦ ⎣0 1 0 0⎦ ⎣0 0 1 0 0 0 0 0 0 1 0 0 0 1 T Ω4 = Λ4 µ04 P0(4) ⎡ ⎤ ⎡ 1 −.196 + 1.502t +.539 + .460t +.4343 + .106t 0 0 1 0 0 ⎢0 ⎥ ⎢0 1 =⎣ ⎦ × ⎣1 0 0 0 1 0 0 0 0 1 0 0 3
0 0 1 0
⎤ 0 0⎥ ⎥, 0⎦
1 ⎤ 0 0⎥ , 0⎦ 1 ⎤ 0 0⎥ , 1⎦ 0
⎤ 0 .071⎥ 0 ⎦ 1 1 0 0 0
⎤ 0 0⎥ . 0⎦ 1
It is easy to check that the above factorization reduces the computational complexity of the analysis–synthesis algorithm.
IX. Conclusion We have shown how the recursive matrix machinery can be fruitfully used to represent and easily handle the linear operators describing the action of an M-band filter bank. The present work gives a new outlook to this topic,
SUBBAND SIGNAL PROCESSING
61
stating the theory in time-domain language, while still maintaining the main advantages of the polyphase approach. Hence, it permits simple and transparent proofs of the theoretical results, leading, at the same time, to computationally efficient implementations of basic operations of filter theory. In conclusion, the present paper can be seen as an improvement of the existing results on this subject, and it provides a simple tool for the study and the construction of M-band systems. Acknowledgments This research was supported by MIUR, Cofin2000 and R.F.O. projects, and C.N.R. Grant No. 99.1707.
References Akansu, A. N., and Smith, M. J. T., eds. (1996). Subband and Wavelet Transforms. Boston: Kluwer Academic. Bacchelli, S. (1999). Block Toeplitz and Hurwitz matrices: A recursive approach. Adv. Appl. Math. 23, 199–210. Bacchelli, S., and Lazzaro, D. (2001). Some practical applications of block recursive matrices. Comput. Math. Appl. 41, 1183–1198. Barnabei, M., Brini, A., and Nicoletti, G. (1982). Recursive matrices and umbral calculus. J. Algebra 75, 546–573. Barnabei, M., Guerrini, C., and Montefusco, L. B. (1998). Some algebraic aspects of signal processing. Linear Algebra Appl. 284(1–3), 3–17. Barnabei, M., Guerrini, C., and Montefusco, L. B. (2000). An algebraic framework for biorthogonal M-band Filters, in Recent Trends in Numerical Analysis, edited by D. Trigiante. Nova Science Publ., pp. 17–34. Barnabei, M., and Montefusco, L. B. (1998). Recursive properties of Toeplitz and Hurwitz matrices. Linear Algebra Appl. 274, 367–388. Daubechies, I., and Sweldens, W. (1998). Factoring wavelet transforms into lifting steps. J. Fourier Anal. Appl. 4(3), 247–269. Gantmacher, F. R. (1959). The Theory of Matrices. New York: Chelsea. ¨ Heger, I. (1858). Uber die Aufl¨osung eines Systemes von mehreren unbest immtem Gleichungen des erten Grades in ganzen Zahlen. Denkschriften der K¨oniglichen Akademie der Wissenschaften (Wien), Mathematisch-naturwissenshaftliche Klasse. 14(2), 1–122. Lazzaro, D. (1999). Biorthogonal M-band filter construction using the lifting scheme. Num. Algorithm (22), 53–72. Montefusco, L. B., and Lazzaro, D. (1997). Discrete orthogonal transform and M-band wavelets for image compression, in Surface Fitting and Multiresolution Methods, edited by L. Schumacker, A. Le Mehaute, and C. Rabut. pp. 261–270. Nayebi, K., Barnwell, T. P., and Smith, M. J. T. (1987). Time domain conditions for exact reconstruction in analysis/synthesis systems based on maximally decimated filter banks. Proc. Southeastern Symposium on System Theory, pp. 498–503. Strang, G., and Nguyen, T. (1996). Wavelets and Filter Banks. Wellesley-Cambridge Press.
62
BARNABEI AND MONTEFUSCO
Suter, B. W. (1998). Multirate and Wavelet Signal Processing. Boston: Academic Press. Sweldens, W. (1996). The lifting scheme: A custom-designed, construction of biorthogonal wavelets. Appl. Comput. Harmon. Anal. 3(2), 186–200. Sweldens, W. (1997). The lifting scheme: A construction of second generation wavelets. Siam J. Math. Anal. 29(2), 511–546. Tolhuizen, L. M. G., Hollmann, H. D., and Kalker, A. C. M. (1995). On the realizability of bi-orthogonal M-dimensional 2-band filter banks. IEEE Trans. Signal Processing 43(3) 640– 648. Vaidyanathan, P. P. (1993). Multirate Systems and Filter Banks. Englewood Cliffs, NJ: Prentice Hall Signal Processing Series. Vaidyanathan, P. P., and Mitra, S. K. (March 1988). Polyphase networks, block digital filtering, LPTV systems, and alias-free QMF banks: A unified approach based on pseudocirculats. IEEE Trans. Acoust. Speech Signal Processing ASSP-36, 381–391. Vetterli, M., and Kovacevic, J. (1995). Wavelets and Subband Coding. Englewood Cliffs, NJ: Prentice Hall Signal Processing Series. Woods, J. W. (1991). Subband Image Coding. Boston: Kluwer Academic.
ADVANCES IN IMAGING AND ELECTRON PHYSICS, VOL. 125
Determining the Locations of Chemical Species in Ordered Compounds: ALCHEMI I. P. JONES Center for Electron Microscopy, School of Engineering, University of Birmingham, Birmingham B15 2TT, United Kingdom
I. Background . . . . . . . . . . . . . . . . . . . . . . . . . . II. Fundamentals . . . . . . . . . . . . . . . . . . . . . . . . . A. What Is ALCHEMI? . . . . . . . . . . . . . . . . . . . . . B. The Two Basic ALCHEMI Analyses . . . . . . . . . . . . . . 1. Dilute Solutions . . . . . . . . . . . . . . . . . . . . . . 2. Concentrated and Less Strongly Ordered Solutions . . . . . . . 3. A Different Way of Doing Things . . . . . . . . . . . . . . C. The Accuracy of ALCHEMI . . . . . . . . . . . . . . . . . . D. Delocalization and EELS ALCHEMI . . . . . . . . . . . . . . E. Optimizing ALCHEMI . . . . . . . . . . . . . . . . . . . . III. ALCHEMI Results . . . . . . . . . . . . . . . . . . . . . . . A. Minerals . . . . . . . . . . . . . . . . . . . . . . . . . . B. Intermetallics . . . . . . . . . . . . . . . . . . . . . . . . C. Functional Materials . . . . . . . . . . . . . . . . . . . . . D. Quasicrystals . . . . . . . . . . . . . . . . . . . . . . . . E. Investigations Involving a Priori Calculations of Electron Distributions F. EELS ALCHEMI . . . . . . . . . . . . . . . . . . . . . . G. Concentrated Solutions . . . . . . . . . . . . . . . . . . . . IV. Predicting Sublattice Occupancies . . . . . . . . . . . . . . . . . V. Competing (or Supplementary) Techniques . . . . . . . . . . . . . VI. Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . VII. Current Challenges and Future Directions (a Personal View) . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . .
63 64 64 66 66 67 71 73 75 76 77 77 81 93 93 93 97 97 100 102 106 106 107
I. Background The term ALCHEMI was coined by Spence and Taftø in 1983. It stands for Atom Location by Channeling Enhanced Microanalysis. The channeling of electrons between and along the atom planes is exploited via the electrons’ inelastic interactions with the atoms. These give rise to energy losses in the beam and excite x-rays, both of which enable the identification of the chemical natures of the atoms at various positions in the structure. All diffracted waves, whether associated with photons or with particles with rest mass, adopt an inhomogeneous distribution in a regular crystalline or quasicrystalline structure. 63 Copyright 2002, Elsevier Science (USA). All rights reserved. ISSN 1076-5670/02 $35.00
64
I. P. JONES
For heavy particles (e.g., neutrons), this may be understood in terms of particle channeling between the atoms. For x-rays and electrons, the most convenient description is in terms of dynamic diffraction theories. The distribution of radiation or particles in the solid is sensitive to its angle of incidence, particularly around channeling orientations, which, for massy particles, are parallel to the channels between atoms and, for wavelike radiation, are close to Bragg positions. The likelihood of inelastic interactions, including the generation of x-rays, therefore varies from point to point in the solid. In a chemically ordered crystal or quasicrystal, this opens up the possibility of identifying separately the chemical nature of the atoms at each location. Such a technique has been employed using x-rays [the “Borrmann effect” (Borrmann, 1941; Batterman, 1969] and also using ion beams (Morgan, 1973). (See Spence and Taftø, 1982, 1983, and Spence, 1992), for reviews and for further information.) In the electron context, it had for some time prior to 1983 been realized that the channeling of the beam electrons in a transmission electron microscope (TEM), which gives rise via (mainly) thermal diffuse scattering to anomalous absorption and the characteristic bend contours of TEM (Hashimoto et al., 1962), would also give rise to anomalous generation of x-rays (Duncumb, 1962). Cherns et al. (1973) showed, among other things, that the x-ray emission oscillated with specimen thickness, demonstrating practically that the generated x-ray intensity depended on the whole electron wavefunction (including Bloch wave interference) rather than on the individual Bloch waves, mirroring an earlier discussion with respect to electron absorption. Taftø (1979) showed that such channeling in an ordered compound Ax By could give rise to a change in the ratio of characteristic x-ray signals IA :IB . He was clearly aware of the potential of this effect for identifying the chemical nature of the atoms at various positions in the unit cell. The crucial subsequent contribution of Taftø (1982), Taftø and Spence (1982a,b), and Spence and Taftø (1982, 1983) was to show how much firm quantitative chemical information of this type could be deduced from such an experiment without the need for complicated Bloch wave calculations. Earlier reviews of ALCHEMI have been published by Otten (1983), Krishnan (1988), Spence (1992), Buseck and Self (1992), and Horita (1998).
II. Fundamentals A. What Is ALCHEMI? When a TEM specimen is oriented near the Bragg position for a given reflection, the beam electrons are channeled (Hirsch et al., 1965; see Fig. 1a). When the specimen is negative of the Bragg position (negative refers to the
CHEMICAL SPECIES LOCATIONS IN ORDERED COMPOUNDS
65
Figure 1. (a) Bloch wave channeling concentrates the beam electrons on the atom planes when s is −ve and between the planes when s is +ve. (b) In an ordered crystal the electrons are channeled along the heavier atoms (here the unshaded circles) and the apparent chemical composition changes. (Reprinted with permission from Jones, I. P., 1996. How ordered are intermetallics?, in Towards the Millennium. London: Institute of Materials, pp. 267–280.)
deviation parameter “s”; negative “s” corresponds to tilting the crystal toward the symmetry orientation), the electrons are channeled along the atom planes. When the specimen is tilted positive of the Bragg position, the electrons are channeled between the atom planes. This gives rise to the familiar black bend contour, the anomalous absorption depending largely on thermal diffuse scattering. The characteristic x-ray production, which contributes a small amount
66
I. P. JONES
to electron absorption, follows a similar (but not identical) distribution, since both inelastic processes occur close to the atom nuclei. When an ordered specimen is oriented close to a fundamental reflection Bragg position, a similar scenario ensues, and the enhanced x-ray emission reflects the overall composition of the reflecting planes (i.e., that of the crystal as a whole). When an ordered specimen is oriented close to a superlattice Bragg position (a superlattice reflection is one which would disappear if order turned to disorder), the electrons are channeled more down one type of atom position than another. If each type of atom position is considered to be the origin of a sublattice, then tilting the crystal about a superlattice Bragg position favors one sublattice or another (see Fig. 1(b)). In an ALCHEMI experiment, the x-ray or EEL (electron energy loss) spectrum is recorded at kinematic (i.e., not strongly diffracting) and superlattice dynamic orientations, and these spectra are used to infer information concerning the chemical compositions of the two or more sublattices, or, to put it another way, the various atom locations in the crystal. The dynamic orientation may be systematic or axial. There are two ways of analyzing the data. One is appropriate for dilute solutions and one for concentrated solutions. In the former type of analysis [which was the original analysis given by Taftø (1982), Spence and Taftø (1982, 1983), and Taftø and Spence (1982a,b)], the solute atoms whose position is being defined by the experiment are assumed not to perturb the host lattice concentrations or locations. The total amount of solute (i.e., the overall chemical composition of the compound) is not required to be known. In the concentrated solution analysis, the composition is required to be known, and all types of atoms enter the analysis on an equal footing. The dilute solution analysis is a limiting case of the concentrated solution analysis. B. The Two Basic ALCHEMI Analyses 1. Dilute Solutions This is the original analysis given by Taftø (1982), Taftø and Spence (1982a,b), and Spence and Taftø (1982, 1983) and subsequently restated and tidied up by Bentley (1986), Goo (1986), and Otten and Buseck (1987). If A and B occupy separate sublattices and a small amount of C is added, how does C partition between the A and B sublattices? Note that we are not asking to know how much C there is, nor what is the exact chemical composition of the compound. In this case, as Taftø and Spence outlined, the A and B sublattices act as internal calibrators of the dynamic channeling. Imagine that the element C partitions such that a fraction p of it occupies the A sublattice and thus a fraction (1 − p)
CHEMICAL SPECIES LOCATIONS IN ORDERED COMPOUNDS
67
occupies the B sublattice. Furthermore, imagine that the numbers of A, B and C x-rays from the kinematic analysis are NA , NB , and NC and that those from the channeling experiment are N′A , N′B , and N′C . Then if RA = N′A /NA etc., the channeling electron beam current along the A sublattice relative to that in the kinematic condition is RA and for the B sublattice is RB . Then N′c = R C = pRA + (1 − p)RB Nc and p=
RC − RB RA − R B
And there is no more to it than that. The beauty of this is that it is an easy experiment, and does not involve any complicated (and uncertain) dynamic electron diffraction calculations. With a splendid and magisterial disregard for detail, Spence and Taftø did not notice until the proof stage of their J. Mic. (1983) paper that their equations are overdefined. This (in fact trivial) blemish was subsequently removed (independently) by Goo (1986), Bentley (1986), and Otten and Buseck (1987). The simplicity and robustness of this basic ALCHEMI technique explains why it has been so much applied. A list of analyses is given later in this review. Everything written above applies equally well whether a systematic row orientation or a zone axis orientation is used. The former is usually called planar and the latter axial ALCHEMI. Krishnan and Thomas (1984) have examined the extension of this approach to more complicated situations involving several solute species. Their paper is a nice illustration of the advantages of the geometric approach introduced below. More complicated situations (geometrically, chemically . . ..) are perhaps better tackled initially on their own merits. 2. Concentrated and Less Strongly Ordered Solutions Here we do not assume that A occupies one sublattice and that B occupies the other. The state of order is represented by an ordering tie line, or OTL (Matsumura et al., 1991; Hou and Fraser, 1997) (at least for two sublattices) (see Fig. 2). The compositions of each sublattice, SL1 and SL2, describe the state of order of the compound. A kinematic measurement returns the overall composition, which is known and enables the k factors (Cliff and Lorimer, 1975) to be determined. In the simplest case of two equally inhabited sublattices, the overall composition is midway between SL1 and SL2. The dynamic
68
I. P. JONES
Figure 2. SL1 and SL2 are the chemical compositions of two equally populated sublattices and the overall composition is midway between them. In an ALCHEMI experiment the apparent chemical composition moves toward SL1 or SL2.
measurement of chemical concentration will move toward SL1 or SL2,∗ depending upon which type of plane is favored by the particular channeling conditions obtaining (Jones, 1996; Hou et al., 1996). Thus the ALCHEMI experiment defines the slope of the OTL. Because the channeling is never perfect, it cannot define the length of the OTL and thus the absolute state of order of the compound. To do this requires a further, independent, piece of information. Four possibilities are (i) Assume some geometric constraint—e.g., that the tie line intersects (ends at) one side of the triangle. (ii) Measure a superlattice extinction distance. (iii) Calculate the extent of the channeling. (iv) Constrain the beam using a different method. The simplest example of (i) might be thought to be where the tie line intersects two sides of the triangle. This is equivalent to the situation analyzed by Method I, but in fact the OTL analysis as described here (Method II) is not applicable because if the composition of the compound were known exactly, then the state of order would be known also. It is also worth noting that on entropic ∗ Exactly which way the apparent composition moves is quite informative, since tilting negative of the Bragg position will favor the sublattice with the higher structure factor (usually the heavier one). A particularly elegant example of this may be found in Taftø and Gjønnes (1988).
CHEMICAL SPECIES LOCATIONS IN ORDERED COMPOUNDS
69
Figure 3. The chemical composition of one sublattice is assumed. This, along with a knowledge of the overall composition, allows the composition of the other sublattice to be fixed by an ALCHEMI experiment.
grounds this situation, strictly speaking, is forbidden, although it is a useful approximation. There are several examples in the literature of analyses which assume that the tie line is anchored at one side of the triangle. These include those of Otten (1983), Shindo et al. (1986, 1988), Walls (1992), Anderson and Bentley (1994), Horita et al. (1995), and Hao et al. (2000a). These presentations are all algebraic, but add little to the simple geometric construction of Figure 3. Approach (ii), where the supplementary, independent piece of information is a superlattice extinction distance, was introduced and pioneered by Matsumura and colleagues in an investigation of CuAuPd alloys (Matsumura et al., 1991, 1998; Kuwano et al., 1996; and Morimura et al., 1997). In their original experiments, the 001 superlattice extinction distance was measured via the intersecting Kikuchi line method of Gjønnes and Høier (1971). They subsequently applied the method to other compounds and structures (see Horita et al., 1995). This approach does not seem to have been taken up by other researchers, although a combination of approaches (ii) and (iii) (see also below) has obvious attractions. Approach (iii), where the absolute length of the OTL is determined by an iterated Bloch Wave simulation, was introduced by Hou and Fraser (1997) and subsequently used by Sarosi et al. (see Jones, 2001). A conventional Bloch wave calculation predicts the electron intensity at every depth through the foil. This is averaged and then folded with the x-ray generation profile for the atoms. This may be a delta function, or may include delocalization
Figure 4. Real-space crystallography. (a) A scan along undoped BSCCO unit cell establishes the periodicity of the X-ray signal. (b) Static analyses from the HAADF image establish the rough position of the dopant Dy atoms in the unit cell. (From Shang et al., 1999. Phil. Mag. Lett. 79, 741–745, by permission of Taylor and Francis Ltd, http://www.tandf.co.uk/journals)
CHEMICAL SPECIES LOCATIONS IN ORDERED COMPOUNDS
71
(see Section II.D). The absorbed electrons are assumed to contribute the average composition over their remaining trajectories. Because the real and imaginary extinction distances depend upon the answer (the ordering state of the crystal), the calculation must be iterated until the correct experimental x-ray signals are predicted. This occurs quite rapidly (2 or 3 iterations). In ALCHEMI, the dynamic scattering of the electron beam effectively produces a series of small probes at each atom column, repeated over the area of the beam. It is also possible to perform this experiment in real space (considering ALCHEMI as a reciprocal space method) by condensing down the electrons to form a single probe which can be scanned along the unit cell (see Fig. 4; Shang et al., 1999). This approach (iv) really falls outside the remit of this review (ALCHEMI). For this type of analysis (Method II, concentrated solutions), it is worth noting that for three components and three sublattices the OTL must be replaced by an ordering tie triangle, and for four sublattices, a quadrilateral. For four components, we must use a tetrahedron, with the possibility of ordering tie lines, triangles, quadrilaterals (not necessarily planar), again. The ordering state for m sublattices and n elements is described by an m × n matrix, p. Assuming that the overall composition is known, p has (m − 1)(n − 1) degrees of freedom (or parameters to be determined) (Jones and Pratt, 1983). For example, for 3 elements/components and three sublattices: Element → Sublattice ↓
P11 P21 P31
P12 P22 P32
P13 P23 P33
the unshaded elements are independent. 3. A Different Way of Doing Things An approach rather different from the previous two is to measure the variation in x-ray signal either across a range of systematic orientations (Zaluzec and Smith, 2001; Anderson, 2001) or around a zone axis (Josefsson et al., 1994; Rossouw et al., 1996a,b, 1997, 2000; Bastow and Rossouw, 1998; Saitoh et al., 2000; and Rossouw, 2001). These x-ray channeling patterns are called ICPs by Rossouw and colleagues (Incoherent Channeling Patterns) and their measurement derives from original work by Bielicki (1983) and early development by Christenson and Eades (1989). Similarity of detail gives a qualitative clue as to where an element resides (see Fig. 5). A recent attempt to quantify the axial ICP technique (in the sense of extracting sublattice occupancies) was unsuccessful
72
I. P. JONES
a
b
Figure 5. (a) 111 ICPs for TiAl containing 1% Ta. The Ta ICP is similar to the Ti ICP and dissimilar to the Al ICP. Therefore, Ta is on the Ti sublattice. (b) Calculated versions of (a). (Courtesy Chris Rossouw.)
CHEMICAL SPECIES LOCATIONS IN ORDERED COMPOUNDS
73
(Rossouw et al., 2001). There appears to be no report of quantification of the planar version, either. Where the technique has proved particularly helpful is in placing solutes where there is no unequivocal reference lattice. When there is a sparsely inhabited interstitial lattice, which does not sufficiently contribute its own diffracted beams (e.g., Rossouw and Miller, 1999a,b) or a situation where different sublattices do not have detectably different chemical compositions (e.g., Rossouw et al., 2001), it is necessary to compute the ALCHEMI response and compare the result with experiment. The ICP approach (whether axial or systematic) provides a convenient framework for this. In the ALCHEMI results section, I have included a table of examples of this type of investigation (see Table 5, pp. 98–99). Although ICPs do not seem to have been applied to bulk specimens (for this purpose), there seems no reason why they should not be.
C. The Accuracy of ALCHEMI It is clearly desirable to have some idea of how accurate ALCHEMI measurements are. The approach universally adopted (either explicitly or implicitly) is to plot the ALCHEMI results in some form against the strength of channeling, on the principle that the stronger the channeling the more significant is the measurement and the more the weight which should be given to it. Standard regression analysis returns estimates of the errors involved. Thus, in terms of the OTL analysis described above, a straight line is fitted through the experimental points and the further away from the fulcrum (the overall concentration) the experimental measurements lie (i.e., the stronger the channeling), the more effect they have on the OTL. The extension to more components and sublattices is trivial. The best known and most commonly applied approach is that introduced by Rossouw and colleagues and named by them “statistical ALCHEMI” (Rossouw et al., 1989, 1996a). Using nomenclature similar to that in the 1989 paper, and taking the simplest situation where element C is added to ordered compound AB: IA (1 − pxC ) NA = kA IA = channeling intensity on A sublattice. kA is an x-ray generation/ proportionality factor IB (1 − (1 − p)xC ) kB IA xC p + IB xC (1 − p) NC = kC
NB =
74
I. P. JONES
Eliminating IA and IB ,
# " (1 − p)kB NB xC pkA NA + NC = kC 1 − xC p 1 − xC (1 − p)
Writing
NC = αA NA + αB NB
(this defines the OTL)
(1)
then αA = Thus
xC pkA kC 1 − xC p xC =
and
αA αA +
kA kC
αB =
+
xC (1 − p)kB kC 1 − xC (1 − p) αB
αB +
kB kC
and p=
αA αB $ % =1− $ % kA kB xC αA + xC αB + kC kC
(2)
Best fit values of αA and αB are determined from the experimental data, and one of these is then used to calculate p. In this formulation, xC is determined from a combination of channeling chemical analyses, but it could just as easily be determined from a kinematic analysis. Note also that kA /kC and kB /kC require to be known and so the analysis in this form is subject to the logical quibble referred to above, that if, for example, xA were measured exactly, p would be known automatically. In the experimental implementation of Rossouw et al., the electron beam is scanned around a zone axis under computer control for an extended period. It is the weight of data thus accrued which makes the statistical ALCHEMI results reported apparently the most precise, currently. (Whether they are the most accurate remains to be seen.) The advantage of using axial rather than systematic ALCHEMI is that the channeling can be stronger, giving more significant results more quickly. The disadvantage is that delocalization (see Section II.D) corrections appear to be larger (also see below). Statistical ALCHEMI is said (by its authors) to be less affected by delocalization than other approaches, although there appears to be no justification of, nor evidence for, this assertion. In fact, in later versions of their method, Rossouw and colleagues include an additional constant in the fitting Eq. (1) which was said to account for delocalization and some of the deficiencies of their EDX software. Certainly, delocalization will, as this implies, move the OTL (Walls, 1992; Fraser, Hou,
CHEMICAL SPECIES LOCATIONS IN ORDERED COMPOUNDS
75
and Amancherla, private communication). Including a constant like this would be expected to improve the precision of the fit, although whether it improves the accuracy is not obvious (see also L´ab´ar et al., 2000). Other authors who have used straight line fits of ALCHEMI results against strength of channeling include Anderson (1997), L´ab´ar et al., 2000; Hao et al., 2000a; and Jiang, to be published). Otten (1983) and Hao, Yang et al. (1999), among others, have analyzed error transmission in Method I of ALCHEMI.
D. Delocalization and EELS ALCHEMI It is a fundamental assumption of ALCHEMI that the original ionization events all take place at the nuclei of the atoms, and thus that atoms on the same crystal plane see the same electron beam intensity through the specimen. In fact, the probability of ionization goes through a maximum at a distance from the nucleus which increases as the energy required decreases. In classical terms, this is the impact parameter. This effect is called delocalization. Also leading to delocalization of the ionization event from the crystal lattice point is thermal vibration of the atoms. In an ordered crystal, the different atoms may vibrate with different amplitudes. Pre-ALCHEMI, Bourdillon et al. (1981) measured delocalization effects in copper and copper–platinum, using a selection of K, L, and M x-ray peaks. They also estimated impact parameters using time dependent perturbation theory, giving the formula b=
hv 1.24 2π E
(3)
(v, speed of incoming electrons; E, energy loss.) Subsequent to the inception of ALCHEMI, Pennycook (1988) reestimated b as $ %" $ % $ %# hv E 16E 1/2 b= ln ln (4) 2πEt Et Et (v, speed of incoming electrons; E their energy; Et threshold for ionization) for x-ray excitation and " $ %# hv 4E −1/2 ln 2π E for EELS, and advocated the use of simple correction factors which depend on energy. Other approximate treatments are due to Ma and Gjønnes (1992), Walls (1992), Anderson and Bentley (1994), and Horita (1996). (See Horita et al., (1993), for a review and a description of a more experimental approach.)
76
I. P. JONES
More exact and lengthy treatments have been reported by Rossouw and Maslen (1987), von Hugo et al. (1988), N¨uchter and Sigle (1995), and by Allen and Rossouw (1993), Allen et al. (1994), Josefsson et al. (1994), Oxley and Allen (1998, 1999, 2000), and Oxley et al. (1999), the latter series including impressive agreement between theory and experiment for the variation of the oxygen K peak with orientation for a spinel, as well as for other examples. These calculations are not trivial. Furthermore, simple corrections as advocated by Pennycook and others appear not to be very accurate (Oxley and Allen, 2000) and have not found general acceptance. What is generally accepted, however, is that delocalization corrections are much less important for systematic ALCHEMI than for zone axis ALCHEMI (Rossouw and Maslen, 1987; Spence et al., 1988; Qian et al., 1991; Munroe and Baker, 1992; Horita et al., 1993; Hao et al., 2000a; L´ab´ar, 1999; and Rossouw et al., 2001)∗ and decreases as beam voltage decreases (Munroe and Baker, 1992; Ma and Gjønnes, 1992). A reasonable strategy would therefore seem to be (i) to seek to avoid delocalization effects by using, where possible, x-ray peaks of similar energies (e.g., Tian et al., 1992), (ii) if this is not possible, to consider using planar, rather than axial, ALCHEMI [not always feasible—e.g., Otten and Buseck (1987)]. (iii) Finally, if delocalization must be taken into account, it seems sensible to perform a full calculation, perhaps as part of a larger calculation— for example, the Bloch wave calculation approach (iii) to Method II (see Section II.B). The application of EELS to ALCHEMI has been inhibited by the problems of delocalization [which can be worse than for x-ray generation (Qian et al. (1992)] and of more difficult quantification procedures as compared with EDX. Thus, EELS has not made much of a contribution to standard ALCHEMI analyses. Where it does show considerable promise, however, is in identifying the oxidation states of particular ions at specific crystallographic sites. Strangely, this aspect seems to have been neglected ever since the early work of Taftø and Krivanek (1982) on Fe++ and Fe+++ distributions in a chromite spinel (see also Taftø, 1984; Self and Buseck, 1983; and Krishnan, 1989).
E. Optimizing ALCHEMI The statistical aspects of ALCHEMI have been discussed in Section II.C, and the question of planar vs axial ALCHEMI in Section II.D. It is always ∗ Presumably the comment to the opposite effect by Reviere et al. (1993) is a typographical error, especially since it is contradicted earlier in the same paper.
CHEMICAL SPECIES LOCATIONS IN ORDERED COMPOUNDS
77
beneficial to cool the specimen (Spence et al., 1986; Rossouw and Maslen, 1987). Aside from reducing any differences in thermal vibration amplitudes between different elements, more significantly, reducing the temperature reduces the rate at which electrons are dechanneled into diffuse waves which only contribute a uniform background. In addition, like so much microanalysis, the accuracy of ALCHEMI is often limited by the build-up of contamination, which is slowed down by cooling the specimen. Following an earlier paper by Nakata et al. (1991c), Jiang, Hou et al. (1999) have analyzed planar ALCHEMI and have shown, theoretically and experimentally, that (i) the best channeling orientation is the symmetry orientation (this is because the –ve s effects of both ± g sum) (ii) the optimum beam voltage corresponds to a level of dynamic interaction (Jones, 1976) ws = 21/2 + ξg /ξ2g . (iii) Under these conditions, the maximum value of channeling (i.e., the proportion of the OTL which can be visited) is about 0.3–0.5, although this value can be much reduced by absorption. (iv) The optimum foil thickness is ξg /4 to ξg /3 (at the optimum level of dynamic interaction). (v) Beam convergence may safely be increased to approximately the Bragg angle. (vi) Channeling and ALCHEMI experiments disappear at the critical voltage. Beam voltage [(ii) and (vi) above], but not in the context of delocalization, has been investigated and discussed previously by Thomas et al. (1985), Krishnan et al. (1986), and Fujimoto et al. (1994), and there appears to be some disagreement about the effect of a critical voltage on ALCHEMI experiments, particularly for zone axis orientations (see also Horita et al., 1997). Secondary defects (e.g., antiphase domain boundaries (Jiang, Rong et al., 1999) predictably degrade ALCHEMI measurements as their density increases. III. ALCHEMI Results The results of ALCHEMI experiments reported in the literature are presented in the form of a series of tables with commentaries. The classification into tables is not mutually exclusive: some compounds appear in more than one table.
A. Minerals I put Table 1 first because this is where ALCHEMI started. All of the analyses (except those involving a priori calculations of the Bloch wave distribution) are of the “dilute” variety, which may reflect the fact that the chemical bonding
TABLE 1 ALCHEMI Results: Minerals Host compound
Addition
Amphibole
Result
Reference
78
Is not possible to analyze via ALCHEMI Mn on Mg sites, Fe slight preference for Mg Consistent with how formula is written after delocalization correction
Otten (1987)
(X + Z) sites: Mg, Si, Ca, Fe Y sites: Al
Balboni et al. (1994)
Answer implicit in how formula is down written and to some extent in the chemical analysis 1. X3 Y2 Z3 O12 X dodecahedral 2+ Y octahedral 3+ Z tetrahedral 4+ 2. Straight line fit vs. channeling—see Section II.C 3. Calculations here are supportive, rather than essential. 4. Need axial rather than planar ALCHEMI to separate all three types of sites.
Dolomite
Mn, Fe
Garnet (pyrope) Mg3 Al2 Si3 O12 (Mg2.33 Fe0.53 Ca0.33 Mn0.01 ) (Al1.74 Ti0.04 Cr0.04 )Si3 O8 Garnet (pyrope) Mg3 Al2 Si3 O12 (Mg2.883 Fe0.074 Ca0.029 ) Al1.997 Si3 O12 Garnet: Almandine Fe3 Al2 Si3 O12
Fe, Mn, Ca Ti, Cr
Ca, Mn
Ca, Mn on Fe (dodecahedral) site
L´ab´ar (1999)
Garnet: Grossular Ca3 Al2 Si3 O12 Ilmenite
Fe
Fe on Al (octahedral) site
L´ab´ar et al. (2000, 2001)
Mg, Al, Mn, Cr Cr
Mg and Mn on Fe sites, Al on Ti sites, Cr slight preference for Ti 0, 0.25, 0 site
McCormick and Smyth (1987) Rossouw and Miller (1999a,b)
Mullite
Comment
McCormick and Smyth (1987) 1. O8 or O12 ? 2. Mn undetectable 3. Planar ALCHEMI too weak
See Table V
Olivine (Mg0.9 Fe0.1 Ni0.004 Mn0.002 )2 SiO4 Olivine
Ni, Fe, Mn, Cr
% Occupancy of M1 sites: Fe 50, Mn 0, Ni 100
Taftø and Spence (1982a)
% Occupancy of M1 sites: Fe 45, Mn 13, Ni 83, Cr∼64. Doesn’t change much with annealing.
McCormick et al. (1987)
Good agreement with x-ray results
Smyth and Taftø (1982)
At this stage ALCHEMI was called CHEXE! I will leave it for you to judge whether it would have caught on quite so quickly without a name change. There must be a lesson here somewhere.
All Al on M1+T. Most Fe and Mg on M1. 71% of Al on T1 site
McCormick (1986)
020 planar ALCHEMI. Results imply most of the vacancies are on M2. Only one reference lattice, but Al:Si known.
U, Th, Sr, Zr, Mo on Ca sublattice &2% Fe on Ca 28% Fe on Ti Sr and U → Ca Zr → Ti Irradiation results in loss of order
Taftø, Clarke, and Spence (1983) Rossouw et al. (1988b) Zaluzec and Smith (2001)
75% Fe on tetrahedral; Mn on tetrahedral; Ti on octahedral
Taftø, Clarke, and Spence (1983)
Olivine (forsterite) % on M1 of un-heat treated 6 days/300◦ C 48 h/600◦ C 45 h/900◦ C 24 h/1000◦ C
79
Omphacite (nonstoichiometric pyroxene) Orthoclase (K0.90 Na0.08 ) (Al0.99 Fe0.01 Si3.01 )O8 Perovskite CaTiO3 Perovskite CaTiO3
U, Th, Sr, Zr, Mo, Fe Sr, Zr, U
Perovskite CaTiO3
Ulvo-spinel
Fe, Ti, Mn
Fe 50 47 50 50 50
Ni 97 87 83 83 80
Mn Ca 1 0 15 0 15 0 15 0 15 0
Taftø and Buseck (1983)
X-ray profiles across systematic rows give sensitive detection of disordering. Various details in irradiated profiles referred to but not interpreted. Equivalent to high resolution 1-D ICP. (See Calculated table.)
(continues)
TABLE 1—Continued Host compound Spinels: ZnCr0.4 Fe1.6 O4 MgAl2 O4 TiFe2 O4 Spinel (Cr0.37 Fe0.23 Al0.23 Mg0.17 )3 O4
Addition
Ti, V
Spinel (Cr0.37 Fe0.23 Al0.23 Mg0.17 )3 O4 Spinel Cr2 MnO4
Fe
Spinel MgO(Al2 O3 )n , n=1 and 2.4
Spinel MgAl2 O4 Spinel MgAl2 O4
Spinel ZnAl2 O4 3 K-feldspars: sanidine low microcline orthoclase Zirconolite CaZrTi2 O7
3% Fe, 0.3% Mn
Yb
Result
Reference
Cr, Fe octahedral, Zn tetrahedral “normal” spinel “inverse” spinel Cr, Al, Ti and V octahedral. Mg and 3/4 of Fe are tetrahedral Fe2+ on tetrahedral sites, Fe3+ on octahedral sites (Cr1.79 Fe0.21 )(Mn0.9 Fe0.1 )O4
Taftø and Liliental (1982)
Qualitative “pre-ALCHEMI” ALCHEMI
Taftø (1982)
Quantitative “pre-ALCHEMI” ALCHEMI
Taftø and Krivanek (1982)
“pre-ALCHEMI” EELS ALCHEMI Original “statistical” ALCHEMI paper. Corrected for oxygen delocalization by full calculation
Unirradiated n =1 90% Al3+ on O sites, 60% Mg2+ on T sites n = 2.4 27% Mg2+ and 20% Al3+ on T sites Irradiated Disordered—stoichiometric more so than nonstoichiometric Normal spinel (Al octahedral, Mg tetrahedral) If MgAl2 O4 = Mg1−x Alx [Al2−x Mgx ]O4 , x measured at 0.17 62% of Fe and all Mn on Zn (tetrahedral) sites 2t1 : 0.52 (random, as expected) 0.92 0.67 20% of Yb on Ca sublattice, rest on Zr sublattice
Rossouw et al. (1989) Soeda et al. (2000)
Comment
Qian et al. (1992)
Used both x-rays and EELS
Anderson (2001)
Similar method to that of Zaluzec and Smith (2001) (see above). (See Table V.)
Taftø and Spence (1982b) McLaren and FitzGerald (1982) Turner et al. (1991)
Al:Si ordering. 2t1 = fraction of Al on T1 sites. Use fact that Al:Si = 1:3 (see Taftø and Buseck (1983)) Assume Yb not on Ti sites.
CHEMICAL SPECIES LOCATIONS IN ORDERED COMPOUNDS
81
in minerals is of the covalent or ionic type and that this is less tolerant of partial (dis)ordering than the metallic bond. Earth science applications of ALCHEMI were reviewed in 1988 by Smyth and McCormick.
B. Intermetallics Tables 2A–2D have been organized in terms of host intermetallic and, secondarily, date of publication. I have generally used upper case for composition (at% throughout) except where a well-defined stoichiometric compound is involved, in which case I have used subscripts, but always I have made clarity of presentation the controlling factor. The most notable feature is how good the overall agreement is between different authors, which should give us confidence in the results. The host lattice composition has an important influence on occupancy by a third element: several authors have followed the very useful policy of examining a series of compounds such as A50−x B50 Cx , A50−x/2 B50−x/2 Cx , and A50 B50−x Cx , in which case it is the middle member of the series which reveals most clearly element C’s propensities. In terms of a Bragg–Williams interpretation (see Section 4), the occupancies reflect a competition between A–B, A–C, and B–C bond enthalpies. Only when the three bond enthalpies are comparable, in the sense that HA−C /HA−B ∼ HB−C /HA−B (Ochiai et al., 1984; Chiba et al., 1991; and Jiang, 1999) will there be any noticeable effect of A–B stoichiometry on C occupancy. When HA−C /HA−B ≫ HB−C /HA−B , for example, C always substitutes for B and vice versa. Currently the most satisfactory way of comparing and interpreting ALCHEMI results from different, perhaps complicated, alloys of this type is to derive bond enthalpies by matching with Bragg–Williams calculations. There is an obvious connection between sublattice occupancies and the underlying phase diagram, primarily because both depend on the (pairwise aspect of the) bonding. Ochiai et al. (1984) correlated the directions of solubility lobes on the ternary isothermal cross-sections with sublattice occupancy, and there has been general subsequent confirmation of this (see Fig. 6; but see Wu et al., 1989, and Kao et al., 1994). Ultimately (if the solubility lobes illustrated in Fig. 6 become lines), the phase diagram can virtually dictate the ALCHEMI result [see, for example, Chu et al., 1998 (Nb33 Cr42 V25 ) and Takasugi et al., 1990 (Ni3 (Si,Ti)]. This is equally—perhaps more—true of other classes of materials; for example, see Table 1, Balboni et al. (1994). Yang and Hao (1999, 2000) and Yang et al. (2000) have extended this correlation, via bond enthalpies, to the prediction of phase boundaries. This is not always an immediately obvious connection (Rossouw et al., 1996b).
TABLE 2A ALCHEMI Results: Intermetallics: B2 Compounds Host compound
Addition
Nb75Al15Ti10 Nb60Al15Ti25 Nb45Al15Ti40 Nb79Ti10Al11
82
Fe50Al40Ni10 Ni50Al40Fe10
Nb51Ti34Al15 Nb57Ti28Al15 Nb68Ti17Al15 Nb50Ti25Al25 Nb36Ti24Al40 Nb35V40Al25 Nb65V20Al15 NiAl 3, 5, 10% V
V
Result
Reference
Al occupies opposite sublattice from Nb and Ti.
Hou and Fraser (1997)
Al tends to adopt opposite sublattice from Nb and Ti (see Leonard and Vasudevan (2000) above). Al adopts opposite sublattice from Ni and Fe. Ni occupies Fe sublattice in first alloy but Fe occupies Al sublattice in second alloy. Al tends to adopt opposite sublattice from Nb and Ti (i.e., Al-Nb and Al-Ti bonds stronger than Nb-Ti).
Amancherla et al. (2000)
Al all on one sublattice (assumed), Nb and V fill up remaining space impartially. V on Al sublattice
Rong et al. (2002)
Leonard and Vasudevan (2000)
Darolia et al. (1989)
Comment Introduced Bloch wave calculation of OTL length. See also Table 6. OTL framework, but only slopes reported. Includes data from Hou and Fraser (1997). Implies stronger Ni-Al than Fe-Al bonding. Good agreement with Bragg–Williams calculations. See Table 6. OTL framework, but no independent measurement, so ordering tendency (i.e., slope) only. See Table 6.
How much V is in solution? Consistent with phase diagram (note Heusler phase Ni2 AlV).
NiAl: Ni50Al47V3 Ni51Al47.5Mn1.5 Ni51.3Al43.5Cr5.2 NiAl: Ni50Al49Cr1 NiAl: Ni50Al40Fe10 Ni40Al50Fe10 Ni47Al51Fe2 Ni49.75Al49.75Fe0.5 NiAl: Ni50Al48Cu2 Ni48Al50Cu2
83
Ni50Al47-xTi3 Cux, x=1,3,6 Ni50-xAl47Ti3 Cux,x=1,3 NiAl: Ni50-xAl50Fex Ni50-x/2Al50-x/2Fex Ni50Al50-xFex x = 0.25, 2, 5 and 10 FeAl: Fe66Al28Cr6 Ni50Al30Fe20
V, Mn, Cr
V, Mn, Cr all on Al sublattice
Munroe and Baker (1990a, 1992)
Cr Fe
Cr on Al sublattice % Fe on Ni sublattice: 25 88 94 55
Field et al. (1991) Anderson et al. (1995)
Cu All Cu on Al 80% Cu on Ni
Bastow and Rossouw (1998)
Ti inhabits Al sites Cu is indifferent
Wilson and Howe (1999)
Fe
Fe has slight preference for Ni sublattice (e.g., 60 : 40 in x/2 alloys)
Anderson et al. (1999)
Cr
% Occupancy of Fe sublattice 29 30
Munroe and Baker (1990b)
Fe has slight preference for Ni sublattice
NMR gave precise result, but needed calibrating by ALCHEMI. ICPs play supporting role.
Compare results with APFIM, EXAFS, magnetic susceptibility and NMR. See also Intermetallics: other structures table. Assumption of perfect Ni and Al ordering clearly incorrect: need to iterate to improve accuracy (see also Shindo et al. (1988)). (continues )
TABLE 2A—Continued Host compound FeAl: (Fe60 Al40 )95 Cr5 (Fe60 Al40 )90 Cr10 FeAl: Fe49Al50X1
84
FeAl: Fe50Al45X5, X=Ti, V, Cr, Mn, Co, Ni, Cu Fe52Al45Ti3 NiTi Ni50Ti50-xXx Ni50-x/2Ti50-x/2Xx Ni50-xTi50Xx 0<x≤3
Addition
Result
X = Cu, Ni, Co, Mn, Cr, V, Ti Ti, V, Cr, Mn, Co, Ni, Cu
All Cr in solution (up to 6%) on Al sites % X on Fe: Cu Ni Co Mn Cr V T: 96 98 100 100 52 5 93 % X on Fe: Ti V Cr Mn Co Ni Cu 15 19 26 74 99 99 100
Cr
Cr, Mn, Fe, Co, Cu Co, Fe, Pd on Ni. Sc on Ti. Others indifferently distributed, depanding on composition
Ti61.5Al24.5Nb14 Ti65Al24Nb11
(Ti65Al35)90.8V3Fe 6.2 Ti46V30Cr14Al10
(Ti48Al2)(Ti14Nb12Al24) 55% of Nb on Ti sites
V, Fe
V, Fe prefer Al sublattice Cr on Ti sublattice Al on V sublattice
Reference Munroe and Baker (1990c) Kong and Munroe (1994) Anderson (1997)
Nakata, et al. (1991a,b). See also Shimizu and Tadaki (1992) and Tadaki, Nakata, and Shimizu (1995) (reviews) Banerjee et al. (1987) Qian et al. (1991)
Inkson et al. (1993) Li et al. (1998)
Comment
Host element mixing allowed. Planar ALCHEMI: delocalization unimportant. Aging caused significant changes. Good agreement with Bragg–Williams.
B2 structure quenched in. Axial and planar gave same answer. Delocalization more effect for axial.
TABLE 2B ALCHEMI Results: Intermetallics: L12 Compounds Host compound
Addition
Al62Ti27Cr10.5X0.5
Al74.2Ti19Ni6.8
85
Al3 Ti: Al62.8Ti23.8Cu13.4 Al65.0Ti23.1Fe11.9 Al65.1Ti23.0Ni11.9 Ni3 Al: Ni73Al21Fe6 Ni70Al24Co6 Ni76Al21Hf3 Ni3 Al: Ni70Al25Co Ni75Al20Cr5 Ni70Al25Fe5 Ni72.5Al22.5Fe5 Ni75Al20Fe5 Ni3 (Co, Cr) Ni3 Al: Ni75Al20Mn5/600◦ C Ni75Al20Mn5/900◦ C Ni75Al16Mn9/600◦ C Ni3 Al: Ni76Al22Hf2B0.24
Hf Zr W Ni
Result % X occupancy of Al sublattices: 0.0 0.0 10.4 Ni prefers Al sublattice
Cu, Fe, Ni
Reference
Comment
Rossouw et al. (2000)
ICPs play supporting role
Munroe and Baker (1990a, 1991) Ma and Gjønnes (1992)
Implied by formula
Cu, Fe, Ni all on Al Fe, Co, Hf
Fe, Cr, Co
Al
% X on Al: 50 16 100 % Occupancy on Ni sublattice 94 6 70 41 23 Al prefers (Co, Cr) sublattice.
Mn
% Mn on Ni sites 2 25 26
Hf
66% Hf on Al sublattice
Bentley (1986, 1989a)
Axial ALCHEMI. Correction of Spence–Taftø formula.
Shindo et al. (1988)
Ni-Co bonds not exceptionally strong. Al-Cr bonds not exceptionally strong. Fe has moderate preference for Al sublattice: Fe-Ni bonds stronger than Fe-Al Ni3 (Co, Cr) part of two phase mixture in sprayed coating
Fox and Tatlock (1989) Shindo et al. (1990)
Munroe and Baker (1992)
(continues)
TABLE 2B—Continued Host compound Ni3 Al: Ni75Al23Pd2 Ni74Al24Pd2 Ni73Al25Pd2
Addition Pd
Ni3 Al: Ni78Al17Ta5 Ni75Al21Ta4 Ni74Al22Ta4
86
Ni3 Al: Ni75Al17Ti8
Ni3 Al Ni3 Al: Ni76Al21Hf3 Ni3 Al: Ni75.3Al24Zr0.7 Ni73.8Al25.5Zr0.7 Ni3 Al: Ni75Al20Mn5/ 600◦ C Ni75Al20Mn5/900◦ C Ni75Al16Mn9/600◦ C
Result
Reference
% Pd on Ni sites: 100 75 92
Chiba et al. (1991)
All Ta on Al sublattice
Tian et al. (1992) Horita et al. (1997)
Horita et al. (1993) Ti
% Ti on Al sites = 100
2% Re Hf
65% Re on Al sites
Miyazaki et al. (1994) Anderson and Bentley (1995)
78% Hf on Al sites Gu et al. (1997) Zr
Zr on Al sites
Mn
% Mn on Ni sites 2 25 26
Shindo et al. (1990)
Comment Shindo et al. (1986, 1988) analysis. The second and third results make a strange combination. Quoted errors are high. Implies strong Ni-Ta bonding. Consistent with phase diagram. Axial ALCHEMI—minimize delocalization corrections by using TaMα and AlK (similar energies). [100] zone axis gives weak channeling. Axial ALCHEMI. Delocalization corrections necessary. Empirical approach based on disordered alloy of similar composition. Good discussion of delocalization. Axial ALCHEMI with delocalization correction
TABLE 2C ALCHEMI Results: Intermetallics: L10 Compounds Host compound
Addition
Result
Reference
87
TiAl: Ti43Al55Nb2 TiAl: Ti48Al47Nb5 Ti43Al52Nb5
Nb Nb
Nb on Ti sublattice Nb “follows” Ti
Shindo et al. (1986) Konitzer et al. (1986)
TiAl: Ti50Al45Ga5 Ti46.5Al48.5Ga5 TiAl: Ti50Al48V2 Ti44Al54V2 TiAl: Ti50Al48Cr2 Ti44Al54Cr2 TiAl: Ti46Al54Nb2 Ti45.5Al53Mn1.5 Ti47.4Al50V2.6 Ti46.9Al51Cr2.1 TiAl: Ti50Al48Zr2
Ga
All Ga on Al sublattice
Ren et al. (1991)
V
Slightly more V on Ti sublattice V on Ti sublattice Cr mainly on Al sublattice Cr on both sublattices % X on Ti site: 89 3 98 43 All Zr on Ti
Huang and Hall (1991a)
Cr Nb, Mn, V, Cr
Zr
Comment
Authors conscious Ti and Al may not be perfectly ordered
Huang and Hall (1991b)
V small preference for Ti sublattice Cr prefers Al sublattice
Mohandas and Beaven (1991)
Axial and planar ALCHEMI gave same result
Chen et al. (1992) (continues)
TABLE 2C—Continued Host compound
Addition
TiAl: Ti49.5Al49.5Mn1 Ti48Al50Mn2 Ti48al48Mn4 Ti47.5Al47.5Mn5 Ti48Al49Cr3 2% Zr alloy unspecified (but see Chen et al. (1992) (previous line of table)) Ti49Al49Zr1Cr1
88
TiAl TiAl: Ti51Al41Mo8 TiAl: (TiAl)95 Mn5 TiAl: Ti50Al47Cr3 Ti48Al48Cr4 Ti47Al50Cr3 Ti50Al49Ni1 Ti49Al49Ni2 Ti49Al50Ni1 Ti50Al49Zr1 Ti49Al49Zr2 Ti49Al50Zr1
Result
Reference
Comment
Reviere et al. (1993) Mn occupies two sublattices equally
31% of Cr occupies Ti sublattice 96% Zr occupies Ti sublattice
V, Fe (<1% each) Mo Mn Cr, Ni, Zr
These fractions unaffected in quaternary alloy Fe prefers Al sublattice Mo on Al sublattice Mn occupies each sublattice equally % Occupancy of Ti sublattice: 4 48 62 12 0 0 58 98 81
Inkson et al. (1993) Jiang et al. (1995) Holmestad et al. (1995) Shindo (1995) Cr indifferent/amphoteric
Ni prefers Al sublattice
Zr prefers Ti sublattice
TiAl: (Ti52 Al48 )99 X1
X = V, Cr, Mn, Ga, Zr, Nb, Mo, Hf, Ta, W
Ti53Al46X1 Ti51Al46X3 Ti49Al46X5 Ti48Al50X2 Ti47Al51X2
X = Fe,Mn,Cr,V,Nb
89
Ti44Al51X5 Ti42Al53X5 TiAl: Ti46Al51X3 CuAu; Cu50Au50-xPdx, 0<x<25 Cu50Au50-xPdx, x=10, 15 Cu90-yAuyPd10, y=35−47 (CuAu)100−z Pdz , z=5−15 Cu50Au45Ni5 Cu47.5Au47.5Ni5 Cu45Au50Ni5
X also = Ta, Zr, Mo, Ni, Sn, Ga
X = Mn, Cr, Nb Pd
% Occupancy on Ti sublattice varies fairly uniformly from 1 to 0 across the series Hf, Zr, Nb, Ta, V, Mn, W, Mo, Cr, Ga. Ti site occupancy p = 0 for Fe, rises as Al increases for Mn
Rossouw et al. (1996b)
% Occupancy on Ti sublattice: Mn 88%, Nb 89%, Cr 73% 90% Pd on Au sublattice
Jaouen et al. (2000)
Usually Cu on one sublattice and Au, Pd on the other. Au moves to Cu sublattice before Pd. In z series, Pd prefers Au sublattice.
Hao, Xu et al. (1999), Hao, Yang et al. (2000a,b)
Bentley and Hisatsune (1989), Bentley (1989b) Matsumura et al. (1991) Morimura et al. (1992) Kuwano et al. (1996) Morimura et al. (1997) Matsumura et al. (1998)
Ni prefers Cu sublattice
Insufficient solubility of Y, La.
Delocalization correction as per Walls (1992)
Introduction of IKL-ALCHEMI method. (see Section II.B.2) Contains bulk of results. Good description of method. Good agreement with CVM calculations.
TABLE 2D ALCHEMI Results: Intermetallics: Other Compounds Host compound C15 Cr2 Nb (cubic Laves): Nb33Cr42V25
90
C15 Cr2 Nb (cubic Laves): Nb33.3Cr66.7-xXx (I) Nb33.3-x/2 Cr66.7-x/2Xx (II) Nb33.3-xCr66.7Xx (III)
D03 FeAl: Fe66Al28Cr6 D019 Ti3 Al: Ti75Al20Nb5 Ti70Al25Nb5 D019 Ti3 Al: (Ti58 Al42 )99 X1
Addition
Result
Reference
V
V on Cr sublattice
Chu et al. (1998)
% of X occupying Nb sublattice: 0 35 24 93
Okaniwa et al. (1999)
V Type I V Type II V Type III Mo Type III
W Type II W Type III Ti Type II Ti Type III Cr Nb
X = V, Cr, Mn, Ga, Zr, Nb, Mo, Hf, Ta, W
83 100 77 94 % Occupancy of Fe sublattice: 30 Nb “follows” Ti
% Occupancy of Ti sublattice falls from 0.9 to 0.6 across the series Ta, Hf, Zr, Mo, W, Nb, Mn, Cr, V with Ga at 0.2
Comment Possible alloys reflect phase diagram (see Section III.B) V prefers Cr sublattice
Mo does not have strong preference for Cr sublattice W prefers Nb sublattice Ti prefers Nb sublattice Munroe and Baker (1990c)
See also Table 2A (B2)
Konitzer et al. (1986)
Authors conscious Ti and Al may not be perfectly ordered
Rossouw et al. (1996b)
D019 Ti3 Al: Ti72Al26X2, X=V, Ga, Zr, Nb, Sn Ti72.5Al26X1.5, X=Cr, Mn Ti73Al26X1, X=Mo D022 Al3 (V0.6 Ti0.4 ) D023 Al11 Ti4 Zn
V, Cr, Mn, Ga, Zr, Nb, Mo, Sn
Hao, Xu et al. (1999)
Axial ALCHEMI with simple treatment of delocalization
Zn
87% V on ‘V, Ti’ sublattice Zn spread roughly evenly over 3 Al sublattices
Rossouw et al. (2000) Bahierathan and Taftø (1991)
Ni
All Ni on Cu(1)
Nakata et al. (1990). See also Shimizu and Tadaki (1992), Nakata et al. (1993, 1994) and Tadaki, Nakata and Shimizu (1995) (reviews) Tadaki et al. (1990a). See also Shimizu and Tadaki (1992), Nakata et al. (1993, 1994) and Tadaki, Nakata and Shimizu (1995) (reviews)
ICP plays supporting role This might be expected to involve some sort of calculation (see, e.g., Rossouw et al. (2001) and Table 5). Expected DO3 . Shindo et al. (1986) analysis. Assume no Al on Cu(1). Quenching/annealing/aging made no difference.
L21 : Cu67.7Al28.6Ni3.7
91
Ga, Sn on Al sublattice V, Cr, Mn, Zr, Nb, Mo on Ti sublattice (exclusively in all cases)
L21 : Cu35Au21Zn44 (∼ CuAuZn2 )
I: 0.21Au, 0.04Cu II: 0.44Zn, 0.06 Cu III: 0.25Cu
Shindo et al. (1986, 1988) analysis. 3 sublattices, but certain assumptions, since I and III occupy same 200 plane.
(continues)
TABLE 2D—Continued Host compound
Addition
M18R martensite from previous line as host Mg12 (La0.43 Ce0.57 ) Ni3 (Si,Ti) (Si>Ti) Ti2 AlNb
Ti2 AlNb
Result Consistent with Tadaki et al. (1990a)
1% Zn
Zn occupies f sublattice Ti substitutes for Si 3 sublattices: 8g, 4c1 , 4c2 4c1 assumed all Al At high T (950◦ C) composition of 8g ∼ that of 4c2 (designated O1) At low T (700◦ C) 8g: 80% Ti; 4c2 : 40–50% Ti (designated O2) See text, Section III.G
Reference Tadaki et al. (1990b). See also Tadaki, Kondoh et al. (1995). Rossouw et al. (2001) Takasugi et al. (1990) Banerjee et al. (1988) Muraleedharan et al. (1995)
Sarosi et al. (2000) See also Jones (2001)
Comment Cu and Au mix during aging See Table 5
CHEMICAL SPECIES LOCATIONS IN ORDERED COMPOUNDS
93
Figure 6. Solubility lobes for Ni3 Al-X. The direction of a lobe indicates which sublattice X prefers. The narrower the lobe, the more exact is the indication. (Reprinted from Ochiai, S., Oya, Y., and Suzuki, T., 1984. Alloying behaviour of Ni3Al, Ni3Ga, Ni3Si and Ni3Ge. Acta Met. 32, 289–298, with permission from Elsevier Science.)
C. Functional Materials In Table 3A (electronic materials), quite a number of the investigations involve calculations, chiefly because interstitials are quite common in these relatively openly packed structures. There is a surprisingly small number of references for magnetic (Table 3B) and superconducting (Table 3C) materials in view of the fact that solution is the only way to influence the intrinsic properties and that which sublattice is occupied is crucial. D. Quasicrystals Nothing new in principle is involved in Table 4, but again there is a paucity of reported results. E. Investigations Involving a Priori Calculations of Electron Distributions There is a requirement for a priori calculations of the electron distribution when there is no reference lattice available. This generally corresponds to the situation where the position in the unit cell of an interstitial species, although required, is unknown because its occupancy is too sparse to give rise to new diffracted beams (Table 5). The exception to this is the work reported by Rossouw et al. (2001) where three of the sublattices had sufficiently similar chemical compositions as to preclude experimental differentiation.
TABLE 3A ALCHEMI Results: Functional Materials: Electronic Host compound BaTiO3: (Ba0.85 Ca0.15 O)0.98 TiO2 (Ba0.85 Ca0.15 O)1.02 TiO2
Addition
Result
Ca 9% Ca on Ti sublattice 26% Ca on Ti sublattice
Cd1+x In2−2x Snx O4 , 0<x<0.7
94
0.04% Zn
001 Al on 001 GaAs
220 Planar ALCHEMI + calculation shows Al sits over open channels in GaAs Static atomic displacements measured by comparison of angular variation of X-ray production with calculation Sb (implanted) Balance between substitutional, interstitial, and random (precipitate) changes during annealing As substitutional 3 × 1019 As/cm3 Ge Interstitial
Si
Si SiC
Chan et al. (1984)
1. CdIn2 O4 “normal” spinel Brewer et al. 2. As x increases, so does disorder (2000) 3. Sn substitutes for In 9% Zn occupies the H interstitial site. Most Christenson and of the Zn substitutes for Ga. Eades (1989)
GaAs
In0.53 Ga0.47 As
Reference
Comment Can be checked independently by electrical conductivity measurements—qualitative but not quantitative agreement For other spinels see minerals table
Simple Bloch Wave calculations used to locate interstitial Zn (See also Table 5.)
Al-Khafaji et al. (1992) Glas and H´enoc (1987) Pennycook et al. (1984)
See also Rossouw, Potter et al. (1988)
Taftø et al. (1983) Kaiser et al. (2000), Kaiser (2001)
Distribution of As x-rays about 220 Bragg position same as for Si Ge implanted as Ge+ . Planar ALCHEMI matched to Bloch wave calculations. (See also Table 5)
TABLE 3B ALCHEMI Results: Functional Materials: Magnetic Host compound Sendust (Fe2 AlSi) (Fe 9.6wt%Si 5.5wt%Al)
Addition 4%Cr 3%Co+0.5%Nb
95
Sm2 (CoTM)17
Sm2 (Co0.76 Fe0.15 Cu0.07 Ti0.02 )17 Sm(Co0.44 Fe0.11 Cu0.45 )5 Y1.7 Sm0.6 Lu0.7 Fe5 O12
2%Ti Mn, Fe
Result Cr on A1 , A2 , slight preference for A2 , perhaps Co on A1 and A2 , Nb not sufficiently soluble Ti on A2 or B 88% Mn and 85% Fe occupy the mixed planes, the rest on the Co planes % Element on “S” plane: Co 53; Fe 75; Cu 19; Ti 16 Co 73; Fe 12; Cu 9 Sm and Lu octahedral, not tetrahedral or dodecahedral
Reference Sato and Matsuhata (1990)
Comment 002 planar ALCHEMI. 3 sublattices: cf Ti2 AlNb (Jones (2001))
Krishnan et al. (1984) Liu et al. (1989)
Krishnan et al. (1985)
1. S ∼ = Co planes 2. Contradicts Krishnan et al. (1984)—see above Sm and Lu interstitial. (Also Table 5)
TABLE 3C ALCHEMI Results: Functional Materials: Superconductors Host compound
Addition
Nb3 Sn
Ti, Ta
Nb3 Sn
Mo, Zr
PbBiSrCaCuO V3 (Ge0.8 Al0.2 ) YBa2 Cu3 O7−δ : YBa2 Cu3−x Cox O7−δ , x=0.1–1.0
Co
Result
Reference
92% Ti and 96% Ta occupy Nb sites Mo on Nb sublattice Zr undetectable
Taftø et al. (1984) Taftø et al. (1985)
Pb partitions 42% on Cu, 3% on Bi and 55% on Sr Al on Ge sublattice
Goodman and Miller (1993) Taftø et al. (1985) Shindo et al. (1987)
Co on Cu sublattice
Comment
No apparent connection between sublattice occupancy and upper critical magnetic field
Implicit in formula Occupancy implicit in formula (see Section III.B)
CHEMICAL SPECIES LOCATIONS IN ORDERED COMPOUNDS
97
TABLE 4 ALCHEMI Results: Quasicrystals Host compound Al65Cu20Fe15 Al74Mn20Si6 Al68Mn20Ru8Si4 Al65Cu16Co15Si4 (decagonal)
Al72Ni20Co8
Addition
Result
Reference
Fe and Cu on one set of sites, Al on another Si prefers Al sites Ru not on Al sites Si atoms located midway between (00001) planes containing Co, Cu and Al ICPs distinguish Al from TM, but not Ni from Co
Shindo et al. (1989) Shindo et al. (1990) Sigle (1993), N¨uchter and Sigle (1994, 1995) Saitoh et al. (2000)
Comment
Required calculation (sparse interstitial lattice) Decagonal quasicrystal. (See also Table 5)
F. EELS ALCHEMI As with functional materials, it is the small number of results which surprises (Table 6).
G. Concentrated Solutions A relatively small number of results has been published concerning concentrated or substantially disordered solutions (Table 7). Matsumura and colleagues (Matsumura et al., 1991, 1993, 1996, 1998), Horita et al. (1995), Kuwano et al. (1996), Morimura et al. (1992, 1997), and Tomokiyo and Matsumura (1998) have followed OTL development during the annealing of a series of L10 CuAuPd alloys (see Fig. 7). In each case, the ends of the tie-line delineate an S-shaped curve. Hou and Fraser (1997), Amancherla et al. (2000), and Leonard and Vasudevan (2000) have examined a series of B2 NbTiAl alloys (some of them metastable) and FeAlTi alloys. [Figure 8 is from Hou and Fraser (1997) and from Amancherla et al. (2000).] The most obvious feature of these results is the crucial role played by aluminum in producing the B2 order, as also concluded by Banerjee et al. (1988), by Rong et al. (2002), and by Li et al. (1998) for TiAlV. Sarosi et al. (2000) (also see Jones, 2001) have determined ordering triangles for orthorhombic Ti2 AlNb. The three sublattices whose compositions constitute the corners of the triangle are Wyckoff 8g, 4c1 , and 4c2 . Superlattice
TABLE 5 ALCHEMI Results: Experiments Requiring Calculation Host compound
Addition
Result
98
Al65Cu16Co15Si4
Si atoms located midway between (00001) planes containing Co, Cu and Al
Al72 Ni20 Co8
ICPs distinguish Al from TM, but not Ni from Co Irradiation results in loss of order
CaTiO3
GaAs
001 Al on 001 GaAs
In0.53 Ga0.47 As
0.04% Zn
9% Zn occupies the H interstitial site. Most of the Zn substitutes for Ga. 220 planar ALCHEMI + calculation shows Al sits over open channels in GaAs Static atomic displacements measured by comparison of angular variation of X-ray production with calculation
Reference Sigle (1993), N¨uchter and Sigle (1994, 1995) Saitoh et al. (2000) Zaluzec and Smith (2001)
Christenson and Eades (1989) Al-Khafaji et al. (1992) Glas and H´enoc (1987)
Comment Decagonal quasicrystal
Decagonal quasicrystal (See also Table 4) X-ray profiles across systematic rows give sensitive detection of disordering. Various details in irradiated profiles referred to but not interpreted. Equivalent to high resolution 1-D ICP. Simple Bloch Wave calculations used to locate interstitial Zn
Mg12 (La0.43 Ce0.57 )
1% Zn
Zn occupies f sublattice
Rossouw et al. (2001)
Mullite
Cr
0, 0.25, 0 site
Rossouw and Miller (1999)
Si
Sb (implanted)
Pennycook et al. (1984)
SiC
Ge
Balance between substitutional, interstitial and random (precipitate) changes during annealing Interstitial
99 Spinel MgAl2 O4
Y1.7 Sm0.6 Lu0.7 Fe5 O12
If MgAl2 O4 = Mg1−x Alx [Al2−x Mgx ]O4 , x measured at 0.17 Sm and Lu octahedral, not tetrahedral or dodecahedral
1. Mg occupies f, i and j sublattices so require ICP simulation 2. Attempt to quantify occupancy defeated by pattern distortion. 3. Interesting discussion of planar vs zonal ALCHEMI. 4. See also Table 2D Sparse occupation of otherwise unoccupied interstitial sublattice necessitates ICP simulation. See also Rossouw, Potter et al. (1988)
Kaiser et al. (2000), Kaiser (2001) Anderson (2001)
Ge implanted as Ge+ . Planar ALCHEMI matched to Bloch wave calculations. Similar method to that of Zaluzec and Smith (2001) (see above)
Krishnan et al. (1985)
Sm and Lu interstitial
100
I. P. JONES TABLE 6 ALCHEMI Results: EELS Host compound
Chromite spinel: (Cr0.37 Fe0.23 Al0.23 Mg0.17 )3 O4
Addition
Result Tetrahedral Fe2+ , octahedral Fe3+
Reference
Comment
Taftø and See also Taftø Krivanek and Gjønnes (1982) (1988)
systematic rows are of two types, consisting either of alternating 8g and 4(c1 + c2 ) subplanes or of 4(g + c1 ) and 4(g + c2 ) subplanes. Conjoint use of these defines the shape of the ordering tie triangle; its size (reflecting the absolute state of order) was determined by iterative Bloch wave calculations of the electron distribution in the specimens. Neutron diffraction experiments gave good agreement (see Fig. 9). These authors examined also how the shape of the ordering tie triangle varied with temperature in another Ti2 AlNb alloy.
IV. Predicting Sublattice Occupancies There are three types of methods available. In order of increasing sophistication (and, one would imagine, accuracy) they are r r r
Bragg–Williams Quasi-chemical [nowadays always CVM (Cluster Variation Method)] Ab initio and other more general approaches (e.g., Monte Carlo)
Examples of each approach (not in any sense an exhaustive list) may be found as follows: Bragg–Williams: Bragg and Williams (1934), Chang and Neumann (1982), Jones and Pratt (1983), Ochiai et al. (1984), Shindo et al. (1988), Nandy et al. (1990) [see also Raju et al. (1991)], Nakata et al. (1991a,b), Chiba et al. (1991), Sparks et al. (1991), Kao et al. (1994), Shindo (1995), Hao, Xu et al. (1999), Jiang (1999), Amancherla et al. (2000), and Yang et al. (2000) CVM: Wu et al. (1989), Oates and Wenzl (1996), Morimura et al. (1997), Xu et al. (1997), and Abe and Onodera (1999) Ab initio etc: Erschbaumer et al. (1993), Song et al. (1994), Matsumura, Hino et al. (1996), Fu and Zou (1996), Wolf et al. (1996), and Woodward et al. (1998). Bragg–Williams seems to tell most of the story, particularly given the current state of experimental data.
TABLE 7 ALCHEMI Results: Concentrated or Disordered Solutions Host compound Cu50Au50-xPdx, x=10, 15 Cu90-yAuyPd10, y=35-47 (CuAu)100−z Pdz , z=5–15 Cu50Au45Ni5 Cu47.5Au47.5Ni5 Cu45Au50Ni5 Nb75Al15Ti10 Nb60Al15Ti25 Nb45Al15Ti40 Nb79Ti10Al11
Fe50Al40Ni10 Ni50Al40Fe10
Addition
Result
Reference
Usually Cu on one sublattice and Au, Pd on the other. Au moves to Cu sublattice before Pd. In z series, Pd prefers Au sublattice. Ni prefers Cu sublattice (beware typos in Table 2)
Matsumura et al. (1991) Morimura et al. (1992) Kuwano et al. (1996) Morimura et al. (1997) Matsumura et al. (1998) Hou and Fraser (1997)
Al occupies opposite sublattice from Nb and Ti. Al tends to adopt opposite sublattice from Nb and Ti (see Leonard and Vasudevan (2000) above). Al adopts opposite sublattice from Ni and Fe. Ni occupies Fe sublattice in first alloy but Fe occupies Al sublattice in second alloy.
Amancherla et al. (2000)
Nb51Ti34Al15 Nb57Ti28Al15 Nb68Ti17Al15 Nb50Ti25Al25 Nb36Ti24Al40
Al tends to adopt opposite sublattice from Nb and Ti (i.e. Al-Nb and Al-Ti bonds stronger than Nb-Ti).
Leonard and Vasudevan (2000)
Ti2 AlNb
See text, Section III.F
Sarosi et al. (2000). See also Jones (2001)
Comment Introduction of IKL-ALCHEMI method. (see Section II.B.2) Contains bulk of results. Good description of method. Good agreement with CVM calculations. Introduced Bloch wave calculation of OTL length OTL framework, but only slopes reported. Includes data from Hou and Fraser (1997). Implies stronger Ni-Al than Fe-Al bonding. Good agreement with Bragg–Williams calculations. OTL framework, but no independent measurement so ordering tendency (i.e., slope) only
102
I. P. JONES
Figure 7. The ordering of Cu45Au30Pd25 at 573K. The OTL rotates as it lengthens. (Reprinted from Matsumura, S., Furuse, T., and Oki, K., 1998. Time-evolution of long range ordering in CuAuPd ternary alloys. Mat. Trans. JIM 39, 159–168, with permission from the Japan Institute of Metals.)
V. Competing (or Supplementary) Techniques These have been summarized (largely) by Doi et al. (1990): r r r r r r r r r r r r r
X-ray diffraction Neutron diffraction Electron diffraction M¨ossbauer NMR (Nuclear Magnetic Resonance) Magnetic susceptibility EXAFS (Extended X-ray Absorption Fine Structure) APFIM (Atom Probe Field Ion Microscopy) PAC (Perturbed Angular Correlation) Ion channeling RBS (Rutherford Backscattering) Extension of single phase region in ternary phase diagram Real-space crystallography
Doi et al. (1990) provide references to many of these. Other examples, largely taken from references already alluded to in this review, are X-ray diffraction: This is the classical method. Many early references from Johansson and Linde (1925) onward. More recently, Rajamani et al. (1975), McCormick et al. (1987), Porter et al. (1989), Munroe and
CHEMICAL SPECIES LOCATIONS IN ORDERED COMPOUNDS
103
a
b
Figure 8. OTLs for (a) the NbTiAl system. (Reprinted from Hou, D.-H., and Fraser, H. L., 1997. The ordering scheme in Nb aluminides with the B2 crystal structure. Scripta Mat. 36, 617–623, with permission of Elsevier Science.) (includes data from other authors) and (b) the FeNiAl system. (Reprinted from Amancherla, S., Banerjee, R., Banerjee, S., and Fraser, H. L., 2000. Ordering in ternary B2 alloys. Int. J. Refractory Metals Hard Mater. 18, 245–252, with permission of Elsevier Science.)
Baker (1991), Sparks et al. (1991), Babu and Seehra (1991), Morris and G¨unter (1992), Kim and Smith (1997), Kim et al. (1999, 2000) Neutron diffraction: Mozer et al. (1990), Sarosi et al. (to be published—see Jones, 2001) Electron diffraction Matsumura et al. (1991) NMR and magnetic resonance: Golberg and Shevakin (1995) (quoted by Anderson et al., 1999), Shinohara et al. (1995), Bastow and Rossouw (1998)
104
I. P. JONES
EXAFS: Chartier et al. (1994) (quoted by Anderson et al., 1999) and Jaouen et al. (2000) APFIM: Miller and Horton (1986), Duncan et al. (1994), Kim and Smith (1997), Kim et al. (1999, 2000) Extension of single phase region in ternary phase diagram: Ochiai et al. (1984) and subsequent references—Section III.B. Real-space crystallography: Shang et al. (1999) Of course, for every method there is an area where it reigns supreme. The main selling points of ALCHEMI are r r r r
It is robust and easy It does not need standardizing It can be done on very small areas It is still as easy and powerful with 10 solutes as with one.
Figure 9. The three sublattices of the orthorhombic Ti2 AlNb phase. O2 can be considered a structure intermediate between α2 (D019 ) and B2. (From Banerjee et al., 1988). (b) ALCHEMI experiments using the two different types of systematic row define the shape of the ordering tie triangle (the inner tie triangle, for example). An iterated Bloch wave calculation defines the absolute state of order (the outer tie triangle) (see color insert). (c) Neutron diffraction results are in good agreement. (From Sarosi, Jones, and Hriljac, to be published—see Jones, 2001).
CHEMICAL SPECIES LOCATIONS IN ORDERED COMPOUNDS
c
Figure 9. (Continued)
105
106
I. P. JONES
VI. Summary 1. Eighty percent of the results (in terms of detail) come very easily; the other 20% (exact delocalization correction, exact OTL length) require far more effort. 2. There are two ALCHEMI analyses, for dilute and for concentrated solutions. For the concentrated solution analysis, the chemical composition of the compound is required plus one independent (i.e., non-ALCHEMI) assumption or measurement or calculation. 3. Axial ALCHEMI gives larger variations in channeling than does planar ALCHEMI, but is more prey to delocalization. 4. Sparsely occupied interstitial and chemically indistinguishable sublattices require a priori calculations of the electron distribution. 5. Statistics are improved by plotting the ALCHEMI results vs. the strength of channeling, in some fashion. 6. Either delocalization should be avoided as much as possible, or it should be corrected for by a full calculation. 7. The optimum conditions for systematic ALCHEMI are as outlined in Section II.E. 8. To compare compounds with different chemical compositions, use A50−x/2 B50−x/2 Cx or a series including this and/or deduce A-B-C bond enthalpies. 9. Ordering is only sensitive to host atom stoichiometry when HA−C / HA−B ∼ HB−C /HA−B . 10. There is a paucity of results for oxidation state occupancy, for functional materials (magnetic and superconducting) and of detailed results (concentrated solution ALCHEMI) for given compounds over ranges of composition and temperature. 11. Bragg–Williams calculations appear to be capable of predicting most experimental ALCHEMI measurements given the latter’s current state of accuracy. VII. Current Challenges and Future Directions (a Personal View) As detailed in Section VI.10 above, a general look through this review suggests that the following areas are ripe for investigation: r
r
r
A detailed look at one compound over its composition and temperature range The investigation of the oxidation state for atoms at various sites using PEELS/ALCHEMI Functional (especially magnetic and superconducting) materials
CHEMICAL SPECIES LOCATIONS IN ORDERED COMPOUNDS
107
Acknowledgments I thank John Spence for generously giving me access to his formidable archive of ALCHEMI papers and to both John and Chris Rossouw for helpful comments. I also thank Hamish Fraser and Doug Konitzer for introducing me to ALCHEMI back in 1984.
References Abe, T., and Onodera, H. (1999). Prediction of atomic configurations in alloys. Comput. Mater. Sci. 14, 311–317. Al-Khafaji, M. A., Cherns, D., Rossouw, C. J., and Woolf, D. A. (1992). Interface atomic structure determination from Al(001)/GaAs(001) bicrystal. Phil. Mag. B 65, 385–389. Allen, L. J., Josefsson, T. W., and Rossouw, C. J. (1994). Interaction delocalization in characteristic X-ray emission from light elements. Ultramicroscopy 55, 258–267. Allen, L. J., and Rossouw, C. J. (1993). Delocalization in electron-impact ionization in a crystalline environment. Phys. Rev. B 47, 2446–2452. Amancherla, S., Banerjee, R., Banerjee, S., and Fraser, H. L. (2000). Ordering in ternary B2 alloys. Int. J. Refractory Metals Hard Mater. 18, 245–252. Anderson, I. M. (1997). ALCHEMI study of site distributions of 3d-transition metals in B2-ordered iron aluminides. Acta Mater. 45, 3897–3909. Anderson, I. M. (2001). The study of structural disorder in spinel by ALCHEMI. Microsc. Microanal. (MSA 2001) 7 (Supplement 2: Proceedings), 356–357. Anderson, I. M., and Bentley, J. (1994). ALCHEMI formulated for delocalization and anti-site defects. Proc. ICEM13 (Paris, France) 1, 609–610. Anderson, I. M., and Bentley, J. (1995). Site occupancies in aluminum intermetallics by ALCHEMI. EMAG95 (IoP Conference series #147), 531–534. Anderson, I. M., Duncan, A. J., and Bentley, J. (1995). Determination of site occupancies in aluminide intermetallics by ALCHEMI. High Temp. Ordered Intermetallic Alloys VI, 443– 448. Anderson, I. M., Duncan, A. J., and Bentley, J. (1999). Site-distributions of Fe alloying additions to B2-ordered NiAl. Intermetallics 7, 1017–1024. Babu, V. S., and Seehra, M. S. (1991). Site selectivity of Mn atoms in γ -TiAl alloys determined by x-ray scattering. J. Mater. Res. 6, 339–342. Bahierathan, B., and Taftø, J. (1991). Crystal site determination of Zn atoms in the intermetallic material Al11 Ti4 Zn by electron channeling. Micron Microsc. Acta 22, 107–108. Balboni, R., Frabboni, S., Rinaldi, R., and Spigarelli, S. (1994). X-ray-emission modulation by electron channeling and site occupancy in garnets. Mikrochim. Acta 114, 187–194. Banerjee, D., Gogia, A. K., Nandi, T. K., and Joshi, V. A. (1988). A new ordered orthorhombic phase in a Ti3 Al-Nb alloy. Acta Met. 36, 871–882. Banerjee, D., Nandy, T. K., and Gogia, A. K. (1987). Site occupation in the ordered beta phase of ternary TiAlNb alloys. Scripta Met. 21, 597–600. Bastow, T. J., and Rossouw, C. J. (1998). Lattice site determination of dilute Cu in β-phase NiAl by 63 Cu nuclear magnetic resonance and X-ray emission channeling patterns. Phil. Mag. Lett. 78, 461–467. Batterman, B. (1969). Detection of foreign atom sites by their x-ray fluorescence scattering. Phys. Rev. Lett. 22, 703–705.
108
I. P. JONES
Bentley, J. (1986). Site occupations in L12 ordered alloys by axial electron channeling microanalysis. Proc. 44th EMSA, 704–705. Bentley, J. (1989a). Practical limitations of ALCHEMI measurements on ordered intermetallic alloys. EMAG89, 285–288. Bentley, J. (1989b). Electron microscopy of phase transformations in spinodal and ordered alloys. Ultramicroscopy 30, 157–171. Bentley, J., and Hisatsune, K. (1989). Sublattice occupancies of Pd in Cu0.5 Au0.5−x Pdx alloys by ALCHEMI. Proc. 47th EMSA, 234–235. Bielicki, T. A. (1983). Applications of digital mapping to the channeling effect in EDX analysis. EMAG83, 67–70. ¨ Borrmann, G. (1941). Uber Extinktions diagramme von Quare. Physik Z. 42, 157–162. Bourdillon, A. J., Self, P. G., and Stobbs, W. M. (1981). Crystallographic orientation effects in energy dispersive X-ray analysis. Phil. Mag. A, 44, 1335–1350. Bragg, W. L., and Williams, E. J. (1934). The effect of thermal agitation on atomic arrangement in alloys. Proc. R. Soc. A145, 699–730. Brewer, L. N., Kammler, D. R., Mason, T. O., and Dravid, V. P. (2000). Combined CBED/ALCHEMI analysis of crystallography and cation distributions in the transparent conductive oxide Cd1+X In2−2X SnX O4 . Proc. MSA2000 (IOP conf. 165). Buseck, P. R., and Self, P. (1992). Electron energy-loss spectroscopy (EELS) and electron channeling (ALCHEMI). Rev. Min. 27, 141–180. Chan, H. M., Harmer, M. P., Lal, M., and Smyth, D. M. (1984). Calcium site occupancy in BaTiO3 . MRS Symp. 31, 345–350. Chang, Y. A., and Neumann, J. P. (1982). Thermodynamics and defect structure of intermetallic phases with the B2 (CsCl) structure. Prog. Solid. State. Chem. 14, 221–301. Chartier, P., Balasubramanian, M., Brewe, D. L., Manzur, T., Pease, D. M., Budnick, J. I., Huang, L., Law, C. C., Russell, S. M., and Kimball, C. W. (1994). Site selectivity in Fe doped β phase NiAl. JAP 75, 3842–3846. Chen, X. F., Reviere, R. D., Oliver, B. F., and Brooks, C. R. (1992). The site location of Zr atoms dissolved in TiAl. Scripta Met. Mat. 27, 45–49. Cherns, D., Howie, A., and Jacobs, M. H. (1973). Characteristic X-ray production in thin crystals. Z. Naturf. 28a, 565–571. Chiba, A., Shindo, D., and Hanada, S. (1991). Site occupation determination of Pd in Ni3Al by ALCHEMI. Acta Met. Mat. 39, 13–18. Christenson, K. K., and Eades, J. A. (1988). ALCHEMI imaging as an aid to lattice site determination. EUREM88 2, 15–16. Christenson, K. K., and Eades, J. A. (1989). ALCHEMI studies on the location of zinc in GaAs. Proc. 47th EMSA, 584–585. Chu, F., Thoma, D. J., Kotula, P. G., Gerstl, S., Mitchell, T. E., Anderson, I. M., and Bentley, J. (1998). Phase stability and defect structure of the C15 Laves phase Nb(Cr,V)2 . Acta Mater. 46, 1759–1769. Cliff, G., and Lorimer, G. W. (1975). The quantitative analysis of thin specimens. J. Microsc. 103, 203–207. Darolia, R., Lahrman, D. F., Field, R. D., and Freeman, A. J. (1989). Alloy modeling and experimental correlation for ductility enhancement in NiAl. High Temp. Ordered Intermetallic Alloys III, 113–118. Doi, H., Hashimoto, K., Kasahara, K., and Tsujimoto, T. (1990). Site determination of third elements in TiAl compound by X-ray diffractometry. Mat. Trans. JIM 31, 975–982; in Japanese, (1989). JJIM 53, 1089–1097. Duncan, A. J., Kaufman, M. J., Liu, C. T., and Miller, M. K. (1994). Site occupation of iron in intermetallic NiAl. Appl. Surf. Sci. 76/77, 155–159.
CHEMICAL SPECIES LOCATIONS IN ORDERED COMPOUNDS
109
Duncumb, P. (1962). Enhanced X-ray emission from extinction contours in a single crystal gold film. Phil. Mag. 7, 2101–2105. Erschbaumer, H., Podloucky, R., Rogl, P., and Temnitschka, G. (1993). Atomic modelling of Nb, V, Cr, and Mn substitutions in γ -TiAl. I: c/a ratio and site preference. Intermetallics 1, 99–106. Field, R. D., Lahrman, D. F., and Darolia, R. (1991). The effect of alloying on slip systems in 001 oriented NiAl single crystals. Acta Met. Mat. 39, 2961–2969. Fox, P., and Tatlock, G. J. (1989). Ordered phases in CoNiCrAlY overlay coatings on turbine blades. EMAG89, 167–170. Fu, C. L., and Zou, J. (1996). Site preference of ternary alloying additions in FeAl and NiAl by first-principles calculations. Acta Mater. 44, 1471–1478. Fujimoto, T., Tomokiyo, Y., Tanaka, Y., and Hayashi, Y. (1994). Effect of the critical voltage on the characteristic x-ray production. Annu. Rep. HVEM Lab., Kyushu Univ. 18, 23. Gjønnes, J., and Høier, R. (1971). The application of non-systematic many-beam dynamic effects to structure-factor determination. Acta Cryst. A27, 313–316. Glas, F., and H´enoc, P. (1987). Study of static atomic displacements by channelled-electronbeam-induced X-ray emission: application to In0.53 Ga0.47 As alloys. Phil. Mag. A 56, 311– 328. Golberg, D., and Shevakin, A. (1995). Iron in NiAl intermetallic compound: Atomic arrangements and associated magnetic properties and electronic structures. Intermetallics 3, 293–301. Goo, E. (1986). Reformulation of atom location by channeling enhanced microanalysis. Appl. Phys. Lett. 48, 1779. Goodman, P., and Miller, P. (1993). Reassessment of the symmetry of the 221 PbBiSrCaCuO structure using LACBED and high resolution SAD: The relevance of Cowley’s theory of disorder scattering to a real-space structural analysis. Ultramicroscopy 52, 549–556. Gu, Y., Liu, Y., Lin, D., Guo, J., Cao, S., and Zhang, Z. (1997). Site occupation determination of zirconium in Ni3 Al by ALCHEMI. J. Trace Microprobe Techniques 15, 613–615. Hao, Y. L., Xu, D. S., Cui, X. Y., Yang, R., and Li, D. (1999). The site occupancies of alloying elements in TiAl and Ti3 Al alloys. Acta Mater. 47, 1129–1139. Hao, Y. L., Yang, R., Cui, Y. Y., and Li, D. (2000a). Atom location by channeling enhanced microanalysis study of site distributions of alloying elements in titanium aluminides. J. Mat. Res. 15, 2475–2481. Hao, Y. L., Yang, R., Cui, Y. Y., and Li, D. (2000b). The effect of Ti/Al ratio on the site occupancies of alloying elements in γ -TiAl. Intermetallics 8, 633–636. Hao, Y. L., Yang, R., Cui, Y. Y., and Li, D. (1999). Estimation of site occupancy error due to statistical noise for the Ratio ALCHEMI method. Scripta Mat. 41, 1305–1310. Hashimoto, H., Howie, A., and Whelan, M. J. (1962). Anomalous electron absorption effects in metal foils: Theory and comparison with experiment. Proc. R. Soc. A269, 80–103. Hirsch, P. B., Howie, A., Nicholson, R. B., Pashley, D., and Whelan, M. J. (1965). Electron Microscopy of Thin Crystals. London: Butterworths. Holmestad, R., Zuo, J. M., Spence, J. C. H., Høier, R., and Horita, Z. (1995). Effect of Mn doping on charge density in γ -TiAl by quantitative convergent beam electron diffraction. Phil. Mag. A 72, 579–601. Horita, Z. (1996). General formulation for ALCHEMI including delocalization effect. Ultramicroscopy 66, 1–4. Horita, Z. (1998). Quantitative X-ray microanalysis in analytical electron microscopy. Mat. Trans. JIM 39, 947–958. Horita, Z., Kuninaka, H., Sano, T., Nemoto, M., and Spence, J. C. H. (1993). Delocalization corrections using a disordered structure for atom location by channeling-enhanced microanalysis in the Ni-Al system. Phil. Mag. A 67, 425–432.
110
I. P. JONES
Horita, Z., Matsumura, S., and Baba, T. (1995). General formulation for ALCHEMI. Ultramicroscopy 58, 327–335. Horita, Z., McCartney, M. R., and Kuninaka, H. (1997). Channelling-enhanced microanalysis using [111] and [001] zone-axis beam incidence for L12 -type Ni3 (Al,Ta). Phil. Mag. A 75, 153–167. Hou, D.-H., and Fraser, H. L. (1997). The ordering scheme in Nb aluminides with the B2 crystal structure. Scripta Mat. 36, 617–623. Hou, D.-H., Jones, I. P., and Fraser, H. L. (1996). The ordering tie-line method for sublattice occupancy in intermetallic compounds. Phil. Mag. A 74, 741–760. Huang, S.-C., and Hall, E. L. (1991a). Characterization of the effect of vanadium additions to TiAl base alloys. Acta Met. Mat. 39, 1053–1060. Huang, S.-C., and Hall, E. L. (1991b). The effects of Cr additions to binary TiAl-base alloys. Met. Trans. 22A, 2619–2627. Inkson, B. J., Boothroyd, C. B., and Humphreys, C. J. (1993). Microstructure of a γ -α 2-β Ti-Al alloy containing iron and vanadium. Acta Met. Mat. 41, 2867–2876. Jaouen, M., Sikora, T., Hug, G., and Walls, M. G. (2000). An experimental and theoretical study of bonding mechanisms in ternary γ -TiAl + X (X = Cr, Mn, Nb) alloys. J. Physique IV 10, 79–84. Jiang, N. (1999). Predicting sublattice occupancy in ternary intermetallic compounds: An analytical approach. Phil. Mag. A 79, 2953–2963. Jiang, N., Chen, Z., and Jones, I. P. (1995). Sublattice ordering in TiAl-Mo: An experimental (ALCHEMI) and theoretical investigation. Proc. EMAG85 (IOP conf. 147), 539–542. Jiang, N., Hou, D. H., Jones, I. P., and Fraser, H. L. (1999). Optimizing the ALCHEMI technique. Phil. Mag. A 79, 2525–2538. Jiang, N., Rong, T. S., Jones, I. P., and Aindow, M. (1999). On the effect of antiphase domain boundaries on ALCHEMI. Phys. Stat. Sol. (b) 214, 237–243. Johannson, C. H., and Linde, J. O. (1925). R¨ontgenographische Bestimmung der Atamanordnung in den Mischkristallreihen Au-Cu und Pd-Cu. Ann. Physik 78, 439–460. Jones, I. P. (1976). The use of reduced parameters in many-beam electron diffraction theory. Phil. Mag. 33, 311–330. Jones, I. P. (1996). How ordered are intermetallics?, in Towards the Millennium. London: Institute of Materials, pp. 267–280. Jones, I. P. (2001). Chemical crystallography: Real space vs reciprocal space. EMAG2001 (IoP conference series #168), 231–238. Jones, I. P., and Pratt, J. N. (1983). A pair-wise interaction model of the sigma phase. III A general approach. Calphad 8, 257–264. Josefsson, T. W., Allen, L. J., Miller, P. R., and Rossouw, C. J. (1994). K-shell ionization under zone-axis electron-diffraction conditions. Phys. Rev. B 50, 6673–6684. Kaiser, U. (2001). Nanocrystal formation in hexagonal SiC after Ge+ ion implantation. J. El. Mic. 50, 251–263. Kaiser, U., Saitoh, K., and Chuvilin, A. (2000). Analysis of strain and defect formation in lowdimensional structures in SiC. Mater. Sci. Forum 353, 259–262. Kao, C. R., Pike, L. M., Chen, S.-L., and Chang, Y. A. (1994). Site preference of substitutional additions to triple-defect B2 intermetallic compounds. Intermetallics 2, 235–247. Kim, S., and Smith, G. D. W. (1997). AP-FIM investigation on gamma-based titanium aluminides. Mater. Sci. Eng. A 239-240, 229–234. Kim, S., Nguyen-Manh, D., Smith, G. D. W., and Pettifor, D. G. (2000). Site preference of Ru and Pd additions to γ -based TiAl intermetallics. Phil. Mag. A 80, 2489–2508. Kim, S., Larson, D. J., and Smith, G. D. W. (1999). Alloying element partitioning in TiAl-Ru intermetallic alloys. Intermetallics 7, 1283–1290.
CHEMICAL SPECIES LOCATIONS IN ORDERED COMPOUNDS
111
Kong, C. H., and Munroe, P. R. (1994). The effect of ternary additions on the vacancy hardening of FeAl. Scripta Met. Mat. 30, 1079–1084. Konitzer, D. G., Jones, I. P., and Fraser, H. L. (1986). Site occupancy in solid solutions of Nb in the intermetallic compounds TiAl and Ti3 Al. Scripta Met. 20, 265–268. Krishnan, K. M. (1987). Ultramicroscopy 23, 199–203. Krishnan, K. M. (1988). Atomic site and species determinations using channeling and related effects in analytical electron microscopy. Ultramicroscopy 24, 125–142. Krishnan, K. M. (1989). Site occupancy-valence measurements using channeling and related effects in microanalysis. Mat. Sci. Eng. B3, 397–402. Krishnan, K. M. (1990). Channeling and related effects in electron-microscopy—the current status. Scanning Microsc. 157–170. Krishnan, K. M., Rabenberg, L., Mishra, R. K., and Thomas, G. (1984). Site occupation of ternary elements in Sm2 (CoTM)17 compounds. JAP 55, 2058–2060. Krishnan, K. M., Rez, P., and Thomas, G. (1985). Crystallographic site-occupancy refinements in thin-film oxides by channeling-enhanced microanalysis. Acta Cryst. B41, 396– 405. Krishnan, K. M., Rez, P., Thomas, G., Yokota, Y., and Hashimoto, H. (1986). The combined effect of acceleration voltage and incident beam orientation on the characteristic X-ray production in thin crystals. Phil. Mag. B 53, 339–348. Krishnan, K. M., and Thomas, G. (1984). A generalisation of atom location by channeling enhanced microanalysis. J. Microsc. 136, 97–101. Kuwano, N., Matsumura, S., Furuse, T., and Oki, K. (1996). Quantitative analysis of ordered structure in multinary alloys by the IKL-ALCHEMI method and its application to ordering kinetics. J. El. Mic. 45, 93–98. L´ab´ar, J. L. (1999). Dynamic electron scattering distinguishes dodecahedral and tetrahedral crystallographic sites in garnet structure. Appl. Phys. Lett. 75, 70–72. L´ab´ar, J. L., Morgiel, J., T´oth, L., and D´odony, I. (2000). Sites are separable in garnets with ALCHEMI. Mikrochim. Acta 132, 489–492. Leonard, K. J., and Vasudevan, V. K. (2000). Phase equilibria and solid state transformations in Nb-rich Nb-Ti-Al intermetallic alloys. Intermetallics 8, 1257–1268. Li, Y. G., Blenkinsop, P. A., Loretto, M. H., and Walker, N. A. (1998). Effect of aluminium ordering of highly stabilised β-Ti-V-Cr alloys. Mater. Sci. Tech. 14, 732–737. Liu, A., Shindo, D., Hiraga, K., and Hirabayashi, M. (1989). Site occupation of additional elements in Sm-Co-based permanent magnet by ALCHEMI. J. El. Mic. 38, 223–229. Ma, Y., and Gjønnes, J. (1992). Ternary atom location in L12 -structured intermetallic phases— Al62.5+X Ti25−Y (Fe, Ni, or Cu)12.5−Z using ALCHEMI. J. Mat. Res. 7, 2049–2058. Matsumura, S., Furuse, T., and Oki, K. (1993). Kinetics of L10 ordering in CuAuPd ternary alloys, in Diffusion in Ordered Alloys, edited by B. Fultz, R. W. Cahn, and D. Gupta. Warrendale, PA: TMS, pp. 137–149. Matsumura, S., Furuse, T., and Oki, K. (1998). Time-evolution of long range ordering in CuAuPd ternary alloys. Mat. Trans. JIM 39, 159–168. Matsumura, S., Hayata, J., and Kinoshita, C. (1996). Energy-filtered electron diffraction for cation distribution in magnesia aluminate spinel crystals. Annu. Rep. HVEM Lab, Kyushu Univ. 20, 9–12. Matsumura, S., Hino, T., Hata, S., and Oki, K. (1996). Time evolution of 1/2,1/2,1/2∗ special point ordering in fcc-based binary alloy studied by Monte-Carlo simulation. Mat. Trans. JIM 37, 1748–1757. Matsumura, S., Morimura, T., and Oki, K. (1991). An analytical electron diffraction technique for the determination of long-range order parameters in multi-component ordered alloys. Mat. Trans. JIM 32, 905–910.
112
I. P. JONES
McCormick, T. (1986). Crystal-chemical aspects of non-stoichiometric pyroxenes. Am. Min. 71, 1434–1440. McCormick, T. C., and Smyth, J. R. (1987). Minor-element distributions in ilmenite and dolomite by electron channeling-enhanced X-ray emission. Am. Min. 72, 778–781. McCormick, T. C., Smyth, J. R., and Lofgren, G. E. (1987). Site occupancies of minor elements in synthetic olivines as determined by channeling-enhanced X-ray emission. Phys. Chem. Min. 14, 368–372. McLaren, A. C., and FitzGerald, J. D. (1982). CBED and ALCHEMI investigation of local symmetry and Al, Si ordering in K-feldspars. Phys. Chem. Min. 14, 281–292. Miller, M. K., and Horton, J. A. (1986). Site occupation determination by APFIM for Hf, Fe and Co in Ni3 Al. Scripta Met. 20, 1125–1130. Miyazaki, S., Murata, Y., and Morinaga, M. (1994). Site occupancy of Re atoms in Ni3 Al (γ ′ ) and γ -γ ′ phase-equilibria in Ni-Al-Re ternary alloys. Tetsu to Hagane-JISI Japan 80, 161–165. Mohandas, E., and Beaven, P. A. (1991). Site occupation of Nb, V, Mn and Cr in γ -TiAl. Scripta Met. Mat. 25, 2023–2027. Morgan, D. V. (1973). Channelling: Theory, Observation and Applications. Amsterdam: NorthHolland. Morimura, T., Matsumura, S., Hasaka, M., and Oki, K. (1992). An analytical electron diffraction study of L10 -type atomic long-range order depending on alloy-composition in a ternary system of CuAuPd. Electron Microsc. (EUREM92) (Granada, Spain) 2, 301–2. Morimura, T., Matsumura, S., Hasaka, M., and Tsukamoto, H. (1997). Phase equilibria of L10 type order in Cu-Au-Pd and Cu-Au-Ni ternary systems. Phil. Mag. A 76, 1235–1249. Morris, D. G., and G¨unter, S. (1992). Ordering, ternary atom location and ageing in L12 trialuminide alloys. Acta Met. Mat. 40, 3065–3073. Mozer, B., Bendersky, L. A., Boettinger, W. J., and Rowe, R. G. (1990). Neutron powder diffraction study of the orthorhombic Ti2 AlNb phase. Scripta Met. Mat. 24, 2363–2368. Munroe, P. R., and Baker, I. (1990a). An ALCHEMI investigation of ternary site occupancy in NiAl-based alloys. Proc. XIIth ICEM (Seattle, USA), 448–449. Munroe, P. R., and Baker, I. (1990b). Application of ALCHEMI to the study of non-stoichiometric compounds. Proc. XIIth ICEM (Seattle, USA) 472–473. Munroe, P. R., and Baker, I. (1990c). Microstructure and mechanical properties of Fe-40Al+Cr alloys. Scripta Met. Mat. 24, 2273–2278. Munroe, P. R., and Baker, I. (1991). Ternary atom site location in L12 -structured intermetallic compounds. J. Mat. Res. 6, 943–949. Munroe, P. R., and Baker, I. (1992). Effect of accelerating voltage on planar and axial channeling in ordered intermetallic compounds. J. Mat. Res. 7, 2119–2125. Muraleedharan, K., Nandy, T. K., and Banerjee, D. (1995). Phase stability and ordering behaviour of the O phase in TiAlNb alloys. Intermetallics 3, 187–199. Nakata, Y., Tadaki, T., and Shimizu, K. (1990). Site determination of Ni atoms in Cu-Al-Ni shape memory alloys by electron channeling enhanced microanalysis. Mat. Trans. JIM 31, 652–658. Nakata, Y., Tadaki, T., and Shimizu, K. (1991a). Atom location of the third element in Ti-Ni-X shape memory alloys determined by the electron channeling enhanced microanalysis. Mat. Trans. JIM 32, 580–586. Nakata, Y., Tadaki, T., and Shimizu, K. (1991b). Composition dependence of the atom location of the third element in Ti-Ni-X shape memory alloys. Mat. Trans. JIM 32, 1120–1127. Nakata, Y., Tadaki, T., and Shimizu, K. (1991c). Evaluation of accuracy in the ALCHEMI method. Ultramicroscopy 39, 72–80. Nakata, Y., Yamamoto, O., and Shimizu, K. (1993). Effect of aging in Cu-Zn-Al shape memory alloys. Mat. Trans. JIM 34, 429–437. Nakata, Y., Yamamoto, O., Shimizu, K., and Tadaki, T. (1994). Structural changes of Cu-Zn-Al
CHEMICAL SPECIES LOCATIONS IN ORDERED COMPOUNDS
113
shape-memory alloys during martensite aging determined by the ALCHEMI method. Mater. Char. 32, 205–211. Nandy, T. K., Banerjee, D., and Gogia, A. K. (1990). Site substitution behaviour of TiAl intermetallic. Scripta Met. Mat. 24, 2019–2022. N¨uchter, W., and Sigle, W. (1994). The structure of decagonal Al-Cu-Co-Si: An electron channeling study. Phil. Mag. Lett. 70, 103–109. N¨uchter, W., and Sigle, W. (1995). Electron channeling: A method in real-space crystallography and a comparison with the atomic location by channeling-enhanced microanalysis. Phil. Mag. A 71, 165–186. Oates, W. A., and Wenzl, H. (1996). The cluster/site approximation for multicomponent solutions—a practical alternative to the cluster variation method. Scripta Mat. 35, 623–627. Ochiai, S., Oya, Y., and Suzuki, T. (1984). Alloying behaviour of Ni3 Al, Ni3 Ga, Ni3 Si and Ni3 Ge. Acta Met. 32, 289–298. Okaniwa, H., Shindo, D., Yoshida, M., and Takasugi, T. (1999). Determination of site occupancy of additives X (X = V, Mo, W and Ti) in the Nb-Cr-X Laves phase by ALCHEMI. Acta Mat. 47, 1987–1992. Otten, M. T. (1983). A practical guide to ALCHEMI. Philips Electron Optics Bull. 126, 21–28. Otten, M. T. (1987). The ALCHEMI technique: Its applications and limitations. Ultramicroscopy 21, 200. Otten, M. T., and Buseck, P. R. (1987). The determination of site occupancies in garnet by planar and axial ALCHEMI. Ultramicroscopy 23, 151–158. Oxley, M. P., and Allen, L. J. (1998). Delocalization of the effective interaction for inner-shell ionization in crystals. Phys. Rev. B 57, 3273–3282. Oxley, M. P., and Allen, L. J. (1999). Impact parameters for ionization by high-energy electrons. Ultramicroscopy 80, 125–131. Oxley, M. P., and Allen, L. J. (2000). Atomic scattering factors for K-shell and L-shell ionization by fast electrons. Acta Cryst. A56, 470–490. Oxley, M. P., Allen, L. J., and Rossouw, C. J. (1999). Correction terms and approximations for atom location by channeling enhanced microanalysis. Ultramicroscopy 80, 109–124. Pennycook, S. J. (1988). Delocalisation corrections for electron channeling analysis. Ultramicroscopy 26, 239–248; Scanning Microsc. 2, 21–32. Pennycook, S. J., and Narayan, J. (1985). Atom location by axial-electron-channeling analysis. Phys. Rev. Lett. 54, 1543–546. Pennycook, S. J., Narayan, J., and Holland, O. W. (1984). Spatially resolved measurement of substitutional dopant concentrations in semiconductors. Appl. Phys. Lett. 44, 547–549. Porter, W. D., Hisatsune, K., Sparks, C. J., Oliver, W. C., and Dhere, A. (1989). Phase and microstructure of Fe modified Al3 Ti. High Temp. Ordered Intermetallic Alloys III, 657–662. Qian, W. D., Spence, J. C. H., Kuwabara, M., and Strychor, R. (1991). Least-squares axial ALCHEMI for Nb site determination in a TiAl intermetallic alloy. Scripta Met. Mat. 25, 337–341. Qian, W., Totdal, B., Høier, R., and Spence, J. C. H. (1992). Channeling effects on oxygencharacteristic X-ray-emission and their use as reference sites for ALCHEMI. Ultramicroscopy 41, 147–151. Rajamani, V., Brown, G. E., and Prewitt, C. T. (1975). Cation ordering of Ni-Mg olivine. Am. Min. 60, 292–299. Raju, S., Mohandas, E., and Raghunathan, V. S. (1991). Some observations on the paper “Site Substitution Behaviour of TiAl Intermetallic” by T. K. Nandy, D. Banerjee, and A. K. Gogia. Scripta Met. Mat. 25, 975–978. Ren, Y., Chen, G., and Oliver, B. F. (1991). Determination of the lattice site location of Ga in TiAl. Scripta Met. Mat. 25, 249–254.
114
I. P. JONES
Reviere, R. D., Chen, X. F., Oliver, B. F., and Brooks, C. R. (1993). Substitution behaviour of Mn, Cr, and Zr in ternary and quaternary alloys of TiAl. Mater. Sci. Eng. A172, 95–100. Rong, T. S., Horspool, D. N., and Aindow, M. (2002). Microstructure and mechanical behaviour of Nb-Al-V alloys with 10–25 at.% Al and 20–40 at.% V–I: Microstructural observations. Intermetallics 10, 1–12. Rossouw, C. J. (2001). The role of simulation in ALCHEMI. Microsc. Microanal. (MSA 2001) 7 (Supplement 2: Proceedings), 352–353. Rossouw, C. J., Bettles, C. J., Davis, T. J., Forwood, C. T., Miller, P. R., and Venkatesan, K. (2001). Location of Zn within the Mg12 (Lax Ce1−x ) lattice by X-ray incoherent channeling patterns. Acta Cryst. A57, 321–332. Rossouw, C. J., Forwood, C. T., and Gibson, M. A. (2000). Atomic site occupancy in D022 and L12 titanium trialuminides. Phil. Mag. A 80, 1583–1603. Rossouw, C. J., Forwood, C. T., Gibson, M. A., and Miller, P. R. (1996a). Statistical ALCHEMI: General formulation and method with application to Ti-Al ternary alloys. Phil. Mag. A 74, 57–76. Rossouw, C. J., Forwood, C. T., Gibson, M. A., and Miller, P. R. (1996b). Zone-axis convergentbeam electron diffraction and ALCHEMI analysis of Ti-Al alloys with ternary additions. Phil. Mag. A 74, 77–102. Rossouw, C. J., Forwood, C. T., Gibson, M. A., and Miller, P. R. (1997). Generation and absorption of characteristic x-rays under dynamical electron diffraction. Micron 28, 125–137. Rossouw, C. J., and Maslen, V. W. (1987). Localisation and ALCHEMI for zone axis orientations. Ultramicroscopy 21, 277–289. Rossouw, C. J., and Miller, P. R. (1999a). Analysis of incoherent channeling patterns formed by x-ray emission from host lattice species and interstitial Cr in mullite. J. El. Mic. 48, 849–864. Rossouw, C. J., and Miller, P. R. (1999b). Location of interstitial Cr in mullite by incoherent channeling patterns from characteristic X-ray emission. Am. Min. 84, 965–969. Rossouw, C. J., Potter, D. I., and Glanvill, S. R. (1988). Electron channeling analysis of aluminium and xenon-implanted aluminium. Ultramicroscopy 26, 87–96. Rossouw, C. J., Turner, P. S., and White, T. J. (1988a). Axial electron-channeling analysis of perovskite. I. Theory and experiment for CaTiO3 . Phil. Mag. B 57, 209–225. Rossouw, C. J., Turner, P. S., and White, T. J. (1988b). Axial electron-channeling analysis of perovskite. II. Site identification of Sr, Zr and U impurities. Phil. Mag. B 57, 227–241. Rossouw, C. J., Turner, P. S., White, T. J., and O’Connor, A. J. (1989). Statistical analysis of electron channeling microanalytical data for the determination of site occupancies of impurities. Phil. Mag. Lett. 60, 225–232. Saitoh, K., Tanaka, M., Tsai, A. P., and Rossouw, C. J. (2000). ALCHEMI study of an Al72 Ni20 Co8 decagonal quasicrystal. J. Phys. Soc. Japan 69, 2379–2382. Sarosi, P. M., Partridge, A., and Jones, I. P. (2000). The orthorhombic Ti2 AlNb phase. Proc. XIIth ICEM, Czechoslovak Soc. Electron Microsc., Brno 2, 73–74. Sato, K., and Matsuhata, H. (1990). Atomic site determination of Ti, Cr and Co in Sendust (Fe-9.6wt%Si-5.5wt%Al) by channeling enhanced microanalysis, in Microbeam Analysis 1990, edited by J. R. Michael and P. Ingram. San Francisco: San Francisco Press, pp. 293– 296. Self, P. G., and Buseck, P. R. (1983). Low-energy limit to channeling effects in the inelastic scattering of fast electrons. Phil. Mag. 48, L21–L26. Shang, P., Yang, G., Jones, I. P., Gough, C. E., Abell, J. S., Nadarzinski, K., and Kawasaki, M. (1999). Locating dysprosium dopant atoms in the Bi2 Sr2 CaCu2 O y unit cell: Direct-space chemical crystallography. Phil. Mag. Lett. 79, 741–745. Shimizu, K., and Tadaki, T. (1992). Recent studies on the precise crystal-structural analyses of martensitic transformations. Mat. Trans. JIM 33, 165–177.
CHEMICAL SPECIES LOCATIONS IN ORDERED COMPOUNDS
115
Shindo, D. (1995). Recent progress in materials analysis by transmission electron microscopy. Mat. Trans. JIM 36, 473–482. Shindo, D., Hirabayashi, M., Kawabata, T., and Kikuchi, M. (1986). A channeling enhanced microanalysis on niobium atom location in an Al-43%Ti-2%Nb intermetallic compound. J. El. Mic. 35, 409–414. Shindo, D., Hiraga, K., Hirabayashi, M., Tokiwa, A., Kikuchi, M., Syono, Y., Nakatsu, O., Kobayashi, N., Muto, Y., and Aoyagi, E. (1987). Effect of Co substitution on Tc in YBa2 Cu3−x Cox O7−y (x=0∼1). Jap. JAP 26, L1667–L1669. Shindo, D., Hiraga, K., Takasugi, T., and Tsai, A.-P. (1990). Electron channeling enhanced microanalysis on Ni-Al-Mn and Al-Mn-Si. Mat. Trans. JIM 31, 647–651. Shindo, D., Hiraga, K., Williams, T., Hirabayashi, M., Inoue, A., and Masumoto, T. (1989). Electron channeling effect in an Al-Fe-Cu quasicrystal. Jap. JAP 28, L688–L689. Shindo, D., Kikuchi, M., Hirabayashi, M., Hanada, S., and Izumi, O. (1988). Site determination of Fe, Co, and Cr atoms added in Ni3 Al by electron channeling enhanced microanalysis. Trans. JIM 29, 956–961. Shinohara, T., Tadaki, T., and Shimizu, K. (1995). Martensitic phase-transformation in Au-Cu-Zn alloys studied by (Cu)-C-63 NMR. Mat. Trans. JIM 36, 1214–1218. Sigle, W. (1993). Structure refinement of quasicrystalline Al62 Cu20 Co15 Si3 by electron channeling. Phil. Mag. Lett. 68, 39–43. Smyth, J. R., and McCormick, T. C. (1988). Earth science applications of ALCHEMI. Ultramicroscopy 26, 77–86. Smyth, J. R., and Taftø, J. (1982). Major and minor element site occupancies in heated natural forsterite. Geoph. Res. Lett. 9, 1113–1116. Soeda, T., Matsumura, S., Kinoshita, C., and Zaluzec, N. J. (2000). Cation disordering in magnesium aluminate spinel crystals induced by electron or ion irradiation. J. Nucl. Mat. 283-287, 952–956. Song, Y., Tang, S. P., Xu, J. H., Mryasov, O. N., Freeman, A. J., Woodward, C., and Dimiduk, D. M. (1994). Ti-Ti bonding in γ -Tial and f.c.c. Ti. Phil. Mag. B 70, 987–1002. Sparks, C. J., Specht, E. D., Ice, G. E., Zschack, P., and Schneibel, J. (1991). Bonding energies and long-range order in the trialuminides. High Temp. Ordered Intermetallic Alloys IV (MRS Symposium #213), 363–368. (See also following paper by Kumar et al. ibid., pg. 369.) Spence, J. C. H. (1992). Electron channeling and its uses. in Electron Diffraction Techniques Vol. 1, edited by J. M. Cowley. Oxford: OUP, Chapter 10, pp. 465–532. Spence, J. C. H., Graham, R. J., and Shindo, D. (1986). Cold ALCHEMI: Impurity atom site location and the temperature dependence of dechanneling. MRS Symp. 62, 153–162. Spence, J. C. H., Kuwabara, M., and Kim, Y. (1988). Localization effects on quantification in axial and planar ALCHEMI. Ultramicroscopy 26, 103–112. Spence, J. C. H., and Taftø, J. (1982). Atomic site and species determination using the channeling effect in electron diffraction. SEM82 II, 523–531. Spence, J. C. H., and Taftø, J. (1983). ALCHEMI: A new technique for locating atoms in small crystals. J. Microsc. 130, 147–154. Tadaki, T., Kondoh, N., Nakata, Y., and Shimizu, K. (1995). Atomic configuration in bainite of a Cu45Zn40Au15 alloy examined by ALCHEMI. J. Phys. IV 5 (C8), 985–990. Tadaki, T., Nakata, Y., and Shimizu, K. (1995). Occupancy sites of constituent atoms and their effects on the martensitic transformations in some Cu-based and TiNi-based ternary alloys. J. Phys. 5 (C8), 81–90. Tadaki, T., Okazaki, H., Nakata, Y., and Shimizu, K. (1990a). Atomic configuration determined by ALCHEMI and X-ray diffraction of the L21 type parent phase in a Cu-Au-Zn shape memory alloy. Mat. Trans. JIM 31, 935–940.
116
I. P. JONES
Tadaki, T., Okazaki, H., Nakata, Y., and Shimizu, K. (1990b). Atomic configuration studied by ALCHEMI and X-ray diffraction of a stabilised M18R martensite in a β phase Cu-Au-Zn alloy. Mat. Trans. JIM 31, 941–947. Taftø, J. (1979). Channeling effects in electron induced x-ray emission from diatomic crystals. Z. Naturforsch. 34a, 452–458. Taftø, J. (1982). The cation-atom distribution in a (Cr,Fe,Al,Mg)3 O4 spinel as revealed from the channeling effect in electron-induced X-ray emission. J. Appl. Cryst. 15, 378–381. Taftø, J. (1984). Absorption edge fine structure study with subunit cell spatial resolution using electron channeling. Nucl. Instr. Methods Phys. Res. B2, 733–736. Taftø, J., and Buseck, P. R. (1983). Quantitative study of Al-Si ordering in an orthoclase feldspar using an analytical transmission electron microscope. Am. Min. 68, 944–950. Taftø, J., Clarke, D. R., and Spence, J. C. H. (1983). Determination of crystal site occupancy in an improved Synroc D ceramic using analytical electron microscopy. Scientific Basis of Waste Management VI (MRS Symposium #15), 9–13. Taftø, J., and Gjønnes, J. (1988). Microcrystallography using electron channeling: Structure factor phase information and crystal site determination. Ultramicroscopy 26, 97–102. Taftø, J., and Krivanek, O. L. (1982). Site-specific valence determination by electron energy-loss spectroscopy. Phys. Rev. Lett. 48, 560–563. Taftø, J., and Liliental, Z. (1982). Studies of the cation atom distribution in ZnCrx Fe2−x O4 spinels using the channeling effect in electron-induced X-ray emission. J. Appl. Cryst. 15, 260–265. Taftø, J., Sabatini, R. L., and Suenaga, M. (1985). Crystal site occupancy of alloying elements in polycrystalline Nb3 Sn superconductors by electron channeling, in Proc. 43rd EMSA, edited by G. W. Bailey. San Francisco: San Francisco Press, pp. 196–199. Taftø, J., and Spence, J. C. H. (1982a). Crystal site location of iron and trace elements in a magnesium-iron olivine by a new crystallographic technique. Science 218, 49–51. Taftø, J., and Spence, J. C. H. (1982b). Atomic site determination using the channeling effect in electron-induced X-ray emission. Ultramicroscopy 9, 243–247. Tafto, J., Spence, J. C. H., and Fejes, P. (1983). Crystal site location of dopants in semiconductors using a 100keV electron probe. JAP 54, 5014–5015. Taftø, J., Suenaga, M., and Welch, D. O. (1984). Crystal site determination of dilute alloying elements in polycrystalline Nb3Sn superconductors using a transmission electron microscope. JAP 55, 2058–2060. Takasugi, T., Shindo, D., Izumi, O., and Hirabayashi, M. (1990). Metallographic and structural observations in the pseudo-binary section Ni3 Si-Ni3 Ti of the Ni-Si-Ti system. Acta Met. Mat. 38, 739–745. Thomas, G., Krishnan, K. M., Yokota, Y., and Hashimoto, H. (1985). Voltage and orientation dependence of characteristic X-ray production in thin crystals. Proc. 43r d EMSA, 414–415. Tian, W. H., Horita, Z., Honda, Y., Kuninaka, H., Sano, T., and Nemoto, M. (1992). Analytical electron microscopy study of γ /γ ′ phase equilibria in Ni-Al-Ta alloys. Mat. Trans. JIM 33, 1084–1092. Tomokiyo, Y., and Matsumura, S. (1998). Recent development in quantitative electron diffraction for crystallography of materials. Mat. Trans. JIM 39, 927–937. Turner, P. S., White, T. J., O’Connor, A. J., and Rossouw, C. J. (1991). Advances in ALCHEMI analysis. J. Microsc. 162, 369–378. von Hugo, D., Kohl, H., and Rose, H. (1988). Effect of delocalised excitation in channeling enhanced microanalysis. Optik 79, 19–24. Walls, M. G. (1992). A formulation of ALCHEMI for materials containing light elements. Microsc. Microanal. Microstruc. 3, 443–451. Walls, M. G. (1993a). Multivariate ALCHEMI in the presence of anti-site defects. J. Phys. IV 3(C7), 2155–2158.
CHEMICAL SPECIES LOCATIONS IN ORDERED COMPOUNDS
117
Walls, M. G. (1993b). The application of multivariate ALCHEMI to material containing light elements. Proc. EMAG93 (IOP conf. 138), 95–98. Wilson, A. W., and Howe, J. M. (1999). Statistical ALCHEMI study of the site occupancies of Ti and Cu in NiAl. Scripta Mat. 41, 327–331. Wolf, W., Podloucky, R., Rogl, P., and Erschbaumer, H. (1996). Atomic modelling of Nb, V, Cr and Mn substitutions in γ -TiAl. 2: Electronic structure and site preference. Intermetallics 4, 201–209. Woodward, C., Kajihara, S., and Yang, L. H. (1998). Site preferences and formation energies of substitutional Si, Nb, Mo, Ta, and W solid solutions in L10 Ti-Al. Phys. Rev. B 57, 13459– 13470. Wu, Y. P., Tso, N. C., Sanchez, J. M., and Tien, J. K. (1989). Modeling of ternary site occupation in L12 ordered intermetallics. Acta Met. 37, 2835–2840. Xu, D. S., Song, Y., Li, D., and Hu, Z. Q. (1997). Site occupancy of alloying elements in Ti3 Al compounds. Mat. Sci. Eng. A234-A236, 230–233. Yang, R., and Hao, Y. L. (1999). Estimation of (γ + α2 ) equilibrium in two-phase Ti-Al-X alloys by means of sublattice site occupancies of X in TiAl and Ti3 Al. Scripta Mat. 41, 341–346. Yang, R., and Hao, Y. L. (2000). The influence of alloying on the α2 /(α2 + γ )/γ phase boundaries in TiAl based systems. Acta Mat. 48, 1313–1324. Yang, R., Hao, Y., Song, Y., and Guo, Z.-X. (2000). Site occupancy of alloying additions in titanium aluminides and its application to phase equilibrium evaluation. Z. Metallkd. 91, 296– 301. Zaluzec, N. J., and Smith, K. L. (2001). HARECXS measurements of elemental disordering. Microsc. Microanal. (MSA 2001) 7 (Suppl. 2: Proceedings), 354–355.
This Page Intentionally Left Blank
ADVANCES IN IMAGING AND ELECTRON PHYSICS, VOL. 125
Aspects of Mathematical Morphology K. MICHIELSEN,1∗ H. DE RAEDT,1∗ AND J. TH. M. DE HOSSON2∗ 1
Institute for Theoretical Physics, Materials Science Center, University of Groningen, NL-9747 AG Groningen, The Netherlands 2 Department of Applied Physics, Materials Science Center, and Netherlands Institute for Metals Research, University of Groningen, NL-9747 AG Groningen, The Netherlands
I. Introduction . . . . . . . . . . . . . . . . . . . . . . II. Integral Geometry: Theory . . . . . . . . . . . . . . . . A. Image Measurements . . . . . . . . . . . . . . . . B. Minkowski Addition and Subtraction . . . . . . . . . . C. Parallel Sets . . . . . . . . . . . . . . . . . . . . D. Convex Sets and Minkowski Functionals . . . . . . . . E. Convex Rings and Additive Image Functionals . . . . . . F. Relation to Topology and Differential Geometry . . . . . G. Application to Images . . . . . . . . . . . . . . . . H. Integral Geometry on a Hypercubic Lattice . . . . . . . I. Integral Geometry on a Lattice: Alternative Formulation . . III. Integral Geometry in Practice . . . . . . . . . . . . . . A. Minkowski Functionals . . . . . . . . . . . . . . . . B. Computer Program . . . . . . . . . . . . . . . . . C. Analysis of Point Patterns . . . . . . . . . . . . . . . D. Analysis of Digitized and Thresholded Images . . . . . . E. What Integral Geometry Cannot Do . . . . . . . . . . F. Reducing Digitization Errors . . . . . . . . . . . . . G. Normalization of Image Functionals . . . . . . . . . . IV. Illustrative Examples . . . . . . . . . . . . . . . . . . A. Regular Lattices . . . . . . . . . . . . . . . . . . . B. Dislocations . . . . . . . . . . . . . . . . . . . . C. Random Point Sets . . . . . . . . . . . . . . . . . D. Topology of Triply Periodic Minimal Surfaces . . . . . . V. Computer Tomography Images of Metal Foams . . . . . . . A. Computation of 3D Minkowski Functionals . . . . . . . B. Aluminum Foams with Closed-Cell and Open-Cell Structure VI. Summary . . . . . . . . . . . . . . . . . . . . . . . Appendix A: Algorithm . . . . . . . . . . . . . . . . . Appendix B: Programming Example (Fortran 90) . . . . . . Appendix C: Derivation of Eq. (36) . . . . . . . . . . . . Appendix D: Proof of Eq. (56) . . . . . . . . . . . . . . Appendix E: Proof of Eq. (57) . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
120 122 122 123 124 126 128 131 132 135 137 141 142 143 144 145 147 148 149 149 149 153 157 162 170 172 173 176 176 178 182 184 188 190
∗ E-mail addresses:
[email protected];
[email protected]; hossonj@phys. rug.nl
119 Copyright 2002, Elsevier Science (USA). All rights reserved. ISSN 1076-5670/02 $35.00
120
MICHIELSEN ET AL.
I. Introduction Geometrical patterns are encountered in many different fields of science and technology (Ball, 1998; Hyde et al., 1997). Very often these patterns come in the form of photographic images. In general, the purpose of image analysis is to find out what is in these images (Russ, 1995; Castleman, 1996; Gonzalez and Woods, 1993; Rosenfeld and Kak, 1982; da Fontoura and Cesar, Jr., 2001). Describing this information in words is one extreme form of characterizing the image; another extreme form is to assign one or more numbers to the observation. In this paper we only consider the latter. In particular we will analyze two-dimensional (2D) and three-dimensional (3D) patterns by numerical representations of the corresponding images in terms of two-valued functions. Therefore, a numerical characterization of features in the image requires that the image has been digitized, i.e., that the image has been converted to numerical form (Russ, 1995; Castleman, 1996; Gonzalez and Woods, 1993; Rosenfeld and Kak, 1982). This conversion may include additional digital image processing steps (Russ, 1995; Castleman, 1996; Gonzalez and Woods, 1993; Rosenfeld and Kak, 1982) to enhance the quality of the images. If the image contains color or gray-level information, the digitization process should include the mapping of the spatial and color/brightness information in the image onto a collection of black-and-white image elements (Russ, 1995; Castleman, 1996; Gonzalez and Woods, 1993; Rosenfeld and Kak, 1982). For simplicity we will use the term pixel to refer to both 2D and 3D image elements. Morphology is a branch of biology dealing with the form and structure of animals and plants. The same word is used for the study of the geometry and topology of patterns. Integral-Geometry Morphological Image Analysis (MIA) employs additive image functionals to assign numbers to the shape and connectivity of patterns formed by the pixels in the image. Integral geometry (Hadwiger, 1957; Santal´o, 1976; Stoyan et al., 1989) provides a rigorous mathematical framework to define these image functionals. A fundamental theorem (discussed below) of integral geometry (Hadwiger, 1957) states that under certain conditions, the number of different additive image functionals is equal to the dimension of the pattern plus one. Thus, in the case of a 2D (3D) image there are exactly 3 (4) of these functionals, called quermassintegrals or Minkowski functionals. For a given image, the first step in MIA is to compute these functionals themselves. The second step is to study the behavior of the three or four numbers as a function of some control parameters, such as time, and density. A remarkable feature of MIA is the big contrast between the simplicity of implemention and use and the level of sophistication of the mathematical theory. Indeed, as will be explained below, the calculation of the image
ASPECTS OF MATHEMATICAL MORPHOLOGY
121
functionals merely amounts to the proper counting of faces, edges, and vertices of pixels. The application of MIA requires little computational effort. Another appealing feature of MIA is that the image functionals have a geometrically and topologically clear interpretation. For 2D images, they correspond to the area, boundary length, and connectivity number. The four functionals for 3D images are the volume, surface area, integral mean curvature, and connectivity number. This paper gives an overview of the various aspects of MIA, with an emphasis on the practical application. MIA has proven to be very useful to describe the morphology of porous media and complex fluids (Mecke and Wagner, 1991; Mecke, 1996a, 1998a,b), the large-scale distribution of matter in the Universe (Mecke 1998b; Mellot, 1990; Mecke et al., 1994), regional seismicity realizations (Makarenko et al., 2000), quantum motion in billiards (Kole et al., 2001), microemulsions (Mecke, 1998b; Likos et al., 1995), patterns in reaction diffusion systems (Mecke, 1996b, 1998b), spinodal decomposition kinetics (Mecke, 1998b; Mecke and Sofonea, 1997; Sofonea and Mecke, 1999), and the dewetting structure in liquid crystal and liquid metal films (Herminghaus et al., 1998), and in polymer films (Jacobs et al., 1998). In many cases, additional information can be extracted from the pattern by making assumptions about size, shape, and distribution of the objects. Usually this involves making a probabilistic model of the pattern and comparing the Minkowski functionals of the model with those of the images. Applications of this stochastic-geometry approach to model natural phenomena can be found in Stoyan et al. (1989). In the preceding discussion, we took for granted that the digitized images are free of noise and other artifacts that may affect the geometry and topology of the structures of interest. Such perfect images are easily generated by computer and are very useful for the development of theoretical concepts and models. Unfortunately, genuine images or patterns obtained from computer simulations are seldom perfect. Therefore, some form of image processing may be necessary before attempting to make measurements of the features in the image. Digital image processing is very important for many industrial, medical, and scientific applications. There is a vast amount of literature on this subject, so we can only cite a few books here (Russ, 1995; Castleman, 1996; Gonzalez and Woods, 1993; Rosenfeld and Kak, 1982). There also is a huge number of different processing steps and methods. The types of measurements that will be performed on the image is an important factor in making a selection of the most appropriate processing steps. In MIA, the geometric and topological content of the image are of prime importance, and this should be reflected in the operations that are used to enhance the image quality. The morphological image processing (MIP) technique (Matheron, 1975; Serra, 1982; Giardina and Dougherty, 1988; Terol-Villalobos,
122
MICHIELSEN ET AL.
2001) is well adapted for this purpose because MIP and MIA are based on the same mathematical roots. In the present review we will not discuss this very important aspect of image processing but focus entirely on the problem of pattern characterization in terms of morphological descriptors. This review is structured as follows. Section II reviews the mathematical concepts of integral geometry as far as they are relevant to MIA. In Section III, we describe how to use MIA in practice and provide algorithms (including examples of computer code) to compute the morphological descriptors. In Section IV, we apply MIA to simple point patterns, random point sets, and geometric objects (minimal surfaces) to illustrate the salient features of MIA. Section V discusses the application of MIA to computer tomography images of metal foams. Our conclusions are given in Section VI.
II. Integral Geometry: Theory In this section, we review the theoretical framework that lies at the heart of integral-geometry-based morphological image analysis. For simplicity we adopt the following conventions: r
r
r
We make no distinction between 2D and 3D images; for example, we always write “pixel” if we also actually mean voxel. We consider blackand-white images only, unless mentioned explicitly. The background color of an image is white. Pixels that are set are black, others are white. This corresponds to the way one normally draws and prints black-and-white images on paper. On a display, the roles of black and white are reversed. Definitions are printed slanted.
A. Image Measurements The purpose of image analysis is to extract information from the image. Integral (and differential) geometry provides information on the geometric and topological structure of sets of points in Euclidean space. In an image, the presence of a point in space is represented by a pixel that is black. Thus, objects in Euclidean space are represented by a collection of black pixels. Empty space corresponds to white pixels. A function that assigns a number to a black-and-white image is called an image functional (Giardina and Dougherty, 1988). An image functional performs a measurement of certain properties or features in the image, such as the brightness, or location of objects, their surface, perimeter, and size
ASPECTS OF MATHEMATICAL MORPHOLOGY
123
distribution. An example of an image functional ϕ is the area of black pixels on a background of white pixels. If P1 and P2 are two patterns of black pixels, we obviously have ϕ(P1 ∪ P2 ) = ϕ(P1 ) + ϕ(P2 ) − ϕ(P1 ∩ P2 ).
(1)
The last term in (1) compensates for the double counting of black pixels that are common to P1 and P2 . Image functionals that share property (1) are called additive. Intuitively it may seem obvious to require image functionals to be additive. In general, one would like to avoid counting a feature in an image more than once. Additive image functionals play a central role in integral geometry, but this does not mean that these are the only useful image functionals. Indeed there are many image functionals that are not additive but yield valuable information on specific features of an image (Russ, 1995). As a prominent example, we mention the two-point correlation function of the positions of the black pixels (i.e., the Fourier transform of the structure factor). This is a nonadditive functional but it certainly yields very useful information about the spatial distribution of the black pixels. Another important property we would like to have is that the value of the image functional does not depend on the choice of the coordinate system. Formally, we say that an image functional ϕ is motion invariant if ϕ(s A) = ϕ(A) for s ∈ S . Here S denotes the group of all symmetry operations in the d-dimensional Euclidean space Rd . A key point of the theory reviewed in the eight subsections to come is that there are only d + 1 fundamentally different, motion invariant, additive image functionals that describe the morphological content of a d-dimensional array of pixels. These functionals are directly related to simple geometric concepts. The reader who is not interested in the mathematical aspects of integral geometry can skip the rest of this section and go directly to Section III. B. Minkowski Addition and Subtraction Consider two sets of points in Euclidean space (e.g., two images), specified by the position vectors A = {a1 , . . . , an } and B = {b1 , . . . , bm }. Translating every point of A by bi , for i = 1, . . . , m and collecting all points yields a new set of points C = A⊕B = B⊕ A=
&
i=1,...,n j=1,...,m
ai + b j ,
(2)
124
MICHIELSEN ET AL.
which is called the Minkowski sum of the sets A and B. The operation ⊕ is called Minkowski addition. Similarly Minkowski subtraction ⊖ is defined by ' ai + b j , (3) C = A⊖B = B⊖ A= i=1,...,n j=1,...,m
that is, we translate every point of A by every element of B and keep only those points that intersect. Minkowski addition increases the number of points. Minkowksi subtraction removes points from an image. Unless B contains the origin, image A ⊖ B is not necessarily a subset of image A. The operations ⊕ and ⊖ play a central role in morphological image processing (Giardina and Dougherty, 1988). In this context, the former is often called dilation and has the effect of “inflating” the image A. The latter is called erosion and results in a “shrinking” of A. (For a review of the properties of Minkowski algebra, see, e.g., (Hadwiger, 1957; Giardina and Dougherty, 1988).
C. Parallel Sets Consider the set of points of a line L of length a embedded in one-dimensional (1D) Euclidean space. We take a similar segment Sr(1) of length 2r and put the center of this line at each point of the line L. What does the union of all these points look like? Obviously, it is another line that is longer than L. The sets L (black line), Sr(1) (gray line), and L r (union of black and gray lines), the result of this operation, are shown in Figure 1. In terms of the operations discussed above, it is clear that L r = L ⊕ Sr(1) . The length l of L r is given by l(L r ) = a + 2r = l(L) + 2r.
(4)
The set L r is called the parallel set of L at a distance r . The 1D case readily extends to two and three dimensions. In Euclidean space, the parallel set Ar of A is defined by Ar = A ⊕ Sr(d) ,
(5)
Figure 1. Minkowski sum of a line segment L (black area) of length a and a line segment Sr(1) of length 2r yields the parallel set L r (black and gray area) at a distance r .
ASPECTS OF MATHEMATICAL MORPHOLOGY
125
Figure 2. Minkowski sum of a disk Sr(2) of radius r and a circular disk D of radius a, a square Q of edge length a, and an equilateral triangle T of side length a yields the parallel sets (union of the black and gray area) Dr , Q r , and Tr respectively.
where Sr(d) denotes a d-dimensional sphere of radius r . Let us consider some simple examples: a circular disc D of radius a, a square Q of edge length a, and an equilateral triangle T of side length a embedded in 2D Euclidean space. Take a disc of radius r and perform the same operation as in the 1D case. Put the center of the disc of radius r at each point of D (or Q or T ) and consider the union of all points. The resulting parallel sets Dr , Q r , and Tr are shown in Figure 2. The area U of Dr , Q r , and Tr is given by U (Dr ) = πa 2 + 2πar + πr 2 ,
U (Q r ) = a 2 + 4ar + πr 2 , √ 3 2 a + 3ar + πr 2 . U (Tr ) = 4
(6a) (6b) (6c)
The formulae (6) suggest that there may be a general relationship between the area of the original set and its parallel set at a distance r . It is not difficult to see that the areas of the three parallel sets can be written as U (K r ) = U (K ) + P(K )r + πr 2 ,
(7)
126
MICHIELSEN ET AL.
where P(K ) denotes the boundary length (or perimeter) of the geometric object K . As a last example, we consider a cube C of edge length a embedded in 3D space. A simple calculation shows that the volume V of the parallel set Cr can be written as V (Cr ) = a 3 + 6a 2r + 3aπr 2 +
4π 3 r . 3
(8)
Again, (8) suggests the generalization V (K r ) = V (K ) + S(K )r + 2π B(K )r 2 +
4π 3 r , 3
(9)
where S(K ) is the surface area and B(K ) is the mean breadth. The examples presented above suggest that for a sufficiently simple 3D (2D) geometric object, the change in the volume (area) can be computed from the original volume, area, and mean breadth (area and perimeter), as long as we inflate or deflate the object without changing its topology. In fact, these relations hold only for convex sets. Convex sets play an important role in integral geometry and are the key to the morphological characterization of sets of points in Euclidean space. Obviously, sets of pixels can be analyzed using these concepts too. However, to be useful in practice, there should be no constraints on the shape of the objects. The purpose of the next two subsections is to discuss the generalization of the above concept to objects of arbitrary shape.
D. Convex Sets and Minkowski Functionals A collection of points K in the d-dimensional Euclidean space Rd is called a convex set if for every pair of points in K , the entire line segment joining them also lies in K . A convex set with nonempty interior is called a convex body. A single point x ∈ Rd is also a convex set and convex body. We will consider only convex sets that are bounded and closed, i.e., that are compact. The class of all compact convex sets is denoted by K. The parallel set K r = K ⊕ Sr(d) of a compact convex set K ∈ K at a distance r is the union of all closed spheres of radius r , the centers of which are points of K (Santal´o, 1976). The operation of taking a parallel set preserves the properties of convexity and compactness, i.e., K r ∈ K (Stoyan et al., 1989).
ASPECTS OF MATHEMATICAL MORPHOLOGY
127
TABLE 1 Relation between the Minkowski Functionals Wν(d) (K ) of a Convex Set K and Conventional Geometric Quantities
W0(d) (K ) W1(d) (K ) W2(d) (K ) W3(d) (K )
d=1
d=2
d=3
l(K ) 2 — —
U (K ) P(K )/2 π —
V (K ) S(K )/3 2π B(K )/3 4π/3
The general expression for the volume v (d) of the parallel body K r at a distance r of a convex body K is given by the Steiner formula (Hadwiger, 1957) d $ % d (d) Wν(d) (K )r ν , (10) v (K r ) = ν ν=0
where the Wν(d) (K ) are called quermassintegrals or Minkowski functionals. Their relation to the familiar geometric quantities is given in Table 1. Clearly, (10) contains the results for the simple examples given in the previous subsection as special cases. It can be shown (Hadwiger, 1957) that the Minkowski functionals are r r
r
r
Motion invariant: ϕ is motion invariant if ϕ(s K ) = ϕ(K ) for s ∈ S . C-additive: A functional is C-additive if ϕ(K 1 ∪ K 2 ) = ϕ(K 1 ) + ϕ(K 2 ) − ϕ(K 1 ∩ K 2 ) for K 1 , K 2 ∈ K and K 1 ∪ K 2 ∈ K. The notion of C-additive (means additive on the set K) is not just a technical one because the union of two convex sets is not necessarily convex, although the intersection is. Continuity: ϕ is continuous if liml→∞ ϕ(K l ) = ϕ(K ) whenever {K l } is a sequence of compact sets such that liml→∞ K l = K in the Hausdorff metric (Giardina and Doughery, 1988). Intuitively, this continuity property of ϕ means that whenever the compact convex sets K l approach the compact convex set K , ϕ(K l ) also approaches ϕ(K ). This is a rather technical condition that is satisfied when we limit ourselves to sets of pixels. Monotonically increasing: ϕ is monotonically increasing ϕ(A) ≤ ϕ(B) if A ⊆ B. In simple terms, the value of the Minkowski functionals of a set A will not decrease if A becomes larger.
A fundamental result in integral geometry is the completeness of the family of Minkowski functionals. A theorem by Hadwiger (Hadwiger, 1957) states that every motion invariant, C-additive and continuous functional ϕ over K
128
MICHIELSEN ET AL.
can be written as ϕ(K ) =
d
aν Wν(d) (K ),
(11)
ν=0
with suitable coefficients aν ∈ R. A similar result holds if ϕ(K ) is C-additive and monotonically increasing. Then the numbers aν are nonnegative. In other words, the d + 1 Minkowski functionals form a complete system of morphological measures on the class of convex sets K (Hadwiger, 1957). MIA uses additive image functionals to characterize images. Of course we prefer to use motion invariant, additive image functionals. If we could replace “C-additive” by “additive,” then Hadwiger’s theorem would tell us that there are no more, no less than d + 1 different additive image functionals. This would imply that we would have to switch to nonadditive or coordinate-systemdependent image functionals to find additional nonmorphological structure in the image. However, in general, an image is not a convex set of points. The extension of Hadwiger’s theorem to additive instead of C-additive image functionals requires further consideration. On the positive side, in the end, it turns out that Hadwiger’s completeness theorem holds for arbitrary images as well.
E. Convex Rings and Additive Image Functionals The results of the previous subsection can be generalized to a much more general class of objects by considering the convex ring (Hadwiger, 1957) R, the class of all subsets A of Rd which can be expressed as finite unions of compact convex sets A=
l & i=1
K i ; K i ∈ K.
(12)
If A1 and A2 both belong to R, then so do A1 ∪ A2 and A1 ∩ A2 . As before, an additive functional ϕ has the property ϕ(A1 ∪ A2 ) = ϕ(A1 ) + ϕ(A2 ) − ϕ(A1 ∩ A2 ). Motion invariance of ϕ on R is defined as for ϕ on K. Obviously, any image is an instance of the convex ring R, the pixels being the convex sets and elements of K. Fundamental to the extension from K to R is the Euler characteristic or connectivity number χ defined as (Hadwiger, 1957) 1 K = ∅ , (13) χ (K ) = 0 K =∅
ASPECTS OF MATHEMATICAL MORPHOLOGY
129
for all K ∈ K. The Euler characteristic is an additive, motion-invariant functional on R (Hadwiger, 1957). For an element A of the convex ring R, the use of the property of additivity of χ yields l & χ(A) = χ Ki =
i
i=1
χ (K i ) −
i< j
χ(K i ∩ K j ) + · · · + (−1)l+1 χ (K 1 ∩ · · · ∩ K l ).
(14)
The value of χ(A) is independent of the representation of A as a finite union of compact convex sets (Hadwiger, 1957). Note that all sets appearing on the right-hand side of (14) are convex so that we can use (13) to compute the numerical (integer) value of χ(A). The Euler characteristic can be used to define the Minkowski functionals for all elements of the convex ring A ∈ R (Hadwiger, 1957). Recalling that a single point x ∈ Rd is a convex set, we can write the characteristic function of the (set A as I A (x) = χ(A ∩ x). Then the volume of A is given by W0(d) (A) = S I A (sx)ds. Here ds denotes the motion-invariant kinematic density (Hadwiger, 1957; Santal´o, 1976) and the integration is over all elements of S (Hadwiger, 1957; Santal´o, 1976). The expression of the volume suggests the following definition (Hadwiger, 1957) of the Minkowski functionals on R: ) χ(A ∩ s E ν )ds ν = 0, . . . , d − 1, Wν(d) (A) = S
Wd(d) (A)
ωd = π d/2 / Ŵ(1 + d/2),
= ωd χ(A)
(15)
where E ν is a ν-dimensional plane in Rd . The normalization is chosen such that for a d-dimensional sphere Sr(d) with radius r , Wν(d) (Sr(d) ) = ωd r d−ν , where ωd denotes the volume of the unit sphere (ω0 = 1, ω1 = 2, ω2 = π , ω3 = 4π/3) (Mecke, 1998b). The Minkowski functionals inherit from χ the property of additivity l & Wν(d) (A) = Wν(d) Wν(d) (K i ) − Wν(d) (K i ∩ K j ) + · · · Ki = i=1
i
+ (−1)
i< j
(l+1)
Wν(d) (K 1
∩ · · · ∩ K l ),
(16)
and motion invariance. It is not difficult to see that Hadwiger’s completeness theorem (11) carries over from the class of convex sets K to the ring R. Replacing K by
130
MICHIELSEN ET AL.
* A = li=1 K i where K i ∈ K and using (16) repeatedly, one finds that (11) holds for any element of R (Hadwiger, 1957). Thus, the d + 1 Minkowski functionals form a complete system of additive functionals on the set of objects that are unions of a finite number of convex sets (Hadwiger, 1957). For completeness and also because we make use of it later, we state one more important result in integral geometry, the so-called kinematic formulae (Hadwiger, 1957) )
Mμ(d) (A S
∩ s B)ds =
μ $ % μ ν=0
ν
(d) Mν(d) (A)Mμ−ν (B),
(17)
where Mμ(d) (A) =
ωd−μ (d) W (A); μ = 0, . . . , d, ωμ ωd μ
(18)
defines the normalized Minkowski functionals. The relation between the normalized Minkowski functionals Mν(d) (K ) of a convex set K and conventional geometric quantities are given in Table 2. The kinematic formulae (17) are very useful in stereology and stochastic geometry (Hadwiger, 1957; Santal´o, 1976; Matheron, 1975). They also play a key role in deriving configurational averages of Minkowski functionals. In translating these abstract mathematical concepts to a practical scheme, it is important to keep in mind the conditions under which the above theorems hold. Fortunately, in practice, this is easy to do. The crucial step is to decompose the image into a union of convex sets so that we can use the theoretical results that hold on the convex ring. We address this issue in Section II.G.
TABLE 2 Relation between the Normalized Minkowski Functionals Mν(d) (K ) of a Convex set K and Conventional Geometric Quantities in Euclidean Spacea
M0(d) (K ) M1(d) (K ) M2(d) (K ) M3(d) (K ) a
d=1
d=2
d=3
l(K ) χ (K )/2 — —
U (K ) P(K )/2π χ(K )/π —
V (k) S(k)/8 B(K )/π 3χ (K )/4π
Note: The Euler characteristic of a convex set is equal to one.
ASPECTS OF MATHEMATICAL MORPHOLOGY
131
Figure 3. Two- and 3D figures with various connectivity numbers (Euler characteristics) χ.
F. Relation to Topology and Differential Geometry The Euler characteristic χ is identical to the one defined in algebraic topology (Hadwiger, 1957). For d = 2, χ(A) equals the number of connected components minus the number of holes. In three dimensions, χ(A) is given by the number of connected components minus the number of tunnels plus the number of cavities. Some examples are shown in Figure 3. The Euler characteristic describes A in a purely topological way, that is, without reference to any kind of metric. Very often, one is interested in the topology of the surface ∂ A of A (Hyde et al., 1997; Mellot, 1990). The Euler characteristic of ∂ A is directly related to that of A, namely χ (∂ A) = χ(A)[1 − (−1)n ],
(19)
where n is the dimension of the body A (n ≤ d) (Likos et al., 1995). The principal curvatures of a surface are useful quantities for the numerical characterization of the surface of a 3D body. They are defined as follows. Consider a point on the surface and the vector through this point, normal to the surface. A plane containing this normal vector intersects the surface. This intersection is a planar curve with a curvature called the normal curvature. Rotation of the plane about the normal produces various planar curves with different values of normal curvature. The extreme values of the normal curvatures are called the principal curvatures κ1 and κ2 of a surface. These two curvatures can be combined to give two useful measures of the curvature of
132
MICHIELSEN ET AL.
a surface, namely the Gaussian and mean curvature defined as G = κ1 κ2 and H = (κ1 + κ2 )/2, respectively. The integral mean curvature H and integral Gaussian curvature G are given by % ) $ 1 1 1 d f, (20) + H (A) = 2 ∂ A R1 R2 and G(A) =
)
∂A
1 d f, R1 R2
(21)
respectively. Here, R1 = 1/κ1 and R2 = 1/κ2 are the principal radii of curvature of A and d f is the area element on A. For H and G to be well defined, the boundary ∂ A should be regular. The mean breadth is proportional to the integral mean curvature: H (A) = 2π B(A).
(22)
The Euler characteristic of ∂ A is closely related to the integral Gaussian curvature G and the genus g (number of handles, i.e., number of holes in the closed surface): G(A) = 2π χ (∂ A),
χ(∂ A) = 2(1 − g).
(23)
Note that integral geometry imposes no regularity conditions on the boundary ∂ A of the objects: H (A) and χ (A) are always well defined.
G. Application to Images Each pixel in a 2D (3D) black-and-white image is a convex set. Therefore, such images may be considered as an element of the convex ring R, and we can invoke integral geometry to build additive image functionals to measure features in the image. However, as mentioned before, some care has to be taken because the Minkowski functionals take known values on convex sets only. The key to the practical application of integral geometry to images is the additivity of χ (see (14)): We can compute the Minkowski functionals of an image A by decomposing A into convex sets K i . However, if we take for {K i } all black pixels (assuming the background consists of white pixels), then we would have to compute all the intersections that appear in (14). Although this can be done, it is much more expedient to take a slightly different route. First we write each pixel K as the union of the disjoint collection of its interior body, interior faces (in 3D only), open edges and vertices (Likos et al., 1995). We will denote the interior of a set A by A˘ = A \ ∂ A. The values of the Minkowski functionals of the open interior of a n-dimensional body A ∈ R
133
ASPECTS OF MATHEMATICAL MORPHOLOGY TABLE 3 Minkowski Functionals Wν(2) (ν = 0, . . . , d = 2) for the Open Sets N˘ m , the Basic Building Blocks of a Square in 2D Euclidean Space a m
N˘ m
W0(2)
W1(2)
W2(2)
0 1 2
P˘ L˘ Q˘
0 0 a2
0 a −2a
π −π π
a ˘ ˘ open edge of Q: open square of edge length a; L: ˘ vertex. length a; P:
embedded in Rd (n ≤ d) are given by (Likos et al., 1995) ˘ = (−1)d+n+ν Wν(d) (A); ν = 0, . . . , d. Wν(d) ( A)
(24)
By making use of the additivity of the Minkowski functionals (see (16)) and the fact that there is no overlap between open sets on a lattice, the values of the Minkowski functionals on the whole pattern P may be obtained from Wν(d) (P ) = Wν(d) ( N˘ m )n m (P ); ν = 0, . . . , d, (25) m
where n m (P ) denotes the number of open sets N˘ m of type m present in P . On a square and cubic lattice there are d + 1 open sets N˘ m : N˘ 0 corresponds to a vertex, N˘ 1 to an open line segment, N˘ 2 to an open square on both the 2D square and the 3D cubic lattice, and N˘ 3 to an open cube on the 3D cubic lattice. The calculation of the Minkowski functionals for the building blocks N˘ m of a 2D square and a 3D cubic lattice is given elsewhere (Michielsen and De Raedt, 2001). For convenience, their values are reproduced in Tables 3 and 4, respectively. The procedure to calculate n m (P ) for a square and cubic lattice is TABLE 4 Minkowski Functionals Wν(3) (ν = 0, . . . , d = 3) for the Open Sets N˘ m , the Basic Building Blocks of a Cube in 3D Euclidean Space a m
N˘ m
W0(3)
W1(3)
W2(3)
W3(3)
0 1 2 3
P˘ L˘ Q˘ C˘
0 0 0 a3
0 0 2a 2 /3 −2a 2
0 πa/3 −2πa/3 πa
4π/3 −4π/3 4π/3 −4π/3
a ˘ ˘ open square of edge length a; L: ˘ open C: open cube of edge length a; Q: ˘ vertex. edge of length a; P:
134
MICHIELSEN ET AL.
Figure 4. Decomposition of the pixels of a 2D black-and-white pattern (left) into squares, edges, and vertices (right).
given in Appendix A. Note that the value of the Minkowski functionals depends on the type of lattice, i.e., the shape of the pixels the pattern is represented on. An example of a decomposition of a pattern on a hexagonal lattice is given in Sofonea and Mecke (1999). In Figure 4 we show a simple example of the decomposition of the pixels of a 2D black-and-white pattern (left) into squares, and edges and vertices (right). For this example, the number of squares n 2 = 8, number of edges n 1 = 24 and number of vertices n 0 = 16. Hence W0(2) = U = 8a 2 , W1(2) = P/2 = −16a + 24a = 8a, and W2(2) = π χ = 8π − 24π + 16π = 0, where a denotes the edge length of the open square and open edge. We further illustrate the procedure to compute the Minkowski functionals by considering the 2D checkerboard pattern with an even number L 0 of cells, of edge length one, in each direction. We consider free and periodic boundary conditions (see Figure 5).
Figure 5. 4 × 4 checkerboard pattern. The black line denotes the boundary. Left: Free boundaries. Right: Periodic boundaries.
ASPECTS OF MATHEMATICAL MORPHOLOGY
135
The left image in Figure 5 shows the 4 × 4 checkerboard lattice with free boundary conditions, that is, the pattern is completely surrounded by white pixels. The right image shows the same pattern but with periodic boundary conditions. For the L 0 × L 0 checkerboard P F with free boundary conditions, we find n 0 (P F ) = (L 0 + 1)2 − 2, n 1 (P F ) = 2L 20 , n 2 (P F ) = L 20 /2 and hence U (P F ) = L 20 /2, P(P F ) = 2L 20 and χ(P F ) = L 20 /2 − (L 0 − 1)2 . Note that this value of χ corresponds to the value we find if we calculate χ as the number of connected components minus the number of holes, since the number of connected components (black structure) equals one and the number of holes equals (L 0 /2 − 1)(L 0 − 2). For the L 0 × L 0 checkerboard P P with periodic boundary conditions, we find n 0 (P P ) = L 20 , n 1 (P P ) = 2L 20 , n 2 (P P ) = L 20 /2 which yields U (P P ) = L 20 /2, P(P P ) = 2L 20 and χ (P P ) = −L 20 /2. Note that χ (P P )/L 20 = lim L 0 →∞ χ(P F )/L 20 = −1/2. Summarizing: In practice the calculation of the Minkowski functionals of a black-and-white image amounts to counting vertices and edges, among others.
H. Integral Geometry on a Hypercubic Lattice Most modern image acquisition systems are pixel based. In many cases the pixels represent an average of the signal (light, electrons, etc.) over some area or volume of the sample. In other cases, for instance in scanning electron microscope images taken at low magnification, the dimension of the sampled volume is smaller than the dimension of the pixel in the image (Russ, 1995). Then the pixels represent points that are discrete and well separated and it is no longer evident that the set of pixels corresponds to an image of an object in Euclidean space. In this case, it is more appropriate and correct to regard the image as a collection of points on a square or cubic lattice instead of in continuum space. Not surprisingly, the concepts of integral geometry can be used to characterize structure in these situations as well. In fact, the theory reviewed above directly applies if we replace the group S of all symmetry operations in Rd by the group S ′ of translations, rotations and reflections that leave the finite d-dimensional, regular (hypercubic) lattice Zd invariant (Klain, 1997, 1998; Voss, 1992; Mecke, 2000). In the following we closely follow Mecke (2000). In analogy with Eq. (15), on the d-dimensional lattice the Minkowski functionals are defined by Vd(d) (A) = χ(A), $ %−1 d (d) Vν (A) = (2d d!)−1 χ (A ∩ s E ν )ds; ν ′ s∈S
ν = 0, . . . , d − 1, (26)
136
MICHIELSEN ET AL.
Figure 6. Basic elements of integral geometry in Z2 : An elementary square (a), two connected elementary squares (b), two 0-dimensional planes (lattice points) E 0 and two 1D planes (horizontal and vertical lines through the lattice points) E 1 .
where E ν is a ν-dimensional plane in Zd . The sum in (26) denotes the sum over all translations, rotations, and reflections in Zd . The factor 2d d! reflects the number of symmetry operations that leave the d-cubic lattice invariant. Figure 6 shows the basic elements of integral geometry in Z2 : an elementary square (a), two connected elementary squares (b), two 0-dimensional planes (lattice points) E 0 , and two 1D planes (horizontal and vertical lines through the lattice points) E 1 . As in the Euclidean case, the Minkowski functionals (26) also have an intuitively clear meaning. In Table 5 we summarize the relationships between the Minkowski functionals on Zd and the conventional morphological quantities.
TABLE 5 Relation between the Minkowski Functionals Vν(d) (A) on a d-Dimensional Hypercubic Lattice Zd and the Length l, Area U , Perimeter P, Volume V , Surface Area S, Integral Mean Breadth B and Euler Characteristic χ
V0(d) V1(d) V2(d) V3(d)
d=1
d=2
d=3
l χ — —
U P/4 χ —
V S/6 2B/3 χ
ASPECTS OF MATHEMATICAL MORPHOLOGY
137
Unlike in the Euclidean case, the Minkowski functionals (26) are no longer homogeneous functions of the length scale (which is fixed to one on Zd ). As a consequence, kinematic formulae (17) no longer hold but have to be replaced by (Mecke, 2000) μ ν $ %$ % 1 (d) κ μ (d) Vν(d) (A)Vμ−κ V (A ∩ s B)ds = (B). ν ν 2d d! s∈S ′ μ ν=0 κ=0
(27)
In Section IV.C, we make use of Eq. (27) to compute analytically the configurational averages of the Minkowski functionals for random distributions of points in Zd .
I. Integral Geometry on a Lattice: Alternative Formulation On the 2D square lattice Z2 , it is a simple matter to identify the fundamental geometric elements. Indeed, any image can be decomposed in vertices (points), line segments (edges) connecting the vertices, and the squares enclosed by vertices and edges. In Euclidean space, these fundamental elements correspond to the open sets used in Section II.G. Obviously, this construction can be made for any dimension d and any shape of the pixels. In terms of these fundamental elements, the basic equations of integral geometry take a particularly simple form. The basic reason is that the intersection of a set of vertices, for example, and a set of edges, for example, is empty. We now proceed with this alternative formulation of integral geometry and obtain the (simple) formulae that will actually be used in MIA. We consider the d-cubic lattice Zd . This lattice can be viewed as the union of the disjoint sets of vertices O0 , edges O1 , faces O2 , cubes O3 , etc. Obviously, in Euclidean space these are the basic open elements (see Section II.G). Any pattern A on Zd can be decomposed into the elements Oν , ν = 0, . . . , d. This decomposition is trivially unique. An example for d = 2 is given in Figure 7. Let the functional Fν (A) count the number of elements Oν in A. It is easy to see that the Fν are functionals on Zd that take positive integer values and are r r
r
additive: Fν (A ∪ B) = Fν (A) + Fν (B) − Fν (A ∩ B) and Fν (∅) = 0 motion invariant: Fν (s A) = Fν (A) where s denotes all translations, rotations and reflections on Zd monotonically increasing: Fν (A) ≤ Fν (B) for A ⊆ B.
Since there is no other type of fundamental elements in Zd than the Oν s (i.e., there is nothing else to be counted), any additive and motion-invariant
138
MICHIELSEN ET AL.
Figure 7. A 2D lattice Z2 decomposed into disjoint sets of vertices O0 , edges O1 , and faces O2 (black objects). The simplexes S0 , S1 and S2 and 2-cubes E 2 (1) and E 2 (3) are shown in gray.
functional ϕ on Zd must be a linear combination of the form
ϕ(A) =
d
aν Fν (A),
(28)
ν=0
where aν = ϕ(Oν ). In this formulation of integral geometry on a lattice, Hadwiger’s representation theorem is almost self-evident. The Fν play the role (but are not the same as) the Minkowski functionals. We proceed to write the conventional functionals describing the morphology of subsets A ⊂ Zd as a linear combination of the functionals Fν (A). It is expedient to introduce the notion of a simplex Sν , ν = 0, . . . , d. A vertex (S0 ), an edge with a vertex at each end (S1 ), a square surrounded by four edges and four vertices (S2 ), etc., are examples of simplexes (see Fig. 7). Clearly, in Euclidean space the equivalent of a simplex is a closed convex set. Every pattern in Zd has a unique decomposition in terms of simplexes Sν , ν = 0, . . . , d. In the case of a d-dimensional digital image, each pixel is a simplex Sd . Hence the decomposition only contains simplexes Sd . The topology of a subset A is characterized by the Euler characteristic. In analogy with the definition of the Euler characteristic χ of a convex body, we define the additive and motion-invariant functional χ by χ (∅) = 0 and χ (Sν ) = 1 for all ν = 0, . . . , d. By construction, since every pattern A has a unique decomposition in terms of the simplexes Sν , χ (A) is uniquely determined by χ (Sν ), hence χ (A) is unique.
ASPECTS OF MATHEMATICAL MORPHOLOGY
139
Let E d (L) denote a d-cube of linear size L (see Fig. 7 for d = 2). Simple counting shows that the number of Oν in E d (L) is given by $ % d L ν (L + 1)d−ν . Fν (E d (L)) = (29) ν We now derive the classical alternating formula for the Euler characteristic (Klain and Rota, 1997). Let us define an additive functional by setting (∅) = 0 and (Oν ) = (−1)ν . Then, since E d (L) is a disjoint union of the Oν s, we can use the additivity of to write (E d (L)) = = =
d
(Oν )Fν (E d (L)),
(30a)
(−1)ν Fν (E d (L)),
(30b)
$ % d L ν (L + 1)d−ν = 1. (−1) ν
(30c)
ν=0
d ν=0
d ν=0
ν
Note that (30) also holds for simplexes since Sν = E ν (1). Therefore, (Sν ) = χ (Sν ) for all ν = 0, . . . , d and hence (A) = χ (A) for any A. Invoking the additivity of Fν , Eq. (30a) implies that χ(A) =
d ν=0
χ(Oν )Fν (A) =
d (−1)ν Fν (A) ν=0
= F0 (A) − F1 (A) + F2 (A) − · · · + (−1)d Fd (A),
(31)
which is the well-known discrete Euler formula (Klain and Rota, 1997). The logic used to derive this formula can also be used to derive the relationships between the functionals Fν on Zd and the conventional geometric quantities. We illustrate this for the case of d = 2. First of all, it is trivial that the area U (E 2 (L)) = F2 (E 2 (L)) and hence U (A) = F2 (A). We have already found the expression for the Euler characteristic, namely χ(A) = F2 (A) − F1 (A) + F0 (A). Let us write for the perimeter P(A) = a2 F2 (A) + a1 F1 (A) + a0 F0 (A). Since we know that P(A = E 2 (1)) = 4, P(A = E 2 (2)) = 8, and P(A = E 2 (3)) = 12, we can set up a linear set of equations for the unknowns a0 , a1 , and a2 and solve it. We find that P(A) = −4F2 (A) + 2F1 (A). Repeating this procedure for all other quantities of interest we find for d = 1 l(A) = F1 (A),
(32)
140
MICHIELSEN ET AL.
for d = 2 U (A) = F2 (A),
P(A) = −4F2 (A) + 2F1 (A),
(33)
and finally for d = 3 V (A) = F3 (A), S(A) = −6F3 (A) + 2F2 (A), 2B(A) = 3F3 (A) − 2F2 (A) + F1 (A).
(34)
Expressions (32)–(34), together with Eq. (31), will be used as the starting point for the algorithms to compute the morphological descriptors (see Section III). Not surprisingly, these expressions are identical to those obtained by starting from the Euclidian version of integral geometry, followed by the procedure described in Section II.G. For completeness, we also derive the discrete analogue of the kinematic formulae in terms of the functionals Fμ . As might be anticipated from the discussion above, they should take a very simple form, and indeed they do. In general, for fixed B, s∈S ′ Fμ (A ∩ s B) is a motion invariant, addititive functional of A. Likewise for fixed A, s∈S ′ Fμ (A ∩ s B) = s∈S ′ Fμ (s A ∩ B) is a motion invariant, addititive functional of B. Hence we have d 1 F (A ∩ s B) = cκν Fκ (A)Fν (B); μ = 0, . . . , d. (35) μ 2d 2! s∈S ′ κ,ν=0 Since s∈S ′ Fμ (A ∩ s B) = s∈S ′ Fμ (s A ∩ B), we must have cκν = cνκ . The remaining constants cκν can be computed by looking at specific examples for A and B. An example of such a calculation is given in Appendix C. In general, we find for the kinematic formulae $ %−1 1 d Fμ (A ∩ s B) = Fμ (A)Fμ (B); μ = 0, . . . , d. (36) 2 ν 2 d! s∈S ′
The kinematic formulae Eq. (17) and Eq. (27) have a more complicated structure than Eq. (36) but this is hardly a surprise: The left-hand side of Eq. (36) only counts the number of elements of the type μ in A ∩ s B and therefore the right-hand side cannot depend on the number of elements in A or B that are of a type different from μ. The Minkowski functionals that enter the kinematic formulae Eq. (17) and Eq. (27) do not have this property. The kinematic formulae for the conventional geometric quantities ϕ follow from d 1 1 ϕ(A ∩ s B) = 2 ϕ(Oμ )Fμ (A ∩ s B). 2d d! s∈S ′ 2 d! s∈S ′ μ=0
(37)
141
ASPECTS OF MATHEMATICAL MORPHOLOGY TABLE 6 Values for the Geometric Quantities ϕ(Oν ), ν = 0, . . . d, Where ϕ Stands for l, U , P, V , S, B or χ
O0 O1 O2 O3
l
U
P
V
S
B
χ
0 1 — —
0 0 1 —
0 2 −4 —
0 0 0 1
0 0 2 −6
0 1/2 −1 3/2
1 −1 1 −1
The values for ϕ(Oμ ) can be derived from Eqs. (28) and Eqs. (31)–(34) and are summarized in Table 6. For instance, the kinematic formula for the Euler characteristic reads d 1 1 χ(Oμ )Fμ (A ∩ s B) χ(A ∩ s B) = 2d d! s∈S ′ 22 d! s∈S ′ μ=0 =
d μ=0
$ %−1 d Fμ (A)Fμ (B); (−1) μ μ
μ = 0, . . . , d, (38)
which is similar to the expression of Theorem 3.2.5 in (Klain and Rota, 1997) and has a much more simple structure than Eq. (27). Summarizing: Given a set of pixels representing a black-and-white image, there are two ways to perform an integral-geometry analysis of the data. If we consider the pixels as the only information we have, then we use integral geometry on a lattice. If we know or have good arguments to assume that continuum space is a better model, we adopt the Euclidean formulation. It will be clear from our discussion below of the practical realization of integralgeometry-based image analysis, that one can switch from one to the other formulation without difficulty.
III. Integral Geometry in Practice An appealing feature of the integral-geometry approach is that the complexity of the mathematical framework is in no way comparable to the simplicity of its practical application. In this section, we focus on the practical aspects. Therefore, we will omit most mathematical justification, references to relevant work, and discussions of examples. As before, we consider only black-andwhite images.
142
MICHIELSEN ET AL.
From the theory reviewed above, it follows that the d + 1 Minkowski functionals form a complete system of additive image functionals on the set of objects that are unions of a finite number of convex sets (Hadwiger, 1957). In ordinary language this implies that if we restict ourselves to using additive image functionals we only have to compute the d + 1 Minkowski functionals to characterize the morphology of the pattern. In Section II.G, or equivalently, Section II.I, we described the basic ideas behind a simple and efficient procedure to compute the Minkowski functionals. In the next two subsections we translate these ideas into a computer program. In Subsections III.C and III.D, we focus on the second step of MIA: The study of the dependence of these functionals on some control parameters. In the remaining subsections, we discuss several technical topics that are relevant to practical applications.
A. Minkowski Functionals Consider a 2D lattice filled with black pixels on a white background (see Fig. 4). For simplicity, we will assume that the pixels are squares and that the linear size of each square has been normalized to one. We want to characterize the geometry and topology of the pattern formed by the black pixels. According to Hadwiger’s completeness theorem, there are three additive image functionals, called Minkowski functionals, that describe the morphological content of this 2D pattern, namely the area U , the perimeter P, and the Euler characteristic χ . The latter describes the connectivity (topology) of the pattern: In 2D χ equals the number of regions of connected black pixels minus the number of completely enclosed regions of white pixels. Two black pixels are “connected” if and only if they are nearest neighbors or next-nearest neighbors of each other or can be connected by a chain of black pixels that are nearest and/or nextnearest neighbors. Using this definition we find that the Euler characteristic of the pattern shown in Fig. 4 is zero. Conceptually the procedure (that easily extends to three dimensions) to compute these three numbers consists of two steps. First, we decompose each black pixel into 4 vertices, 4 edges, and the interior of the pixel (see Fig. 4). Then we count the total number of squares n 2 , edges n 1 , and vertices n 0 . By definition we have Fν = n ν for ν = 0, . . . , d (see Table 3). A simple algorithm and computer program to count these numbers are described in Appendix A and B, respectively. Using Eq. (25) and Table 4, or directly from (33), it follows that the area U , perimeter P, and Euler characteristic χ are given by U = n 2 , P = −4n 2 + 2n 1 , χ = n 2 − n 1 + n 0 .
(39)
For the example shown in Fig. 4, the number of squares n 2 = 8, number of
143
ASPECTS OF MATHEMATICAL MORPHOLOGY
edges n 1 = 24, and number of vertices n 0 = 16 and we find U = 8, P = 16, and χ = 0. For a 3D cubic lattice filled with black and white pixels, the four Minkowski functionals are the volume V , the surface area S, the mean breadth B (see Section II.C), and the Euler characteristic χ . In 3D χ equals the number of regions of connected black pixels plus the number of completely enclosed regions of white pixels minus the number of tunnels, i.e., regions of white pixels piercing regions of connected black pixels. As in the 2D case, the first step in the calculation of these four numbers is to consider each black pixel as the union of 8 vertices, 12 edges, 6 faces, and the interior of the cube. From Eq. (25) and Table 4, or from (34), it follows that V = n3,
S = −6n 3 + 2n 2 ,
2B = 3n 3 − 2n 2 + n 1 , (40)
χ = −n 3 + n 2 − n 1 + n 0 ,
where n 3 is the number of cubes. Thus, as in the 2D case, the morphological characterization of a 3D pattern reduces to the counting of the elementary geometric objects (vertices, edges, faces, cubes) that constitute the pattern. For reference, in Table 7, we collect some basic results for the normalized Minkowski functionals of simple shapes in 3D Euclidean space. B. Computer Program Technically the only real “problem” with the procedure described above is to avoid counting, e.g., a face, an edge or vertex more than once. However, this problem is easily solved, as illustrated by the algorithm we will briefly discuss now. In Appendix B, we list a computer program to compute V , S, B, and χ for a 3D black-and-white pattern. For an example of a program for 2D images, see Michielsen and De Raedt (2001). For some applications, notably those TABLE 7 Normalized Minkowski Functionals Mν(3) (K ) for the Elementary Convex Sets of Linear Size a in 3D Euclidean Space
M0(3) /a 3 8M1(3) /a 2 π M2(3) /a 4π M3(3) /3
V /a 3 S/a 2 B/a χ
Vertex
Line
Square
Cube
0 0 0 1
0 0 1/2 1
0 2 1 1
1 6 3/2 1
144
MICHIELSEN ET AL.
where the patterns are the result of computer simulation, it is useful to adopt periodic boundary conditions. Therefore, we present computer code for the case of periodic boundary conditions too. Alternatively, one can embed the original image into a larger one, formed by surrounding the original image by one extra layer of pixels. The value of these pixels is determined by making use of the periodic boundary conditions. Then one can use the code that does not explicitly use the boundary conditions. Conceptually, what this program does is to build up the whole image using vertices, edges, etc. In practice, this is accomplished by adding active (= black in the example above) pixels to an initially empty (= white in the example above) image (held in array tmp(.)) one by one. Just before adding the active pixel to the current image (in tmp(.)) subroutine “minko 3D free” determines the change in V , S, B, and χ that would result if this pixel were actually added to the current image. This change is calculated by first decomposing this cubic pixel as discussed above and then checking whether, e.g., a face overlaps with a face of another active pixel in the current image. Then the pixel is made active in the current image and the changes are added to the current values of V , S, B, and χ. Inspection of “minko 3D free” shows that all it does is check to see if the pixel-to-be-added has active nearest neighbors and/or next-nearest neighbors and count the number of faces, edges, and vertices accordingly. Clearly, the number of arithmetic operations required to compute V , S, B, and χ scales linearly with the number of pixels of the image. Thus, the numerical procedure is efficient.
C. Analysis of Point Patterns Many systems observed in nature may be modelled by point patterns. For example, a system of particles may be viewed as a collection of points defined by the position of the particles. These points are usually called the germs of the model (Stoyan et al., 1989; Matheron, 1975). In order to study the morphological properties of the set of points (degree of randomness, clustering, periodic ordering, etc.), it is useful to attach to the points discs (spheres) of radius r . Those discs (spheres) are called the grains of the model (Stoyan et al., 1989; Matheron, 1975). The study of the coverage of the image by the grains gives information about the distribution of the germs. Mapping the point pattern onto a square (cubic) lattice yields a black-andwhite image. Black pixels represent the germs of the model. On the pixel lattice we can construct the grains of the model in two different ways. In the first method, we consider the germs to be discs (spheres) of radius r = 0. We enlarge the discs (spheres) by making black all pixels that are positioned at a distance smaller or equal to r > 0 from the germs. The grains form discrete
ASPECTS OF MATHEMATICAL MORPHOLOGY
145
Figure 8. Graining procedure of a point pattern in two dimensions. Left: The grains are discrete approximations to a sphere with radius three in the Euclidean space. Right: The grains are squares of edge length seven. The light gray pixels indicate the positions of the germs.
approximations to discs (spheres) in the Euclidean space. An example of this graining procedure in two dimensions is shown on the left-hand side of Figure 8 for grains of radius r = 3. The right-hand side of Figure 8 illustrates the second graining procedure (for d = 2), where we take the germs to be squares (cubes) of edge length r = 1 and the grains to be enlarged squares (cubes) of edge length r = 2n + 1, n > 0. Note that the growing of the cubic grains leads to a faster complete coverage of the image than the growing of the circular grains. For this category of problems, MIA consists of the calculation of the three (four) numbers U , P, and χ (V , S, B and χ ) as a function of the grain size r . A schematic representation of this procedure for the case of 2D point patterns is shown in Figure 9.
D. Analysis of Digitized and Thresholded Images In general, the intensity (or gray level) in experimental images may be thought of as a continuous function of the position in the image. In order to analyze such images by computer, we first have to digitize them (Russ, 1995; Castleman, 1996; Gonzalez and Woods, 1993; Rosenfeld and Kak, 1982). The digitization process requires the mapping of the image on a grid and a quantization of the gray level. Usually 2D (3D) images are partitioned into square (cubic) regions. Each square (cube) is centered at a lattice point, corresponding to a pixel. In general, the range of gray levels is divided into intervals and the gray level at any lattice point is required to take only one of these values.
146
MICHIELSEN ET AL.
χ
Figure 9. Schematic representation of how to use MIA to analyze the morphological properties of (patterns that can be interpreted in terms of) 2D point patterns.
The output of image analysis should be a description of the given image. Thus we have to define the various objects building up the image, that is, we need a method to distinguish objects from the background (Russ, 1995; Rosenfeld and Kak, 1982). The simplest method of reducing gray-scale images to two-valued images or black-and-white images is to make use of a threshold. If the given image P (x) with x ∈ Rd has gray level range [a, b], and q is any number between a and b, the result of thresholding P (x, q) at q is the twovalued image P (x, q) defined by (Russ, 1995; Castleman, 1996; Gonzalez and Woods, 1993; Rosenfeld and Kak, 1982) 1 P (x) ≥ q . (41) P (x, q) = 0 P (x) < q By definition, if P (x, q) = 0, x is part of the background, and if P (x, q) = 1, x is part of an object. In practice, not all thresholds q yield useful P (x, q). If q is too large, too many objects are classified as background or if q is too small, the opposite happens. Other thresholding operations may also be considered
ASPECTS OF MATHEMATICAL MORPHOLOGY
147
Figure 10. Schematic representation of how to use MIA to analyze the morphological properties of 2D digitized images.
(Russ, 1995; Castleman, 1996; Gonzalez and Woods, 1993; Rosenfeld and Kak, 1982). For this type of image, MIA consists of the calculation of the three (four) numbers U , P, and χ (V , S, B and χ) as a function of the threshold q. A schematic representation of this procedure for the case of 2D gray-scale images is shown in Figure 10.
E. What Integral Geometry Cannot Do Although Hadwiger’s theorem on the completeness of the d + 1 Minkowski functionals is rather powerful, the term “completeness” should not be taken too
148
MICHIELSEN ET AL.
literally. An almost trivial example may serve to illustrate this point. Consider a triangle in 2D Euclidean space. Can we use the Minkowski functionals to “completely” characterize this triangle? The answer is no. It is not difficult to show that the length of the edges of the triangle are given by
P U 1 + cos θ a= + ± 4 P sin θ U 1 + cos θ P + ∓ b= 4 P sin θ c=
+$ +$
P U 1 + cos θ + 4 P sin θ
%2
−
2U , sin θ
(42a)
U 1 + cos θ P + 4 P sin θ
%2
−
2U , sin θ
(42b)
2U 1 + cos θ P − , 2 P sin θ
(42c)
where θ denotes the angle between the sides of length a and b and U and P are the area and perimeter, respectively. From (42), it is clear that the Minkowski functionals U , P, and χ = 1 do not specify the triangle completely. Different choices of the angle θ results in different triangles with the same area U and perimeter P. For MIA to yield useful information about the morphological content of an image, it is necessary to study the dependence of the Minkowski functionals on one or more control parameters (e.g., threshold, grain size, etc.).
F. Reducing Digitization Errors Often the lattice of pixels represents an image of objects in Euclidean space. The Minkowski functionals computed using this lattice approximate the Minkowski functionals in Euclidean space. By digitizing the 2D (3D) image, we have introduced square (cubic) distortions in the objects, causing a directional bias. For example, digitizing a 2D (3D) image transforms a smooth contour (surface) to a more stepwise contour (surface). The more complicated the image, the better the digital approximations are likely to become since the parts of the stepwise boundary or surface will exhibit each orientation more often. The most problematic structures may be isotropic ones. There are several methods to correct for systematic errors caused by digitization of the image (Russ, 1995; Serra, 1982). Many of them can be used to improve the accuracy of the approximations to the area, perimeter, etc. We will not treat problems of digitization here because it is not of fundamental importance for the application of integral-geometry concepts.
ASPECTS OF MATHEMATICAL MORPHOLOGY
149
G. Normalization of Image Functionals For presentation purposes it is convenient to introduce the following normalized quantities:
and
, = U/L 2 , U
, V = V /L 3 ,
, = P/L N 1/2 , P
, S = S/L 2 N 1/3 ,
, χ = χ/N ;
, = 2B/L N 2/3 , B
d = 2,
, χ = χ /N ;
(43a)
d = 3, (43b)
where L denotes the linear size of the square (cube) and N denotes the number of germs.
IV. Illustrative Examples In this section, we apply MIA to point sets and complicated surfaces. These examples serve to illustrate the two basic modes of analysis discussed in Sections III.C and III.D. We first apply MIA to simple cubic, face-centered cubic (FCC) and body-centered cubic (BCC) lattice structures with and without imperfections. Then we compute the mean value of the Minkowski functionals of random point sets, i.e., the average over all configurations, grain sizes, and shapes. Finally we compute the Euler characteristic of a selection of minimal surfaces.
A. Regular Lattices The FCC and BCC lattices are of great importance, since an enormous variety of solids and several complex fluids (Gast and Russel, 1998) crystallize in these forms. The simple cubic (SC) lattice, however, is relatively rare but is often used in theoretical models. The SC lattice may be generated from the following set of primitive vectors a1 = L 0 [100],
a2 = L 0 [010],
a3 = L 0 [001],
(44)
where L 0 denotes the lattice constant. A symmetric set of primitive vectors for the FCC cubic lattice is a1 =
L0 [011], 2
a2 =
L0 [101], 2
a3 =
L0 [110], 2
(45)
150
MICHIELSEN ET AL.
and for the BCC cubic lattice is a1 =
L0 ¯ [111], 2
a2 =
L0 ¯ [111], 2
a3 =
L0 ¯ [111]. 2
(46)
To compute the Minkowski functionals for the SC, FCC, and BCC lattices we imbed these lattices in a cubic lattice with lattice constant one, and make use of (44)–(46) to determine the position of the black pixels. Then we follow the procedure described in Section III.C to transform the resulting point pattern into a pattern of “spherical” grains of radius r and to study the behavior of the Minkowski functionals as a function of r . An example of the graining procedure is shown in Figure 11 for the SC lattice with periodic boundary conditions and L 0 = 4. The thick solid line indicates the dimensions of the conventional unit cell, simply called the unit cell from now on.
Figure 11. Graining procedure for the SC lattice with periodic boundary conditions and L 0 = 4. The thick solid line indicates the dimensions of the unit cell.
ASPECTS OF MATHEMATICAL MORPHOLOGY
151
Figure 12. Minkowski functionals as a function of r for the perfect SC (dotted curve), FCC (solid curve), and BCC (dashed curve) lattice with M = 1 and L 0 = 32 with periodic boundary conditions.
,, , , and , Figure 12 shows the Minkowski functionals V S, B, χ as a function of r for the SC (dotted curve), FCC (solid curve), and BCC (dashed curve) lattice without imperfections. The SC, FCC, and BCC lattices with periodic boundaries consist of one unit cell of linear dimension L 0 = 32. Because of the normalization (43), the curves for more than one unit cell will be the same as the ones shown in Figure 12. Figure 12 clearly shows that the behavior of the Minkowski functionals as a function of r differs for √ the various √ lattice types. The area , S reaches a maximum if r equals L 0 /2, L 0 2/4, L 0 3/4, for the SC, FCC, and BCC lattice, respectively. At this value for r , the Euler characteristic , χ starts to deviate from one because the “spheres” touch each other. For the SC lattice, , χ jumps to -2 independent of L 0 (result not shown). In the case of the FCC (BCC) lattice and for sufficiently large L 0 (L 0 ≥ 16), the Euler characteristic jumps to a large negative (positive) value. For the SC, FCC, and BCC lattice with r = 0, the Euler characteristic per unit cell equals 1, 4, and 2, respectively. This corresponds to the number of “spheres” per unit cell.
152
MICHIELSEN ET AL.
Figure 13. Minkowski functionals as a function of r for BCC lattice structures with M = 8 and L 0 = 16 with periodic boundary conditions. Solid line: Perfect BCC lattice. Dashed line: BCC lattice to which ±30% of impurities have been added at randomly chosen positions. Dotted line: BCC lattice of which ±30% of randomly chosen basic lattice points have been moved over a randomly chosen distance 0 or 1. Dash-dotted line: BCC lattice of which all the basic lattice points have been moved over a randomly chosen distance 0 or 1.
Crystal structures formed in materials are not perfect. Therefore, it is of interest to study the influence of defects on the curves shown in Figure 12 for the BCC lattice. Imperfections in the crystal structure may be formed by the absence or by small displacements of some of the basic lattice points. Also, the presence of impurities, creating extra lattice points, causes an imperfect crystal structure. In Figure 13 we show the Minkowski functionals as a function of r for perfect and imperfect BCC lattice structures. The solid curve depicts the data for a perfect BCC lattice containing M = 8 unit cells of linear dimension L 0 = 16. The dashed curve shows the data for the same BCC lattice to which ±30% of defects have been added at randomly chosen positions. The dotted curve depicts the results of displacing ±30% randomly chosen basic lattice points over a random distance 0 or 1. Apart from some minor changes, the
ASPECTS OF MATHEMATICAL MORPHOLOGY
153
three curves behave in the same way. Only if we move all the lattice points , and , over a random distance 0 or 1 (dash-dotted lines), the curves for B χ differ qualitatively from the ones of the perfect BCC lattice. Therefore, we may conclude that the presence of small amounts of defects in the crystal structure does not alter the characteristic behavior of the Minkowski functionals as a function of r . An appealing feature of MIA is that it is capable of distinghuishing different lattice types even if the amount of lattice points is relatively small. MIA is not very sensitive to finite-size effects (Mecke, 2000).
B. Dislocations Dislocations comprise an important class of defects because they are the basic carriers of plasticity in crystalline material. This is even so, to a rather great extent, for quasi-crystalline and even noncrystalline solid material that can be described in terms of a network of disclinations, i.e., groups of dislocations, ordered and disordered, respectively. In a well-annealed crystal, the dislocation density, defined as the total length of dislocations per unit volume, is usually between 109 and 1012 m−2 (De Hosson et al., 1983). Dislocations represent discontinuities in displacements and they can also be used to describe, at least in a mathematical sense, a macroscopic static crack. As such, the dislocation can be considered to be the basic building block of a crack. This idea that a crack can be thought of as an array of discrete coplanar and parallel dislocations was introduced by Eshelby et al. (1951) and by Leibfried (1951). For dislocationbased fracture mechanics, references are made to Lardner’s (1974) book and to the recently published excellent text by Weertman (1996). The two simplest kinds of dislocations are the edge and screw dislocations (Hirth and Lothe, 1968). An edge dislocation can be considered by inserting an extra half plane of atoms into the lattice. The edge of the extra half plane of atoms is called the dislocation line. The presence of a screw dislocation in the crystal transforms the crystal planes into a helicoidal surface, that is, the atom planes perpendicular to the dislocation line are turned into a spiral ramp. The atoms in a crystal containing a dislocation are displaced from their perfect lattice positions. If a complete circuit is made around a dislocation line, the displacement of the end point must differ from the displacement of the starting point by the length of the Burgers vector b. In the case of a screw dislocation, the Burgers vector is parallel to the dislocation line. The Burgers vector of an edge dislocation is perpendicular to the dislocation line. The slip plane is uniquely defined as the plane that contains both the dislocation line and the Burgers vector (Hirth and Lothe, 1968). The glide of an edge dislocation
154
MICHIELSEN ET AL.
is therefore limited to a specific plane. However, the dislocation line and the Burgers vector of a screw dislocation do not define a unique plane and hence the glide of the dislocation is not restricted to a specific plane. Movement of the screw dislocation produces a displacement parallel to the dislocation line. In what follows, we orient the coordinate system such that the z axis coincides with the dislocation line. The y axis is chosen to be perpendicular to the slip plane, the plane on which a dislocation line glides. Hence, for a screw dislocation, the Burgers vector determines the position of the z axis and for an edge dislocation it fixes the x axis. In the case of a screw dislocation, there are no displacements in the x and y direction (u x = u y = 0). The dislocation in the z direction increases uniformly from zero to b ≡ |b| and for the dislocation in an isotropic linear elastic medium the displacement reads (Hirth and Lothe, 1968) uz =
y b tan−1 , 2π x
(47)
where b is the magnitude of the Burgers vector of the dislocation. For an edge dislocation, the displacements are given by (Hirth and Lothe, 1968) " # xy b −1 y ux = tan + 2π x 2(1 − ν)(x 2 + y 2 ) " # 1 − 2ν x 2 − y2 b 2 2 uy = − ln(x + y ) + 2π 4(1 − ν) 4(1 − ν)(x 2 + y 2 ) u z = 0,
(48)
where ν denotes Poisson’s ratio. Typical values of ν for metallic and ceramic solids lie in the range 0.2–0.45 (Hirth and Lothe, 1968). The displacement fields of Eqs. (47) and (48) describe the fields of stationary dislocations that are affected by the velocity of a moving dislocation in a drag-controlled regime (De Hosson et al., 2001; Roos et al., 2001a,b). Possible Burgers vectors are determined by the crystallographic structure of the crystal. A dislocation whose Burgers vector is a lattice translation vector is known as a perfect dislocation (Hirth and Lothe, 1968). We consider only Burgers vectors that are the shortest lattice translation vectors on the characteristic slip planes. For the FCC lattice it is observed experimentally that slip takes place on the {111} planes along the 110 directions (Hirth and Lothe, 1968). We choose the Burgers vector to point along the [110] direction. Thus the smallest possible Burgers vector of a perfect dislocation is b = [110]L 0 /2
ASPECTS OF MATHEMATICAL MORPHOLOGY
155
√ with length b = L 0 / 2. For the BCC lattice, the smallest possible √ Burgers vector of a perfect dislocation is b = [111]L 0 /2 with length b = 3L 0 /2 (Hirth and Lothe, 1968). Experimentally, slip has been observed on the {110}, {112}, and {123} planes (Hirth and Lothe, 1968). We consider as slip planes the {110} planes, which are the most densely packed. To compute the Minkowski functionals for the FCC and BCC lattices with a screw or edge dislocation, we first create a perfect FCC and BCC lattice by imbedding these lattices in a cubic lattice with lattice constant one and use (44)–(46) to determine the positions of the black pixels. For the four different cases, we rotate the coordinate system such that ¯ ¯ 2] ¯ FCC, screw : z = [100], y = [111], x = y × z = [11 ¯ ¯ BCC, screw : z = [111], y = [110], x = y × z = [112] ¯ ¯ FCC, edge : x = [110], y = [111], z = x × y = [112]
¯ ¯ BCC, edge : x = [111], y = [110], z = x × y = [1¯ 12],
(49)
and we compute the atom displacements using Eqs. (47) and (48). Then we rotate back to the original (x, y, z) coordinate system. Finally, we follow the procedure described in Section III.C to transform the resulting point pattern into a pattern of “spherical” grains of radius r or cubic grains of edge length r , and study the behavior of the Minkowski functionals, as a function of r . For the computation of the Minkowski functionals, we make use of free boundary conditions. , for FCC and BCC crystals containing screw or The results for , V, , S, and B edge dislocations differ only very little from the results for the perfect crystal structures. Therefore, we only present the results for , χ . Figure 14 depicts the Euler characteristic for a perfect FCC lattice (Fig. 14a,d), an FCC lattice with a screw dislocation in the center (Fig. 14b,e), and an FCC lattice with an edge dislocation located at the crystal center (Fig. 14c,f). The FCC lattice contains M = 5 unit cells of linear dimension L 0 = 16. In Figure 14a,b,c the grains are spheres and in Figure 14d,e,f the grains are cubes. Similar results for the BCC lattice are shown in Figure 15. As seen from Figures 14 and 15, the Euler characteristics as a function of r for the FCC and BCC lattices show a completely different behavior, independent of the choice of the shape of the grains. This was already concluded from Figure 12 where spherical grains and periodic boundaries were used. Figure 14 shows that using spherical grains to study the Euler characteristic as a function of r makes it difficult to detect a dislocation in an FCC lattice. Only very small changes in the curve for the Euler characteristic are seen (except for the change in the amplitudes of the peaks). Using cubic grains instead of spherical grains gives a clear signature
156
MICHIELSEN ET AL.
Figure 14. Euler characteristic as a function of r for FCC lattice structures with M = 5 and L 0 = 16 with free boundary conditions. (a) Perfect FCC lattice, grains are spheres. (b) FCC lattice with screw dislocation at the center, grains are spheres. (c) FCC lattice with edge dislocation at the center, grains are spheres. (d) Perfect FCC lattice, grains are cubes. (e) FCC lattice with screw dislocation at the center, grains are cubes. (f) FCC lattice with edge dislocation at the centre, grains are cubes.
of the presence of a dislocation in , χ . However, the distinction between a screw and an edge dislocation is not clear. For the BCC lattice, it is better to use spherical than cubic grains to detect dislocations in the crystal, as seen from Figure 15. Also, for the BCC lattice, it is not easy to distinguish between an edge and a screw dislocation. If we increase the number of unit cells in the crystal, the signature of the dislocation in the Euler characteristic weakens (results not shown). This is due to the large contribution from the perfect lattice part to , χ compared to the contribution from the part of the crystal containing the dislocation. Hence,
ASPECTS OF MATHEMATICAL MORPHOLOGY
157
Figure 15. Euler characteristic as a function of r for BCC lattice structures with M = 5 and L 0 = 16 with free boundary conditions. (a) Perfect BCC lattice, grains are spheres. (b) BCC lattice with screw dislocation at the center, grains are spheres. (c) BCC lattice with edge dislocation at the center, grains are spheres. (d) Perfect BCC lattice, grains are cubes. (e) BCC lattice with screw dislocation at the center, grains are cubes. (f) BCC lattice with edge dislocation at the center, grains are cubes.
for large crystal structures, local measurements of the Euler characteristic are necessary to detect dislocations.
C. Random Point Sets We consider a collection of N points pi , with positions generated from a , ⊂ Rd of uniformly uncorrelated random distribution, in a convex domain Euclidean space. The mean density of points equals ρ = N /, where
158
MICHIELSEN ET AL.
, We attach to every point pi (germ) a grain Ai ∈ R, denotes the volume of . where R denotes the convex ring, the class of all subsets A of Rd which can be expressed as finite unions of compact convex sets (see Section II.E). A configuration of the grains Ai gives rise to a set A N ∈ R
AN =
N &
si Ai ,
(50)
i=1
˜ where si ∈ S , under the assumption that the translations are restricted to . Here S denotes the group of all symmetry operations in Rd . This random distribution of grains includes the Boolean model (Matheron, 1975), a basic model in stereology and stochastic geometry (Santal´o, 1976; Stoyan et al., 1989). A nice feature of this model is that the configurational average of the Minkowski functionals of A N can be calculated analytically (Mecke and Wagner, 1991; Michielsen and De Raedt, 2001; Davy, 1976; Kellerer, 1984). This is useful to assess the validity of numerical calculations. We first consider the case of Euclidean space. In the bulk limit N , → ∞, ρ = N / fixed, the closed form expressions for the configurational averages Mν /N N , ν = 0, . . . , d are known exactly (Mecke and Wagner, 1991; Michielsen and De Raedt, 2001; Davy, 1976; Kellerer, 1984) and are given by M0 /N N = (1 − e−ρm 0 )/ρ,
M1 /N N = m 1 e−ρm 0 , M2 /N N = m 2 − m 21 ρ e−ρm 0 , M3 /N N = m 3 − 3m 1 m 2 ρ + m 31 ρ 2 e−ρm 0 ,
(51a) (51b) (51c) (51d)
where we introduced the notation Mν ≡ Mν(d) , Mν N denote the configurational average of the Minkowski functionals over realizations with density ρ and m ν denote the mean values of the Minkowski functionals of a single grain. In 2D Euclidean space, (51) reduces to ,(ρ) = (1 − e−ρu ), U , P(ρ) = pρ 1/2 e−ρu , $ % 1 2 p ρ e−ρu , , χ (ρ) = 1 − 4π
(52a) (52b) (52c)
where u and p denote the mean values of the area and perimeter of a single grain. Note that in 2D (51d) has no meaning. The Euler characteristic of a single grain equals one.
ASPECTS OF MATHEMATICAL MORPHOLOGY
159
In 3D Euclidean space we find , V (ρ) = (1 − e−ρv ), , S(ρ) = sρ 2/3 e−ρv , % $ π s2 1/3 , B(ρ) = 2bρ ρ e−ρv , 1− 64 b % $ π 3 2 −ρv 1 s ρ e , , χ (ρ) = 1 − sbρ + 2 384
(53a) (53b) (53c) (53d)
where the mean values of the volume, area, and mean breadth of a single grain are denoted by v, s, and b, respectively. The Euler characteristic of a single grain equals one. The mean values of the Minkowski functionals depend on the (averaged) shape of the single grains. For instance, in the 2D case where the (averaged) grains are circular discs of radius r , u = πr 2 and p = 2πr . If the (averaged) grains are spheres of radius r , we employ (53) with v = 4πr 3 /3, s = 4πr 2 and b = 2r . In the latter case we obtain , V = 1 − e−n , , S = 4πρ 2/3r 2 e−n , $ % 2 , = 4ρ 1/3r 1 − 3π n e−n , B 32 $ % 3π 2 2 −n , χ = 1 − 3n + n e , 32
(54a) (54b) (54c) (54d)
with n = 4πr 3 ρ/3. We adopt the procedure outlined in Section III.C to compute the morphological properties of a uniform random distribution of points in a cube of edge length L, subject to periodic boundaries. In Figure 16, we depict the Minkowski functionals as a function of r for two random point sets with L = 128 and different density. The solid (dashed) lines show the data for N = 1024 (N = 512). For both cases the behavior of the Minkowski functionals as a function of r is very similar: The curves show the same qualitative behavior (the grains have the same shape in both cases) and are only shifted with respect to each other. Results (not shown) for various other system sizes and densities show similar, minor quantitative differences. For small r , the grains are isolated leading to a small covered volume and surface area and to a positive Euler characteristic. For large r , the grains largely overlap and cover almost completely the whole cube. Only small cavities remain. This gives rise to a large covered volume, a small surface area, and a positive Euler characteristic which approaches zero in
160
MICHIELSEN ET AL.
Figure 16. Minkowski functionals for random point sets as a function of the radius r of the “spherical” grains. Periodic boundary conditions have been employed. Solid lines: 1024 points in a cubic box of edge length 128. Dash-dotted lines: Fit to this data, using the expressions given by Eq. (55). Dashed lines: 512 points in a cubic box of edge length 128. Dotted lines: Fit to this data, using the expressions given by Eq. (55).
the case of the completely covered cube. For intermediate r , the coverage has a tunnel-like structure with a negative Euler characteristic and a large surface area. The dash-dotted (N = 1024) and dotted (N = 512) lines in Figure 16 are the results obtained by fitting , VF = 1 − e−n , , S F = sρ 2/3 e−n , % $ 2 ,F = 2 b − πρs ρ 1/3 e−n , B 64 $ % ρsb πρ 2 s 3 , χF = 1 − f4 + f 5 e−n , 2 384
(55a) (55b) (55c) (55d)
ASPECTS OF MATHEMATICAL MORPHOLOGY
161
to the data with n = 4πρ f 1r 3 /3, s = 4π f 2r 2 , and b = 2 f 3r . The functional ,F , and , SF, B χ F is chosen to be the same as for perfect spherical behavior of , VF , , grains in the Euclidean space (see (54)). The fitting parameters f 1 , . . . , f 5 have been introduced to take into account that in practice we are working on a lattice and are approximating spheres by discrete structures. We find for N = 512 and N = 1024, f 1 = 0.108, f 2 = 0.32, f 3 = 0.8, f 4 = 0.72, and f 5 = 0.77 for the dash-dotted (dotted) line by fitting the solid (dashed) line. The Minkowski functionals of random point sets generically display the behavior shown in Figure 16. In general, the fitting parameters f i considerably deviate from their Euclidian value ( f i = 1). Clearly the use of a lattice introduces some artifacts, which, after all, is not unexpected. Recall that some of these artifacts will remain if we use a finer mesh (Michielsen and De Raedt, 2001). On a regular d-dimensional lattice, it may be more natural to work with hypercubes instead of digital approximations of the corresponding Euclidean shapes. This suggests the use of integral geometry on a lattice (see Sections II.H and II.I). Thus, we consider a collection of N pixels pi in a hypercubic domain Z ⊂ Zd of volume |Z | = L d . The positions of the pixels are generated from a uniformly uncorrelated random distribution. The mean density of pixels equals ρ = N /|Z |. As before, we attach to every germ pi a hypercubic grain Ci . In Appendices D and E we give a derivation of the lattice equivalent of (51) by making use of the kinematic formulae (27) and (36), respectively. Our derivation differs from the one given in Mecke and Wagner (1991) in the sense that it uses another technique and is valid for small systems too. In the bulk limit N , |Z | → ∞ with ρ fixed, the averaged Minkowski functionals Vν ≡ Vν(d) of random configurations of grains on a lattice are given by -
-
-
-
V0 N V1 N V2 N V3 N
. . . .
N
= (1 − e−ρv0 )/ρ
N
= e−ρv0 (1 − e−ρv1 )/ρ
N
N
= −e−ρv0 1 − 2e−ρv1 + e−ρ(2v1 +v2 ) /ρ
= e−ρv0 1 − 3e−ρv1 + 3e−ρ(2v1 +v2 ) − e−ρ(3v1 +3v2 +v3 ) /ρ,
(56)
where vν denote the mean values of Vν for one single grain. Analogously, in the bulk limit N , |Z | → ∞ with ρ fixed, the averaged functionals Fν of random
162
MICHIELSEN ET AL.
configurations of grains on a lattice are given by $ %−1 !/ - . $ % Fν d d 1 − exp −ρ f ν = ρ; ν = 0, . . . , d, ν ν N N
(57)
where f ν denote the mean values of Fν for one single grain. Rewriting formulae (56) and (57) in terms of the morphological functionals U , P, χ for two dimensions and V , S, B, χ for three dimensions and specializing to (averaged) square and cubic grains so that vν = r d−ν and f ν = (dν )r ν (r + 1)d−ν for ν = 0, . . . , d, we have 2
, = 1 − e−ρr , U 2
, P = 4e−ρr (1 − e−ρr )/ρ 1/2 , 2 , χ = e−ρr − 1 + 2e−ρr − e−ρ(2r +1) /ρ,
(58a) (58b) (58c)
for the case of a 2D lattice, and 3
, V = 1 − e−ρr , 3
(59a) 2
, S = 6e−ρr (1 − e−ρr )/ρ 1/3 , , = 3e−ρr 3 − 1 + 2e−ρr 2 − e−ρ(2r 2 +r ) /ρ 2/3 , B 3 2 2 2 , χ = e−ρr 1 − 3e−ρr + 3e−ρ(2r +r ) − e−ρ(3r +3r +1) /ρ,
(59b) (59c) (59d)
for the 3D lattice. In (58) and (59) the linear size of the (averaged) square and cubic grain is denoted by r . Note that r is a positive integer. In Figures 17 and 18 we show the Minkowski functionals for one realization of a 2D (3D) random points set. Square (cubic) grains were used to compute the Minkowski functionals. The linear size of the system L = 128 and the number of points N = 1024. The solid lines are the results obtained from (58) and (59). There is excellent agreement between the numerical data and the theoretical results. Note that there is no need to use adjustable parameters if we adopt the lattice version of integral geometry.
D. Topology of Triply Periodic Minimal Surfaces A minimal surface in R3 is defined as a surface ∂ P for which the mean curvature H = (1/R1 + 1/R2 )/2 is zero at each of its points, where R1 and R2 are the two principal radii of curvature. Hence the Gaussian curvature G = 1/R1 R2 is always nonpositive. For every closed circuit on the surface, the area is a minimum. We will consider the triply periodic minimal surfaces (TPMS),
ASPECTS OF MATHEMATICAL MORPHOLOGY
163
Figure 17. Minkowski functionals for random point sets on a square lattice subject to periodic boundary conditions. The grains are squares of linear size r . Markers: N = 1024 points in a square of edge length L = 128. Solid lines: Analytical result for the infinite system, as obtained from the lattice formulation of integral geometry.
minimal surfaces that are periodic in three independent directions. Structures related to TPMS may form spontaneously in physicochemical and in biological systems (Hyde et al., 1997; Klinowski et al., 1996). Examples may be found in various crystal structures (Hyde et al., 1997; von Schnering and Nesper, 1987; St. Andersson et al., 1984; Mackay, 1985, 1988; Barnes et al., 1990), lipid-containing systems (Luzzati et al., 1968, 1993; Longley and McIntosh, 1983; Larsson, 1983; Fontell, 1990; Mariani et al., 1999), polymers (Thomas et al., 1986, 1988; Hasegawa et al., 1987; Anderson and Thomas, 1988; Mogi et al., 1992; Xie et al., 1993; Matsushita et al., 1994; Schulz et al., 1994; Hajduk et al., 1994, 1995; Jinnai et al., 1997, 2001; Sakurai et al., 1998; Hamersky et al., 1998; Vigild et al., 1998; Bates and Fredrickson, 1999), skeletal elements in sea urchins (Donnay and Pawson, 1969; Nissen, 1969), and cell membranes (Larsson, 1989). Recently, TPMS became of interest in
164
MICHIELSEN ET AL.
Figure 18. Minkowski functionals for random point sets on a cubic lattice subject to periodic boundary conditions. The grains are cubes of linear size r . Markers: N = 1024 points in a cube of size L = 128. Solid lines: Analytical result for the infinite system, as obtained from the lattice formulation of integral geometry.
the analysis of the relations between the geometry and topology of the surface and surface diffusion (Holyst et al., 1999; Plewczy´nski and Holyst, 2000, 2001). A TPMS is either free of self-intersections or may intersect itself in a more or less complicated way. Each TPMS without self-intersections is two-sided and subdivides R3 into two infinite, connected but disjunct regions. These two regions, or labyrinths, are not simply connected. They interpenetrate each other in a complicated way. The two labyrinths may differ in shape or they may be congruent, that is, there exist symmetry operations mapping one labyrinth onto the other. In the latter case, the surface is called a balance surface (Fischer and Koch, 1987). Balance surfaces divide space into two labyrinths with equal volume fractions. The symmetry of a balance surface is described by a group– subgroup pair H/I of spacegroups, where H contains all isometries of R3 ,
,
ASPECTS OF MATHEMATICAL MORPHOLOGY
165
which map the surface onto itself. An isometry of H maps each side of the surface and each labyrinth either onto itself or onto the other side and the other labyrinth (Fischer and Koch, 1987). I contains only those isometries which map each side of the surface and each labyrinth onto itself. If the two sides of a balance surface are “colored” so that they are symmetrically distinct, black–white space groups instead of the group–subgroup pairs with index 2 may be used to describe its symmetry (Fischer and Koch, 1987). In this case, the surface is called oriented. Nonbalance surfaces have I ≡ H and divide space into two labyrinths with unequal volume fractions. The topology of a TPMS can be characterized by means of the Euler characteristic χ which is related to the genus g of the surface by means of g = 1 − χ(∂ P )/2 (see Section II.F). A finite surface of genus g is the topological equivalent of a sphere with g handles. In this sense, a TPMS has an infinite Euler characteristic and genus. Therefore, to characterize the topology of a TPMS, the genus and Euler characteristic are calculated per unit cell. There are two common choices of unit cells, the lattice fundamental region and the crystallographic cell (Grosse-Brauckmann, 1997). The lattice fundamental region contains the smallest region of the surface that reproduces the complete surface upon translation of this unit cell alone. The crystallographic cell is the smallest cube generating space by the lattice and can contain many lattice fundamental regions. We give our data for the crystallographic cell, simply called the unit cell from now on. The topology of a TPMS can be determined in different ways: 1. by means of the genus calculated by making use of the labyrinth graphs (Schoen, 1970; Hyde, 1989), 2. by means of the Euler characteristic determined with the aid of any tiling on the surface (Fischer and Koch, 1989), 3. by means of the genus computed making use of the flat points of the surface (Hyde, 1989), 4. by means of the simplex decomposition method (Aksimentiev et al., 2002), 5. by means of the Euler characteristic obtained with MIA. In previous work, we analyzed the topology (via the computation of the Euler characteristic) and the geometry of the P (primitive), the D (double diamond) and the G (gyroid) surfaces (Schoen, 1970; Schwarz, 1890) by means of integral-geometry-based MIA (Michielsen and De Raedt, 2001; Michielsen et al., 2000). Here we use the same method to characterize the topology of the minimal balance surfaces S (Fischer and Koch, 1987), C(P) (Neovius, 1883), C(Y)
166
MICHIELSEN ET AL.
(Fischer and Koch, 1987), ± Y (Fischer and Koch, 1987), C(± Y) (Koch and Fischer, 1988) and the minimal nonbalance surfaces I-WP (Schoen, 1970) and F-RD (Schoen, 1970). These structures serve as a good test case for integral-geometry-based MIA because compared to the P, D, and G surfaces, these minimal surfaces typically have a more complicated structure per unit cell. TPMS, as well as other periodic surfaces, such as equipotential (and zero potential) and Fermi surfaces, can be approximated by periodic nodal surfaces (von Schnering and Nesper, 1987, 1991; Mackay, 1988, 1993; Roaf, 1963). These can be obtained from the roots of the series |F(hkl)| cos(2π(hx + ky + lz) − αhkl ) = 0, (60) hkl
where αhkl and F(hkl) denote a phase shift and the structure factor, respectively. Truncating the series to the leading term determines the principal nodal surfaces of a given symmetry (von Schnering and Nesper, 1991). The nodal P surface is given by cos X + cos Y + cos Z = 0,
(61)
where X = 2π x/L 0 , Y = 2π y/L 0 , Z = 2π z/L 0 and L 0 denotes the length of the crystallographic unit cell. The nodal primitive P surface is shown in Figure 19 together with the nodal double diamond D surface. The latter is
Figure 19. Unit cube for the nodal primitive P surface (a) and the nodal double diamond D surface (b). The surfaces are generated from Eqs. (61) and (62), respectively.
ASPECTS OF MATHEMATICAL MORPHOLOGY
167
Figure 20. Unit cube for the nodal gyroid G surface (a) and the nodal S surface (b). The surfaces are generated from Eqs. (63) and (64), respectively.
defined by sin X sin Y sin Z + sin X cos Y cos Z + cos X sin Y cos Z + cos X cos Y sin Z = 0.
(62)
The nodal gyroid surface is shown in Figure 20 and is defined by sin X cos Y + sin Y cos Z + cos X sin Z = 0.
(63)
Figure 20 also shows the nodal S surface given by cos 2X sin Y cos Z + cos X cos 2Y sin Z + sin X cos Y cos 2Z = 0.
(64)
The nodal Neovius surface C(P) and the nodal C(Y) surface are depicted in Figure 21. Their representations read cos X + cos Y + cos Z + 4 cos X cos Y cos Z = 0,
(65)
and − sin X sin Y sin Z + sin 2X sin Y + sin 2Y sin Z + sin X sin 2Z − cos X cos Y cos Z + sin 2X cos Z + cos X sin 2Y + cos Y sin 2Z = 0, (66)
168
MICHIELSEN ET AL.
Figure 21. Unit cube for the nodal Neovius C(P) surface (a) and the nodal C(Y) surface (b). The surfaces are generated from Eqs. (65) and (66), respectively.
respectively. The nodal ± Y surface is given by 2 cos X cos Y cos Z + sin 2X sin Y + sin X sin 2Z + sin 2Y sin Z = 0, (67) and is shown in Figure 22 together with the nodal surface for its complementary surface C(± Y ) which is given by −2 cos X cos Y cos Z + sin 2X sin Y + sin X sin 2Z + sin 2Y sin Z = 0. (68)
Figure 22. Unit cube for the nodal ± Y surface (a) and the nodal C(± Y) surface (b). The surfaces are generated from Eqs. (67) and (68), respectively.
ASPECTS OF MATHEMATICAL MORPHOLOGY
169
Figure 23. Unit cube for the nodal I-WP surface (a) and the nodal F-RD surface (b). The surfaces are generated from Eqs. (69) and (70), respectively.
Finally, the minimal nonbalance surfaces I-WP and F-RD are depicted in Figure 23. They are represented by 2(cos X cos Y + cos Y cos Z + cos X cos Z ) − cos 2X − cos 2Y − cos 2Z = 0,
(69)
and 4 cos X cos Y cos Z − cos 2X cos 2Y − cos 2Y cos 2Z − cos 2X cos 2Z = 0,
(70)
respectively. It is known that the properties of the nodal approximations can differ considerably from those of real TPMS (Barnes et al., 1990). The quality of the nodal approximation depends on the number of terms in the Fourier series and varies considerably for different structures (Schwarz and Gompper, 1999; Gandy et al., 2001). Table 8 gives our results for the Euler characteristic of the nodal surfaces for one unit cell of length L 0 = 128 together with the numbers found in literature (Fischer and Koch, 1989; Schwarz and Gompper, 1999). As seen from Table 8 the values calculated using the integral-geometry approach are in good agreement with the numbers found in literature. In conclusion, integral-geometry-based MIA is a convenient tool to study the topology of challenging surfaces such as some TPMS. In particular, to
170
MICHIELSEN ET AL.
TABLE 8 Minimal Surfaces, Group–Subgroup Pairs H-I and the Euler Characteristic Minimal surface P D G S C(P) C(Y) ± Y C(± Y) I-WP F-RD
H
I
χ (a) (∂ P )
χ (b) (∂ P )
χ M I A (∂ P )
¯ I m 3m ¯ Pn 3m ¯ I a 3d ¯ I a 3d ¯ I m 3m I 41 32 I a 3¯ I a 3¯ ¯ I m 3m ¯ Fm 3m
¯ Pm 3m ¯ Fd 3m I 41 32 ¯ I 43d ¯ Pm 3m P43 32 Pa 3¯ Pa 3¯ ¯ I m 3m ¯ Fm 3m
−4 −4 −4 −20 −16 −24 −40 −24 −6 −10
−4 −16 −8 −40 −16 −24 – – −12 −40
−4 −16 −8 −40 −16 −24 −40 −24 −12 −40
Note. χ (a) (∂ P ) denotes the Euler characteristic per lattice fundamental region of the minimal surfaces obtained by means of labyrinth graphs and surface tilings (Fischer and Koch, 1989), χ (b) (∂ P ) denotes the Euler characteristic per unit cell of the nodal approximations calculated by triangulating the surface with the marching cube algorithm (Schwarz and Gompper, 1999), χ M I A (∂ P ) = 2χ M I A (P ) denotes the Euler characteristic per unit cell of the nodal approximations as obtained by means of the integral-geometry-based MIA.
study the topology of the TPMS, MIA does not require the use of labyrinth graphs or surface tilings (Fischer and Koch, 1989).
V. Computer Tomography Images of Metal Foams Metal foams have recently become a popular topic of research interest in the materials science community although these materials have existed for over almost 50 years (Elliot, 1956). At present, metal foams containing up to 95% porosity are being explored for applications that require a high specific stiffness and strength and high mechanical energy absorption and for heat exchangers (Ashby et al., 1998). Various studies on open-cell foams have shown that both the stiffness and the strength are dominated by bending, the former of which scales with ρ 2 and the latter with ρ 3/2 (with ρ the relative density) (Gibson and Ashby, 1997). In Figure 24, the bending of a strut in an open-cell foam (Duocel) at an early and a later stage is shown as observed by in situ tensile experiments in a Philips-XL30-FEG-ESEM. The mechanical response of closed-cell foams is more complicated because the deformation of the cell faces, as well as the edges, must be included to give a complete description. In metallic foams, both size effects (Andrews et al., 2001) and topology play a crucial role. However, only a few studies have investigated topological criteria that control
ASPECTS OF MATHEMATICAL MORPHOLOGY
171
Figure 24. In situ deformation in a Philips XL30-FEG-ESEM of Duocel 40 PPI foam with a relative density of approximately 7%. The alloy composition is A6061.
the deformation mechanism by analyzing the rigidity of frameworks composed of inextensional struts (Deshpande et al., 2001). The commercially available metal foams have random microstructures, but microstructures of periodic architectures can also be constructed with topologies that lead to properties superior to their stochastic analogues (Evans et al., 2001). For materials scientists, it is of interest to investigate the topology of the metal foams without destroying the sample. Using a computer tomograph (CT) scanner makes it possible to visualize the interior of a foam in a nondestructive manner and to produce a 3D image of the sample. Information about the morphological properties of the foams can be obtained from the computation of the Minkowski functionals for the 3D CT images. In this section, we first explain how we calculate the 3D Minkowski functionals from the CT images. Then we apply the technique to two aluminum foam samples of a different type. We demonstrate that using the morphological image
172
MICHIELSEN ET AL.
analysis technique to analyse the 3D CT images gives information about the morphological properties of the foam and might be used to classify the metal foams.
A. Computation of 3D Minkowski Functionals High-resolution CT scanners can produce huge data sets of 2D slices (of typically 1 Mb each). These 2D slices are used to reconstruct the 3D image (Russ, 1995). Our 3D images are cubes cut out from the complete CT image. Before we compute the Minkowski functionals, we first set a threshold to the 3D image. The program to calculate the Minkowski functionals for the 3D images always holds four slices in memory at a time. Basically, we use the same method and program as discussed in Section III.B and Appendix B; that is, we add active (black) pixels to an initially empty (white) image one by one. Before we add the pixel, we check if the pixel-to-be-added has active nearest neighbors and/or next-nearest neighbors and count the number of faces, edges, and vertices accordingly. At the onset of the computation, the first three slices are completely empty (all pixels are white) and the fourth slice contains the pixels of the first slice of the CT scan. We remove the pixels from the fourth slice (one by one, from left to right and from front to back) and add them to the second slice after checking its nearest neighbors and/or next-nearest neighbors in all directions (bottom direction is first slice and top direction is third slice) and counting the number of faces, edges, and vertices. We calculate the changes to V , S, B, and χ and proceed. If the fourth slice is emptied, we make a copy of the second slice in the first slice and load the second CT slice in the fourth slice. We now proceed in the same way as for the first step. We continue until all CT slices are processed. The number of arithmetic operations required to calculate the Minkowski functionals scales linearly with the number of active pixels in the image. The memory needed equals four times the memory required to store one CT slice. The computational and memory demands can be slightly reduced. The method described above uses the program given in Appendix B. This program is written for more general applications than the one we need here. The program can handle the situation in which we remove one pixel from an image of which we already calculated the Minkowski functionals and compute the change in the Minkowski functionals. This requires the check of all nearest and next-nearest neighbors (26 in total). In the CT case, we build up the image by adding pixel by pixel. Then only half of the neighbors need to be checked. Moreover, only storage for three slices is required. The procedure to calculate the Minkowski functionals in this more economic way is as follows. At startup of the computation, the first two slices are completely empty (all pixels are white) and the third slice contains the pixels
ASPECTS OF MATHEMATICAL MORPHOLOGY
173
of the first slice of the CT scan. We remove the pixels from the third slice (one by one, from left to right and from front to back) and add them to the second slice after checking its nearest neighbors and/or next-nearest neighbors to the left, front and bottom (first slice) and counting the number of faces, edges, and vertices. We calculate the changes to V , S, B, and χ and proceed. If the third slice is emptied, we make a copy of the second slice in the first slice and load the second CT slice in the third slice. We now proceed in the same way as for the first step. We continue until all CT slices are processed. B. Aluminum Foams with Closed-Cell and Open-Cell Structure An example of a 2D CT image of a closed-cell (left) and open-cell (right) aluminum foam is shown in Figure 25. We used hundreds of these 2D slices to reconstruct the 3D image of a closed-cell and open-cell aluminum foam. Figure 26 shows the 3D images of the two different aluminum foam samples. The image on the left corresponds to a closed-cell foam and the one on the right to a foam with a more open structure. In order to study the Minkowski functionals for these aluminum foams, we first put a threshold to the cubic image. For the given images, only very small thresholds are relevant. Considering medium to large thresholds (q > 50) leads to loose parts and completely disconnected structures in the image. For these large thresholds the resulting image no longer resembles the real foam structure. In Figure 27 and Figure 28, we depict the Minkowski functionals as a function of the threshold q for the closed-cell and open-cell structure, respectively. For the analysis of the foam structures we only consider 0 < q < 50 because
Figure 25. 2D computer tomography image of a closed-cell (left) and open-cell (right) aluminum foam.
174
MICHIELSEN ET AL.
Figure 26. 3D computer tomography images of two aluminum foams of different types. Left: Closed-cell aluminum foam. Right: Open-cell aluminum foam.
Figure 27. Minkowski functionals as a function of the threshold q as obtained from a 3D CT image of a dense aluminum foam.
ASPECTS OF MATHEMATICAL MORPHOLOGY
175
Figure 28. Minkowski functionals as a function of the threshold q as obtained from a 3D CT image of an open aluminum foam.
for larger q the image is almost completely empty. The covered volume for the closed-cell foam is much larger (four times) than the covered volume for the foam with the more open structure. The surface areas of the coverages show a similar feature. This suggests that in the aluminum foam with open structure, large open structures (cavities and tunnels) filled with air are present and that the closed-cell aluminum foam contains many small cavities and tunnels filled with air. This can also be seen from the structure images in Figure 26. For the open-cell aluminum foam, the mean breadth is positive and rather constant for 0 < q < 50. In the case of the closed-cell foam, the mean breadth changes from negative to positive values for 0 < q < 50. This indicates that in the closedcell foam more tunnels are present. For both foams, the Euler characteristic is negative, which means that on the average the surface is hyperbolic and contains a lot of tunnels. For the foam with the closed-cell structure, the Euler characteristic is much more negative than the Euler characteristic for the foam with the more open structure. The more negative χ becomes, the more tunnels in the structure and the more complex the structure (Nishikawa et al., 1998).
176
MICHIELSEN ET AL.
The analysis of the 3D Minkowski functionals clearly indicates that the two aluminum foams have a different internal structure. A more detailed morphological characterization of the foams can be made by investigating local instead of averaged Minkowski functionals. Additional information can also be obtained from the computation of the 2D Minkowski functionals of various slices. We have demonstrated that the morphological image analysis technique is useful in giving information of the 3D morphological properties of metal foams. Although the amount of data obtained by CT scans is huge, the memory and computation time requirements for the calculation of the Minkowski functionals are low. To study the morphological properties of various metal foams in more detail, more samples need to be analyzed and compared with each other. This challenging problem is left for future research.
VI. Summary Integral-geometry morphological image analysis characterizes patterns in terms of numerical quantities, called Minkowski functionals. These morphological descriptors have an intuitively clear geometrical and topological interpretation. Integral-geometry morphological image analysis yields information on structure in patterns. In most cases, this information is complementary to the one obtained from two-point correlation functions. A remarkable feature of MIA is the big contrast between the level of sophistication of the underlying mathematics and the ease with which MIA can be implemented and used. MIA does not require the surface to be regular, nor is there any need to introduce labyrinth graphs or surface tilings to compute derivatives. MIA is applied directly to the digitized representation of the patterns, it can be implemented with a few lines of computer code; it is computationally inexpensive, and is easy to use in practice. Therefore, we believe it should be part of everyone’s toolbox for analyzing geometrical objects and patterns.
Appendix A: Algorithm We describe a procedure to determine how the number of open bodies of each type changes when one adds (removes) one black pixel to (from) a given pattern. Using this procedure, it is easy to compute the Minkowski functionals for a given pattern, namely by adding the black pixels one-by-one to an initially empty (white) image.
ASPECTS OF MATHEMATICAL MORPHOLOGY
177
In 2D, the number n 2 (P ) of open squares building up the objects in the L x × L y image P (x, q) = P (i, j, q); (i = 1, . . . , L x , j = 1, . . . , L y ) increases (decreases) with one if one adds (removes) one black pixel at the position x = (i, j) to (from) the image. Therefore, if we add an black pixel, n 2 (P ) = 1,
(A.1)
where we introduce the symbol to indicate that we compute the difference. Similarly, the change in the number of open edges, n 1 (P ) is given by [Q(i + α, j, q) + Q(i, j + α, q)], (A.2) n 1 (P ) = α=±1
where Q(x, q) = 1 − P (x, q). For the change in the number of vertices, n 0 (P ), we find n 0 (P ) = Q(i + α, j, q)Q(i + α, j + β, q)Q(i, j + β, q). (A.3) α,β=±1
In 3D, the number n 3 (P ) of open cubes building up the objects in the L x × L y × j = 1, . . . , L y , L z image P (x, q) = P (i, j, k, q); (i = 1, . . . , L x , k = 1, . . . , L z ) increases (decreases) with one if one adds (removes) one black pixel to (from) the image at the position x = (i, j, k), i.e n 3 (P ) = 1. The change in n 2 (P ), the number of open faces, may be computed from [Q(i + α, j, k, q) + Q(i, j + α, k, q) + Q(i, j, k + α, q)]. n 2 (P ) = α=±1
(A.4)
The change in n 1 (P ), the number of open edges, reads n 1 (P ) = [Q(i + α, j, k, q)Q(i + α, j + β, k, q)Q(i, j + β, k, q) α,β=±1
+ Q(i, j + α, k, q)Q(i, j + α, k + β, q)Q(i, j, k + β, q)
+ Q(i + α, j, k, q)Q(i + α, j, k + β, q)Q(i, j, k + β, q)]. (A.5)
For the change in n 0 (P ), the number of vertices, we find Q(i + α, j, k, q)Q(i + α, j + β, k, q)Q(i, j + β, k, q) n 0 (P ) = α,β,γ =±1
× Q(i + α, j, k + γ , q)Q(i + α, j + β, k + γ , q) × Q(i, j + β, k + γ , q)Q(i, j, k + γ , q).
(A.6)
178
MICHIELSEN ET AL.
Appendix B: Programming Example (Fortran 90) ! ! Minkowski functionals 3D computes the Minkowski functionals ! (volume,surface,integral mean curvature,euler) for a 3D image, ! represented by the 1D array LATTICE(.). A pixel at (jx,jy,jz) is ! ! ! ! ! !
black if LATTICE(jx+Lx*(jy+Ly*jz))=1,otherwise LATTICE(jx+Lx*(jy+Ly*jz))=0. Here 0 < jx < Lx, 0 < jy < Ly, and 0 < jz < Lz. The array TMP(.) is used as work space. Putting FREE BOUNDARIES = 0 returns Minkowski functionals for periodic boundary conditions, other values return Minkowski functionals for the image on in a infinitely large empty background. subroutine minkowski functionals(Lx,Ly,Lz,free boundaries,tmp, & lattice,volume,surface,curvature,euler) implicit integer (a-z) integer lattice(0:*),tmp(0:*) vol=0 sur=0 cur=0 eul=0 if(free boundaries.eq.0) then ! periodic boundary conditions tmp(0:Lx*Ly*Lz-1)=0 else tmp(0:(Lx+2)*(Ly+2)*(Lz+2)-1)=0 endif do jz=0,Lz-1 do jy=0,Ly-1 do jx=0,Lx-1 i=jx+Lx*(jy+Ly*jz) if( lattice(i) > 0 ) then ! black pixel if(free boundaries.eq.0) then ! periodic boundary conditions call minko 3D periodic(Lx,Ly,Lz,jx,jy,jz,tmp,v,s,c,e) tmp(i)=1 ! can only be 0 or 1 in minko 3D periodic else call minko 3D free(Lx+2,Ly+2,Lz+2,jx+1,jy+1,jz+1,tmp,v,s,c,e) tmp(jx+1+(Lx+2)*(jy+1+(Ly+2)*(jz+1)))=1 endif vol=vol+v sur=sur+s
ASPECTS OF MATHEMATICAL MORPHOLOGY
179
cur=cur+c eul=eul+e endif enddo enddo enddo volume=vol surface=sur curvature=cur euler=eul end subroutine minko 3D periodic(Lx,Ly,Lz,jx,jy,jz,lattice, & volume,surface,curv,euler3D) implicit integer (a-z) integer lattice(0:LX*LY*Lz-1) parameter( & , & !(a*a*a, where a is lattice displacement) volume body=1 & !(-6*a*a, open body) surface body=-6 , & !(2*a*a, open face) surface face=2 , curv body=3 , , curv face=-2 , curv edge=1 euler3D body=-1 , euler3D face=1 , euler3D edge=-1 , euler3D vertex=1)
& & & & & &
nfaces=0 nedges=0 nvert=0 do i0=-1,1,2 jxi=jx+i0 if(jxi.lt.0) then jxi=Lx+jxi else if(jxi.ge.Lx) then jxi=jxi-Lx endif
!(3*a, open body) !(-2*a, open face) !(a, open line) !(open body) !(open face) !(open line) !(vertices)
180
MICHIELSEN ET AL. jyi=jy+i0 if(jyi.lt.0) then jyi=Ly+jyi else if(jyi.ge.Ly) then jyi=jyi-Ly endif jzi=jz+i0 if(jzi.lt.0) then jzi=Lz+jzi else if(jzi.ge.Lz) then jzi=jzi-Lz endif kc1=1-lattice(jxi+Lx*(jy+Ly*jz)) kc2=1-lattice(jx+Lx*(jyi+Ly*jz)) kc3=1-lattice(jx+Lx*(jy+Ly*jzi)) nfaces=nfaces+kc1+kc2+kc3 do j0=-1,1,2 jyj=jy+j0 if(jyj.lt.0) then jyj=Ly+jyj else if(jyj.ge.Ly) then jyj=jyj-Ly endif jzj=jz+j0 if(jzj.lt.0) then jzj=Lz+jzj else if(jzj.ge.Lz) then jzj=jzj-Lz endif k4=Lx*(jyj+Ly*jz) k7=Lx*(jy+Ly*jzj) kc7=1-lattice(jx+k7) kc1kc4kc5=kc1*(1-lattice(jxi+k4))*(1-lattice(jx+k4)) nedges=nedges+kc1kc4kc5+kc2*(1-lattice(jx+Lx*(jyi+Ly*jzj)))*kc7 & +kc1*(1-lattice(jxi+k7))*kc7 if(kc1kc4kc5.ne.0) then do k0=-1,1,2 jzk=jz+k0 if(jzk.lt.0) then jzk=Lz+jzk else if(jzk.ge.Lz) then jzk=jzk-Lz endif
ASPECTS OF MATHEMATICAL MORPHOLOGY
181
k9=Lx*(jy+Ly*jzk) k10=Lx*(jyj+Ly*jzk) nvert=nvert+(1-lattice(jxi+k9))*(1-lattice(jxi+k10)) & *(1-lattice(jx+k10))*(1-lattice(jx+k9)) enddo ! k0 endif ! kc1kc4kc5 enddo ! j0 enddo ! i0 volume=volume body surface=surface body+surface face*nfaces curv=curv body+curv face*nfaces+curv edge*nedges euler3D=euler3D body+euler3D face*nfaces & +euler3D edge*nedges+euler3D vertex*nvert return end subroutine minko 3D free(Lx,Ly,Lz,jx,jy,jz,lattice, & volume,surface,curvature,euler3D) implicit integer (a-z) integer lattice(0:LX*LY*Lz-1) parameter( & , & !(a*a*a, where a is lattice displacement) volume body=1 surface body=-6 surface face=2 curv body=3 curv face=-2 curv edge=1 euler3D body=-1 euler3D face=1
, , , , , , ,
& & & & & & &
!(-6*a*a, open body) !(2*a*a, open face) !(3*a, open body) !(-2*a, open face) !(a, open line) !(open body) !(open face)
euler3D edge=-1 , euler3D vertex=1)
&
!(open line) !(vertices)
nfaces=0 nedges=0 nvert=0 do i0=-1,1,2 jxi=jx+i0 jyi=jy+i0 jzi=jz+i0 kc1=1-lattice(jxi+Lx*(jy+Ly*jz)) kc2=1-lattice(jx+Lx*(jyi+Ly*jz)) kc3=1-lattice(jx+Lx*(jy+Ly*jzi))
182
MICHIELSEN ET AL. nfaces=nfaces+kc1+kc2+kc3 do j0=-1,1,2 jyj=jy+j0 jzj=jz+j0 k4=Lx*(jyj+Ly*jz) k7=Lx*(jy+Ly*jzj) kc7=1-lattice(jx+k7) kc1kc4kc5=kc1*(1-lattice(jxi+k4))*(1-lattice(jx+k4)) nedges=nedges+kc1kc4kc5+kc2*(1-lattice(jx+Lx*(jyi+Ly*jzj)))*kc7 & +kc1*(1-lattice(jxi+k7))*kc7 if(kc1kc4kc5.ne.0) then do k0=-1,1,2 jzk=jz+k0 k9=Lx*(jy+Ly*jzk) k10=Lx*(jyj+Ly*jzk) nvert=nvert+(1-lattice(jxi+k9))*(1-lattice(jxi+k10)) & *(1-lattice(jx+k10))*(1-lattice(jx+k9)) enddo ! k0 endif ! kc1kc4kc5 enddo ! j0 enddo ! i0 volume=volume body surface=surface body+surface face*nfaces curvature=curv body+curv face*nfaces+curv edge*nedges euler3D=euler3D body+euler3D face*nfaces & +euler3D edge*nedges+euler3D vertex*nvert end
Appendix C: Derivation of Eq. (36) We sketch the procedure to compute the coefficients cμν by considering the case d = 2. From Eq.(35) it follows that 1 Fμ (A ∩ s B) = c00 F0 (A)F0 (B) + c11 F1 (A)F1 (B) + c22 F2 (A)F2 (B) 8 s∈S ′ + c01 (F0 (A)F1 (B) + F1 (A)F0 (B))
+ c02 (F0 (A)F2 (B) + F2 (A)F0 (B)) + c12 (F1 (A)F2 (B) + F2 (A)F1 (B)) .
(C.1)
To obtain values for the constants cκν (κ = 0, 1, 2 ; ν = κ, 2), we take for A
183
ASPECTS OF MATHEMATICAL MORPHOLOGY TABLE 9 Quantities Used to Determine the Coefficients cμν in Eq. (C.1). A
B
F0 (A)
F0 (B)
F1 (A)
F1 (B)
F2 (A)
F2 (B)
S2 S2 S2 S1 S1 S0
S2 S1 S0 S1 S0 S0
4 4 4 2 2 1
4 2 1 2 1 1
4 4 4 1 1 0
4 1 0 1 0 0
1 1 1 0 0 0
1 0 0 0 0 0
s∈S ′ (A
∩ s B)
8{4{S0 }, 4{S1 }, {S2 }} 16{2{S0 }, {S1 }} 32{S0 } 4{6{S0 }, {S1 }} 16{S0 } 8{S0 }
and/or B the simplexes Sν for ν = 0, 1, 2. For all cases, the values for Fμ (A) and Fμ (B) are given in Table 9. Computation of the left-hand side of (C.1) requires an evaluation of all possible intersections of A and B. Figure 29 and Figure 30 show schematically how the intersections (in gray) may be obtained for the cases A = B = S1 and A = B= S2 , respectively. The ninth column of Table 9 summarizes the results for s∈S ′ (A ∩ s B) for all possible combinations of A and B. Once all the elements of the intersection are obtained, it is straightforward to compute the left-hand side of (C.1) for μ = 0, 1, 2. From (C.1) and the entries in Table 9, we obtain for μ = 0 the following set of equations ⎧ 16 = 16c00 + 16c11 + c22 + 32c01 + 8c02 + 8c12 ⎪ ⎪ ⎪ ⎪ 8 = 8c00 + 4c11 + 12c01 + 2c02 + c12 ⎪ ⎨ 4 = 4c00 + 4c01 + c02 . 4 = 4c00 + c11 + 4c01 ⎪ ⎪ ⎪ ⎪ ⎪ ⎩2 = 2c00 + c01 1 = c00
(C.2)
Figure 29. Intersections (in gray) of the simplexes A = S1 and B = S1 that appear in the expression of the kinematic formulae Eq.(36). Because of the rotational and reflection symmetry of B, each intersection contributes to Eq.(36) with a weight of four.
184
MICHIELSEN ET AL.
Figure 30. Intersections (in gray) of the simplexes A = S2 and B = S2 that appear in the expression of the kinematic formulae Eq. (36). Because of the rotational and reflection symmetry of B, each intersection contributes to Eq. (36) with a weight of eight.
Straightforward algebra then shows that 1 F0 (A ∩ s B) = F0 (A)F0 (B). 8 s∈S ′
(C.3)
Similarly, for μ = 1 and μ = 2 we obtain
1 1 F1 (A ∩ s B) = F1 (A)F1 (B), 8 s∈S ′ 2
(C.4)
1 F2 (A ∩ s B) = F2 (A)F2 (B), 8 s∈S ′
(C.5)
and
in agreement with Eq. (36).
Appendix D: Proof of Eq. (56) We will use the kinematic formulae (27) to compute the configurational average of the Minkowski functionals Vν(d) for a system of grains that are distributed randomly (and uniformly) in a hypercubic domain Z ⊂ Zd of volume |Z |. The grains are assumed to be either identical or, in the case that they have random shapes and size, have the same shape-and-size probability distribution. In this appendix, we will adopt the lattice version of integral geometry. Results for Euclidean space can be found elsewhere (Michielsen and De Raedt, 2001; Mecke, 2000).
185
ASPECTS OF MATHEMATICAL MORPHOLOGY
First, the aim is to compute the mean value of Vν(d) (A N ), i.e. the average over all configurations, grain sizes, and shapes. We first consider the configurational average of a single grain. Let us write the image A N formed by all grains as
A N = A N −1 ∪ s N A N .
(D.1)
We will sum over all possible symmetry operations s N of the( grain A N on the cubic lattice. With some misuse of notation we will write for this sum ( and define dsi = 2d d!|Z | ≡ . Making use of the properties of additivity and motion invariance of the Minkowski functionals, application of kinematic formulae (27) yields )
Vμ(d) (A N )
ds N = Vμ(d) (A N −1 ) + Vμ(d) (A N ) −
)
Vμ(d) (A N −1 ∩ s N A N )
= Vμ(d) (A N −1 ) + Vμ(d) (A N ) μ ν $ %$ % 1 ν μ (d) Vν(d) (A N −1 )Vμ−κ (A N ), − κ ν |Z |
ds N
(D.2)
ν=0 κ=0
for the configurational average over the single grain A N . It is clear that we can repeat this procedure, i.e., sum over all translations, rotations, and reflections of grain A N −1 and so on. The mathematical structure of this recursive procedure is most easily seen by introducing a matrix notation. With the shorthand Vμ ≡ Vμ(d=3) , (D. 2) reads )
VN
ds N = Q N V N −1 + R N ,
(D.3)
where the matrices V N , Q N and R N are given by ⎛ ⎞ ⎞ V0 (A N ) V0 (A N ) ⎜V (A )⎟ ⎜ V (A ) ⎟ ⎜ 1 N ⎟ ⎜ 1 N ⎟ VN = ⎜ ⎟, ⎟ , RN = ⎜ ⎝V2 (A N )⎠ ⎝ V2 (A N ) ⎠ V3 (A N ) V3 (A N ), ⎛ ⎞ αN 0 0 0 βN 0 0⎟ ⎜a , QN = ⎝ N bN cN γN 0 ⎠ d N 3e N f N δ N ⎛
(D.4a)
(D.4b)
186
MICHIELSEN ET AL.
with αN = 1 − γN = 1 − δN = 1 −
V0 (A N ) , |Z |
βN = 1 −
V0 (A N ) + V1 (A N ) , |Z |
V0 (A N ) + 2V1 (A N ) + V2 (A N ) , |Z |
V0 (A N ) + 3V1 (A N ) + 3V2 (A N ) + V3 (A N ) , |Z |
aN = βN − αN ,
c N = 2(γ N − β N ),
b N = α N − 2β N + γ N ,
d N = α N − 3β N + 3γ N − δ N ,
e N = 3(β N − 2γ N − δ N ),
f N = 3(α N − 2γ N + δ N ).
(D.5)
Repeating the steps that lead to (D.3) the configurational average over two grains A N and A N −1 reads ) ) ) ds N ds N −1 ds N −1 + RN = Q N V N −1 VN 2 = Q N Q N −1 V N −2 + V N R N −1 + R N ,
and the average over all possible configurations can be written as ) ) ds N · · · ds1 = Q N · · · Q2 R1 + Q N · · · Q3 R2 + · · · · · · VN N + Q N R N −1 + R N .
(D.6)
(D.7)
We now use the assumptions about the properties of the individual grains. If all grains are identical, we have Q = Qi and R = Ri for i = 1, . . . , N . Likewise, if the distribution of size and shape of the grains is the same for all grains, averaging (D.7) over this distribution also yields Q = Qi and R = Ri for all i. Evidently the latter case contains the former. Thus, we can simplify the notation by dropping the subscript of α N etc. Averaging (D.7) over the size and shape of the grains yields ) ) ds N . . . ds1 V N ≡ . . . V N = (1 + Q + · · · + Q N −1 )R. (D.8) N By mathematical induction it can be shown that ⎛ 0 0 αn n ⎜aUn (α, β) β 0 ⎜ Qn = ⎜ (n) cUn (β, γ ) γn ⎝ Q 1,3 Q (n) 1,4
Q (n) 2,4
f Un (γ , δ)
⎞ 0 0⎟ ⎟ ⎟, 0⎠
δn
(D.9)
ASPECTS OF MATHEMATICAL MORPHOLOGY
187
where Q (n) 1,3 = bUn (α, γ ) + acVn−1 (α, β, γ ), Q (n) 1,4 = dUn (α, δ)+aeVn−1 (α, β, δ)+b f Vn−1 (α, γ , δ)+ac f Wn−2 (α, β, γ , δ), Q (n) 2,4 = eUn (β, δ) + c f Vn−1 (β, γ , δ),
(D.10)
and Un (x, y) = xUn−1 + y n =
x n − yn , x−y
Un+1 (x, y) − Un+1 (x, z) , y−z (D.11) Wn (x, y, z, t) = x Wn−1 (x, y, z, t) + Vn (y, z, t). Vn (x, y, z) = x Vn−1 (x, y, z) + Un (y, z) =
Let us write vν ≡ Vν (Ai ) for the average over size and shape of the Minkowski functionals for each single grain Ai . We have v0 , |Z | v0 + v1 β =1− , |Z | α =1−
γ =1− δ =1−
v0 + 2v1 + v2 , |Z |
v0 + 3v1 + 3v2 + v3 , |Z |
and somewhat tedious but straightforward algebra yields ⎛ ⎞ 1 − αN ⎟ ⎜ αN − β N ⎜ ⎟ V N = |Z | ⎜ ⎟. N N N ⎝ −α + 2β − γ ⎠ α N − 3β N + 3γ N − δ N
(D.12)
(D.13)
In the bulk limit (N , |Z |) → ∞ with the density of particles ρ = N /|Z | fixed, we have $ % 0 v0 ρ 1 N v0 N lim α N = lim 1 − = e−ρv0 , (D.14) = lim 1 − N →∞ N →∞ N →∞ |Z | N
188
MICHIELSEN ET AL.
and
V N N →∞ N lim
⎛
⎞ (1 − e−ρv0 )/ρ ⎜ ⎟ e−ρv0 (1 − e−ρv1 )/ρ ⎜ ⎟ =⎜ ⎟. −e−ρv0 1 − 2e−ρv1 + e−ρ(2v1 +v2 ) /ρ ⎝ ⎠ −ρ(2v1 +v2 ) −ρ(3v1 +3v2 +v3 ) −ρv0 −ρv1 e + 3e −e /ρ 1 − 3e (D.15)
Expressions (D.15) agree with those of Mecke (2000).
Appendix E: Proof of Eq. (57) Here we start from the alternative formulation of integral geometry on a lattice (see Section II.I) and use kinematic formulae (36) to compute the configurational average of the functionals Fν . We adopt the same strategy as in Appendix D to compute the mean value of Fν (A N ). Making use of the properties of additivity and motion invariance of the functionals Fν , application of kinematic formulae (36) yields for the configurational average over the single grain A N 1 1 Fμ (A N ) = Fμ (A N −1 ) + Fμ (A N ) − Fμ (A N −1 ∩ s N A N ) s N ∈S ′ sN $ % 1 d −1 Fμ (A N −1 )Fμ (A N ), = Fμ (A N −1 ) + Fμ (A N ) − |Z | μ (E.1) where ≡ s∈S ′ 1 = 2d d!|Z | and |Z | denotes the number of lattice points of the finite lattice Zd . As in Appendix D, it is convenient to write (E.1) in matrix notation. We have 1 F N = Q N F N −1 + R N , s N ∈S ′ where, for d = 3, the matrices F N , Q N and R N are given by ⎞ ⎛ ⎞ ⎛ F0 (A N ) F0 (A N ) ⎜ F (A )⎟ ⎜ F (A ) ⎟ ⎜ 1 N ⎟ ⎜ 1 N ⎟ = FN = ⎜ , R ⎟, ⎟ ⎜ N ⎝ F2 (A N )⎠ ⎝ F2 (A N )⎠ F3 (A N ) F3 (A N )
(E.2)
(E.3a)
189
ASPECTS OF MATHEMATICAL MORPHOLOGY
⎛
⎜ ⎜ QN = ⎜ ⎜ ⎝
1−
F0 (A N ) |Z |
0
1−
0 0
0
0
0
F1 (A N ) 3|Z |
0
0
F2 (A N ) 3|Z |
0
0 0
1−
0
1−
⎞
F3 (A N ) |Z |
⎟ ⎟ ⎟ . (E.3b) ⎟ ⎠
Note that in constrast to Eq.(D.4b), (E.3b) is a diagonal matrix, leading to considerable simplification of the subsequent algebra. Repeated use of recursion (E.2) and averaging over all configurations, grain sizes and shapes yields F N ≡
1 N
FN
s1 ,...,s N ∈S ′
ds N . . . ds1 = (1 + Q + · · · + Q N −1 )R. N
(E.4)
It is easy to show that ⎛ 1−α N 1−α
1 + Q + ··· + Q
N −1
⎜ ⎜ 0 =⎜ ⎜ ⎝ 0 0
0
0
1−β N 1−β
0
0
1−γ N 1−γ
0
0
0
⎞
⎟ 0 ⎟ ⎟, ⎟ 0 ⎠
(E.5)
1−δ N 1−δ
where α = 1 − f 0 /|Z |, δ = 1 − f 3 /|Z |,
β = 1 − f 1 /3|Z |,
γ = 1 − f 2 /3|Z |, (E.6)
and f ν ≡ Fν (Ai ) denotes the average over size and shape of the functional Fν for a single grain Ai . Straightforward algebra yields ⎛ ⎞ 1 − αN ⎜3(1 − β N )⎟ ⎜ ⎟ F N = |Z | ⎜ (E.7) ⎟. ⎝3(1 − γ N )⎠ 1 − δN
In the bulk limit (N , |Z |) → ∞ with the density of particles ρ = N /|Z | fixed, we have % % $ $ f0ρ N f0 N = lim 1 − = e−ρ f0 , (E.8) lim α N = lim 1 − N →∞ N →∞ N →∞ |Z | N
190
MICHIELSEN ET AL.
and
lim
N →∞
F N N
⎞ (1 − e−ρ f0 )/ρ ⎜3(1 − e−ρ f1 /3 )/ρ ⎟ ⎜ ⎟ =⎜ ⎟. ⎝3(1 − e−ρ f2 /3 )/ρ ⎠ (1 − e−ρ f3 )/ρ ⎛
(E.9)
In general, for any d > 0 we find $ % $ %−1 !/ Fν N d d 1 − exp −ρ f ν lim = ρ; ν = 0, . . . , d. ν ν N →∞ N (E.10)
Acknowledgments The authors thank Freek Pasop of SkyScan for providing us the micro-CT images of aluminum foam, our master student Rutger van Merkerk for the SEM images, and Patrick Onck for discussions on modeling studies of the mechanical properties of metallic foams. Part of this research has been financially supported by the “Stichting Nationale Computer Faciliteiten (NCF).”
References Aksimentiev, A., Fialkowski, M., and Holyst, R. (2002). Morphology of surfaces in mesoscopic polymers, surfactants, electrons, or reaction-diffusion systems: Methods, simulations, and measurements, Adv. Chem. Phys. 121, 141–239. Anderson, D. M., and Thomas, E. L. (1988). Microdomain morphology of star copolymers in the strong-segregation limit, Macromolecules 21, 3221–3230. Andersson, St., Hyde, S. T., and von Schnering, H. G. (1984). The intrinsic curvature of solids, Z. Kristallogr. 168, 1–17. Andrews, E. W., Gioux, G., Onck, P., and Gibson, L. J. (2001). Size effects in ductile cellular solids, Int. J. Mech. Sci. 43, 701–713. Ashby, M. F., Evans, A. G., Hutchinson, J. W., and Fleck, N. A. (1998). Metal foams: A design guide. Cambridge: University of Cambridge. Ball, P. (1998). The Self-Made Tapestry: Pattern Formation in Nature. Oxford: Oxford Univ. Press. Barnes, I. S., Hyde, S. T., and Ninham, B. W. (1990). The caesium chloride zero potential surface is not the Schwarz P-surface, Coll. Phys. C7, 19–24. Bates, F. S., and Fredrickson, G. H. (1999). Block Copolymers-Designer Soft Materials. Advances in synthetic chemistry and statistical theory provide unparalleled control over molecular scale morphology in this class of macromolecules, Phys. Today 52, 32–38. Castleman, K. R. (1996). Digital Image Processing. Englewood Cliffs, NJ: Prentice Hall. ,
,
ASPECTS OF MATHEMATICAL MORPHOLOGY
191
da Fontoura Costa, L., and Cesar, Jr., R. M. (2001). Shape Analysis and Classification: Theory and Practice. Florida: CRC Press. Davy, P. (1976). Projected thick sections through multi-dimensional particle aggregates, J. Appl. Prob. 13, 714–722. De Hosson, J. Th. M., Kanert, O., and Sleeswyk, A. W. (1983). Dislocations in solids investigated by means of nuclear magnetic resonace, in Dislocations in Solids, p. 441–534. edited by F. R. N. Nabarro. Amsterdam: North-Holland. De Hosson, J.Th. M., Roos, A., and Metselaar, E. D. (2001). Temperature rise due to fast-moving dislocations, Philos. Mag. 81, 1099–1120. Deshpande, V. S., Ashby, M. F., and Fleck, N. A. (2001). Foam topology bending versus stretching dominated architectures, Acta Mater. 49, 1035–1040. Donnay, G., and Pawson, D. L. (1969). X-ray diffraction studies of echinoderm plates, Science 166, 1147–1150. Elliot, J. C. (1956). Method of producing metal foam, U.S. patent No. 2,751,289. Eshelby, J. D., Frank, F. C., and Nabarro, F. R. N. (1951). The equilibrium of linear arrays of dislocations, Phil. Mag. 42, 351–364. Evans, A. G., Hutchinson, J. W., Fleck, N. A., Ashby, M. F., and Wadley, H. N. G. (2001). The topological design of multifunctional cellular metals, Prog. Mat. Sci. 46, 309–327. Fischer, W., and Koch, E. (1987). On 3-periodic minimal surfaces, Z. Kristallogr. 179, 31–52. Fischer, W., and Koch, E. (1989). Genera of minimal balance surfaces, Acta. Cryst. A45, 726–732. Fontell, K. (1990). Cubic phases in surfactant and surfactant-like lipid systems, Colloid. Polym. Sci. 268, 264–285. Gandy, P. J. F., Bardhan, S., Mackay, A. L., and Klinowski, J. (2001). Nodal surface approximations to the P, D, G and I-WP triply periodic minimal surfaces, Chem. Phys. Lett. 336, 187–195. Gast, A. P., and Russel, W. B. (1998). Simple ordering in complex fluids, Colloidal particles suspended in solution provide intriguing models for studying phase transitions, Phys. Today 51, 24–30. Giardina, C. R., and Dougherty, E. R. (1988). Morphological Methods in Image and Signal Processing. New Jersey: Prentice Hall. Gibson, L., and Ashby, M. F. (1997). Cellular solids, 2nd ed. Cambridge: Cambridge Univ. Press. Gonzalez, R. C., and Woods, R. E. (1993). Digital Image Processing. Reading, MA: AddisonWesley. Grosse-Brauckmann, K. (1997). On Gyroid Interfaces. J. Colloid. Interface Sci. 187, 418–428. Hadwiger, H. (1957). Vorlesungen u¨ ber Inhalt, Oberfl¨ache und Isoperimetrie. Berlin: Springer Verlag. Hajduk, D. A., Harper, P. E., Gruner, S. M., Honeker, C. C., Kim, G., Thomas, E. L., and Fetters, L. J. (1994). The gyroid: A new equilibrium morphology in weakly segregated diblock copolymers, Macromolecules 27, 4063–4075. Hajduk, D. A., Harper, P. E., Gruner, S. M., Honeker, C. C., Thomas, E. L., and Fetters, L. J. (1995). A reevaluation of bicontinuous cubic phases in starblock copolymers, Macromolecules 28, 2570–2573. Hamersky, M. W., Hillmeyer, M. A., Tirrell, M., Bates, F. S., Lodge, T. P., and von Meerwall, E. D. (1998). Block copolymer self-diffusion in the gyroid and cylinder morphologies, Macromolecules 31, 5363–5370. Hasegawa, H., Tanaka, H., Yamasaki, K., and Hashimoto, T. (1987). Bicontinuous microdomain morphology of block copolymers, 1. Tetrapod-network structure of polystyrenepolyisopropene diblock polymers. Macromolecules 20, 1651–1662. Herminghaus, S., Jacobs, K., Mecke, K., Bischof, J., Fery, A., Ibn-Elhaj, M., and Schlagowski, S. (1998). Spinodal dewetting in liquid crystal and liquid metal films, Science 282, 916–919.
192
MICHIELSEN ET AL.
Hirth, J. P., and Lothe, J. (1968). Theory of dislocations. New York: Mc Graw Hill. Holyst, R., Plewczy´nski, D., Aksimentiev, A., and Burdzy, K. (1999). Diffusion on curved, periodic surfaces, Phys. Rev. E60, 302–307. Hyde, S. T. (1989). The topology and geometry of infinite periodic surfaces, Z. Kristallogr. 187, 165–185. Hyde, S., Andersson, S., Larsson, K., Blum, Z., Landh, T., Lidin, S., and Ninham, B. W. (1997). The Language of Shape; The Role of Curvature in Condensed Matter: Physics, Chemistry and Biology. Amsterdam: Elsevier. Jacobs, K., Herminghaus, S., and Mecke, K. R. (1998). Thin liquid polymer films rupture via defects, Langmuir 14, 965–969. Jinnai, H., Hashimoto, T., Lee, D., and Chen, S.-H. (1997). Morphological characterization of bicontinuous phase-separated polymer blends and one-phase microemulsions, Macromolecules 30, 130–136. Jinnai, H., Kajihara, T., Watashiba, H., Nishikawa, Y., and Spontak, R. J. (2001). Interfacial and topological measurements of bicontinuous polymer morphologies, Phys. Rev. E64, 010803. Kellerer, H. G. (1984). Minkowski Functionals of Poisson Processes, Z. Wahrscheinlichkeitstheorie verw. Gebiete 67, 63–84. Klain, D. A. (1997). Kinematic formulas for finite lattices, Ann. Combinatorics 1, 353–366. Klain, D. A. (1998). Kinematic formulas for finite vector spaces, Discr. Math. 179, 121–132. Klain, D. A., and Rota, G.-C. (1997). Introduction to Geometric Probability. Cambridge: Cambridge Univ. Press. Klinowski, J., Mackay, A. L., and Terrones, H. (1996). Curved surfaces in chemical structure, Phil. Trans. R. Soc. Lond. A354, 1975–1987. Koch, E., and Fischer, W. (1988). On 3-periodic minimal surfaces with non-cubic symmetry, Z. Krystallogr. 183, 129–152. Kole, J. S., Michielsen, K., and Raedt, H. De (2001). Morphological image analysis of quantum motion in billiards, Phys. Rev. E63, 016201. Lardner, R. W. (1974). Mathematical Theory of Dislocations and Fracture. Toronto: Toronto Univ. Press. Larsson, K. (1983). Two cubic phases in monoolein-water system, Nature 304, 664–664. Larsson, K. (1989). Cubic lipid–water phases: Structures and biomembrane aspects, J. Phys. Chem. 93, 7304–7314. Leibfried, G. (1951). Verteilung von Versetzungen im statischen Gleichgewicht, Z. Phys. 130, 214–226. Likos, C. N., Mecke, K. R., and Wagner, H. (1995). Statistical morphology of random interfaces in microemulsions, J. Chem. Phys. 102, 9350–9361. Longley, W., and McIntosh, T. J. (1983). A bicontinuous tetrahedral structure in a liquidcrystalline lipid, Nature 304, 612–614. Luzatti, V., Vargas, R., Mariani, P., Gulik, A., and Delacroix, H. (1993). Cubic phases of lipidcontaining systems. Elements of a theory and biological connotations, J. Mol. Biol. 229, 540–551. Luzzati, V., Tardieu, A., Gulik-Krzywicki, T., Rivas, E., and Reiss-Husson, F. (1968). Structure of the cubic phases of lipid–water systems, Nature 220, 485–488. Mackay, A. L. (1985). Periodic minimal surfaces, Nature 314, 604–606. Mackay, A. L. (1988). Equipotential surfaces in periodic charge distributions, Angew. Chem. 27, 849–850. Mackay, A. L. (1993). Crystallographic surfaces, Proc. R. Soc. Lond. A442, 47–59. Makarenko, N. G., Karimova, L. M., Terekhov, A. G., and Kardashev, A. V. (2000). Minkowksi functionals and comparison of discrete samples in seismology, Phys. Solid Earth 36, 305–309. ,
ASPECTS OF MATHEMATICAL MORPHOLOGY
193
Mariani, P., Luzzati, V., and Delacroix, H. (1999). Cubic phases of lipid-containing systems, J. Mol. Biol. 204, 165–189. Matheron, G. (1975). Random Sets and Integral Geometry. New York: John Wiley and Sons. Matsushita, Y., Tamura, M., and Noda, I. (1994). Tricontinuous double-diamond structure formed by a styrene-isopropene-2-vinylpyridine triblock copolymer, Macromolecules 27, 3680–3682. Mecke, K. R. (1996a). A morphological model for complex fluids, J. Phys. Condens. Matter 8, 9663–9667. Mecke, K. R. (1996b). Morphological characterization of patterns in reaction-diffusion systems, Phys. Rev. E53, 4794–4800. Mecke, K. R. (1998a). Morphological thermodynamics of composite media, Fluid Phase Equilibria 150–151, 591–598. Mecke, K. R. (1998b). Integral geometry in statistical physics, Int. J. Mod. Phys. B12, 861–899. Mecke, K. R. (2000). In: Statistical Physics and Spatial Statistiscs, edited by K. R. Mecke and D. Stoyan. Lecture Notes in Physics 554. Berlin: Springer. Mecke, K. R., Buchert, T., and Wagner, H. (1994). Robust morphological measures for large-scale structure in the Universe, Astron. Astrophys. 288, 697–704. Mecke, K. R., and Sofonea, V. (1997). Morphology of spinodal decomposition, Phys. Rev. E56, R3761–R3764. Mecke, K. R., and Wagner, H. (1991). Euler characteristic and related measures for random geometric sets, J. Stat. Phys. 64, 843–850. Mellot, A. L. (1990). The topology of large-scale structure in the universe, Phys. Rep. 193, 1–39. Michielsen, K., and De Raedt, H. (2001). Integral-Geometry Morphological Image Analysis, Phys. Rep. 347, 461–538. Michielsen, K., De Raedt, H., and Fraaije, J. G. E. M. (2000). Morphological characterization of spatial patterns, Prog. Theor. Phys. Supp. 138, 543–548. Mogi, Y., Mori, K., Matsushita, Y., and Noda, I. (1992). Tricontinuous Morphology of Triblock Copolymers of the ABC Type. Macromolecules 25, 5412–5415. Neovius, E. R. (1883). Bestimmung zweier spezieller periodischer Minimalfl¨achen. Helsingfors: Akad. Abhandlungen. Nishikawa, Y., Jinnai, H., Koga, T., Hashimoto, T., and Hyde, S. T. (1998). Measurements of interfacial curvatures of bicontinuous structure from three-dimensional digital images, 1. A parallel surface method, Langmuir 14, 1242–1249. Nissen, H.-U. (1969). Crystal orientation and plate structure in echinoid skeletal units, Science 166, 1150–1152. Plewczy´nski, D., and Holyst, R. (2000). Reorientational angle distribution and diffusion coefficient for nodal and cylindrical surfaces, J. Chem. Phys. 113, 9920–9929. Plewczy´nski, D., and Holyst, R. (2001). Approach to equilibrium of particles diffusing on curved surfaces, Physica A295, 371–378. Roaf, D. J. (1963). The Fermi surfaces of Cu, Ag, Au, Phil. Trans. R. Soc. Lond. A255, 135–152. Roos, A., De Hosson, J. Th. M., and Giessen, E. van der (2001a). A two-dimensional computational methodology for high-speed dislocations in high strain-rate deformation, Comput. Mat. Sci. 20, 1–18. Roos, A., De Hosson, J. Th. M., and van der Giessen, E. (2001b). High-speed dislocations in high strain rate deformations, Comput. Mat. Sci. 20, 19–27. Rosenfeld, A., and Kak, A. C. (1982). Digital Picture Processing. New York: Academic Press. Russ, J. C. (1995). The Image Processing Handbook. Boca Raton, FL: CRC Press. Sakurai, S., Irie, H., Umeda, H., Nomura, S., Lee, H. H., and Kim, J. K. (1998). Gyroid structures and morphological control in binary blends of polystyrene-block-polyisopropene diblock copolymers, Macromolecules 31, 336–343. ,
194
MICHIELSEN ET AL.
Santal´o, L. A. (1976). Integral Geometry and Geometric Probability. Reading, MA: AddisonWesley. von Schnering, H. G., and Nesper, R. (1987). How nature adapts chemical structures to curved surfaces, Angew. Chem. 26, 1059–1080. von Schnering, H. G., and Nesper, R. (1991). Nodal surfaces of Fourier series: fundamental invariants of structured matter, Z. Phys. B83, 407–412. Schoen, A. H. (1970). Infinite Periodic Minimal Surfaces without Self-Intersections, NASA Technical NOTE No. D-5541. Schulz, M. F., Bates, F. S., Almdal, K., and Mortensen, K. (1994). Epitaxial Relationship for Hexagonal-to-Cubic Phase transition in a Block Copolymer Mixture, Phys. Rev. Lett. 73, 86–89. Schwarz, H. A. (1890). Gesammelte Mathematische Abhandlungen. Berlin: Springer. Schwarz, U. S., and Gompper, G. (1999). Systematic approach to bicontinuous cubic phases in ternary amphiphilic systems, Phys. Rev. E59, 5528–5541. Serra, J. (1982). Image Analysis and Mathemathical Morphology. London: Academic Press. Sofonea, V., and Mecke, K. R. (1999). Morphological characterization of spinodal decomposition kinetics, Eur. Phys. J. B8, 99–112. Stoyan, D., Kendall, W. S., and Mecke, J. (1989). Stochastic Geometry and its Applications. Berlin: Akademie Verlag. Terol-Villalobos, Ivan R. (2001). Morphological Image Enhancement and Segmentation, Adv. Imaging Electron Phys. 118, 208–273. Thomas, E. L., Alward, D. B., Kinning, D. J., Martin, D. C., Handlin Jr., D. L., and Fetters, L. J. (1986). Ordered bicontinuous double-diamond structure of star block copolymers: A new equilibrium microdomain morphology, Macromolecules 19, 2197–2202. Thomas, E. L., Anderson, D. M., Henkee, C. S., and Hoffman, D. (1988). Periodic area-minimizing surfaces in block copolymers, Nature 334, 598–601. Vigild, M. E., Almdal, K., Mortensen, K., Hamley, I. W., Fairclough, J. P. A., and Ryan, A. J. (1998). Transformations to and from the gyroid phase in a diblock copolymer, Macromolecules 31, 5702–5716. Voss, K. (1992). Discrete Images, Objects and Functions in Z n , Algorithms and Combinatorics 11. Berlin: Springer. Weertman, J. (1996). Dislocation Based Fracture Mechanics. Singapore: World Scientific. Xie, R., Yang, B., and Jiang, B. (1993). The ordered bicontinuous double-diamond morphology in triblock copolymer homopolymer blends, Macromolecules 26, 7097–7099.
ADVANCES IN IMAGING AND ELECTRON PHYSICS, VOL. 125
Ultrafast Scanning Tunneling Microscopy G. M. STEEVES1 AND M. R. FREEMAN2 1
California NanoSystems Institute, University of California, Santa Barbara, California 93106 2 Department of Physics, University of Alberta, Edmonton, AB T6G 2J1 Canada
I. II. III. IV. V. VI. VII.
Introduction . . . . . . . . . . . . History . . . . . . . . . . . . . . Ultrafast Scanning Probe Microscopy . . Junction Mixing STM . . . . . . . . Distance Modulated STM . . . . . . . Photo-Gated STM . . . . . . . . . . Ultrafast STM by Direct Optical Coupling to the Tunnel Junction . . . . . . . . VIII. Conclusions . . . . . . . . . . . . References . . . . . . . . . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
195 196 199 205 216 218
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
225 228 228
I. Introduction Advances in nanofabrication have allowed the creation of novel devices which operate in a unique regime where both classical and quantum theory govern their behavior. These developments have stimulated a wide interest in all aspects of nanoscale physical phenomena. To facilitate studies on nanoscale systems, efforts have been devoted to forging new techniques in microscopy. The scanning tunneling microscope (STM), forefather of all scanning probe microscopies, was a result of such efforts (Binnig and Rohrer, 1987). STMs have been used to image surface topography with atomic resolution (Binnig and Rohrer, 1982). STM has been used to image the local densities of electron states on superconducting samples using scanning tunneling spectroscopy (Hess et al., 1989; Hudson et al., 2001). Recently, magnetic imaging capabilities have been added to the arsenal of powerful STM modalities through ballistic electron magnetic microscopy (Rippard and Buhrman, 1999) and STM assisted electron spin resonance (Durkan and Welland, 2002). While STM is renowned for its imaging ability, this ability has been limited to the study of static systems; however, with the incorporation of ultrafast optical techniques, a new probe has emerged: an aggregate probe allowing ultrafast dynamic imaging and atomic resolution (Nunes and Freeman, 1993). There are many potential uses for such a probe. Electron transport could be studied in high-speed integrated circuits, single electron transistors, Coulomb 195 Copyright 2002, Elsevier Science (USA). All rights reserved. ISSN 1076-5670/02 $35.00
196
STEEVES AND FREEMAN
blockade structures, carbon nanowires, or DNA. Electronic and magnetic transitions could be observed, and it may even be possible to resolve the question of whether electronic or structural reordering occurs first in a phase transition. The main limitation on the speed of a conventional STM is caused by the relatively low bandwidth of STM electronics. STM electronics have been designed to measure very small currents typically in the range of pico to nanoamps. To accurately measure these small currents, STM electronics operate at low bandwidths to maximize signal to noise. STM scan rates are determined by the mechanical response of an STM’s scan mechanism (typically kHz bandwidths) which also relates to STM electronic response characteristics. This response is adequate for static STM imaging but insufficient for ultrafast STM imaging. Efforts to speed up STM temporal response have achieved feedback response times in the microseconds (Mamin et al., 1994). The ultimate response of a real-time STM measurement is governed by probabilistic tunneling rates; for typical tunneling currents this allows measurements at MHz frequencies as explored by Ochmann et al. (1999). To surpass these limitations, it is necessary to switch to repetitive measurements. Using repetitive measurements allows the average dynamic response to be measured with far greater signal to noise than any single shot measurement. Repetitive measurements are easily facilitated by the availability of pulsed laser systems. Commercial Ti/Sapph laser systems, for example, can produce a train of 60-fs pulses at 76 MHz. A single pulse can be split with a beam splitter to produce two synchronous pulses for pump/probe experiments. Subsequent pulses repeatedly pump and probe the system allowing an average response to be determined. With repetitive pump/probe modulations of an STM’s tunneling current, the averaged dynamic tunneling current can be measured. The component of tunneling current from the dynamic pump/probe modulation can easily be separated from DC tunneling current by switching on and off the pump and probe beams using an optical chopper at a slow (kHz) rate within the bandwidth of normal current preamplifiers. By varying the relative pump/probe delay time, a convolved dynamic response can be measured, which can reveal the dynamics of the underlying system. This the the premise for ultrafast STM.
II. History With this approach in mind, we now consider the type of modulations which will allow stroboscopic ultrafast STM. The first approach suggesting pump/ probe techniques was disclosed anonymously in 1989 (Anonymous, 1989). Entitled “Contactless, High-Speed Electronic Sampling Probe,” the disclosure entails using optical pulses, coupled to a vacuum tunneling probe to sample fast
ULTRAFAST SCANNING TUNNELING MICROSCOPY
197
electrical transients propagating in an integrated circuit. An optical pulse incident on the metal–vacuum–metal tunnel junction is rectified by the nonlinear current–voltage characteristic of the tunnel junction [nonlinear optical rectification was investigated in metal–barrier–metal tunnel junctions by Faris, Gustafson et al. (Faris et al., 1973; Faris and Gustafson, 1974; Fan et al., 1977) as early as 1973, well before the invention of a STM]. An electrical transient in the test circuit, coincident with the optical pulse will modify the tunnel junction response allowing cross correlation measurements between the electrical and optical pulses. Using the current–voltage rectification of an optical pulse, the authors state that this technique should allow better than 5 fs temporal resolution. They also suggest that the tunneling probe could be scanned micrometer distances across the device surface to look for inhomogeneities and defects. They do not speculate on the ultimate spatial resolution of this technique. Direct optical coupling was later explored by Feldstein et al. (1966) and Gerstner et al. (2000), as will be discussed in a subsequent section. Due to the relative obscurity of the aforementioned proposal, nothing was known of time-resolved STM until the work of Hamers and Cahill (1990, 1991). Here Hamers and Cahill successfully demonstrated the use of an STM tip to detect an ultrafast signal. The signal was generated using pump/probe laser pulses incident on a 7 × 7 reconstructed Si (111) surface. The laser pulses induced a surface photovoltage, which was detected capacitively using the STM tip. Capacitive detection was chosen over tunneling current detection as signal to noise was significantly enhanced. Tunneling current detection was proposed to achieve STM spatial resolution in an ultrafast pump/probe measurement; this had yet to be demonstrated. In 1993, four groups demonstrated ultrafast signal detection in which the ultrafast signal was extracted from the tunneling current of an STM. Within an 11-day period, the techniques of photogated STM (PG-STM), distance modulated STM (DM-STM), and junction mixing STM (JM-STM) were introduced. The essential feature of the PG-STM technique, developed by Weiss et al., (1993) is a photoconductive switch in series with a tunneling STM tip. DC tunneling with an unilluminated tip is still possible in this configuration since the dark resistance of the switch is typically comparable to tunnel junction impedances. If a PG-STM is set to tunnel into a dynamically evolving sample, the switch on the STM tip can be briefly opened to gate the instantaneous state of the sample. The STM will only be sensitive to dynamics which are manifest in one of its usual acquisition modes such as topography or differential conductivity. Photogating acts in much the same way that a photographer’s flash captures instantaneous motion. Extremely fast 900-fs time resolution has been demonstrated with PG-STM (Botkin et al., 1996) using silicon-onsapphire switches implanted with O 2+ which possesses extremely fast carrier
198
STEEVES AND FREEMAN
recombination times. Unfortunately, though conceptually simple, this technique was found to be limited in spatial resolution. This limitation is directly related to the geometric capacitance from the segment of the STM tip between photoswitch and sample. Geometric capacitance is thought to hamper spatial resolution in PG-STM to the tens of nanometers. Distance modulated STM, developed by Freeman and Nunes (1993), provides an alternate means for detecting ultrafast sample dynamics. Here the tip sample separation is modulated on an ultrafast time scale to enhance the transient current contribution from the sample. In this implementation, the timeresolved tunneling current originates at the tunnel junction so there should be no loss in spatial resolution, though this has yet to be demonstrated. The only drawback to this technique is the need to mechanically alter the elevation of the STM tip high above a sample on fast time scales. This modulation has been demonstrated on nanosecond time scales using a magnetostrictive STM tip driven by a current coil. There the response is limited by the sound propagation time across the excited volume of the tip. Using an electronic pulsed current source allows a greater degree of flexibility in the pulse amplitude and repetition rate, over optically generated pulses. Picosecond and femtosecond speeds have yet to be shown but are probably needed, to compel adoption of this technique. Photoacoustic pulse generation seems a feasible route to this goal. To overcome the speed limitations of the distance modulated technique Nunes and Freeman developed junction mixing STM. This technique relied on a fast electrical pulse which is launched into the STM tunnel junction. The inherent electrical non-linearities of a tunnel junction allow the pulse to mix with the electronic state of the sample. If the electronic state of the sample is dynamical, then variations in the arrival time of the input pulse will evoke a series of responses. By deconvolving this signal with the shape of the electrical pulse, the evolution of the electronic state of the sample can be determined. It is worth noting that this technique should be capable of imaging any dynamical process which affects the tunneling current of an STM. To this date the junction mixing technique has been used to demonstrate combined 20 ps, 1 nm spatio-temporal resolution. The ultimate spatial resolution of JMSTM is thought to correspond to that of an ordinary STM and calculations suggest that sub-picosecond temporal resolution can be achieved. To explore the boundaries of ultrafast STM time resolution, even the fastest electronic pulses are too slow. If ultrafast STM is to approach temporal resolution of tens of femtoseconds then optical pulses must be directly coupled to the STM tunnel junction. For this idea to be demonstrated a number of experimental challenges must be met, but direct junction coupling stands as the grail in ultrafast STM.
ULTRAFAST SCANNING TUNNELING MICROSCOPY
199
Figure 1. Schematic diagram of experimental apparatus. ÷ N = Cavity dumper, EOM = electrooptic modulator, PP = polarizing prism. (Reprinted with permission from R. J. Hamers and David G. Cahill, 1991. J. Vac. Sci. Technol. B, 9, 514, American Vacuum Society.)
III. Ultrafast Scanning Probe Microscopy Shortly after the 1989 anonymous disclosure was published, a more widely cited idea for ultrafast STM was put forward by Hamers and Cahill (1990, 1991). The 1990 paper described the first experimental attempt to combine laser methods for ultrafast time resolution with a scanning probe microscope (an STM tip was used outside the tunneling regime to detect a change in capacitance between tip and sample). In addition to this experimental demonstration the authors outlined a technique and requirements to add ultrafast time resolution to other types of scanning probe microscopes (their original motivation had been to attempt a time-resolved detection of surface photovoltage using a scanning tunneling microscope, but signal to noise requirements mandated the scanning capacitance scheme). The process that the authors studied was the decay of photoexcited carriers at a Si (111) (7 × 7) surface. Their experimental setup is shown in Figure 1. In this scheme, they proposed that “Using the SXM∗ probe tip as a local detector of a deviation from equilibrium which is a nonlinear function of an externally controlled stimulus (such as an optical pulse), it is possible to achieve unprecedented time resolution which is limited only by the inherent time scale of the underlying physical process.” ∗
Where SXM refers to a generic scanning probe microscopy.
200
STEEVES AND FREEMAN
In their experiment a scanning capacitance microscope was used as the local probe, and optical pulses from a dye laser were used to create a transient surface photovoltage (SPV) which was detected by the probe. Using a capacitive probe allows the tip sample separation to be large minimizing the effects of photothermal expansion of sample and tip due to laser intensity fluctuations. Under continuous illumination the magnitude of the SPV varies as: SPV ≈ A ln(1 + Cδ Q/Q)
(1)
where A and C are constants and δ Q is the photoexcited carrier density (Hamers and Markert, 1990). Under pulsed illumination using 1-ps pulses every 13.1 ns, the SPV can be fit with this same equation (with modified values of A and C) as shown in Figure 2. The fitting curve was generated with A = 28 mV which is almost identical to the value A = 31 mV found under continuous illumination, suggesting that carrier relaxation occurred on time scales slower than 13.1 ns. To probe carrier relaxation on longer time scales, a cavity dumper was used to vary the repetition rate of laser pulses at the sample surface. One picosecond duration excitation pulses would arrive with a spacing (in time) in increments of τML = 13.1 ns. The authors measured the average photovoltage as they varied the repetition rate of their excitation pulses from τML to 40 ×τML while
Figure 2. Dependence of SPV on incident power using 1-ps pulses separated by 13.1 ns. (Reprinted with permission from R. J. Hamers and David G. Cahill, 1991. J. Vac. Sci. Technol. B, 9, 514, American Vacuum Society.)
ULTRAFAST SCANNING TUNNELING MICROSCOPY
201
maintaining a constant time-averaged illumination intensity. In this way, they could look for deviations from Eq (1). When the cavity dumper repetition rate was high with respect to the carrier relaxation time, a large carrier density would be established producing a saturated SPV and measured displacement current signal (through repeated pumping of the sample). For a long repetition rate (compared to the recombination time), most photocarriers recombined, diminishing the SPV before the next excitation pulse, resulting in a diminished current signal. The excitation pulse train was optically chopped at 4 kHz so that a lock-in amplifier could be used for detecting and discriminating the time-resolved signal. An averaging transient recorder measured the displacement current induced in the tip at 4 kHz. Integrating the displacement current over the “on” time of the chopper gave an averaged value proportional to the time-averaged SPV. A SPV plot with respect to the pulse separation time is shown in Figure 3. This curve was fit and a 1-μs carrier decay time was deduced in the limit of small photovoltages. This experiment was the first to use a scanning probe to detect a signal with fast transient origins. The authors did not report on the effect of scanning the detector over the silicon surface, so combined spatiotemporal resolution was not demonstrated. The technique used to achieve time resolution in this case relied solely on properties of the sample under investigation. The saturation of the SPV at different pump powers and pump repetition rates allowed the authors to deduce the carrier recombination time for the Si (111)
Figure 3. Time dependence of surface photovoltage, obtained by measuring the displacement current induced in the tip as a function of delay time between optical pulses, at constant average illumination intensity. Individual points were measured in random order. (Reprinted with permission from R. J. Hamers and David G. Cahill, 1991. J. Vac. Sci. Technol. B, 9, 514, American Vacuum Society.)
202
STEEVES AND FREEMAN
surface, but this technique is not generalizable for time-resolved studies of other material properties. Nonetheless, the authors demonstrated the first use of a local probe to investigate the 1-μs photoexcited carrier recombination time of a Si (111)-(7 × 7) surface. Two years after the work by Hamers and Cahill, the group of Hou, Ho, and Bloom (1992) published results demonstrating picosecond time resolution using a scanning electrostatic force microscope (SFM). Atomic force microscopes (AFM) had only been invented about 6 years earlier by Binnig, Quate, and Gerber (1986) and were already acquiring a great number of industrial applications due to their flexibility and ease of use. Adding ultrafast time resolution to a modified AFM would prove to be not only interesting, but quite2 practical. In the Hou et al. (1992) scheme, the nonlinear force F = − ǫ02zAV2 between SFM tip and the sample would serve to mix the excitation response of the system with a time-varying probe signal. This differed from the Hamers and Cahill experiment where they used repetitive pumping at differing time intervals to build a measurable signal (there was only a pumping stimulus and the frequency of this stimulus was varied). In the Hou et al. (1992) configuration, two different signals, a pump and a probe signal at different frequencies, were mixed in the force nonlinearity to produce a difference frequency signal within the bandwidth of the SFM cantilever and electronics. In demonstrating this technique, the authors combined the 1-V outputs of two synthesized sinusoidal signals at frequencies f and f + f , and launched the combined signal down a transmission line (TRL). The fundamental frequency f = 1 GHz was used and f was varied from 0 → 25 kHz. The deflection amplitude of the cantilever is shown in Figure 4. The 19-kHz peak corresponds to the mechanical resonance frequency of the cantilever. By characterizing the mechanical resonances of the SFM tip, the mixing signal can be obtained. Similar mixing experiments were successful up to 20 GHz, limited by the quality of the experimenters’ high-speed electronics. This group also performed experiments in the time domain using a pair of step-recovery-diode (SRD) comb generators, which produced 110-ps pulses at 500 MHz and 500 MHz + 10 Hz, respectively. These pulse trains were combined and launched onto a coplanar waveguide structure patterned onto a GaAs substrate. The beat pattern from the two pulse trains is shown in Figure 5, where a 130-ps correlation between pulse trains is shown. In their concluding remarks, the authors contend that this technique should be capable of measuring voltage signals with picosecond time resolution and submicron lateral resolution. This technique should be quite useful especially when excitation and probe pulses are close enough in frequency that the mechanical cantilever will respond with sufficient amplitude to be detected. One questions the ultimate resolution of this technique, as it relies on a tip sample force which is capacitive in origin, yet the technique should provide the submicron resolution the authors suppose.
Figure 4. SFM electrical mixing at 1 GHz. (Reprinted with permission from A. S. Hou, C 1992 IEEE.) F. Ho, and D. M. Bloom, 1992. Electron. Lett. 28, 2302.
Figure 5. Equivalent-time correlation trace of two 110-ps pulse trains. (Reprinted with C 1992 permission from A. S. Hou, F. Ho, and D. M. Bloom, 1992. Electron. Lett. 28, 2302. IEEE.)
204
STEEVES AND FREEMAN
The absolute magnitude of the mixed signal is a convolution of the force exerted on the cantilever, due to the voltage pulses being measured, and the mechanical response of the cantilever. This complication was later addressed using a nulling method developed by Bridges et al. (1993) in their time-resolved AFM system. In the work by Bridges et al., the electrostatic force between a flexible tip and sample is monitored and used to demodulate an ultrafast signal. The sample under investigation is repetitively pumped and probed, and by cleverly modulating the amplitude of the repetitive probe signal while adding a variable AC dither, the sample response can be accurately mapped, irrespective of the cantilever displacement. The force between cantilever tip and sample can be written as: FZ = −
1 ∂ C(x, y, z)[v p (t) − vc (x, y, t)]2 2 ∂z
(2)
where vc is the voltage on the circuit element being tested, v p is the voltage on the scanned probe, and C(x, y, z) is dependent on the probe tip/circuit geometry and position. To measure vc one could use a series of probe pulses similar to the technique of Hou et al. (1992), calibrating the response of the cantilever would still be a problem though. The trick that Bridges et al. (1993) used was to use a modulated pulse train to probe the circuit voltage. The initial sampling voltage train was given as vs (t) = G δ (t − τ ) (where Gδ(t − τ ) is a step function at time τ with width δ), but this was modified, giving v p (t) = [A + K cos(ωr t)]vs (t), where wr is the resonant frequency of the cantilever. In the previous equation, the variables A, K , and vs (t) are all user controlled parameters. Using v p (t) in Eq. (2) gives a number of terms in the force equation. Most of these terms will appear as DC contributions, but there will be a term at the modulation frequency ωr , which a lock-in amplifier can detect. This term is expressed as ∂ C(x, y, z) × [Avs (t), vs (t) − vs (t), vc (x, y, t)]K cos(ωr t) ∂z (3) ( where a, b = T1 T a(t)b(t)dt is the inner product over the period T. Using vs (t) in Eq. (3) and evaluating the inner product vs (t), vs (t) noting that δ is the width of the impulse G δ (t − τ ), we rewrite Fz as " # δ ∂ Fz |ω≃ωr = − C(x, y, z) × A · − G δ (t − τ ), vc (x, y, t) K cos(ωr t) ∂z T (4) Under the condition that the width of the impulse approaches zero, we note that G δ (t − τ ), vc (x, y, t) ⇒ vc (x, y, t = τ ) · Tδ . Recalling that vc (x, y, t) is the function we are trying to determine, we see that by adjusting the modulating Fz |ω≃ωr = −
ULTRAFAST SCANNING TUNNELING MICROSCOPY
205
parameter A to null the force Fz , one can easily determine vc . This approach only works when the response of the detection system is very slow compared to the speed of the sampling pulses. The work by Hou et al. (1992) was quite illustrative for early proposals for time-resolving STM operation, especially the junction mixing techniques. In the Bloom work, a pump/probe configuration was used, where pump and probe signals were mixed in the squared voltage nonlinearity exerting a force on the AFM tip. The technique of Greg Bridges et al. (1993) employed this same nonlinearity, but by using a single excitation source for pump and probe pulses, the electronics required were greatly simplified. The nulling procedure overcomes the problems of calibrating cantilever response and allows the determination of the response of a sample to electrical excitation. These techniques offer fast time resolution, demonstrated into the picosecond range, but the spatial resolution of this technique will be limited by the range of the geometric capacitance between tip and sample. For time-resolved microscopies where atomic-scale spatial resolution is necessary, one must look past other scanning probe microscopes to the grandfather of the field, the STM.
IV. Junction Mixing STM The most successful technique in ultrafast STM has been junction mixing. In this technique, a sample is repetitively stimulated (or pumped) where this stimulus will, in some way, affect the tunneling current of the STM. The effect could be to change the tunneling I/V characteristic, through some local surface modification, or to change the bias voltage within the original I/V characteristic. I/V characteristics for metal insulator metal tunneling are typically given by a form of the Simmons equation (1963) I = β(V + γ V 3 ) + O(V 5 ),
(5)
showing a cubic nonlinearity. The probe pulse, in the junction mixing technique, is launched from either the tip or the sample toward the tunnel junction, mixing with the pump pulse at the junction itself. This leads to a time-resolved current contribution which varies with the time delay between pump and probe pulses as they arrive at the tunnel junction. This technique was first demonstrated by Nunes and Freeman (1993). The experimental setup is shown in Figure 6. To distinguish the time-resolved current contribution, an optical chopper was used to chop pump and probe beams at different frequencies and a lock-in amplifier was used to detect at the sum frequency. As a proof of principle experiment, photoconductively generated electrical pulses were sent down a transmission line, with one pulse acting as pump and one as probe. The
206
STEEVES AND FREEMAN
Figure 6. Experimental layout for junction mixing STM.
pulses were generated by a single laser pulse train, split into a pump and probe pulse train. Each pulse train was optically chopped at a different frequency and a phase-sensitive detector was used to monitor the tunneling current at the sum of the two chopping frequencies. A variable delay between pump and probe was the result of an optical delay line. When the electrical pulses are not coincident at the tunnel junction, each pulse causes a slight increase in the tunneling current acquired by the STM electronics. The STM feedback electronics compensates for this increase by withdrawing the STM tip, but the effect is negligible since the duration of the pulses is four orders of magnitude smaller than the repetition rates, and pulse magnitude is comparable to the DC bias voltage. If the optical chopping frequencies are above the bandwidth of the STM feedback electronics, the STM tip position will not be effected by a change in tunneling current at this frequency. When pump and probe pulses are not coincident, there will be no time-resolved signal at the sum frequency as the feedback will have compensated for the additional current at each of the constituent frequencies. When the pulses are coincident, the nonlinearity of the tunnel junction will produce an excess tunneling current contribution as V (I1 + I2 ) > V (I1 ) + V (I2 ). Because this excess current is only manifest at the sum of the chopping frequencies, the feedback electronics will not be fast enough to compensate for it, allowing its detection. To use this scheme, one must ensure that the sum frequency falls within the bandwidth of the STM current preamplifier.
ULTRAFAST SCANNING TUNNELING MICROSCOPY
207
Figure 7. Time-resolved current using JM-STM, indicated by the solid line. Dashed line is a fit assuming an exponentially decaying optically launched voltage pulse. (Reprinted with C 1993 American permission from Geoff Nunes, Jr. and M. R. Freeman, 1993. Science 262, 1029. Association for the Advancement of Science.)
When these electrical pulses arrive simultaneously at the tunnel junction of the STM, the sum voltage leads to a nonlinear increase in tunneling current, which can be seen in Figure 7. Because the mixing between pump and probe pulse is occurring in the tunnel junction, this technique should maintain the spatial resolution of the STM. This method is noncontact and essentially nonperturbative, considering that the current leaking from the circuit under test is on the order of picoamps. There are limitations to the junction mixing technique; for the signal to “mix” in the tunnel junction, the stimulated sample must offer I/V contrast with the unstimulated sample. In the experiment by Nunes and Freeman (1993), both pump and probe pulses were sent down a transmission line to the tunnel junction. This is not a requirement of the technique. The sample can be pumped in any manner which will affect the tunneling current, and can be probed using a voltage pulse sent from the tip or sample into the tunnel junction. If the voltage pulse is to be sent down the tip into the tunnel junction, there must already be a fixed conduction channel between the junction and the STM current preamp which the pulse is coupled onto as shown if Figure 8. This eliminates the possibility that capacitive charging of the tip could lead to a time-resolved signal and is one of the essential differences
208
STEEVES AND FREEMAN
Figure 8. An integrated STM tip with photoconductive switch for JM-STM modeled after a similar design by Groeneveld et al. (1996). In this design there is a direct path from STM tip to the STM current electronics, and there is a photoconductive switch which can be biased with respect to the tip line, so that electrical pulses can be launched into the tunnel junction from the tip. Though fabricated, this tip design has not been tested.
between the junction mixing technique and the photogated STM (PG-STM) technique, which will be discussed later. In the initial junction mixing experiments of Nunes and Freeman (1993), the authors demonstrated the detection of a transient voltage pulse on a transmission line with 130 ps time resolution. When the STM tip was withdrawn from the tunneling regime, the time-resolved signal also diminished at the same rate as the quasi-DC tunneling current. This suggested that the tunnel junction was indeed the origin of the time-resolved signal. Further work by Steeves et al. (1997) confirmed the origin of the time-resolved signal by launching electrical pulses from either side of a transmission line into the tunnel junction. The detection of 10-ps electrical transients in this junction mixing experiment confirmed that the nonlinear mixing of pump and probe pulses was occurring at the tunnel junction since timing arguments ruled out mixing at any other location. Up to this point, it had been shown that the junction mixing STM technique was capable of detecting fast electrical transients traveling along a transmission line into which an STM was tunneling. Since the tunnel junction of the STM had been shown to be the origin of the time-resolved current, it was assumed that the junction mixing technique would preserve the excellent spatial resolution
ULTRAFAST SCANNING TUNNELING MICROSCOPY
209
Figure 9. I/V curves tunneling into titanium and gold surfaces. (Reprinted with permission from G. M. Steeves, A. Y. Elezzabi, and M. R. Freeman, 1998. Appl. Phys. Lett. 72, 504, American Institute of Physics.)
inherent in scanning tunneling microscopy. To confirm this fact and to begin measurements of the spatiotemporal resolution limitations of the junction mixing technique, another experiment was devised by Steeves et al. (1998). To clearly demonstrate combined spatiotemporal resolution, they designed a gold transmission line structure which had 3-μm titanium dots patterned onto it. The characteristic I/V curve tunneling into the gold surface contrasted sharply with that of the titanium as shown in Figure 9. A time-resolved signal originating because of the I/V nonlinearity of the tunnel junction should also show this contrast as an STM tip is scanned from one material to the other. The transmission line structures used in this experiment were made from 100-nm thick, 200-μm wide gold lines, deposited onto a H + ion-implanted (1 × 10−15 cm−2 ion dose at 200 keV) GaAs substrate. Titanium dots, 20 nm thick, with a 3-μm diameter were patterned onto regions of the gold transmission line using sputtering and lift-off. A schematic of the structure is shown here in Figure 10. With this structure, a time-resolved STM experiment was performed where initially a dot was found on the transmission line, and then synchronous pump/probe pulses were used to establish a time-resolved current. Once the time-resolved signal was detected through a lock-in amplifier, an STM image was taken of the titanium dot and its gold transmission line host. Acquire time for these images was quite lengthy, 21 min for a 64 × 64 pixel scan, as a long 300-ms lock-in time constant was necessary to detect the time-resolved tunneling signal. In this configuration, time-resolved and topographic images are acquired simultaneously by monitoring the STM feedback signal and the
210
STEEVES AND FREEMAN
Figure 10. Schematic showing the gold transmission line structure with titanium dots patterned on top. (Reprinted with permission from G. M. Steeves, A. Y. Elezzabi, and M. R. Freeman, 1998. Appl. Phys. Lett. 72, 504, American Institute of Physics.)
time-resolved signal at the chopping frequency as was done in the original Nunes and Freeman (1993) junction mixing experiment. By subsequently varying the relative timing between pump and probe pulses, the transit of an electrical pulse across the titanium dot can be recorded using ultrafast STM as shown in Figure 11 (from Steeves’ Ph.D. thesis, 2001). To analyze the combined spatiotemporal resolution of the microscope, junction mixing STM was performed on the edge of a titanium dot. As the STM tip passed from the gold transmission line to encounter the titanium surface, the time-resolved STM signal was carefully monitored. It was found that the time-resolved signal changed dramatically in amplitude just as the STM began tunneling into titanium, and the resolution of the titanium gold interface established an upper limit on the spatial resolution of junction mixing STM. The temporal resolution in this experiment was determined by the pulse width of the electrical pulses which were created on the gold transmission line, 20 ps. In this way, Steeves et al. (1998) demonstrated combined 20-nm 20-ps spatiotemporal resolution using junction mixing STM. Continuing this work with an ultrahigh vacuum Omicron STM, Khusnatdinov et al. (2000) were able to demonstrate a factor of 20 improvement in spatial resolution, showing combined spatiotemporal resolution of 1 nm and 20 ps. A line scan showing the topographic and time-resolved STM signals at the edge of a titanium dot is shown in Figure 12. Junction mixing STM has been shown to achieve the goals of ultrafast STM. Combined picosecond time-resolution and single nanometer spatial resolution
ULTRAFAST SCANNING TUNNELING MICROSCOPY
211
Figure 11. Time-resolved STM image scans, color contrast represents contrast in timeresolved current between titanium and gold. Each image corresponds to a pump/probe delay time as follows: A = 0 ps, B = 40 ps, C = 45 ps, D = 50 ps, E = 52.5 ps, F = 55 ps, G = 60 ps, H = 80 ps, I = 100 ps. (Reprinted with permission from G. M. Steeves’s Ph. D. thesis, Junction C 2001.) Mixing Scanning Tunneling Microscopy
are the result of a time-resolved signal originating at the STM tunnel junction. Ultimately subangstrom spatial resolution should be possible using this technique. The ultimate time-resolution of this technique has been investigated by Steeves, Elezzabi, Teshima, Said, and Freeman (1998) through modeling the JM-STM tunnel junction using a simple lumped parameter circuit model. The geometry of the model is shown in Figure 13. The circuit elements consist of a transmission line impedance Z , and the tunnel junction elements are a nonlinear resistance Rt (V ) and the geometrical tip/sample capacitance Ct . The
212
STEEVES AND FREEMAN
Figure 12. Topography and time-resolved current of the Au/Ti interface. Time-resolved current (open circles) uses left axis, topography (solid line) uses right axis. (Reprinted with permission from N. N. Khusnatdinov, T. J. Nagle, and G. Nunes, Jr., 2000. Appl. Phys. Lett. 77, 4434, American Institute of Physics.)
form of the nonlinear tunnel junction resistance is derived from the Simmons equation (1963) Rt (V ) =
1 β(1 + γ (V (t) − I (t)Z )2 )
(6)
Voltage pulses are expressed as the integrated charge produced by a convolution of the laser pulse shape competing with carrier relaxation processes: $ % $ % ) t−t0 τ − (t − t0 ) τ sech2 exp dτ (7) V (t − t0 ) = V0 tp tc −∞
Figure 13. Equivalent lumped element model of STM and transmission line. (Reprinted with permission from G. M. Steeves’s Ph.D. thesis, Junction Mixing Scanning Tunneling MiC 2001.) croscopy
ULTRAFAST SCANNING TUNNELING MICROSCOPY
213
The time t0 is adjusted for each voltage pulse to simulate the varying delay times between pump and probe optical pulses in junction mixing experiments. It is convenient to note that the integral can be expressed as a hypergeometrical function which in the case of t0 = 0, can be expressed as 2 1 1 sech2 (t/2) (8) × 2F 1 1, 2; 2 + ; −t tp e + 1 1 + (1/t p ) From these equations, tc is the electrooptic switch carrier recombination time and t p is the laser excitation pulse width. A value for the tip sample capacitance was still needed to begin calculations. Accurate modeling of STM tip sample capacitance was investigated by Kurokawa and Sakai (1997), but in this work a simple method of image calculation was used (Said, 1995). It is worth noting that on nanoscopic scales additional nonlinear capacitive effects may contribute additionally to the circuit model presented here. The low frequency admittance of a quantum point contact was derived by Christen and B¨uttiker (1996). Complementing this analysis, was the work of Wang et al. (1999) who examined the density of states induced nonlinear capacitance for a parallel plate system, and proposed a quantum scanning capacitance microscope. The finite element model used by Steeves et al. (1998) gave a nominal value for Ct = 33 fF. The equation for the time-resolved tunneling current was given as % $ 1 V (t) 1 d V (t) 1 d I (t) = − I (t) + + (9) dt Z dt ZCt Rt C t Rt Z C t where V (t) is the sum of both voltage pulses delivered by the two ultrafast photoconductive switches and different values of t0 . Solving for I (t) numerically using a fourth-order Runge–Kutta solver gave results which were compared with experimental data as shown in Figure 14. The panel on the left shows calculations using the parameters V0 = 0.49V, β = 5.1 nS, γ = 0.75V −2 , Z = 68, Ct = 33 fF, tc = 10 ps, and t p = 2.8 ps. All values used in calculation were measured experimentally with the exception of the amplitude V0 which was used to fit calculation to experiment. Parameters for the right-hand panel were quite different since the time-resolved current in this panel was acquired from a faster transmission line structure. Measured values of β = 23 nS, γ = 1.3V −2 , tc = 4.5 ps, and t p = 2.8 ps were used with a voltage pulse amplitude of V0 = 0.561V , impedance Z = 68, and capacitance Ct = 33 fF. For both panels, the amplitude of the calculated current pulse had to be scaled over the duty cycle of the laser to achieve proper fits with data. Good agreement is seen between experimental results and calculations, though there is some disagreement between the tails of the experimental pulse on the right
214
STEEVES AND FREEMAN
a
b
Figure 14. Time-resolved tunneling currents. Experimental data are indicated by dashed lines, model calculations are represented by solid lines. (a) Data for a 10 ps voltage pulse. (b) Data from a faster 4-ps pulse. (Reprinted with permission from G. M. Steeves, A. Y. Elezzabi, C 1998 R. Teshima, R. A. Said, and M. R. Freeman, 1998. IEEE J. Quantum Electron. 34, 1415. IEEE.)
panel. The authors attribute this to pulse dispersion and reflections in their transmission line structure. Investigating variations of tip sample capacitance on the time-resolved STM signal resulted in curves as displayed in Figure 15. A time-resolved signal was not affected by a varying capacitance for values of Ct < 1 fF. For values greater than 1 fF, the width and amplitude of the time-resolved signal is effected in a roughly linear manner as the signal broadens and diminishes for increasing values of capacitance, shown in the inset. Intuitively, this result seems reasonable; current from a fast voltage pulse encountering a resistor and capacitor in parallel will preferentially couple its high frequency components across the capacitor. This leads to a broader, smaller time-resolved tunneling signal when tip sample capacitance is large. For a fixed tip sample capacitance of 33 fF the speed of the junction mixing technique is then evaluated by examining the calculated time-resolved current as a function of transmission line pulse widths governed by tc . Figure 16 shows calculated time-resolved currents for carrier recombination times ranging from 1 ps down to 300 fs. The inset shows that for carrier recombination times under 1 ps; the correlation pulse width falls off slowly implying that femtosecond time-resolution will be difficult to achieve without reducing tip sample capacitance below 33 fF.
ULTRAFAST SCANNING TUNNELING MICROSCOPY
215
Figure 15. Tunneling junction current time-resolved signals for different tip-transmission line capacitances. Inset shows the width of the correlation pulses as the capacitance is varied. Model parameters are β = 23 nS, γ = 1.3 V−2, V0 = 1 V, tc = 1 ps, and t p = 200 fs. (Reprinted with permission from G. M. Steeves, A. Y. Elezzabi, R. Teshima, R. A. Said, and M. R. Freeman C 1998 IEEE.) (1998). IEEE J. Quantum Electron. 34, 1415.
Figure 16. Calculated time-resolved tunneling current for various electrical excitation pulses. Inset shows the width of the correlation pulses as the width of the voltage pulse along the transmission line is varied. Model parameters are β = 23 nS, γ = 1.3 V−2, V0 = 1 V, Ct = 33 fF, and t p = 200 fs. (Reprinted with permission from G. M. Steeves, A. Y. Elezzabi, R. Teshima, C 1998 IEEE.) R. A. Said, and M. R. Freeman (1998). IEEE J. Quantum Electron. 34, 1415.
216
STEEVES AND FREEMAN
V. Distance Modulated STM Around the same time that the junction mixing technique was first proposed, Freeman and Nunes (1993) reported a completely different technique for timeresolving an STM, the distance modulation technique. In this work, a sample is repetitively stimulated while the distance between tip and sample is varied on a nanosecond time scale. Recall that STM tunneling current varies exponentially with tip/sample separation. Varying this separation on a short time scale allows the gating (or fast sampling) of a signal. This is a technique to mechanically switch the tunneling conductance of the STM without the inherent capacitance problems associated with having an electrical switch on the STM tip. The experimental setup is shown in Figure 17 where the STM tip was electrochemically etched nickel wire (125 μm diameter). Nickel is magnetostrictive, changing its structure by constricting or elongating with applied magnetic field. A pulsed magnetic field can cause an acoustic pulse to be launched along the magnetostrictive nickel tip, modulating STM tip/sample separation on nanosecond time scales. When biased to the steepest part of its magnetization curve, the maximum reversible magnetostriction was ≈0.1 ppm/Oe. Using a static bias field of 30 Oe and applying a 80-ns, 20-Oe pulse resulted in ˚ (measured through time-resolved a maximum absolute tip displacement of 5 A fiber-optic interferometry). The direction of tip deflection can be changed by reversing the polarity of the field pulse. For the experiment described, the effective tunneling work function φ¯ = 0.45 eV gives a characteristic tunneling ˚ which is less than the 5 A ˚ displacement of the tip, range (2k ′ )−1 = 1.45 A
Figure 17. A schematic diagram of the distance modulated STM apparatus.
ULTRAFAST SCANNING TUNNELING MICROSCOPY
217
Figure 18. Measured time-resolved tunneling current obtained using the tunnel distance modulation technique to sample the presence of a voltage pulse on the transmission line sample. Data are shown for both directions of current flow through the magnetic field pulse coil, and are compared with a model calculation based on the data of Fig. 2 of Freeman and Nunes (1993). Note that the vertical scale for the lower curve has been expanded by a factor of 5. DC tunneling parameters used to stabilize the quiescent tip position are 1 nA, 50 mV. (Reprinted with permission from M. R. Freeman and Geoff Nunes, Jr., 1993. Appl. Phys. Lett. 63, 2633, American Institute of Physics.)
indicating that the acoustic pulse will have a very significant effect on the tunneling current signal over its duration. Using the distance modulation technique to sample an electrical pulse sent down a gold transmission line, as shown in Figure 17, gives the curves shown in Figure 18. The fact that the time-resolved signal reversed polarity as the pulse field polarity was reversed illustrated the fact that the time-resolved current contribution decreases as the tip is withdrawn. It does not represent a negative current, just a lowering of current relative to the average since the y-axis zero corresponds to a DC current of 1 nA. This apparent bipolar signal is evidence that tip magnetostriction is responsible for the time-resolved current observed. This technique is quite general in its applications. It poses few restrictions on the types of samples which can be studied. So long as the samples can be stimulated repetitively in a way that the STM can detect, distance modulated STM can be used. The primary hindrance of the technique is the inertia of the tip (limiting distance modulated STM to the nanosecond time domain).
218
STEEVES AND FREEMAN
A technique with “no moving parts” would enable superior time resolution. Motivated by this criterion, the junction mixing STM technique was developed.
VI. Photo-Gated STM 1993 was a banner year for experiments demonstrating the time-resolved operation of the scanning tunneling microscope. Four papers, published over a two-month span by three different groups, brought three competing methods to add ultrafast time resolution to STM operation. The third of the competing methods is photogated STM. In PG-STM (Weiss, Ogletree, et al., 1993), an optical pulse train is split into a pump/probe configuration, the pump beam repetitively stimulates the sample under investigation while the probe beam is directed to an optical switch embedded in series with the tip of the STM. In this work, the photoconductive switch had a finite “dark” resistance of 30 M (small compared to a typical STM tunnel junction resistance of 100 M or more) so that the STM operated without having to illuminate the switch. The idea with this configuration is to use the pump beam to stimulate the sample such that the tunneling current between tip and sample is modified. Without the probe beam, the additional current will be small, the electronics of the STM will integrate this additional current contribution, and the feedback system of the STM will compensate by adjusting the STM tip height. When the probe pulse is implemented, the current contribution from coincidence between pump excitation and probe gating will be enhanced. Under these circumstances, the STM electronics will again integrate the excess current and the STM feedback electronics will adjust the tip height accordingly. To avoid this, the pump and probe beams are optically chopped at different frequencies, where the chopped frequencies are outside the response time of the feedback system, but within the bandwidth of the STM current pre-amp. A lock-in amplifier is used to detect the excess current at the chopping sum or difference frequency. By shifting the phase between pump and probe pulse trains, a cross-correlation between the response of the sample (to the pump pulse), and the response of the photoconducting switch (to the probe pulse), can be created. If one characterizes the response of the photoconductive switch, the response of the sample can be extracted by deconvolution. The experimental setup used by Weiss, Ogletree, et al. (1993) is shown in Figure 19. The cross-correlated current measured by this device is shown in Figures 20 and 21. In Figure 20, each trace is at a different tunneling resistance. Figure 21 shows the time-resolved current measured through tunneling and across a tip in contact with the transmission line. In their results, Weiss, Ogletree, et al. (1993) noted that the time-resolved signal is proportional to the average tunnel currents, from which they concluded that there is no geometrical capacitive contribution to the time-resolved signal.
ULTRAFAST SCANNING TUNNELING MICROSCOPY
219
Figure 19. Photogated STM. One laser pulse excites a voltage pulse on a transmission line. The second pulse photoconductively samples the tunneling current on the tip assembly.
Figure 20. The tunneling current I (t) for different gap resistances (16, 64, and 256 M from top to bottom). (Reprinted with permission from S. Weiss, D. F. Ogletree, D. Botkin, M. Salmeron, and D. S. Chemla, 1993. Appl. Phys. Lett. 63, 2567, American Institute of Physics.)
220
STEEVES AND FREEMAN
Figure 21. Time-resolved current cross-correlation detected on the tip assembly (a) in tunneling (5 nA and 80 mV settings), (b) when the tip is crashed into the sample, and (c) is the time derivative of (b). (Reprinted with permission from S. Weiss, D. F. Ogletree, D. Botkin, M. Salmeron, and D. S. Chemla, 1993. Appl. Phys. Lett. 63, 2567, American Institute of Physics.)
˚ from the surface, the DC and time-resolved By extracting the tunneling tip 50 A currents drop to zero. The authors used this evidence to conclude that the PG˚ This conclusion is not STM technique has spatial resolution of at least 50 A. valid. Even in ordinary STM, I/Z sensitivity and lateral resolution are essentially unrelated. Another interesting observation was that the time-resolved correlation pulse width increases with increasing gap resistance. This the authors attribute to an RC time for the tunnel junction, where R is the gap resistance and C is a quantum capacitance between tip and sample. One perplexing question about this work is the role of tip/sample geometrical capacitance. According to Weiss, Ogletree, et al. (1993), a quantum tip/sample capacitance (≈10−18 F) played a significant role in the shape of the time-resolved current. This was supported by the evidence that the shape of the time-resolved signal changes dramatically if the STM tip is crashed into the sample (transmission line) as shown in Figure 21. The time-resolved signal in contact looked like the integral of the tunneling signal. If a quantum capacitance were the only source of the time-resolved signal, then the geometric tip/sample capacitance, in parallel with the quantum capacitance and 3 orders of magnitude bigger, was not having any contribution.
ULTRAFAST SCANNING TUNNELING MICROSCOPY
221
Figure 22. Photoconductively gated STM tip above a coplanar stripline and its equivalent circuit model shown below. (Adapted with permission from R. H. M. Groeneveld and H. van Kempen, 1996. Appl. Phys. Lett. 69, 2294, American Institute of Physics.)
The true nature of capacitance in PG-STM was not resolved until 1996 by Groeneveld and van Kempen. In their paper, “The capacitive origin of the picosecond electrical transients detected by a photoconductively gated scanning tunneling microscope,” the authors established that PG-STM was a capacitive microscopy where the spatial resolution of the time-resolved signal should be limited to microns. A simple circuit model was used where the tunnel junction was represented by a linear conductance γ in parallel with the geometric capacitance Ct . This tunnel junction is in series with the photoconductive switch with time-dependent conductance gs (t) in parallel with the switch capacitance Cs . This configuration is shown in Figure 22. The time-dependent switch conductance and bias voltage Vin(t) are periodic with period T (13.1 ns), an essential feature since RC times of the tip/sample (e.g., 10 M × 1 fF = 10 ns) are comparable to the period. From this model the time-dependent tip voltage on the tip of the STM (before the switch) is given by Cp
dV d Vin (t) + [gs (t) + γ ]V = Ct + γ Vin (t), dt dt
(10)
222
STEEVES AND FREEMAN
Figure 23. Comparison between the measured (solid curve) and calculated (dashed curve) correlation current for different tunnel conductance γ . In contact γ = 10−2−1 and tunneling γ = 1.4 × 10−8−1, each curve has been independently scaled. (Reprinted with permission from R. H. M. Groeneveld and H. van Kempen, 1996. Appl. Phys. Lett. 69, 2294, American Institute of Physics.)
where C p = Cs + Ct . Integrating the product of V (t) with gs (t) over the period T we get the averaged current: ) 1 T Ic = gs (τ )V (τ )dτ. (11) T 0 If the STM tip is withdrawn from the sample surface and γ drops to zero, then the integrated current becomes zero since no net current can flow through a capacitor. This explains one of the strange results of the aforementioned Weiss publication. Solving Eq. (10) and (11) for an STM tip tunneling and in contact with a transmission line also fits the Weiss, Ogletree, et al. (1993) result as shown in Figure 23. The excellent agreement between the Groeneveld and van Kempen (1996) model and the Weiss et al. results confirmed that the PG-STM technique would not maintain STM spatial resolution in its application. PG-STM used the electronics of a scanning tunneling microscope to detect a time-resolved signal which was capacitive in origin. The last of the pioneering TR-STM papers was published on December 27, 1993, by Takeuchi and Kasahara, two months later than the previously reported experiments. The authors developed a photogated STM technique and used it to show better than 300-ps time resolution. Unfortunately, the photogated technique of Takeuchi and Kasahara also falls victim to the analysis of Groeneveld
ULTRAFAST SCANNING TUNNELING MICROSCOPY
223
Figure 24. Schematic of probe tip and sample device. SBD is Schottky barrier diode. (Adapted with permission from Koichiro Takeuchi and Yukio Kasahara, 1993. Appl. Phys. Lett. 63, 3548, American Institute of Physics.)
and van Kempen as a capacitive microscopy without STM spatial resolution. The experimental setup is shown in Figure 24. The sample (a gold transmission line) is stimulated by the pulsed output of a Schottky barrier diode, driven by a 100-MHz synthesized sinusoid. A Pt/Ir STM tip is attached to a photoconductive switch of Au/Mo patterned onto a semi-insulating InP substrate. The dark resistance of the switch is around 1 M, less than a typical tunneling resistance, so the STM can operate with the photoconductive switch closed. By illuminating the switch with 80-ps pulses (400 μW (average power) at 670 nm) at a 100 MHz + 1 kHz repetition rate from a driven laser diode, additional current from the STM can be gated into the STM electronics, through the 1 k open resistance of the InP switch. The pump frequency of the sample was set to 100 MHz, so that the probe beam would sweep across the period of the stimulated sample in 100,000 cycles. This brought the dynamics of the sample within the bandwidth of the STM electronics, as shown in Figure 25. Implementation of this technique relies on an electrical synthesizer to pump the sample and gate the STM tip current using a photoconductive switch stimulated by a driven laser diode. The speed of the synthesizer will limit the time
224
STEEVES AND FREEMAN
Figure 25. Output of the Schottky barrier diode as measured using a high-speed oscilloscope. (Reprinted with permission from Koichiro Takeuchi and Yukio Kasahara, 1993. Appl. Phys. Lett. 63, 3548, American Institute of Physics.)
resolution of this technique. It would be advantageous to rely instead on all optical methods for generating electrical pulses needed to stimulate the sample under investigation. This technique is essentially a modification of the Weiss et al. photogated STM technique as a photoconductive switch mounted on the STM tip is used to gate the STM tunneling current. This implies that the Takeuchi, Kasahara method for time resolving an STM will not maintain STM spatial resolution. Since the initial PG-STM papers were published, there have been other photogated STM experiments exploring speed limitations (Weiss, Botkin, et al., 1995; Botkin et al., 1996) and more efficient tip geometries (Groenveld et al., 1996) for PG-STM. Even when the role of tip sample capacitance had been elucidated, efforts continued to explore the spatiotemporal limits of this technique. The first attempt at spatiotemporal imaging using photogated STM was by Jensen et al. (1997). In this work, an STM tip with an integrated photogated switch was scanned over a coplanar transmission line structure patterned onto a GaAs substrate. The photogated signal was acquired as the tip was scanned within a 50 × 50 μm maximum scan range. While the tip was above conducting elements of the transmission line, the STM was tunneling but the tip was also scanned over the insulating GaAs substrate. Above the insulating substrate the STM tip was probably in mechanical contact with the GaAs surface, but a time-resolved signal was nonetheless detected. To maintain illumination of the photoconductive switch while the STM tip was being scanned, an optical fiber
ULTRAFAST SCANNING TUNNELING MICROSCOPY
225
was mounted directly onto the switch. The STM position was scanned in 1-μm increments orthogonally to the transmission lines at various pump/probe delay times. Using this technique the modal structure of a voltage pulse on a coplanar waveguide structure was directly imaged, and spatiotemporal resolution of microns-picoseconds was demonstrated. These results are consistent with the Groeneveld interpretation of PG-STM.
VII. Ultrafast STM by Direct Optical Coupling to the Tunnel Junction Returning to the original anonymous proposal for ultrafast STM, we now examine schemes for direct laser coupling to the tunnel junction beginning with that of Feldstein et al. (1996). Motivating their work was the goal of studying chemical dynamics at an interface. To study chemical dynamics, femtosecond time resolution is required, which puts severe limitations on the potential implementations of ultrafast STM. The authors propose two schemes for achieving this goal, and they performed initial experiments using one of their schemes. The first scheme proposed was to perform scanning tunneling spectroscopy (Chen, 1993) on a surface, while modifying the surface local density of states using time-resolved laser spectroscopy (Polanyi and Zewail, 1995). The second was to use the proximity of the STM tip to locally modify the electronic environment of the sample which would then be probed using pumpprobe optical techniques which are sensitive to the electronic state of the test specimen. The former offers the chance to realize femtosecond ultrafast STM, while the latter relies on a near-field tip/sample interaction similar to that used in ultrafast near-field scanning optical microscopy (Levy et al., 1996). Focusing on these approaches, we now consider how to implement such a system. It is important that the sample under investigation be accessible to the scanning tunneling microscope and to the optical pump and probe beams. Using the geometry shown in Figure 26, the authors are able to illuminate the bottom ˚ Ag film) mounted on the hypotenuse of the thin sample (in this case a 525-A of a right-angle prism. The STM tunnels into the sample from above. Pumping and probing the same spot on the sample and measuring the intensity of the probe pulse as a function of delay time yielded an auto-correlation trace shown in Figure 27. The pump pulse generates surface plasmon polaritons which increase the electron temperature in the sample, shifting the plasmon resonance condition for the second probe pulse thus affecting its reflected intensity. If the STM tip was moved within tunneling, the tunneling regime on the other side of the film and its tip/sample separation was modulated at 2 kHz, by 1–10 nm the detected pump/probe correlation signal, measured through probe intensity, was also found to be modulated at 2 kHz. The spatial dependence of this signal
226
STEEVES AND FREEMAN
Figure 26. The sample and probe apparatus consists of a pair of 20-cm focal length lenses, a long pass filter, a fused silica prism mounted on a goniometer, and an STM. (Reprinted with permission from M. J. Feldstein, P. Vohringer, W. Wang, and N. F. Scherer, 1996. J. Phys. Chem. C 1996 American Chemical Society.) 100(12), 4739–4748.
on STM tip position was not investigated. Keil et al. (1998) have attributed this effect to plasmon scattering by the STM tip as the plasmon field is though to extend microns from the sample surface. Using a similar experimental setup, Keil et al. (1998) have gone further, and demonstrated as ultrafast signal mediated by tunneling.
Figure 27. Time-dependent pump-probe signal from Ag film. (Reprinted with permission from M. J. Feldstein, P. Vohringer, W. Wang, and N. F. Scherer, 1996. J. Phys. Chem. 100(12), C 1996 American Chemical Society.) 4739–4748.
ULTRAFAST SCANNING TUNNELING MICROSCOPY
227
Figure 28. (a) Time-resolved tunneling current measured at Vbias = 48 mV, IDC = 0.7 nA with tip under probe beam. (b) Time-resolved signal dependence on DC current, fit to y = ax b where b = 0.49. (Reprinted with permission from Ulrich D. Keil, Taekjip Ha, Jacob R. Jensen, and Jørn M. Hvam, 1998. Appl. Phys. Lett. 72, 3074, American Institute of Physics.)
Using the same overall geometry, though the pump and probe beams were not overlapping (20 μm separation), Keil et al. (1998) were able to observe a time-resolved current contribution with 200 fs time resolution. In their experiment, the pump and probe pulses were modulated at over 600 kHz. This was to minimize the effect of sample and tip heating. The time-resolved tunneling current component was extracted at the difference frequency between pump and probe modulations, 1.4 kHz. Tunneling onto a 40-nm thick Au sample, with a 48-mV bias voltage and 0.7-nA DC tunneling current, they observed a time-resolved tunneling current. The STM tip was tunneling above the probe pulse, and the time-resolved signal shown in Figure 28 peaks 1.2 ps after the pump pulse is incident. Withdrawing the STM tip, the authors find that this signal also diminishes to zero, but the decay follows that of a square root, not an exponential as expected. The width of the time-resolved signal is thought to be limited by the experimental resolution and ultimately limited by the coherence time of a plasmon wave which is influenced by the STM tip.
228
STEEVES AND FREEMAN
In this work, the authors demonstrated the use of an STM tip to detect a ultrafast signal 200 fs in duration. The dependence of this signal on tip/sample separation is perplexing and it is not clear whether this technique will achieve the ultimate resolution of the STM. Further investigations into direct optical manipulation of the STM tunnel junction were carried out by Gerstner et al. (2000), but measurements achieving STM spatial resolution have yet to be made. Nonetheless, direct optically induced sample excitations and optical junction modulation represent the ultimate in spatiotemporal resolution through ultrafast STM. VIII. Conclusions Ultrafast STM combines the unmatched spatial resolution of a scanning tunneling microscope with the ever-increasing temporal resolution of pump/probe ultrafast optical techniques. Ultrafast STM by junction mixing has been used to create atomic scale snap-shot images, with 1 nm/20 ps spatiotemporal resolution. Calculations suggest that time resolution in the femtosecond regime will be possible using the junction mixing technique. This technique can be applied to study any dynamic system on which ordinary STM can resolve contrast. The nature of the dynamics must be repetitive for any pump/probe scheme to function, but recent work has shown that even dynamic systems with nondeterministic responses can be imaged, and the stochastic components analyzed (Freeman et al., 2000). In situations where dynamics must be imaged in time-scales well below 1 ps, the junction mixing STM technique as implemented with photoconductive switches will not be fast enough. In that case, direct optical junction modulation will need to be further developed. Acknowledgment This work was supported by the Natural Sciences and Engineering Research Council of Canada. References Anonymous. (1989). August:# Research Disclosure, 30480. Binnig, G., and Rohrer, H. (1982). Helv. Phys. Acta. 55, 726. Binnig, G., and Rohrer, H. (1987). Rev. Mod. Phys. 56, 615. Binnig, G., Quate, C. F., and Gerber, Ch. (1986). Phys. Rev. Lett. 56, 930. Botkin, D., Glass, J., Chemla, D. S., Ogletree, D. F., Salmeron, M., and Weiss, S. (1996). Appl. Phys. Lett. 69, 1321.
ULTRAFAST SCANNING TUNNELING MICROSCOPY
229
Bridges, G. E., Said, R. A., and Thompson, D. J. (1993). Electron. Lett. 29, 1448. Chen, C. Julian (1993). Introduction to Scanning Tunneling Microscopy. New York: Oxford. Christen, T., and B¨uttiker, M. (1996). Phys. Rev. Lett. 77, 143. Durkan, C., and Welland, M. E. (2002). Appl. Phys. Lett. 80, 458. Fan, B., Faris, S. M., Gustafson, T. K., and Bridges, T. J. (1977). Appl. Phys. Lett. 30, 177. Faris, S. M., and Gustafson, T. K. (1974). Appl. Phys. Lett. 25, 544. Faris, S. M., Gustafson, T. K., and Wiesner, J. C. (1973). IEEE J. Quantum Electron. QE-9, 737. Feldstein, M. J., V¨ohringer, P., Wang, W., and Scherer, N. F. (1996). J. Phys. Chem. 100, 4739. Freeman, M. R., Hunt, R. W., and Steeves, G. M. (2000). Appl. Phys. Lett. 77, 717. Freeman, M. R., and Nunes, Geoff, Jr. (1993). Appl. Phys. Lett. 63, 2633. Gerstrner, V., Knoll, A., Pfeiffer, W., Thon, A., and Gerber, G. (2000). J. Appl. Phys. 88, 4851. Groeneveld, R. H. M., and van Kempen, H. (1996). Appl. Phys. Lett. 69, 2294. Groeneveld, R. H. M., Rasing, Th., Kaufmann, L. M. F., Smalbrugge, E., Wolter, J. H., Melloch, M. R., and van Kempen, H. (1996). J. Vac. Sci. Tech. B 14, 861. Hamers, R. J., and Cahill, David G. (1990). Appl. Phys. Lett. 57, 2031. Hamers, R. J., and Cahill, David G. (1991). J. Vac. Sci. Technol. B 9(2), 514. Hamers, R., and Markert, K. (1990). Phys. Rev. Lett. 64, 1051. Hess, H. F., Robinson, R. B., Dynes, R. C., Valles, J. M., Jr., and Waszczak, J. V. (1989). Phys. Rev. Lett. 62, 214. Hou, A. S., Ho, F., and Bloom, D. M. (1992). Electron. Lett. 28, 2302. Hudson, E. W., Lang, K. M., Madhavan, V., Pan, S. H., Eisaki, H., Uchida, S., and Davis, J. C. (2001). Nature 411, 920. Jensen, Jacob R., Keil, Ulrich D., and Hvam, Jørn M. (1997). Appl. Phys. Lett. 70, 2762. Keil, Ulrich D., Ha, Taekjip, Jensen, Jacob R., and Hvam, Jørn M. (1998). Appl. Phys. Lett. 72, 3074. Khusnatdinov, N. N., Nagle, T. J., and Nunes, Geoff, Jr. (2000). Appl. Phys. Lett. 77, 4434. Kurokawa, S., and Sakai, A. (1997). Sci. Rep. RITU 44, 173. Levy, J., Nikitin, V., Kikkawa, J. M., Cohen, A., Samarth, N., Garcia, R., and Awschalom, D. D. (1996). Phys. Rev. Lett. 76, 1948. Mamin, H. J., Birk, H., Wimmer, P., and Rugar, D. (1994). J. Appl. Phys. 75, 161. Nunes, Geoff, Jr., and Freeman, M. R. (1993). Science 262, 1029. Ochmann, M., M¨unzer, H. J., Boneberg, J., and Leiderer, P. (1999). Rev. Sci. Instrum. 70, 2049. Polanyi, J. C., and Zewail, A. H. (1995). Acc. Chem. Res. 28, 119. Rippard, W. H., and Buhrman, R. A. (1999). Appl. Phys. Lett. 75, 1001. Said, Ra’a A. (1995). Scanning Force Potentiometry Techniques for Semiconductor Circuit Characterization. Ph.D. thesis, University of Manitoba. Simmons, J. G. (1963). J. Appl. Phys. 34, 238. Steeves, G. M., Elezzabi, A. Y., and Freeman, M. R. (1997). Appl. Phys. Lett. 70, 1909. Steeves, G. M., Elezzabi, A. Y., and Freeman, M. R. (1998). Appl. Phys. Lett. 72, 504. Steeves, G. M., Elezzabi, A. Y., Teshima, R., Said, R. A., and Freeman, M. R. (1998). IEEE J. Quantum Electron. 34, 1415. Steeves, G. M. (2001). Junction Mixing Scanning Tunneling Microscopy. Ph.D. thesis, University of Alberta. Takeuchi, K., and Kasahara, Y. (1993). Appl. Phys. Lett. 63, 3548. Wang, Baigeng, Zhao, Xuean, Wang, Jian, and Guo, Hong (1999). Appl. Phys. Lett. 74, 2887. Weiss, S., Ogletree, D. F., Botkin, D., Salmeron, M., and Chemla, D. S. (1993). Appl. Phys. Lett. 63, 2567. Weiss, S., Botkin, D., Ogletree, D. F., Salmeron, M., and Chemla, D. S. (1995). Phys. Status Solidi B 188, 343.
This Page Intentionally Left Blank
ADVANCES IN IMAGING AND ELECTRON PHYSICS, VOL. 125
Low-Density Parity-Check Codes—A Statistical Physics Perspective RENATO VICENTE,1,∗ DAVID SAAD1 AND YOSHIYUKI KABASHIMA2 1
Neural Computing Research Group, University of Aston, Birmingham B4 7ET, United Kingdom Department of Computational Intelligence and Systems Science, Tokyo Institute of Technology, Yokohama 2268502, Japan
2
I. Introduction . . . . . . . . . . . . . . . . . . . . . . A. Error Correction . . . . . . . . . . . . . . . . . . . B. Statistical Physics of Coding . . . . . . . . . . . . . . C. Outline . . . . . . . . . . . . . . . . . . . . . . II. Coding and Statistical Physics . . . . . . . . . . . . . . A. Mathematical Model for a Communication System . . . . 1. Data Source and Sink . . . . . . . . . . . . . . . 2. Source Encoder and Decoder . . . . . . . . . . . . 3. Noisy Channels . . . . . . . . . . . . . . . . . . 4. Channel Encoder and Decoder . . . . . . . . . . . . B. Linear Error-Correcting Codes and the Decoding Problem . C. Probability Propagation Algorithm . . . . . . . . . . . D. Low-Density Parity-Check Codes . . . . . . . . . . . E. Decoding and Statistical Physics . . . . . . . . . . . . III. Sourlas Codes . . . . . . . . . . . . . . . . . . . . . A. Lower Bound for the Probability of Bit Error . . . . . . . B. Replica Theory for the Typical Performance of Sourlas Codes C. Shannon’s Bound . . . . . . . . . . . . . . . . . . D. Decoding with Probability Propagation . . . . . . . . . IV. Gallager Codes . . . . . . . . . . . . . . . . . . . . A. Upper Bound on Achievable Rates . . . . . . . . . . . B. Statistical Physics Formulation . . . . . . . . . . . . . C. Replica Theory . . . . . . . . . . . . . . . . . . . D. Replica Symmetric Solution . . . . . . . . . . . . . . E. Thermodynamic Quantities and Typical Performance . . . F. Codes on a Cactus . . . . . . . . . . . . . . . . . . G. Tree-Like Approximation and the Thermodynamic Limit . . H. Estimating Spinodal Noise Levels . . . . . . . . . . . V. MacKay–Neal Codes . . . . . . . . . . . . . . . . . . A. Upper Bound on Achievable Rates . . . . . . . . . . . B. Statistical Physics Formulation . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
232 232 236 236 237 237 238 238 239 241 242 244 250 250 252 254 256 262 266 270 272 273 275 277 278 282 287 289 291 294 294
∗ Current affiliation: Departamento de F´ısica Geral, Instituto de F´ısica, Universidade de S˜ao Paulo, 05315-970, S˜ao Paulo–SP, Brazil; to whom correspondence should be addressed (
[email protected]).
231 Copyright 2002, Elsevier Science (USA). All rights reserved. ISSN 1076-5670/02 $35.00
232
VICENTE ET AL.
C. Replica Theory . . . . . . . . . . . . . . . . . . . D. Probability Propagation Decoding . . . . . . . . . . . E. Equilibrium Results and Decoding Performance . . . . . . 1. Analytical Solution: The Case of K ≥ 3 . . . . . . . . 2. The Case of K = 2 . . . . . . . . . . . . . . . . 3. The Case of K = 1 and General L > 1 . . . . . . . . F. Error Correction: Regular vs. Irregular Codes . . . . . . . G. The Spinodal Noise Level . . . . . . . . . . . . . . . 1. Biased Messages: K ≥ 3 . . . . . . . . . . . . . . 2. Unbiased Messages . . . . . . . . . . . . . . . . VI. Cascading Codes . . . . . . . . . . . . . . . . . . . . A. Typical PP Decoding and Saddle-Point-Like Equations . . . B. Optimizing Construction Parameters . . . . . . . . . . VII. Conclusions and Perspectives . . . . . . . . . . . . . . . Appendix A. Sourlas Codes: Technical Details . . . . . . . 1. Free-Energy . . . . . . . . . . . . . . . . . . . 2. Replica Symmetric Solution . . . . . . . . . . . . . 3. Local Field Distribution . . . . . . . . . . . . . . 4. Zero Temperature Self-Consistent Equations . . . . . . 5. Symmetric Channels Averages at Nishimori’s Temperature 6. Probability Propagation Equations . . . . . . . . . . Appendix B. Gallager Codes: Technical Details . . . . . . . 1. Replica Theory . . . . . . . . . . . . . . . . . . 2. Replica Symmetric Solution . . . . . . . . . . . . . 3. Energy Density at the Nishimori Condition . . . . . . 4. Recursion Relations . . . . . . . . . . . . . . . . Appendix C. MN Codes: Technical Details . . . . . . . . . 1. Distribution of Syndrome Bits . . . . . . . . . . . . 2. Replica Theory . . . . . . . . . . . . . . . . . . 3. Replica Symmetric Free-Energy . . . . . . . . . . . 4. Viana–Bray Model: Poisson Constructions . . . . . . . References . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
296 301 303 303 307 307 310 312 312 315 317 319 323 325 327 327 329 331 332 333 334 336 336 337 338 339 340 340 341 344 348 349
I. Introduction A. Error Correction The way we communicate has been deeply transformed during the twentieth century. Telegraph, telephone, radio, and television technologies have brought to reality instantaneous long distance communication. Satellite and digital technologies have made global high-fidelity communication possible. Two obvious common features of modern digital communication systems are that typically the message to be transmitted (e.g., images, text, computer programs) is redundant and the medium used for transmission (e.g., deepspace, atmosphere, optical fibers, etc.) is noisy. The key issues in modern communication are, therefore, saving storage space and computing time by
LOW-DENSITY PARITY-CHECK CODES
233
eliminating redundancies (source coding or compression) and making transmissions reliable by employing error-correction techniques (channel coding). Shannon was one of the first to point out these key issues. In his influential 1948 papers, Shannon proved general results on the natural limits of compression and error-correction by setting up the framework for what is now known as information theory. Shannon’s channel coding theorem states that error-free communication is possible if some redundancy is added to the original message in the encoding process. A message encoded at rates R (message information content/codeword length) up to the channel capacity Cchannel can be decoded with a probability of error that decays exponentially with the message length. Shannon’s proof was nonconstructive and assumed encoding with unstructured random codes and impractical (nonpolynomial time) (Cover and Thomas, 1991) decoding schemes. Finding practical codes capable of reaching the natural coding limits is one of the central issues in coding theory. To illustrate the difficulties that may arise when trying to construct high performance codes from first principles, we can use a simple geometric illustration. On the top left of Figure 1 we represent the space of words (a message is a sequence of words), and each circle represents one sequence of binary bits. The word to be sent is represented by a black circle in the left side figure. Corruption by noise in the channel is represented in the top right figure as
Figure 1. In the top figure we illustrate what happens when a word is transmitted without error correction. White circles represent possible word vectors, the black circle represents the word to be sent. The channel noise causes corruption of the original word that is represented by a drift in the top right picture. The dashed circles indicate decision boundaries in the receiver; in the case depicted, noise corruption leads to a transmission error. In the bottom figure we show qualitatively the error-correction mechanism. The redundant information changes the space geometry, increasing the distance between words. The same drift as in the top figure does not result in a transmission error.
234
VICENTE ET AL.
a drift in the original word location. The circle around each word represent spheres that provide a decision boundary for each particular word; any signal inside a certain decision region is recognized as representing the word at the center of the sphere. In the case depicted in Figure 1 the drift caused by noise places the received word within the decision boundary of another word vector, causing a transmission error. Error-correction codes are based on mapping the original space of words onto a higher dimensional space in such a way that the typical distance between encoded words (codewords) increases. If the original space is transformed, the same drift shown in the top of Figure 1 is insufficient to push the received signal outside the decision boundary of the transmitted codeword (bottom figure). Based on this simple picture we can formulate general designing criteria for good error-correcting codes: codewords must be short sequences of binary digits (for fast transmission), the code must allow for a large number of codewords (for a large set of words), and decision spheres must be as large as possible (for large error-correction capability). The general coding problem consists of optimizing one of these conflicting requirements given the other two. So, for example, if the dimension of the lattice and diameter of decision spheres are fixed, the problem is finding the lattice geometry that allows the densest possible sphere packing. This sphere packing problem is included in the famous list of problems introduced by Hilbert (it is actually part of the 18th problem). This problem can be solved for a very limited number of dimensions (Conway and Sloane, 1998), but is very difficult in general. As a consequence, constructive procedures are known only for a limited number of small codes. For a long time, the best practical codes known were Reed–Solomon codes (RS) operating in conjunction with convolutional codes (concatenated codes). The current technological standard are RS codes, proposed in 1960, found almost everywhere from compact disks to mobile phones and digital television. Concatenated codes are the current standard in deep-space missions (e.g., Galileo mission) (MacWilliams and Sloane, 1977; Viterbi and Omura, 1979). Recently, Turbo codes (Berrou et al., 1993) have been proven to outperform concatenated codes and are becoming increasingly more common. These codes are composed of two convolutional codes working in parallel and show practical performance close to Shannon’s bound when decoded with iterative methods known as probability propagation, first studied in the context of coding by Wiberg (1996). Despite the success of concatenated and Turbo codes, the current performance record is owned by Gallager’s low-density parity-check codes (e.g., Chung, 2000; Davey, 1998, 1999). Gallager codes were first proposed in 1962 (Gallager, 1962, 1963) and then were all but forgotten soon after due to computational limitations of the time and due to the success of convolutional codes.
LOW-DENSITY PARITY-CHECK CODES
235
To give an idea of how parity-check codes operate, we exemplify with the simplest code of this type known as Hamming code (Hamming, 1950). A (7, 4) Hamming code, where (7, 4) stands for the number of bits in the codeword and input message, respectively, operates by adding 3 extra bits for each 4 message bits; this is done by a linear transformation G, called the generator matrix, represented by: ⎛ ⎞ 1 0 0 0 ⎜0 1 0 0⎟ ⎜ ⎟ ⎜0 0 1 0⎟ ⎜ ⎟ G = ⎜0 0 0 1⎟ . (1) ⎜0 1 1 1⎟ ⎜ ⎟ ⎝1 0 1 1⎠ 1 1 0 1
When the generator matrix G is applied to a digital message s = (s1 , s2 , s3 , s4 ), we get an encoded message defined by t = Gs composed of 4 message bits plus redundant information (parity-check) as 3 extra bits t5 = s2 ⊕ s3 ⊕ s4 , t6 = s1 ⊕ s3 ⊕ s4 and t7 = s1 ⊕ s2 ⊕ s4 (⊕ indicates binary sums). One interesting point to note is that the transmitted message is such that t5 ⊕ s2 ⊕ s3 ⊕ s4 = 0 and similarly for t6 and t7, what allows direct check of single corrupted bits. The decoding procedure relies in a second operator, known as parity-check matrix, with the property HG = 0. For the generator (1) the parity-check matrix has the following form: ⎛ ⎞ 0 0 0 1 1 1 1 H = ⎝0 1 1 0 0 1 1⎠ . (2) 1 0 1 0 1 0 1 The decoding procedure follows from the observation that the received message is corrupted by noise as r = Gs ⊕ n. By applying the parity-check matrix we get the syndrome Hr = Hn = z. In the (7, 4) Hamming code the syndrome vector gives the binary representation for the position of the bit where an error has occurred (e.g., if n = (0, 0, 1, 0, 0, 0, 0), z = (0, 1, 1)). Due to this nice property, decoding is trivial and this code is known as a perfect single-errorcorrecting code (Hill, 1986). Codes in the low-density parity-check family work along the same principles as the simple Hamming code above, the main differences being that they are much longer, the parity-check matrix is very sparse, and multiple errors can be corrected. However, low-density parity-check codes are not perfect and the decoding problem is, in general, significantly more difficult. Luckily, the sparseness of the matrix allows for the decoding process to be carried out by probability propagation methods similar to those employed in Turbo codes. Throughout
236
VICENTE ET AL.
this chapter we concentrate on low-density parity-check codes (LDPC) that are state of the art concerning performance and operate along simple principles. We study four variations of LDPCs known as Sourlas codes, Gallager codes, MacKay–Neal codes, and Cascading codes. B. Statistical Physics of Coding The history of statistical physics application to error-correcting codes started in 1989 with a paper by Sourlas relating error-correcting codes to spin glass models (Sourlas, 1989). He showed that the Random Energy Model (Derrida, 1981b; Saakian, 1998; Dorlas and Wedagedera, 1999) can be thought of as an ideal code capable of saturating Shannon’s bound at vanishing code rates. He also showed that the SK model (Kirkpatrick and Sherrington, 1978) could operate as a practical code. In 1995, convolutional codes were analyzed by employing the transfermatrix formalism and power series expansions (Amic and Luck, 1995). In 1998, Sourlas work was extended for the case of finite code rates (Kabashima and Saad, 1999a) by employing the replica method. Recently, Turbo codes were also analyzed using the replica method (Montanari and Sourlas, 2000; Montanari, 2000). In this chapter we present the extension of Sourlas work together with the analysis of other members in the family of LDPCs. We rely mainly on replica calculations (Kabashima et al., 2000; Murayama et al., 2000; Vicente et al., 2000b) and mean-field methods (Kabashima and Saad, 1998; Vicente et al., 2000a). The main idea is to develop the application of statistical physics tools for analyzing error-correcting codes. A number of results obtained are rederivations of well known results in information theory, while others put known results into a new perspective. The main differences between the statistical physics analysis and traditional results in coding theory are the emphasis on very large systems from the start (thermodynamic limit) and the calculation of ensemble typical performances instead of worst-case bounds. In this sense statistical physics techniques are complementary to traditional methods. As a byproduct of our analysis we connect the iterative decoding methods of probability propagation with wellknown mean-field techniques, presenting a framework that might allow a systematic improvement of decoding techniques. C. Outline In the next section we provide an overview of results and ideas from information theory that are relevant for understanding of the forthcoming sections. We also discuss more deeply linear encoding and parity-check decoding. We present
LOW-DENSITY PARITY-CHECK CODES
237
the probability propagation algorithm for computing approximate marginal probabilities efficiently and finish by introducing the statistical physics point of view of the decoding problem. In Section III, we investigate the performance of error-correcting codes based on sparse generator matrices proposed by Sourlas. We employ replica methods to calculate the phase diagram for the system at finite code rates. We then discuss the decoding dynamics of the probability propagation algorithm. Sourlas codes are regarded as a first step toward developing techniques to analyze other more practical codes. Section IV provides a statistical physics analysis for Gallager codes. These codes use a dense generator and a sparse parity-check matrix. The code is mapped onto a K-body interaction spin system and typical performance is obtained using the replica method. A mean-field solution is also provided by mapping the problem onto a Bethe-like lattice (Husimi cactus), recovering, in the thermodynamic limit, the replica symmetric results and providing a very good approximation for finite systems of moderate size. We show that the probability propagation decoding algorithm emerges naturally from the analysis, and its performance can be predicted by studying the free-energy landscape. A simple technique is introduced to provide upper bounds for the practical performance. In Section V we investigate MacKay–Neal codes that are a variation of Gallager codes. In these codes, decoding involves two very sparse paritycheck matrices, one for the signal with K nonzero elements in each row and a second for the noise with L nonzero elements. We map MN codes onto a spin system with K + L interacting spins. The typical performance is again obtained by using a replica symmetric theory. A statistical description for the typical PP decoding process for cascading codes is provided in Section VI. We use this description to optimize the construction parameters of a simple code of this type. We close, in Section VII, with concluding remarks. Appendices with technical details are also provided.
II. Coding and Statistical Physics A. Mathematical Model for a Communication System In his papers from 1948, Shannon introduced a mathematical model (schematically represented in Figure 2) incorporating the most basic components of communication systems, and identified key problems and proved some general results. In the following we will introduce the main components of Shannon’s communication model, the mathematical objects involved, as well as related general theorems.
238
VICENTE ET AL.
Figure 2. Mathematical model for a communication system. Each component is discussed in the text.
1. Data Source and Sink A data source can be discrete or continuous. A discrete source is defined by the pair (S , π), where S is a set of m symbols (alphabet) and π is a probability measure over the space of sequences of symbols with any length (messages). In general, any discrete alphabet can be mapped onto sequences of [log m] Boolean digits {0, 1}. Continuous sources can always be made discrete at the expense of introducing some distortion to the signal (Cover and Thomas, 1991). A source is memoryless if each symbol in the sequence is independent of the preceding and succeeding symbols. A data sink is simply the receiver of decoded messages. 2. Source Encoder and Decoder Data sources usually generate redundant messages that can be compressed to vectors of shorter average length. Source encoding, also known as data compression, is the process of mapping sequences of symbols from an alphabet S onto a shorter representation A. Shannon employed the statistical physics idea of entropy to measure the essential information content of a message. As enunciated by Khinchin (1957), the entropy of Shannon is defined as follows: Definition II.1 (Entropy) Let $ a1 p1
a2 p2
··· ···
am pm
%
LOW-DENSITY PARITY-CHECK CODES
239
be a finite scheme, wherea j are mutually exclusive events and p j are associated probabilities with mj=1 p j = 1. The entropy of the scheme in bits (or shannons) is defined as H2 (A) = −
m
p j log2 p j .
(3)
j=1
The entropy is usually interpreted as the amount of information gained by removing the uncertainty and determining which event actually occurs. Shannon (1948) posed and proved a theorem that establishes the maximal shortening of a message by compression as a function of its entropy. The compression coefficient can be defined as μ ≡ lim N →∞ L N /N , where N is the original message length and L N is the average length of compressed messages. As presented by Khinchin (1957), the theorem states: Theorem II.1 (Source compression) Given a discrete source with m symbols and entropy of H bits, for any possible compression code, the compression coefficient is such that H ≤μ log2 m and there exists a code such that μ<
H +ǫ , log2 m
for arbitrarily small ǫ. A compression scheme that yields a coefficient μ within the bounds above, given that the statistical structure π of the source is known, was proposed in 1952 by Huffman. Several practical algorithms are currently known and the design of more efficient and robust schemes is still a very active research area (Nelson and Gailly, 1995). 3. Noisy Channels Message corruption during transmission can be described by a probabilistic model defined by the conditional probability P(r | t) where t and r represent transmitted and received messages, respectively. We can assume that in any of the channels used, only one component tj, j = 1, . . . , M of the original message is being sent. If there is no interference effects between components, the channel is memoryless and the conditional probability factorizes as P(r | t) = 3M P(r j | t j ). j=1
240
VICENTE ET AL.
A memoryless channel model is specified by (T , P(r | t), R), where T and R are input and output alphabets and P(r | t) transition probabilities. The information needed to specify t given the received signal r is the conditional entropy: ! H2 (T | R) = − P(r ) P(t | r ) log2 (P(t | r )) . (4) r ∈R
t∈T
The information on the original signal t conveyed by the received signal r is given by the mutual information I (T ; R) = H2 (T ) − H2 (T | R), where H2(T) is defined in (3). The maximal information per bit that the channel can transport defines the channel capacity (Cover and Thomas, 1991). Definition II.2 (Channel capacity) Given the channel model, the channel capacity is Cchannel = max I (T ; R), P(t)
where I(T; R) is understood as a functional of the transmitted bits distribution P(t). Thus, for example, if Cchannel = 1/2, in the best case, 2 bits must be transmitted for each bit sent. The following channel model (see MacKay, 1999, 2000a) is of particular interest in this chapter: Definition II.3 (Binary symmetric channel) The memoryless binary symmetric channel (BSC) is defined by binary input and output alphabets T = R = {0, 1} and by the conditional probability P(r = t | t) = p
P(r = t | t) = 1 − p.
(5)
The channel capacity of a BSC is given by CBSC = 1 − H2 ( p) = 1 + p log( p) + (1 − p) log(1 − p) In this chapter, we concentrate on the binary symmetric channel due to its simplicity and straightforward mapping onto an Ising spin system. However, there are several other channel types that have been examined in the literature and that play an important role in practical applications (Viterbi and Omura, 1979; Cover and Thomas, 1991). The most important of these is arguably the Gaussian channel; most of the analysis presented in this paper can be carried out in the case of the Gaussian channel as demonstrated in Kabashima and Saad (1999a) and Vicente et al. (1999).
LOW-DENSITY PARITY-CHECK CODES
241
Figure 3. Codebook for the (7, 4) Hamming code defined by (1).
4. Channel Encoder and Decoder Highly reliable communication is possible even through noisy channels. It can be achieved by protecting a message with redundant information using a channel encoder defined as: Definition II.4 ((2N, M) Code) A code of rate R = N /M is an indexed list (codebook) of 2N codewords t(i ) ∈ T each of length M. Each index i in the codebook corresponds to a possible sequence of message bits. In a digital system, a code can be regarded as a map of representations of 2N symbols as Boolean sequences of N bits onto Boolean sequences of M bits. In Figure 3, we show the codebook for the Hamming code defined by (1) that is a (24, 7) code. Each sequence of N = 4 message bits is indexed and converted in a codeword with M = 7 bits. A decoding function g is a map of a channel output r ∈ R back into a codeword. The probability that a symbol i is decoded incorrectly is given by the probability of block error: pBlock = P{g(r ) = i | t = t(i)}.
(6)
The average probability that a decoded bit sˆ j = g j (r ) fails to reproduce the original message bits is the probability of bit error: pb =
N 1 P{ˆs j = s j }. N j=1
(7)
Shannon’s coding theorem is as follows (Cover and Thomas, 1991; MacKay, 2000a). Theorem II.2 (Channel coding) The affirmative part of the theorem states: For every rate R < Cchannel , there exists a sequence of (2MR, M) codes with maximum probability of block error p(M) Block → 0. Conversely, any sequence → 0 must have R ≤ Cchannel . of (2MR, M) codes with p (M) Block
242
VICENTE ET AL.
The negative part of the theorem is a corollary of the affirmative part and states: Error-free communication above the capacity Cchannel is impossible. It is not possible to achieve a rate R with probability of bit error smaller than % $ Cchannel −1 . (8) 1− pb (R) = H2 R This nonconstructive theorem is obtained by assuming ensembles of random codes and impractical decoding schemes. No practical coding scheme (i.e., that can be encoded and decoded in polynomial time) that saturates the channel capacity is known to date. As Shannon’s proof does not deal with complexity issues, there is no guarantee that such practical scheme exists at all. B. Linear Error-Correcting Codes and the Decoding Problem Linear error-correction codes add redundancy to the original message s ∈ {0, 1} N through a linear map like: t = Gs (mod 2),
(9)
where G is an M × N Boolean matrix. The received message r = t + n is a corrupted version of the transmitted message. In the simplest form, optimal decoding consists of finding an optimal estimate sˆ (r ) assuming a model for the noisy channel P(r | t) and a prior distribution for the message source P(s). The definition of the optimal estimator depends on the particular task and loss function assumed. An optimal estimator is defined as follows (see Iba, 1999, and references therein): Definition II.5 (Optimal estimator) An optimal estimator sˆ (r ) for a loss function L(s, sˆ (r )) minimizes the average of L in relation to the posterior distribution P(s | r ). A posterior probability of messages given the corrupted message received can be easily found by applying Bayes theorem: P(r | t) δ (t; Gs) P(s) P(s | r ) = , s P(r | t) δ (t; Gs) P(s)
(10)
where δ(x; y) = 1 if x = y and δ(x; y) = 0, otherwise. If we define our task to be the decoding of perfectly correct messages (i.e., we are interested in minimizing the probability of block error pBlock), we have to employ a two-valued loss function that identifies single mismatches: L(s, sˆ (r )) = 1 −
M j=1
δ(s j ; sˆ j ).
(11)
LOW-DENSITY PARITY-CHECK CODES
243
An optimal estimator for this loss function must minimize the following: L(s, sˆ (r )) P(s|r ) = P(s | r )L(s, sˆ (r )) s
=1−
s
P(s | r )
= 1 − P(ˆs | r ).
M
δ(s j ; sˆ j )
j=1
(12)
Clearly, the optimal estimator in this case is sˆ = argmax S P(s | r ). This estimator is often called the Maximum a Posteriori estimator or simply MAP. If we tolerate a certain degree of error in the decoded message (i.e., we are instead interested in minimizing the probability of bit error pb), the loss function has to be an error counter like: L(s, sˆ (r )) = −
M
s j sˆ j ,
(13)
j=1
where we assume for simplicity the binary alphabet s ∈ {±1} N . The optimal estimator must minimize the following: L(s, sˆ (r )) P(s|r ) = −
M j=1
s j P(s|r ) sˆ j .
(14)
An obvious choice for the estimator is s j P(s|r ) | s j P(s|r ) | = sgn s j P(s|r )
sˆ j =
= argmaxs j P(s j | r ),
(15)
where P(s j | r ) = {sk :k= j} P(s | r ) is the marginal posterior distribution. As suggested by Eq. (15), this estimator is often called the Marginal Posterior Maximizer or MPM for short. Decoding, namely, the computation of estimators, becomes a hard task, in general, as the message size increases. The MAP estimator requires finding a global maximum of the posterior over a space with 2N points and the MPM estimator requires to compute long summations of 2N − 1 terms for finding the two valued marginal posterior. The exponential scaling makes a na¨ıve brute force evaluation quickly impractical. An alternative is to use approximate methods to evaluate posteriors. Popular methods are Monte Carlo sampling and the computationally more efficient probability propagation. In the sequence we will discuss the latter.
244
VICENTE ET AL.
C. Probability Propagation Algorithm The probabilistic dependencies existing in a code can be represented as a bipartite graph (Lauritzen, 1996) where nodes in one layer correspond to the M received bits rμ and nodes in the other layer to the N message bits sj. The connections between the two layers are specified by the generator matrix G. Decoding requires evaluation of posterior probabilities when the received bits r are known (evidence). The evaluation of the MPM estimator requires the computation of the following marginal joint distribution: P(s j , r ) = P(s | r )P(r ) {si :i= j}
= =
{si :i= j}
P(r | s)P(s)
M
{si :i= j} μ=1
P(rμ | si1 · · · si K )
N
P(s j ),
(16)
j=1
where si1 · · · si K are message bits composing the transmitted bit tμ = (Gs)μ = si1 ⊕ · · · ⊕ si K and r is the message received. Equation (16) shows a complex partial factorization that depends on the structure of the generator matrix G. We can encode this complex partial factorization on a directed graph known as a Bayesian network (Pearl, 1988; Castillo et al., 1997; Jensen, 1996; Kschischang and Frey, 1998; Aji and McEliece, 2000; Frey, 1998, Kschischang et al., 2001). As an example, we show in Figure 4 a simple directed bipartite graph encoding the following joint distribution: P(s1 , . . . , s4 , r1 , . . . , r6 ) = P(r1 | s1 , s2 , s3 )P(r2 | s3 )P(r3 | s1 , s2 ) ×P(r4 | s3 , s4 )P(r5 | s3 )P(r6 | s3 ) ×P(s1 )P(s2 )P(s3 )P(s4 )
(17)
Figure 4. Bayesian network representing a linear code of rate 2/3. If there is an arrow from a vertex sj to a vertex rμ , sj is said to be a parent and rμ is said to be a child.
LOW-DENSITY PARITY-CHECK CODES
The generator matrix for the code in Figure 4 is: ⎞ ⎛ 1 1 1 0 ⎜0 1 0 0⎟ ⎟ ⎜ ⎜1 1 0 0⎟ G=⎜ ⎟. ⎜0 0 1 1⎟ ⎝0 0 1 0⎠ 0 0 1 0
245
(18)
Given r , an exact evaluation of the marginal joint distribution (16) in a space of binary variables s ∈ {±1} N would require (N + M)(2 N −1 − 1) + 1 operations. In 1988, Pearl proposed an iterative algorithm that requires O(N ) computational steps to calculate approximate marginal probabilities using Bayesian networks. This algorithm is known as belief propagation (Pearl, 1988), probability propagation (Kschischang and Frey, 1998), generalized distributive law (Aji and McEliece, 2000) or sum-product algorithm (Frey, 1998; Kschischang et al., 2001; see also Opper and Saad, 2001). The probability propagation algorithm is exact when the Bayesian network associated to the particular problem is free of loops. To introduce the probability propagation algorithm we start with the simple chain in Figure 5, which represents the following joint distribution: p(s1 , s2 , s3 , s4 , s5 ) = p(s1 ) p(s2 | s1 ) p(s3 | s2 ) p(s4 | s3 ) p(s5 | s4 ).
(19)
Suppose now that we would like to compute p(s3), we would then have to compute: p(s3 ) = p(s1 ) p(s2 | s1 ) p(s3 | s2 ) p(s4 | s3 ) p(s5 | s4 ). (20) s1 ,s2 ,s4 ,s5
A brute force evaluation of (20) would take 5 × (24 − 1) + 1 = 61 operations in a binary field. The probability propagation algorithm reduces significantly the number of operations needed by rationalizing the order in which they are performed. For Figure 5 we can start by marginalizing vertex s5 and writing: p(s5 | s4 ). (21) R54 (s4 ) = s5
Figure 5. Marginal probabilities can be calculated exactly in a Bayesian chain. R-messages flow from a child to a parent and Q-messages flow from a parent to a child.
246
VICENTE ET AL.
The function R54(s4) can be regarded as a vector (a message) carrying information about vertex s5. In a similar way we can write: p(s4 | s3 )R54 (s4 ). (22) R43 (s3 ) = s4
Again R43(s3) can be seen as a message carrying information about vertices s4 and s5. Note that we can write (21) in the same form as (22) by assuming that R5 (s5 ) = 1 if s5 is not given or R5 (s5 ) = δ(s5 ; s ∗ ) if s5 = s ∗ , where δ(x; y) = 1 if x = y and δ(x; y) = 0, otherwise. We can also gather information from vertices to the left of s3. Firstly, we marginalize s1 by introducing: Q 12 (s1 ) = p(s1 ).
(23)
We then propagate the message Q12(s1) to s2 producing a new message: Q 12 (s1 ) p(s2 | s1 ). (24) Q 23 (s2 ) = s1
The marginal probability p(s3) can be finally computed by: p(s3 ) = Q 23 (s2 )R43 (s3 ) p(s3 | s2 ) s2
= = =
s2
s1
Q 12 (s1 ) p(s2 | s1 )
s1
p(s1 ) p(s2 | s1 )
s2
s1 ,s2 ,s4 ,s5
s4
s4
p(s4 | s3 )R54 (s4 ) p(s3 | s2 )
p(s4 | s3 )
s5
p(s5 | s4 )
p(s1 ) p(s2 | s1 ) p(s3 | s2 ) p(s4 | s3 ) p(s5 | s4 ).
(25)
The evaluation of p(s3) using probability propagation is exact and requires only 16 operations, much less than the 61 operations required for the brute force calculation. A slightly more complex situation is shown in Figure 6 representing the following joint distribution: p(s1 , . . . , s12 ) = p(s6 ) p(s8 ) p(s9 ) p(s10 ) p(s11 ) p(s12 ) p(s1 | s10 ) p(s2 | s11 , s12 ) × p(s3 | s1 , s2 , s9 ) p(s4 | s3 , s8 ) p(s5 | s3 , s6 ) p(s7 | s4 ). (26) Suppose that the variables are binary, s7 and s5 are given evidence vertices and we would like to compute the marginal p(s3). A brute force evaluation would require 11 × (29 − 1) + 1 = 5622 operations. In general, we can just initialize the messages with random values, or make use of prior knowledge that may be available, and update the vertices in a
LOW-DENSITY PARITY-CHECK CODES
247
Figure 6. Marginal probabilities also can be calculated exactly in a Bayesian tree.
random order, but this may require several iterations for convergence to the correct values. In the particular case of trees there is an obvious optimal scheduling that takes only one iteration per vertex to converge: start at the leaves (vertices with a single edge connected to them) and proceed to the next internal level until the intended vertex. For the tree in Figure 6, the optimal schedule would be as follows: r r
Q11,2, Q12,2, Q10,1, Q65, Q93, Q84 and Q74 Q13, Q23 and R43, R53
The Q-messages are just the prior probabilities: Q jμ (s j ) = p(s j ), where j = 6, 8, 9, 10, 11, 12. The R-message between s7 and s4 is: R7 (s7 ) p(s7 | s4 ), R74 (s4 ) =
(27)
(28)
s7
where R7 (s7 ) = δ(s7 , s7∗ ) and s7∗ is the value fixed by the evidence. Following the schedule, we have the following Q-messages: p(s1 | s10 )Q 10,1 (s10 ) Q 13 (s1 ) =
(29)
s10
Q 23 (s2 ) =
s11 ,s12
p(s2 | s11 , s12 )Q 11,2 (s11 )Q 12,2 (s12 ).
(30)
248
VICENTE ET AL.
The remaining R-messages are: p(s4 | s3 , s8 )Q 84 (s8 )R74 (s4 ) R43 (s3 ) =
(31)
p(s5 | s3 , s6 )Q 65 (s6 )R5 (s5 ),
(32)
s4 ,s8
R53 (s3 ) =
s6 ,s5
where R5 (s5 ) = δ(s5 , s5∗ ) and s5∗ is the value fixed by the evidence. Finally we can fuse all the messages in the vertex s3 as follows: p(s3 | s1 , s2 , s9 )Q 13 (s1 )Q 23 (s2 )R43 (s3 )R53 (s3 )Q 93 (s9 ). p(s3 ) =
(33)
s1 ,s2 ,s9
By substituting the expressions for the messages in (33), it is relatively straightforward to verify that this expression gives the exact value for the marginal of (26). In this case, the probability propagation algorithm requires only 432 operations against 5622 operations required by the brute force evaluation. We can now summarize the rules for calculating the message that flows through a particular edge: r
r
Multiply all incoming messages by the local probability table (for example: p(s3 | s1, s2, s9) for vertex s3) and sum over all vertices not attached to the edge that carries the outgoing message. Both Q and R messages must be only functions of the parent in the edge through which the message is flowing.
Probability propagation is only exact if the Bayesian network associated has no cycles. However, we can blindly apply the same algorithm in a general graph hoping that convergence to a good approximation is attained. In this kind of application there is no obvious optimal schedule and nodes can be updated serially, in parallel, or randomly. Before writing the probability propagation equations for a general graph, let us first provide some definitions. Two vertices sj and rμ are adjacent if there is an edge connecting them. If there is an arrow from sj to rμ , sj is said to be a parent and rμ a child. The children of sj are denoted by M( j) and the parents of rμ are L(μ). Linear codes are specified by bipartite graphs (as in Fig. 4) where all parents are in one layer and all children in the other layer. A message is a probability vector Q = (Q 0 , Q 1 ) with Q 0 + Q 1 = 1. The probability propagation algorithm in a bipartite graph operates by passing messages between the two layers through the connection edges, first forward from the top layer (parents) to the bottom layer (children), then backward, and so on iteratively. Child-to-parent messages (backward messages in Fig. 4) are R-messages denoted Rμj , while parent-to-child messages (forward messages) are Q-messages denoted by Q jμ .
LOW-DENSITY PARITY-CHECK CODES
249
Figure 7. Left side: forward (Q) message from parent to child. Right side: backward (R) message from child to parent.
With the help of Figure 7 using the algorithm above, the forward (Q) messages between a parent sj and child rμ are just (see also Davey, 1999): Q ajμ = P(S j = a | {Jν : ν ∈ M( j)μ}) Rνaj , = αμj p(s j = a)
(34) (35)
ν∈M( j)\μ
where αμj is a required normalization, M( j) \ μ stands for all elements in the set M( j) except μ. Similarly, we can get the expression for the backward (R) messages between child rμ and parent sj: si a Rμj = P(rμ | s j = a, {si : i ∈ L(μ) \ j}) Q iμ . (36) {si :i∈L(μ)\ j}
i∈L(μ)\ j
An approximation for the marginal posterior can be obtained by iterating Eqs. (34) and (36) until convergence or some stopping criteria is attained, and fusing all incoming information to a parent node by calculating: a Rμj , (37) Q aj = α j p(s j = a) ν∈M( j)
where α j is a normalization Q aj is an approximation for the marginal posterior P(s j | r ). Initial conditions can be set to the prior probabilities Q sjμ = p(s). It is clear (see also Pearl, 1988) that the probability propagation (PP) algorithm is exact if the associated graph is a tree and that the convergence for the exact marginal posterior occurs within a number of iterations proportional to the diameter of the tree. However, graphs defining error-correcting codes always have cycles and it has been observed empirically that decoding with the PP algorithm also yields good results (Frey and MacKay, 1998; Cheng, 1997) in spite of that. There are a number of studies of probability propagation in loopy graphs with a single cycle (Weiss, 1997) and describing Gaussian joint distributions (Freeman, 1999), but no definite explanation for its good performance in this case is known to date.
250
VICENTE ET AL.
D. Low-Density Parity-Check Codes Marginal posteriors can be calculated in O(N K ) steps, where K is the average connectivity of a child node, by using probability propagation. Therefore, the use of very sparse generator matrices ( μj G μj = O(N )) seems favorable. Moreover, it is possible to prove that the probability of a cycle-free path of length l in a random graph decays with O(K l /N ), that indicates that small cycles are harder to find if the generator matrix is very sparse and that PP decoding is expected to provide better approximations for the marginal posterior (no proof is known for this statement). Encoding is also faster if very sparse matrices are used, requiring O(N ) operations. Despite the advantages, the use of very sparse matrices for encoding has the serious drawback of producing codewords that differ in only O(K ) bits from each other, which leads to a high probability of undetectable errors. Codes with sparse generator matrices are known as Sourlas codes and will be our object of study in the next section. A solution for the bad distance properties of sparse generator codes is to use a dense matrix for encoding (providing a minimum distance between codewords of O(N )), while decoding is carried out in a very sparse graph, allowing efficient use of PP decoding. The method known as parity-check decoding (Hill, 1986; Viterbi and Omura, 1979) is suitable in this situation, as encoding is performed by a generator matrix G, while decoding is done by transforming the corrupted received vector r = Gs +n (mod 2) with a suitable parity-check matrix H having the property HG (mod 2) = 0, yielding the syndrome vector z = Hn (mod 2). Decoding reduces to finding the most probable vector n when the syndrome vector z is known, namely, performing MPM estimates that involve the calculation of the marginal posterior P(n j | z). In 1999, MacKay proved that this decoding method can attain vanishing block error probabilities up to the channel capacity if optimally decoded (not necessarily practically decoded). This type of decoding is the basis for the three families of codes (Gallager, MacKay–Neal, and cascading) that we study in this chapter.
E. Decoding and Statistical Physics The connection between spin systems in statistical physics and digital error correcting codes, first noted by Sourlas (1989), is based on the existence of a simple isomorphism between the additive Boolean group ({0, 1}, ⊕) and the multiplicative binary group ({+1, −1}, ·) defined by: S · X = (−1)s⊕x ,
(38)
LOW-DENSITY PARITY-CHECK CODES
251
where S, X ∈ {+1, −1} and s, x ∈ {0, 1}. Through this isomorphism, every addition on the Boolean group corresponds to a unique product on the binary group and vice-versa. A parity-check bit4 in a linear code is usually formed K by a Boolean sum of K bits of the form j=1 s j that can be mapped onto a 3K K-spin coupling j=1 S j . The same type of mapping can be applied to other error-correcting codes as convolutional codes (Sourlas, 1994b; Amic and Luck, 1995) and Turbo codes (Montanari and Sourlas, 2000; Montanari, 2000). The decoding problem depends on posteriors like P(S | J), where J is the evidence (received message or syndrome vector). By applying Bayes’ theorem this posterior can, in general, be written in the form: Pαγ (S | J) =
1 exp ln Pα (J | S) + ln Pγ (S) , Z (J)
(39)
where α and γ are hyperparameters assumed to describe features like the encoding scheme, source distribution, and noise level. This form suggests the following family of Gibbs measures: 1 exp −β Hαγ (S; J) Z Hαγ (S; J) = − ln Pα (J | S) − ln Pγ (S),
Pαβγ (S | J) =
(40) (41)
where J can be regarded as quenched disorder in the system. It is not difficult to see that the MAP estimator is represented by the ground state of the Hamiltonian (40), i.e., by the sign of thermal averages Sˆ jMAP = sgn(S j β→∞ ) at zero temperature. On the other hand, the MPM estimator is provided by the sign of thermal averages Sˆ jMPM = sgn(S j β=1 ) at temperature one. We have seen that if we are concerned with the probability of bit error pe the optimal choice for an estimator is MPM, this is equivalent to decoding at finite temperature β = 1, known as the Nishimori temperature (Nishimori, 1980, 1993, 2001; Ruj´an, 1993). The evaluation of typical quantities involves the calculation of averages over the quenched disorder (evidence) J, namely, averages over: Pα∗ γ ∗ (J) = Pα∗ (J | S)Pγ ∗ (S), (42) S
∗
∗
where α and γ represent the “real” hyperparameters, in other words, the hyperparameters actually used for generating the evidence J. Those “real” hyperparameters are, in general, not known to the receiver, but can be estimated from the data. To calculate these estimates we can start by writing free-energy like negative log-likelihoods for the hyperparameters: F(α, γ ) Pα∗ γ ∗ = −lnPαγ (J) Pα∗ γ ∗ .
(43)
252
VICENTE ET AL.
This log-likelihood can be regarded as measuring the typical plausibility of α and γ , given the data J (Berger, 1993). This function can be minimized to find the most plausible hyperparameters (known as type II maximum likelihood hyperparameters or just ML-II hyperparameters) (Berger, 1993). The ML-II hyperparameters correspond in this case to α = α ∗ and γ = γ ∗ , i.e., the “real” hyperparameters must be used in the posterior for decoding. This fact is a consequence of the following inequality: F(α ∗ , γ ∗ ) Pα∗ γ ∗ ≤ F(α, γ ) Pα∗ γ ∗ .
(44)
The proof of (44) follows directly from the information inequality (Iba, 1999; Cover and Thomas, 1991), i.e., the nonnegativity of the KL-divergence: D(Pα∗ γ ∗ Pαγ ) ≥ 0 %. - $ Pα∗ γ ∗ (J) ≥0 ln Pαγ (J) Pα∗ γ ∗ −ln Pα∗ γ ∗ (J) Pα∗ γ ∗ ≤ −ln Pαγ (J) Pα∗ γ ∗ .
(45)
When the true and assumed hyperparameters agree, we say that we are at the Nishimori condition (Iba, 1999; Nishimori, 2001). At the Nishimori condition many calculations simplify and can be done exactly (for an example, see Appendix B.3). Throughout this chapter we assume, unless otherwise stated, the Nishimori condition. For background reading about statistical physics methods in general, Nishimori’s condition, and its relevance to the current calculation we refer the reader to Nishimori (2001).
III. Sourlas Codes The code of Sourlas is based on the idea of using a linear operator G (generator matrix) to transform a message vector s ∈ {0, 1} N onto a higher dimensional vector t ∈ {0, 1} M . The encoded vector is t = Gs (mod 2), each bit tk being the Boolean sum of K message bits (parity-check). This vector is transmitted through a noisy channel and a corrupted M dimensional vector r is received. Decoding consists of producing an estimate sˆ of the original message. This estimate can be generated by considering a probabilistic model for the communication system. Reduced (order N) time/space requirements for the encoding process and the existence of fast (polynomial time) decoding algorithms are guaranteed by choosing sparse generator matrices, namely, a matrix G with exactly K nonzero elements per row and C nonzero elements per column, where K and C are of order 1. The rate of such a code, in the case of unbiased
LOW-DENSITY PARITY-CHECK CODES
253
messages, is evidently R = N /M, as the total number of nonzero elements in G is M K = N C the rate is also R = K /C. In the statistical physics language a binary message vector ξ ∈ {±1} N is encoded to a higher dimensional vector J 0 ∈ {±1} M defined as Ji01 ,i2 ...i K = ξi1 ξi2 · · · ξi K , where M sets of K indices are randomly chosen. A corrupted version J of the encoded message J 0 has to be decoded for retrieving the original message. The decoding process is the process of calculating an estimate ξˆ to the original message by minimizing a given expected loss L(ξ, ξˆ ) P(J |ξ ) P(ξ ) averaged over the indicated probability distributions (Iba, 1999). Thedefinition of the loss depends on the particular task; the overlap L(ξ, ξˆ ) = j ξ j ξˆ j can be used for decoding binary messages. As discussed in Section II.B, an optimal estimator for this particular loss function is ξˆ j = signS j P(S j |J ) (Iba, 1999), where S is an N-dimensional binary vector representing the dynamic variables of the decoding process and P(S j | J) = Sk ,k= j P(S | J) is the marginal posterior probability. Using Bayes theorem, the posterior probability can be written as: ln P(S | J) = ln P(J | S) + ln P(S) + const. The likelihood P(J | S) has the form: 5 5 P(J | S) = P Ji1···i K 5 Ji01···i K P Ji01···i K 5 S . chosensets Ji0 ···i 1
The term
P(Ji01···i K
(46) (47)
K
| S) models the deterministic encoding process being: 5 (48) P Ji01···i K 5 S = δ Ji01···i K ; Si1 · · · Si K .
The noisy channel is modeled by the term P(Ji1···i K | Ji01···i K ). For the simple case of a memoryless binary symmetric channel (BSC), J is a corrupted version of the transmitted message J 0 where each bit is independently flipped with probability p during transmission, in this case (Sourlas, 1994a): 5 5 1 ln P Ji1···i K 5 Ji01···i K = 1 + Ji01···i K ln P Ji1···i K 5 + 1 2 5 1 + 1 − Ji01···i K ln P Ji1···i K 5 − 1 2 % $ 1− p 1 Ji1···i K Ji01···i K . (49) = const + ln 2 p Putting equations together, we obtain the following Hamiltonian: ln P(S | J) = −β N H(S) + const = βN
μ
Aμ Jμ
i∈L(μ)
(50) Si + β N′
N j=1
S j + const,
(51)
254
VICENTE ET AL.
where a set of indices is denoted L(μ) = i 1 , . . . i K and A is a tensor with the properties Aμ ∈ {0, 1} and {μ:i∈L(μ)} Aμ = C∀i, which determines the M components of the codeword J 0 . The interaction term is at Nishimori’s temperature β N = 21 ln( 1−p p ) (Nishimori, 1980, 1993; Iba, 1999; Ruj´an, 1993), 1− p and β N′ = 21 ln( pξ ξ ) is the message prior temperature, namely, the prior distribution of message bits is assumed to be P(S j = +1) = 1 − pξ and P(S j = −1) = pξ . The decoding procedure translates to finding the thermodynamic spin averages for the system defined by the Hamiltonian (50) at a certain temperature (Nishimori temperature for optimal decoding); as the original message is binary, the retrieved message bits are given by the signs of the corresponding averages. The performance of the error-correcting process can be measured by the overlap between actual message bits and their estimates for a given scenario characterized by code rate, corruption process, and information content of the message. To assess the typical properties, we average this overlap over all possible codes A and noise realizations (possible corrupted vectors J) given the message ξ and then over all possible messages: 6 7 N 1 (52) ξi signSi A,J |ξ ρ= N i=1 ξ
Here signSi is the sign of the spins thermal average corresponding to the Bayesian optimal decoding. The average error per bit is, therefore, given by pb = (1 − ρ)/2. The number of checks per bit is analogous to the spin system connectivity and the number of bits in each check is analogous to the number of spins per interaction. The code of Sourlas has been studied in the case of extensive connectivity, where the number of bonds C ∼ KN −1 scales with the system −1 size. In this case it can be mapped onto known problems in statistical physics such as the SK (Kirkpatrick and Sherrington, 1978) (K = 2) and random energy (REM) (Derrida, 1981a) (K → ∞) models. It has been shown that the REM saturates Shannon’s bound (Sourlas, 1989). However, it has a rather limited practical relevance as the choice of extensive connectivity corresponds to a vanishingly small code rate.
A. Lower Bound for the Probability of Bit Error It has been observed in Montanari and Sourlas (2000) that a sparse generator code can only attain vanishing probability of bit error if K → ∞. This fact alone does not rule out the practical use of such codes as they can still be
LOW-DENSITY PARITY-CHECK CODES
255
used if a controlled probability of error is allowed or as part of a concatenated code. Before engaging in a relatively complex analysis, it is of theoretical interest to establish a detailed picture of how the minimum bit error attainable decays with K. This can be done in quite a simple manner suggested in Montanari and Sourlas (2000). Let us suppose that messages are unbiased and random and that the channel is a BSC of noise level p. Assume, without loss of generality, that the message ξ j = 1 for j is sent. The bit error probability can be all N expressed as the sum pb = l=1 pb (l), where pb (l) represents the probability of decoding incorrectly any l bits. Clearly pb ≥ pb (1). The probability of decoding incorrectly a single bit can be easily evaluated. A bit j engages in exactly C interactions with different groups of K bits in a way that their contribution to the Hamiltonian is: H j = −S j Jμ Si , (53) μ∈M( j)
i∈L(μ)\ j
where M( j) is the set of all index sets that contain j. If all bits but j are set to Si = 1, an error in j only can be detected if its contribution to the Hamiltonian is positive; if μ∈M( j) Aμ Jμ ≤ 0 the error is undetectable. The probability of error in a single bit is therefore 8 9 pb (1) = P Jμ ≤ 0 , (54) μ∈M( j)
where Aμ = 1 for exactly C terms and Jμ can be simply regarded as a random variable taking values +1 and −1 with probabilities 1 − p and p, respectively; therefore: pb ≥
l≤C
C! (1 − p)C−l pl . (C − l)! l! l∈N,C−2l≤0
(55)
A lower bound for for pb in the large C regime can be obtained by using the DeMoivre–Laplace limit theorem (Feller, 1950), writing: $ % $ % (1 − p)C 4p (1 − p)2 C 2 1 pb ≥ erfc exp − ≈√ , (56) 2 8p 64 p 2 π (1 − p)C (∞ where erfc(x) = √2π x du exp(−u 2 ) and the asymptotic behavior is given in Gradshteyn and Ryzhik (1994, page 940). This bound implies that K → ∞ is a necessary condition for a vanishing bit error probability in sparse generator codes at finite rates R = K /C.
256
VICENTE ET AL.
B. Replica Theory for the Typical Performance of Sourlas Codes To calculate the typical performance of Sourlas codes we employ the statistical physics technique known as replica theory. To simplify analysis we use the gauge transformation (Fradkin et al., 1978) Si "→ Si ξi and Ji1···i K "→ Ji1···i K ξi1 · · · ξi K that maps any general message to the configuration defined as ξi∗ = 1∀i (ferromagnetic configuration). By introducing the external field F ≡ β N′ /β we rewrite the Hamiltonian in the form:
H(S) = −
i 1···i K
Ai1···i K Ji1···i K Si1 · · · Si K − F
N
ξj Sj,
(57)
j=1
With the gauge transformation, the bits of the uncorrupted encoded message become Ji01···i K = 1 and, for the BSC, the corrupted bits can be described as random variables with probability: P (J ) = (1 − p) δ(J − 1) + p δ(J + 1),
(58)
where p is the channel flip rate. For deriving the typical properties we calculate the free-energy following the replica theory prescription: 5 1 1 ∂ 55 f = − lim Z n A,ξ,J , (59) β N →∞ N ∂n 5n=0
where Z n A,ξ,J represents an analytical continuation in the interval n ∈ [0, 1] of the replicated partition function: ; : α α α Z n A,ξ,J = Tr{S αj } eβ F α,k ξk Sk +β α,μ Aμ Jμ Si1···Si K
A,J,ξ
.
(60)
The overlap ρ can be rewritten using gauged variables as: ρ=
N < = 1 signSi A,J |ξ ∗ ξ , N i=1
(61)
where ξ ∗ denotes the transformation of a message ξ into the ferromagnetic configuration. To compute the replicated partition function we closely follow Wong and Sherrington (1987a). We average uniformly over all codes A such that
257
LOW-DENSITY PARITY-CHECK CODES
i 1 =i,i 2 ···i K
Ai1···i K = C∀i to find: 8
n
Z A,ξ,J = exp N Extrq,qˆ −C
n
l=0 α1 ··· αl
n C C K + Tl q C− K K l=0 α1 ··· αl α1 ··· αl qα1 ··· αl qˆ α1···αl
C !9 n < β Fξ S α = α qˆ α1 ··· αl S α1 . . . S αl + ln Tr{S α } e , ξ l=0 α1 ··· αl
(62)
where Tl = tanhl (β J ) J , as in Viana and Bray (1985), q0 = 1 and Extrq,qˆ ˆ denotes the extremum of f (details in Appendix A.1). At the extremum f (q, q) of (62) the order parameters acquire a form similar to those of Wong and Sherrington (1987a): qˆ α1 ,...,αl = Tl qαK1−1 ,...,αl 6 −1 7 n l α1 αl αi qˆ α1 ··· αl S · · · S . qα1 ,...,αl = S i=1
l=0 α1 ··· αl
(63)
X
where C n < β Fξ S α = α α α , X = e qˆ α1 ··· αl S 1 · · · S l ξ
(64)
l=0 α1 ··· αl
and · · ·X = Tr{Sα } (· · ·)X /Tr{Sα } [(· · ·)]. To compute the partition function it is necessary to assume a replica symmetric (RS) ans¨atz. It can be done by introducing auxiliary fields π (x) and π(y) ˆ (see also Wong and Sherrington, 1987a): ) qˆ α1 ··· αl = dy π(y) ˆ tanhl (βy), qα1 ··· αl = for l = 1, 2, . . . .
)
d x π (x) tanhl (βx)
(65)
258
VICENTE ET AL.
Plugging (65) into the replicated partition function (62), taking the limit n → 0 and using Eq. (59) (see Appendix A.2 for details): 1 f = − Extrπ,πˆ {α ln cosh β β !6 !7 ) K K d xl π (xl ) ln 1 + tanh β J tanh βx j +α l=1
−C −C +
)
)
j=1
J
d x d y π(x) πˆ (y) ln [1 + tanh βx tanh βy]
)
dy π(y) ˆ ln cosh βy C
dyl πˆ (yl )
l=1
!6
ln 2 cosh β
C j=1
y j + Fξ
!7 ⎫ ⎬ ξ
⎭
, (66)
where α = C/K . The saddle-point equations obtained by calculating functional variatons of Eq. (66) provide a closed set of relations between π (x) and π(y) ˆ
π (x) =
)
π(y) ˆ =
)
C−1
dyl π(y ˆ l)
l=1
K −1
!6
d xl π (xl )
l=1
δ x−
!6
δ y−
C−1 j=1
y j − Fξ
!7
ξ
atanh tanh β J
3 K −1 j=1
tanh βx j
β
!7
. J
(67)
Later we will show that this self-consistent pair of equations can be seen as a mean-field description of probability propagation decoding. Using the RS ans¨atz one can find that the local field distribution is (see Appendix A.3):
P(h) =
)
C l=1
!6
dyl πˆ (yl )
δ h−
C j=1
y j − Fξ
where πˆ (y) is given by the saddle-point equations (67).
!7
ξ
,
(68)
LOW-DENSITY PARITY-CHECK CODES
The overlap (52) can be calculated using: ) ρ = dh sign(h) P(h).
259
(69)
The code performance is assessed by assuming a prior distribution for the message, solving the saddle-point equations (67) numerically and then computing the overlap. For Eq. (66) to be valid, the fixed point given by (67) must be stable and the related entropy must be nonnegative. Instabilities within the RS space can be probed by calculating second functional derivatives at the extremum defining the free-energy (66). The solution is expected to be unstable within the space of symmetric replicas for sufficiently low temperatures (large β). For high temperatures we can expand the above expression around small β values to find the stability condition: (
J J xπK −2 ≥ 0
(70)
The average xπ = d x π(x)x vanishes in the paramagnetic phase and is positive (nonzero when K is even) in the ferromagnetic phase, satisfying the stability condition. We now restrict our study to the unbiased case (F = 0), which is of practical relevance, since it is always possible to compress a biased message to an unbiased one. For the case K → ∞, C = α K we can obtain solutions to the saddle-point equations at arbitrary temperatures. The first saddle-point Eq. (67) can be approximated by: ) C−1 yl ≈ (C − 1)yπˆ = (C − 1) dy y π(y). ˆ (71) x= l=1
If yπˆ = 0 (paramagnetic phase) then π(x) must be concentrated at x = 0 implying that π(x) = δ(x) and π(y) ˆ = δ(y) are the only possible solutions. Equation (71) also implies that x ≈ O(K ) in the ferromagnetic phase. Using Eq. (71) and the second saddle-point Eq. (67) we find a self-consistent equation for the mean field yπˆ : . 1 tanh[tanh(β J ) [tanh(β(C − 1)yπˆ )] K −1 ] . (72) yπˆ = β J For the BSC we average over the distribution (58). Computing the average, ˜ K )/K , we obtain in using C = α K and rescaling the temperature β = β(ln the limit K → ∞: K ˜ yπˆ ≈ (1 − 2 p) tanh(βαy , (73) πˆ ln(K ))
260
VICENTE ET AL.
where p is the channel flip probability. The mean field yπˆ = 0 is always a solution to this equation (paramagnetic solution); at βc = ln(K )/(2α K (1 − 2 p)) an extra nontrivial ferromagnetic solution emerges with yπˆ = 1 − 2 p. The connection with the overlap ρ is given by Eqs. (68) and Eq. (69) implying that ρ = 1 for the ferromagnetic solution. It is remarkable that the temperature where the ferromagnetic solution emerges is βc ∼ O(ln(K )/K ). Paramagnetic–ferromagnetic barriers emerge at reasonably high temperatures, in a simulated annealing process, implying metastability and, consequently, a very slow convergence. It seems to advocate the use of small K values in practical applications. For β > βc both paramagnetic and ferromagnetic solutions exist. The ferromagnetic free-energy can be obtained from Eq. (66) using Eq. (71), resulting in f FERRO = −α(1 − 2 p). The corresponding entropy is sFERRO = 0. The paramagnetic free-energy is obtained by plugging π (x) = δ(x) and π(y) ˆ = δ(y) into Eq. (66): 1 f PARA = − (α ln(cosh β) + ln 2), β
(74)
sPARA = α(ln(cosh β) − β tanh β) + ln 2.
(75)
Paramagnetic solutions are unphysical for α > (ln 2)/ [β tanh β − ln(cosh β)], since the corresponding entropy is negative. To complete the picture of the phase diagram we have to introduce a replica symmetry breaking scenario that yields sensible physics. In general, to construct a symmetry breaking solution in finite connectivity systems (see Monasson, 1998b; Franz et al., 2001) is a difficult task. We choose as a first approach a one-step replica symmetry breaking scheme, known as the frozen spins solution, that yields exact results for the REM (Gross and Mezard, 1984; Parisi, 1980). We assume that ergodicity breaks in such a way that the space of configurations is divided in n/m islands. Inside each of these islands there are m identical configurations, implying that the system can freeze in any of n/m microstates. Therefore, in the space of replicas we have the following situation: N 1 β S αj S j = 1, if α and β are in the same island N j=1 N 1 β S α S = q, otherwise. N j=1 j j
(76)
261
LOW-DENSITY PARITY-CHECK CODES
By assuming the above structure the replicated partition function has the form: 6 7 n < n = α H(S ) ZRSB A,ξ,J = Tr{Sαj } exp −β α=1
6
= TrA S 1 ,...,S n/m B exp −βm = =
6 <
j
n/m α
j
A,J,ξ
n/m
α
H(S )
α=1
Tr{S αj } exp (−βm H(Sα ))
7
7
A,J,ξ
A,J,ξ
n/m = ZRS , A,ξ,J
(77)
where in the first line we have used the ans¨atz with n/m islands with m identical configurations in each and in the last step we have used that the overlap between any two different islands is q. From (77) we have: 5 = ∂ 55 < n ln ZRSB (β)A,ξ,J = Z (β) A,ξ,J ∂n 5n=0 RSB
1 ln ZRS (βm)A,ξ,J . (78) m The number of configurations per island m must extremize the free-energy, therefore, we have: =
∂ ln ZRSB (β)A,ξ,J = 0, ∂m
(79)
what is equivalent to
5 # " 5 1 2 ∂ 5 ˜ ˜ sRS (βg ) = −β ln ZRS (β)A,ξ,J ˜ ∂ β˜ 5β=β ˜ g β = 0,
(80)
where we introduced β˜ = βm. In this way m = βg /β, with βg being a root of the replica symmetric paramagnetic entropy (74), satisfying: α(ln(cosh βg ) − βg tanh βg ) + ln 2 = 0
(81)
The RSB-spin glass free-energy is given by fPARA (74) at temperature β g: f RSB-SG = −
1 (α ln (cosh βg ) + ln 2), βg
(82)
consequently the entropy is sRSB-SG = 0. In Figure 8 we show the phase diagram for a given code rate R in the plane of temperature T and noise level p.
262
VICENTE ET AL.
Figure 8. Phase diagram in the plane of temperature T versus noise level p for K → ∞ and C = α K , with α = 4. The dotted line indicates the Nishimori temperature TN. Full lines represent phase coexistence. The critical noise level is pc. The necessary condition for stability of the ferromagnetic phase within the replica symmetric space is satisfied above the dashed line.
C. Shannon’s Bound The channel-coding theorem asserts that up to a critical code rate Rc, which equals the channel capacity (Shannon’s bound), it is possible to recover information with arbitrarily small probability of error. For the BSC: Rc =
1 = 1 + p log2 p + (1 − p) log2 (1 − p). αc
(83)
The code of Sourlas, in the case where K → ∞ and C ∼ O(N K ), can be mapped onto the REM and has been shown to saturates the channel capacity in the limit R → 0 (Sourlas, 1989). Shannon’s bound can also be attained by Sourlas code at zero temperature for K → ∞ but with connectivity C = α K . In this limit the model is analogous to the diluted REM analyzed by Saakian (1998). The errorless phase is manifested in a ferromagnetic phase with total alignment (ρ = 1), only attainable for infinite K. Up to a certain critical noise level, a noise level increase produces ergodicity breaking leading to a spin glass phase where the misalignment is maximal (ρ = 0). The ferromagnetic– spin glass transition corresponds to the transition from errorless decoding to decoding with errors described by the channel coding theorem. A paramagnetic
263
LOW-DENSITY PARITY-CHECK CODES
phase is also present when the transmitted information is insufficient to recover the original message (R > 1). At zero temperature, saddle-point Eq. (67) can be rewritten as: ! ! ) C−1 C−1 π (x) = dyl πˆ (yl ) δ x − yj (84) l=1
πˆ (y) =
)
6
K −1
j=1
!
d xl π (xl )
l=1
× δ y − sign J
K −1 l=1
!7
xl min(| J |, . . . , | x K −1 |)
,
(85)
J
The solutions for these saddle-point equations may result in very structured probability distributions. As an approximation we choose the simplest selfconsistent family of solutions which are, since J = ±1, given by: π(y) ˆ = p+ δ(y − 1) + p0 δ(y) + p− δ(y + 1) π(x) = with
C−1
l=1−C
(86)
T[ p± , p0 ;C−1] (l) δ(x − l),
T[ p+ , p0 , p− ;C−1] (l) =
′ (C − 1)! k h m p p p , k! h! m! + 0 − {k,h,m}
(87)
where the prime indicates that k,h,m are such that k − h = l; k + h + m = C − 1. Evidence for this simple ans¨atz comes from Monte Carlo integration of Eq. (67) at very low temperatures, that shows solutions comprising three dominant peaks and a relatively weak regular part. Plugging this ans¨atz (86) in the saddle-point equations, we write a closed set of equations in p± and p0 that can be solved numerically. Solutions are of three types: ferromagnetic ( p+ > p− ), paramagnetic ( p0 = 1), and replica symmetric spin glass ( p− = p+ ). Computing freeenergies and entropies enables one to construct the phase diagram. At zero temperature, the paramagnetic free-energy is f PARA = −α and the entropy is sPARA = (1 − α) ln 2; this phase is physical only for α < 1, as is expected since it corresponds exactly to the regime where the transmitted information is insufficient to recover the actual message (R > 1). The ferromagnetic free-energy does not depend on the temperature, having the form f FERRO = −α(1 − 2 p) with entropy sFERRO = 0. We can find the ferromagnetic–spin glass coexistence line that corresponds to the maximum performance of a Sourlas code by equating Eq. (82) and fFERRO. Observing
264
VICENTE ET AL.
Figure 9. Phase diagram in the plane code rate R versus noise level p for K → ∞ and C = α K at zero temperature. The ferromagnetic–spin glass coexistence line corresponds to Shannon’s bound.
that βg = β N ( pc ) (as seen in Fig. 8) we find that this transition coincides with the channel capacity (83). It is interesting to note that in the large K regime both RS–ferromagnetic and RSB–spin glass free-energies (for T < Tg ) do not depend on the temperature, it means that Shannon’s bound is saturated also for finite temperatures up to Tg. In Figure 9 we represent the complete zero temperature phase diagram. The bound obtained depends on the stability of the ferromagnetic and paramagnetic solutions within the space of symmetric replicas at zero temperature. Instabilities are found in the ferromagnetic phase for p > 0. These instabilities within the replica symmetric space puts in question our result of saturating Shannon’s bound, since a correction to the ferromagnetic solution could change the ferromagnetic–spin glass transition line. However, the instability vanishes for high temperatures, which supports the ferromagnetic–spin glass transition line obtained and possible saturation of the bound in some region. Shannon’s bound can only be attained in the limit K → ∞; however, there are some possible drawbacks in using high K values due to large barriers which are expected to occur between the paramagnetic and ferromagnetic phases. We now consider the finite K case, for which we can solve the RS saddle-point Eqs. (67) for arbitrary temperatures using Monte Carlo integration. We can also obtain solutions for the zero temperature case using Eqs. (86) iteratively.
LOW-DENSITY PARITY-CHECK CODES
265
Figure 10. Top: zero temperature overlap ρ as a function of the noise level p for various K values at code rate R = 1/2, as obtained by the iterative method. Bottom: RS-ferromagnetic free-energies (white circles for K = 2 and from the left: K = 3, 4, 5, and 6) and RSB-spin glass free-energy (dotted line) as functions of the noise level p. The arrow indicates the region where the RSB–spin glass phase starts to dominate. Inset: a detailed view of the RS–RSB transition region.
It has been shown that K > 2 extensively connected models (Gross and Mezard, 1984) exhibit Parisi-type order functions with similar discontinuous structure as found in the K → ∞ case; it was also shown that the one-step RSB frozen spins solution, employed to describe the spin glass phase, is locally stable within the complete replica space and zero field (unbiased messages case) at all temperatures. We, therefore, assume that the ferromagnetic–spin glass transition for K > 2 is described by the frozen spins RSB solution. At the top of Figure 10 we show the zero temperature overlap ρ as a function of the noise level p at code rate R = 1/2 obtained by using the three-peaks ans¨atz. Note that the RSB–spin glass phase dominates for p > pc (see bottom of Fig. 10). In the bottom figure we plot RS free-energies and RSB frozen spins free-energy, from which we determine the noise level pc for coexistence of ferromagnetic and spin-glass phases (pointed by an arrow). Above the transition, the system enters in a paramagnetic or RS spin glass phase with free-energies for K = 3, 4, 5, and 6 that are lower than the RSB spin glass free-energy; nevertheless, the entropy is negative and these free-energies are therefore unphysical. It is remarkable that the coexistence value does not
266
VICENTE ET AL.
change significantly for finite K in comparison to infinite K. Remind that Shannon’s bound cannot be attained for finite K, since ρ → 1 ( pb → 0) only if K → ∞. It is known that the K = 2 model with extensive connectivity (SK model) requires a full Parisi solution to recover the concavity of the free-energy (Mezard et al., 1987). No stable solution is known for the intensively connected model (Viana–Bray model). Probability propagation only solves the decoding problem approximately, the approximated solutions are similar to those obtained by supposing replica symmetry. Thus, the theoretical relevance of the RS results for K = 2 are to be evaluated by comparison with simulations of probability propagation decoding.
D. Decoding with Probability Propagation ˆ The decoding task consists of evaluating estimates of the form ξ j = signS j P(S j |J ) . The marginal posterior P(S j | J) = Sl ,l= j P(S | J) can be, in principle, calculated simply by using Bayes theorem and a proper model for the encoding and corruption processes (namely, coding by a sparse generator matrix with K bit long parity-checks and a memoryless BSC channel) to write:
P(S j | J) =
N 1 P(Si ), P(Jμ | Si1 · · · Si K ) P(J) Sl ,l= j μ i=1
(88)
where P(J) is a normalization dependent on J only. A brute force evaluation of the above marginal on a space of binary vectors S ∈ {±1} N with M checks would take (M + N + 1)2 N operations, what becomes infeasible very quickly. To illustrate how dramatically the computational requirements increase, assume a code of rate R = 1/2, if N = 10 the number of operations required is 31,744, if one increases the message size to N = 1000, 3 × 10304 operations are required! Monte Carlo sampling is an alternative to brute force evaluation; it consists of generating a number (much less than 2N) of typical vectors S. By using this to estimate the marginal posterior, however, the sample size required can prove to be equally prohibitive. As a solution to these resource problems, we can explore the structure of (88) to devise an algorithm that produces an approximation to P(S j | J) in O(N ) operations. We start by concentrating on one particular site Sj; this site interacts directly with a number of other sites through C couplings denoted by Ji1··· i K and {Jμ } = Jμ(1) , . . . , Jμ(C−1) . Suppose now that we isolate only the interaction via coupling Ji1···i K , if the bipartite Bayesian network representing
267
LOW-DENSITY PARITY-CHECK CODES
the dependencies in the problem is a tree, it is possible to write: 5 P(S j ) P Ji1···i K 5 S j , Si1 · · · Si K −1 P S j | Ji1···i K = P Ji1···i K {Si ···Si } 1
×
K −1 l=1
K −1
P(Sil | {Jμ : μ ∈ M(il )}).
(89)
Terms like P(Sil | {Jμ }) can be interpreted simply as updated priors for Sil . 3 In a tree, these terms factorize like P(Sil | {Jμ }) = C−1 j=1 P(Sil | Jμ( j) ) and a recursive relation can be obtained, introducing: Q νx j = P(S j = x | {Jμ : μ ∈ M( j) \ ν})
and Rνx j =
{Si :i∈L(ν)\ j}
P(Jν | S j , {Si : i ∈ L(ν) \ j})
(90)
Si Q νi ,
(91)
i∈L(ν)\ j
where M( j) is the set of couplings linked to site j and L(ν) is the set of sites linked to coupling ν. Equation (89) can be rewritten as: x = aμj P(S j = x) Rνx j . (92) Q μj ν∈M( j)\μ
Equations (91) and (92) can be solved iteratively, requiring (2 K K C + 2C 2 )N T operations with T being the (order 1) number of steps needed for convergence. These computational requirements may be further reduced by using Markov chain Monte Carlo methods (MacKay, 1999). An approximation to the marginal posterior (88) is obtained by counting the influence of all C interactions over each site j and using the assumed factorization property to write: Rνx j . (93) Q xj = a j P(S j = x) ν∈M( j)
This is an approximation in the sense that the recursion obtained from (89) is only guaranteed to converge to the correct posterior if the system has a tree structure, i.e., every coupling appears only once as one goes backwards in the recursive chain. By taking advantage of the normalization conditions for the distributions −1 +1 −1 Q +1 μj + Q μj = 1 and Rμj + Rμj = 1, one can change variables and reduce −1 ˆ μj = the number of equations by a factor of two m μj = Q +1 μj − Q μj and m +1 −1 Rμj − Rμj .
268
VICENTE ET AL.
The analogy with statistical physics can be exposed by first observing that: P(Jμ | S j , {Si : i ∈ L(μ) \ j}) ∼ exp − β Jμ (94) Si . i∈L(μ)
That can be also written in the more convenient form: 1 Sj . P(Jμ | S j , {Si : i ∈ L(μ) \ j}) ∼ cosh(β Jμ ) 1 + tanh(β Jμ ) 2 j∈L(μ)
(95)
Plugging Eq. (95) for the likelihood in Eqs. (92), using the fact that the prior probability is given by P(S j ) = 12 (1 + tanh(β N′ S j )) and computing m μj and mˆ μj (see Appendix A.6) one obtains: mˆ μj = tanh(β Jμ ) m μl l∈L(μ)\ j
m μj = tanh
ν∈M(l)\μ
atanh(mˆ ν j ) + β N′ .
The pseudo-posterior can then be calculated: ′ atanh(mˆ ν j ) + β N , m j = tanh
(96)
(97)
ν∈M(l)
providing Bayes optimal decoding ξˆ j = sign(m j ). Equations (96) depend on the received message J. In order to make the analysis message independent, we can use a gauge transformation mˆ μj "→ ξ j mˆ μj and m μj "→ ξ j m μj to write: mˆ μj = tanh(β J ) m μl l∈L(μ)\ j
m μj = tanh
ν∈M(l)\μ
tanh−1 (mˆ ν j ) + β N′ ξ j .
(98)
In the new variables, a decoding success corresponds to mˆ μj > 0 and m μj = 1 for all μ and j. By transforming these variables as mˆ = tanh(βy) and m = tanh(βx) and considering the actual message and noise as quenched disorder,
LOW-DENSITY PARITY-CHECK CODES
269
Eqs. (98) can be rewritten as: 6 7 K −1 1 tanh−1 tanh(β J ) tanh(βx j ) y= β j=1 J 6 7 C−1 x= yj + ξ F . j=1
(99)
ξ
For a large number of iterations, one can expect the ensemble of probability networks to converge to an equilibrium distribution where mˆ and m are random ˆ variables sampled from distributions φ(y) and φ(x), respectively. The above ˆ relations lead to a dynamics of the distributions φ(y) and φ (x), that is exactly as the one obtained when solving iteratively RS saddle-point Eqs. (67). The ˆ probability distributions φ(y) and φ (x) can be, therefore, identified with πˆ (y) and π (x), respectively, and the RS solutions correspond to decoding a generic message using probability propagation averaged over an ensemble of different codes, noise, and signals. Equations (96) are now used to show the agreement between the simulated decoding and analytical calculations. For each run, a fixed code is used to generate 20000-bit codewords from 10000-bit messages; corrupted versions of the codewords are then decoded using (96). Numerical solutions for 10 individual runs are presented in Figures 11 and 12, initial conditions are chosen as mˆ μl = 0 and m μl = tanh(β N′ ) reflecting the prior beliefs. In Figure 11 we show results for K = 2 and C = 4 in the unbiased case, at code rate R = 1/2 (prior probability P(S j = +1) = pξ = 0.5) and low temperature T = 0.26 (we avoided T = 0 due to numerical difficulties). Solving the saddle-point Eqs. (67) numerically using Monte Carlo integration methods we obtain solutions with good agreement to simulated decoding. In the same figure we show the performance for the case of biased messages (P(S j = +1) = pξ = 0.1), at code rate R = 1/4. Also here the agreement with Monte Carlo integrations is satisfactory. The third curve in Figure 11 shows the performance for biased messages at the Nishimori temperature TN, as expected, it is far superior compared to low temperature performance and the agreement with Monte Carlo results is even better. In Figure 12 we show the results obtained for K = 5 and C = 10. For unbiased messages the system is extremely sensitive to the choice of initial conditions and does not perform well on average even at the Nishimori temperature. For biased messages ( pξ = 0.1, R = 1/4) results are far better and in agreement with Monte Carlo integration of the RS saddle-point equations. The experiments show that probability propagation methods may be used successfully for decoding Sourlas-type codes in practice, and provide solutions that are consistent with the RS analytical solutions.
270
VICENTE ET AL.
Figure 11. Overlap as a function of the flip probability p for decoding using TAP equations for K = 2. From the bottom: Monte Carlo solution of the RS saddle-point equations for unbiased message ( pξ = 0.5 at T = 0.26 (line) and 10 independent runs of TAP decoding for each flip probability (plus signs), T = 0.26 and biased messages ( pξ = 0.5) at the Nishimori temperature TN.
IV. Gallager Codes In 1962, Gallager proposed a coding scheme which involves sparse linear transformations of binary messages in the decoding stage, while encoding uses a dense matrix. His proposal was overshadowed by convolutional codes due to computational limitations. The best computer available to Gallager in 1962 was an IBM 7090 costing $3 million and with disk capacity of 1 megabyte, while convolutional codes, in comparison, only demanded a simple system of shift registers to process one byte at a time. Gallager codes have been rediscovered recently by MacKay and Neal (1995) who proposed a closely related code, to be discussed in Section V. This almost coincided with the breakthrough discovery of high performance Turbo codes (Berrou et al., 1993). Variations of Gallager codes have displayed performance comparable (sometimes superior) to Turbo codes (Davey, 1998, 1999), qualifying them as state-of-the-art codes. A Gallager code is defined by a binary matrix A = [C1 | C2 ], concatenating two very sparse matrices known to both sender and receiver, with C 2 (of dimensionality (M − N ) × (M − N )) being invertible and C 1 of dimensionality (M − N ) × N . A non-systematic Gallager code is defined by a random
LOW-DENSITY PARITY-CHECK CODES
271
Figure 12. Overlap as a function of the flip probability p for decoding using TAP equations for K = 5. The dotted line is the replica symmetric saddle-point equations Monte Carlo integration for unbiased messages ( pξ = 0.5) at the Nishimori temperature TN. The bottom error bars correspond to 10 simulations using the TAP decoding. The decoding performs badly on average in this scenario. The upper curves are for biased messages ( pξ = 0.1) at the Nishimori temperature TN. The simulations agree with results obtained using the replica symmetric ans¨atz and Monte Carlo integration.
matrix A of dimensionality (M − N ) × M. This matrix can, in general, be organized in a systematic form by eliminating a number ǫ ∼ O(1) of rows and columns. Encoding refers to the generation of an M dimensional binary vector t ∈ {0, 1} M (M > N ) from the original message ξ ∈ {0, 1} N by t = G T ξ (mod 2),
(100)
where all operations are performed in the field {0, 1} and are indicated by (mod 2). The generator matrix is G = [I | C −1 2 C 1 ] (mod 2),
(101)
where I is the N × N identity matrix, implying that AG T (mod 2) = 0 and that the first N bits of t are set to the message ξ. Note that the generator matrix is dense and each transmitted parity-check carries information about an O(N ) number of message bits. In regular Gallager codes the number of nonzero elements in each row of A is chosen to be exactly K . The
272
VICENTE ET AL.
number of elements per column is then C = (1 − R)K , where the code rate is R = N /M (for unbiased messages). The encoded vector t is then corrupted by noise represented by the vector ζ ∈ {0, 1} M with components independently drawn from P(ζ ) = (1 − p)δ(ζ ) + pδ(ζ − 1). The received vector takes the form r = G T ξ + ζ (mod 2).
(102)
Decoding is carried out by multiplying the received message by the matrix A to produce the syndrome vector z = Ar = Aζ (mod 2),
(103)
from which an estimate τˆ for the noise vector can be produced. An estimate for the original message is then obtained as the first N bits of r + τˆ (mod 2). The Bayes optimal estimator (also known as marginal posterior maximizer, MPM) for the noise is defined as τˆ j = argmaxτ j P(τ j | z). The performance of this estimator can be measured by the bit error probability pb = 1 − 1/M Mj=1 δ[τˆ j ; ζ j ], where δ[;] is the Kronecker delta. Knowing the matrices C 2 and C 1, the syndrome vector z and the noise level p, it is possible to apply Bayes theorem and compute the posterior probability P(τ | z) =
1 χ [z = Aτ (mod 2)] P(τ ), Z
(104)
where χ [X] is an indicator function providing 1 if X is true and 0 otherwise. To posterior P(τ j | z) = compute the MPM one has to compute the marginal M i= j P(τ | z), which in general requires O (2 ) operations, thus becoming impractical for long messages. To solve this problem we can take advantage of the sparseness of A and use probability propagation for decoding, requiring O(M) operations to perform the same task.
A. Upper Bound on Achievable Rates It was pointed by MacKay in 1999 that an upper bound for rates achievable for Gallager codes can be found from information theoretic arguments. This upper bound is based on the fact that each bit of the syndrome vector z = Aζ(mod 2) is a sum of K noise bits independently drawn from a bimodal delta distribution P(ζ ) with P(ζ = 0) = 1 − p. The probability of z j = 1 is pz1 (K ) = 12 − 21 (1 − 2 p) K (see Appendix C.1 for details). Therefore, the maximal information content in the syndrome vector is (M − N )H2 ( pz1 (K )) (in bits or shannons), where H2 (x) is the binary entropy. In the decoding process one has to extract information from the syndrome vector in order to reconstruct
LOW-DENSITY PARITY-CHECK CODES
273
Figure 13. (a) Bounds for the rate R as a function of the noise level p for several values of K . From bottom to top: K = 2 to 10,20 and Shannon limit. (b) Bounds for several values of C. From bottom to top C = 2, 3, 4, 5 and Shannon limit.
a noise vector ζ which has an information content of MH2(p). It clearly means that a necessary condition for successful decoding is: (M − N )H2 pz1 (K ) ≥ M H2 ( p) (1 − R)H2 pz1 (K ) ≥ H2 ( p) R ≤1−
H2 ( p) . H2 pz1 (K )
(105)
In Figure 13a we plot this bound by fixing K and finding the minimum value for C such that R = 1 − C/K verifies (105). Observe that as K → ∞, pz1 (K) → 1/2 and R → 1 − H2(p) that corresponds to Shannon’s bound. In Figure 13b we plot the bound by fixing C and finding the maximum K such that R = 1 − C/K satisfies (105), recovering the curves presented in MacKay (1999). Note that K → ∞ implies C → ∞ and vice versa. Gallager codes only can attain Shannon’s bound asymptotically in the limit of large K or, equivalently, large C.
B. Statistical Physics Formulation The connection to statistical physics is made by replacing the field {0, 1} by Ising spins {±1} and mod 2 sums by products (Sourlas, 3 1989). The syndrome vector acquires the form of a multispin coupling Jμ = j∈L(μ) ζ j where j = 1, . . . , M and μ = 1, . . . , (M − N ). The K indices of nonzero elements in the row μ of A are given by L(μ) = { j1 , . . . , jK }, and in a column l are given by M(l) = {μ1 , . . . , μC }.
274
VICENTE ET AL.
The following family of posterior probabilities can be introduced: 1 exp[−β Hγ (τ ; J )] Z M M−N Jμ τj. τj − 1 − F Hγ (τ ; J ) = −γ
Pγ (τ | J ) =
μ=1
(106)
j=1
j∈L(μ)
The Hamiltonian depends on hyperparameters γ and F. For optimal decoding, γ and F have to be set to specific values that best represent how the encoding process and corruption were performed (Nishimori condition (Iba, 1999)). Therefore, γ must be taken to infinity to reflect the hard constraints in Eq. (104) and F = atanh(1 − 2 p), reflecting the channel noise level p. The temperature β must simultaneously be chosen to be the Nishimori temperature β N = 1, that will keep the hyperparameters in the correct value. The disorder in (106) is trivial and can be gauged to Jμ "→ 1 by using τ j "→ τ j ζ j . The resulting Hamiltonian is a multispin ferromagnet with finite connectivity in a random field ζ j F: M−N M gauge Hγ (τ ; ζ) = −γ τj − 1 − F ζjτj. (107) μ=1
j=1
j∈L(μ)
At the Nishimori condition γ → ∞ and the model is even simpler, corresponding to a paramagnet with restricted configuration space on a nonuniform external field:
Hgauge (τ ∈ ; ζ) = −F where
8
= τ :
j∈L(μ)
M
(108)
ζjτj,
j=1
9
τ j = 1, μ = 1, . . . , M − N .
(109)
The optimal decoding process simply corresponds to finding local magnetizations at the Nishimori temperature m j = τ j β N and calculating Bayesian estimates as τˆ j = sgn(m j ). In the {±1} representation the probability of bit error, acquires the form pb =
M 1 1 − ζ j sgn(m j ), 2 2M j=1
(110)
275
LOW-DENSITY PARITY-CHECK CODES
connecting the code performance with the computation of local magnetizations.
C. Replica Theory In this section we use the replica theory for analyzing the typical performance of Gallager codes along the same lines discussed for Sourlas codes. We start by rewriting the gauged Hamiltonian (107) in a form more suitable for computing averages over different codes:
Hγgauge (τ ; ζ) = −γ
i 1···i K
Ai1···i K (τi1 · · · τi K − 1) − F
M
(111)
ζjτj,
j=1
where Ai1···i K ∈ {0, 1} is a random symmetric tensor with the properties: Ai1···i K = M − N Ai1 ,...,i K = C∀l, (112) i 1···i K
i 1 ,...,i j =l,...,i K
that selects M − N sets of indices (construction). The construction {Ai1···i K } and the noise vector ζ are to be regarded as quenched disorder. As usual, the aim is to compute the free-energy: f =−
1 1 lim ln Z A,ζ , β M→∞ M
(113)
from which the typical macroscopic (thermodynamic) behavior can be obtained. The partition function Z is: Z = Trτ exp −β Hγgauge (τ ; ζ) . (114)
The free-energy can be evaluated calculating following expression 5 1 ∂ 55 1 Z n A,ζ , f = − lim β M→∞ M ∂ n 5n=0 where
n
Z A,ζ =
τ
M
1
,...,τ n
×
6
j=1
6
exp Fζβ
n
i 1···i K α=1
exp
n α=1
τ jα
7
ζ
βγ Ai1···i K τiα1 · · · τiαK
−1
7
A
(115)
.
(116)
276
VICENTE ET AL.
The average over constructions (· · ·)A takes the form: M 1 (· · ·)A = δ Ai = j,...,i K − C (· · ·) N {A} j=1 i1 = j,i2 ,...,i K 1 ! C M 1 dZj 1 i 1 = j,i 2 ,...,i K Ai 1 = j,...,i K = Zj (· · ·), N {A} j=1 2πi Z C+1 j and the average (· · ·)ζ over the noise is: (· · ·)ζ = (1 − p)δ(ζ − 1) + pδ(ζ + 1) (· · ·).
(117)
(118)
ζ =−1,+1
By computing the averages above and introducing auxiliary variables through the identity ) M 1 Z i τiα1 · · · τiαm = 1 (119) dqα1···αm δ qα1···αm − M i
one finds, after using standard techniques (see Appendix B.1 for details), the following expression for the replicated partition function: % ) $ n 1 dq0 d qˆ 0 dqα d qˆ α n ··· Z A,ζ = N 2πi 2πi α=1 n MK Tm qαK1···αm K ! m=0 α1···αm ! n qα1···αm qˆ α1···αm −M
× exp
m=0 α1···αm
× ×
M j=1
C
Tr{τ α }
6
exp Fβζ
n α=1
τ
α
!7
ζ
n ! d Z exp Z m=0 α1···αm qˆ α1···αm τ α1 · · · τ αm , 2πi Z C+1 (120)
where Tm = e−nβγ coshn (βγ ) tanhm (βγ ). Comparing this expression with that obtained for the code of Sourlas in Eq. (A.7), one can see that the differences are
277
LOW-DENSITY PARITY-CHECK CODES
the dimensionality M for Gallager codes instead of N for Sourlas (reflecting the fact that in the former the noise vector of dimension M is the dynamic variable) and the absence of disorder in the couplings, yielding a slightly modified definition for the constants Tm .
D. Replica Symmetric Solution The replica symmetric ans¨atz consists of assuming the following form for the order parameters: qα1···αm =
)
d x π (x) x m
qˆ α1···αm =
)
d xˆ πˆ (xˆ ) xˆ m .
(121)
By performing the limit γ → ∞, plugging (121) into (120), computing the normalization constant N , integrating in the complex variable Z and computing the trace (see Appendix B.2) we find: " $) % ˆ (1 + x x) ˆ n −1 Z n A,ζ = Extrπ,πˆ exp −MC d x d xˆ π(x) πˆ (x) +
⎛
×⎝
MC K
) K
) C j=1
j=1
d x j π(x j ) 1 +
d xˆ j π( ˆ xˆ j )
6
σ =±1
K j=1
eσβ Fζ
xj
n
−1
!
!n 7 ⎞ M ⎫ ⎬ ⎠ (1 + σ xˆ j ) . ⎭ j=1
C
ζ
(122)
Using (115): 8 ) 1 C ˆ ln(1 + x x) ˆ ln 2 + C d x d xˆ π(x) πˆ (x) f = Extrπ,πˆ β K ) K K C xj d x j π(x j ) ln 1 + − K j=1 j=1 6 !7 9 ) C C σβ Fζ − e d xˆ j π( ˆ xˆ j ) ln (1 + σ xˆ j ) . j=1
σ =±1
j=1
ζ
(123)
278
VICENTE ET AL.
The extremization above yields a pair of saddle-point equations: ! ) K K −1 −1 ˆ = xj d x j π(x j ) δ xˆ − π( ˆ x) π(x) =
) C−1 l=1
(124)
j=1
j=1
6
d xˆ l πˆ (xˆ l ) δ x − tanh β Fζ +
C−1
atanh xˆ l
l=1
!7
, ζ
where β = 1 (Nishimori temperature) and F = 12 ln ( 1−p p ) for optimal decoding. Following M the derivation of Appendix A.3 very closely, the typical overlap ρ = M1 j=1 ζ j τˆ j A,ζ between the estimate τˆ j = sgn(τ j β ) and the actual noise ζ j is given by: ) (125) ρ = dh P(h) sgn(h) P(h) =
) C l=1
6
d xˆ l πˆ (xˆ l ) δ h − tanh β Fζ +
C
atanh xˆ l
l=1
!7
. ζ
E. Thermodynamic Quantities and Typical Performance The typical performance of a code as predicted by the replica symmetric theory can be assessed by solving (124) numerically and computing the overlap ρ using (125). The numerical calculation can be done by representing distributions π and πˆ by histograms (we have used representations with 20000 bins), and performing Monte Carlo integrations in an iterative fashion until a solution is found. Overlaps can be obtained by plugging the distribution πˆ that is a solution for (124) into (125). Numerical calculations show the emergence of two solution types; the first corresponds to a totally aligned (ferromagnetic) state with ρ = 1 described by: πFERRO (x) = δ[x − 1]
ˆ = δ[xˆ − 1]. πˆ FERRO (x)
(126)
The ferromagnetic solution is the only stable solution up to a specific noise level ps. Above ps another stable solution with ρ < 1 (suboptimal ferromagnetic) can be obtained numerically. This solution is depicted in Figure 14 for K = 4, C = 3 and p = 0.20. The ferromagnetic state is always a stable solution for (124) and is present for all choices of noise level or construction parameters C and K . The stability can be verified by introducing small perturbations to the solution and observing that the solution is recovered after a number of iterations of (124).
LOW-DENSITY PARITY-CHECK CODES
279
Figure 14. Suboptimal ferromagnetic solution π NFERRO(x) for the saddle-point Eqs. (124) obtained numerically. Parameters are K = 4, C = 3 and p = 0.20. Circles correspond to an experimental histogram obtained by decoding with probability propagation in 100 runs for 10 different random constructions.
The free-energy for the ferromagnetic state at Nishimori’s temperature is simply f FERRO = −F(1 − 2 p). In Figure 15 we show free-energies for K = 4 and R = 1/4, pc indicates the noise level where coexistence between the ferromagnetic and suboptimal ferromagnetic phases occurs. This coexistence noise level coincides, within the numerical precision, with the information theoretic upper bound of Section IV.A. In Figure 16 we show pictorially how the replica symmetric free-energy landscape changes with the noise level p. In Figure 17 we show the overlap as a function of the noise level, as obtained for K = 4 and R = 1/4 (therefore C = 3). Full lines indicate values corresponding to states of minimum free-energy that are predicted thermodynamically. The general idea is that the macroscopic behavior of the system is dominated by the global minimum of the free-energy (thermodynamic equilibrium state). After a sufficiently long time the system eventually visits configurations consistent with the minimum free-energy state staying there almost all of the time. The whole dynamics is ignored and only the stable equilibrium, in a thermodynamic sense, is taken into account. Also in Figure 17 we show results obtained by simulating probability propagation decoding (black circles). The practical decoding stays in a metastable (in the thermodynamic sense) state between ps and pc, and the practical maximum noise level corrected is actually given by ps. Returning to the pictorial representation in Figure 16, the noise level ps that provides the practical threshold is signalled by the
280
VICENTE ET AL.
Figure 15. Free-energies for K = 4, C = 3 and R = 1/4. The full line corresponds to the free-energy of thermodynamic states. Up to ps only the ferromagnetic state is present. The ferromagnetic state then dominates the thermodynamics up to pc, where thermodynamic coexistence with suboptimal ferromagnetic states takes place. Dashed lines correspond to replica symmetric free-energies of nondominant metastable states.
appearance of spinodal points in the replica symmetric free-energy, defined as points separating (meta)stable and unstable regions in the space of thermodynamic configurations (ρ). The noise level ps may, therefore, be called spinodal noise level. The solutions obtained must produce nonnegative entropies to be physically meaningful. The entropy can be computed from the free-energy (123) as s = β 2 ∂∂βf yielding: s = β(u(β) − f ) (127) 6 7 3 ) C τβ Fζ C τ =±1 τ e j=1 (1 + τ xˆ j ) u(β) = − , d xˆ j πˆ ∗ (xˆ j ) Fζ 3 C τβ Fζ j=1 j=1 (1 + τ xˆ j ) ζ τ =±1 e
where πˆ ∗ is a solution for the saddle-point Eqs. (124) and u(β) corresponds to the internal energy density at temperature β. For the ferromagnetic state sFERRO = 0, what indicates that the replica symmetric ferromagnetic solution is physical and that the number of microstates consistent with the ferromagnetic state is at most of polynomial order in N. The entropy of the suboptimal ferromagnetic state can be obtained numerically. Up to the spinodal noise level ps the entropy vanishes as only the ferromagnetic state is stable. Above ps, the entropy of the replica symmetric suboptimal ferromagnetic state is negative
LOW-DENSITY PARITY-CHECK CODES
281
Figure 16. Pictorial representation of the replica symmetric free-energy landscape changing with the noise level p. Up to ps there is only one stable state F corresponding to the ferromagnetic state with ρ = 1. At ps, a second stable suboptimal ferromagnetic state F′ emerges with ρ < 1, as the noise level increases, coexistence is attained at pc. Above pc, F′ becomes the global minimum dominating the system thermodynamics.
and, therefore, unphysical. At pc, the entropy of the suboptimal ferromagnetic state becomes positive again. The internal energy density obtained numerically is depicted in Figure 18 with u = −F(1 − 2 p) for both ferromagnetic and suboptimal ferromagnetic states, justified by assuming Nishimori’s condition γ → ∞, β = 1 and F = atanh(1 − 2 p) (Iba, 1999); (see Appendix B.3).
Figure 17. Overlaps for K = 4, C = 3, and R = 1/4. The full line corresponds to overlaps predicted by thermodynamic considerations. Up to ps only the ferromagnetic ρ = 1 state is present, it then dominates the thermodynamics up to pc, where coexistence with suboptimal ferromagnetic states takes place. Dashed lines correspond to overlaps of nondominant states.
282
VICENTE ET AL.
Figure 18. Internal energy density for K = 4, C = 3 and R = 1/4 for both ferromagnetic and suboptimal ferromagnetic states. The equality is a consequence of using the Nishimori condition (see Appendix B.3).
The unphysical behavior of the suboptimal ferromagnetic solution between ps and pc indicates that the replica symmetric ans¨atz does not provide the correct physical description of the system. The construction of a complete one-step replica symmetry breaking theory turns out to be a difficult task in the family of models we focus on here (Wong and Sherrington, 1988; Monasson, 1998a,b), although it may be possible in principle using a new method, recently introduced by Mezard and Parisi (2001). An alternative is to consider a frozen spins solution. In this case the entropy in the interval ps < p < pc is corrected to sRSB = 0 and the free-energy and internal energy are frozen to the values at pc . Any candidate for a physical description for the system would have to be compared with simulations to be validated. Nevertheless, our aim here is to predict the behavior of a particular decoding algorithm, namely, probability propagation. In the next section, we will show that, to this end, the replica symmetric theory will be sufficient. F. Codes on a Cactus In this section we present a statistical physics treatment of Gallager codes by employing a mean-field approximation based on the use of a generalized tree structure (Bethe lattice (Wong and Sherrington, 1987b)) known as Husimi
LOW-DENSITY PARITY-CHECK CODES
283
Figure 19. First step in the construction of Husimi cactus with K = 3 and connectivity C = 4.
cactus that is exactly solvable (Gujrati, 1995; Bowman and Levin, 1982; Rieger and Kirkpatrick, 1992; Goldschmidt, 1991). There are many different ways of building mean-field theories. One can make a perturbative expansion around a tractable model (Plefka, 1982; Tanaka, 2000) or assume a tractable structure and variationally determine the model parameters (Saul and Jordan, 1998). In the approximation we employ, the tractable structure is tree-like and the couplings Jμ are just assumed to be those of a model with cycles. In this framework the probability propagation decoding algorithm (PP) emerges naturally, providing an alternative view to the relationship between PP decoding and mean-field approximations already observed in (Kabashima and Saad (1998)). Moreover, this approach has the advantage of being slightly more controlled and easier to understand than replica calculations. A Husimi cactus with connectivity C is generated starting with a polygon of K vertices with one Ising spin in each vertex (generation 0). All spins in a polygon interact through a single coupling Jμ and one of them is called the base spin. In Figure 19 we show the first step in the construction of a Husimi cactus; in a generic step, the base spins of the (C − 1) (K − 1) polygons in generation n − 1 are attached to K − 1 vertices of a polygon in the next generation n. This process is iterated until a maximum generation nmax is reached; the graph is then completed by attaching C uncorrelated branches of nmax generations at their base spins. In this way each spin inside the graph is connected to C polygons exactly. The local magnetization at the center mj can be obtained by fixing boundary (initial) conditions in the zeroth generation and iterating the related recursion equations until generation nmax is reached. Carrying out the calculation in the thermodynamic limit corresponds to having nmax ∼ ln M generations and M → ∞. The Hamiltonian of the model has the form (106) where L(μ) denotes the polygon μ of the lattice. Due to the tree-like structure, local quantities far from the boundary can be calculated recursively by specifying boundary conditions. The typical decoding performance can therefore be computed exactly without resorting to replica calculations (Gujrati, 1995).
284
VICENTE ET AL.
We adopt the approach presented in Rieger and Kirkpatrick (1992) for obtaining recursion relations. The probability distribution Pμk (τk ) for the base spin of the polygon μ is connected to (C − 1) (K − 1) distributions Pν j (τ j ), with ν ∈ M( j) \ μ (all polygons linked to j but μ) of polygons in the previous generation: ! 1 Tr{τ j } exp βγ Jμ τk τ j − 1 + β Fτk Pμk (τk ) = N j∈L(μ)\k × Pν j (τ j ), (128) ν∈M( j)\μ j∈L(μ)\k
where the trace is over the spins τ j such that j ∈ L(μ) \ k. The effective field xˆ ν j on a base spin j due to neighbors in polygon ν can be written as: Pν j (−) , (129) e−2xˆ ν j = e2β F Pν j (+)
Combining (128) and (129) we find the recursion relation (see Appendix B.4 for details): e
−2xˆ μk
=
Tr{τ j } e−βγ Jμ
Tr{τ j } e+βγ Jμ
3
3
j∈L(μ)\k j∈L(μ)\k
τj+ τj+
j∈L(μ)\k (β F+ j∈L(μ)\k (β F+
ν∈M( j)\μ
ν∈M( j)\μ
xˆ ν j )τ j xˆ ν j )τ j
.
By computing the traces and taking γ → ∞ and β = 1 one obtains: ! xˆ ν j tanh F + xˆ μk = atanh Jμ j∈L(μ)\k
(130)
(131)
ν∈M( j)\μ
The effective local magnetization due to interactions with the nearest neighbors in one branch is given by mˆ μj = tanh(xˆ μj ). The effective local field on a base spin j of a polygon μ due to C − 1 branches in the previous generation and due to the external field is xμj = F + ν∈M( j)\μ xˆ ν j ; the effective local magnetization is therefore m μj = tanh(xμj ). Equation (131) can then be rewritten in terms of mˆ μj and m μj and the PP equations (MacKay, 1999; Kabashima and Saad 1998; Kschischang and Frey, 1998) can be recovered: atanh(mˆ νk ) m μk = tanh F + ν∈M(k)\μ
mˆ μk = Jμ
m μj
(132)
j∈L(μ)\k
Once the magnetization on the boundary (zeroth generation) are assigned, the local magnetization mj in the central site is determined by iterating (132)
285
LOW-DENSITY PARITY-CHECK CODES
and computing:
m j = tanh F +
atanh(mˆ ν j )
ν∈M( j)
(133)
A free-energy can be obtained by integration of (132) (Murayama et al., 2000; Vicente et al., 2000b; Bowman and Levin, 1982). The Eqs. (132) describing PP decoding represent extrema of the following free-energy: M−N M−N ln 1 + Jμ F ({m μk , mˆ μk }) = ln(1 + m μi mˆ μi ) − m μi μ=1
μ=1 i∈L(μ)
−
M
ln e
F
j=1
μ∈M( j)
(1 + mˆ μj ) + e
i∈L(μ)
−F
μ∈M( j)
(1 − mˆ μj )
!
(134)
The iteration of the maps (132) is actually one out of many different methods of finding stable extrema of this free-energy. The decoding process can be performed by iterating the multidimensional map (132) using some defined scheduling. Assume that the iterations are performed in parallel using the following procedure: (i) Effective local magnetizations are initialized as m μk = 1 − 2 p, reflecting prior probabilities. (ii) Conjugate magnetizations mˆ μk are updated. (iii) Magnetizations m μk are computed. (iv) If convergence or a maximal number of iterations is attained, stop. Otherwise go to step (ii). Equations (132) have fixed points that are inconveniently dependent on the particular noise vector ζ. By applying the gauge transformation Jμ "→ 1 and τ j "→ τ j ζ j we get a map with noise-independent fixed points that has the following form: atanh(mˆ νk ) (135) m μk = tanh ζk F + ν∈M(k)\μ
mˆ μk =
(136)
m μj .
j∈L(μ)\k
In terms of effective fields xμk and xˆ μk we have: xˆ μk = atanh xˆ νk tanh(xμj ) . xμk = ζk F + ν∈M(k)\μ
j∈L(μ)\k
(137)
286
VICENTE ET AL.
The above equations provide a microscopic description for the dynamics of a probability propagation decoder; a macroscopic description can be constructed by retaining only statistical information about the system, namely, by describing the evolution of histograms of variables xμk and xˆ μk . Assume that the effective fields xμk and xˆ μk are random variables indepenˆ x), ˆ respectively; in the same dently sampled from the distributions P(x) and P( way assume that ζ j is sampled from P(ζ ) = (1 − p) δ(ζ − 1) + δ(ζ + 1). A recursion relation in the space of probability distributions (Bowman and Levin, 1982) can be found from Eq. (137): ! ) ) C−1 C−1 xˆ l Pn (x) = dζ P(ζ ) d xˆ l Pˆ n−1 (xˆ l ) δ x − Fζ − l=1
ˆ = Pˆ n−1 (x)
) K −1 j=1
d x j Pn−1 (x j ) δ xˆ − atanh
l=1
K −1
tanh(x j )
j=1
!
, (138)
where Pn(x) is the distribution of effective fields at the nth generation due to the previous generations and external fields, in the thermodynamic limit the distribution far from the boundary will be P∞ (x) (generation n → ∞). The local field distribution at the central site is computed by replacing C − 1 by C in the first Eq. (138), taking into account C polygons in the generation just before the central site, and inserting the distribution P∞ (x): ! ) ) C C d xˆ l Pˆ ∞ (xˆ l ) δ x − Fζ − xˆ l . (139) P(h) = dζ P(ζ ) l=1
l=1
ˆ satisfy Eqs. (124) obtained by the It is easy to see that P∞ (x) and Pˆ ∞ (x) replica symmetric theory (Kabashima et al., 2000; Murayama et al., 2000; Vicente et al., 2000b), if the variables describing fields are transformed to those of local magnetizations through x "→ tanh(βx). In Figure 14 we show empirical histograms obtained by performing 100 runs of PP decoding for 10 different codes of size M = 5000 and compare with a distribution obtained by solving equations like (138). The practical PP decoding is performed by setting initial conditions as m μj = 1 − 2 p to correspond to the prior probabilities and iterating (132) until stationarity or a maximum number of iterations is attained (MacKay, 1999). The estimate for the noise vector is then produced by computing τˆ j = sign(m j ). At each decoding step the system can be described by histograms of variables (132), this is equivalent to iterating (138) (a similar idea was presented in MacKay (1999) and Davey (1998)). In Figure 20 we summarize the transitions obtained for K = 6 and K = 10. A dashed line indicates Shannon’s limit, the full line represents the information theoretic upper bound of Section IV.A, white circles stand for the coexistence line obtained numerically. Diamonds represent spinodal noise levels obtained
LOW-DENSITY PARITY-CHECK CODES
287
Figure 20. Transitions for Gallager codes with K = 6 (left) and K = 10 (right). Shannon’s bound (dashed line), information theory upper bound (full line), and thermodynamic transition obtained numerically (䊊). Transitions obtained by Monte Carlo integration of Eq. (138) (♦) and by simulations of PP decoding (+, M = 5000 averaged over 20 runs) are also shown. Black squares are estimates for practical thresholds based on Sec. IV.H. In both figures, symbols are chosen larger than the error bars.
by solving (138) numerically and (+) are results obtained by performing 20 runs using PP decoding. It is interesting to observe that the practical performance tends to get worse as K grows large, which agrees with the general belief that decoding gets harder as Shannon’s limit is approached.
G. Tree-Like Approximation and the Thermodynamic Limit The geometric structure of a Gallager code defined by the matrix A can be represented by a bipartite graph as in Figure 21 (Tanner graph) (Kschischang
Figure 21. Tanner graph representing the neighborhood of a bit node in an irregular Gallager code. Black circles represent checks and white circles represent bits.
288
VICENTE ET AL.
and Frey, 1998) with bit and check nodes (in this case, we show an irregular constraction where the values of K and C are not fixed). Each column j of A represents a bit node and each row μ represents a check node; Aμj = 1 means that there is an edge linking bit j to check μ. It is possible to show (Richardson and Urbanke, 2001) that for a random ensemble of regular codes, the probability of completing a cycle after walking l edges starting from an arbitrary node is upper bounded by P [l; K , C, M] ≤ l 2 K l /M. It implies that for very large M only cycles of at least order ln M survive. In the thermodynamic limit M → ∞ and the probability P [l; K , C, M] → 0 for any finite l and the bulk of the system is effectively tree-like. By mapping each check node to a polygon with K bit nodes as vertices, one can map a Tanner graph into a Husimi lattice that is effectively a tree for any number of generations of order less than ln M. In Figure 22 we show that the number of iterations of (132) required for convergence far from the threshold does not scale with the system size, therefore, it is expected that the interior of a tree-like lattice approximates a Gallager code with increasing accuracy as the system size increases. Figure 23 shows that the approximation is fairly good even for sizes as small as M = 100 when compared to theoretical results and simulations for size M = 5000. Nevertheless, the difference increases as the spinodal noise level approaches, what seems to indicate the breakdown of the approximation. A possible explanation is that convergence times larger than O(ln M) may be required in this region. An interesting analysis of the convergence properties
Figure 22. PP decoding convergence time as a function of the code size (M − N) for K = 4C = 3 and p = 0.05, therefore, well below the threshold. The convergence time clearly does not scale with the system size.
LOW-DENSITY PARITY-CHECK CODES
289
Figure 23. Mean normalized overlap ρ between the actual noise vector ζ and decoded noise τˆ for a Gallager code with K = 4 and C = 3 (therefore R = 1/4). Theoretical values ( ) obtained by Monte Carlo integration of Eqs.(138) and averages of 20 simulations of PP decoding for code word lengths M = 5000 (䊉) and M = 100 (full line). Symbols are chosen larger than the error bars.
of probability propagation algorithms for some specific graphical models can be found in Weiss (1997). H. Estimating Spinodal Noise Levels We now estimate the threshold noise level ps by introducing a measure for the number of parity checks violated by a bit τ l: El = − Jμ τl τj − 1 . (140) j∈L(μ)\l
μ∈M(l)
By using gauged variables:
El = −
μ∈M(l)
τl
j∈L(μ)\l
τj − 1 .
(141)
Suppose that random guesses are generated by sampling the prior distribution, their typical overlap will be ρ = 1 − 2 p. Assume now that the vectors sampled are corrected by flipping τ l if El = C. If the landscape has a single dominant minimum, we expect that this procedure will tend to increase the overlap ρ between τ and the actual noise vector ζ in the first step up to the noise level ps, where suboptimal microscopic configurations are expected to emerge. Above ps, there is a large number of suboptimal ferromagnetic microstates with an overlap around ρ = 1 − 2 p (see Fig. 23), and we expect that if a single bit of a randomly guessed vector is corrected, the overlap will then
290
VICENTE ET AL.
either increase or decrease, staying unchanged on average. A vanishing variation in the mean overlap would, therefore, signal the emergence of suboptimal microstates at ps. The probability that a bit τl = +1 is corrected is: 8 9 P(El = C | τl = +1) = P τ j = −1 . (142) j∈L(μ)\l
μ∈M(l)
For a a bit τl = −1: P(El = C | τl = −1) =
μ∈M(l)
1− P
8
j∈L(μ)\l
τ j = −1
9!
.
(143)
Considering vectors sampled from a prior P(τ ) = (1 − p) δ(τ − 1) + p δ(τ + 1) we have: 8 9 1 1 P (144) τ j = −1 = − (1 − 2 p) K −1 . 2 2 j∈L(μ)\l
The gauged overlap is defined as ρ = Mj=1 S j and the variation on the overlap after flipping a bit l is ρ = ρ1 − ρ0 = Sl1 − Sl0 . The mean variation in the overlap due to a flip in a bit τ l with El = C is therefore: 1 ρ = P(τl = +1 | El = C) − P(τl = −1 | El = C) 2 τ =±1 τl P(E l = C | τl )P(τl ) , = l τl =±1 P(E l = C | τl ))P(τl )
(145)
1 [1 − (1 − 2 p) K −1 ]C (1 − p) − [1 + (1 − 2 p) K −1 ]C p ρ = . 2 [1 − (1 − 2 p) K −1 ]C (1 − p) + [1 + (1 − 2 p) K −1 ]C p
(146)
where we applied the Bayes theorem to obtain the last line. By plugging the prior probability (142) and (144) into the above expression we get:
At ps we have ρ = 0 and:
# " ps 1 − (1 − 2 ps ) K −1 . = 1 − ps 1 + (1 − 2 ps ) K −1
(147)
The above equation can be solved numerically yielding reasonably accurate estimates for practical thresholds ps as can be seen in Figure 20. MacKay (1999) and Gallager (1962, 1963) introduced probabilistic decoding algorithms whose performance analysis is essentially the same those as
LOW-DENSITY PARITY-CHECK CODES
291
presented here. However, the results obtained in Section IV.C put the analysis into a broader perspective: algorithms that generate decoding solutions in polynomial time, as is the case of probabilistic decoding or probability propagation, seem to be bounded by the practical threshold ps due to the presence of suboptimal solutions. On the other hand, decoding in exponential time is always possible up to the thermodynamic transition at pc (with pc attaining channel capacity if K → ∞), by performing an exhaustive search for the global minimum of the free-energy (134).
V. MacKay–Neal Codes MacKay–Neal (MN) codes were introduced in 1995 as a variation on Gallager codes. As in the case of Gallager codes (see Section IV), MN codes are defined by two very sparse matrices, but with the difference that information on both noise and signal is incorporated to the syndrome vector. MN codes are also decoded using sparse matrices, while encoding uses a dense matrix, which yields good distance properties and a decoding problem solvable in linear time by using the methods of probability propagation. Cascading codes, a class of constructions inside the MN family recently proposed by Kanter and Saad (1999, 2000a,b) have been shown to outperform some of the cutting-edge Gallager and turbo code constructions. We will discuss cascading codes in the next secion, but this fact alone justifies a thorough study of MN codes. Theorems showing the asymptotic goodness of the MN family have been proved in (MacKay, 1999). By assuming equal message and noise biases (for a BSC), it was proved that the probability of error vanishes as the message length increases and that it is possible to get as close as desired to channel capacity by increasing the number of nonzero elements in a column of the very sparse matrices defining the code. It can also be shown by a simple upper bound that MN codes, unlike Gallager codes, might, as well, attain Shannon’s bound for a finite number of nonzero elements in the columns of the very sparse matrices, given that unbiased messages are used. This upper bound does not guarantee that channel capacity can be attained in polynomial time or even that it can be attained at all. Results obtained using statistical physics techniques (Kabashima et al., 2000; Murayama et al., 2000; Vicente et al., 2000a,b) seem to indicate that Shannon’s bound can actually be approached with exponential time decoding. This feature is considered to be new and somewhat surprising (D. MacKay, personal communication, 2000). Statistical physics has been applied to analyze MN codes and its variants (Kabashima et al., 2000; Murayama et al., 2000; Vicente et al., 2000b). In this
292
VICENTE ET AL.
analysis we use the replica symmetric theory to obtain all relevant thermodynamic quantities and to calculate the phase diagram. The theory also yields a noise level where suboptimal solutions emerge that are in connection with the practical thresholds observed when probability propagation decoding is used. Assuming that a message is represented by a binary vector ξ ∈ {0, 1} N sampled independently from the distribution P(ξ ) = (1 − pξ ) δ(ξ ) + pξ δ(ξ − 1), the MN encoding process consists of producing a binary vector t ∈ {0, 1} M defined by t = Gξ (mod 2),
(148)
where all operations are performed in the field {0, 1} and are indicated by (mod 2). The code rate is, therefore, R = N /M. The generator matrix G is an M × N dense matrix defined by G = Cn−1 Cs (mod 2),
(149)
with Cn being an M × M binary invertible sparse matrix and Cs an M × N binary sparse matrix. The transmitted vector t is then corrupted by noise. We here assume a memoryless binary symmetric channel (BSC), namely, noise is represented by a binary vector ζ ∈ {0, 1} M with components independently drawn from the distribution P(ζ ) = (1 − p) δ(ζ ) + p δ(ζ − 1). The received vector takes the form r = Gξ + ζ (mod 2).
(150)
Decoding is performed by preprocessing the received message with the matrix Cn and producing the syndrome vector z = Cnr = Cs ξ + Cn ζ (mod 2),
(151)
from which an estimate ξˆ for the message can be directly obtained. An MN code is called regular if the number of elements set to one in each row of Cs is chosen to be K and the number of elements in each column is set to be C. For the square matrix Cn the number of elements in each row (or column) is set to L. In this case the total number of ones in the matrix Cs is M K = N C, yielding that the rate can alternatively be expressed as R = K /C. In contrast, an MN code is called irregular if each row m in Cs and Cn contains Km and Lm nonzero elements, respectively. In the same way, each column j of Cs contains Cj nonzero elements and each column l of Cn contains Dl nonzero elements.
LOW-DENSITY PARITY-CHECK CODES
293
Counting the number of nonzero elements in the matrices leads to the following relations: N j=1
Cj =
M
Kμ
M l=1
μ=1
Dl =
M
L μ,
(152)
μ=1
The code rate is, therefore, R = K /C, where: K =
M 1 Kμ M μ=1
C=
N 1 Cj. N j=1
(153)
The Bayes optimal estimator ξˆ for the message ξ is ξˆ j = argmax S j P(S j | z). The performance of this estimator is measured by the probability of bit error pb = 1 − 1/N Nj=1 δ[ξˆ j ; ξ j ], where δ[; ] is the Kronecker delta. Knowing the matrices Cs and Cn , the syndrome vector z, the noise level p, and the message bias pξ , the posterior probability is computed by applying Bayes theorem: P(S, τ | z) =
1 χ [z = Cs S + Cn τ (mod 2)] P(S)P(τ ), Z
(154)
where χ [X] is an indicator function providing 1 if X is true and 0 otherwise. To obtain the estimate one has to compute the marginal posterior P(S j | z) =
{Si :i= j} τ
P(S, τ | z),
(155)
which requires O(2 N ) operations and is impractical for long messages. Again we can use the sparseness of [Cs | Cn ] and the methods of probability propagation for decoding, which requires only O(N ) operations. When p = pξ , MN and Gallager codes are equivalent under a proper transformation of parameters, as the code rate is R = N /M for MN codes and R = 1 − N /M for Gallager codes. The main difference between the codes is in the syndrome vector z. For MN codes, the syndrome vector incorporates information on both message and noise while for Gallager codes, only information on the noise is present (see Eq. (103)). This feature opens the possibility of adjusting the code behavior by controlling the message bias pξ . An MN code can be thought of as a nonlinear code (MacKay, 2000b). Redundancy in the original message could be removed (introduced) by using a source (de)compressor defined by some nonlinear function ξ = g(ξ 0 ; pξ ), and encoding would then be t = Gg(ξ 0 ; pξ ) (mod 2). In the following we show that other new features emerge due to the introduction of the parameter pξ .
294
VICENTE ET AL.
A. Upper Bound on Achievable Rates In a regular MN code, the syndrome vector z = Cs S + Cn τ (mod 2) is a sum of K message bits drawn from the distribution P(ξ ) = (1 − pξ ) δ(ξ ) + pξ δ(ξ − 1) and L noise bits drawn from P(ζ ) = (1 − p) δ(ζ ) + p δ(ζ − 1). The probability of z j = 1 is (see Appendix C.1) pz1 (K , L) =
1 1 − (1 − 2 pξ ) K (1 − 2 p) L . 2 2
(156)
The maximum information content in the syndrome vector is M H2 ( pz1 (K , L)) (in bits or shannons), where H2(x) is the binary entropy. The amount of information needed to reconstruct both the message vector ξ and the noise vector ζ is N H2 ( pξ ) + M H2 ( p) (in bits or shannons). Thus, it is a necessary condition for successful decoding that: M H2 pz1 (K , L) ≥ N H2 ( pξ ) + M H2 ( p) H2 pz1 (K , L) − H2 ( p) ≥ R H2 ( pξ ) H2 pz1 (K , L) − H2 ( p) . (157) R≤ H2 ( pξ ) For the case pξ = p and L = C, we can recover bounds (105) for Gallager codes with dimensions and parameters redefined as M ′ = M + N , N ′ = N and K ′ = K + L. In MacKay (1999), a theorem stating that channel capacity can be attained when K → ∞ was proved for this particular case. If unbiased ( pξ = 1/2) messages are used, H2 ( pξ ) = 1, H2 ( pz1 (K , L)) = 1 and the bound (157) becomes R ≤ 1 − H2 ( p),
(158)
i.e., MN codes may be capable of attaining channel capacity even for finite K and L, given that unbiased messages are used.
B. Statistical Physics Formulation The statistical physics formulation for MN codes is a straightforward extension of the formulation presented for Gallager codes. The field ({0, 1}, +(mod 2)) is replaced by ({±1}, ×) (Sourlas, 1989) and the syndrome vector acquires the form: ζl (159) Jμ = ξj j∈Ls (μ)
l∈Ln (μ)
where j = 1, . . . , N , l = 1, . . . , M and μ = 1, . . . , M.
295
LOW-DENSITY PARITY-CHECK CODES
The K μ indices of nonzero elements in the row μ of the signal matrix Cs are given by Ls (μ) = { j1 , . . . , jK μ }, and in a column j are given by Ms ( j) = {μ1 , . . . , μC j }. In the same way, for the noise matrix Cn , the Lμ indices of nonzero elements in the row μ are given by Ln (μ) = { j1 , . . . , jL μ }, and in a column l are given by Mn (l) = {μ1 , . . . , μ Dl }. Under the assumption that priors P(S) and P(τ ) are completely factorizable, the posterior (154) corresponds to the limit γ → ∞ and β = 1 (Nishimori temperature) of: 1 exp[−β Hγ (S, τ ; J )] (160) Z N M Sj τl − 1 − Fs S j − Fn τl ,
Pγ (S, τ | J ) =
Hγ (S, τ ; J ) = −γ
M μ=1
Jμ
j∈Ls (μ)
l∈Ln (μ)
j=1
l=1
1− p
with Fs = 21 atanh( pξ ξ ) and Fn = 12 atanh( 1−p p ) (Nishimori condition (Iba, 1999)). By applying the gauge transformation S j "→ S j ξ j and τl "→ τl ζl the couplings can be gauged out Jμ "→ 1, eliminating the disorder. The model is free of frustration (as in Toulouse, 1977, the model is flat). Similar to Gallager codes, the resulting Hamiltonian consists of two sublattices interacting via multispin ferromagnetic iterations with finite connectivity in random fields ξ j Fs and ζl Fn : M gauge Sj Hγ (S, τ ; ξ, ζ) = −γ τl 1 μ=1
− Fs
N j=1
j∈Ls (μ)
l∈Ln (μ)
ξ j S j − Fn
M
(161)
ζl τl .
l=1
At the Nishimori condition γ → ∞, the model can also be regarded as a paramagnet with restricted configuration space on a nonuniform external field:
Hgauge ((S, τ ) ∈ ; ξ, ζ) = −Fs where
8
= (S, τ ) :
j∈Ls (μ)
Sj
l∈Ln (μ)
N j=1
ξ j S j − Fn
M
ζl τl ,
(162)
9
(163)
l=1
τl = 1, μ = 1, . . . , M .
Optimal decoding consists of finding local magnetizations at the Nishimori temperature in the signal sublattice m j = S j β N and calculating Bayesian estimates ξˆ j = sgn(m j ).
296
VICENTE ET AL.
The probability of bit error is pb =
N 1 1 ξ j sgn(m j ), − 2 2N j=1
(164)
connecting the code performance with the computation of local magnetizations. C. Replica Theory The replica theory for MN codes is the theory constructed for Gallager codes, with the introduction of extra dynamic variables S. The gauged Hamiltonian (161) is written as: Hγgauge (S, τ ; ξ, ζ) = −γ A jl (S j1 · · · S jK τl1 · · · τl L − 1) jl
− Fs
N j=1
ξ j S j − Fn
M
ζl τl ,
(165)
l=1
where jl is a shorthand for j1 · · · jK l1 · · · l L . Code constructions are described by the tensor Ail ∈ {0, 1} that specifies a set of indices j1 · · · jK l1 · · · l L corresponding to nonzero elements in a particular row of the matrix [Cs | Cn ]. To cope with noninvertible Cn matrices, we can start by considering an ensemble with uniformly generated M × M matrices. The noninvertible matrices can be made invertible by eliminating a ǫ ∼ O(1) number of rows and columns, resulting in an ensemble of (M − ǫ) × (M − ǫ) invertible Cn matrices and (M − ǫ) × (N − ǫ)Cs matrices. As we are interested in the thermodynamic limit, we can neglect O(1) differences and compute the averages in the original space of M × M matrices. The averages are then performed over an ensemble of codes generated as follows: M (i) Sets of numbers {C j } Nj=1 and {Dl }l=1 are sampled independently from distributions PC and P D , respectively; (ii) Tensors A jl are generated such that A jl = M, jl
j1 = j··· j K l1···l L
A jl = C j
j1··· j K l1 =l···l L
A jl = Dl .
297
LOW-DENSITY PARITY-CHECK CODES
The free-energy is computed by the replica method as: 5 1 ∂ 55 1 Z n A,ξ,ζ f = − lim β N →∞ N ∂n 5n=0
The replicated partition function is: 7 6 N n α n Sj Z A,ξ,ζ = exp Fs ξβ S1 ,...,Sn τ 1 ,...,τ n j=1
× ×
M l=1
6
6
α=1
exp Fn ζβ
n jl α=1
n
τlα
α=1
7
(166)
ξ
ζ
exp βγ A jl S αj1 · · · S αjK τlα1 · · · τlαL − 1
7
.
A
(167)
The average over constructions (· · ·)A is:
1 P D (Dl ) δ PC (C j ) (· · ·)A = A jl − C j N {C j ,Dl } j=1 j1 = j,i 2 ,..., j K l l=1 A jl − Dl (· · ·) ×δ M
N
jl1 =l,l2 ,...,l K
=
N
PC (C j )
{C j ,Dl } j=1
M
P D (Dl )
l=1
! C N dZj 1 1 i 1 = j,i 2 ,...,i K l A j1 = j,..., j K l Z × N {A} j=1 2πi Z Cj j +1 j ! C M dYl 1 jl1 =l,l2 ,...,l L A jl1 =l,...,l L × Y (· · ·), 2πi YlDl +1 l l=1
(168)
where the first sum is over profiles {Cj, Dl} composed by N numbers drawn independently from PC (C) and M numbers drawn from P D (D). The second sum is over constructions A consistent with the profile {Cj, Dl}. The signal average (· · ·)ξ has the form: (· · ·)ξ = (1 − pξ ) δ(ξ − 1) + pξ δ(ξ + 1) (· · ·). (169) ξ =−1,+1
298
VICENTE ET AL.
Similarly, the noise average (· · ·)ζ is: (· · ·)ζ = (1 − p) δ(ζ − 1) + p δ(ζ + 1) (· · ·).
(170)
ζ =−1,+1
Along the same steps described for Gallager codes, we compute averages above and introduce auxiliary variables via ) N 1 αm α1 Z i Si · · · Si =1 (171) dqα1···αm δ qα1···αm − N i ) M 1 αm α1 Yi τi · · · τi =1 (172) drα1···αm δ rα1···αm − M i Using the same types of techniques employed in the case of Gallager codes (see Appendix C.2 for details), we obtain the following expression for the replicated partition function: Z n A,ξ,ζ =
N
PC (C j )
j=1 C j
dq0 d qˆ 0 × 2πi
−N
% n
m=0 α1···αm
N 1 Tr{S αj } × N j=1
× × ×
dqα d qˆ α 2πi α=1
$
dr0 d rˆ0 ··· 2πi
n ML NK Tm qαK1···αm rαL1···αm K !L! m=0 α1···αm
n
C
P D (Dl )
l=1 Dl
$
× exp
M
qα1···αm qˆ α1···αm − M
6
exp Fs βξ
n
n
m=0 α1···αm
S αj
α=1
n
% n
drα d rˆα 2πi α=1
rα1···αm rˆα1···αm
ξ
α1 α1···αm qˆ α1···αm S j · · · C +1 Zj j
S αj m
C
! n α α dYl exp Yl m=0 α1···αm rˆα1···αm τl 1 · · · τl m , 2πi YlDl +1
Tr{τlα }
exp Fn βζ
n α=1
τlα
!7
!
!
l=1
M
···
!7
d Z j exp Z j 2πi 6
m=0
ζ
(173)
299
LOW-DENSITY PARITY-CHECK CODES
where Tm = e−nβγ coshn (βγ ) tanhm (βγ ). Note that the above expression is an extension of Eq. (120). The replica symmetry assumption is enforced by using the ans¨atze: ) ) ˆ xˆ m qˆ α1 ···αm = d xˆ πˆ (x) (174) qα1 ···αm = d x π (x)x m
and
rα1 ···αm =
)
dy φ(y) y m
rˆα1 ···αm =
)
ˆ yˆ ) yˆ m . d yˆ φ(
(175)
By plugging the above ans¨atze, using the limit γ → ∞ and standard techniques (see Appendix C.3 for details) the following expression for the freeenergy: 8 ) 1 ˆ ln (1 + x x) ˆ α ln 2 + C d x π(x) d xˆ πˆ (x) f = Extr{π,π, ˆ ˆ φ,φ} β ) ˆ yˆ ) ln (1 + y yˆ ) + α D dy φ(y) d yˆ φ( −α −
)
K
PC
C
−α
d x j π (x j )
j=1
)
PD
D
C
!
L l=1
d xˆ j π( ˆ xˆ j )
j=1
)
D l=1
dyl φ(yl ) ln 1 +
!6
ˆ yˆ l ) d yˆ l φ(
!
ln
!6
σ =±1
ln
K
xj
j=1
L
yl
l=1
C (1 + σ xˆ j ) eσ ξβ Fs j=1
!7
D (1 + σ yˆ l )
eσ ζβ Fn
σ =±1
l=1
ξ
!7 9
,
ζ
(176)
where C = C C PC (C), D = D D P D (D) and α = M/N = C/K . By performing the extremization above, restricted to the space of normalized functions, we find the following saddle-point equations: ! ) K L K −1 −1 L ˆ = yl xj d x j π(x j ) dyl φ(yl )δ xˆ − π( ˆ x) j=1
π (x) =
1
C
6
C
C PC
d xˆ l πˆ ( xˆ l )
l=1
l=1
j=1
l=1
) C−1
× δ x − tanh β Fs ξ +
C−1 l=1
atanh xˆ l
!7
, ξ
300
VICENTE ET AL.
ˆ yˆ ) = φ(
) L−1
φ(y) =
1
dyl φ(yl )
6
D
d x j π(x j )δ yˆ −
j=1
l=1
D
K
DPD
) D−1
L−1
yl
l=1
K
xj
j=1
!
ˆ yˆ l ) d yˆ l φ(
l=1
× δ y − tanh β Fn ζ +
D−1
atanh yˆ l
l=1
!7
(177)
. ζ
The typical overlap ρ = N1 Nj=1 ξ j ξˆ j A,ζ,ξ between the estimate ξˆ j = sgn(S j β N ) and the actual signal ξ j is given by (see Appendix A.3): ρ= P(h) =
)
dh P(h) sgn(h)
C
6
PC (C)
) C
(178)
d xˆ l πˆ (xˆ l )
l=1
× δ h − tanh β Fs ξ +
C
atanh xˆ l
l=1
!7
.
ξ
The intensive entropy is simply s = β 2 ∂∂βf yielding: s = β(u(β) − f ) (179) 7 6 3 ) C Fs ξ σ =±1 σ eσβ Fs ξ j (1 + σ xˆ j ) 3 u=− d xˆ j πˆ ∗ (xˆ j ) PC σβ Fs ξ σ =±1 e j (1 + σ xˆ j ) C j=1 ξ 7 6 3 ) σβ F ζ D n ˆ j) Fn ζ σ =±1 σ e j (1 + σ y ∗ 3 −α PD d yˆ j φˆ ( yˆ j ) σβ Fn ζ ˆ j) σ =±1 e j (1 + σ y D j=1 ζ
where starred distributions are solutions for (177) and u(β) is the internal energy density. For optimal decoding the temperature must be chosen to be β = 1 (Nishimori temperature) and the fields are Fs =
$ % 1 − pξ 1 ln 2 pξ
Fn =
% $ 1− p 1 . ln 2 p
LOW-DENSITY PARITY-CHECK CODES
301
D. Probability Propagation Decoding In Sections III and IV we derived probability propagation equations first by assuming a set of factorization properties and writing a closed set of equations that allowed the iterative computation of the (approximate) marginal posterior, and second by computing local magnetizations on the interior of a Husimi cactus (Bethe approximation). The two methods are equivalent as the factorization properties assumed in the former are encoded in the geometry of the lattice assumed in the latter. Here we use insights provided in the last sections to build a decoding algorithm for MN codes directly. From the replica symmetric free-energy (176) we can write the following Bethe free-energy: ˆ = F (m, m)
M 1 M ln 2 + ln 1 + m sμi mˆ sμi N N μ=1 i∈Ls (μ)
M 1 ln 1 + m nμj mˆ nμj N μ=1 j∈Ln (μ) M 1 s n − ln 1 + Jμ m μi m μj N μ=1 i∈Ls (μ) j∈Ln (μ) ! N 1 s σ Fs e ln 1 + σ mˆ μi − N i=1 σ =± μ∈Ms (i) ! M 1 n σ Fn 1 + σ mˆ μj . ln e − N j=1 σ =± μ∈Mn ( j)
+
(180)
The variables m sμj (m nμj ) are cavity effective magnetizations of signal (noise) bits interacting through the coupling μ, obtained by removing one of the C couplings in Ms ( j) (Mn ( j)) from the system. The variables mˆ sμj (mˆ nμj ) correspond to effective magnetizations of signal (noise) bits due to the coupling μ only. The decoding solutions are fixed points of the free-energy (181) given by: ˆ ∂ F (m, m) =0 s ∂m μj
ˆ ∂ F (m, m) =0 s ∂ mˆ μj
(181)
ˆ ∂ F (m, m) =0 n ∂m μj
ˆ ∂ F (m, m) =0 n ∂ mˆ μj
(182)
302
VICENTE ET AL.
The solutions for the above equations are the equations being solved by the probability propagation decoding algorithm:
m sμl = tanh mˆ sμj = Jμ m nμl
ν∈Ms (l)\μ
i∈Ls (μ)\ j
= tanh
mˆ nμj = Jμ
m sμi
m sμi
i∈Ls (μ)
!
m nμl ,
(183) (184)
l∈Ln (μ)
ν∈Mn (l)\μ
atanh mˆ sνl + Fs
atanh mˆ nνl + Fn
!
m nμl .
(185) (186)
l∈Ln (μ)\ j
The estimate for the message is ξˆ j = sgn(m sj ), where mjs is the local magnetization due to all couplings linked to the site j, can be computed as: m sj = tanh
ν∈Ms ( j)
atanh mˆ sν j + Fs
!
(187)
One possibility for the decoding dynamics is to update Eqs. (183) and (185) until a certain halting criteria is reached, and then computing the estimate for the message using Eq. (187). The initial conditions are set to reflect the prior knowledge about the message m sμj (0) = 1 − 2 pξ and noise m nμl (0) = 1 − 2 p. As the prior information is limited, a polynomial time decoding algorithm (like PP) will work only if the solution is unique or the initial conditions are inside the correct basin of attraction. In this case, the 2(NK + MC) Eqs. (181) only need to be iterated an O(1) number of times to get a successful decoding. On the other hand, when there are many solutions, it is possible to obtain improved decoding in exponential time by choosing random initial conditions and comparing free-energies of the solutions obtained, selecting a global minimum. Observe that the free-energy described here is not equivalent to the variational mean-field free-energy introduced in MacKay (1995, 1999). Here no essential correlations are disregarded, except those related to the presence of loops. In the next section, we will analyze the landscape of the replica symmetric free-energy for three families of construction parameters and will be able to predict the practical performance of a PP decoding algorithm.
LOW-DENSITY PARITY-CHECK CODES
303
E. Equilibrium Results and Decoding Performance The saddle-point Eqs. (177) can be solved by using Monte Carlo integration iteratively. In this section, we show that MN codes can be divided, as far as performance is concerned, into three parameter groups: K ≥ 3, K = 2, and K = 1, L > 1. We, therefore, treat each of these cases separately in the following. 1. Analytical Solution: The Case of K ≥ 3 Replica symmetric results for the cases of K ≥ 3 can be obtained analytically; therefore, we focus first on this simple case. For unbiased messages (Fs = 0), we can easily verify that the ferromagnetic state, characterized by ρ = 1, and the probability distributions π (x) = δ(x − 1) π( ˆ xˆ ) = δ(xˆ − 1) φ(y) = δ(y − 1)
ˆ yˆ ) = δ( yˆ − 1) φ(
(188)
and the paramagnetic state of ρ = 0 with the probability distributions π (x) = δ(x) ˆ = δ(x) ˆ πˆ (x)
ˆ yˆ ) = δ( yˆ ) φ( φ(y) =
1 + tanh(Fn ) δ(y − tanh(Fn )) 2 1 − tanh(Fn ) δ(y + tanh(Fn )), + 2
(189)
satisfy replica symmetric saddle-point Eqs. (177). Other solutions could be obtained numerically. To check for that, we represented the distributions with histograms of 20,000 bins and iterated Eqs. (177) 100–500 times with 2 × 105 Monte Carlo sampling steps for each iteration. No solutions other than ferromagnetic and paramagnetic have been observed. The thermodynamically dominant state is found by evaluating the freeenergy of the two solutions using Eq. (176), which yields f FERRO = −
C Fn tanh(Fn ), K
(190)
304
VICENTE ET AL.
for the ferromagnetic solution and C C ln 2 − ln 2 − ln (2 cosh(Fn )) , (191) K K for the paramagnetic solution. Figure 24(a) describes schematically the nature of the solutions for this case, in terms of the replica symmetric free-energy and overlap obtained, for various noise levels p and unbiased messages pξ = 1/2. The coexistence line in the code rate versus noise level plane is given by f PARA =
f FERRO − f PARA =
ln 2 [Rc − 1 + H2 ( p)] = 0. Rc
(192)
This can be rewritten as Rc = 1 − H2 ( p) = 1 + p log2 ( p) + (1 − p) log2 (1 − p),
(193)
which coincides with channel capacity and is represented in Figure 25(a) together with the overlap ρ as a function of the noise level p. Equation (193) seems to indicate that all constructions with K ≥ 3 may attain error-free data transmission for R < Rc in the limit where both message and codeword lengths N and M become infinite, thus saturating Shannon’s bound. However, as described in Fig. 24(a), the paramagnetic state is also stable for any noise level, which has dynamic implications if a replica symmetric freeenergy is to be used for decoding (as is the case in probability propagation decoding). To validate the solutions obtained we have to make sure that the entropy is positive. Entropies can be computed by simply plugging distributions (189) and (190) into Eq. (179). The energy densities for the unbiased case are u = u PARA = u FERRO = −α Fn (1 − 2 p), since the Nishimori condition is employed (see Appendix B.3). Ferromagnetic entropies are sFERRO = u − f FERRO = 0 and sPARA = u − f PARA
C C ln 2 + ln 2 + ln (2 cosh(Fn )) . (194) K K It can be seen by using a simple argument that sPARA is negative below pc. For p < pc , f PARA > f FERRO and u − sPARA > u − sFERRO . This indicates that the distribution (190) is nonphysical below pc, despite being a solution of replica symmetric saddle-point equations. This result seems to indicate that the replica symmetric free-energy does not provide the right description below pc. A simple alternative is to use the frozen spins solution as the formulation of a theory with replica symmetry breaking for highly diluted = −α Fn (1 − 2 p) −
Figure 24. Figures on the left side show schematic representations free-energy landscapes while figures on the right show overlaps ρ a function of the noise level p; thick and thin lines denote stable solutions of lower and higher free energies, respectively, dashed lines correspond to unstable solutions. (a) K ≥ 3—The solid line in the horizontal axis represents the phase where the ferromagnetic solution (F, ρ = 1) is thermodynamically dominant. The paramagnetic solution (P, ρ = 0) becomes dominant at pc, which coincides with the channel capacity. (b) K = 2—The ferromagnetic solution and its mirror image are the only minima of the free-energy up to ps (solid line). Above ps suboptimal ferromagnetic solutions (F′ , ρ < 1) emerge. The thermodynamic transition occurs at p3 is below the maximum noise level given by the channel capacity, which implies that these codes do not saturate Shannon’s bound even if optimally decoded. (c) K = 1— The solid line in the horizontal axis represents the range of noise levels where the ferromagnetic state (F) is the only minimum of the free-energy. The suboptimal ferromagnetic state (F′ ) appears in the region represented by the dashed line. The dynamic transition is denoted by ps, where F′ first appears. For higher noise levels, the system becomes bistable and an additional unstable solution for the saddle point equations necessarily appears. The thermodynamic transition occurs at the noise level p1 where F′ becomes dominant.
306
VICENTE ET AL.
Figure 25. Transition lines in the plane rate R versus the flip rate p, obtained from numerical solutions and the TAP approach (N = 104 ), and averaged over 10 different initial conditions with error bars much smaller than the symbols size. (a) Numerical solutions for K = L = 3, C = 6 and varying input bias fs ( ) and TAP solutions for both unbiased (+) and biased (♦) messages; initial conditions were chosen close to the analytical ones. The critical rate is multiplied by the source information content to obtain the maximal information transmission rate, which clearly does not go beyond R = 3/6 in the case of biased messages; for unbiased patterns, H2 ( f s ) = 1. (b) For the unbiased case of K = L = 2, initial conditions for the TAP (+) and the numerical solutions (♦) were chosen to be of almost zero magnetization. (c) For the case of K = 1, L = 2 and unbiased messages. We show numerical solutions of the analytical equations (♦) and those obtained by the TAP approach (+). The dashed line indicates the performance of K = L = 2 codes for comparison. Codes with K = 1, L = 2 outperform K = L = 2 for code rates R < 1/3.
systems, which is a difficult task (see, e.g., Wong and Sherrington, 1988; Monasson, 1998b). Nevertheless, the practical performance of the probability propagation decoding is described by the replica symmetric theory, the presence of paramagnetic stable states implies the failure of PP decoding at any noise level. Even without knowing the correct physics below pc, it is possible to use an
LOW-DENSITY PARITY-CHECK CODES
307
exhaustive search for the global minimum of the free-energy in Section V.D to attain Shannon’s bound in exponential time. 2. The Case of K = 2 All codes with K ≥ 3 potentially saturate Shannon’s bound and are characterized by a first-order phase transition between the ferromagnetic and paramagnetic solutions. Solutions for the case with K = 2 can be obtained numerically, yielding significantly different physical behavior as shown in Figure 24(b). At very large noise levels, the paramagnetic solution (190) gives the unique extremum of the free-energy until the noise level reaches p1, at which the ferromagnetic solution (189) of higher free-energy becomes locally stable. As the noise level decreases to p2, the paramagnetic solution becomes unstable and a suboptimal ferromagnetic solution and its mirror image emerge. Those solutions have lower free-energy than the ferromagnetic solution until the noise level reaches p3. Below p3, the ferromagnetic solution becomes the global minimum of the free-energy, while the suboptimal ferromagnetic solutions remain locally stable. However, the suboptimal solutions disappear at the spinodal noise level ps and the ferromagnetic solution (and its mirror image) becomes the unique stable solution of the saddle-point Eqs. (177). The analysis implies that p3, the critical noise level below which the ferromagnetic solution becomes thermodynamically dominant, is lower than pc = H2−1 (1 − R) which corresponds to Shannon’s bound. Namely, K = 2 does not saturate Shannon’s bound in contrast to K ≥ 3 codes even if decoded in exponential time. Nevertheless, it turns out that the free-energy landscape, with a unique minimum for noise levels 0 < p < ps , offers significant advantages in the decoding dynamics compared to that of codes with K ≥ 3, allowing for the successful use of polynomial time probability propagation decoding. 3. The Case of K = 1 and General L > 1 The choice of K = 1, independent of the value chosen for L > 1, exhibits a different behavior presented schematically in Figure 24(c); also in this case there are no simple analytical solutions and all solutions in this scenario but the ferromagnetic one have been obtained numerically. The first important difference to be noted is that the paramagnetic state (190) is no longer a solution of the saddle-point Eqs. (177) and is being replaced by a suboptimal ferromagnetic state, very much like Gallager codes. Convergence to ρ = 1 solution can only be guaranteed for noise levels p < ps , where only the ferromagnetic solution is present.
308
VICENTE ET AL.
Figure 26. Free-energies obtained by solving the analytical equations using Monte Carlo integrations for K = 1, R = 1/6 and several values of L. Full lines represent the ferromagnetic free-energy (FERRO, higher on the right) and the suboptimal ferromagnetic free-energy (higher on the left) for values of L = 2, . . . , 7. The dashed line indicates Shannon’s bound and the arrows represent the spinodal point values ps for L = 2, . . . , 7. The thermodynamic transition coincides with Shannon’s bound.
The K = 1 codes do not saturate Shannon’s bound in practice; however, we have found that at rates R < 1/3 they outperform the K = L = 2 code (see Fig. 25) while offering improved decoding times when probability propagation is used. Studying the replica symmetric free-energy in this case shows that as the corruption rate increases, suboptimal ferromagnetic solutions (stable and unstable) emerge at the spinodal point ps. When the noise increases further, this suboptimal state becomes the global minimum at p1, dominating the system’s thermodynamics. The transition at p1 must occur at noise levels lower or equal to the value predicted by Shannon’s bound. In Figure 26 we show free-energy values computed for a given code rate and several values of L, denoting Shannon’s bound by a dashed line; the thermodynamic transition observed numerically (i.e., the point where the ferromagnetic free-energy equals the suboptimal ferromagnetic free-energy) is closely below Shannon’s bound within the numerical precision used. Spinodal noise levels are indicated by arrows. In Figure 27 we show spinodal noise levels as a function of L as predicted by the replica symmetric theory (circles) and obtained by running PP decoding of codes with size 104. The optimal parameter choice is L = 2. Due to the simplicity of the saddle-point Eqs. (177) we can deduce the asymptotic behavior of K = 1 and L = 2 codes for small rates (large C) by computing the two first cumulants of the distributions π, πˆ , φ, and φˆ (Gaussian approximation). A decoding failure corresponds to h ∼ O(1) and
309
LOW-DENSITY PARITY-CHECK CODES
Figure 27. Spinodal point noise level ps for K = 1, R = 1/6 and several choices of L. Numerical solutions are denoted by circles and PP decoding solutions (10 runs with size N = 104 ) by black triangles. Symbols are larger than the error bars.
ˆ ∼ O(1/C) and σxˆ ∼ O(1/C). For that, y must σh2 ∼ O(1). It implies that x be small and we can use atanh(tanh(y1 )tanh(y2 )) ≈ y1 y2 and write: x ∼ O(1) ˆ ≈ y2 x σxˆ2
2 2
σx2 ∼ O(1)
(195) (196)
≈ y − y
4
(197)
y = yˆ + (1 − 2 p)Fn
σ y2 = σ yˆ2 + 4 f (1 − p)Fn2
yˆ ≈ tanh(x)y
(198) (199)
σ yˆ2 ≈ tanh2 (x)y 2 − tanh(x)2 y2
(200)
To simplify further we can assume that p → 0.5. Therefore, Fn ≈ (1 − 2 p). The critical observation is that in order√to have h ∼ O(1), we need that xˆ ∼ O(1/C) and consequently y ∼ O(1/ C). Manipulating the set of equations above: y ≈ tanhxy + (1 − 2 f )2 By imposing the condition over y : C −1/2 ∼ (1 − 2 p)2 (1 − tanhx)−1 In terms of the code rate R = 1/C: R∼
(1 − 2 p)4 (1 − tanhx)2
(201)
310
VICENTE ET AL.
Figure 28. Asymptotic behavior of the transition for small rates. The full line represents Shannon’s bound, circles represent transitions obtained by using only the first cumulants, and squares correspond to the Gaussian approximation.
The asymptotic behavior of Shannon’s bound is given by: R∼
(1 − 2 p)2 ln 2
(202)
Thus, the K = 1 and L = 2 codes are not optimal asymptotically (large C values). In Figure 28 we verify the relation (201) by iterating first cumulant equations in the delta approximation and first and second cumulant equations in the Gaussian approximation.
F. Error Correction: Regular vs. Irregular Codes Matrix construction irregularity can improve the practical performance of MN codes. This fact was first reported in the information theory literature (see, e.g., Davey, 1998, 1999; Luby et al., 1998). Here we analyze this problem by using the language and tools of statistical physics. We now use the simplest irregular constructions as an illustration; here, the connectivities of the signal matrix Cs are described by a simple bimodal probability distribution:
PC (C) = (1 − θ) δ(C − Co ) + θ δ(C − Ce ).
(203)
312
VICENTE ET AL.
transition. The thermodynamic transition coincides with the upper bound (u.b.) in Section V.A and is very close to, but below, Shannon’s limit which is shown for comparison. Similar behavior was observed in regular MN codes with K = 1. G. The Spinodal Noise Level The PP algorithm can be regarded as an iterative solution of fixed-point equations for the free-energy (181) which is sensitive to the presence of local minima in the system. One can expect convergence to the global minimum of the free-energy from all initial conditions when there is a single minimum or when the landscape is dominated by the basin of attraction of this minimum when random initial conditions are used. To analyze this point, we run decoding experiments starting from initial conditions m sμj (0) and m nμl (0) that are random perturbations of the ferromagnetic solution drawn from the following distributions: P m sμj (0) = (1 − λs ) δ m sμj (0) − ξ j + λs δ m sμj (0) + ξ j (204) and
P m nμl (0) = (1 − λn ) δ m nμl (0) − τl + λn δ m nμl (0) + τl ,
(205)
where for convenience we choose 0 ≤ λs = λn = λ ≤ 0.5. We performed PP decoding several times for different values of λ and noise level p. For λ ≤ 0.026, we observed that the system converges to the ferromagnetic state for all constructions, message biases pξ , and noise levels p examined. It implies that this state is always stable. The convergence occurs for any λ for noise levels below the transition observed in practice. These observations suggest that the ferromagnetic basin of attraction dominates the landscape up to some noise level ps. The fact that no other solution is ever observed in this region suggests that ps is the noise level where suboptimal solutions actually appear, namely, it is the noise level that corresponds to the appearance of spinodal points in the free-energy. The same was observed for regular MN codes with K = 1 or K = 2. We have shown that MN codes can be divided into three categories with different equilibrium properties: (i) K ≥ 3, (ii) K = 2, and (iii) general L > 1, K = 1. In the next two subsections we will discuss these cases separately. 1. Biased Messages: K ≥ 3 To show how irregularity affects codes with this choice of parameters, we choose K , L = 3, Co = 4, Ce = 30 and biased messages with pξ = 0.3. These
LOW-DENSITY PARITY-CHECK CODES
311
Figure 29. (a) Overlap as a function of the noise level p for codes with K = L = 3 and C = 15 with message bias pξ = 0.3. Analytical RS solutions for the regular code are denoted as ♦ and for the irregular code; with Co = 4 and Ce = 30 denoted as . Results are averages over 10 runs of the PP algorithm in an irregular code of size N = 6000 starting from fixed initial conditions (see the text); they are plotted as 䊉 in the rightmost curve for comparison. PP results for the regular case agree with the theoretical solutions and have been omitted to avoid overloading the figure. (b) Free-energies for the ferromagnetic state (full line) and for the failure state (line with 䊊). The transitions observed in (a) are indicated by the dashed lines. Arrows indicate the thermodynamic (T) transition, the upper bound (u.b.) of Section V.A, and Shannon’s bound.
The mean connectivity is C = (1 − θ)Co + θCe and Co < C < Ce ; bits in a group with connectivity Co will be referred as ordinary bits and bits in a group with connectivity Ce as elite bits. The noise matrix Cn is chosen to be regular. To gain some insight into the effect of irregularity on solving the PP Eqs. (183) and (185), we performed several runs starting from the fixed initial conditions m sμj (0) = 1 − 2 pξ and m nμl (0) = 1 − 2 p as prescribed in the last section. For comparison, we also iterated the saddle-point Eqs. (177) obtained by the replica symmetric (RS) analysis, setting the initial conditions to be π0 (x) = (1 − pξ ) δ(x − m sμj (0)) + pξ δ(x + m sμj (0)) and ρ0 (y) = (1 − p) δ(y − m nμl (0)) + p δ(y + m nμl (0)), as suggested from the interpretation of the fields π (x) and ρ(y) in the last section. In Figure 29(a) we show a typical curve for the overlap ρ as a function of the noise level p. The RS theory agrees very well with PP decoding results. The addition of irregularity improves the performance considerably. In Figure 29(b) we show the free-energies of the two emerging states. The free-energy for the ferromagnetic state with overlap ρ = 1 is shown as a full line; the failure suboptimal ferromagnetic state (in Fig. 29(a) with overlap ρ = 0.4) is shown as a line marked with 䊊. The transitions seen in Fig. 29(a) are denoted by dashed lines. It is clear that they are far below the thermodynamic (T) transition, indicating that the system becomes trapped in suboptimal ferromagnetic states for noise levels p between the observed transitions and the thermodynamic
LOW-DENSITY PARITY-CHECK CODES
313
Figure 30. Spinodal noise level ps for regular and irregular codes. In both constructions, parameters are set as K = L = 3. Irregular codes with Co = 4 and Ce = 30 are used. PP decoding is carried out with N = 5000 and a maximum of 500 iterations; they are denoted by + (regular) and ∗ (irregular). Numerical solutions for the RS saddle-point equations are denoted by ♦ (regular) and 䊊 (irregular). Shannon’s limit is represented by a full line and the upper bound of Section V.A. is represented by a dashed line. The symbols are chosen to be larger than the actual error bars.
choices are arbitrary but illustrate what happens with the practical decoding performance. In Figure 30 we show the transition from the decoding phase to a failure phase as a function of the noise level p for several rates R in both regular and irregular codes. Practical decoding (♦ and 䊊) results are obtained for systems of size N = 5000 with a maximum number of iterations set to 500. Random initial conditions are chosen and the whole process repeated 20 times. The practical transition point is found when the number of failures equals the number of successes. These experiments were compared with the theoretical values for ps obtained by solving the RS saddle-point Eqs. (177) (represented as + and ∗ in Fig. 30) and finding the noise level for which a second solution appears. For comparison the coding limit is represented in the same figure by a full line. As the constructions used are chosen arbitrarily, one can expect that these transitions can be further improved, even though the improvement shown in Figuer 30 is already fairly significant. The analytical solution obtained for K ≥ 3 and unbiased messages pξ = 1/2 implies that the system is bistable for arbitrary code constructions when these parameters are chosen. The spinodal noise level is then ps = 0 in this case and cannot be improved by adding irregularity to the construction. Up to the noise
314
VICENTE ET AL.
Figure 31. Spinodal noise level ps for irregular codes as a function of the message bias pξ . The construction is parameterized by K = L = 3, Co = 4, and Ce = 30 with C = 15. PP decoding is carried out with N = 5000 and a maximum of 500 iterations, and is represented by +, while theoretical RS solutions are represented by ♦. The full line indicates Shannon’s limit. Symbols are larger than the actual error bars
level pc, the ferromagnetic solution is the global minimum of the free-energy, and therefore Shannon’s limit is achievable in exponential time; however, the bistability makes these constructions unsuitable for practical decoding with a PP algorithm when unbiased messages are considered. The situation improves when biased messages are used. Fixing the matrices Cn and Cs , one can determine how the spinodal noise level ps depends on the bias pξ . In Figure 31 we compare simulation results with the theoretical predictions of ps as a function of pξ . The spinodal noise level ps collapses to zero as pξ increases toward the unbiased case. It obviously suggests using biased messages for practical MN codes with parameters K ≥ 3 and PP decoding. The qualitative pictures of the energy landscape for coding with biased and unbiased messages with K ≥ 3 differ significatively. In Figure 32 this landscape is sketched as a function of the noise level p for a given bias. Up to the spinodal noise level ps, the landscape is totally dominated by the ferromagnetic state F. At the spinodal noise level, another suboptimal state F ′ emerges, dominating the decoding dynamics. At pc, the suboptimal state F ′ becomes the global minimum. The bold horizontal line represents the region where the ferromagnetic solution with ρ = 1 dominates the decoding dynamics. In the
LOW-DENSITY PARITY-CHECK CODES
315
Figure 32. Pictorial representation of the free-energy landscape for codes with K ≥ 3 and biased messages pξ < 0.5 as a function of the noise level p. Up to the spinodal noise level ps, there is only the ferromagnetic state F. At ps, another state F ′ appears, dominating the decoding dynamics. The critical noise level pc indicates the point where the state F ′ becomes the global minimum (thermodynamic transition).
region represented by the dashed line, decoding dynamics is dominated by suboptimal ferromagnetic ρ < 1 solutions. 2. Unbiased Messages For the remaining parameter choices, namely general L > 1, K = 1, and K = 2, it was shown that unbiased coding is generally possible, yielding close to Shannon’s limit performance. In the K ≥ 3 case, the practical performance is defined by the spinodal noise level ps and the addition of irregularity modifies ps. In the general L , K = 1 family we illustrate the effect of irregularity by the choice of L = 2, Co = 4, and Ce = 10. In Figure 33 we show the transitions observed by performing 20 decoding experiments with messages of length N = 5000 and a maximal number of iterations set to 500 (+ for regular and ∗ for irregular). We compare the experimental results with theoretical predictions based on the RS saddle-point Eqs. (177) (♦ for regular and 䊊 for irregular). Shannon’s limit is represented by a full line. The improvement is modest, as expected, since regular codes already present close-to-optimal performance. Discrepancies between the theoretical and numerical results are due to finite size effects. We also performed a set of experiments using K = L = 2 with Co = 3 and Ce = 8, the same system size N = 5000 and maximal number of decoding iterations 500. The transitions obtained experimentally and predicted by theory are shown in Figure 34.
316
VICENTE ET AL.
Figure 33. Spinodal noise level ps for regular and irregular codes. The constructions are of K = 1 and L = 2, irregular codes are parameterized by Co = 4 and Ce = 10. PP decoding is carried out with N = 5000 and a maximum of 500 iterations; they are denoted by + (regular) and ∗ (irregular). Numerical solutions for RS equations are denoted by ♦ (regular) and 䊊 (irregular). The coding limit is represented by a line. Symbols are larger than the actual error bars.
Figure 34. Spinodal noise level values ps for regular and irregular codes. Constructions are of K = 2 and L = 2, irregular codes are parameterized by Co = 3 and Ce = 8. PP decoding is carried out with N = 5000 and a maximum of 500 iterations; they are denoted by + (regular) and ∗ (irregular). Theoretical predictions are denoted by ♦ (regular) and 䊊 (irregular). The coding limit is represented by a line. Symbols are larger than the actual error bars.
LOW-DENSITY PARITY-CHECK CODES
319
{α j } completely specify the construction. A further constraint to the parameters set {α j } is provided by the choice of a code rate, as the inverse code rate is α = M/N = mj=1 α j . Encoding and decoding using cascading codes are performed in exactly the same fashion as described in Section V for MN codes. A binary vector t ∈ {0, 1} M defined by t = Gξ (mod 2),
(206)
is produced, where all operations are performed in the field {0, 1} and are indicated by (mod 2). The code rate is R = N /M. The generator matrix G is a M × N dense matrix defined by G = C −1 n C s (mod 2).
(207)
The transmitted vector τ is then corrupted by noise. Assuming a memoryless BSC, noise is represented by a binary vector ζ ∈ {0, 1} M with components independently drawn from the distribution P(ζ ) = (1 − p) δ(ζ ) + p δ(ζ − 1). The received vector is r = Gξ + ζ (mod 2).
(208)
Decoding is performed by computing the syndrome vector z = Cnr = Cs ξ + Cn ζ (mod 2),
(209)
from which an estimate ξˆ for the message can be obtained.
A. Typical PP Decoding and Saddle-Point-Like Equations In this section, we show how a statistical description for the typical PP decoding can be constructed without using replica calculations. To keep the analysis as simple as possible, we exemplify the procedure with a KS code with two signal matrices denoted 1s and 2s and two noise submatrices denoted 1n and 2n. The channel is chosen to be a memoryless BSC. The number of nonzero elements per row is K 1 and K 2 , respectively, and the inverse rate is α = α1 + α2 . Therefore, for a fixed code rate, the code construction is specified by a single parameter α1 . We present one code in this family in Figure 37. The PP decoding dynamics for these codes is described by Eqs. (185). However, due to the irregular character of the construction, sites inside each one of the submatrices are connected differently. Remembering the statistical physics formulation of MN codes presented in Section V.B, nonzero row elements in the matrices depicted in Figure 37 correspond to sites taking part in one multispin interaction. Therefore, signal sites in the submatrix 1s interact with other
LOW-DENSITY PARITY-CHECK CODES
317
Figure 35. Bit error probability pb as a function of the signal to noise ratio for codes of rate R = 1/2, sizes N = 1000 (right) and N = 10000 (left) in a memoryless Gaussian channel. Black triangles represent cascading codes, dashed lines represent Turbo codes and dotted lines represent optimized irregular Gallager codes of similar sizes (Kanter and Saad, 2000b).
VI. Cascading Codes Kanter and Saad (KS) recently proposed a variation of MN codes that has been shown to be capable of attaining close-to-channel capacity performance and outperforming Turbo codes (Kanter and Saad, 1999, 2000a,b). The central idea is to explore the superior dynamic properties (i.e., large basin of attraction) of MN codes with K = 1, 2 and the potential for attaining channel capacity of MN codes with K > 2 by introducing constructions with intermediate properties. This is done by employing irregular constructions like the one depicted in Figure 35, with the number of nonzero elements per row set to several different values K 1 , . . . , K m . In Figure 35 we show a performance comparison (presented in (Kanter and Saad, 2000b) of Turbo, KS, and Gallager codes with optimized irregular constructions (Richardson et al., 2001) for a memoryless Gaussian channel. The bit error probability pb is plotted against the signal to noise ratio in decibels (10 log10 (S/N)) for codes of sizes N = 1000 and N = 10000. The introduction of multispin interactions of several different orders and of more structured matrices makes the statistical physics of the problem much harder to solve. We, therefore, adopt a different approach: first we write the probability propagation equations and find an appropriate macroscopic
318
VICENTE ET AL.
Figure 36. Cascading construction with three signal submatrices with K 1 , K 2 and K 3 nonzero elements per row, respectively. The number of nonzero elements per column is kept fixed to C. The noise matrix Cn is composed by two submatrices, the nonzero elements are denoted by lines. The inverse Cn−1 is also represented.
description in terms of field distributions, we then solve saddle-point-like equations for the field distributions to find the typical performance. Cascading codes are specific constructions of MN codes. The signal matrix Cs is defined by m random submatrices with K 1 , K 2 , . . . , K m nonzero elements per row, respectively. The matrix Cn is composed of two submatrices: C (1) ni j = −1 = δ . The inverse C used in the encoding process δi, j + δi, j+ and C (2) i, j ni j n is easily obtainable. In Figure 36 we represent a KS code with three signal submatrices, the nonzero elements in the noise matrix Cn are denoted by lines, we also represent the inverse of the noise matrix C −1 n . The signal matrix Cs is subdivided into M j × N submatrices, with j = 1, . . . , m. The total number of nonzero elements is given by N C = mj=1 M j K j what yields C = mj=1 α j K j , where α j = M j /N . The code construction is, therefore, parameterized by the set {(α j , K j )}. If we fix {K j }, the parameters
321
LOW-DENSITY PARITY-CHECK CODES
For the submatrix 1s we have: m (1s) atanh mˆ (1s) + νj μj = tanh ν∈M1s ( j)\μ
(1n) (1n) mˆ (1s) μj = Jμ m μμ m μμ+
ν∈M2s ( j)
m (1s) μl ,
atanh mˆ (2s) + Fs νj
! (211)
l∈L1s (μ)\ j
where the second equation represents interactions with two noise sites and and K 1 − 1 signal sites. The first equation represents the α1 K 1 + α2 K 2 − 1 multispin interactions the site j participates in. Similarly, for the submatrix 2s we have: ! (1s) (2s) (2s) atanh mˆ ν j + atanh mˆ ν j + Fs m μj = tanh ν∈M1s ( j)
(2n) mˆ (2s) μj = Jμ m μ
ν∈M2s ( j)\ν
m (2s) μl
(212)
l∈L2s (μ)\ j
For the submatrix 1n we have:
E D (1n) ˆ m = tanh atanh + F m (1n) n νj μj (1n) mˆ (1n) m (1s) μj = Jμ m μi μl ,
(213) (214)
l∈L1s (μ)
where either j = μ, i = μ + or j = μ + , i = μ. Finally, for submatrix 2n we have: m (2n) = tanh [Fn ] μ mˆ (2n) = Jμ m (2s) μl μ
(215) (216)
l∈L2s (μ)
The pseudo-posterior and decoded message are given by: m j = tanh
ν∈M1s ( j)
ξˆ j = sgn(m j ).
atanh mˆ (1s) + νj
ν∈M2s ( j)
! (2s) atanh mˆ ν j
(217) (218)
The above equations provide a microscopic description for the PP decoding process. We can produce a macroscopic description for the typical decoding process by writing equations for probability distributions related to the dynamic variables. It is important to stress that the equations describing the PP decoding are entirely deterministic when couplings Jμ and initial conditions are given. The randomness comes into the problem when quenched averages over messages, noise, and constructions are introduced.
320
VICENTE ET AL.
Figure 37. Cascading code with two signal matrices with parameters K 1 and K 2 . Note that noise sites inside the shaded regions take part in a different number of interactions than the ordinary sites.
K 1 − 1 signal sites in 1s and exactly two noise sites in 1n. Moreover, the same site takes part in other α1 K 1 + α2 K 2 − 1 multispin couplings in both 1s and 2s. Sites in submatrix 2s interact with one noise site in 2n and K 2 − 1 signal sites in 2s, taking part in other α1 K 1 + α2 K 2 − 1 multispin interaction. Noise sites in the submatrix 1n interact with another noise site and with K 1 signal sites in 1s. Finally, noise sites in 2n interact with K 2 sites in 2s. Thus, the Hamiltonian for a KS code takes the following form:
H = −γ −γ
M1 (Jμ Si1 · · · Si K1 τμ τμ+ − 1) μ=1
M
μ=M1 +1
(Jμ Si1 · · · Si K2 τμ − 1) − Fn
M l=1
τl − Fs
N
S j , (210)
j=1
where Jμ = ξi1 · · · ξi K1 ζμ ζμ+ , for μ = 1, . . . , M1 and Jμ = ξi1 · · · ξi K2 ζμ for μ = M1 + 1, . . . , M. Additionally, Nishimori’s condition requires that γ → ∞, Fs = atanh(1 − 2 pξ ) and Fn = atanh(1 − 2 p), where the prior probabilities are defined as in the previous chapters. We can write PP decoding equations for each one of the submatrices 1s, 2s, 1n and 2n. The shaded regions in Figure 37 have to be described by different equations, but can be disregard if the width is of O(1), implying /N → 0 for N → ∞.
322
VICENTE ET AL.
By performing the gauge transformation (as) ˆ μj mˆ ν(as) j → ξjm
(as) (as) m μj → ξ j m μj
(an) mˆ μj → ζ j mˆ (an) μj
(an) m (an) μj → ζ j m μj
Jμ → 1
(219) (220)
(a = 1, 2),
(221)
introducing effective fields xμj = atanh(m μj ), xˆ μj = atanh(mˆ μj ) and assum(as) (as) (an) ing that xμj , xˆ μj , yμj , yˆ (an) μj are independently drawn from distributions ˆ ˆ Pa (x), Pa (xˆ ), Ra (y), Ra ( yˆ ), respectively, we get the following saddle-pointlike equations (for simplicity, we restrict the treatment to the case of unbiased messages Fs = 0). For the submatrix 1s: P1 (x) =
)
α1 K 1 −1
d xˆ j Pˆ 1 (xˆ j )
j=1
×δ x − ˆ = Pˆ 1 (x)
) K 1 −1
α 1 K2
dw ˆ l Pˆ 2 (w ˆ l)
l=1
α1 K 1 −1 j=1
xj −
α 2 K2
wl
l=1
!
(222)
d x j P1 (x j ) dy1 R1 (y1 ) dy2 R1 (y2 )
j=1
× δ xˆ − atanh tanh(y1 )tanh(y2 )
K 1 −1
tanh(x j )
j=1
!
(223)
For 2s: P2 (x) =
) α 1 K1
d xˆ j Pˆ 1 (xˆ j )
) K 2 −1
ˆ l) dw ˆ l Pˆ 2 (w
l=1
j=1
×δ x − ˆ = Pˆ 2 (x)
α1 K 2 −1
α 1 K1 j=1
xj −
α2 K 2 −1
wl
l=1
!
(224)
d x j P2 (x j ) dy R2 (y)
j=1
× δ xˆ − atanh tanh(y)
K 2 −1 j=1
!
tanh(x j )
(225)
323
LOW-DENSITY PARITY-CHECK CODES
For 1n we have: R1 (y) = Rˆ 1 ( yˆ ) =
)
d yˆ Rˆ 1 ( yˆ ) δ [y − yˆ − ζ Fn ]ζ
) K1
d x j P1 (x j ) dy R1 (y)
j=1
× δ xˆ − atanh tanh(y)
K1
tanh(x j )
j=1
!
(226)
Finally, for submatrix 2n: R2 (y) = δ [y − ζ Fn ]ζ ! ) K2 K2 Rˆ 2 ( yˆ ) = tanh(x j ) d x j P2 (x j )δ xˆ − atanh
(227)
j=1
j=1
The typical overlap can then be obtained as in the case of MN codes by computing: ) ρ = dh P(h) sgn(h) (228) P(h) =
) α 1 K1 j=1
d xˆ j Pˆ 1 (xˆ j )
α 1 K2 l=1
dw ˆ l Pˆ 2 (w ˆ l) δ h −
α 1 K1 j=1
xj −
α 2 K2 l=1
wl
!
(229)
The numerical solution of these equations provides the typical overlap for cascading codes with two signal matrices parameterized by α1 (α2 = α − α1). In Figure 38 we compare results obtained by solving the above equations numerically (Monte Carlo integration with 4000 bins) and PP decoding simulations (10 runs, N = 5000) with R = 1/5 and α1 = 3. The agreement between theory and experiments supports the assumptions employed to obtain the saddle-point-like equations.
B. Optimizing Construction Parameters Equations (222) to (229) can be used to optimize code constructions within a given family. For the family introduced in Figure 37 with fixed parameters K1 and K2, the optimization requires finding the value of α 1 that produces the highest threshold ps. In Figure 39 we show the threshold (spinodal noise level) ps for a KS code with K 1 = 1, K 2 = 3 and rate R = 1/5 (α = 5). The optimal performance is obtained by selecting α1 = 3 and is very close to the channel capacity.
324
VICENTE ET AL.
Figure 38. Monte Carlo integration of field distributions and simulations for a KS code with two signal matrices (K 1 = 1 and K 2 = 3), α = 5 (R = 1/5) and α1 = 3. Circles: full statistics (4000 bins). Squares: simulations N = 5000.
Figure 39. Spinodal noise level ps as a function of α 1 for a KS code with K 1 = 1, K 2 = 3 and R = 1/5 (α = 5). Circles: Monte Carlo integrations of saddle-point equations (4000 bins). Squares: PP decoding simulations (10 runs with size N = 5000). The best performance is reached for α1 = 3 and is close to the channel capacity for a BSC (indicated by a dashed line).
LOW-DENSITY PARITY-CHECK CODES
325
VII. Conclusions and Perspectives In this chapter we have analyzed error-correcting codes based on very sparse matrices by mapping them onto spin systems of the statistical physics. The equivalence between coding concepts and statistical physics is summarized in a table. Coding theory
Statistical physics
Message bits s Received bits r Syndrome bits z Bit error probability pe Posterior probability MAP estimator MPM estimator
Spins S Multispin disordered couplings J (Sourlas) Multispin couplings J (Gallager, MN, KS) Gauged magnetization ρ (overlap) Boltzmann weight Ground state Thermal average at Nishimori’s temperature
In the statistical physics framework, random parity-check matrices (or generator matrices as in the case of Sourlas codes), random messages, and noise are treated as quenched disorder and the replica method is employed to compute the free-energy. Under the assumption of replica symmetry, we found in most of the cases that two phases emerge: a successful decoding (ρ = 1) and failure (ρ < 1) phases. For MN codes with K = 2 or K = 1, three phases emerge, representing successful decoding, failure, and catastrophic failure. The general picture that emerges shows a phase transition between successful and failure states that coincides with the information theory upper bounds in most cases, the exception being MN codes with K = 2 (and to some extent K = 1) where the transition is bellow the upper bound. A careful analysis of replica symmetric quantities reveals unphysical behavior for low noise levels with the appearance of negative entropies. This question is resolved in the case of Sourlas codes with K → ∞ by the introduction of a simple frozen spins first-step replica symmetry breaking ansatz. Despite the difficulties in the replica symmetric analysis, threshold noise values observed in simulations using probability propagation (PP) decoding agree with the noise level where metastable states (or spinodal points) appear in the replica symmetric free-energy. A mean-field (Bethe) theory based on the use of a tree-like lattice (Husimi cactus) exposes the relationship between PP decoding and statistical physics and supports the agreement between theory and simulations as PP decoding can be reinterpreted as a method for finding local minima of a Bethe freeenergy. Those minima can be described by distributions of cavity local fields that are solutions of the replica symmetric saddle-point equations.
326
VICENTE ET AL.
The performance of the decoding process with probability propagation can be obtained by looking at the Bethe free-energy landscape (or the replica symmetric landscape); in this way we can show that information theoretic upper bounds can be attained by looking for global minima of the Bethe free-energy, which may require computing time that grows exponentially with the system size. In practical time scales, simple decoding procedures that simply find minima become trapped in metastable states. That is the reason practical thresholds are linked to the appearance of spinodal points in the Bethe free-energy. For cascading codes, we adopted a different approach for the analysis. Using the insights obtained in the analysis of the other codes, we started by writing down the PP decoding equations and writing the Bethe free-energy and the saddle-point-like equations for distributions of cavity fields. The transitions predicted by these saddle-point-like equations were shown to agree with experiments. We then employed this procedure to optimize parameters of one simple family of cascading codes. By studying the replica symmetric landscape we classified the various codes by their construction parameters, we also showed that modifications in code construction, such as the use of irregular matrices, can improve the performance by changing the way the free-energy landscape evolves with the noise level. We summarize in a table the results obtained.
Channel capacity Sourlas Gallager MacKay–Neal Cascading
K →∞ K →∞ K >2 Still unclear
Practical decoding of unbiased messages K =2 Any K K = 1, any L > 1 or K = 2 K j = 1, 2 for some j
These results shed light on the properties that limit the theoretical and practical performance of parity-check codes, explain the differences between Gallager and MN constructions, and explore the role of irregularity in LDPC error-correcting codes. Some new directions are now being pursued and are worth mentioning. The statistical physics of Gallager codes with nonbinary alphabets is investigated in Nakamura et al. (2001). In Kabashima et al. (2001), the performance of errorcorrecting codes in the case of finite message lengths has been addressed, yielding tighter general reliability bounds. New analytical methods to investigate practical noise thresholds using statistical physics have been proposed in van Mourik et al. (2001) and in Kabashima et al. (2001), while the nature of Gallager codes phase diagram was studied in detail in Montanari (2001).
327
LOW-DENSITY PARITY-CHECK CODES
We believe that methods developed over the years in the statistical physics community can make a significant contribution in other areas of information theory. Research in some of these areas, such as CDMA and image restoration, is already underway.
Appendix A. Sourlas Codes: Technical Details 1. Free-Energy To compute free-energies we need to calculate the replicated partition function (62). We can start from Eq. (60): < = (A.1) Z n A,ξ,J = Tr{S αj } exp − β H(n) ({Sα }) A,J,ξ , where H(n) ({Sα }) represents the replicated Hamiltonian and α the replica indices. First, we average over the parity-check tensors A; for that, an appropriate distribution has to be introduced, denoting μ ≡ i 1 , . . . , i K for a specific set of indices: 7 6 1 α (n) −βH ({S }) n δ Aμ − C Tr{Sαj } e , (A.2) Z = N {A} i μ\i J,ξ
where the δ distribution imposes a restriction on the connectivity per spin, N is a normalization coefficient, and the notation μ\i means the set μ except the element i. Using integral representations for the delta functions and rearranging: 6 C 1 d Zi 1 n Z = Tr{S αj } N 2πi Z iC+1 i Aμ 7 . (A.3) exp − β H(n) ({Sα }) Zi × {A}
μ
i∈μ
J,ξ
Remembering that A ∈ {0, 1}, and using the expression (50) for the Hamiltonian, we can change the order of the summation and the product above and sum over A: 6 C 1 d Zi 1 α n Z = Tr{S αj } eβ F α,i ξi Si C+1 N 2πi Zi i !7 α × Z i exp β Jμ . (A.4) Si 1+ μ
i∈μ
α
i∈μ
J,ξ
328
VICENTE ET AL.
3 3 Using the identity exp(β Jμ i∈μ Siα ) = cosh(β) [1 + ( i∈μ Siα ) tanh(β Jμ )], we can perform the product over α to write: C d Zi 1 < β F ξi Sα = 1 α,i i e Z n = Tr{Sαj } C+1 ξ N 2πi Z i i n Siα × Z i cosh (β) 1 + tanh(β J ) J 1+ μ
α
i∈μ
+ tanh2 (β J ) J
Siα1
α1 α2 i∈μ
j∈μ
S αj 2 + · · ·
!
i∈μ
(A.5)
.
Defining as an ordered set of sets, and observing that for μ1 , μ2 , . . . , μl large N, μ1···μl (· · ·) = l!1 ( μ (· · ·))l , we can perform the product over the sets μ and replace the energy series by an exponential: C d Zi 1 < β F α,iξi S α = 1 i Z n = Tr{S αj } e ξ N 2πi Z iC+1 i n × exp cosh (β) Z i + tanh(β J ) J z i Siα μ
+ tanh2 (β J ) J
α
i∈μ
α1 α2 μ
i∈μ
Z i Siα1 Siα2 + · · ·
!
μ
i∈μ
.
(A.6)
Observing that μ = 1/K ! i1 ,...i K , defining Tl = coshn (β J ) tanhl (β J ) J and introducing auxiliary variables qα1···αm = N1 i Z i Siα1 · · · Siαm we find: C $) % ) d Zi 1 ˆ ˆ dq d q 1 d q dq α α 0 0 × ··· Z n A,ξ,J = N 2πi Z iC+1 2πi 2πi α i ! NK K K K × exp T0 q0 + T1 qα1 α2 + · · · qα + T2 K! α α1 α2 ! qα qˆ α + qα1 α2 qˆ α1 α2 + · · · × exp −N q0 qˆ 0 + α
× Tr{S αj }
<
eβ F
α α,i ξi Si
=
ξ
exp
i
α1 α2
qˆ 0 Z i +
α
qˆ α Z i Siα + · · ·
!
.
(A.7)
LOW-DENSITY PARITY-CHECK CODES
The normalization constant is given by: Aμ − C , N = δ i
{A}
329
(A.8)
μ\i
and can be computed using exactly the same methods as above, resulting in: C $) % d Zi 1 dq0 d qˆ 0 N = 2πi Z iC+1 2πi i ! NK K × exp Zi . q − N q0 qˆ 0 + qˆ 0 (A.9) K! 0 i
Computing the integrals over Zis and using Laplace method to compute the integrals over q0 and qˆ 0 we obtain: $ C %#2 " K qˆ 0 N K . (A.10) q − N q0 qˆ 0 + N ln N = exp Extrq0 ,qˆ 0 K! 0 C! The extremum point is given by q0 = N (1−K )/K [(K − 1)!C]1/K and qˆ 0 = (C N )(K −1/K ) [(K − 1)!]−1/K . Replacing the auxiliary variables in Eq. (A.7) using qα1···αm /q0 → qα1···αm and qˆ α1···αm /q0 → qˆ α1···αm , computing the integrals over Zi and using Laplace method to evaluate the integrals, we finally find Eq. (62). 2. Replica Symmetric Solution The replica symmetric free-energy (66) can be obtained by plugging the ans¨atz (65) into Eq. (A.7). Using Laplace method we obtain: " #2 C 1 n (A.11) exp N Extrπ,πˆ G1 − C G2 + G3 , Z A,ξ,J = N K where:
G1 = T0 + T1 + T2
K ) α
K )
α1 α2
( d x j π (x j ) tanh(βx j ))
j
j
( d x j π (x j ) tanh2 (βx j )) + · · · ,
(A.12)
331
LOW-DENSITY PARITY-CHECK CODES
Putting everything together, using Eq. (59) and some simple manipulation we find Eq. (66). 3. Local Field Distribution Here we derive explicitly Eq. (68). The gauge transformed overlap can be written as N 1 sign(m i )A,J,ξ , (A.19) ρ= N i=1
introducing the notation m i = Si , where · · · is a gauged average. For an arbitrary natural number p, one can compute pth moment of mi 6 7 < p= p −β nα=1 H(α) 1 2 Si · Si · · · · · Si e , (A.20) m i A,J,ξ = lim n→0
S1 ,...,Sn
A,J,ξ
(α)
where H denotes the gauged Hamiltonian of the αth replica. By performing the same steps described in the Appendices A.1 and A.2, introducing the auxiliary functions π (x) and π(y) ˆ defined in Eqs. (65), one obtains 6 7 ) C C < p= m i A,J,ξ = dy j π(y ˆ j ) tanh p β Fξ + β yj . (A.21) j=1
j=1
ξ
Employing the identity sign(x) + 1 = 2 lim
n→∞
n m=0
2n! (2n − m)!m!
$
1+x 2
%2n−m $
1−x 2
%m
(A.22)
which holds for any arbitrary real number x ∈ [−1, 1] and Eqs. (A.21) and (A.22), one obtains ) sign(m i )A,J,ξ + 1 = 2 dh P(h) × lim
n→∞
=
)
n m=0
C2n,m
$
1+h 2
%2n−m $
dh P(h) sign(h),
thus reproducing Eq. (68).
%m
(A.23)
where we introduced the local fields distribution 6 7 ) C C P(h) = , yj dy j πˆ (y j ) δ h − Fξ − j=1
1−h 2
j=1
ξ
(A.24)
330
VICENTE ET AL.
G2 = 1 + + and
)
d x d y π(x) π(y) ˆ tanh(βx) tanh(βy)
α
)
d x d y π (x) π(y) ˆ tanh2 (βx) tanh2 (βy) + · · · (A.13)
α1 α2
⎡6 8 C 7 d Zi 1 α Tr{S αj } ⎣ expβ F ξi Si 2πi Z iC+1 i α,i ξ ) Zi + dy πˆ (y) tanh(βy) Z i Siα × exp qˆ 0
1 G3 = ln N
α
i
+
α1 α2
Z i Siα1 Siα2
i
)
i
dy πˆ (y) tanh2 (βy) + · · ·
! 9
. (A.14)
The equation for G1 can be worked out by using the definition of Tm and the fact that ( α1 ...αl 1) = nl to write: 6 n 7 ) K K n G1 = cosh (β J ) . d x j π(x j ) 1 + tanh(β J ) tanh(βx j ) j=1
j=1
J
(A.15)
Following exactly the same steps we obtain: ) G2 = d x d y π(x) πˆ (y) (1 + tanh(βx) tanh(βy))n ,
and
8
G3 = ln Tr{Sα } ×
C
6
exp β Fξ
α
dZ 1 exp qˆ 0 Z 2πi Z C+1
)
S
α
7
ξ
dy π(y) ˆ
n α=1
(A.16)
!9
(1 + S α tanh(βy))
.
(A.17)
Computing the integral over Zi and the trace, we finally find: !n 9 8 ) C C < = qˆ C0 (1 + σ tanh(βyl )) dyl π(y ˆ l) G3 = ln eσβ Fξ ξ . C! l=1 l=1 σ =±1
(A.18)
332
VICENTE ET AL.
4. Zero Temperature Self-Consistent Equations In this section we describe how one can write a set of self-consistent equations to solve the zero temperature saddle-point Eqs. (84). Supposing a three peaks ans¨atz given by: πˆ (y) = p+ δ(y − 1) + p0 δ(y) + p− δ(y + 1) π(x) =
C−1
l=1−C
T[ p± , p0 ;C−1] (l) δ(x − l),
(A.25) (A.26)
with T[ p+ , p0 , p− ;C] (l) =
(C − 1)! k h m p p p . k!h!m! + 0 − {k,h,m; k−h=l; k+h+m=C−1}
(A.27)
We can consider the problem as a random walk, where π(y) ˆ describes the probability of one step of length y (y > 0 means one step to the right) and π(x) describes the probability of being at distance x from the origin after C − 1 steps. With this idea in mind, it is relatively easy to understand T[ p+ , p0 , p− ;C−1] (l) as the probability of walking the distance l after C − 1 steps with the probabilities p+ , p− and p0 of, respectively, moving right, left, and staying at the same position. We define the probabilities of walking right/left as ψ± = lC−1 T[ p+ , p0 , p− ;C−1] (±l). Using second saddle-point Eqs. (84): p+ =
)
K −1
d xl π (xl )
l=1
!6
δ 1 − sign J
K −1 l=1
xl min(|J |, | x1 |, . . . |
!7
. J
(A.28)
The right side of the above equality can be read as the probability of making K − 1 independent walks, such that after C − 1 steps: none is at origin and an even (for J = +1) or odd (for J = −1) number of walks is at the left side. Using this reasoning for p− and p0, we can finally write: p+ = (1 − p) +p +
⌊
⌊ K 2−1 ⌋
j=0
K −1 2 ⌋−1
j=0
p ψ−K −1
(K − 1)! 2 j K −2 j−1 ψ− ψ+ 2 j!(K − 1 − 2 j)!
(K − 1)! 2 j+1 K −2 j−2 ψ − ψ+ (2 j + 1)!(K − 2 − 2 j)!
odd(K − 1)
(A.29)
LOW-DENSITY PARITY-CHECK CODES
p− = (1 − p)
⌊ K 2−1 ⌋−1
j=0
333
(K − 1)! 2 j+1 K −2 j−2 ψ − ψ+ (K − 2 j − 2)!(2 j + 1)!
⌊ K 2−1 ⌋−1
(K − 1)! 2 j K −2 j−1 ψ− ψ + + (1 − p)ψ−K −1 odd(K − 1), (K − 2 j − 1)!2 j! j=0 (A.30) where odd(x) = 1(0) if x is odd (even). Using that p+ + p− + p0 = 1, one can obtain p0. A similar set of equations can be obtained for a five-peaks ans¨atz leading to the same set of solutions for the ferromagnetic and paramagnetic phases. The paramagnetic solution p0 = 1 is always a solution, for C > K a ferromagnetic solution with p+ > p− > 0 emerges. +p
5. Symmetric Channels Averages at Nishimori’s Temperature Here we establish the identity J J = J tanh(β N J ) J for symmetric channels. It was shown in Sourlas (1994a) that: % $ p(J | 1) 1 , (A.31) β N J = ln 2 p(J | −1) where β N is the Nishimori temperature and p(J | J 0 ) are the probabilities that a transmitted bit J 0 is received as J. From this we can easily find: tanh (β N J ) =
p(J | 1) − p(J | −1) . p(J | 1) + p(J | −1)
(A.32)
p(J | 1) − p(−J | 1) . p(J | 1) + p(−J | 1)
(A.33)
In a symmetric channel ( p(J | −J 0 ) = p(−J | J 0 )), it is also represented as tanh (β N J ) = Therefore,
J tanh (β N J ) J = Tr J p(J | 1)
J p(J | 1) p(J | 1) + p(−J | 1)
+ Tr J p(J | 1) = Tr J p(J | 1)
(−J ) p(−J | 1) p(J | 1) + p(−J | 1)
J p(J | 1) p(J | 1) + p(−J | 1)
+ Tr J p(−J | 1) = Tr J J p(J | 1) = J J .
J p(J | 1) p(−J | 1) + p(J | 1) (A.34)
334
VICENTE ET AL.
6. Probability Propagation Equations In this section we derive the probability propagation Eqs. (36) and (34) in the form (96). We start by introducing the following representation for the Sk Sk variables Q μk and Rμk : 1 (1 + m μk Sk ) 2
Sk Q μk =
Sk Rμk =
1 (1 + mˆ μk Sk ). 2
(A.35)
We can now put (91), (95), and (A.35) together to write: Sk Rμj
1 = aμ ×
1 cosh(β Jμ ) 1 + tanh(β Jμ ) Sj 2 j∈Lμ {Sk :k∈L(μ)\ j}
k∈L(μ)\ j
1 1 = K 2 aμ
1 1 + m μk Sk 2
{Sk :k∈L(μ)\ j}
× 1+
k∈L(μ)\ j
cosh(β Jμ ) 1 + tanh(β Jμ )
m μk Sk +
k=l∈L(μ)\ j
Sj
j∈L(μ)
m μk m μl Sk Sl + · · ·
1 1 m μk = K cosh(β Jμ ) 1 + tanh(β Jμ )S j 2 aμ k∈L(μ)\ j 1 1 + tanh(β Jμ )S j m μk . = 2 k∈L(μ)\ j
(A.36)
To obtain the last line, we used that the normalization constant is aμ = 1 cosh(β Jμ ). Writing the above equation in terms of the new variable mˆ μk 2 K −1 we obtain the first Eq. (96): (+) (−) − Rμk mˆ μk = Rμk 1 1 1 + tanh(β Jμ ) 1 − tanh(β Jμ ) = m μk − m μk 2 2 k∈L(μ)\ j k∈L(μ)\ j = tanh(β Jμ ) m μk . (A.37) k∈L(μ)\ j
335
LOW-DENSITY PARITY-CHECK CODES
To obtain the second Eq. (96), we write: 1 1 Sk = aμk (1 + tanh(β N′ Sk )) (1 + mˆ νk Sk ). Q μk 2 2 ν∈M(k)\μ
(A.38)
In the new variables m μk : 1 = aμk K 2
m μk
8
(1 + tanh(β N′ ))
− (1 −
tanh(β N′ ))
(1 + mˆ νk )
ν∈M(k)\μ
ν∈M(k)\μ
(1 − mˆ νk )
9
(A.39)
By using the identity eσ x = cosh(x)(1 + σ tanh(x)) we can write: exp m μk =
D
ν∈M(k)\μ
atanh(m νk ) + β N′
E
3 −1 K aμk 2 cosh(β N′ ) ν∈M(k)\μ cosh(atanh(m νk )) E D exp − ν∈M(k)\μ atanh(m νk ) − β N′ − −1 K 3 aμk 2 cosh(β N′ ) ν∈M(k)\μ cosh(atanh(m νk ))
(A.40)
Computing the normalization aμj along the same lines gives:
−1 aμk =
exp
D
ν∈M(k)\μ
atanh(m νk ) + β N′
E
3 2 K cosh(β N′ ) ν∈M(k)\μ cosh(atanh(m νk )) D E exp − ν∈M(k)\μ atanh(m νk ) − β N′ 3 + K 2 cosh(β N′ ) ν∈M(k)\μ cosh(atanh(m νk ))
(A.41)
Inserting (A.41) into (A.40) gives:
m μk = tanh
ν∈M(k)\μ
atanh(m νk ) +
β N′
!
.
(A.42)
337
LOW-DENSITY PARITY-CHECK CODES
to write: n
Z A,ζ
6 7 n M 1 α τj exp Fζβ = N τ 1 ,...,τ n j=1 α=1 ζ ! C M dZj 1 × 2πi Z C+1 j {A} j=1 n cosh (βγ ) 1+ (Z i1 · · · Z i K ) × enβγ i 1···i K 9 n α α × 1 + τi1 · · · τi K tanh(βγ ) .
(B.4)
α=1
By following Appendix A.1 from Eq. (A.5), we can finally find Eq. (120). 2. Replica Symmetric Solution As in the code of Sourlas (Appendix A.2), the replicated partition function can be put into the form: " #2 1 C (B.5) exp M Extrπ,πˆ G1 − C G2 + G3 . Z n A,ζ = N K Introducing the replica symmetric ans¨atz (121) into the functions G1 , G2 , and G3 , we obtain: G1 (n) = T0 + T1 qαK + T2 qαK1 α2 + · · · α
=
) K coshn (βγ ) enγβ
+
j=1
α1 α2
d x j π(x j ) 1 +
K n! tanh(βγ ) xj (n − 1)! j=1 !
K n! tanh2 (βγ ) x 2j + · · · (n − 2)!2! j=1
coshn (βγ ) = enγβ 1 −→ n 2
γ →∞
) K j=1
) K j=1
d x j π(x j ) 1 + tanh(βγ )
d x j π(x j ) 1 +
K j=1
xj
!n
,
K j=1
xj
!n (B.6)
336
VICENTE ET AL.
Appendix B. Gallager Codes: Technical Details 1. Replica Theory The replica theory for Gallager codes is very similar to the theory obtained for Sourlas codes (see Appendix A). We start with Eq. (116): 7 6 n M α n τj exp Fζβ Z A,ζ = τ 1 ,...,τ n j=1
×
6
α=1
n
i 1···i K α=1
ζ
exp βγ Ai1···i K τiα1 · · · τiαK − 1
7
.
(B.1)
A
The average over constructions A is then introduced using Eq. (117): 7 6 M n 1 n α Z A,ζ = τj exp Fζβ N τ 1 ,...,τ n j=1 α=1 ζ ! C M dZj 1 i 1 = j,i 2 ,...,i K Ai 1 = j,...,i K Zj × 2πi Z C+1 j {A} j=1 ! n α α τi1 · · · τi K − 1 . exp βγ Ai1···i K × (B.2) i 1···i K
α=1
After observing that M j=1
Zj
i 1 = j,i 2 ,...,i K
Ai1 = j,...,i K
=
(Z i1 · · · Z i K )Ai1···i K ,
i 1···i K
we can compute the sum over Ai1···i K ∈ {0, 1}: 7 6 n M 1 Z n A,ζ = τ jα exp Fζβ N τ 1 ,...,τ n j=1 α=1 ζ ! C M dZj 1 × 2πi Z C+1 j j=1 8 9 n α Z i1 · · · Z i K α × exp βγ τi1 · · · τi K 1+ . (B.3) enβγ i 1···i K α=1 We can now use the identity exσ = cosh(x)(1 + σ tanh(x)), where σ = ±1,
LOW-DENSITY PARITY-CHECK CODES
339
where the hyperparameters γ ∗ , F ∗ are used in the Hamiltonian H and β ∗ is the temperature, while γ , F and β are the actual parameters of the encoding and corruption processes. The Nishimori condition is defined by setting the temperature and all hyperparameters of the Hamiltonian to the values in the encoding and corruption processes. If this is done, the expression for the energy can be rewritten: J ,τ H(γ , F)Pγβ ({Jμ } | τ )PFβ (τ ) . (B.12) U= J ,τ Pγβ ({Jμ } | τ )PFβ (ζ)
By plugging (106) for the likelihood Pγβ ({Jμ } | τ ) and for the prior PFβ (ζ); setting the hyperparameters to γ → ∞, β = 1 and F = atanh(1 − 2 p) and performing the summation over J first, we easily get: U = −F(1 − 2 p). (B.13) M Note that this expression is independent of the macroscopic state of the system. u = lim
M→∞
4. Recursion Relations We start by introducing the effective field xˆ ν j : tanh(β xˆ ν j ) =
Pν j (+)e−β F − Pν j (−)e+β F . Pν j (+)e−β F + Pν j (−)e+β F
(B.14)
Equation (129) can be easily obtained from the equation above. Equation (130) is then obtained by introducing Eq. (128) into Eq. (129), and performing a straightforward manipulation, we obtain Eq. (131): 3′′ 3′ 3′′ Tr{τ j } eβγ (−Jμ j τ j −1) ν j eβ Fτ j +β xˆ ν j (τ j −1) , (B.15) exp(−2β xˆ μk ) = 3′′ 3′ 3′′ Tr{τ j } eβγ (+Jμ τk j τ j −1) ν j eβ Fτ j +β xˆ ν j (τ j −1)
where
Pν j (τ j )e−β Fτ j exp β xˆ ν j (τ j − 1) = Pν j (+)e−β F 3′′ 3′ and the products ν and j are over ν ∈ M( j) \ μ and j ∈ L(μ) \ k, respectively. The above equation can be rewritten as: 1E D0 ′ 3′′ 3′′ Tr{τ j } j e(β F+ ν xˆ ν j )τ j 1 − Jμ j τ j tanh(βγ ) D0 1E . e−2β xˆ μk = (B.16) ′ 3′′ 3′′ Tr{τ j } j e(β F+ ν xˆ ν j )τ j 1 + Jμ j τ j tanh(βγ )
338
VICENTE ET AL.
where we use the Nishimori condition γ → ∞, β = 1 to obtain the last line. G2 (n) = 1 + qα qˆ α + qα1 α2 qˆ α1 α2 + · · · α
=
)
and 1 G3 (n) = lnTr{τ α } M
α1 α2
ˆ [1 + x x] ˆ n. d xd xˆ π (x)π( ˆ x) 6
exp Fβζ
n
τα
α=1
(B.7)
!7
ζ
n ! d Z exp Z m=0 α1···αm qˆ α1···αm τ α1 · · · τ αm × 2πi Z C+1 6 !7 n 1 α τ = lnTr{τ α } exp Fβζ M α=1 ζ ( ! 3n C α ˆ ˆ) d Z exp Z d xˆ πˆ (x) α=1 (1 + τ x × 2πi Z C+1 !n ) C C < = 1 qˆ C0 d xˆ l π( ˆ xˆ l ) (1 + τ xˆ l ) (B.8) e Fβζ τ ζ = ln M C! l=1 l=1 τ =±1 C
By using Eq. (115) we can write " # ∂ 55 C 1 G1 (n) − C G2 (n) + G3 (n) , f = − Extrπ,πˆ 5n=0 β ∂n K
(B.9)
what yields the free-energy (123).
3. Energy Density at the Nishimori Condition In general, the average internal energy is evaluated as: U = H(γ ∗ , F ∗ )β ∗ J ,ζ ζ Pγβ ({Jμ } | ζ)PFβ (ζ) = ˜ ˜ ˜ J J˜ ,ζ˜ Pγβ ({J μ } | ζ)PFβ (ζ) H(γ ∗ , F ∗ )Pγ ∗ β ∗ ({Jμ } | τ )PF ∗ β ∗ (τ ) × τ , τ˜ Pγ ∗ β ∗ ({Jμ } | τ˜ )PF ∗ β ∗ (τ˜ )
(B.10)
(B.11)
341
LOW-DENSITY PARITY-CHECK CODES
From Eqs. (C.2) and (C.3) above we can write: 1 − 2 pz1 (K ) =
K
K! (−1)l pl (1 − p) K −l (K − l)!l! l odd
= (1 − p − p) K = (1 − 2 p) K .
(C.4)
From which we find: pz1 (K ) =
1 1 − (1 − 2 p) K . 2 2
(C.5)
For MN codes, syndrome bits have the form: (C.6)
z μ = ξ j1 ⊕ · · · ⊕ ξ jK ⊕ ζl1 ⊕ · · · ⊕ ζl L ,
where signal bits ξ j are randomly drawn with probability P(ξ = 1) = pξ and noise bits ζ l are drawn with probability P(ζ = 1) = p. The probability pz0 (K , L) of z μ = 0 is therefore: pz0 (K , L) = pz0 (K ) pz0 (L) + pz1 (K ) pz1 (L)
= 1 − pz1 (K ) − pz1 (L) + 2 pz1 (K ) pz1 (L).
(C.7)
where pzx (K ) and pz0 (L) stand for probabilities involving the K signal bits and L noise bits, respectively. By plugging Eq. (C.5) into Eq. (C.7), we get: pz1 (K , L) = 1 − pz0 (K , L) =
1 1 − (1 − 2 pξ ) K (1 − 2 p) L . 2 2
(C.8)
2. Replica Theory For MN codes the replicated partition function has the following form: 7 6 N n α n exp Fs ξβ Sj Z A,ξ,ζ = S1 ,...,Sn τ 1 ,...,τ n j=1
× ×
M l=1
6
6
exp Fn ζβ
n jl α=1
α=1
n α=1
τlα
7
ξ
ζ
exp βγ A jl S αj1 · · · S αjK τlα1 · · · τlαL − 1
7
. A
(C.9)
340
VICENTE ET AL.
By introducing the Nishimori condition β = 1 and γ → ∞ and computing traces: 3 3 xμj τ − Jμ j∈L(μ)\k τ =±1 τ exμj τ j∈L(μ)\k τ =±1 e 3 exp(−2β xˆ μk ) = 3 xμj τ + J xμj τ μ j∈L(μ)\k τ =±1 e j∈L(μ)\k τ =±1 τ e 3 1 − Jμ j∈L(μ)\k tanh(xμj ) 3 , (B.17) = 1 + Jμ j∈L(μ)\k tanh(xμj ) where we have introduced
xμj = F +
xˆ ν j .
ν∈M( j)\μ
A brief manipulation of the equation above yields Eq. (131).
Appendix C. MN Codes: Technical Details 1. Distribution of Syndrome Bits In this section we evaluate probabilities pzx associated with syndrome bits in MN and Gallager codes. In the case of Gallager codes, a syndrome bit μ has the form z μ = ζl1 ⊕ · · · ⊕ ζl K ,
(C.1)
where ζ ∈ {0, 1} and ⊕ denotes mod 2 sums. Each bit ζ l is randomly drawn with probabilities P(ζ = 1) = p and P(ζ = 0) = 1 − p. The probability pz0 (K ) of z μ = 0 equates with the probability of having an even number of ζl = 1 in the summation, therefore: pz0 (K ) = = Consequently pz1 (K ) =
K
K! pl (1 − p) K −l (K − l)!l! l even K
(−1)l
l even
K
l odd
=−
K! pl (1 − p) K −l . (K − l)!l!
(C.2)
K! pl (1 − p) K −l (K − l)!l!
K
K! (−1)l pl (1 − p) K −l . (K − l)!l! l odd
(C.3)
343
LOW-DENSITY PARITY-CHECK CODES
We use the identity exσ = cosh(x)(1 + σ tanh(x)), where σ = ±1, to write: n
Z A,ξ,ζ =
N
S ,...,S τ 1 ,...,τ n j=1 1
×
n
M j=1
×
6
6
exp Fs ξβ
exp Fn ζβ
N
8
1+
n
PC (C j )
{C j ,Dl } j=1
1 N
×
C
M
S αj
α=1
τ jα
α=1
n
7
7
ξ
ζ
P D (Dl )
l=1
dZj 1 2πi Z Cj j +1
C
dYl 1 2πi YlDl +1
coshn (βγ ) (Z i1 · · · Z i K Yl1 · · · Yl L ) enβγ il 9 n × 1 + Siα1 · · · SiαK τlα1 · · · τlαL tanh(βγ ) . ×
(C.12)
α=1
The product in the replica index α yields: n n tanhm (βγ ) 1 + Siα1 · · · SiαK τlα1 · · · τlαL tanh(βγ ) = m=0
α=1
×
α1 ,...,αm
Siα11 · · ·
Siα1m · · ·
SiαK1
···
SiαKm τlα1 1 · · · τlα1 m τlαL1
· · · τlαLm
!
,
(C.13)
where α1 , . . . , αm = {α1 , . . . , αm : α1 < · · · < αm }. The product in the multi-indices il can be computed by observing that the following relation holds in the thermodynamic limit: mmax 1 + ψil = il
m=0 il1 ,...,ilm
with mmax ∼ (N K M L )/K !L!.
N →∞
ψil1 · · · ψilm −→ exp
il
!
ψil , (C.14)
342
VICENTE ET AL.
By introducing averages over constructions (117) as described in Appendix B.1 we find: 6 7 N n n α Z A,ξ,ζ = Sj exp Fs ξβ S1 ,...,Sn τ 1 ,...,τ n j=1
× ×
M j=1
6
α=1
exp Fn ζβ N
n
τ jα
α=1
PC (C j )
{C j ,Dl } j=1
M
7
ξ
ζ
P D (Dl )
l=1
! C N 1 dZj 1 j1 = j, j2 ,..., j K ,l A j1 = j,..., j K ,l Z × N {A} j=1 2πi Z Cj j +1 j ! C M dYl 1 j,l1 =l,l2 ,...,l L A j,l1 =l,...,l L Y × 2πi YlDl +1 l l=1 ! n α exp βγ A jl S j1 · · · S αjK τlα1 · · · τlαL − 1 . (C.10) × jl
α=1
Computing the sum over A we get: n
Z A,ξ,ζ =
N
S1 ,...,Sn τ 1 ,...,τ n j=1
× ×
M j=1
6
exp Fn ζβ N
exp Fs ξβ
{C j ,Dl } j=1
C
6
n
τ jα
α=1
PC (C j )
M
n α=1
7
7
ξ
ζ
P D (Dl )
l=1
C
S αj
1 N
dZj 1 dYl 1 C j +1 2πi Z j 2πi YlDl +1 8 Z i · · · Z i K Yl1 · · · Yl L × 1+ 1 enβγ il ×
×
n α=1
9 α exp βγ Si1 · · · SiαK τlα1 · · · τlαL .
(C.11)
345
LOW-DENSITY PARITY-CHECK CODES
The variables can be normalized as: qα1···αm "→ qα1···αm q0
rα1···αm "→ rα1···αm . r0
(C.21)
By plugging Eqs. (C.17), (C.18), the above transformation into (173) and by using Laplace’s method, we obtain: 8 n C n Z A,ξ,ζ = Extrq,r,q,ˆ Tm qαK1···αm rαL1···αm ˆ r exp N K m=1 α1···αm − NC
n
qα1 ···αm qˆ α1···αm
− ML
n
rα1 ···αm rˆα1···αm
× ×
m=1 α1···αm
m=1 α1···αm
N
PC (C j )
P D (Dl )
l=1 Dl
j=1 C j
N Cj! C
j=1
M
!
qˆ 0 j
Tr{S αj }
6
exp Fs βξ
n α=1
Sα
!7
ξ
! d Z j exp Z j m=0 α1···αm qˆ α1···αm S α1 · · · S αm × C +1 2πi Zj j 6 !7 M n Dl ! α τl × Tr{τlα } exp Fn βζ rˆ0Dl l=1 α=1 ζ n !9 C α1 dYl exp Yl m=0 α1···αm rˆα1···αm τ · · · τ αm × 2πi YlDl +1 C
n
(C.22) where Tm = e−nβγ coshn (βγ )tanhm (βγ ). We can rewrite the replicated partition function as: 8 !9 C Z n A,ξ,ζ = exp N Extrq,r,q,ˆ G1 − C G2 − L G3 + G4 + G5 ˆr K (C.23)
344
VICENTE ET AL.
We find Eq. (173) by putting Eqs. (C.14) and (C.13) into (C.12) and using the following identities to introduce auxiliary variables: ! ) N 1 αm α1 Z j Sj · · · Sj =1 dqα1···αm δ qα1···αm − N j=1 ! ) M 1 αm α1 Yl τl · · · τl drα1···αm δ rα1···αm − =1 (C.15) M l=1 3. Replica Symmetric Free-Energy We first compute the normalization N for a given: %) $ % ) $ dq0 d qˆ 0 dr0 d rˆ0 N = 2πi 2πi # " L K M N × exp T0 q0K r0L − N q0 qˆ 0 − Mr0rˆ0 K !L! M C N C dYl exp [Yl rˆ0 ] d Z j exp Z j qˆ 0 × C j +1 2πi Z j 2πi YlDl +1 l=1 j=1
(C.16)
By using Cauchy’s integrals to integrate in Z j and Yl and Laplace’s method, we get: " L K M N N = exp Extrq0 ,qˆ 0 ,r0 ,ˆr0 T0 q0K r0L − N q0 qˆ 0 − Mr0rˆ0 K !L! !9 C M N rˆ0L l qˆ 0 j + . (C.17) ln + ln Cj! Ll ! l=1 j=1 The extremization above yields the following equations: q0 qˆ 0 =
N 1 Cj = C N j=1
r0rˆ0 =
M 1 Ll = L M l=1
q0K r0L = C
(K − 1)!L! . N K −1 M L
(C.18)
(C.19) (C.20)
347
LOW-DENSITY PARITY-CHECK CODES
!7 6 n N Cj! 1 α Tr{S α } exp Fs βξ Sj ln PC (C j ) G4 (n) = C N j=1 qˆ 0 j Cj α=1 ξ ! n C αm α1 d Z j exp Z j m=0 α1···αm qˆ α1···αm S j · · · S j × C +1 2πi Zj j !7 6 N n 1 Cj! α = ln PC (C j ) Sj Tr{S αj } exp Fs βξ C N j=1 qˆ 0 j Cj α=1 ξ ! ( 3n C α ˆ ˆ ˆ x) d Z j exp Z j d xˆ π( α=1 (1 + S j x) × C j +1 2πi Zj !n ) Cj Cj < = Fs βξ S = ln e d xˆ l πˆ (xˆ l ) PC (C j ) (1 + S xˆ i ) ξ Cj
l=1
S=±1
i=1
(C.29)
In the same way:
!7 6 M n 1 Dl ! α G5 (n) = ln P D (Dl ) τl Tr{τ α } exp Fn βζ M l=1 rˆ0Dl Dl α=1 ζ ! n C αm α1 dYl exp Yl m=0 α1···αm rˆα1···αm τl · · · τl × 2πi YlDl +1 !7 6 M n 1 Dl ! = P D (Dl ) ln τlα Tr{Dlα } exp Fn βζ Dl M l=1 ˆ r Dl α=1 0 ζ ! ( 3n C ˆ yˆ ) α=1 (1 + τlα yˆ ) dYl exp Yl d yˆ φ( × 2πi YlDl +1 !n ) Dl Dl < = ˆ yˆ l ) (1 + τ yˆ i ) PC (Dl ) d yˆ l φ( e Fn βζ τ = ln ζ
Dl
l=1
τ =±1
i=1
(C.30) By using Eq. (166) we can write ! 5 C ∂ 55 1 f = − Extrπ,π,φ, G1 (n) − C G2 (n) − L G3 (n) + G4 (n) + G5 (n) , ˆ φˆ β ∂ n 5n=0 K
(C.31)
what yields free-energy (176).
346
VICENTE ET AL.
Introducing the replica symmetric ans¨atze: ) ) ˆ xˆ m qˆ α1 ···αm = d xˆ πˆ (x) qα1 ···αm = d x π (x) x m
(C.24)
and rα1 ···αm =
)
dy φ(y) y
m
rˆα1 ···αm =
)
ˆ yˆ ) yˆ m . d yˆ φ(
(C.25)
By introducing Nishimori’s condition γ → ∞ and β = 1, we can work each term on (C.23) out and find: qαK1 α2 rαL1 α2 + · · · qαK rαL + T2 G1 (n) = T0 + T1 α
=
coshn (βγ ) enγβ × 1+
) K
α1 α2
dx j
j=1
L
dyl φ(yl )
l=1
K L n! tanh(βγ ) xj yl (n − 1)! j=1 l=1
! K L n! tanh2 (βγ ) x 2j yl2 + · · · + (n − 2)!2! j=1 l=1 coshn (βγ ) = enγβ
) K
d x j π (x j )
j=1
L l=1
dyl φ(yl ) 1 + tanh(βγ )
K j=1
xj
L l=1
!n
yl
!n ) K L K L 1 d x j π (x j ) dyl φ(yl ) 1 + xj yl , (C.26) −→ n 2 j=1 l=1 j=1 l=1 G2 (n) = 1 + qα qˆ α + qα1 α2 qˆ α1 α2 + · · · γ →∞
α
=
)
α1 α2
ˆ [1 + x x] ˆ n. d xd xˆ π(x)πˆ (x)
(C.27)
Similarly,
G3 (n) = 1 + =
)
α
rα rˆα +
α1 α2
rα1 α2 rˆα1 α2 + · · ·
ˆ yˆ ) [1 + y yˆ ]n . dyd yˆ φ(y) φ(
(C.28)
LOW-DENSITY PARITY-CHECK CODES
349
Since the variance of a Poisson distribution is given by the square root of the mean in the thermodynamic limit: 9 8 M→∞ (C.40) P A jl = x −→ δ (x − M) . jl
The Poisson distribution for the construction variables C and L will imply that a fraction N e−C of the signal bits and Me−L of the noise bits will be decoupled from the system. These unchecked bits have to be estimate by randomly sampling the prior probability P(S j ), implying that the overlap ρ is upper bounded by: E 1 D ρ≤ N − N e−C + N e−C (1 − 2 pξ ) N ≤ 1 − e−C + e−C (1 − 2 pξ )
≤ 1 − 2 pξ e−C .
(C.41)
Therefore, a VB-like code has necessarily an error-floor that decays exponentially with the C chosen.
Acknowledgments Support by Grants-in-Aid, MEXT (13680400) and JSPS (YK), The Royal Society and EPSRC-GR/N00562 (DS) is acknowledged. We acknowledge the contribution of Tatsuto Murayama to this research effort.
References Aji, S., and McEliece, R. (2000). The generalized distributive law. IEEE Trans. Inf. Theory 46, 325–343. Amic, C. D. E., and Luck, J. (1995). Zero-temperature error-correcting code for a binary symmetric channel. J. Phys. A 28, 135–147. Berger, J. (1993). Statistical Decision Theory and Bayesian Analysis. New York: Springer-Verlag. Berrou, C., Glavieux, A., and Thitimajshima, P. (1993). Near Shannon limit error-correcting coding and decoding: Turbo codex, in Proc. IEEE Int. Conf. Commun. (ICC), Geneva, Switzerland. pp. 1064–1070. Bowman, D., and Levin, K. (1982). Spin-glass in the Bethe approximation: Insights and problems. Phys. Rev. B 25, 3438–3441. Castillo, E., Guti´errez, J., and Hadi, A. (1997). Expert Systems and Probabilistic Network Models. New York: Springer-Verlag. Cheng, J.-F. (1997). Iterative decoding. Ph.D. thesis, California Institute of Technology, Pasadena, CA.
348
VICENTE ET AL.
4. Viana–Bray Model: Poisson Constructions The Viana-Bray (VB) model is a multispin system with random couplings and strong dilution (Viana and Bray, 1985). We can introduce a VB version of our statistical mechanical formulation for MN codes. The Hamiltonian for a VB-like code is identical to Eq. (160): Hγgauge (S, τ ; ξ, ζ) = −γ A jl S j1 · · · S jK τl1 · · · τl L − 1 jl
−Fs
N j=1
ξ j S j − Fn
M
ζl τl .
(C.32)
l=1
The variables A jl are independently drawn from the distribution: $ P(A) = 1 −
L!K ! M L−1 N K
%
δ(A) +
L!K ! δ(A − 1). M L−1 N K
The above distribution will yield the following averages: 7 6 A jl = M 6
jl
6
A jl
A jl
7
j1··· j K l1 =l···l L
(C.34)
A
7
j1 = j··· j K l1···l L
(C.33)
A
A
=C
(C.35)
= L.
(C.36)
In the thermodynamic limit the above summations are random variabels with a Poisson distributions: 8 9 Mx A jl = x = e−M P (C.37) x! jl 8 9 x −C C (C.38) P A jl = x = e x! j1 = j··· j K l1···l L 8 9 x L A jl = x = e−L . (C.39) P x! j1··· j K l1 =l···l L
LOW-DENSITY PARITY-CHECK CODES
351
Khinchin, A. (1957). Mathematical Foundations of Information Theory. Dover Publications, Inc., New York, NY. Kabashima, Y., Murayama, T., and Saad, D. (2000). Typical performance of Gallager-type errorcorrecting codes. Phys. Rev. Lett. 84, 1355–1358. Kabashima, Y., Nakamura, K., and van Mourik, J. (June 2001). Statistical mechanics of typical set decoding. cond-mat/0106323. Kirkpatrick, S., and Sherrington, D. (1978). Infinite-ranged models of spin-glasses. Phys. Rev. B 17, 4384–4403. Kabashima, Y., and Saad, D. (1998). Belief propagation vs. TAP for decoding corrupted messages. Europhys. Lett. 44, 668–674. Kabashima, Y., and Saad, D. (1999). Statistical physics of error-correcting codes. Europhys. Lett. 45, 97–103. Kanter, I., and Saad, D. (1999). Error-correcting codes that nearly saturate Shannon’s bound. Phys. Rev. Lett. 83, 2660–2663. Kanter, I., and Saad, D. (2000a). Cascading parity-check error-correcting codes. Phys. Rev. E 61, 2137–2140. Kanter, I., and Saad, D. (2000b). Finite-size effects and error-free communication in Gaussian channels. J. Phys. A 33, 1675–1681. Kabashima, Y., Sazuka, N., Nakamura, K., and Saad, D. (2001). Tighter decoding reliability bound for gallager’s error-correcting codes. Phys. Rev. E 64, art. no. 046113. Lauritzen, S. (1996). Graphical Models. Oxford University Press, New York, NY. Luby, M., Mitzenmacher, M., Shokrollahi, A., and Spielman, D. (1998). Improved low-density parity-check codes using irregular graphs and belief propagation. Digital Systems Research Center, http://www.research.digital.com/SRC/, Tech. Report SRC 1998-009. MacKay, D. (1995). Free energy minimization algorithm for decoding and cryptanalysis. Electronics Letters 31, 446–447. MacKay, D. (1999). Good error-correcting codes based on very sparse matrices. IEEE Trans. on Info. Theory 45, 399–431. MacKay, D. (2000a). Information theory, inference and learning algorithms. Available at http://wol.ra.phy.cam.ac.uk/mackay/. MacKay, D. (2000b). Relationships between sparse graph codes. Proceedings of the 2000 Workshop on Information-Based Induction Sciences (IBIS2000). Izu, Japan, pp. 257–270. Murayama, T., Kabashima, Y., Saad, D., and Vicente, R. (2000). Statistical physics of regular low-density parity-check error-correcting codes. Phys. Rev. E 62, 1577–1591. MacKay, D., and Neal, R. (1995). Good codes based on very sparse matrices. Lecture Notes in Computer Science, Vol. 1025, Spriger, Berlin, pp. 100–111. Monasson, R. (1998a). Optimization problems and replica symmetry breaking in finite connectivity spin glasses. J. Phys. A 31, 513–529. Monasson, R. (1998b). Some remarks on hierarchical replica symmetry breaking in finiteconnectivity systems. Philos. Mag. B 77, 1515–1521. Montanari, A. (2000). Turbo codes: the phase transition. Eur. Phys. J. B 18, 121–136. Montanari, A. (2001). The glassy phase of gallager codes. Eur. Phys. J. B 23, 121–136. Mezard, M., and Parisi, G. (2001). The bethe lattice spin glass revisited. Eur. Phys. J. B 20, 217–233. Mezard, M., Parisi, G., and Virasoro, M. (1987). Spin Glass Theory and Beyond. World Scientific Publishing Co., Singapore. MacWilliams, F., and Sloane, N. (1977). The Theory of Error-Correcting Codes. North-Holland, Amsterdam. Montanari, A., and Sourlas, N. (2000). The statistical mechanics of turbo codes. Eur. Phys. J. B 18, 107–119. Nelson, M., and Gailly, J. (1995). The Data Compression Book. M & T Books, New York, NY.
350
VICENTE ET AL.
Chung, S.-Y. (2000). On the construction of some capacity-approaching coding schemes. Ph.D. thesis, Massachusetts Institute of Technology. Conway, J., and Sloane, N. (1998). Sphere Packings, Lattices and Groups. New York: SpringerVerlag. Cover, T., and Thomas, J. (1991). Elements of Information Theory. New York: John Wiley & Sons. Davey, M. (1998). Record-breaking correction using low-density parity-check codes. Hamilton prize essay, Gonville and Caius College, Cambridge. Davey, M. (December 1999). Error-correction using low-density parity-check codes. Ph.D. thesis, University of Cambridge. Derrida, B. (1981). Random-energy model: An exactly solvable model of disordered systems. Phys. Rev. B 24, 2613–2626. Derrida, B. (1981). Random Energy Model: An exactly solvable model of disordered systems. Phys. Rev. B 24, 238–251. Dorlas, T., and Wedagedera, J. (1999). Phase diagram of the random energy model with higher order ferromagnetic term and error correcting codes due to Sourlas. Phys. Rev. Lett. 83, 4441– 4444. Feller, W. (1950). An Introduction to Probability Theory and Its Applications. Vol. 1, John Wiley & Sons, New York, NY. Fradkin, E., Huberman, B., and Shenker, S. (1978). Gauge symmetries in random magnetic systems. Phys. Rev. B 18, 4879–4814. Franz, S., Leone, M., Ricci-Tersenghi, F., and Zecchina, R. (2001). Exact solutions for diluted spin glasses and optimization problems. Phys. Rev. Lett. 87, No. 127209. Frey, B., and MacKay, D. (1998). A revolution: Belief propagation in graphs with cycles. in Advances in Neural Information Processing Systems 10, edited by Jordan, M., Kearns, M., and Solla, S., Cambridge, MA: The MIT Press, pp. 479–485. Frey, B. (1998). Graphical Models for Machine Learning and Digital Communication. Cambridge, MA: MIT Press. Freeman, Y. W. W. (1999). Correctness of belief propagation in Gaussian graphical models of arbitrary topology. Tech. Report TR UCB–CSD-99-1046, UC Berkeley CS Department TR UCB–CSD-99-1046. Gallager, R. (1962). Low density parity check codes. IRE Trans. Inf. Theory. IT-8, 21–28. Gallager, R. (1963). Low-density parity-check codes. Research Monograph series. No. 21. Cambridge, MA: MIT Press. Gross, D., and Mezard, M. (1984). The simplest spin glass. Nucl. Phys. B 240, 431–452. Goldschmidt, Y. (1991). Spin glass on the finite-connectivity lattice: The replica solution without replicas. Phys. Rev. B 43, 8148–8152. Gradshteyn, I., and Ryzhik, I. (1994). Table of Integrals, Series and Products. London: Academic Press, London. Gujrati, P. (1995). Bethe or Bethe-like lattice calculations are more reliable than conventional mean-field calculations. Phys. Rev. Lett. 74, 809–812. Hamming, R. (1950). Error detecting and error correcting codes. Bell Sys. Tech. J. 26, 147–160. Hill, R. (1986). A First Course in Coding Theory. Oxford, Claredon Press. Huffman, D. (1952). A method for construction of minimum redundancy codes. Proc. IRE 40, 1098–1101. Iba, Y. (1999). The Nishimori line and Bayesian statistics. J. Phys. A 32, 3875–3888. Jensen, F. (1996). An Introduction to Bayesian Networks, London, UCL Press. Kschischang, F., and Frey, B. (1998). Iterative decoding of compound codes by probability propagation in graphical models. IEEE J. Selected Areas Commun. 2, 153–159. Kschischang, F., Frey, B., and Loeliger, H.-A. (2001). Factor graphs and the sum-product algorithm. IEEE Trans. Inf. Theory 47, 498–519.
352
VICENTE ET AL.
Nishimori, H. (1980). Exact results and critical properties of the Ising model with competing interactions. J. Phys. C 13, 4071–4076. Nishimori, H. (1993). Optimal decoding for error-correcting codes. J. Phys. Soc. Japan 62, 2973–2975. Nishimori, H. (2001). Statistical Physics of Spin Glasses and Information Processing. Oxford University Press, Oxford, UK. Nakamura, K., Kabashima, Y., and Saad, D. (2001). Statistical mechanics of low-density parity check error-correcting codes over galois fields. Europhys. Lett. 56, 610–616. Opper, M., and Saad, D. (2001). Advanced Mean Field Methods: Theory and Practice. MIT Press, Cambridge, MA. Parisi, G. (1980). The order parameter for spin glasses: a function on the interval 0-1, J. Phys. A 13, 1101–1112. Pearl, J. (1988). Probabilistic Reasoning in Intelligent Systems. Morgan Kaufmann Publishers, Inc., San Francisco, CA. Plefka, T. (1982). Convergence condition of the TAP equation for the infinite-ranged ising spin glass model. J. Phys. A 15, 1971–1978. Rieger, H., and Kirkpatrick, T. (1992). Disordered p-spin interaction models on Husimi trees. Phys. Rev. B 45, 9772–9777. Richardson, T., Shokrollahi, A., and Urbanke, R. (2001). Design of prvably good low-density parity check codes. IEEE Trans. Inf. Theory 47, 619–637. Richardson, T., and Urbanke, R. (2001). The capacity of low-density parity check codes under message-passing decoding. IEEE Trans. Inf. Theory 47, 599–618. Ruj´an, P. (1993). Finite temperature error-correcting codes. Phys. Rev. Lett. 70, 2968–2971. Saakian, D. (1998). Diluted generalized random energy model. JETP Lett. 67, 440–444. Shannon, C. (1948). Mathematical theory of communication. Bell Sys. Tech. J. 27, (pt. I) 379–423 (pt. II) 623–656. Saul, L., and Jordan, M. (1998). Exploiting tractable substructures in intractable, in Advances in Neural Information Processing Systems 10 edited by Touretzky, D., Mozer, M., and Hasselmo, M. E., Cambridge, MA: MIT Press, pp. 479–485. Sourlas, N. (1989). Spin-glass models as error-correcting codes. Nature 339, 693–695. Sourlas, N. (1994). Spin-glasses, error-correcting codes and finite-temperature decoding. Europhys. Lett. 25, 159–164. Sourlas, N. (1994). Statistical mechanics and error-correcting codes, in From Statistical Physics to Statistical Inference and Back edited by Grassberger, P., and Nadal, J.-P., NATO ASI Series, Vol. 428, Kluwer Academic Publishers. pp. 195–204. Tanaka, T. (2000). Information geometry of mean field approximation. Neural Computation 12, 1951–1968. Toulouse, G. (1977). Theory of the frustration effect in spin glasses: I. Commun. Phys. 2, 115–119. Viana, L., and Bray, A. (1985). Phase diagrams for dilute spin glasses. J. Phys. C 18, 3037–3051. van Mourik, J., Saad, D., and Kabashima, Y. (2001). Weight vs. magnetization enumerator for gallager codes, in Cryptography and Coding, 8-th IMA International Conference (Berlin, Germany) (Honary, B., ed.), Springer, pp. 148–157. Viterbi, A., and Omura, J. (1979). Principles of Digital Communication and Coding. McGrawHill Book Co., Singapore. Vicente, R., Saad, D., and Kabashima, Y. (1999). Finite-connectivity systems as error-correcting codes. Phys. Rev. E 60, 5352–5366. Vicente, R., Saad, D., and Kabashima, Y. (2000). Error-correcting code on a cactus: a solvable model. Europhys. Lett. 51, 698–704. Vicente, R., Saad, D., and Kabashima, Y. (2000). Statistical mechanics of irregular low-density parity-check codes. J. Phys. A 33, 6527–6542.
LOW-DENSITY PARITY-CHECK CODES
353
Weiss, Y. (1997). Belief propagation and revision in networks with loops. Tech. Report A.I. Memo 1616, MIT. Wiberg, N. (1996). Codes and decoding on general graphs. Ph.D. thesis, Dep. of Electrical Engineering, Link¨oping University. Wong, K., and Sherrington, D. (1987). Graph bipartitioning and spin glasses on a random network of fixed finite valence. J. Phys. A 20, L793–L799. Wong, K., and Sherrington, D. (1987). Graph bipartitioning and the Bethe spin-glass. J. Phys. A 20, L785–L791. Wong, K., and Sherrington, D. (1988). Intensively connected spin glasses: towards a replicasymmetry-breaking solution of the groud state. J. Phys. A 21, L459–L466.
This Page Intentionally Left Blank
ADVANCES IN IMAGING AND ELECTRON PHYSICS, VOL. 125
Computer-Aided Crystallographic Analysis in the TEM STEFAN ZAEFFERER Max Planck Institut for Iron Research, D-40237 D¨usseldorf, Germany
I. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . II. Electron Diffraction and Diffraction Contrast . . . . . . . . . . . . . A. Selected Area Diffraction Patterns and Diffracted Intensities . . . . . B. Convergent Beam Diffraction Patterns . . . . . . . . . . . . . . C. Kikuchi Diffraction Patterns . . . . . . . . . . . . . . . . . . D. Diffraction Contrast . . . . . . . . . . . . . . . . . . . . . . III. A Universal Procedure for Orientation Determination from Electron Diffraction Patterns . . . . . . . . . . . . . . . . . . . . . . . A. Calculation of the Reciprocal Lattice . . . . . . . . . . . . . . . B. Orientation Determination Step I: Pattern Acquisition . . . . . . . . C. Orientation Determination Step II: Determination of Diffraction Vectors D. Orientation Determination Step III: Indexing of Diffraction Vectors and Orientation Calculation . . . . . . . . . . . . . . . . . . . E. Orientation Determination Step IV: Display and Control of Results . . IV. Automation of Orientation Determination in TEM . . . . . . . . . . . A. Kikuchi Patterns . . . . . . . . . . . . . . . . . . . . . . . B. Spot Patterns . . . . . . . . . . . . . . . . . . . . . . . . . 1. Determination of Spot Positions and Intensities . . . . . . . . . 2. Intensity-Corrected Orientation Determination from Spot Patterns . 3. Application to Highly Deformed Metals . . . . . . . . . . . . 4. Limitations . . . . . . . . . . . . . . . . . . . . . . . . C. Dark Field Imaging by Conical Scanning . . . . . . . . . . . . . D. Local Texture Measurements from Debye–Scherrer Ring Patterns . . . V. Characterization of Grain Boundaries . . . . . . . . . . . . . . . . A. Misorientation Calculation . . . . . . . . . . . . . . . . . . . B. Determination of Twins . . . . . . . . . . . . . . . . . . . . C. Determination of Foil Thickness and Boundary Plane . . . . . . . . VI. Determination of Slip Systems . . . . . . . . . . . . . . . . . . . A. Determination of Burgers Vectors of Perfect Dislocations in Elastically Isotropic Materials . . . . . . . . . . . . . . . . . . . . . . B. Burgers Vector Determination in Other Cases . . . . . . . . . . . C. Determination of the Dislocation Line Direction . . . . . . . . . . VII. Phase and Lattice Parameter Determination . . . . . . . . . . . . . A. Fit of Lattice Parameters . . . . . . . . . . . . . . . . . . . . B. Phase Discrimination . . . . . . . . . . . . . . . . . . . . . VIII. Calibration . . . . . . . . . . . . . . . . . . . . . . . . . . . A. Determination of Diffraction Length and Accelerating Voltage . . . . B. Determination of the Magnetic Rotation . . . . . . . . . . . . . IX. Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . .
. . . . . .
356 358 361 365 366 368
. . . .
. . . .
369 370 371 372
. . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . .
373 375 377 377 380 380 382 384 387 388 390 391 392 393 395 397
. . . . . . . . . . .
. . . . . . . . . . .
398 400 401 403 404 407 408 409 410 411 413
355 Copyright 2002, Elsevier Science (USA). All rights reserved. ISSN 1076-5670/02 $35.00
356
STEFAN ZAEFFERER
I. Introduction Analytical transmission electron microscopy (TEM) is one of the most powerful techniques in material science. Not only does it allow the direct observation of the microstructure inside a material, but also, by analysis of electron diffraction patterns, the quantitative characterization of the microstructural elements in areas ranging in size from several 10 μm to a few nanometers. Quantitative characterization includes, for example, the crystal symmetry and structure of phases, the crystallographic orientation of grains, misorientation between grains, the character of dislocations (Burgers vectors and line direction), twin systems, types of interfaces, and so on. Furthermore, TEM allows the determination of the chemical composition of very small regions by energy dispersive x-ray analysis (EDX) or electron energy loss spectrometry (EELS). The usefulness of diffraction pattern analysis for the quantitative investigation of crystalline materials in the TEM is evident; however, the possibilities are often not fully exploited because of the difficulties related to crystallography and diffraction. New computer programs which are directly linked to the microscope now make these tasks much easier and allow the microscopist to concentrate on the material science of the samples rather than to deal with the indexing of diffraction patterns or with stereographic projections. It is the aim of this paper to present some of the possibilities given by such computer software. A program created by the author will serve as a thread through the examples given here. The work presented here deals only with classical analytical TEM where the image contrast of a crystalline sample develops due to diffraction processes in the sample. The direct observation of the crystal lattice by phase contrast (high resolution imaging) is not treated, although the software presented might provide some help for this kind of work as well. The experimental and computer work of the author has been inspired to a large extend by the books of Edington (1991). Images created by diffraction contrast can most easily be quantitatively interpreted if the crystal is illuminated under well-defined diffraction conditions. In the case of two-beam conditions where only the transmitted and one diffracted beam are strongly excited, the image contrast allows the characterization of lattice defects like dislocations, stacking faults, or grain boundaries. If many beams are equally strongly excited (Laue diffraction), which is the case when the primary beam is parallel to a principal zone axis, information about the crystal symmetry of a crystal can be obtained. To achieve these different conditions, the crystal has to be oriented appropriately by tilting the sample. To perform tilting effectively, it is best to know the crystals orientation with respect to a fixed microscope coordinate system and calculate from that position the required tilt angles. These tasks, the determination of crystal orientations, and the calculation of appropriate tilt angles can be done effectively
COMPUTER-AIDED CRYSTALLOGRAPHIC ANALYSIS IN TEM
357
by computer programs and should be the basis of computer-aided TEM. At best, such a program is directly linked to the microscope (via video camera and/or other interfaces) so that the work can be performed online. The determination of crystal orientations is carried out by the analysis of a diffraction pattern (spot or line (Kikuchi) pattern). Based on the crystal orientation and on a predetermined calibration parameter, a computer program is able to simulate the diffraction mode of the microscope for this crystal. The author’s software, for example, simulates the tilting of the observed crystal in a sample holder (for example, a double tilt holder) while diffraction pattern, stereographic projection, and other representations of the crystal in its current tilt position can be displayed on the computer screen. In this way, the microscope is “doubled”: while the diffraction image is simulated on the computer monitor, the user observes the real image of the crystal—under the same tilt conditions—in the microscope in order to check the contrast of dislocations, grain boundaries, etc., for a quantitative characterization. The TEM is switched to diffraction mode only for the determination of the first orientation, the control of the current diffraction conditions or the observation of special features in the diffraction patterns like satellite spots, streaks, and so on. In addition to the simulation of the diffraction mode, the author’s program facilitates the search for particular diffraction conditions (for example twobeam conditions for the determination of Burgers vectors of dislocations). Also direct measurements of elements in the real image can be performed (for example, the position of a grain boundary or a dislocation line) and related to the crystallography of the observed crystal. The computer programs presented in this work are almost exclusively based only on the kinematic theory of electron diffraction. This allows a high calculation speed, but it is well known that the kinematic theory is, in most cases, not appropriate to quantitatively describe TEM images. However, it will be shown throughout this paper that in many cases crystallographic investigations are not seriously limited by this simplification of the theory. In detail, this paper will cover the following topics. (i) A scheme for the determination of crystal orientations from single crystal diffraction patterns. Particularly, a universal algorithm for the orientation determination from spot and Kikuchi patterns of any crystal structure is presented. The algorithm works equally with diffraction patterns including one or more zone axis or with spot patterns from arbitrary orientations. (ii) Digital image processing algorithms for the automatic evaluation of spot and Kikuchi patterns. For spot patterns, positions and intensities are determined and the information used for indexing. This is of special interest for the orientation determination in highly deformed metals where no Kikuchi patterns are available.
358
STEFAN ZAEFFERER
(iii) Other techniques for orientation determination, not based on single diffraction patterns but on dark field images or Debye–Scherrer ring patterns. (iv) The simulation of the diffraction mode of the TEM and its use for analytical quantitative work at the TEM. (v) The characterization of grain boundaries. This includes the determination of the misorientation across the boundary for crystals of any crystal structure. Furthermore, the inclination of the boundary and the crystallographic boundary plane can be determined. As a prerequisite for this, a simple technique for the foil thickness determination using extinction lines is presented. (vi) The determination of dislocations characterized by their Burgers vector, crystallographic line direction, their slip plane and their type (screw, edge, mixed). Only simple geometric concepts are used. (vii) The distinction of phases and the fit of lattice parameters for notexactly-known phases by simple geometric models. (viii) Procedures for the calibration of microscope parameters, including diffraction length, acceleration voltage, and magnetic rotation of image and diffraction pattern. The possibilities given by computer-aided TEM are illustrated by examples from different research projects in materials science. A larger number of examples is taken from a work on the determination of deformation mechanisms in Ti alloys (Zaefferer, 2002) where glide and twinning mechanisms had to be determined. Other examples stem from an investigation of the recrystallization mechanisms in an Fe36%Ni alloy (Zaefferer et al., 2001). Here orientations had to be measured in an extremely highly deformed microstructure. Further examples are taken from investigations on the atomic stacking order in TiAl ordered material, from twin determination in Al2O3, from phase distinction in different asbestos fibers, and others. The work presented here has partly been already published in a much shorter form in Zaefferer (2000).
II. Electron Diffraction and Diffraction Contrast The following chapter will serve as a brief introduction of the physical concepts used in the computer procedures described in the following sections. The aim is to recall the relationship between crystal lattice, diffraction at this lattice, and the image arising from diffraction mainly by use of simple geometric models. It is by no means intended to give a physically complete description of the diffraction process. The reader is referred to the monograph of Hirsch et al. (1965) which still gives the most complete coverage of the theory and praxis of diffraction in the TEM. It is assumed that the reader is familiar with
COMPUTER-AIDED CRYSTALLOGRAPHIC ANALYSIS IN TEM
359
Figure 1. Illustration of the Bragg equation. Diffracted intensity appears only if the electron waves excited on the lattice planes hkl are exactly in phase.
the concept of the reciprocal lattice and its relationship to the crystal lattice and to the diffraction process, but a very short repetition will be given in the following. The reciprocal lattice of a crystal lattice is formed by vectors, hhkl, that are perpendicular to the crystal lattice planes (hkl) and whose lengths are equal to the reciprocal interplanar spacing of these planes |hhkl | = 1/dhkl . The geometry of electron diffraction is described most characteristically by the Bragg equation nλ = 2d sin θ
(1)
which describes the angle relative to a crystal lattice plane (the Bragg angle θ) under which a maximum of diffracted intensity is observed. The equation is illustrated graphically in Figure 1. An intensity maximum under the angle θ occurs if the scattering of the incoming electron wave, with the wave length λ, occurs in phase on all lattice planes (hkl) with the interplanar spacing d. The diffraction process can be easily described by the wave vector k which is a vector perpendicular to the wave front with a length 1/λ, that is, k is a vector of the reciprocal space. The difference vector between primary and diffracted wave vector k = k − k′ is called the diffraction vector, g. As visible in Figure 1, g is perpendicular to the diffracting planes, that is, it is parallel to the reciprocal vector hhkl of these planes. The length of the diffraction vector is |g| = 2 · sin θ · |k| = 2 sin θ · 1/λ. With Eq. (1), it is found |g| = 1/dhkl . Thus, the diffraction vector g is equal to the reciprocal lattice vector h. Electron diffraction in the TEM leads to three fundamentally different pattern forms which may be applied in different situations. These are spot patterns formed by selected area diffraction (SADP), convergent beam electron
TABLE 1 Some Properties of Different Diffraction Pattern Types with Special Regard to the Application for Orientation Determination Conventional SAD spot pattern (SAD)
Microbeam spot pattern (MBSP)
Spatial resolution Precision of orientation measurement
0.5 μm 5◦
10–50 nm 5◦ , ≤1◦ With spot intensity analysis
Orientation determination in highly deformed metals Ambiguity of orientation measurement Useful sample thickness
Not used due to low spatial resolution
Well suited
Often 180◦ ambiguity
Microscope requirements
SAD diffraction
Other applications besides orientation determination
Definition and control of diffraction conditions, bright and dark field observations
Vanishes in thick samples
Convergent beam diffraction pattern (CBED)
Kikuchi pattern (TKP)
10–100 nm Normally not used for orientation determination -“-
10–50 nm ≤0.1◦ Medium well suited
180◦ Ambiguity less frequent Vanishes in thick samples
-“-
180◦ Ambiguity rare
Best use in thin samples
Microbeam diffraction, clean vacuum, digital camera for spot intensity measurement Definition and control of diffraction conditions
Microbeam diffraction, clean vacuum
Vanishes in thin samples (unless voltage is reduced) Microbeam diffraction, clean vacuum
Determination of space group, sample thickness, Burgers vectors, lattice parameters (HOLZ-lines)
Performance of tilt experiments, determination of lattice parameters
COMPUTER-AIDED CRYSTALLOGRAPHIC ANALYSIS IN TEM
361
diffraction patterns (CBEDP) (also called Kossel–M¨ollenstedt patterns), and transmission Kikuchi patterns (TKP). CBEDP created with a small convergence angle (see Section II.B) will also be called microbeam spot diffraction (MBSD) patterns. The different pattern forms are described and discussed in the following. Table 1 gives an overview of some of their characteristic properties especially with respect to orientation measurements
A. Selected Area Diffraction Patterns and Diffracted Intensities The most widely known (and may be used?) form of diffraction patterns is the SADP. It is created by illuminating the sample with a parallel electron beam. An aperture is inserted into the intermediate image plane to select the area in the sample, which contributes to the pattern (minimum diameter about 0.5 μm). The (parallel) primary beam is diffracted according to the Bragg equation, creating focused spots in the diffraction pattern. The position of these spots can be easily determined with the help of the Ewald construction (Fig. 2a): since all diffracted beams must have the same wavelength as the primary beam (elastic scattering), the locus of all possible diffracted wave vectors k′ in reciprocal space is a sphere with the radius 1/λ, the Ewald sphere. According to Eq. (1), diffracted intensity appears only in those directions where the diffraction vector is equal to a reciprocal lattice vector, i.e., only in those directions k′ where the Ewald sphere touches a reciprocal lattice vector. It must now be kept in mind that the Bragg equation is only valid for an infinite stack of lattice planes while TEM samples are usually very thin. This leads to a relaxation of the Bragg equation, that is, diffracted intensity also appears when Eq. (1) is not exactly fulfilled. In the Ewald construction in Figure 2a, this fact is shown by enlarging the lattice points to lattice streaks. The smaller the extension of the diffracting crystal in a given direction (for example, the thinner the specimen), the longer the streaks in the reciprocal lattice in the same direction (for a thin foil, for example, perpendicular to the specimen surface). As a consequence, the Ewald sphere might now intersect a reciprocal lattice streak even if it does not pass exactly through a reciprocal lattice point and diffracted intensity will appear also for conditions where Eq. (1) is not exactly fulfilled. The distance between a reciprocal lattice point and the actual Ewald sphere intersection is described by the deviation vector s. The length of s, s, is called the excitation error. In a simple model (see below), the intensity I of a diffracted beam quickly decreases with increasing s as it is shown schematically in Figure 2a. Concluding Figure 2, an SAD pattern can be interpreted as the projection, from reciprocal into real space, of the intersections of the Ewald sphere with the reciprocal lattice. The spot intensities depend, among others (see below), on the value of s. As will be shown in Section IV.B, it is possible to determine
362
STEFAN ZAEFFERER
Figure 2. (a) The Ewald construction. The graphic shows reciprocal space (the reciprocal lattice and the Ewald sphere) and reciprocal space (the crystal lattice and the diffraction pattern) in one graph. This is, of course, in principle forbidden, since the dimensions of both spaces are different (1/Length and Length). The presentation has been chosen to make clear the relationship of directions in the crystal, in the reciprocal lattice, and in the diffraction pattern. A strongly and a weakly excited beam a displayed. For the strongly excited beam on the right side, the Ewald sphere cuts the center of the reciprocal lattice streak. The Bragg equation is satisfied and for the electron wave vectors is k′ − k = g1. k is the wave vector of the primary beam, k′ the one of the diffracted beam. For the weakly excited beam the reciprocal lattice streak is cut away from its centre. In this case, the Bragg equation is not exactly satisfied and the wave vector equation is k′ − k = g2 − s, where s is the excitation error. (b) Typical spot diffraction pattern from bcc iron close to the 110 zone axis. (c) Computer simulation of the pattern in (b), recalculated after automatic orientation determination.
COMPUTER-AIDED CRYSTALLOGRAPHIC ANALYSIS IN TEM
363
from the positions and intensities of the spots in the pattern the position of the reciprocal lattice vectors and thus the orientation of the crystal. The correct analysis of diffraction patterns and images created by diffraction contrast requires the knowledge of the dependence of the diffracted intensities on crystal orientation, crystallography of the crystal, and sample and beam conditions. A simple approach to this is given by the kinematic theory of electron diffraction∗ which assumes that the scattering process in the sample is weak. In the case that only one set of lattice planes is in diffracting position and thus only one diffracted beam and the transmitted beam appear in the diffraction pattern (two-beam conditions), the intensity of the diffracted beam is given by I ∼ [sin(πts)/(πs)]2 ,
(2a)
where t is the specimen thickness. The kinematic theory has only very limited validity (see below) because electron scattering is a particularly strong event. Therefore, in most cases the dynamic theory of electron diffraction† has to be applied. The dynamic theory is mathematically complicated, but an easy analytical expression for the diffracted intensity can be given for the two-beam case. For the case of a perfect crystal (i.e., without lattice defects) the two-beam approximation yields the following equation for the diffracted and transmitted intensities, Ig and It: F s 2 + 1/ξg2 (2b) Ig (s, t) = (π/ξg )2 · sin2 (πtseff )/(π t seff )2 with seff =
and
It = 1 − I g . The variable ξ g is called the extinction distance. In the exact Bragg case (s = 0) it represents the wavelength of the oscillation of the diffracted intensity with the depth in the specimen. It is calculated according to ξg = π Vc cos(θ)/λ |Fg |. ∗
(3)
The kinematic theory of electron diffraction assumes that scattering in the sample is weak. In this case, a primary electron suffers only one scattering event and the diffracted electron beam is not rediffracted into the primary beam. Due to weak scattering, the diffracted intensity is very small compared to the primary intensity. The assumption of weak scattering is only true for very thin samples. Usually TEM samples do not satisfy this condition. † The dynamic theory of electron diffraction gives up the wrong assumption of the kinematic theory that electron scattering is weak. Therefore, a diffracted beam may show diffracted intensities that are high compared to the primary beam intensity, and rediffraction of a diffracted beam into the primary beam may occur.
364
STEFAN ZAEFFERER
Vc is the volume of the crystal lattice unit cell, λ the electron wavelength, θ the Bragg angle for the diffraction vector g, and F the structure factor of g. The structure factor F represents the contribution of the crystal structure to the diffracted intensity. It is calculated from the positions of all n atoms in the unit cell and their atomic scattering amplitude according to Fg = n f n (sin(θ)/λ) exp{2πi[hxn + kyn + lz n ]}.
(4)
h, k, l are the coordinates of the diffraction vector g and x, y, z the coordinates of the atoms in the unit cell. The atomic scattering amplitude f, finally, represents the scattering strength of every single atom in the unit cell. For every atom, the value fel for electron scattering is related to the value of fx for x-ray scattering by the following formula: f el (sin(θ)/λ) = m 0 e2 /(2 h) (λ/ sin(θ)) (Z − f x (sin(θ)/λ)),
(5)
where m0 and e are the mass, respectively, the charge of the electron, h is Planck’s number, and Z the atomic number of the atom. The values for fx have been determined by quantum mechanical calculations and can be found tabulated, for example, in the International Tables for X-Ray Crystallography (1999) as a function of sin(θ)/λ(= 1/(2|g|)). They can also be approximated by polynomial functions using tabulated coefficients (see Eq. (6)). Equation (2b) for dynamic conditions converts into Eq. (2a) for kinematic conditions when scattering is weak. Mathematically this is expressed by 1/ξ g ≪ s. Physically this corresponds to one or more of the following cases: (i) large deviation s between diffraction vector and reciprocal lattice vector (large deviation from exact Bragg conditions) (ii) weak scattering power of the crystal structure, i.e., small structure factor F or large unit cell volume Vc (iii) weak scattering power of the primary electrons, i.e., high energy electrons (high accelerating voltage). Equation (2b) describes a rather complicated intensity distribution for the spots in an SADP with strongly oscillating intensities depending on the thickness of the sample and on the excitation error s as it is graphically shown in Figure 3a (see color insert). These strong intensity variations are, however, normally not observed in diffraction patterns. Rather, the spot intensities fall, for a given t, monotonically with increasing s. There are two reasons for this observation. First, it has to be kept in mind that Eq. (2b) is valid only for the two-beam case. With the appearance of further beams, the oscillations become less significant. Second, thin foils used for TEM are always slightly buckled
COMPUTER-AIDED CRYSTALLOGRAPHIC ANALYSIS IN TEM
365
and the area contributing to a diffraction pattern thus has a range of s values. The intensity in a spot therefore represents the integral over a certain s-range as it is displayed in Figure 3b (see color insert). It can be seen here that the integration over s smoothes out not only the strong oscillations over s appearing for larger t but also the oscillations over t. The above considerations on geometry and intensities of spot patterns show that SADPs bear some complications for precise orientation determination or crystallographic work, although SADPs are often considered to be simple to understand and to use. It has to be kept in mind first that an SADP is the projection of the intersection of a sphere with a regular lattice, which makes the geometry of such a pattern relatively complicated. Second, for the correct crystal orientation determination from a spot pattern, the deviation of the Ewald sphere to the “active” reciprocal lattice points must be taken into account. It can only be estimated from the intensities of the spots. Therefore, a precise crystal orientation determination requires the measurement of both the positions and the intensities of the spots in the pattern. Furthermore, an SADP always stems from a relatively large area (Ø > 0.5 μm) and represents therefore only an average over the orientations in this region. Last but not least, a spot pattern usually shows only a small part of the reciprocal space, which makes tilt experiments relatively complicated. Nevertheless, spot patterns are sometimes indispensable in TEM; for example, to select well-defined dark field conditions or when crystal orientations in highly deformed metals are to be determined where no other diffraction patterns are available. Computer-aided TEM helps to overcome to a large extent the problems related to spot patterns, as will be further discussed in Section IV.B.
B. Convergent Beam Diffraction Patterns Like an SADP, a convergent beam diffraction pattern (CBEDP) is formed by elastic, coherent scattering of the primary beam. However, in contrast to an SADP, the sample is illuminated with a convergent rather than a parallel primary beam. The convergence is achieved by focusing the beam onto the specimen. The angle of convergence can be controlled by the size of the probe on the sample and the size of the condenser aperture. The primary electrons enter the sample under a range of different directions given by the width of the illumination cone, and diffraction appears within this range of directions. The diffraction pattern, in turn, does not consist of sharp diffraction spots but of disks whose diameters are proportional to the convergence angle. A CBED pattern can be understood as an array of individual spot patterns, each one for another beam incidence direction, laid side by side within the borders of the convergence angle of the primary beam (quoted after Deininger et al., 1994).
366
STEFAN ZAEFFERER
The interior of these disks is, if their diameter is large enough, not of uniform brightness but shows numerous details given by the different beam incidence directions for every position in the disk (“rocking curve”). These details can be used to directly characterize the illuminated area of the sample without observing the image of the sample itself. It is, for example, possible to determine the sample thickness, the character of dislocations and stacking faults, the crystallographic symmetry, and lattice constants of the material. CBED is a very powerful technique and is described in detail in several monographs and textbooks on TEM as, for example, Tanaka et al. (1988, 1994), Mansfield (1984), or Williams and Carter (1996). It is not the aim of this text to describe details of the technique. Rather, the use of CBEDP for orientation determination, an aspect that is usually disregarded in the literature, will be discussed here in more detail. If the convergence of the primary beam is not too large (i.e., with a small condenser aperture), the spots in the diffraction pattern are still relatively small and show uniform intensities as in a typical SADP. Since the pattern is created by elastic scattering of the primary beam, as SADP, and for small convergence of the primary beam the Ewald sphere construction can still be applied for the interpretation of the geometry of the pattern (Fig. 4, see color insert). The spot intensities are in this case integral intensities over all incidence angles in the primary beam, i.e., over a range of the deviation parameter s. They, show, thus, a behavior similar to that of SADP, with the difference that in the latter the buckling of the foil leads to the integration of the intensities over s. CBED patterns created under a small convergence angle will be referred to in the following as microbeam spot diffraction (MBSD) patterns. As for SADP, the determination of a crystal orientation from an MBSD pattern can only be carried out with reasonable precision if the positions and the intensities of the spots are known and the practical procedure for this is the same as for SADP (see Section IV.B). The big advantage of MBSD compared to SAD for orientation determination is that an MBSD pattern is created by a finely focused beam which allows a much higher spatial resolution of about 10 nm, compared to 500 nm reachable with SADP. Compared to the orientation determination from Kikuchi patterns (see next section), MBSD patterns have the great advantage of being relatively insensitive to lattice defects and can thus be used to measure orientations in highly deformed metals. This point will be discussed in more details in Section IV.B.3.
C. Kikuchi Diffraction Patterns Kikuchi patterns appear in thicker samples or at low accelerating voltages. Under these conditions, incoherent scattering of primary electrons becomes
COMPUTER-AIDED CRYSTALLOGRAPHIC ANALYSIS IN TEM
367
a frequent event. The incoherently scattered electrons can be regarded as an electron source inside of the crystal emitting electrons in all directions, as shown schematically in Figure 5a (see color insert). In contrast to SADP and CBEDP, Kikuchi patterns are not formed by elastic scattering of the primary beam electrons but by elastic scattering of these first incoherently scattered electrons. Since the “new” primary electrons are traveling in all directions, the locus of all electrons that are elastically diffracted on one set of lattice planes (hkl) is a pair of cones around the normal of these lattice planes (i.e., around their reciprocal lattice vector) as is shown in Figure 5b (see color insert). The electron intensity in the cone further away from the primary beam is higher than in the background (diffraction from a direction of high electron density into a direction of low electron density), and the cone is called the excess cone. The excess electrons in this cone are missing in the background intensity in the other, symmetrically positioned cone which is called the deficient cone (diffraction from a direction of low electron density into a direction of high electron density). Since the Bragg angle in TEM is small, the two cones intersect the observation plane (for example, the fluorescent screen) as two almost straight parallel lines. The line further away from the primary beam, the excess line (formed by the excess cone), appears brighter than the background, the deficient line darker than the background. Together they enclose the Kikuchi band which often appears brighter than the background. The geometric interpretation of a Kikuchi pattern is very simple (Fig. 5b): the position of the center line of a Kikuchi band corresponds to the intersection of the elongated diffracting plane with the observation plane (gnomonic projection). Thus, the spatial direction of the related reciprocal lattice vector can be directly determined from the center line position. The width of the band is related to the exact Bragg angle because the deviation vector s is 0 for both of the enveloping Kikuchi lines. Compared to spot type patterns, TKP have several important advantages. First, the accuracy of orientation determination from Kikuchi lines is one or more orders of magnitude higher than for spot patterns, which is due to the fact that the Bragg condition is exactly fulfilled for Kikuchi lines. Second, the visible solid angle of the reciprocal space is very large in a TKP (more than 20◦ compared to about 5◦ for spot patterns), which is due to the large range of primary electron directions created by incoherent scattering. This normally allows observation of a sufficient number of zone axes in one pattern in order to determine the orientation of a crystal unequivocally. Also tilt experiments can be performed with great ease, and crystal symmetries may be observed directly. Third, for many applications of TKP, especially for orientation determination and tilt experiments, only the geometry of the pattern has to be taken into account while the intensity of the lines is not of great importance. This makes experiments and calculations easier and more straightforward.
368
STEFAN ZAEFFERER
A serious disadvantage of TKP is their sensitivity to lattice defects. With increasing density of defects, TKP blur out and can thus not be used for the orientation determination in highly disturbed lattices. This point is further discussed in Section IV.B.3. Another disadvantage of TKP for orientation measurements is the difficulty of automating the detection of line positions by a software algorithm. The automatic orientation determination, however, is an important demand of current TEM where many crystal orientations are to be determined in a reasonable time span. The available techniques and their difficulties are discussed shortly in Section IV.A.
D. Diffraction Contrast The aim of the following short section is to introduce some important terms and concepts which are used later in this paper. It is expected that the reader is familiar with the concepts of diffraction contrast, as they are described in all textbooks on TEM. In analytical TEM, the contrast in the image of a crystalline sample appears mainly due to electron diffraction. The intensity of diffraction at a given position depends, as described by Eq. (2b), on the orientation of the crystal, i.e., on the active diffraction vectors ghkl and their appropriate deviation vector s, on the thickness of the crystal, and on the elemental composition at the position of observation. Additionally, every lattice defect that disturbs the coherence of the scattering process, such as a dislocation, a stacking fault, or a grain boundary, leads to a change of the diffracted intensity. By inserting an aperture into the back focal plane of the objective lens, either the transmitted electrons or a fraction of the diffracted electrons are selected to form the image. Transmitted electrons form a bright field (BF) image, diffracted electrons a dark field (DF) image. The less the sample diffracts at a certain position, the brighter the appropriate area appears in the BF image. In a DF image, in contrast, all areas appear bright, which diffract into the direction chosen by the position of the diffraction aperture. In the particular case of two-beam conditions, the BF image and the DF image show exactly opposite contrast. If, particularly, s = 0, Eq. (2b) becomes very simple and can be used, for example, to determine the thickness of a sample by thickness fringes at grain boundaries or by convergent beam diffraction (c.f. Section V.C). Furthermore, the two-beam case is of great importance for the quantitative characterization of lattice defects such as dislocations. In this case, the fact is that particular lattice planes are not distorted by the elastic strain field of the dislocation. Hence, if diffraction on these lattice planes is excited under two-beam conditions, diffraction will appear as in a perfect lattice and the dislocation will be invisible. This invisibility criterion can be used to determine the Burgers vector of the dislocation. The
COMPUTER-AIDED CRYSTALLOGRAPHIC ANALYSIS IN TEM
369
exact procedure and the support a microscopist can get, in this case, from a computer program is further discussed in Section VI.A. The diffraction image of a sample is a projection of the features in the sample onto the observation plane. Due to this projection, information about the three-dimensional position of a feature (grain boundaries, dislocations, etc.) is lost. However, by tilting the sample, the features can be put into another spatial position which also changes their projection. From two images, taken under different tilt angles, the three-dimensional position of this feature can be recalculated and a computer program may facilitate this calculations. An important problem for these large-angle tilt experiments is to keep the observed feature in the field of view even for rotations about a noneucentric axis when the image moves strongly. The problem becomes worse due to the fact that the diffraction conditions change during tilting and the observed features temporarily may go out of contrast. Especially in these cases the “virtual microscope,” a computer program which simulates the diffraction mode of the TEM, provides valuable help. The appropriate tilt angles are determined on the computer monitor using, for example, the simulation of a Kikuchi pattern, while the microscope is used in image mode, and image drift during tilting can be easily corrected. The use of such a computer program for the crystallographic analysis of diffraction contrast is one subject in Sections V and VI.
III. A Universal Procedure for Orientation Determination from Electron Diffraction Patterns The basis of computer-aided TEM is the determination of a crystal orientation from a diffraction pattern. A flexible software algorithm for the interactive determination of orientations should be as general as possible and should have as few as possible case distinctions. Particularly, line and spot patterns should be treated, as far as possible, by the same algorithm and no distinctions should be made between cases where several zone axes appear in the pattern and those where only one is visible. The algorithm should furthermore be independent of the crystal structure of the observed crystal. The advantage of such an algorithm is not only shortness of the code but especially the robustness of the indexing procedure. An example where such an algorithm is of importance is the orientation determination from a mix of diffraction spots and Kikuchi lines as it may become necessary, for example, when the primary beam enters the crystal close to a prominent zone axis and not enough sharp Kikuchi lines are visible for an unambiguous orientation determination. In this case, the spots allows the approximate orientation determination while the additionally measured Kikuchi line(s) increase the accuracy of the orientation determination significantly.
370
STEFAN ZAEFFERER
The computer-aided determination of a crystal orientation from a diffraction pattern requires four steps (I–IV). First the pattern is acquired and Kikuchi line or spot coordinates are measured. Second, the diffraction vector lengths and the angles between them have to be calculated from the measured coordinates. The next step is the consistent indexing of these vectors and the calculation of the crystal orientation from the indexed vectors. Finally, the pattern is recalculated and displayed on the screen to allow the user to check for the correctness of the solution found. In step III, the indexing of the measured diffraction vectors is based on a comparison of the positions of the reciprocal lattice vectors for a given crystal structure with the positions of the measured diffraction vectors. For this end, the reciprocal lattice for the given crystal structure is calculated in advance and stored in a look-up table. Some considerations about the creation of this look-up table are discussed in the following chapter. Subsequently the four steps for the orientation determination are presented in detail. A. Calculation of the Reciprocal Lattice To allow quick indexing and consistent crystallographic work, the vectors of the reciprocal lattice are stored in a look-up table which is organized in families of symmetrically equivalent vectors according to the symmetry of the reciprocal lattice. In principal, the reciprocal lattice shows the point symmetry of the crystal structure (32 available symmetry groups), but it is often sufficient (and easier to program) to use the Laue symmetry which additionally contains a center of symmetry (only 11 symmetry groups). The use of the Laue symmetry is justified in those cases of electron diffraction where the diffracted intensities from planes hkl are the same as for the planes -h-k-l. Von Laue (1948) has shown that this is the case for SAD patterns but not for Kikuchi patterns because for the latter the electron source is situated inside of the crystal. In noncentrosymmetric crystal structures, the Kikuchi line intensities for planes hkl and -h-k-l may thus be different. Nevertheless, for the determination of orientations from Kikuchi patterns, the limitation to the Laue group symmetry is sufficient in most cases because only the pattern geometry and not the line intensities is required for this task. All members of a family of symmetrically equivalent reciprocal lattice vectors have the same structure factor F whose value has to be known for the calculation of the diffracted intensity of a reflection in the diffraction pattern. F is calculated according to Eq. (4), which requires the knowledge of all atoms in the unit cell with their positions and atomic scattering amplitudes. The atom positions in the unit cell are created from a basic unit of atoms (the basis) by applying the space group symmetry of the structure (which is different from the point group symmetry!). The calculation of the atomic scattering
COMPUTER-AIDED CRYSTALLOGRAPHIC ANALYSIS IN TEM
371
amplitudes according to Eq. (5) requires the knowledge of the scattering amplitudes for x-rays, fx for the different atomic species in the structure. They can either be determined directly from tabulated values or calculated from tabulated polynomial coefficients. The latter procedure has been adapted by Kraus and Nolze (1993) for a computer program calculating x-ray diffraction diagrams using a polynomial function developed by Cromer and Mann (1968) and Haidu (1972): 4 aq exp[−bq (sin(θ)/λ)2 ] + c. (6) f x (sin(θ )/λ) = n=1
The material constants aq, bq, and c are tabulated for every atomic species in the International Tables (1999) and have been transferred by Kraus and Nolze (1993) into a simple text file. The computer program created by the author uses Eqs. (5) and (6) to calculate the structure factor for the case of electron diffraction. The knowledge of structure factor and atomic scattering amplitude is indispensable for the analysis of spot intensities for orientation determination from spot patterns (see Section IV.B.2) and for a realistic simulation of patterns on the computer screen. It is also of importance for the reliable indexing of Kikuchi patterns of complicated crystal structures. In this case, the number of reciprocal lattice vectors available for the indexing procedure may be limited by the definition of a minimum value for the structure factor. This results in higher speed and higher reliability of the indexing procedure. B. Orientation Determination Step I: Pattern Acquisition For quick and direct access to the diffraction pattern, the microscope is best equipped with a video or CCD camera, mounted either on the bottom flange of the microscope column, below the microscope chamber, or at the wide angle port above it. In general, for the applications presented here, the camera should have a large dynamic in order to allow the acquisition of diffraction patterns which often show very high contrast. The sensitivity of the camera, on the other hand, is less important. Some of the examples presented in this paper have been prepared, for example, on a Jeol 2000 EX TEM with a Lhesa LH 4036-B video camera mounted to the bottom flange. This camera shows a medium sensitivity and a large dynamic range which makes it well suited for the required task. A disadvantage of such a system is that a rather small diffraction length (≤100 mm) has to be used to record a sufficiently large part of the diffraction pattern on the small camera chip. New high-resolution CCD cameras mounted to the wide-angle port are able to record the whole image visible on the viewing screen of the microscope and the diffraction length or image magnification does not have to be particularly small.
372
STEFAN ZAEFFERER
The camera image is transferred to the computer and then displayed on the monitor. Spot or line coordinates in this image can be measured either manually by clicking with a mouse cursor on the selected positions or automatically by digital image processing (DIP). Automation of the detection is almost indispensable for a precise orientation determination from spot patterns, which requires the measurement of spot positions and intensities. The appropriate procedure will be described in Section IV.B.1. In contrast, the automated determination of line coordinates in Kikuchi patterns is necessary mainly to facilitate and speed up the measurement but does normally not improve the accuracy of the measurement. Also, a good working automated procedure may be applied to the determination of the Kikuchi pattern quality which gives important information about the degree of local lattice perfection. Appropriate DIPs will be discussed further in Section IV.A. If no video camera is available, a second online acquisition method developed by Weiland and Schwarzer (1985) might be of interest. In their technique, the diffraction pattern is shifted on the microscope screen by varying the voltages in the x-y beam deflection coils of the TEM. Different positions in the image (for example, the spots in a spot pattern) are successively brought into superposition with a fixed marker on the screen and the related coil voltages are acquired with an AD converter board in the computer. The software converts these data into coordinates in the observed image. Compared to the use of a camera, this method is cheap and simple but does not permit the determination of intensities. The system described has been installed on a Philips EM 430 TEM (Weiland and Schwarzer, 1985).
C. Orientation Determination Step II: Determination of Diffraction Vectors The aim of step II is to determine from the measured spot or line coordinates the related diffraction vectors in the microscope reference coordinate system and to calculate their lengths and the angles between them. The procedure is different for spot and Kikuchi patterns. For Kikuchi patterns, the calculation of diffraction vectors is straightforward. It takes into account that the center line of a Kikuchi band is the gnomonic projection of the diffracting lattice plane (Fig. 5a). In spot patterns, the exact position of a diffraction vector can only be calculated if its deviation from the Ewald sphere, s, is known (see Fig. 2a). s is related to the intensity of a diffraction spot but it is impossible to determine its value from just the intensity, first because the sign of s cannot be determined (i.e., the Ewald sphere could pass above or below the reciprocal lattice point) and second because the exact form of the function I(s, t) (I: intensity, t: specimen thickness) is unknown. Thus, for the determination of an orientation, another way is adapted which is based on the determination
COMPUTER-AIDED CRYSTALLOGRAPHIC ANALYSIS IN TEM
373
of those spots, under all visible ones, which correspond to a minimum value of s. In the picture of the Ewald construction, these spots correspond to those positions where the Ewald sphere passes closest to the reciprocal lattice points. From their diffraction vectors, the orientation can than be accurately calculated. The procedure will be discussed in detail in Section IV.B.2. D. Orientation Determination Step III: Indexing of Diffraction Vectors and Orientation Calculation Indexing of diffraction vectors is based on the comparison of precalculated values for the vector lengths and the angles between them with the values determined from the pattern. To use a similar indexing procedure for all crystal structures, all calculations are carried out in a Cartesian coordinate system. The indexing look-up table therefore contains the reciprocal lattice vectors in Cartesian coordinates. The transformation from the crystal coordinate system (superscript “c”) to a Cartesian coordinate system (superscript “o”) is done using the following formulas. For quicker calculations the vectors are also normalized (superscript “n”): 5 o 5 o 5 5 (7) and gon go(hkl) = gc(hkl) A−1 (hkl) = g(hkl) / g(hkl)
with the diffraction vector length 5 o 5 5g 5 = 1/d(hkl) = go go 1/2 (hkl) (hkl) (hkl)
(8)
A denotes the crystal matrix (Schumann, 1979): ⎞ ⎛ a b cos γ c cos β ⎟ ⎜ A = ⎝0 b sin γ c (cos α − cos β cos γ ) / sin γ ⎠ 0 0 c[1 + 2 cos α cos β cos γ − (cos2 α + cos2 β + cos2 γ )]1/2/sin γ
(9)
The angles between the diffraction vectors are then simply calculated by . on α1,2 = arcos gon (10) (hkl)1 g(hkl)2
During indexing, the measured values are compared one after the other with the values from the look-up table until all angles and vector lengths coincide within certain tolerances. If no consistent indexing can be found within the given limits, the tolerances are increased repeatedly until they reach a predefined maximum. If still no indexing is possible, the algorithm may exclude a certain number of diffraction vectors from the indexing procedure and attempt indexing again with all possible combinations of diffraction vectors. The latter case may arise, for example, in the case of pattern overlapping
374
STEFAN ZAEFFERER
where spots or Kikuchi lines are considered which do not all belong to the same crystal orientation. The same difficulty may appear in the case of spot patterns where some diffraction vectors show a too-large distance s from the Ewald sphere. From the consistently indexed diffraction vectors, the orientation of the crystal is calculated. For this end, an algorithm is used which permits the orientation calculation even if all measured diffraction vectors belong to the same zone axis. This is sometimes the case for spot patterns but may happen as well with Kikuchi patterns. Conventional algorithms either deal only with vectors from one zone axis (normally used in the case of spot pattern indexing assuming a flat Ewald sphere—see for example Rhoades (1976)), or require three vectors from two different zone axes (used in Kikuchi pattern analysis, where it is required to measure three lines which do not all pass through the same intersection—see, for example, Schwarzer (1989) or Schwarzer and Weiland (1984)). These algorithms are not sufficiently flexible to deal with all possible cases appearing in diffraction patterns from arbitrary crystal orientations. For the algorithm proposed here, the following equation system is solved for all pairs of indexed diffraction vectors g in order to determine the crystal direction parallel to the beam direction and one direction perpendicular to it. Usually the direction parallel to the x-axis of the diffraction pattern is chosen. g(hkl),1 r[uvw] = d(hkl),1 . cos <) g(hkl)1 , r[uvw] g(hkl),2 r[uvw] = d(hkl),2 . cos <) g(hkl)2 , r[uvw] 5 5 5r[uvw] 5 = 1 (11)
All vectors are given in Cartesian, normalized coordinates. r denominates the direction sought, either the primary beam direction or the second, perpendicular reference direction. The angles between the indexed diffraction vector g and either of the two reference directions r are known from measurement. Equation (11) has two possible solutions for r for each pair of vectors. By comparison of all solutions for r, the correct direction is chosen. In cases where only reflections from one zone axis have been measured, both directions r are possible solutions, which means that there is an ambiguity in the orientation calculation. The final direction is calculated as the average over all solutions. This averaging is a very important step, especially in the case of spot patterns. In the picture of the Ewald construction, this corresponds to positioning the reciprocal lattice so that all indexed reciprocal lattice vectors have a minimum distance to the Ewald sphere. The orientation determined by Eq. (11) is given with respect to a coordinate system fixed to the viewing screen or photographic plate. In most cases, however, the orientation with respect to the goniometer axis direction is required.
COMPUTER-AIDED CRYSTALLOGRAPHIC ANALYSIS IN TEM
375
Therefore, the calculated orientation is rotated around the beam direction to compensate for the magnetic rotation of the diffraction pattern between sample and viewing screen. At least in older microscopes, the magnetic rotation changes as a function of diffraction length and accelerating voltage, and the appropriate values have to be taken into account. An alternative indexing algorithm has been proposed by Adams et al. (1993) and successfully used for the orientation determination from electron backscatter diffraction (EBSD) patterns in the scanning electron microscope. The algorithm is based on the combination of the measured diffraction vectors into all possible groups of three vectors. Each triple is indexed separately and one orientation is calculated for each. As final orientation, that one is chosen that has been calculated most frequently (which has the largest number of “votes”). Although this algorithm works reliably in the case of EBSD patterns, it is not really useful for TEM diffraction patterns, especially for spot patterns. One reason is that the choice of possible diffraction vectors is much higher in TEM diffraction patterns than in EBSD patterns, another reason is the unknown deviation vector s in spot patterns which leads to too many possible solutions for the independent indexing of the vector triples.
E. Orientation Determination Step IV: Display and Control of Results The last step in the interactive orientation determination procedure is the display of the result and the choice of the correct solution by the user. It often happens that several solutions are found, especially if the indexing tolerances are large or wrong reflexes are included in the measurement. Therefore, a measure determining the most probable solution is necessary. Three different values can be used. The first is the indexing confidence index, given by the ratio of the number of consistently indexed diffraction vectors over all measured vectors. A second is the average angular difference between measured and recalculated diffraction vectors. In the case of spot patterns, finally, the average difference of the length of measured and recalculated diffraction vectors can be used. A further selection algorithm based on the number of votes for a particular orientation has already been mentioned in the previous chapter. It is of use mainly in the case of EBSD patterns. The diffraction pattern for the most probable solution is finally displayed on the screen for comparison with the real pattern. The user has to decide whether the proposed solution is correct. If the orientation is calculated from reflections of only one zone axis, there is an ambiguity of 180◦ in the orientation. In this case, the program will propose both possible orientations one after the other. The decision as to which of these solutions is the correct one can be made in two ways. First, if lines or spots which do not belong to the same zone axis
376
STEFAN ZAEFFERER
appear in the outer part of the pattern (spots or lines from first or higher order Laue zones FOLZ or HOLZ), a comparison of the recalculated and the original pattern may reveal the correct solution. Second, a sample rotation of a couple of degrees may be simulated on the computer and the appropriate diffraction pattern displayed on the screen. The sample is then tilted in the same way, and it is checked for which of the two possible solutions the real and the recalculated pattern of the tilted sample coincide. The visual comparison of measured and recalculated pattern requires a realistic simulation of the intensities of spots and lines. It will be discussed shortly in the following whether the kinematic theory is sufficient or the dynamic theory has to be used for this task. Dynamic conditions are clearly active when Kikuchi lines are visible, hence the intensity of Kikuchi lines can only be correctly simulated using the dynamic theory. The geometry of a Kikuchi pattern, however, is not affected by the choice of the theory, and a sufficiently realistic kinematical simulation can already be achieved if only those lines are drawn on the screen, which possess a kinematic intensity (equal to the square of the structure factor) larger than a given minimum. As an example, Figure 6 (see color insert) shows an application to the determination of the stacking order of Ti and Al atoms in neighboring lamella in the ordered γ phase of the intermetallic alloy TiAl. The positions of super lattice reflections (double lines in Fig. 6b) lead to the easy recognition of the correct orientation. Spot patterns are usually also taken from regions which are too thick for the kinematic theory to be valid and, in principle, the dynamic theory must also be used in this case to correctly simulate the spot intensities in dependence of foil thickness t and deviation parameters s. In a real spot pattern, however, a behavior similar to that expected from the kinematic theory is observed, that is, the spot intensities decrease monotonically with s for any crystal thickness and their maximum intensity for s = 0 is the higher, the larger is the structure factor for the respective reciprocal lattice vector. The reasons for this (foil buckling, beam convergence, multibeam case) have been discussed in Section II.A. The author’s program therefore only uses kinematic intensity calculations, and this assumption has turned out to give astonishingly realistic pattern simulations in many cases. For a quick and sufficiently realistic simulation, the [sin(π ts)/s]2 dependence of the spot intensities (Eq. (2a)) is approximated by a simple triangular function. The foil thickness t (respectively, the reciprocal lattice streak length ∼1/t) is variable in order to adapt the appearance of the recalculated pattern to the current foil thickness. As an example, Figure 2b shows a spot pattern from α-iron and its simulation. The possibility of superimposing Kikuchi and spot patterns in any intensity ratio gives an even more realistic simulation and facilitates the recognition of the correct orientation solution.
COMPUTER-AIDED CRYSTALLOGRAPHIC ANALYSIS IN TEM
377
The use of the kinematic theory in the author’s program first of all allows a high speed of calculation which is necessary to give quick access to the diffraction pattern, for example, for the simulation of tilt experiments. Dynamic intensity calculations for the usual multibeam case would be much too slow for this task. A second reason for the use of the simple kinematic theory is that usually the exact foil thickness at the point of observation, which is indispensable for dynamic calculations, is unknown and its determination complicated and time consuming. The interactive determination of orientations is flexible and robust because the user decides which diffraction spots or lines are taken into account and checks whether the solution found is correct. The whole procedure for the determination of one orientation usually takes between 10 and 30 s, necessary to measure the appropriate coordinates by mouse clicks. For the conventional work at the TEM, where only few crystals are investigated, for example, in order to study lattice defects, this measurement time is of no matter. Nevertheless, for a larger number of orientation measurements, for example, for studies on the local texture, and in order to ease the work at the TEM, automatic procedures for the spot and line detection have been developed which significantly reduce the measurement time. Automatic algorithms are also indispensable when the intensities of spots in spot patterns are to be detected. Appropriate algorithms are discussed in the following sections. In cases where a large number of orientations are to be determined, for example, in order to create an orientation map of a local area, most time is spent not in the indexing or measurement procedure but during the manual selection of the measurement position. This time can be significantly reduced by techniques which allow the completely automatic measurement of such maps. They are discussed further in Sections IV.C and IV.D.
IV. Automation of Orientation Determination in TEM A. Kikuchi Patterns The automatic detection of Kikuchi lines by digital image processing (DIP) has been originally developed for the evaluation of backscatter Kikuchi patterns (BKP) (also called electron backscatter diffraction, EBSD) in the SEM. Nowadays it has become a widely used tool for automatic crystal orientation mapping (ACOM). The book of Schwartz et al. (2000) gives a good overview of current applications. For the DIP, a Hough transform (HT) (Hough, 1962) has been most successfully applied. The use of an HT for Kikuchi line detection was first introduced by Krieger-Lassen et al. (1992) (see also
378
STEFAN ZAEFFERER
Krieger-Lassen, 1994), and later extended to orientation imaging microscopy (OIM) by Adams et al. (1993). The HT is an image processing procedure, which is generally used for the detection of parameterizable shapes in an image. For the detection of lines, the normal parameterization of a line is used in the form ρ = x cos ω + y sin ω
with
−R > ρ > R
and 0 > ω > π,
(12)
where ρ is the distance of the line from the origin, R the radius of the image, and ω the angle of the normal of the line with the x-axis. Each line in an image can be represented unequivocally by a point (ρ, ω) in a Cartesian coordinate system ρ over ω, called the Hough space. In the HT algorithm, all intensities of pixels (x, y) representing a straight line in a digital image are added and the sum is assigned to the appropriate position (ρ, ω) in the Hough space. In this way, bright or dark lines in the original image correspond to bright or dark spots in the Hough space. By the application of an HT, the problem of detecting lines in the real image is thus simplified to the problem of the detection of points in the Hough space. In fact, since a BKP image does not show really sharp Kikuchi lines but rather bright bands (see Fig. 7b), the HT does not create sharp points in the Hough space but butterflyshaped spots. These have to be detected by a convolution of the Hough image with an appropriate mask. The HT algorithm has been highly optimized for the evaluation of BKP and currently more than 30 orientations per second can be determined by ACOM. Soon after the introduction of the HT for BKP evaluation, efforts were made to use the same algorithm for the evaluation of transmission Kikuchi patterns, TKP (Zaefferer and Schwarzer, 1994; Krieger-Lassen, 1995; Schwarzer and Sukkau, 1998). These investigations did not lead to reliably working algorithms but pointed out the difficulties that have to be faced when dealing with TKP. The basic problem is that the density of crystallographic information (the number of Bragg-diffracted electrons over all electrons) is much lower in a TKP than in a BKP, and the detection algorithms therefore must be more sensitive. Figure 7 shows a typical TKP and a BKP to point out the difficulties in detail. The first difference between both kinds of patterns is the strongly varying background in TKP. In the treatment of BKP, the background is removed by the subtraction of an “empty” image which has been created by the integration over a large number of different patterns. This is not possible with TKP because here the background may change drastically with the sample thickness. Instead, the background may be eliminated by the use of a high-pass filter which allows only the high frequency Kikuchi lines to pass through into the filtered image. The use of such a software-based filter increases, of course, the calculation time for the automatic pattern evaluation. A second problem is the variable appearance of TKP Kikuchi lines. Couples of sharp dark and bright lines may appear as
COMPUTER-AIDED CRYSTALLOGRAPHIC ANALYSIS IN TEM
379
Figure 7. Comparison of a transmission and a backscatter Kikuchi pattern (TKP and BKP). (a) TKP showing sharp Kikuchi line couples, a diffuse Kikuchi band, strong background intensity variation, and aligned diffraction spots. (b) BKP with constantly bright Kikuchi bands and a homogeneous background intensity.
well as couples of only dark lines or bright diffuse bands. In contrast, in a BKP only bright bands appear. In the HT, the peaks of a TKP therefore appear quite different from those of a BKP, and the usual peak detection algorithms cannot be applied. The author recently developed a procedure using the Hough backmapping technique, introduced by Krieger-Lassen (1998), which allows the reliable detection of most kinds of lines and bands. The technique has two advantages. First, bright and dark lines are mapped automatically into two separate parameter spaces where they both appear as bright spots, hence only maxima have to be detected in the Hough space. Second, the algorithm results in a very sparse parameter space where especially the weak dark lines can easily be detected. A further problem for the application of an HT for TKP is the appearance of aligned spots which may be misinterpreted by the HT algorithm as bright lines. The only way around this problem is the use of a robust indexing algorithm which is capable of sorting out lines which do not fit to the others. A last difficulty arises due to the often weak contrast of the dark line of a Kikuchi line pair. It is, however, especially in the case of wide Kikuchi bands, not sufficient for a reliable orientation determination, if only the bright line is detected. Here also, the only way around this is a robust indexing algorithm. It seems that all procedures developed up to now suffer from a lack of reliability for the large variety of Kikuchi patterns. Nevertheless, the use of the backmapping technique in addition to the Hough transform appears to be a promising way for the automatic TKP evaluation. In addition, with the
380
STEFAN ZAEFFERER
increasing calculation power of computers it will be possible to calculate highresolution Hough transforms in a reasonable time (quicker than the manual measurement). This will allow determination with higher accuracy and reliability the position of the sharp Kikuchi lines. One example where a high resolution HT is already successfully applied is the detection of HOLZ lines in CBED patterns for the determination of lattice strain (Kr¨amer and Mayer, 1999). Here, however, only clear lines with constant contrast (dark lines on a white background) are to be detected, which makes the procedure significantly easier.
B. Spot Patterns Although the accuracy of orientation determination from Kikuchi patterns is superior to that from spot patterns, there are situations when Kikuchi patterns are not available and spot patterns (SAD or MBSD) have to be used. This is, for example, the case when orientations are to be measured in highly deformed or in very thin samples. Additionally, the detection of spots is much simpler and more robust than the line detection, and automation can be achieved more easily. The only disadvantage of the orientation determination from spot patterns is the rather large inaccuracy of ± 5◦ if only the positions of spots are taken into account. It is therefore desirable to have a procedure which enhances the accuracy of the orientation determination from spot patterns. As has been mentioned, this can be achieved by taking into account the spot positions and intensities. In the following, a DIP algorithm for the detection of positions and intensities of spots, as it is used in the author’s program, will be described first. Second, the procedure for the intensity-corrected orientation determination will be presented in more detail. Finally, the most important application for this technique, the determination of orientations in highly deformed metals and the limitations of the technique, will be discussed. 1. Determination of Spot Positions and Intensities The detection of diffraction spots is much easier than the detection of Kikuchi lines. The DIP is carried out without any previous background subtraction procedure and usual image sizes in the order of 300 × 300 pixels can be treated in a split second. Larger images are usually binned to half the original size. The DIP procedure consists of three steps. First, the spot positions are searched; second, the exact spot maximum is determined; and third, the spot intensity for each spot is calculated by a numerical integration over the whole peak area followed by a subtraction of the local background. The different steps are illustrated in Figure 8.
COMPUTER-AIDED CRYSTALLOGRAPHIC ANALYSIS IN TEM
381
Figure 8. Illustration of the digital image processing procedure for spot patterns. (a) Diffraction pattern from a 94% cold rolled sample of Fe 36% Ni, recorded with a Lhesa TV LH 4036-B TV-rate camera on a Jeol 2000EX microscope at 200 kV. (b) Intensity distribution along the white line in (a).
The first step uses a gradient search procedure to find the footpoints of the spots. Therefore, the gradient (∂i(x, y)/∂ x)2 + (∂i(x, y)/∂ y)2 (where i(x, y) denotes the image intensity at position x, y) is checked at each point in the image. If the gradient is larger than a predefined minimum, a possible footpoint was found. In the second step, the maximum of the peak is sought starting from the footpoint. To this end, all values in a 5 × 5 pixel mask are monitored. The position of the mask is successively moved to the highest pixel in the mask until the assumed peak maximum is found. Next, the distance between the footpoint and the maximum, i.e., the peak width, is calculated. If the width is larger than a predefined value, it is supposed that no true spot has been found and the gradient search procedure is continued. A special procedure has to
382
STEFAN ZAEFFERER
be applied in the case of plateau maxima. Plateau maxima appear when the camera locally goes into saturation due to high intensities. The treatment of these maxima consists of the determination of all border points of the plateau. Subsequently, the most probable peak position is calculated as the center of gravity of these points. If a true spot is found, the intensity of this spot is determined by a summation over all pixel intensities inside a circle around the maximum position with a radius of the peak width. The local background intensity is determined as the average over all pixel values on the peak limiting circle and subtracted from the peak intensity. The procedure described not only determines the positions and intensities of spots in spot patterns but also those of spots which appear in Kikuchi patterns. As has been discussed above, the appearance of intense spots which are often arranged in straight lines complicates the application of a Hough transform for the automated line determination. The described DIP algorithm permits removal of the spots from the image and may in this way enable a more reliable line detection algorithm. 2. Intensity-Corrected Orientation Determination from Spot Patterns From the determined spot positions and intensities, the orientation can be calculated with much improved accuracy. The basic principle is to determine, among all appearing spots, those which are closest to the Ewald sphere (see also Section III.C). For these spots, the exact (or nearly exact) position of the diffraction vector in reciprocal space can be calculated from the spot position in the pattern and the known curvature of the Ewald sphere. The distance of a diffraction vector from the Ewald sphere is determined by the analysis of the related spot intensity. In the frame of the kinematic theory of electron diffraction (c.f. Section IV.B.4 concerning this choice) the intensity of a spot (hkl) is given by the product of the square of the structure factor, F, and a deviation factor, d, which is a function of the deviation shkl of the diffraction vector ghkl from the Ewald sphere. From (2a) and (3) we get: Ihkl = [λ|Fg |/(Vc cos(θ))]2 . d(s)
(12a)
d(s) = [sin(πts)/(πs)]2 .
(12b)
with
d(s) becomes maximum for s → 0 when the corresponding diffraction vector is closest to the Ewald sphere. It is the goal of the following to show how d(s) can be determined from the measured intensity I.
COMPUTER-AIDED CRYSTALLOGRAPHIC ANALYSIS IN TEM
383
The structure factor, F, is calculated according to Eq. (4) as the sum over all n atoms in the unit cell of the product of a geometric factor and the atomic scattering amplitude f n (sin(θ)/λ). In the case of pure elements and disordered alloys, on which the discussion will be focused in the following, the scattering amplitude fn is constant for all atom positions, and the structure factor Eq. (4) ′ can be split up further into a geometric structure factor Fhkl and the atomic scattering amplitude: ′ Fhkl = f (sin(θ)/λ) n exp[2πi(hxn + kyn + lz n )] = f (sin(θ)/λ) .Fhkl
(13)
The atomic scattering amplitude f for electrons as a function of sin(θ)/λ can be calculated by Eqs. (5) and (6). The value of sin(θ)/λ is determined for each diffraction spot from the distance R of the spot to the primary beam in the diffraction pattern because sin(θ)/λ = 1/2d ∼ R (1/d: length of the diffraction vector). Equation (12a) is now rearranged so that all known values (known either from the pattern measurement or from the microscope conditions) are grouped together. This value is called the corrected intensity I c: c ′2 = Ihkl / f 2 (sin(θ)/λ) . (Vc cos(θ)/λ)2 = Fhkl d(s) Ihkl
(14)
For simple crystal structures (bcc, fcc, hcp, primitive lattices), the geometric structure factor F ′ is either 0 or 1. If it is 0, the spot is invisible, if it is 1, the corrected intensity Ic is equal to d(s). A maximum of I c thus corresponds to a minimum distance of a diffraction vector to the Ewald sphere. The numerical implementation of the theory described is simple: In a first step, the spot intensities in the diffraction pattern (recorded by a video or CCD camera) are determined by digital image processing as has been described previously. Subsequently, for every determined spot, the corrected intensity is calculated from its measured intensity and its distance to the primary beam. Finally, the spots are ordered according to their corrected intensities I c. From these spots only those with the highest I c are taken, for which it is assumed that they are closest to the Ewald sphere. From the position of these spots in the pattern and the known curvature of the Ewald sphere, the exact position of the diffraction vector can be calculated. The subsequent orientation calculation (Section III) determines the best position of the Ewald sphere relative to all spots that have been taken into account. Figure 9 shows two microbeam spot patterns, recorded from 94% deformed Fe 36% Ni. The orientations determined are less than 1◦ misoriented from the true orientation as could be verified by comparison with the results obtained from the extremely faint Kikuchi patterns.
384
raw image
STEFAN ZAEFFERER
detected spots
indexed pattern
Figure 9. Illustration of the automatic intensity-corrected orientation determination from spot patterns taken under same conditions as Fig. 8. Upper part: Off-zone axis pattern (orientation (−0.681 0.733 0.009)0.657 0.605 0.449). Lower part: Several zone axes in one spot pattern (orientation (−0;664 0.532 −0.526)−0.339 −0.847 −0.352). In the recalculated patterns, spot indices are not shown for better visibility. The size of the circles indicates the diffraction spot intensities. The slight disagreement between intensities of the measured and recalculated patterns is due to the use of a triangular function instead of Eq. (2a). The magnetic rotation of the pattern with respect to the goniometer direction (GD) and transverse direction (TD) is indicated. The crystal sketches are drawn with respect to GD parallel to the horizontal direction and the inverse beam direction normal to the paper plane.
3. Application to Highly Deformed Metals As has been mentioned already in the introduction to Section IV, the use of spot patterns for orientation determination is indispensable in the case of highly deformed metals. The orientation measurement in these materials is, for example, of importance for studies on deformation mechanisms or mechanisms of nucleation during recrystallization. In the following chapter, the applicability of spot and Kikuchi diffraction patterns for these cases will be discussed and some applications of the technique presented.
COMPUTER-AIDED CRYSTALLOGRAPHIC ANALYSIS IN TEM
385
The use of Kikuchi patterns for the measurement of orientations of crystals with a high density of lattice defects is limited: since the Bragg condition for diffraction is fulfilled exactly, the position of a Kikuchi line is very sensitive to the position of the diffracting crystal planes, and already the existence of a single dislocation can be observed by a slight shift of the Kikuchi line. If the electron beam irradiates a volume containing dislocations, the intensity which is normally diffracted into one Kikuchi cone is now distributed among several slightly separated cones, as shown schematically in Figure 10b (see color insert). The corresponding Kikuchi lines thus become more diffuse and tend to disappear with increasing dislocation density. It is thus necessary to irradiate a more or less undistorted volume element in the sample, i.e., to “shoot” in between the dislocations in order to create a clear Kikuchi pattern. The higher the dislocation density, the more difficult it is to find a dislocation free crystal column. Depending on the arrangement of dislocations (in groups or separated), evaluable diffraction patterns can be obtained only up to about 80% of deformation, even with a very small electron probe. Materials with a high stacking fault energy (SFE) may allow higher deformations than materials with a low SFE since in the former dislocations are more easily grouped together in cells, thus creating dislocation free volume elements. If even higher deformed metals are to be investigated, microbeam spot patterns have to be used which are still visible at these deformations. The reason for this insensitivity to deformation is two-fold. First, if the lattice defects create a slight rotation of the crystal orientation about the beam direction, the spots in a diffraction pattern just move on circles around the primary beam, that is, they get widened but the center of their intensity continues to be at the position of the average orientation. The second reason is related to the relaxation of the Bragg equation for spot patterns and is illustrated in Figure 10a. Although the crystal orientation in the probed area may change due to lattice defects, the reciprocal lattice streaks are still cut by the Ewald sphere. Therefore, the spot positions are not changed but their intensities are averaged over the different positions of the reciprocal lattice with respect to the Ewald sphere. The important result is that the spot pattern is still visible and can be used for orientation determination, up to very high lattice defect densities. An example for the application of this technique is given in Figure 11 (see color insert; from Zaefferer, S. (2000). J. Appl. Cryst. 33, 10–25. Blackwell Science, Oxford, with permission) which is taken from a study on the recrystallization mechanisms of a heavily cold-rolled sample of Fe36% Ni (Zaefferer et al., 2001). After 94% of cold rolling and complete recrystallization, this alloy develops a sharp and almost pure cube texture (preferred crystal orientation (100)001) and the origin of this texture was studied. The fcc alloy Fe 36% Ni has a high stacking fault energy but recovers very little during cold deformation due to its high degree of alloying and its high recrystallization temperature. It therefore keeps a high dislocation density inside the deformation bands,
386
STEFAN ZAEFFERER
which makes orientation measurements in the highly deformed material very difficult. Figure 11a shows a TEM micrograph of the 94% deformed material. Two different regions can be distinguished. One region shows dislocation cells, the other region thin and elongated lamella and microbands. The orientations of all subgrains were measured using the microbeam Kikuchi and spot techniques. The results are shown in the form of orientation maps in Figure 11b. The dislocation cell structure shows cube-oriented grains, whereas the lamella and microbands show orientations typical for cold-rolled alloys of this type. Additionally, the quality of diffraction patterns, which gives some information about the local degree of recovery, is mapped into the microstructure in Figure 11c. Here, as measure of the pattern quality, only the number of visible Kikuchi lines was taken. This is a rather rough and subjective measure, but it turned out to give valuable information about the local defect density. In the lamella bands, only spot patterns could be obtained (i.e., the number of Kikuchi lines was 0), whereas in cube grains Kikuchi patterns appeared. In general, the quality of diffraction patterns was all the better (i.e., the degree of recovery all the higher), the closer a grain was to the cube orientation. Figure 11d shows the misorientation angles of cells situated along the white arrow in Figure 11b. It is visible that the cells in the cube-oriented area build up a strong orientation gradient, ranging from {100}001 to an orientation close to {210}001. In lamella bands, in contrast, only a weak orientation gradient developed. From a number of similar investigations the study could finally confirm an oriented nucleation mechanism for the cube texture formation in Fe 36% Ni and explain important details of the nucleation process itself. Another extreme example of the application of the spot pattern technique is given in Figure 12 (see color insert; from Zaefferer, S. et al. (2001). Acta. Mater. 49, 1105–1122, with permission from Elsevier Science) which shows an orientation map from a region with shear bands in a heavily cold-rolled Fe 36% Ni alloy. Shear bands are regions of concentrated shear deformation and show an extremely high density of dislocations. In fact, the white areas in the map correspond to regions in the center of the shear band where the local lattice rotation due to defects was so high that even from a region as small as 30 nm no spot patterns but only broken ring patterns were received. In this case, no orientation could be determined. A smaller beam size (which was not available with the microscope type used) would have allowed the measurement even in these areas. The measurements indicated that small areas inside of the shear band showed orientations completely different from the matrix, some of them, especially, were in twin orientation to the surrounding matrix. It is known that the nucleation of recrystallization in shear bands may lead to orientations different from the matrix orientation (Nes and Hutchinson, 1989); however, the exact orientations inside of shear bands could, until now, not be investigated in detail.
COMPUTER-AIDED CRYSTALLOGRAPHIC ANALYSIS IN TEM
387
4. Limitations The use of spot patterns for the orientation determination of crystals is only necessary in cases when Kikuchi patterns are not available, i.e., in very thin or in highly deformed samples. Spot patterns might also be preferred in the case of fully automated orientation mapping because the automatic procedure for the evaluation of spot patterns is much easier and more robust than that for Kikuchi patterns. In all other cases it is advisable to use Kikuchi patterns, especially because they allow a significantly higher precision of the orientation determination. For the use of spot patterns it must be carefully ascertained in which cases the use of the described procedure—which is based on the kinematic theory of electron diffraction—is justified and may give reliable results. In the case of very thin crystals (i.e., for thicknesses smaller than a quarter of the smallest extinction distance ξ g, min), the kinematic theory and thus the procedure for the intensity correction described previously applies directly. For thicker crystals, dynamic conditions are active, and the oscillation of the diffracted intensity dependent on crystal thickness, t, and excitation error, s, have to be taken into account. The intensity oscillations with the crystal thickness can be easily observed at grain boundaries (boundary fringes) or at edges (thickness fringes) if the crystal is observed under two-beam conditions. If the dynamic theory applies in this strict form, a maximum value of d(s) can no longer be related to the case s = 0, i.e., to a minimum of the distance between a reciprocal lattice vector and the Ewald sphere. This means the procedure described in the previous section cannot be applied in this case. There are, however, conditions when the procedure may be applied even for thicker samples: With increasing deformation of the crystal, the undisturbed, coherently scattering volume elements become smaller. If the diameter of the coherently scattering regions, i.e., the average distance between two crystal defects (dislocations, stacking faults), becomes small compared to ξ g, the oscillation effects disappear and the behavior of the intensity becomes similar to that given by the kinematic theory. This can be observed in highly deformed samples: no grain boundary fringes appear and the contrast is reduced due to the appearance of inelastic scattering. Additionally, as has been discussed in Section II.A, the buckling of the foil or the beam convergence also lead to a disappearance of the strong intensity oscillations (c.f. Fig. 3b). In this case again the intensity goes through a maximum for s → 0 for all values of t, although this maximum might be quite weak for certain values of t. It seems, therefore, also for the case of dynamical diffraction, in principle possible to estimate the position of the reciprocal lattice points from those spots with the highest corrected intensities. However, since the maximum might be weak, the maximum detection will be rather uncertain and the orientation determination less accurate than in the kinematic case.
388
STEFAN ZAEFFERER
It can be concluded that the procedure for the automated orientation determination from spot patterns reliably applies only in the case of very thin or very highly deformed samples. As a matter of fact, these are as well the only cases when the procedure has to be applied since Kikuchi patterns are not available! It should be finally pointed out that the procedure, as described above, only works for pure elements or disordered alloys with simple crystal structures because only in these cases the geometric structure factor of an observed diffraction spot is known before indexing (it is always 1 for visible spots). In other cases, a modified algorithm could be used. Here, in a first step, the orientation of the crystal would be roughly determined from only the spot positions. In a second step, the now already-known structure factors of the indexed spots are used to improve the orientation determination.
C. Dark Field Imaging by Conical Scanning The techniques described above can in principle be relatively easily extended to a mapping technique similar to the EBSD technique in the SEM. To this end, the beam is scanned over the sample and the orientation is automatically determined for every beam position. Depending on the sample conditions (thin or thick, low or high defect density), the Kikuchi or spot (MBSD) method has to be used. If samples with varying conditions have to be measured, only the spot pattern method may be applied. A problem with either of these techniques might be the stability of the microscope over the relatively long measurement times (TEM are usually optimized only for short-term stability): electronic drift might change the patterns so strongly that they cannot be evaluated any longer; sample drift might move the desired area out of the field of view. It is therefore highly desirable to have a technique that allows high speed measurement and high robustness for the measurement under different sample conditions. A probably suitable measurement technique has been recently developed by Wright and Dingley (1998), based on ideas of Schwarzer (1983) and Humphreys (1983) and is already commercially available (Dingley and Wright, 2000). The technique will be called in the following dark field imaging by conical scanning, DFCS. It is based on the consecutive recording of dark field images under different beam incidence angles. Figure 13 (see color insert) shows the principle: the primary beam incidence is scanned in steps around cones (angle β) with successively larger opening angles ω (Fig. 13a). The maximum beam tilt angle ω achievable with standard microscopes is in the order of about 4◦ . In every dark field image, characterized by the beam incidence angles (β, ω), those positions (x, y) light up which fulfill the Bragg conditions for a particular set of lattice planes (hkl) (Fig. 13b). From all intensities recorded
COMPUTER-AIDED CRYSTALLOGRAPHIC ANALYSIS IN TEM
389
for the different angles (β, ω) for one given position (x, y), a CBED pattern of this particular position can be reconstructed (Fig. 13c). The orientation determination from this spot pattern is subsequently carried out in the same way as has been described in the previous section and the limitations that have been discussed for CBED and MBSD patterns also apply to the method presented here. Since all pixels in one dark field image are recorded simultaneously, all diffraction patterns of all positions in the image can be reconstructed after one (β, ω)-scan. The data acquisition time for one complete scan with sufficiently high angular resolution is currently about 20 min. The postprocessing time (i.e., reconstruction and evaluation of spot patterns from the measured dark field images), is significantly longer, but, of course, does not need any TEM time. An example of first measurement result is given in Figure 13d, showing the reconstructed microstructure of highly deformed aluminum. Here, however, no complete orientations were determined. Rather the fact was used that all grains in this microstructure were close to one 110 zone axis and the color coding in Figure 13d just gives a qualitative measure of the proximity of a position to this zone axis calculated from the number of visible spots in the image. The spatial resolution of the technique is first of all determined by the chosen magnification and can in principle be as high as the resolution of the dark field image. In praxis the resolution is deteriorated by the misalignment of the microscope and by sample drift, so that the pixel positions in the recorded images do no longer correspond to exactly the same positions in the sample. Image cross-correlation may help to align the different images and correct to a certain extent for sample drift and misalignment. Besides image shift, the largest problem to deal with in the further development of the technique is the relatively small number of spots visible in the reconstructed images. This is due to the large intensity range that has to be spanned by the measurement. Inside of one image a large intensity range appears due to the fact that all grains in the imaged area, which might exhibit very different diffraction conditions, are recorded at the same time. This requires a large intensity dynamics of the used camera. An even larger intensity range is encountered with changing the angle ω from positions close to the primary beam to positions up to about 4◦ away from it (the intensity falls proportional to the square of the atomic scattering amplitude f (sin(θ)/λ)). This strong intensity variation can only be dealt with by raising the gain of the camera with increasing ω whereby its dynamic range must be kept high. If the intensity problems can be solved, the DFCS method will probably be the best method when extreme spatial resolution has to be achieved on highly deformed or thin samples. For the application to thick samples with a low defect density, the same restrictions have to be taken into account that have been discussed for spot patterns in the previous section.
390
STEFAN ZAEFFERER
D. Local Texture Measurements from Debye–Scherrer Ring Patterns Last but not least a technique will be described briefly that is not used for the measurement of single crystal orientations but for texture measurements, for the determination of the statistical orientation distribution of a small area of the sample. The technique has been developed independently by Schwarzer (1983) and Humphreys (1983) and later further improved by Schwarzer and co-workers (see below). A polycrystalline area of the sample is illuminated with a parallel electron beam in SAD mode. In this case, the diffraction pattern does not consist of discrete diffraction points from single grains but of Debye– Scherrer rings formed by the superposition of many single crystal patterns. Depending on the orientation distribution in the illuminated area, the diffracted intensity is more or less homogeneously distributed along the rings. By measuring the intensity distribution along the rings, the lattice plane (or pole) density distribution for all planes parallel to the primary beam can be determined. These data correspond to density values along one large circle (in the respective stereographic projection of the pole positions, the pole figure.) The sample is then successively tilted in small steps around the goniometer axis and the Debye–Scherrer ring intensities are measured. In this way incomplete pole figures can be constructed for all rings visible in the pattern. The larger the achievable sample tilt angle, the smaller the lens-shaped unmeasured area in the center of the pole figure (Schwarzer and Weiland, 1986). Finally, from several incomplete pole figures the orientation distribution function (ODF) can be calculated using, for example, the harmonic series expansion method of Bunge (1982). Schwarzer (1985) compiled the different principles for the measurement of the ring intensities that have been developed. A first possibility is to deflect the beam in front of the specimen (prespecimen deflection method). This method is very similar to that described in the previous section. Instead of measuring the dark field image intensity for every point, the dark field intensity is measured integrally over the whole image for every deflection position. A second possibility is to deflect the diffracted ring behind the specimen (postspecimen deflection method) in a way that the ring moves over a Faraday cage, positioned in the middle of the observation screen in order to measure the ring intensities (Schwarzer, 1983). The problem with this method is that, due to lens imperfections, the diffraction rings are not circles and the correct movement of the rings over the Faraday cage is difficult. This problem does not appear, and intensity acquisition becomes much simpler, if the diffraction pattern is recorded by a camera and the ring position detected by a computer program (Schwarzer et al., 1996). The last development in this field was the use of a special sample holder which allows production of diffraction patterns in gracing beam incidence (Sch¨afer and Schwarzer, 1998). Here the sample surface
COMPUTER-AIDED CRYSTALLOGRAPHIC ANALYSIS IN TEM
391
is positioned almost parallel to the beam and the half rings which are visible (the other half is shadowed by the sample) are recorded by a CCD camera. This method allows the spatially resolved texture determination of very thin surface layers on bulk samples or on thin foils. The texture measurement with electrons has interesting applications, especially those where the spatial resolution of other techniques (neither texture measurement with x-rays nor the single grain orientation determination with electrons) is not sufficient. A typical example is the measurement of orientation distributions in shear bands. Measurements on shear bands have already been presented in Section IV.B.3, and it was shown there that the spatial resolution of single grain orientation measurements was not sufficient to resolve the orientation of cells directly in the shear band. The Debye–Scherrer method would here allow measurement of the texture in this region. Schwarzer and Weiland (1986) present one measurement in a shear band but, unfortunately, the method has, up to now, not been applied seriously to problems in material science.
V. Characterization of Grain Boundaries Grain boundaries separate crystals of different orientations from each other and are the most obvious lattice defects in crystalline matter. Grain boundaries strongly influence almost all properties of a material, but especially the mechanical behavior. In metals, for example, they generally increase the yield stress for plastic deformation because they are obstacles for the movement of dislocations. In other cases, cracks may propagate along grain boundaries because the atomic binding forces are lower at grain boundaries (see, for example, Watanabe and Tsurekawa, 1999). Grain boundaries also seriously influence the electric, magnetic, chemical, and diffusion properties of a material. The extent to which a grain boundary influences the materials properties depends strongly on the characteristics of the boundary. To these characteristics belong the misorientation across a grain boundary and the crystallographic plane that forms the boundary in either of the crystals. In multiphase materials, grain boundaries may also separate crystals of different phases. The study of these phase boundaries may reveal important information about phase transformation mechanisms. Examples are the orientation relationships between the bcc α and fcc γ phase in a duplex steel or between the hcp α phase and the bcc β phase in titanium alloys. In this case, for example, the characteristics of a phase boundary allows explanation of details of the phase transformation during heating or cooling. Much literature is available on the characterization of grain boundaries. An introduction to experimental techniques and mathematical concepts is given
392
STEFAN ZAEFFERER
by Randle (1993); more detailed mathematical concepts are, for example, presented in Bollmann (1970). In the following, some straightforward and simple concepts for both the misorientation calculation and the boundary plane determination on the basis of TEM observations are introduced.
A. Misorientation Calculation Once the orientations of two neighboring crystals in a sample have been determined, the misorientation between them can be calculated disregarding their particular phase. The misorientation G between two crystals can be understood as the orientation of one of the crystals with respect to the other one (while the orientation of a crystal expresses its orientation with respect to a sample reference system). It is defined as follows: G = G1 · G−1 2 ,
(15a)
where G1 and G2 are the orientations of the two crystals. Due to the crystal symmetry, each crystal orientation can be represented in n different ways, where n is the number of point group or Laue group symmetry elements S: Gi = S i · G
(i = 1 . . . n)
(15b)
The misorientation between two crystals of the same or of different phases can thus be expressed in n · m ways, where n and m are the number of symmetry elements of each of the crystals. Most generally, the misorientation between two crystals is thus expressed by: Gij = (S1,i · G1 ) · (S2, j · G2 )−1
(i = 1 . . . n, j = 1 . . . m)
(15c)
Misorientations are usually not expressed as rotation matrices but in the form of a rotation axis u and a rotation angle ω around this axis. From the misorientation matrix, the rotation axis–rotation angle pair is calculated using the following equations: ω = cos((G x x + G yy + G zz − 1)/2) u i = [(G ii − cos(ω))/(1 − cos(ω))]1/2 ,
i = x, y, z
(16) (17)
It is a convention to choose under all possible symmetrically equivalent misorientations that one which has the smallest misorientation angle ω and whose rotation axis u is located in the standard orientation triangle of the first crystal. The value ω is then also called the orientation distance.
COMPUTER-AIDED CRYSTALLOGRAPHIC ANALYSIS IN TEM
393
The misorientation between two grains can also be classified by the coincidence site lattice (CSL) formed by the two crystals. A coincidence site lattice is formed only in special positions of the two lattices when certain lattice sites of both crystals coincide. The CSL is characterized by the value which is the ratio of the volume of the CSL unit cell and the volume of the crystal lattice unit cell. The value of is always an integer and odd. For more information about this the reader is referred to the books mentioned above.
B. Determination of Twins A special type of grain boundary is formed between crystals in twin orientation relationship. The matrix and the twin lattices have a high number of coincidence sites and show thus the smallest value = 3 (except small angle grain boundaries for which = 1). Twin boundaries have special properties, especially a very low boundary energy and a low mobility during grain growth processes. These properties have been utilized to make a material more stable against grain growth by increasing the number of growth twins in metals (Thomson and Randle, 1996). In some metals mechanical twinning is an important deformation mechanism and it is of great interest to know which kinds of twins are formed under which kinds of deformation and how they are influenced by the composition of the material (c.f. Philippe et al., 1988; Zaefferer, 2002). The examples above show that it is of importance to characterize twin boundaries precisely by electron microscopy. In general, two different types of twinning systems can be distinguished, rotation and reflection twins. Rotation twins can be described crystallographically by a 180◦ rotation of the matrix crystal around the shear direction of the twin followed by an inversion operation. Reflection twins are described by a mirror operation of the matrix crystal at the twinning plane. In centrosymmetric structures, the latter twin type can also be described by a 180◦ rotation around a direction perpendicular to the twinning plane. Therefore, a twin can always be recognized from the misorientation calculation because a rotation angle close to 180◦ will be found under all possible n2 solutions (n, number of symmetry elements, see above). The rotation axis coincides with either the shear direction η1 or the normal to the twinning plane K 1 . In the software written by the author, these values are directly displayed in crystal coordinates, so the twinning system can be recognized instantly. Figure 14 shows a twin boundary in an Al2O3 bicrystal. The boundary reveals two types of secondary boundary dislocations. The Burgers vectors of these dislocations have been determined by HRTEM (Argilier, 1997). From the measured spacing between the dislocations, a deviation from the true twin orientation of 0.8◦ was calculated. Second, matrix and twin orientations
394
STEFAN ZAEFFERER
Figure 14. Twin boundary in an Al2O3 bicrystal exhibiting misfit boundary dislocations. (Micrograph by S. Argilier.) The stereographic projection below shows the two measured crystal orientations after a (simulated) rotation to a symmetrical position.
were measured by the Kikuchi technique described in Section III and the misorientations calculated.∗ Table 2 shows a part of the output of the ∗ The author’s program allows one to store any number of orientations of crystals of similar or different phases in memory. By mouse clicks, those crystals are chosen between which the misorientation if to be calculated. The program then displays the misorientation and a superimposed stereographic projections of both crystals.
COMPUTER-AIDED CRYSTALLOGRAPHIC ANALYSIS IN TEM
395
TABLE 2 Selection of the Misorientation Calculation Table for a Twin in Al2O3. The Twin Orientation Relationship Is Easily Revealed by the Appearance of the 180◦ Misorientation Angle Misorientation axis in crystal coordinates 0.440 0.880 0.180 −0.699 0.700 0.143 −0.891 −0.445 −0.091 −0.409 −0.818 0.404 0.874 0.436 −0.216 0.667 −0.669 0.329 0.699 −0.700 −0.143 −0.880 −0.440 0.180
Plane perpendicular to the misorientation axis in the crystal coordinates
Misorientation angle (◦ )
(0.000 0.442 0.897) (−0.575 0.575 0.582) (−0.702 0.000 −0.713) (−0.000 −0.200 0.980) (0.378 −0.000 −0.926) ( 0.354 −0.354 0.866) (0.574 −0.574 −0.584) (−0.442 0.000 0.897)
179,93 124,05 124,11 179,96 86,57 86,51 124,11 179,93
misorientation calculations. The twin boundary is easily revealed. The deviation of the misorientation axis from the true twin misorientation axis was calculated to be 0.75◦ around a direction close to the 11-20 direction. The accuracy of the measurement obtained with the method described can be exceeded by CBED (c.f. Morniroli, 1996). However, the speed (no more than some minutes for two precise measurements) and ease of the procedure is an important argument for the method described.
C. Determination of Foil Thickness and Boundary Plane The misorientation alone is not sufficient to completely characterize a grain boundary, because the crystallographic position of the boundary is another free variable. It can be determined from a tilt experiment, provided the sample is at least 2 to 3 times thicker than the extinction length for a low-indexed diffraction vector: In a first step, the thickness of the specimen has to be measured. Therefore the orientation of one of the crystals is determined close to the grain boundary. The software is then able to calculate the tilt angles necessary to excite two beam conditions (s = 0) for a low-indexed diffraction vector and the sample is set to this position. If the crystal is thick enough and the density of lattice defects is low, the grain boundary now shows characteristic thickness fringes, as described by Eq. (2b). The distance in foil thickness between two dark fringes is the extinction distance ξ g for the excited diffraction vector g. Figure 15a shows the situation schematically. From the number of dark fringes the projected thickness of the crystal can be easily calculated: d ′ = (n + 1/2)ξg
(18a)
396
STEFAN ZAEFFERER
Figure 15. Determination of specimen thickness and crystallographic grain boundary plane. (a) Sketch of the geometric setup. (b) Crystal in titanium excited under two-beam conditions with g = (01–10) showing strong grain boundary fringes. Sample tilt: 4◦ (x-axis) 1.6◦ (y-axis). Number of thickness fringes: 3. Extinction distance ξ(01−10) = 94 nm. Therefore, a thickness of about 330 nm and a grain boundary inclination of ±55◦ is determined. From the inclination angle and the crystal direction along the arrow a boundary plane of either (hkil) = (10-11) or (−10-12) for the most left position marked in the image is calculated. The distinction between both has to be made with a further tilt experiment.
COMPUTER-AIDED CRYSTALLOGRAPHIC ANALYSIS IN TEM
397
The true foil thickness is determined from the tilt angle β which is calculated as the angle between the current beam direction and the predetermined foil normal. It is thus: d = d ′ cos(β)
(18b)
It should be mentioned that the foil thickness can also be determined, very elegantly, from convergent beam electron diffraction patterns. Here also welldefined two-beam conditions are required but no grain boundary has to be used. Instead, the “thickness” fringes are directly visible—and be counted there—in the diffraction spot itself, because the spot fine structure shows the rocking curve of the crystal. Tanaka et al. (1994) describes the procedure in detail. From the measurement of the foil thickness the crystallographic plane of any interface can now be easily determined: first the width w of the projected interface is measured. In the author’s program, the measurement is done directly in the acquired image by marking the interface with two mouse clicks. In the next step, the program calculates the inclination angle ω of the interface from the width w and the previously determined foil thickness (for the untilted crystal is w = w′ sin(β)). Next, the crystallographic direction of the surface–interface intersection u has to be determined. This also is done by marking the position of the intersection in the image with mouse clicks. Since the current orientation of the crystal is known (the computer program keeps track of all tilt events), the crystallographic direction u can be calculated, taking into account the magnetic rotation of the image for the image magnification chosen. From u, the known foil normal n and the inclination angle ω the plane normal p is calculated by rotating the vector n about the vector u by an angle ω. The whole procedure is illustrated in Figure 15b which shows a grain in a titanium specimen.
VI. Determination of Slip Systems Dislocations are the second important lattice defects visible in TEM images. They can be distinguished into lattice dislocations, which are found inside a crystal and interface dislocations found in all kinds of interfaces. The discussion here will focus only on the former. Lattice dislocations are, together with mechanical twinning, the microscopic mechanisms of plastic deformation. Their characterization is therefore indispensable for the investigation of the plastic behavior of a material. During plastic deformation, dislocations are created in dislocation sources (for example, Frank–Read sources) and then glide on more or less well-defined slip planes.∗ The passage of a dislocation ∗ Another important form of movement is thermally activated dislocation climb, which will not be regarded further here.
398
STEFAN ZAEFFERER
through the lattice leads to a shear deformation whose direction and magnitude is given by the Burgers vector, b, which is the most characteristic value of a dislocation. Additionally, the crystallographic direction of the dislocation line u is important. If b and u are not parallel to each other, they define a plane in which the dislocation can move. The normal to this slip plane, ns, is the third characteristic value of a dislocation. If b and u are perpendicular to each other, the dislocation is called an edge dislocation; in other relative positions of b and u (except of b parallel u), it is called a mixed dislocation. If b and u are parallel, the dislocation is called a screw dislocation. In this case, the dislocation can, in principle, move on all planes that contain the Burgers vector. If the Burgers vector is equal to a lattice vector, the passage of the dislocation does not change the order of the lattice. These dislocations are called perfect dislocations. In contrast, dislocations with Burgers vectors smaller than a lattice vector are called partial dislocations. Their appearance in a crystal lattice always leads to the formation of a stacking fault. In the following, a simple computer-aided method for the determination of perfect dislocations in materials with only a weak elastic anisotropy (like Ti, W, or Al) is presented. Other cases and the limitations of the method are discussed in the subsequent section. Finally, a geometric method for the determination of the dislocation line direction u and the slip plane normal ns is presented, which is applicable to all kinds of dislocations. Generally, all techniques described here require that the dislocations are individually observable even over larger specimen tilts (≥20◦ ). This limits, in the case of deformation studies in metals, the possible deformation to values corresponding to samples deformed by maximum 10 to 15% of tensile strain.
A. Determination of Burgers Vectors of Perfect Dislocations in Elastically Isotropic Materials The principle of the Burgers vector determination in TEM is based on the observation of the dislocation contrast under several well-defined diffraction conditions (see, for example, Edington, 1991, for details). From the continuums theory of elasticity, it is found that for elastically isotropic material the lattice planes (hkl) that contain the Burgers vector are not or only little distorted by the dislocation. Therefore, diffraction on such planes with a diffraction vector ghkl occurs as in a perfect lattice. In the image created under two-beam conditions with this diffraction vector, the dislocation is invisible or shows only faint contrast. The Burgers vector b and the active diffraction vector g are perpendicular to each other, i.e., g · b = 0.
(19)
COMPUTER-AIDED CRYSTALLOGRAPHIC ANALYSIS IN TEM
399
Only pure screw dislocations show invisibility for all cases where Eq. (19) is fulfilled. For edge and mixed dislocations, two further conditions have to be satisfied: g · be = 0 and g · (b × u) = 0,
(20)
where be denominates the edge component of the dislocation. However, a contrast minimum (“effective invisibility”) is reached also for edge or mixed dislocations even if only Eq. (19) is fulfilled. Thus, the Burgers vector can be determined by finding at least two linear independent diffraction vectors for which the dislocation, line image disappears. In praxis, the crystal is tilted to excite different diffraction vectors under two-beam conditions. Each time, the contrast of the dislocation is controlled until at least two consistent diffraction vectors have been found where the dislocation is invisible. It should be noted that the contrast of dislocations is not always unequivocal. Dislocations may be visible when Eq. (19) is fulfilled or almost invisible when it is not. Therefore, it is important to establish the extinction conditions for a sufficiently large number of different (nonlinear) diffraction vectors. The rather long and complicated procedures of tilt experiments can be significantly facilitated by the use of a computer program that determines the appropriate tilt angles for two-beam conditions by a reasonably good simulation of the diffraction mode. The computer-aided procedure for the determination is as follows (Fig. 16, see color insert): In a first step, the orientation of the grain of interest is determined and the recalculated diffraction pattern displayed on the computer monitor. As long as there is nothing known about the dislocation under consideration the user has to guess a possible Burgers vector and enter this into the program. The computer then calculates all diffraction vectors that fulfill the g · b = 0 criterion and displays the corresponding Kikuchi lines or diffraction spots in the simulated pattern in a different color. The user may now determine on the computer a sample rotation for which one of the indicated diffraction vectors is excited under two-beam conditions, i.e., for which the primary beam coincides with just one Kikuchi line. Finally, the sample is tilted to these predetermined tilt angles while the microscope is working in imaging mode. In this way, the image shift due to noneucentric tilt can easily be corrected. After tilting, the contrast of the dislocation is checked. If it vanishes, the g · b = 0 criterion may be fulfilled and the procedure is repeated for the next marked diffraction vector. If the dislocation is visible for any of these vectors, the procedure must be continued with another guess for b until the dislocation contrast vanishes for at least two, better yet three, of the marked diffraction vectors. A completely different approach for the determination of Burgers vectors is proposed by Cherns and Preston (1986) and worked out with detailed examples
400
STEFAN ZAEFFERER
by Tananka et al. (1994). The technique is based on CBED patterns formed with an electron probe which is so far defocused that the whole strain field of the dislocation is illuminated (defocused CBED). The sample is tilted in a way that three linear independent diffraction vectors are about equally strongly excited. The Kikuchi lines visible inside the diffraction spots of a reflection g now show, depending on the value of n = g · b, a characteristic break-up into several line segments. From the known diffraction vector, g, and the number of line segments, the value n and subsequently the Burgers vector can be determined. As for the conventional techniques for the Burgers vector determination, the sample has to be tilted in order to achieve well-defined diffraction conditions. However, the method of Tanaka has the big advantage that different values of n can be used, that is, only one tilt position is necessary. Additionally, the effect of the line splitting is more easily revealed than the invisibility of a dislocation. On the other hand, the method requires the careful choice of the correct ratio t/ξ g, that is, the foil thickness t must be known, otherwise the line splitting becomes uninterpretable.
B. Burgers Vector Determination in Other Cases Edington (1991) has compiled a large number of situations for the invisibility or visibility of partial dislocation and the included stacking fault in elastically isotropic material of fcc structure. Generally, as in the case described above, the product of the Burgers vector, bp, of the partial and the diffraction vector g must be controlled and the image contrast observed under two-beam conditions (s ≈ 0). Invisibility of partial dislocations of Shockley type (bp = 1/6112) can, for example, occur if the included stacking fault is visible and g · bp = ±1/3. Normally, the invisibility or visibility also depends on the foil thickness and the dislocation line direction u. Again, the author’s computer program may facilitate the necessary tilt experiments insofar as the user may enter different values for the product g · bp and for bp (taken, for example, from Edington, 1991) and the program displays, as described above, the Kikuchi lines with the appropriate g vectors. The appropriate tilt angles are quickly determined and the dislocation and stacking fault contrast can be checked for the different possibilities. Also, the program allows calculation of the foil thickness (see above, Section V.C) and the line direction (see below, Section VI.C), so that some cases of partial dislocations may be readily solved. In more complicated structures and especially in the case of elastically anisotropic materials the simple geometric model for the invisibility of dislocations is no longer valid because the strain field of a dislocation becomes more complicated. In this case, the correct Burgers vector of a dislocation
COMPUTER-AIDED CRYSTALLOGRAPHIC ANALYSIS IN TEM
401
can only be determined by the technique of image matching, as proposed, for example, in the textbooks of Edington (1991) or by Williams and Carter (1996), or by the already-mentioned technique of defocused CBED, described by Tanaka (1994). The image matching technique is based on the comparison of the real image of the dislocation with a computer image, simulated for the same imaging conditions. Williams and Carter (1996) present a long list of computer programs for image simulation. All of these programs need as input the exact geometry of the dislocation (i.e., the dislocation line direction u and the crystal thickness t) and the diffraction conditions (i.e., the active diffraction vector g and the value of s). Again, the program described here can deliver the necessary data.
C. Determination of the Dislocation Line Direction The crystallographic line direction, u, of a dislocation is determined from two images of the dislocation taken under two different tilt positions of the sample. The principle of measurement is sketched in Figure 17. Again, the orientation of the crystal has to be determined first. The sample is then tilted to a position where the dislocation exhibits good contrast. An appropriate position, for
Figure 17. Sketch of the geometric setup of the determination of the dislocation line direction.
402
STEFAN ZAEFFERER
example, a two-beam situation with g · b = 0, can be predetermined using the computer program. In any way, the program must keep track of the currently set tilt angles and orientation. Now an image of the dislocation is captured and the coordinates of two points on the dislocation line are measured by mouse clicks. From these coordinates and the current crystal orientation, the computer calculates the crystallographic orientation of the projected dislocation line, taking into account the magnetic rotation of the image for the current magnification. Additionally, the plane p1, containing the projected dislocation line and the current crystal direction parallel to the electron beam (semitransparent plane in Fig. 17) is calculated. The sample is now tilted at least 20◦ about an axis which is not parallel to the normal of p1, to another position with good dislocation contrast. There the line measurement procedure is repeated and a second plane, p2, determined. The intersection of the planes p1 and p2 is the three-dimensional crystallographic dislocation line direction u. Additionally the program calculates the angle between the two planes. The larger this angle, the more precise the determination of u. Generally, the angle should not be smaller than 15◦ degree to yield an accuracy of u of a few degrees. This is normally sufficient, especially since the change of u due to the dislocation curvature is often significantly larger. From the dislocation line direction u and the Burgers vector b, the computer program also determines the dislocation type and the slip plane normal ns. If the dislocation is of edge or mixed type, the slip plane is simply calculated by the vector product of b and u. In the case of a screw dislocation, the slip plane is in principle not fixed. However, it is often observed that even screw dislocations occupy a defined slip plane, indicated by the appearance of slip lines or slip bands (Fig. 18). In these cases, the measurement of the slip line direction or of the direction of the intersection of the slip band with the surface and the Burgers vector permits determination of the slip plane. With the described computer-aided procedures, the time for the determination of a complete slip system in a titanium alloy was reduced to about 10 to 15 min. Therefore, a larger, statistically meaningful number of slip systems could be determined in a study on the influence of the alloy composition on the activation of slip systems and on the development of the deformation texture in titanium alloys (Zaefferer 1996, 2002). Figure 19 (see color insert; from Zaefferer, S. (2000). J. Appl. Cryst. 33, 10–25. Blackwell Science, Oxford, with permission) shows one example of results of this study. From the measured crystal orientations and the known deformation tensor, the slip systems with the highest shear stresses were calculated using the Sachs model (Sachs, 1928; see also Kocks, 1970). The slip systems were determined and subsequently compared with the calculated ones. This comparison led to an estimation of the critical resolved shear stresses of different slip systems dependent on the oxygen content of the alloy.
COMPUTER-AIDED CRYSTALLOGRAPHIC ANALYSIS IN TEM
403
Figure 18. Principle of slip plane determination in case of screw dislocations arranged in slip bands. (a) Micrograph of screw dislocations in an α-titanium alloy. (b) Sketch of measurement principle. s denotes the intersection vector of the sample surface and the slip band, sp the vector s projected into the image, u is the dislocation line direction, b the Burgers vector.
VII. Phase and Lattice Parameter Determination The determination of crystal structure and lattice parameters of phases too small for x-ray diffraction is a very important task of TEM. Volumes as small as 10−3 μm3 can be investigated, and a complete space group determination may be performed. The determination of the atomic structure, in contrast, is difficult due to the strong appearance of dynamic diffraction. Also, due to the small Bragg angles in TEM electron diffraction, the determination of lattice parameters is about an order of magnitude less precise than that determined by x-ray diffraction techniques.
404
STEFAN ZAEFFERER
In the following, only two aspects out of the large field of structure determination in the TEM will be regarded further because they are based on purely geometric models and have been implemented in the author’s program. One is the fit of lattice parameters for substances which are approximately known; the second is the discrimination between different possible phases from diffraction patterns. A third aspect, the determination of space group information from diffraction patterns, will not be treated here, but it shall be mentioned because this task is also mainly geometry based. Most completely, the space group can be determined by a systematic investigation of the symmetry and fine structure of CBED from well-defined zone axes. The method is described in detail by Tanaka et al. (1988) or in the textbook by Williams and Carter (1996). Also the symmetry of Kikuchi patterns can be used to reveal, at least partially, the space group of a material as has been demonstrated, for example, by Baba-Kishi and Dingley (1989) for the case of backscatter Kikuchi patterns.
A. Fit of Lattice Parameters Two different procedures may be used to achieve an exact fit of lattice parameters. First, a given diffraction pattern may be recalculated with various lattice parameters until the best coincidence of experimental recalculated pattern is found. A second method is to calculate the lattice parameters directly from a set of measured diffraction vectors and their known indexation. This procedure allows the quick adjustment of lattice parameters a, b, c, for any crystal structure and will be described in the following. The precondition is that the symmetry and the lattice parameters are known so that a diffraction pattern can be correctly indexed. The procedure works most easily with Kikuchi patterns because the positions of Kikuchi lines exactly correspond to the position of the diffraction vector. In a first step, the pattern is indexed and the crystal orientation is calculated. For this purpose, particularly wide Kikuchi bands should be chosen for measurement since they give the best precision in the lattice parameters. Several possible crystal symmetries and/or crystal parameters may be tried until a correct indexing is achieved. The components of the diffraction vectors with respect to the microscope coordinate system are known from the measurement: ⎞ ⎛ sin γg cos αg ⎜ ⎟ gm = |g| ⎝ sin γg sin αg ⎠ (21) cos γg
COMPUTER-AIDED CRYSTALLOGRAPHIC ANALYSIS IN TEM
405
The measured angles α g and γ g define the direction of the diffraction vector with respect to the goniometer axis and the foil plane. |g| is the length of the measured diffraction vector. Using the crystal orientation G, the vector is now transformed into the crystal coordinate system given in Cartesian coordinates: go = G gm
(22)
The vector go is related to the appropriate reciprocal lattice vector in the crystal system gc which is known from indexing: gc = go A
(23)
(crystal matrix A, defined in equation (9)). If the lattice parameters α, β, and γ are assumed to be constant, the matrix A is only a function of a, b, and c. It is thus gc = Ggm A
(24)
From this linear equation system a, b, c can be calculated for each measured diffraction vector. The final values are calculated as an average over all determined values. The procedure described can be used for an iterative fit for lattice parameters. Therefore, the reciprocal lattice is recalculated with the lattice parameters determined. The pattern is reindexed, the orientation determined, and the new lattice parameters are again recalculated. With well-calibrated microscope parameters (accelerating voltage, U, and the camera length, L), a correctly set diffraction focus and the use of Kikuchi lines with high indices, a precision of about 1 to 2% is achieved. The precision is most seriously influenced by the fact that Kikuchi patterns usually span a relatively large solid angle (lines may be more than 10◦ away from the primary beam). This leads to pattern distortions in regions far away from the primary beam and limits the precision of the measurement. Spot patterns, in contrast, show only a small opening angle (in the order of the Bragg angle) and may therefore be used for a fine tuning of the roughly determined values. It must be taken into account, however, that spot patterns do not allow a high accuracy for the determination of the position of the diffraction vectors. The most precise determination of lattice parameters is reached by the use of higher order Laue zone (HOLZ) Kikuchi lines, which combine the small opening angle of spot patterns with the high precision of Kikuchi lines. An appropriate method has been proposed, for example, by Matsumura et al. (1986). More recently Kr¨amer et al. (2000) developed a similar method for the precise determination of elastic strain in thin aluminum films. To reach a sufficiently high precision for this task, they use a dynamic correction for the recalculated line positions and determine the experimental lines with a high resolution Hough transform (c.f. Section IV.A).
406
STEFAN ZAEFFERER
Figure 20. Fit of lattice parameters. (a) Micrograph of unknown precipitation in T60 (Ti + 2000 ppm O). (b) Kikuchi pattern of the precipitate, superimposed with a recalculated pattern after a fit of lattice parameters. The pattern was calculated using a face centered tetragonal lattice. (From Zaefferer, S. (2000). J. Appl. Cryst. 33, 10–25. Blackwell Science, Oxford, with permission.)
The procedure described has been applied to determine the phase and the lattice parameters of an unknown, needle-shaped precipitate in the titanium alloy T60 (Fig. 20). T60 is almost pure titanium (hcp), containing 2000 ppm of oxygen. The camera length and the accelerating voltage were calibrated from Kikuchi patterns (c.f. Section VIII) of the matrix using the lattice parameters of Ti, containing 2000 ppm of oxygen (Zwicker, 1974). Subsequently the Kikuchi patterns of the precipitate were analyzed, trying several different phases proposed by Williams (1973) and Banerjee et al. (1988). The patterns could be consistently indexed assuming a tetragonally distorted fcc lattice. The lattice parameters were calculated to be a = b = 0.416 nm and c = 0.447 nm. This phase is reported by Banerjee et al. (1988) to be an artifact caused by hydrogen adsorption during the preparation of the thin foil. The cited authors,
COMPUTER-AIDED CRYSTALLOGRAPHIC ANALYSIS IN TEM
407
however, mention slightly different lattice parameters. Once again, it should be pointed out that the precision of the method is inferior to other methods. The speed of measurement, however, is rather high. The phase determination for the case described did not take longer than a quarter of an hour (sufficient information on possible phases was available).
B. Phase Discrimination It is often the case that the phases in a sample are not completely unknown but have to be chosen from a number of possibilities. These possibilities might arise from the overall chemical composition of the sample, from the local composition of the crystal, measured, for example, by EDX analysis, or from other considerations (for example, the origin of the sample). The choice between the different phases can, in many cases, be easily made by the analysis of one or two electron diffraction patterns. To this end, the author’s computer program allows loading of the crystallographic information for all possible phases in memory and indexing a given diffraction pattern for each of these phases, as has been described in Section III. The decision as to which of the phases is correct is made, as described in Section III.E, by the indexing confidence index, the average angular deviation between measured and indexed diffraction vectors,
Figure 21. Example for phase discrimination by electron diffraction. The upper left image is a spot diffraction pattern of an unknown asbestos mineral. The pattern has been indexed with different crystal structures for different possible minerals (c.f. Table 3 for the crystal structures and indexing results). The recalculated patterns are shown ordered for their fit from up to down and left to right. In no case do the simulated intensities coincide exactly with the measured pattern, which might be due to dynamic diffraction conditions.
408
STEFAN ZAEFFERER
TABLE 3 Lattice Parameters and Indexing Result for the Indexing of the Diffraction Pattern of an Unknown Asbestos Fiber. The Results Are Sorted from Top to Bottom for the Quality of the Indexing. The “Deviation Angle” Is the Sum of All Angular Deviations between Measured and Recalculated Planes or the Sum of the Bragg Angle Deviations between Measured and Recalculated Planes Lattice parameters [A], [◦ ]
Deviation angle [◦ ]
Name
a
b
c
α
β
γ
Indexing confidence
Actinolitea Tremolitea Crocidolitea Antophylliteb Gruneritea Lizarditec
9.84 9.82 9.89 18.54 9.57 5.34
18.1 18.05 17.95 18.02 18.22 5.34
5.28 5.26 5.31 5.28 5.33 7.24
90 90 90 90 90 90
104.7 104.7 107.5 90 102.1 90
90 90 90 90 90 120
1 1 1 1 0.969 No solution
a
Planes
Bragg
0.923 0.931 1.127 2.722 0.921
0.006 0.006 0.008 0.009 0.005
Monoclinic; borthorhombic; ctrigonal.
and the average length deviation of these vectors. With these values, the first solution proposed by the program is normally the correct one. Otherwise, the correct solution has to be determined by visual control. Generally, the use of Kikuchi patterns allows easier phase discrimination because the observed reciprocal space is larger with these patterns. However, in the case of beam-sensitive samples, Kikuchi patterns might not be available. In this case, spot patterns have to be used. An example for this case is given in Figure 21. Here the decision had to be made as to which kinds of asbestos fibers were contained in a given sample. Since the fibers charge easily, Kikuchi patterns could not be obtained. Six different asbestos minerals were considered and their crystallographic information loaded into the program. The difficulty here was the similarity of the different phases. Table 3 gives an overview over their lattice parameters. Before the pattern was indexed, the microscope parameters were calibrated using a diffraction pattern of pure aluminum. The spot pattern of the fiber was indexed and as best fit the mineral actinolite was determined. The indexing results are compiled as well in Table 3. VIII. Calibration All tasks for computer-aided TEM presented above need well-calibrated microscope parameters like the accelerating voltage, U, and the camera length, L, and the magnetic rotations in diffraction and imaging mode, ρ d and ρ i. Again,
COMPUTER-AIDED CRYSTALLOGRAPHIC ANALYSIS IN TEM
409
the procedures presented in the following are based on simple geometric concepts using the kinematic theory of electron diffraction for the calibration of L and U. If higher precision is required, for example, for the determination of elastic lattice strain (see, for example, Deiniger et al., 1994), dynamic effects and the crystal potential have to be taken into account.
A. Determination of Diffraction Length and Accelerating Voltage For the calibration of U and L, a sharp diffraction pattern, preferably a Kikuchi pattern, of a well-known substance is necessary. Two different methods are available, and they may be applied depending on whether a given diffraction pattern can be indexed or not. If the diffraction pattern cannot be indexed, a trial-and-error procedure is first applied to roughly determine the value of L. For this purpose, a diffraction pattern is acquired and some Kikuchi lines or spots in the pattern are measured. The accelerating voltage is, as a first guess, taken from the microscope reading. The program then tries to index the pattern with different diffraction lengths, starting from a minimum value and raising it in user-defined steps up to a maximum. The diffraction length for which the recalculated pattern fits best to the experimental one is the result of this procedure. The diffraction pattern can now be correctly indexed and the second procedure is applicable. The second procedure is based on the calculation of the diffraction length and accelerating voltage from the geometry of the diffraction vectors of a pattern. If a Kikuchi pattern is used, both values L and U can be determined; from spot patterns, only L or U can be calculated. From the position of the Kikuchi lines in the pattern, the camera length is calculated as the projection length of the gnomonic projection. The widths of the Kikuchi bands are used to calculate the camera constant λL (λ: electron wavelength). Together with the previously determined camera length, the accelerating voltage is then calculated. With the new values of L and U, the orientation is recalculated and the procedure repeated until the values of L and U do not change any longer. It has been found that the procedure described sometimes yields values for U which deviate by more than 10% from the value indicated at the microscope, although the recalculated diffraction pattern seems to superimpose perfectly with the experimental one. The reason for this is a wrongly-set diffraction focus, which leads, in Kikuchi patterns, to a shift of the line positions in the outer part of the image although the pattern continues to appear focused. Unfortunately, the correct diffraction focus point seems to be different in the outer part and the center part of the diffraction pattern, so that the focus cannot be set by the use of spot patterns (which focus the center part). If highest accuracy for the calibration is required, it is therefore indispensable to work with CBED patterns and use HOLZ lines for the calibration of U.
410
STEFAN ZAEFFERER
B. Determination of the Magnetic Rotation The knowledge of the magnetic rotation of diffraction pattern and image, ρ d and ρ i, is indispensable for all tasks presented in this work. In textbooks about TEM, usually a method is introduced which allows the determination of the relative rotation of diffraction pattern and image. This procedure uses a MoO3 crystal whose morphology is related to its crystal orientation. The crystal is observed in an unfocused diffraction pattern which shows the crystal image and the diffraction pattern at the same time. In this way, the relative rotation ρ d-ρ i between image and diffraction mode can be measured. For the work presented here it is, however, necessary to know the absolute values of ρ d and ρ i. In the following, a simple and quick technique is introduced which can be carried out using a simple metallic (but nonmagnetic) specimen. The procedure for the determination of ρ d uses the fact that a rotation of the sample around the goniometer axis appears as a movement of the lines in a Kikuchi pattern perpendicular to the rotation axis. The rotation axis can thus easily be determined from two Kikuchi patterns taken from the same crystal before and after a sample rotation around the goniometer axis. In both patterns, the same pole (crossing of two Kikuchi lines) is measured by mouse clicks and the program calculates the magnetic rotation. Figure 22 illustrates the procedure. Care must be taken that the sample height is correctly set to ensure the eucentric tilt of the specimen. A similar principle is used for the measurement of the image rotation ρ i. Here, the sample is not rotated about the goniometer axis but moved along it. This time, the same point in the image is measured before and after the movement. Both rotation values, ρ d and ρ i, are measured relative to a right-handed microscope coordinate system. A standard convention is to define the positive
Figure 22. Principle of the determination of the magnetic rotation in diffraction mode. The left image is taken after a specimen tilt of 8◦ around the goniometer axis with respect to the right image. The marked pole has performed the indicated movement.
COMPUTER-AIDED CRYSTALLOGRAPHIC ANALYSIS IN TEM
411
x-direction to point out of the microscope along the goniometer axis and the positive z-direction to point against the electron beam direction. This convention fixes the axes and senses for all rotation angles.
IX. Conclusions Techniques for computer-aided crystallography in analytical TEM have been presented and demonstrated mainly by the example of a computer program written by the author. The techniques include, first of all, different methods for the semiautomated and completely automated determination of crystal orientations from different forms of electron diffraction patterns. Further, the analysis of different lattice defects like grain boundaries and dislocations is discussed. Finally, methods for the determination of crystal lattice parameters and for the calibration of microscope parameters are presented. In contrast to other approaches, the algorithms presented here completely avoid the use of the dynamic theory of electron diffraction. Rather, the kinematic theory and relatively simple geometric concepts are used to allow the quick access to interesting materials properties without lengthy and complicated calculations. It is one goal of this paper to show how far these concepts can be applied and where are their limits. To this end, Section II presents some more theoretical considerations about the application of dynamic and kinematic theory. It is shown there, and throughout the paper, that the kinematic theory may give useful results also in those cases when dynamic effects are clearly active. In Section III, concepts for the determination of crystal orientations are introduced and compared especially under the aspect of reaching the highest spatial resolution and orientation accuracy. Although the use of Kikuchi patterns is certainly the most precise and straightforward method, the use of spot patterns can be very interesting and becomes even indispensable in the case of orientation measurements in highly deformed metals. The accuracy of the spot pattern orientation determination can be much improved by taking into account positions and intensities of the diffraction spots. Section IV introduces algorithms for the automation of crystal orientation determination. Two completely different approaches for the orientation determination (not developed by the author), using either Debye–Scherrer ring patterns or dark field intensity measurements, are here presented as well and compared with the sequential single grain orientation measurement techniques. Sections V and VI deal with the analysis of lattice defects and it is shown how far the concept of a “virtual microscope” can ease these tasks. The term virtual microscope here means the simulation of the diffraction mode of the microscope on the computer. While the microscope works in imaging mode, allowing observation of the contrast of lattice defects, the computer simulates
412
STEFAN ZAEFFERER
the diffraction mode and is used to determine the necessary tilt angles for, for example, two-beam conditions leading to specific contrast in the image. Using these concepts, the thickness of a sample can be determined by aid of thickness fringes on grain boundaries, and the crystallography of grain boundaries can be investigated. Dislocations can be studied using the g · b = n criterion for the Burgers vector determination. The dislocation line direction is measured from the image of a dislocation under different tilt angles. From line direction and Burgers vector, the type and glide plane of the dislocation can be determined. The methods presented are quick and effective but not applicable in all cases. For such cases, it is discussed which other methods, using convergent beam electron diffraction and image matching methods, have to be applied. An important task in TEM is the determination of phases and lattice parameters. In Section VII, a procedure is introduced that allows the quick but only little precise adjustment of lattice parameters on the basis of Kikuchi patterns. Another method permits the discrimination between phases by indexing diffraction patterns with different possible crystal structures. Here again, the limits of the techniques are discussed and alternative, more precise methods (particularly convergent beam techniques) proposed. Finally, some procedures for the computer-aided calibration of microscope parameters (i.e., the accelerating voltage, the camera length, and the magnetic rotation in diffraction and imaging mode) are presented in the last section. The combination of several of the described procedures in one single computer program can be used as a rather universal tool for crystallographic work at the TEM. Especially when such a program is connected online to the TEM, via a camera transmitting the images to the computer or, even better, by some control of the microscope (especially the goniometer), it may facilitate the TEM work enormously. The examples presented to illustrate these possibilities are taken from studies on deformation mechanisms in titanium, recrystallization mechanisms in an Fe-Ni alloy, from orientation determination and grain boundary studies in TiAl intermetallics and Al2O3, and phase discrimination between different asbestos fibers.
Acknowledgment The author is especially grateful to Professor R. Schwarzer, Technische Universit¨at Clausthal, Germany, who taught him the basics of transmission electron microscopy. His work on the evaluation of Kikuchi patterns inspired the first TEM computer programs of the author. The author is also grateful to Professor Bunge, director of the Institut f¨ur Metallkunde in Clausthal, where the work on deformation mechanisms in titanium was carried out. The work on highly deformed metals reported here has been carried out by the group of Professor R. Penelle and
COMPUTER-AIDED CRYSTALLOGRAPHIC ANALYSIS IN TEM
413
Dr. T. Baudin, Universit´e Paris-Sud, France, who are gratefully acknowledged. A large part of computer work has been carried out by the group of Professor T. Maki and T. Furuhara, University of Kyoto, Japan, who are also gratefully acknowledged. The author further thanks D. Dingley and S. Wright, TexSem Laboratories, Salt Lake City, Utah, USA, for the permission to use graphics on their dark field diffraction method and for many helpful discussions. The financial support of the work of the author by the European Community (BRITE-EURAM program BREU 0117), of the Acad´emie de Sciences, France, and of the Japanese Society for the Promotion of Science (JSPS) is gratefully acknowledged. Last but not least the author thanks A. Takada for her help in preparing the graphics in this paper.
References Adams, B. L., Wright, S. I., and Kunze, K. (1993). Orientation imaging: The emergence of a new microscopy. Met. Trans. 24A, 819–831. Baba-Kishi, K. Z., and Dingley, D. J. (1989). Backscatter Kikuchi diffraction in the SEM for identification of crystallographic point groups. Scanning 11, 305–312. Banerjee, D., Shelton, C. G., Ralph, B., and Williams, J. C. (1988). A resolution of the interface phase problem in titanium alloys. Acta Metall. 36, 125–141. Bollmann, W. (1970). Crystal Defects and Crystalline Interfaces. Berlin: Springer-Verlag. Bunge, H. J. (1982). Texture Analysis in Material Science. London: Butterworth. Cherns, D., and Preston, A. R. (1986). Proc. 11th Int. Congr. on Electron Microsc., Kyoto, 712. Cromer, D. T., and Mann, J. B. (1968). X-ray scattering factors computed from numerical HartreeFock wave functions. Acta Cryst. A24, 321–324. Deininger, C., Necker, G., and Mayer, J. (1994). Determination of structure factors, lattice strains and accelerating voltage by energy filtered convergent beam electron diffraction. Ultramicroscopy 54, 15–30. Dingley, D. J., and Wright, S. I. (2000). Software “ACT, Automated crystallography for the TEM.” TSL/EDAX, Draper, USA. Edington, J. W. (1991). Practical Electron Microscopy in Material Science. Herndon, VA: TechBooks. Haidu, F. (1972). Revised parameters of the analytic fits for coherent and incoherent scattered x-ray intensities of the first 36 atoms. Acta Cryst. A28, 250–252. Hirsch, P. B., Howie, A., Nicholson, R. B., Pashley, D. W., and Whelan, M. J. (1965). Electron Microscopy of Thin Crystals. London: Butterworth. Hough, P. V. C. (1962). Methods and means for recognizing complex patterns. US patent 3069654. Humphreys, F. J. (1983). The determination of crystallographic textures from selected areas of a specimen by electron diffraction. Textures Microstructures 6, 45. International Tables for X-Ray Crystallography (1999). Vol. C: Mathematical, Physical and Chemical Tables, edited by E. Prince. London: Kluwer Academic. Kocks, U. F. (1970). The relation between polycrystal deformation and single crystal deformation. Met. Trans. 1, 1121–1143. Kr¨amer, S., Mayer, J., Witt, C., Weickenmeier, A., and R¨uhle, M. (2000). Analysis of local strain in aluminium interconnects by energy filtered CBED. Ultramicroscopy 81, 245–262.
414
STEFAN ZAEFFERER
Kr¨amer, S., and Mayer, J. (1999). Using the Hough transform for HOLZ line identification in convergent beam electron diffraction. J. Microsc. 194, 2–11. Kraus, W., and Nolze, G. (1993). POWDER CELL, Programm zur Darstellung von Kristallstrukuren und Berechnung der zugeh¨origen R¨ontgendiffraktogramme f¨ur polykristalline Proben. Deutsche Bundesanstalt f¨ur Materialpr¨ufung, Berlin. Krieger-Lassen, N. C. (1994). Automated determination of crystal orientations from electron backscattering patterns. Ph.D. thesis, Danmarks Tekniske Universitet, Lyngby. Krieger-Lassen, N. C. (1995). Computerized analysis of Kikuchi patterns. Proc. 16th Risø Int. Symp., pp. 405–411. Krieger-Lassen, N. C. (1998). A new procedure for automatic high precision measurements of the position and width of bands in backscatter Kikuchi patterns. Proc. Int. Conf. on Textures and Anisotropy of Polycrystalline Materials. Mater. Sci. Forum 273–275, 201–208. Krieger-Lassen, N. C., Juul Jensen, D., and Conradsen, K. (1992). Image processing procedures for analysis of electron diffraction patterns. Scanning Microsc. 6, 115–121. Mansfield, J. (1984). Convergent Beam Electron Diffraction of Alloy Phases. Bristol and Boston: Adam Hilger Ltd. Matsumura, S., Tomokiyo, Y., Kuwano, N., Kominami, M., Okuyama, T., and Oki, K. (1986). Proc. XIth Int. Congr. on Electron Microscopy, Kyoto, pp. 707–708. Morniroli, J. P. (1996). Interface Sci. 4, 273–283. Nes, E., and Hutchinson, W. B. (1989). in Bilde-Sørensen (ed.), Proc. 10th Int. Risø Symp., Risø, Denmark, 233. Philippe, M. J., Esling, C., and Hochheid, B. (1988). Role of twinning in texture development and in plastic deformation of hexagonal materials. Textures Microstructures 7, 265–301. Randle, V. (1993). The Measurement of Grain Boundary Geometry. Bristol and Philadelphia: Institute of Physics Publishing. Rhoades, B. L. (1976). XIDENT—A computer technique for the direct indexing of electron diffraction spot patterns. Research Report 70/76, University of Canterbury, Christchurch. Sachs, G. (1928). Z. Verein Deutscher Ingenieure 72, 734. Sch¨afer, B., and Schwarzer, R. A. (1998). SAD pole figures in transmission and reflection using a high-grade CCD camera a an area detector. Proc. Int. Conf. on Textures and Anisotropy of Polycrystalline Materials. Mater. Sci. Forum 273–275, 223–228. Schumann, H. (1979). Kristallgeometrie. Leipzig: VEB-Verlag. Schwartz, A. J., Kumar, M., and Adams, B. L. (eds.) (2000). Electron Backscatter Diffraction in Material Science. New York: Kluwer Acad./Plenum. Schwarzer, R. A. (1983). Intensit¨atskorrektur fuer die Messung von Polfiguren im TEM. Beitr. elektronenmikr. Direktabb. Oberfl. (BEDO) 16, 131–134. Schwarzer, R. A. (1985). Polfigurmessung mit einem computergesteuerten Transmissions– Elektronenmikroskop. Beitr. elektronenmikr. Direktabb. Oberfl. (BEDO) 18, 61–68. Schwarzer, R. A. (1989). Die Bestimmung der lokalen Textur mit dem Elektronenmikroskop. Habilitation Thesis, Technische Universit¨at Clausthal. Schwarzer, R. A., Krieger-Lassen, N. C., and Sch¨afer, B. (1996). Electron texture goniometer consisting of a TEM with side entry goniometer and a CCD camera as an area detector. Proc. 11th Int. Conf. on Texture of Materials (ICOTOM-11), pp. 170–175. Schwarzer, R. A., and Sukkau, J. (1998). Automated crystal orientation mapping (ACOM) with a computer-controlled TEM by interpreting transmission Kikuchi patterns. Proc. Int. Conf. on Textures and Anisotropy of Polycrystalline Materials. Mater. Sci. Forum 273–275, 215– 222. Schwarzer, R. A., and Weiland, H. (1984). Computer-aided indexing of Kikuchi patterns from cubic, tetragonal and hexagonal crystals for the determination of grain orientations. Proc. 8th European Congress on Electron Microscopy (EUREM 1984), Budapest, pp. 341–342.
COMPUTER-AIDED CRYSTALLOGRAPHIC ANALYSIS IN TEM
415
Schwarzer, R. A., and Weiland, H. (1986). Electron diffraction polefigure measurements, in Experimental Techniques of Texture Analysis, edited by H. J. Bunge. Oberursel, DGM Informationsgesellschaft Verlag, pp. 287–300. Tanaka, M., Terauchi, M., and Kaneyama, T. (1988). Convergent-Beam Electron Diffraction II. Tokyo: JEOL-Maruzen. Tanaka, M., Terauchi, M., and Tsuda, K. (1994). Convergent-Beam Electron Diffraction III. Tokyo: JEOL-Maruzen. Thomson, C. B., and Randle, V. (1996). A study of twinning in nickel. Scripta Mater. 35, 385–390. von Laue, M. (1948). Materiewellen und ihre Interferenzen. Leipzig: Akademische Verlagsgesellschaft. Watanabe, T., and Tsurekawa, S. (1999). The control of brittleness and development of desirable mechanical properties in polycrystalline systems by grain boundary engineering. Acta Mater. 47, 4171–4185. Weiland, H., and Schwarzer, R. A. (1985). On-line Auswertung von Kikuchi- und ChannelingDiagrammen. Beitr. elektronenmikr. Direktabb. Oberfl. (BEDO) 18, 55–60. Williams, D. B., and Carter, C. B. (1996). Transmission Electron Microscopy, Vol. II. New York: Plenum. Williams, J. C. (1973). Titanium Science and Technology. New York: Plenum. Wright, S. I., and Dingley, D. J. (1998). Orientation imaging in the transmission electron microscope. Proc. Int. Conf. on Textures and Anisotropy of Polycrystalline Materials. Mater. Sci. Forum 273–275, 209–214. Zaefferer, S. (1996). Entwicklung und Anwendungen eines elektronenmikroskopischen on-line Verfahrens zur Bestimmung von Verformungssystemen in hexagonalen Werkstoffen. Ph.D. Thesis, Technische Universit¨at Clausthal, Papierflieger, Clausthal Zellerfeld. Zaefferer, S. (2000). New developements of computer-aided crystallographic analysis in transmission electron microscopy. J. Appl. Cryst. 33, 10–25. Zaefferer, S. (2002). A study on active deformation systems in titanium alloys: Dependence on alloy composition and correlation with the deformation texture. Accepted for publication in Mater. Sci. Eng. Zaefferer, S., and Schwarzer, R. A. (1994). Automated measurement of single grain orientations in the TEM. Z. Metallkunde 85, 585–591. Zaefferer S., Baudin T., and Penelle, R. (2001). A study on the formation mechanisms of the cube recrystallization texture in cold rolled Fe-36% Ni alloys. Acta Mater. 49, 1105–1122. Zwicker, U. (1974). Titan und Titanlegierungen. Berlin: Springer.
This Page Intentionally Left Blank
Index
A Ab initio methods, 100 Accelerating voltage, determination of, 409 Additive image functionals, 123, 128–130 Advance, 14 ALCHEMI (Atom Location by Channeling Enhanced Microanalysis) accuracy of, 73–75 competing/supplementary techniques, 102–105 concentrated and less ordered solutions analyzed by, 67–71 defined, 64–66 delocalization, 74, 75–76 dilute solutions analyzed by, 66–67 electron energy loss (EEL), 66, 76 future for, 106 incoherent channeling patterns (ICPs), 71–73 optimizing, 76–77 ordering tie line (OTL), 67–69, 73 origin of term, 63 planar versus axial, 75–76 predicting sublattice occupancies, 100 statistical, 73–75 ALCHEMI experiments, results of concentrated solutions, 97, 101 electron distributions, calculations of, 93, 98–99 electron energy loss (EEL), 97, 100
functional materials, 93, 94–96 intermetallics, 81–93 minerals, 77–81 quasicrystals, 93, 97 Alias–component formulation, 2 Alias-free filter banks, 40–42 Alias-free reconstruction, 40 Aluminum foams, computer tomography images of, 170–176 Analysis filter vector, 34 Analysis operator, 21–22 description of, 30–31 matrix description of, 33–35 Analysis-synthesis operator, 22 description of, 32–33 Atomic force microscopes (AFM), 202 Atom probe field ion microscopy (APFIM), 102, 104 Automatic crystal orientation mapping (ACOM), 377, 378 Axial ALCHEMI, 75–76
B Backscatter Kikuchi patterns (BKPs), 377–380 Balance surfaces, 164–165 Bayesian network, 244, 266–267 Bayesian tree, 247 Bayes optimal estimator, 272, 293 Bayes theorem, 242, 251, 253, 266, 272, 293 Belief propagation, 245 Binary symmetric channel (BSC), 240, 266 417
418
INDEX
Black bend contour, 65 Bloch waves, 64 simulation, 69 Block Toeplitz matrices, 23–26 Body-centered cubic (BCC) lattices, 149–153 Boolean model, 158 Borrmann effect, 64 Boundary plane, determination of, 397 Boundary value, 8 Bragg equation, 359, 361 Bragg position, 64–66 Bragg-Williams, 100 Burgers vectors, 153–155, 398–401
C Cascading codes, 317–318 decoding, 319–323 encoding, 319 optimizing construction parameters, 323–324 Channel(s) binary symmetric, 240 capacity, 240 coding, 233 encoding and decoding, 241–242 memoryless, 239–240 noisy, 239–241 Cluster variation method (CVM), 100 Codes/coding See also Low-density parity-check codes error correction, 232–236 Hamming, 235 linear error-correcting codes and decoding problem, 242–243 mathematical model for a communication system, 237–242
probability propagation algorithm, 244–249 statistical physics of, 236 Communication system, Shannon’s, 237–242 Compact convex sets, 126 Compression, 233 coefficient, 239 data, 238–239 Computer tomography images of metal foams, 170–176 Concatenated/convolutional codes, 234 Conical scanning, dark field imaging by, 388–389 Connectivity number, 128 Convergent beam electron diffraction patterns (CBEDPs), 359, 361, 365–366 Convex body, 126 Convex rings, 128–130 Convex sets, 126–128 Counter-identity matrix, 14 Cross-talk, free from, 47 Crystallography, real-space, 102, 104 Curvatures, principal, 131–132
D Dark field imaging by conical scanning, 388–389 Data compression, 238–239 Data sink, 238 Data source, 238 Debey-Scherrer ring patterns, 390–391 Decoding, 250–252 cascading, 319–323 Gallager, 272 MacKay–Neal, 292
INDEX
with probability propagation, 266–269, 301–302, 319–323 Sourlas, 252–254, 266–269 Delay, 14 Delocalization, 74, 75–76 DeMoivre–Laplace limit theorem, 255 Diagonal matrix, 47 Diffraction contrast, 368–369 Diffraction length, determination of, 409 Diffraction patterns acquisition, 371–372 description of, 358–368 display and control of results, 375–377 orientation determination and, 369–377 Diffraction vectors determination of, 372–373 indexing of, 373–375 Digital image processing (DIP), 121, 377, 380–382 Digitization errors, reducing, 148 Digitized images, analysis of, 145–147 Dilation, 124 Discrete Euler formula, 139 Dislocation line direction, determination of, 401–403 Dislocations, 153–157 determination of, 397–403 Distance modulated STM (DM-STM), 197, 198, 216–218 Distortion function, 40 Dual (biorthogonal) filter vectors, 44 Duality map, 6, 13–14 Dual lifting scheme, 54–55
419
E Edge dislocations, 153–157 EDX software, 74, 76 Electron backscatter diffraction (EBSD) patterns, 375, 377–380 Electron diffraction, 358–369 Electron distributions, ALCHEMI results and calculations of, 93, 98–99 Electron energy loss (EEL), 66, 76 ALCHEMI results, 97, 100 Electron energy loss spectrometry (EELS), 356 Electronic materials, ALCHEMI results, 93, 94 Energy dispersive x-ray (EDX), 356 Entropy, 238–239 Erosion, 124 Error correction, 232–236, 310–312 Euler characteristic, 128–129, 131, 139 triply periodic minimal surfaces, topology of, 165–170 Ewald construction, 361 Extended x-ray absorption fine structure (EXAFS), 102, 104
F Face-centered cubic (FCC) lattices, 149–153 Factorization into lifting steps, 56–60 Ferromagnetic solutions, 260, 263–264, 278–282 Filter banks See also M-channel filter bank systems analysis, 30–31
420
INDEX
Filter banks (Cont.) analysis, matrix description of, 33–35 analysis-synthesis, 32–33 synthesis, 31–32 synthesis, matrix description of, 35–37 Filter bank theory, applications of, 2, 30 Filter theory, recursive operators and basic linear operators, 13–15 decimation operators, 15–18 interchanging and combining basic operators, 19–22 Foil thickness, determination of, 395–397 Frozen spins solution, 260
G Gallager codes, 234 decoding, 272 encoding, 271 energy density at Nishimori condition, 338–339 Husimi cactus and mean-field theories, 282–287 recursion relations, 339–340 regular, 271 replica symmetric solution, 277–278, 337–338 replica theory, 275–277, 336–337 spinodal noise levels, estimating, 289–291 statistical physics formulation, 273–275 Tanner graphs, 287–288 thermodynamic quantities and typical performance, 278–282 tree-like approximation and thermodynamic limit, 287–289
upper bound for achievable rates, 272–273 variations, 270 Gaussian channel, 240 Generalized distributive law, 245 Generator matrix, 235, 245, 271 Germs, of a model, 144 Gibbs measures, 251 Grain boundaries, characterization of, 391–397 Grains, of a model, 144
H Hadwiger’s completeness theorem, 127–128, 142, 147–148 Hamiltonian, 253–254, 255, 256, 275, 296 Hamming code, 235 Hankel matrices, 9–12, 14 banded, 11–12 HOLZ lines, 380, 405 Hough transform (HT), 377–379, 405 h-step shift, 14 Hurwitz matrices, 3, 9–12 products of, 26–30 transposed, 15, 22 Husimi cactus and mean-field theories, 282–287 Hypercubic lattice, 135–137 Hyperparameters, 251–252
I Image functionals, 122–123 additive, 123, 128–130 normalization of, 149 Incoherent channeling patterns (ICPs), 71–73 Information theory, 233 Integral Gaussian curvature, 132
INDEX
Integral geometry algorithm, 176–177 application to images, 132–135 computer program, use of, 143–144, 178–182 computer tomography images of metal foams, 170–176 convex rings and additive image functionals, 128–130 convex sets, 126–128 digitization errors, reducing, 148 digitized and thresholded images, analysis of, 145–147 disadvantages of, 147–148 image measurements, 122–123 kinematic formulae, 130, 140–141, 182–190 on lattices, 137–141 on lattices, hypercubic, 135–137 Minkowski addition and subtraction, 123–124 Minkowski functionals, 120, 127–128, 142–143 morphology, use of term, 120 parallel sets, 124–126 point patterns, analysis of, 144–145 relation to topology and differential geometry, 131–132 Integral geometry, examples body-centered cubic (BCC) lattices, 149–153 dislocations, 153–157 face-centered cubic (FCC) lattices, 149–153 point sets, random, 157–162 simple cubic (SC) lattices, 149–153 triply periodic minimal surfaces, topology of, 162–170 Integral mean curvature, 132
421
Intermetallics, ALCHEMI results, 81–93
J Junction mixing STM (JM-STM), 197, 198, 205–215
K k-decimated matrix, 26 k-decimated polynomials, 15–16 k-decimated vector, 26 k-decimation operators, 16 k-downsampling operator, 15 k-expander, 14 Kikuchi line method, 69, 369 Kikuchi patterns. See Transmission Kikuchi patterns (TKPs) Kinematic formulae, 130, 140–141 computation of configurational average of Minkowski functionals, 184–188 computation of mean value, 188–190 derivation of, 182–184 Kinematic theory of electron diffraction, 363, 377, 382 Kossel–M¨ollenstedt patterns, 361 Kronecker delta, 272 k-upsampling operator, 14
L Lattice parameters, determination of, 404–407 Lattices, determination of dislocations, 397–403 Lattices, integral geometry and alternative formulation, 137–141 body-centered cubic (BCC), 149–153
422
INDEX
Lattices, integral geometry (Cont.) face-centered cubic (FCC), 149–153 on hypercubic, 135–137 simple cubic (SC), 149–153 Lattices, reciprocal and crystal, 359–361 calculation of reciprocal, 370–371 Laue symmetry, 370 Laurent polynomials, 3 See also Filter theory, recursive operators and block, 23–26 composition of, 13 defined, 7 dual polynomial of, 13 matrix representation of operations on, 12–13 product of, 12–13 Laurent series definitions and properties of, 4–7 reciprocal series, 6 sum and product of, 5 Lifting. See M-band lifting Linear error-correcting codes and decoding problem, 242–243 Local field distribution, 258, 331 Low-density parity-check codes (LDPCs) applications, 250 cascading, 317–324 Gallager, 234, 270–291 how they work, 235–236 MacKay–Neal, 291–316 Sourlas, 252–270
M MacKay–Neal (MN) codes decoding, 292 decoding with probability propagation, 301–302
development of, 291 equilibrium results and decoding performance, 303–310 error correction, 310–312 K = 1 and general L > 1, 307–310 K = 2, 307 K ≥ 3, 303–307 regular versus irregular, 292 replica symmetric free-energy, 344–347 replica theory, 292, 296–300, 341–344 spinodal noise levels, estimating, 312–316 statistical physics formulation, 291–292, 294–296 syndrome bits, distribution of, 340–341 upper bound for achievable rates, 291, 294 Viana–Bray (VB) model, 266, 348–349 Magnetic materials, ALCHEMI results, 93, 95 Magnetic rotation, determination of, 410–411 M-analysis operator, 34 Marginal posterior maximizer (MPM), 243, 272 Markov chain, 267 Mathematical model for a communication system, 237–242 Maximum a posteriori (MAP) estimator, 243 Maximum likelihood hyperparameters, 252 M-band filter bank systems, 2 M-band lifting, 48 algebraic preliminaries, 49–53 algorithms, 56
INDEX
factorization into lifting steps, 56–60 lifting and dual lifting, 53–55 M-channel analysis bank, 33 M-channel filter bank systems alias-free, 40–42 matrix description of, 37–39 perfect reconstruction, 42–44 M-channel synthesis bank, 35 M-decimated matrix, 35 Mean breadth, 126, 132 Mean-field theories, Husimi cactus and, 282–287 Memoryless channels, 239–240 Messages biased, 312–315 unbiased, 315, 316 Metal foams, computer tomography images of, 170–176 Microbeam spot diffraction (MBSD) patterns, 361 Minerals, ALCHEMI results, 77–81 Minkowski addition and subtraction, 123–124 Minkowski functionals, 120, 127–128, 142–143 See also Integral geometry computation of 3D, 172–173 normalized, 130 Misorientation, calculation of, 392–393 Monomial recursive matrices, 9–12 Monte Carlo integration, 264, 269, 278, 303 Markov chain, 267 sampling, 266 Morphological image analysis (MIA), 120–122 See also Integral geometry Morphological image processing (MIP) technique, 121–122
423
Morphology, use of term, 120 Motion invariant, 123 M-synthesis operator, 36 M-transmultiplexer, 45
N Nishimori condition, 252, 274, 295, 338–339 Nishimori temperature, 251, 269, 274, 295 Noble identities, 20–21 Nodal surfaces, periodic, 166–168 Noisy channels, 239–241 Normalized Minkowski functionals, 130 Nuclear magnetic resonance (NMR), 102, 103
O Optimal estimator, 242–243 Ordering tie line (OTL), 67–69, 73 Ordering tie triangle, 71 Orientation determination, automation of dark field imaging by conical scanning, 388–389 Kikuchi patterns, 377–380 spot patterns, 380–388 texture measurements from Debey–Scherrer ring patterns, 390–391 Orientation determination, diffraction patterns and, 369–377 Orientation distribution function (ODF), 390 Orientation imaging microscopy (OIM), 378
424
INDEX
P Parallel sets, 124–126 Paramagnetic solutions, 260, 263 Parisi solution, 265, 266 Parity-check codes. See Low-density parity-check codes Parity-check matrix, 235 Pascal triangle, 8 Perfect reconstruction filter banks, 42–44 property, 42, 48 transmultiplexers, 46–48 Phases, determination of, 407–408 Photogated STM (PG-STM), 197–198, 218–225 Pixel, use of term, 120 Planar ALCHEMI, 75–76 Point patterns, analysis of, 144–145 Point sets, random, 157–162 Polyphase formulation, 2 Probability of bit error, 241 lower bound for, 254–255 Probability of block error, 241 Probability propagation (PP) algorithm, 244–249 decoding with, 266–269, 301–302, 319–323 equations, 334–335 Product Rule, 9, 10–12
Q Quasicrystals, ALCHEMI results, 93, 97 Quermassintegrals, 120, 127–128
R Random energy model (REM), 236, 254, 260 Reconstruction rule, 16–17
Recurrence rule, 8 Recursive matrices, 3 basics of, 4–7 definitions, 7–9 monomial, 9–12 operations on Laurent polynomials, 12–13 Recursive operators. See Filter theory, recursive operators and Reed–Solomon (RS) codes, 234 Replica symmetric (RS) ans¨atz, 257, 258, 277–278, 299 Replica symmetric spin glass, 260, 261, 263 Replicated partition function, 256–257, 261, 276, 297, 298–299 Replica theory Gallager codes and, 275–277, 336–337 MacKay–Neal (MN) codes and, 292, 296–300, 341–344 Sourlas codes and, 256–261 RSB-spin glass free-energy, 261, 265
S Saddle-point equations, 258, 259, 263, 278, 299–300 cascading codes and, 319–323 zero temperature, 332–333 Scalar Toeplitz matrices, 23–26 Scanning force microscopes (SFM), 202 Scanning tunneling microscopy (STM), ultrafast applications, 195–196 development of, 199–205 direct laser coupling, 225–228 distance modulated (DM-STM), 197, 198, 216–218
INDEX
history, 196–198 junction mixing (JM-STM), 197, 198, 205–215 photogated (PG-STM), 197–198, 218–225 Screw dislocations, 153–157 Selected area diffraction patterns (SADPs), 359, 361–365 Shannon’s bound, 262–266 Shannon’s communication system, 237–242 Simple cubic (SC) lattices, 149–153 Simplexes, 138 SK model, 236, 254, 266 Slip systems, determination of, 397–403 Source coding, 233 Source encoding, 238–239 Sourlas codes decoding, 252–254 decoding with probability propagation, 266–269 free-energy, 327–329 J J = J tanh(β N J ) J , 333 local field distribution, 331 lower bound for probability of bit error, 254–255 probability propagation equations, 334–335 replica symmetric solution, 329–331 replica theory, 256–261 Shannon’s bound, 262–266 zero temperature saddle-point equations, 332–333 Sparse generator matrices, 250 Sphere packing problem, 234 Spinodal noise levels, 280 Gallager codes and estimating, 289–291 MacKay–Neal codes and estimating, 312–316
425
Spot patterns application to highly deformed metals, 384–386 determination of positions and intensities, 380–382 intensity-corrected orientation determination from, 382–384 limitations of, 387–388 Starred k-decimated vector, 26 Statistical ALCHEMI, 73–75 Steiner formula, 127 Subband filter theory, applications of, 1 Subband signal processing algebraic results, 22–30 analysis and synthesis filter banks, 30–37 m-band lifting, 48–60 m-channel filter bank systems, 37–44 recursive matrices, overview of, 4–13 recursive operators, 13–22 transmultiplexers, 44–48 Sublattice, 66 predicting occupancies, 100 Sum-product algorithm, 245 Superconducting materials, ALCHEMI results, 93, 96 Superlattice Bragg position, 66 Syndrome vector, 272 Synthesis filter vector, 36 Synthesis operator, 22 description of, 31–32 matrix description of, 35–37
T Tanner graphs, 287–288 Texture measurements from Debey–Scherrer ring patterns, 390–391
426
INDEX
Thresholded images, analysis of, 145–147 Time-domain formulation, 2–3 Toeplitz matrices, 3, 9–12, 14 banded, 12 block/scalar, 23–26 products of Hurwitz matrices, 26–30 Transfer function matrix, 39 Transform domain formulation, 2 Transmission electron microscopy (TEM), computer-aided crystallographic analysis in, 356–358 automation of orientation determination, 377–391 calibration, 408–411 convergent beam electron diffraction patterns (CBEDPs), 359, 361, 365–366 diffraction contrast, 368–369 diffraction patterns, 358–368 diffraction patterns and orientation determination, 369–377 grain boundaries, characterization of, 391–397 lattice parameters, determination of, 404–407
phases, determination of, 407–408 selected area diffraction patterns (SADPs), 359, 361–365 slip systems, determination of, 397–403 transmission Kikuchi patterns (TKPs), 361, 366–368 Transmission Kikuchi patterns (TKPs), 361, 366–368 automatic detection, 377–380 Transmission matrix, 39 Transmultiplexers matrix description of, 45–46 perfect reconstruction, 46–48 Transposed Hurwitz matrix, 15, 22 Triply periodic minimal surfaces (TPMS), topology of, 162–170 Turbo codes, 234, 236 Twin orientation, determination of, 393–395
V Viana–Bray (VB) model, 266, 348–349
X X-ray diffraction, 102–103
