Lecture Notes in Computer Science Edited by G. Goos, J. Hartmanis and J. van Leeuwen
2082
3
Berlin Heidelberg New York Barcelona Hong Kong London Milan Paris Singapore Tokyo
Michael F. Insana Richard M. Leahy (Eds.)
Information Processing in Medical Imaging 17th International Conference, IPMI 2001 Davis, CA, USA, June 18–22, 2001 Proceedings
13
Series Editors Gerhard Goos, Karlsruhe University, Germany Juris Hartmanis, Cornell University, NY, USA Jan van Leeuwen, Utrecht University, The Netherlands Volume Editors Michael F. Insana University of California, Biomedical Engineering One Shields Avenue, Davis, CA 95616, USA E-mail:
[email protected] Richard M. Leahy University of Southern California, Signal and Image Processing Institute 3740 McClintock Avenue, Los Angeles, CA 90089-2564, USA E-mail:
[email protected] Cataloging-in-Publication Data applied for Die Deutsche Bibliothek - CIP-Einheitsaufnahme Information processing in medical imaging : 17th international conference ; proceedings / IPMI 2001, Davis, CA, USA, June 18 - 22, 2001. Michael F. Insana ; Richard M. Leahy (ed.). - Berlin ; Heidelberg ; New York ; Barcelona ; Hong Kong ; London ; Milan ; Paris ; Singapore ; Tokyo : Springer, 2001 (Lecture notes in computer science ; Vol. 2082) ISBN 3-540-42245-5
CR Subject Classification (1998): I.4, I.2.5-6, I.5, J.1, I.3 ISSN 0302-9743 ISBN 3-540-42245-5 Springer-Verlag Berlin Heidelberg New York This work is subject to copyright. All rights are reserved, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, re-use of illustrations, recitation, broadcasting, reproduction on microfilms or in any other way, and storage in data banks. Duplication of this publication or parts thereof is permitted only under the provisions of the German Copyright Law of September 9, 1965, in its current version, and permission for use must always be obtained from Springer-Verlag. Violations are liable for prosecution under the German Copyright Law. Springer-Verlag Berlin Heidelberg New York a member of BertelsmannSpringer Science+Business Media GmbH http://www.springer.de © Springer-Verlag Berlin Heidelberg 2001 Printed in Germany Typesetting: Camera-ready by author, data conversion by Boller Mediendesign Printed on acid-free paper SPIN: 10839299 06/3142 543210
Preface
The 17th biennial International Conference/Workshop on Information Processing in Medical Imaging (IPMI) was held June 18–22, 2001, on the campus of the University of California, Davis. Following the successful meeting in beautiful Visegr´ad, Hungary, this year’s conference summarized important developments in a broad range of topics regarding the acquisition, analysis, and application of information from medical images. Seventy-eight full manuscripts were submitted to the conference; of these, twenty-two were accepted as oral presentations in six sessions of three or four papers each. Thirty-two excellent submissions that could not be accommodated as oral presentations were presented as posters. Manuscripts from oral presentations were limited to 14 pages, whereas those from poster presentations were limited to 7 pages. Every effort was made to maintain those traditional features of IPMI that have made this conference a unique and exciting experience since the first in 1969. First, papers are presented in single-track sessions, followed by discussion that is unbounded with respect to the schedule. Although unlimited discussion ruins carefully planned meal schedules, many participants welcome the rich, detailed descriptions of essential techniques that often emerge from the discussions. For that reason, IPMI is often viewed as a workshop in contrast to the constrained schedules of most conferences. Second, the focus at IPMI has been to encourage the participation of young investigators, loosely described as students, postdocs, and junior faculty under 35 years of age who are presenting at IPMI for the first time. Looking back to our first encounters at IPMI in the 1980’s, we co-chairs remember the challenge and thrill of having our senior colleagues probe deeply into the science and engineering that authors spent so much time advancing and refining. Truly, this format nurtures new talent in a way that encourages the brightest investigators to engage and advance medical image science. Third, the setting and dress has always been casual, which promotes collegiality and an exchange of information unfettered by the usual formalities. This year, the conference was held on the UC Davis campus, where attendees stayed together in the university housing. The causal approach helps organizers keep costs low, thus encouraging young investigator participation. Of course, the tradition of carrying on discussion into the evening over a beer, this year at Cantina del Cabo in Davis, was a pleasant experience for many. We also took Wednesday afternoon off to enjoy tours in the wine country of Northern California and dinner at the elegant Soga’s restaurant. We organizers also assumed the responsibility of looking forward by encouraging new topics, new authors, and new format elements. First, most sessions at this conference opened with a half-hour talk by a senior investigator who introduced the topics. With the diversity of topics, the depth of presentation, and a
VI
Preface
large number of young investigators, the co-chairs thought it would be helpful to experiment with session introductions that provided a high-level review of the topic. Second, we invited a plenary speaker, Sanjiv Gambhir from UCLA, to review the exciting advances in multimodality molecular imaging. Sam’s interests involve the use of mutiple imaging techniques, including X-ray CT, autoradiography, optical-flourescence imaging, and PET, to explore biochemical and physiological processes in animals and humans. These exciting new techniques include the use of molecular probes, e.g., radiolabelled antisense oligonucleotides, for in vivo imaging of gene expression with PET. The future of medical imaging will require those of us developing methodologies to extend our systems and techniques to include the molecular nanoscale, a formidable challenge indeed. Third, we were happy and surprised by many outstanding submissions in the areas of image quality assessment, molecular and diffusion tensor imaging, and fMRI/EEG/MEG approaches. These three of six session topics reflect the organizers’ and program committee’s desire to extend the topics of IPMI beyond its traditional strengths in image analysis and computer vision, while maintaining an emphasis on mathematical approaches. These changes are experimental and may not survive to become part of the IPMI tradition. Nevertheless, we hope the attendees view these experiments as reflections of the sense of adventure that characterizes IPMI’s approach to imaging research. At the time of year we are writing this preface, threats of rolling blackouts loom ominously throughout our state during the summer months. Perhaps the conference staff should be looking into bicycle powered generators to run the LCD projectors and air conditioners. Instead we have limited our preparation to hoping that California can transcend third-world status before June, while we eagerly await the scientific program and hope it can approach the exciting, enriching experiences provided to us by our conference co-chair predecessors.
March 2001
Michael F. Insana Richard M. Leahy
Acknowledgements
The XVIIth IPMI conference was made possible by the efforts of many hardworking individuals and generous organizations. First, the organizers wish to thank the Scientific Program Committee for their critical reviews that determined the content of the program. Considering they were asked to review an average of 10 full manuscripts in December near the holidays, their efforts were truly heroic. We also extend our gratitude to all authors who submitted papers to the conference, and our regrets to those we turned down, often because of time constraints. We gratefully acknowledge the assistance of the Conference and Event Services staff at UC Davis, particularly Teresa Brown who coordinated most aspects of conference logistics. Michael Insana wishes to thank Terry Griffin at UCD who helped organize communications with authors and attendees. Richard Leahy expresses his gratitude to David Shattuck, Karim Jerbi, and Evren Asma at USC for taking time from their research to provide expert assistance in compiling and checking the proceedings. Finally, we express our appreciation of financial support from the following organizations
The Whitaker Foundation The National Institutes of Health Department of Biomedical Engineering, UC Davis Signal and Image Processing Institute, USC Anonymous Friends of Medical Imaging
Francois Erbsmann Prize Winners
1987 10th IPMI, Utrecht, The Netherlands John M. Gauch, Dept. of Computer Science, University of North Carolina, Chapel Hill, NC, USA JM Gauch, WR Oliver, SM Pizer: Multiresolution shape descriptions and their applications in medical imaging. 1989 11th IPMI, Berkeley, CA, USA Arthur F. Gmitro, Dept. of Radiology, University of Arizona, Tucson, AZ, USA AF Gmitro, V Tresp, V Chen, Y Snell, GR Gindi: Video-rate reconstruction of CT and MR images. 1991 12th IPMI, Wye (Kent), UK H. Isil Bozma, Dept. of Electrical Engineering, Yale University, New Haven, CT, USA HI Bozma, JS Duncan: Model-based recognition of multiple deformable objects using a game-theoretic framework. 1993 13th IPMI, Flagstaff, AZ, USA Jeffrey A. Fessler, Division of Nuclear Medicine, University of Michigan, Ann Arbor, MI, USA JA Fessler: Tomographic reconstruction using information-weighted spline smoothing. 1995 14th IPMI, Brest, France Maurits K. Konings, Dept. of Radiology and Nuclear Medicine, University Hospital Utrecht, The Netherlands MK Konings, WPTM Mali, MA Viergever: Design of a robust strategy to measure intravascular electrical impedance. 1997 15th IPMI, Poultney, VT, USA David Atkinson, UMDS, Radiological Sciences, Guy’s Hospital, London, UK D Atkinson, DLG Hill, PNR Stoyle, PE Summers, SF Keevil: An autofocus algorithm for the automatic correction of motion artifacts in MR images. 1999 16th IPMI, Visegr´ ad, Hungary Liana M. Lorigo, Artificial Intelligence Laboratory, Massachusetts Institute of Technology, Cambridge MA, USA LM Lorigo, O Faugeras, WEL Grimson, R Keriven, R Kikinis, C-F Westin: Co-dimension 2 geodesic active contours for MRA segmentation.
Conference Committee
Chairs Michael F. Insana Richard M. Leahy
University of California, Davis, USA University of Southern California, USA
Scientific Committee Christian Barillot INRIA/CNRS, France Harrison H. Barrett University of Arizona USA Yves Bizais Universit´e de Bretagne Occidentale, France Michael Brady Oxford University, UK Gary Christensen University of Iowa, USA Alan Colchester University of Kent, UK D. Louis Collins McGill University, Canada James S. Duncan Yale University, USA Jeffrey A Fessler University of Michigan, USA Guido Gerig University of North Carolina, Chapel Hill, USA Gene Gindi State University of New York, Stony Brook, USA David Hawkes Guy’s Hospital, London, UK Derek Hill Guy’s Hospital, London, UK Nico Karssemejier University Hospital Nijmegen, The Netherlands Frithjof Kruggel Max-Planck-Institute of Cognitive Neuroscience, Germany Attila Kuba Jozsef Attila University, Hungary Nicholas Lange McLean Hospital, Belmont, MA, USA Kyle J. Myers Food and Drug Administration, USA Stephen M. Pizer University of North Carolina, USA Jerry L. Prince Johns Hopkins University, USA Martin Samal Charles University Prague, Czech Republic Milan Sonka University of Iowa, USA Chris Taylor University of Manchester, UK Andrew Todd-Pokropek University College London, UK Max A. Viergever University Hospital Utrecht, The Netherlands
The 1999 IPMI Board
Yves Bizais Harrison Barrett Randy Brill Alan Colchester Stephen Bacharach Frank Deconinck Robert DiPaola James Duncan Michael Goris Attila Kuba Doug Ortendahl Stephen Pizer Andrew Todd-Pokropek Max Viergever
Table of Contents
Objective Assessment of Image Quality On the Difficulty of Detecting Tumors in Mammograms . . . . . . . . . . . . . . . . . Arthur E. Burgess, Francine L. Jacobson, Philip F. Judy
1
Objective Comparison of Quantitative Imaging Modalities Without the Use of a Gold Standard . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 John Hoppin, Matthew Kupinski, George Kastis, Eric Clarkson, Harrison H. Barrett Theory for Estimating Human-Observer Templates in Two-Alternative Forced-Choice Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 Craig K. Abbey, Miguel P. Eckstein
Shape Modeling The Active Elastic Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36 Xenophon Papademetris, E. Turan Onat, Albert J. Sinusas, Donald P. Dione, R. Todd Constable, James S. Duncan A Minimum Description Length Approach to Statistical Shape Modelling . 50 Rhodri H. Davies, Tim F. Cootes, Chris J. Taylor Multi-scale 3-D Deformable Model Segmentation Based on Medial Description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64 Sarang Joshi, Stephen Pizer, P. Thomas Fletcher, Andrew Thall, Gregg Tracton Automatic 3D ASM Construction via Atlas-Based Landmarking and Volumetric Elastic Registration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78 Alejandro F. Frangi, Daniel Rueckert, Julia A. Schnabel, Wiro J. Niessen
Molecular and Diffusion Tensor Imaging A Regularization Scheme for Diffusion Tensor Magnetic Resonance Images Olivier Coulon, Daniel C. Alexander, Simon R. Arridge
92
Distributed Anatomical Brain Connectivity Derived from Diffusion Tensor Imaging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106 Geoffrey J.M. Parker, Claudia A.M. Wheeler-Kingshott, Gareth J. Barker
XII
Table of Contents
Study of Connectivity in the Brain Using the Full Diffusion Tensor from MRI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121 Philipp G. Batchelor, Derek L.G. Hill, Fernando Calamante, David Atkinson
Poster Session I: Registration and Structural Analysis Incorporating Image Processing in a Clinical Decision Support System . . . . 134 Paul Taylor, Eugenio Alberdi, Richard Lee, John Fox, Margarita Sordo, Andrew Todd-Pokropek Automated Estimation of Brain Volume in Multiple Sclerosis with BICCR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141 D. Louis Collins, Johan Montagnat, Alex P. Zijdenbos, Alan C. Evans, Douglas L. Arnold Automatic Image Registration for MR and Ultrasound Cardiac Images . . . 148 Caterina M. Gallippi, Gregg E. Trahey Estimating Sparse Deformation Fields Using Multiscale Bayesian Priors and 3-D Ultrasound . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155 Andrew P. King, Philipp G. Batchelor, Graeme P. Penney, Jane M. Blackall, Derek L.G. Hill, David J. Hawkes Automatic Registration of Mammograms Based on Linear Structures . . . . . 162 Robert Marti, Reyer Zwiggelaar, Caroline Rubin Tracking Brain Deformations in Time-Sequences of 3D US Images . . . . . . . . 169 Xavier Pennec, Pascal Cachier, Nicholas Ayache Robust Multimodal Image Registration Using Local Frequency Representations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 176 Baba C. Vemuri, Jundong Liu, Jos´e L. Marroquin Steps Toward a Stereo-Camera-Guided Biomechanical Model for Brain Shift Compensation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183 ˇ Oskar Skrinjar, Colin Studholme, Arya Nabavi, James Duncan
Poster Session I: Functional Image Analysis Spatiotemporal Analysis of Functional Images Using the Fixed Effect Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 190 Jayasanka Piyaratna, Jagath C. Rajapakse Spatio-temporal Covariance Model for Medical Images Sequences: Application to Functional MRI Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197 Habib Benali, M´elanie P´el´egrini-Issac, Frithjof Kruggel
Table of Contents
XIII
Microvascular Dynamics in the Nailfolds of Scleroderma Patients Studied Using Na-fluorescein Dye . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 204 Philip D. Allen, Chris J. Taylor, Ariane L. Herrick, Marina Anderson, Tonia Moore Time Curve Analysis Techniques for Dynamic Contrast MRI Studies . . . . . 211 Edward V.R. Di Bella, Arkadiusz Sitek Detecting Functionally Coherent Networks in fMRI Data of the Human Brain Using Replicator Dynamics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 218 Gabriele Lohmann, D. Yves von Cramon Smoothness Prior Information in Principal Component Analysis of Dynamic Image Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 225 ˇ ıdl, Miroslav K´ ˇamal, Werner Backfrieder, V´ aclav Sm´ arn´y, Martin S´ Zsolt Szabo Estimation of Baseline Drifts in fMRI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 232 Fran¸cois G. Meyer, Gregory McCarthy Analyzing the Neocortical Fine-Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 239 Frithjof Kruggel, Martina K. Br¨ uckner, Thomas Arendt, Christopher J. Wiggins, D. Yves von Cramon
fMRI/EEG/MEG Motion Correction Algorithms of the Brain Mapping Community Create Spurious Functional Activations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 246 Luis Freire, Jean-Fran¸cois Mangin Estimability of Spatio-temporal Activation in fMRI . . . . . . . . . . . . . . . . . . . . 259 Andre Lehovich, Harrison H. Barrett, Eric W. Clarkson, Arthur F. Gmitro A New Approach to the MEG/EEG Inverse Problem for the Recovery of Cortical Phase-Synchrony . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 272 Olivier David, Line Garnero, Francisco J. Varela Neural Field Dynamics on the Folded Three-Dimensional Cortical Sheet and Its Forward EEG and MEG . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 286 Viktor K. Jirsa, Kelly J. Jantzen, Armin Fuchs, J.A. Scott Kelso
Deformable Registration A Unified Feature Registration Method for Brain Mapping . . . . . . . . . . . . . . 300 Haili Chui, Lawrence Win, Robert Schultz, James Duncan, Anand Rangarajan
XIV
Table of Contents
Cooperation between Local and Global Approaches to Register Brain Images . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 315 Pierre Hellier, Christian Barillot Landmark and Intensity-Based, Consistent Thin-Plate Spline Image Registration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 329 Hans J. Johnson, Gary E. Christensen Validation of Non-rigid Registration Using Finite Element Methods . . . . . . 344 Julia A. Schnabel, Christine Tanner, Andy D. Castellano Smith, Martin O. Leach, Carmel Hayes, Andreas Degenhard, Rodney Hose, Derek L.G. Hill, David J. Hawkes
Poster Session II: Shape Analysis A Linear Time Algorithm for Computing the Euclidean Distance Transform in Arbitrary Dimensions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 358 Calvin R. Maurer, Jr., Vijay Raghavan, Rensheng Qi An Elliptic Operator for Constructing Conformal Metrics in Geometric Deformable Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 365 Christopher Wyatt, Yaorong Ge Using a Linear Diagnostic Function and Non-rigid Registration to Search for Morphological Differences Between Populations: An Example Involving the Male and Female Corpus Callosum . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 372 David J. Pettey, James C. Gee Shape Constrained Deformable Models for 3D Medical Image Segmentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 380 J¨ urgen Weese, Michael Kaus, Christian Lorenz, Steven Lobregt, Roel Truyen, Vladimir Pekar Stenosis Detection Using a New Shape Space for Second Order 3D-Variations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 388 Qingfen Lin, Per-Erik Danielsson Graph-Based Topology Correction for Brain Cortex Segmentation . . . . . . . . 395 Xiao Han, Chenyang Xu, Ulisses Braga-Neto, Jerry L. Prince Intuitive, Localized Analysis of Shape Variability . . . . . . . . . . . . . . . . . . . . . . 402 Paul Yushkevich, Stephen M. Pizer, Sarang Joshi, J.S. Marron A Sequential 3D Thinning Algorithm and Its Medical Applications . . . . . . . 409 K´ alm´ an Pal´ agyi, Erich Sorantin, Emese Balogh, Attila Kuba, Csongor Halmai, Bal´ azs Erd˝ ohelyi, Klaus Hausegger
Table of Contents
XV
Poster Session II: Functional Image Analysis An Adaptive Level Set Method for Medical Image Segmentation . . . . . . . . . 416 Marc Droske, Bernhard Meyer, Martin Rumpf, Carlo Schaller Partial Volume Segmentation of Cerebral MRI Scans with Mixture Model Clustering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 423 Aljaˇz Noe, James C. Gee Nonlinear Edge Preserving Smoothing and Segmentation of 4-D Medical Images via Scale-Space Fingerprint Analysis . . . . . . . . . . . . . . . . . . . 431 Bryan W. Reutter, V. Ralph Algazi, Ronald H. Huesman Spatio-temporal Segmentation of Active Multiple Sclerosis Lesions in Serial MRI Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 438 Daniel Welti, Guido Gerig, Ernst-Wilhelm Rad¨ u, Ludwig Kappos, Gabor Sz´ekely Time-Continuous Segmentation of Cardiac Image Sequences Using Active Appearance Motion Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 446 Boudewijn P.F. Lelieveldt, Steven C. Mitchell, Johan G. Bosch, Rob J. van der Geest, Milan Sonka, Johan H.C. Reiber Feature Enhancement in Low Quality Images with Application to Echocardiography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 453 Djamal Boukerroui, J. Alison Noble, Michael Brady 3D Vascular Segmentation Using MRA Statistics and Velocity Field Information in PC-MRA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 461 Albert C.S. Chung, J. Alison Noble, Paul Summers, Michael Brady Markov Random Field Models for Segmentation of PET Images . . . . . . . . . 468 Jun L. Chen, Steve R. Gunn, Mark S. Nixon, Roger N. Gunn
Analysis of Brain Structure Statistical Study on Cortical Sulci of Human Brains . . . . . . . . . . . . . . . . . . . . 475 Xiaodong Tao, Xiao Han, Maryam E. Rettmann, Jerry L. Prince, Christos Davatzikos Detecting Disease-Specific Patterns of Brain Structure Using Cortical Pattern Matching and a Population-Based Probabilistic Brain Atlas . . . . . . 488 Paul M. Thompson, Michael S. Mega, Christine Vidal, Judith L. Rapoport, Arthur W. Toga Medial Models Incorporating Object Variability for 3D Shape Analysis . . . 502 Martin Styner, Guido Gerig
XVI
Table of Contents
Deformation Analysis for Shape Based Classification . . . . . . . . . . . . . . . . . . . 517 Polina Golland, W. Eric L. Grimson, Martha E. Shenton, Ron Kikinis
Subject Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 531 Author Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 535
On the Difficulty of Detecting Tumors in Mammograms Arthur E. Burgess, Francine L. Jacobson, and Philip F. Judy Radiology Department, Brigham and Women’s Hospital, 75 Francis St. Harvard Medical School, Boston MA 02115
[email protected],
[email protected],
[email protected]
Abstract. We did human observer experiments using a hybrid image technique to determine the variation of tumor contrast thresholds for detection as a function of tumor sizes. This was done with both mammographic backgrounds and filtered noise with the same power spectra. We obtained the very surprising result that contrast had to be increased as lesion size increased to maintain contrast detectability. All previous investigations with white noise, radiographic and CT imaging system noise have shown the opposite effect. We compared human results to predictions of a number of observer models and found fairly good qualitative agreement. However we found that human performance was better than what would be expected if mammographic structure was assumed to be pure noise. This disagreement can be accounted for by using a simple scaling correction factor.
1
Introduction
Detectability of abnormalities in medical images is determined by a number of factors. Examples are: spatial resolution, image noise, lesion contrast and patient structure. Image display and visual system capabilities are also important for human observers The consequences of these effects can be summarized by the contrast-detail (CD) diagram, a plot of the lesion contrast needed to reach a defined detection accuracy as a function of lesion size. The same CD diagram form has been consistently found in previous work using phantoms and artificial signals in image noise. The contrast threshold decreases steadily as signal size increases. There has been very little formal study of the effect of patient structure on lesion detection [1], [2], [3]. Bochud et al. [4],[5],[6] did experiments designed to determine whether spatial variations in mammograms due to normal patient anatomical structure can be considered to be a form of image noise. They concluded that the effects of structure backgrounds are not fully described by their average power spectrum and that human observers are able to use some information contained in the phase spectrum. They estimated that the effect of anatomical structure variations was three times as important as imaging system noise for microcalcifications and 30 to 60 times as important for an 8 mm simulated module. M.F. Insana, R.M. Leahy (Eds.): IPMI 2001, LNCS 2082, pp. 1–11, 2001. c Springer-Verlag Berlin Heidelberg 2001
2
Arthur E. Burgess, Francine L. Jacobson, and Philip F. Judy
Zheng et al. [7], reported mammogram power spectra of the form P (f ) = K/f 3 . We used this spectrum with the frequency domain integral in equation (2) below to evaluate lesion detectability indices and found the surprising prediction of a positive CD slope - that contrast thresholds would increase as lesion size increases. However the use of the integral form of equation (2) was based on the dubious assumption that second order statistics of mammograms are stationary. The goals of our work were to (1) measure human CD diagrams for lesion detection in mammographic backgrounds and 1/f 3 filtered noise backgrounds with matched average second order statistics, (2) evaluate performance of two models that do not require the assumption of stationarity and (3) compare human performance with model predictions. We will describe the models, experimental methods and results. We will show that the prediction of positive CD diagram slopes is robust. It occurs for both human and model observers with both mammogram and ’1/f 3 ’ filtered noise backgrounds.
2
Models
Many experiments (discussed in references [8] and [9]) have shown that human detection and discrimination performance can be described by models based on matched filters. The models can be described in several ways, using continuous images, g(x,y), signals, s(x,y), noise, n(x,y), and statistically-defined backgrounds, b(x,y), for example. For discrete images, (x,y) are pixel row and column addresses. Digital images and components can be described in column vector notation with the Np pixel values in lexicographic order [10]. In this notation, the image and component vectors are g, s, n, and b. The total image is then described by the two alternative summation forms g(x, y) = s(x, y) + n(x, y) + b(x, y) and g = s + n + b
(1)
When noise and backgrounds have stationary statistics observer performance modeling is most convenient in the frequency domain. Then the signal is described by its Fourier transform, S(u,v) and stochastic components are described by power spectra, Pn (u, v) and Pb (u, v). The discrete models can be evaluated [11], [10] in the spatial domain with noise and background fluctuations described by covariance matrices, Kn and Kb - 2D arrays with Np rows and Np columns. This approach has the advantage that the stationarity assumption can be avoided. Several models will be presented for the simple signal known exactly (SKE) detection task, with the signal 2D profile and potential locations precisely defined and known to the observer. 2.1
Ideal Observer
The ideal observer [12], [13] uses Bayes’ theorem to combine a priori information (about the signal profile and possible locations for example) with new data optimally extracted from the image. The optimum strategy depends on task
On the Difficulty of Detecting Tumors in Mammograms
3
details. For the SKE detection task, the ideal observer uses a prewhitening (PW) matched filter. The detectability index, d’, equations for the SKE case in the frequency and spatial domains are given by ∞ ∞ |S(u, v)|2 (d )2 = = st (Kn + Kb )−1 s. (2) [P (u, v) + P (u, v)]dudv n b −∞ −∞ The Fourier domain integral equation requires the assumption of stationarity for the noise and background second order statistics. The spatial domain vectormatrix form of the equation does not require stationarity. 2.2
Channelized Observer Models
The ideal observer model is impractical to use for images with nonstationary statistics because the covariance matrix size is very large - about 4x109 elements for an ensemble of 256x256 images. Fiete et al. [14] , proposed an optimum linear observer model for cases where ideal observer performance calculation is impractical - which they referred to as the Hotelling model. We will use the term Fisher-Hotelling (FH), since it is similar to Fisher linear discriminant analysis. The covariance matrix size can be reduced dramatically by describing the image data near the potential signal locations by coefficients of a set of smooth basis functions centered on the potential locations rather than by pixel values. One example is the difference of Gaussians (DOG) channel model [15]. Barrett et al.[16] suggested using Laguerre-Gauss (LG) basis functions. For isotropic signals, a radial channel count, Nc , of 6 to 8 is adequate [17]. Each channel is described by a basis vector, tc , and the set of channels is described by a matrix, T, whose columns are the individual basis vectors. The response to the signal is rs = Tt s. The covariance matrix of the channel responses to noise and backgrounds, Kc , has dimension (Nc )2 . We will refer to the two channelized models as FHDOG and FHLG . The detectability equation for this channelized FH model class is (d )2chan = rts K−1 c rs .
3 3.1
(3)
Materials and Methods Observer Experiments
The human observer experiments were designed to comply with the requirements of the SKE detection task and we attempted to optimize experimental conditions to maximize human performance. We used the two-alternative forced choice (2AFC) method and hybrid digital images, with signals added to both digitized mammograms and filtered noise images. The filtered noise had a spectrum P (f ) = K/f 3 , where f is radial frequency. An example display is shown in figure 1. During each trial, two randomly selected backgrounds were displayed side-byside with one containing the signal. The observers selected the side they believed to contain the signal. Three experienced observers took part, two physicists and a radiologist (the authors).
4
Arthur E. Burgess, Francine L. Jacobson, and Philip F. Judy
Fig. 1. Example display for 2AFC experiments. A reference copy of the lesion is shown above the mammographic backgrounds. The two possible lesion locations are surrounded by circle cues [19], which are exaggerated here for publication.
The background regions (each 61x61 mm with 0.12 mm pixels on the mammogram and 0.29 mm pixels on the monitor) were selected from 210 digitized normal craniocaudal mammograms. A log exposure (log E) amplitude scale was used. Backgrounds were confined to the constant thickness region of the breast to eliminate the confounding effect of the large, systematic brightness variations at the periphery. The signals used in the experiments were a simulated nodule and 4 realistic breast lesions (1 fibroadenoma and 3 ductal carcinomas). The nodule equation was s(r) = rect(ρ/2)(1 − ρ2 )1.5 , where ρ is a normalized radial distance (r/R) and R is the nodule radius. Tumor images were extracted from digitized specimen radiographs [23]. The tumors (original size 8 to 18 mm) were rescaled to fit in a common array size (256x256). During experiments the lesions were minified to sizes in the range from 4 to 128 pizels (corresponding to the range 0.5 → 15.6 mm on the mammograms). Images were displayed on a Clinton grayscale monitor with a maximum luminance of 75 cd/m2 and 1024(V)x1280(H) pixels. Each observer did 256 trials for each experimental condition in blocks of 128 trials. The 4 tumor profiles led to 4 slightly different CD diagrams. We used the average contrast threshold (across all sizes) for each lesion to adjust the 4 diagrams up or down to a common relative contrast scale. Additional experiments were done using simulated noise. We used one tumor to determine the CD diagram with white noise with a range of 256 gray levels, a mean of 128 and a pixel standard deviation of 25.6. The purpose was to determine whether the typical CD diagram, with threshold contrast decreasing as signal size increased, would be obtained under our display conditions. We also did experiments using the simulated nodule and filtered noise with a spectrum
On the Difficulty of Detecting Tumors in Mammograms
5
matched to the power-law exponent (3.0) for the ensemble average estimated for the set of mammographic backgrounds. The purpose was to allow comparison of human and model results for an isotropic signal and stationary noise with known statistics. 3.2
Statistics of Mammographic Images
We did measurements in 213 square ROIs (61x61 mm). Two spectrum calculation methods were used. The first was the discrete Fourier transform method [21] with a radial Hanning window. The spectrum was averaged over angle and gave radial slice frequency dependence, P (f ) = K/f 3 below 1 c/mm. We also measured the radial averages of individual periodograms. The exponent distribution had a mean of 2.8 and a standard deviation of 0.35. Maximum entropy method (MEM) spectral estimates [22] were done using row and column projections to determine 1D spectral slices. The average exponents were 2.5 (std. dev. 0.3) and 3.0 (std. dev. 0.5) in the fx and fy directions respectively. We also used a spatial method [26] to evaluate second order statistics. Pixel variance was measured over a range of circle diameters, centered on each of the ROIs and the ensemble averages of variance as a function of size were determined. For power-law noise with an exponent of 3, a plot of log(ensemble average variance) versus log(diameter) should give a slope of one. We obtained a value of 0.99. Pixel variance results for the mammographic background set and the matching filtered noise backgrounds are shown in figure 4A. The covariance matrices for the FHDOG and FHLG models were determined with no signal present. The FHDOG model had 7 channel filters based on a viewing distance of 75 cm from the monitor and center frequency separations of one octave. The FHLG model had 6 basis functions and a free spatial scaling parameter that was adjusted for each nodule radius to maximize d values. Response vectors for each image, rg , were obtained by cross-correlating basis vectors with the image data, centered on the ROIs. Covariance matrices were calculated using the ensemble expectation value, < . . . >, formulation Kc = (rg − rg )(rg − rg )t .
4
(4)
Results
The CD diagram data for human detection of the 4 extracted tumors in mammographic structure and one tumor in white noise are shown in figure 2. The upper data are for 4 different lesions in mammographic backgrounds (in log E units). Contrast thresholds increase for sizes greater than 1 mm with a positive CD diagram slope of 0.3. The lower data are for one lesion in white noise with an interpolated curve through the data as a guide to the eye (it has no theoretical significance). The results are averages for 3 observers with 256 trials for each observer per datum. Standard errors of the estimates for the data are all about 5% of mean values.
6
Arthur E. Burgess, Francine L. Jacobson, and Philip F. Judy
amp (d’=2) [log_E units]
0.05
0.02
mammography background
0.01
0.005
white noise
0.002 0.001 0.5
1
2
5
Lesion size (mm)
10
20
Fig. 2. Contrast thresholds for detection as a function of lesion size.
We compared human nodule detection results with predictions of 3 observer models. Prewhitening (PW) matched filter observer model performance was calculated using numerical integration of the frequency domain version of equation (2). This model is ideal for stationary noise, as is the case in figure 3B. It is a nonideal approximation for mammograms (in figure 3A), where the stationarity assumption is not valid. For mammograms we used the mean parameters (exponent = 2.83) of single image periodograms averaged over angle. FH model performance, which is not dependent on the stationarity assumption, was evaluated using equation (3). The human and model observer nodule detection results for mammographic backgrounds are shown in figure 3A. The models give qualitatively fair agreement to human results. However, the models are incomplete since human internal noise was not included. With human induced internal noise [18] included, the model thresholds would be about 40% higher (this will be discussed below). The best fit regression line to human results has a slope of 0.30. The slopes are 0.40, 0.46 and 0.40 for the PW, FHDOG and FHLG models respectively. Note that humans perform better than FH models at large nodule size. Results for filtered noise backgrounds are shown in figure 3B. All model observers have better performance than humans. The best fit regression line to human results has a slope of 0.44. The regression line slopes are 0.50, 0.51 and 0.50 for the ideal (PW), FHDOG and FHLG models respectively. Since the filtered noise is known to be stationary, the performance of the PW model is ideal. Human efficiency ranges from 24 to 40%. Observer efficiency, η, for a given task is defined using η = (dO /dI )2 , where the subscripts indicate the observer being tested and the ideal observer. Typical efficiencies for humans have been found in to be in the 30 to 60% range for a variety of simple tasks. To a first
On the Difficulty of Detecting Tumors in Mammograms
7
approximation, human inefficiency is due to 2 types of internal noise - static and induced [18]. For large external noise levels, static internal noise can be neglected and induced internal noise can be modeled by scaling the image noise and background power spectra (or covariance matrices) by a factor of (1+ϕ), where the value of ϕ is selected (typically 0.3 to 1) to provide a fit to human results. 0.1
0.1
human avg Ideal FH_DOG FH_LG
human avg
PW
amp (d’=2)
amp (d’=2)
FH_DOG
FH_LG
A 0.01 1
diameter [mm] (A)
10
B 0.01 1
diameter [mm]
10
(B)
Fig. 3. The CD diagrams for simulated nodule detection in (A) mammographic backgrounds and (B) matching power-law noise. Error bars are about the size of the symbols. PW matched filter observer model performance for mammograms is not ideal because of nonstationary statistics
Figure 4A shows pixel variance measurements for the mammogram image set and the matching filtered noise to demonstrate the close agreement in second order statistics. There was equally good agreement between radial averages of ensemble power spectra. The human and FHLG model results for the two types of backgrounds are replotted in figure 4B. This illustrates the salient point that human thresholds for 2AFC detection are very different while those for the FHLG are quite similar for the two sets of images that we tried to match on the basis of second order statistics.
5
Discussion
In most previous experimental CD diagram measurements, thresholds decreased as signal size increased. We obtained the same result in our white noise control experiment. Our most important result is that CD diagrams for mammograms are completely different. We found that thresholds increased as lesion size increased for lesions larger than 1 mm with a positive slope of 0.3 on a log-log
8
Arthur E. Burgess, Francine L. Jacobson, and Philip F. Judy 10
-2
0.1 human, mammo human, noise FH_LG, mammo FH_LG, noise
10
amp (d’=2)
pixel variance
noise mammograms
-3
slope = 0.99
A 10
-4
1
10
ROI diameter [mm] (A)
B 0.01 1
diameter [mm] (B)
10
Fig. 4. (A) The pixel variance measurement results (log E 2 units) for mammogram and filtered noise backgrounds with matched power spectra. (B) The CD diagrams for humans and the FHLG model for the two types of statistically matched backgrounds.
plot. We obtained similar results for ’1/f 3 ’ filtered noise, confirming that the positive CD diagram slope is due to the power-law statistics of mammograms. The threshold increase below 1 mm (the size range for microcalcifications) for mammograms is due to imaging system noise dominance in the spectrum at high spatial frequencies. The most important point about model observer results is that they all give similar CD diagram slopes and differ mainly in absolute threshold values. This suggests that the question of nonstationarity of mammogram statistics is not a major issue. There is good agreement between the FH model results for the two types of basis functions (channels), LG and DOG. This is consistent with the results of Abbey [24], who found that model performance was not particularly sensitive to basis function selection. It has been found that a number of other models [25] also give similar CD diagram slopes. There is fair agreement between human and model results for simulated nodule detection in the mammographic backgrounds. However, if induced internal noise was included in the models, thresholds would be about 40% higher than human results. By contrast, the human results with power-law noise would be in good agreement with the models if realistic induced internal noise values were included. The human efficiencies of 24 to 40% for detection in filtered noise are consistent with results from previous experiments with statistically defined backgrounds. As figure 4B demonstrates, the FHLG model results are very similar for the two types of backgrounds, whereas human results are not. This suggests that for observer models, mammographic backgrounds can be considered to be pure random noise (i.e.
On the Difficulty of Detecting Tumors in Mammograms
9
models cannot use anatomical information) while humans can make some use of anatomical information in 2AFC experiments. This difference is not a serious problem for modeling human performance because it can be accounted for by a simple scaling process. There are also differences in slopes of CD diagrams to be accounted for: between humans and models and between the two types of backgrounds. We suspect that the cause of the difference for backgrounds may be that power-law exponents vary between 2 and 4 in the set of mammographic backgrounds. This point requires further study. Our mammographic CD diagram results were based on a collection of mammographic regions with an average power law exponent of about 3. It has been shown [26] using the frequency domain integral form of equation (2), that for scaled signals and stationary noise, the CD diagram slope, m, is related to the power-law noise exponent β, by a linear relationship with m = 0.5(β − 2). This CD slope equation is subject to the constraint that the signal has the same normalized 2D profile as its size changes and that signal energy as a function of frequency decreases sufficiently rapidly that the integral converges. Spectral analysis of individual mammogram regions gave a range of exponents from 2 to 4 and MEM spectral analysis of smaller regions showed that the exponent varies within a mammogram. This has interesting implications for lesion detectability. Consider the consequences of the CD diagram slope equation for detection of a growing tumor with thickness parallel to the x-ray beam proportional to diameter perpendicular to the beam. As the tumor grows, its projected contrast will be determined by the local difference in x-ray attenuation and its thickness, so contrast will increase linearly with diameter as long as the composition of the tumor and surrounding tissue do not change. The tumor becomes detectable, at the selected accuracy criterion, when its trajectory crosses the appropriate CD threshold line. If the tumor is in a region of low exponent, detection probability will change rapidly with size. If the exponent is large, detection probability will change slowly with size. If the spectra, P (f ) = K/f β , have similar values of K, then the lesion will be detectable at smaller sizes for mammoigram regions with smaller power-law exponents. X-ray mammography is the primary method of detecting breast cancer. However, lesions are often extremely difficult to detect in the complex and highly variable normal parenchymal patterns. Our work was designed to develop a quantitative understanding of the statistical properties of images of this breast structure and its effect on detectability of realistic lesions. Our finding that more lesion contrast is needed as the size of the lesion increases helps explain why large lesions can be missed despite careful search of a mammogram [27]. The theoretical prediction that lesion detectability will be dependent on the local statistical properties of patient structure is also important. The experiments described here were done using an artificial 2AFC task with high image contrast designed to allow comparison of human observer results with theoretical model predictions. We have obtained similar results in experiments involving search for the lesion [28]. We recognize that our investigations must be extended to more clinically realistic decision tasks. The importance of this work may increase in the future,
10
Arthur E. Burgess, Francine L. Jacobson, and Philip F. Judy
when mammograms are viewed on CRTs for primary interpretation. The novel CD diagram results for lesion detection in mammograms and our finding that human results can be described by observer models suggest that it may be possible to use these models to develop image processing algorithms that will help increase the accuracy of digital mammogram interpretation.
Acknowledgements Larry Clarke and Maria Kallergi provided the mammograms. Jack Beutel digitized the specimen radiographs and provided H&D curve data. We also thank Craig Abbey, Dev Chakraborty, Kyle Myers and Robert Wagner for very helpful discussions. This research was supported by grant R01-CA58302 from the National Cancer Institute.
References 1. Revesz, G., Kundel, H.L., Graber, M.A.: The influence of structured noise on detection of radiologic abnormalities. Invest. Radiol. 9 (1974) 479–486 2. Kundel, H.L., Nodine, C.F., Thickman, D. Carmody, D., et al.: Nodule detection with and without a chest image. Invest. Radiol. 20 (1985) 94-99 3. Judy, P.F., Swensson, R.G., Nawfel, R.D., Chan K.H.: Contast detail curves for liver CT Med. Phys. 19 (1992) 1167–1174 4. Bochud, F.O., Verdun, F.R., Valley, J.F., Hessler C., et al.: The importance of anatomical noise in mammography Proc. SPIE 3036 (1997) 74–80 5. Bochud, F.O., Abbey, C.K., Eckstein, M.P.: Further inverstigation of the effect of phase spectrum on visual detection in structured backgrounds Proc. SPIE 3663 (1999) 273–281 6. Bochud, F.O., Valley, J.F., Verdun F.R., Hessler C., et al.: Estimate of the noisy component of anatomical backgrounds Med. Phys. 26 (1999) 1365–1370 7. Zheng, B., Chang, Y.-H., Gur, D.: Adaptive computer-aided diagnosis scheme of digitized mammograms. Acad. Radiol. 3 (1996) 806–814 8. Burgess, A.E.: High level visual decision efficiencies, In Blakemore, C. (ed.) Vision: Coding and Efficiency. Cambridge Univ. Press,: London (1990) 431–440 9. Barrett, H.H., Yao, J., Rolland, J.P., Myers, K.J.: Model observers for assessment of image quality. Proc. Nat. Acad. Sci. USA 90 (1993) 9758–9765 10. Abbey, C.K., Bochud, F.O.: Modeling visual signal detection tasks in correlated image noise with linear observer models. In Beutel, J., Kundel, H., van Metter, R.L. (eds.) Handbook Of Medical Imaging: Physics and Psychophysics. SPIE Press,: Bellingham (2000) 629–654 11. Eckstein, M.P., Abbey, C.K., Bochud, F.O.: A practical guide to model observers for visual detection in synthetic and natural noisy images. In Beutel, J., Kundel, H.L., van Metter, R.L. (eds.) Handbook of Medical Imaging. SPIE Press,: Bellingham (2000) 593–628 12. Wagner, R.F., Brown, D.G.: Unified SNR analysis of medical imaging systems. Phys. Med. Biol. 30 (1985) 489–518 13. Myers, K.J.: Ideal observer models of visual signal detection. In Beutel, J., Kundel, H., van Metter, R.L. (eds.) Handbook Of Medical Imaging: physics and Psychophysics. SPIE Press,: Bellingham (2000) 558–592
On the Difficulty of Detecting Tumors in Mammograms
11
14. Fiete, R.D., Barrett, H.H., Smith, W.E., Myers, K.J.: Hotelling trace criterion and its correlation with human-observer performance. J. Opt. Soc. Am. A4 (1987) 945–953 15. Wilson, H., Bergen, J.: A four-mechanism model for threshold spatial vision. Vision Res. 19 (1979) 19–32 16. Barrett, H.H., Abbey, C.K., Gallas, B., Eckstein, M.P.: Stabilized estimates of Hotelling observer detection performance in patient structured noise. Proc. SPIE 3340 (1998) 27–43 17. Burgess, A.E., Li, X., Abbey, C.K.: Visual signal detectability with two noise components: anomalous masking effects. J. Opt. Soc. Am. A14 (1997) 2420–2442 18. Burgess, A.E., Colborne, B.: Visual signal detection IV: Observer inconsistency. J. Opt. Soc. Am. A5 (1988) 617–627 19. Kundel, H.L., Nodine, C.F., Toto, L., Lauver, S.: A circle cue enhances detection of simulated masses on mammographic backgrounds. Proc. SPIE 3032 (1997) 81–84 20. Burgess, A.E., Chakraborty, S.: Producing lesions for hybrid images: extracted tumours and simulated microcalcifications. Proc. SPIE 3663 (1999) 316–322 21. Bendat, J.S., Piersol, A.G.: Random Data: analysis and measurement procedures. John Wiley & Sons, New York (1986) 22. Press, W.H., Flannery, B.P., Teukolsky, S.A., Vettering, W.T.: Numerical Recipes in Fortran. Second Edition. Cambridge Univ. Press, (1992) 23. Burgess, A.E.: Mammographic structure: Data preparation and spatial statistics analysis. Proc. SPIE 3661 (1999) 642–653 24. C.K. Abbey: Assessment of reconstructed images, Ph.D. Dissertation, Univ. of Arizona, 1998. 25. Burgess, A.E.: Evaluation of detection model performance in power-law noise. Proc. SPIE 4324 (2001) , (in press) 26. Burgess, A.E., Jacobson, F.L., Judy, P.F.: On the detection of lesions in mammographic structure. Proc. SPIE 3663 (1999) 304–315 27. Rosenberg, R.D., Hunt, W.C., Williamson, M.R., et al.: Effects of age, breast density, ethnicity, and estrogen replacement therapy on screening mammographic sensitivity and cancer stage at diagnosis: review of 183,134 screening mammograms in Albuquerque, New Mexico, Radiology 209, (1998), 511–518 28. Burgess, A.E., Jacobson, F.L., Judy, P.F.: Breast parenchymal patterns: Human observer lesion detection experiments. Med. Phys. (in press)
Objective Comparison of Quantitative Imaging Modalities Without the Use of a Gold Standard John Hoppin1,4 , Matthew Kupinski2,4 , George Kastis3,4 , Eric Clarkson2,3,4, and Harrison H. Barrett1,2,3,4 1
Program in Applied Mathematics, University of Arizona 2 Department of Radiology, University of Arizona 3 Department of Optical Sciences, University of Arizona 4 Center for Gamma Ray Imaging, University of Arizona
Abstract. Imaging is often used for the purpose of estimating the value of some parameter of interest. For example, a cardiologist may measure the ejection fraction (EF) of the heart in order to know how much blood is being pumped out of the heart on each stroke. In clinical practice, however, it is difficult to evaluate an estimation method because the gold standard is not known, e.g., a cardiologist does not know the true EF of a patient. Thus, researchers have often evaluated an estimation method by plotting its results against the results of another (more accepted) estimation method, which amounts to using one set of estimates as the pseudogold standard. In this paper, we present a maximum likelihood approach for evaluating and comparing different estimation methods without the use of a gold standard with specific emphasis on the problem of evaluating EF estimation methods. Results of numerous simulation studies will be presented and indicate that the method can precisely and accurately estimate the parameters of a regression line without a gold standard, i.e., without the x-axis.
1
Introduction
There are many approaches in the literature to assessing image quality, but there is an emerging consensus in medical imaging that any rigorous approach must specify the information desired from the image (the task) and how that information will be extracted (the observer). Broadly, tasks may be divided into classification and estimation, and the observer can be either a human or a computer algorithm. In medical applications, a classification task is to make a diagnosis, perhaps to determine the presence of a tumor or other lesion. This task is usually performed by a human observer, and task performance can be assessed by psychophysical studies and ROC (receiver operating characteristic) analysis. Scalar figures of merit such as a detectability index or area under the ROC curve can then be used to compare imaging systems. Often, however, the task is not directly a diagnosis but rather an estimation of some quantitative parameter from which a diagnosis can later be derived. An M.F. Insana, R.M. Leahy (Eds.): IPMI 2001, LNCS 2082, pp. 12–23, 2001. c Springer-Verlag Berlin Heidelberg 2001
Objective Comparison of Quantitative Imaging Modalities
13
example is the estimation of cardiac parameters such as blood flow, ventricular volume or ejection fraction (EF). For such tasks, the observer is usually a computer algorithm, though often one with human intervention, for example defining regions of interest. Task performance can be expressed in terms of the bias and variance of the estimate, perhaps combined into a mean-square error as a scalar figure of merit. For both classification and estimation tasks, a major difficulty in objective assessment is lack of a believable standard for the true state of the patient. In ROC analysis for a tumor-detection task, we need to know if the tumor is really present, and for estimation of ejection fraction we need to know the actual value for each patient. In common parlance, we need a gold standard, but it is rare that we have one with real clinical images. For classification tasks, biopsy and histological analysis are usually accepted as gold standards, but even when a pathology report is available, it is subject to error; the biopsy can give information on false-positive fraction but if a lesion is not detected on a particular study and hence not biopsied, its contribution to the false-negative fraction will remain unknown. Similarly, for cardiac studies, ventriculography or ultrasound might be taken as the gold standard for estimation of EF, and nuclear medicine or dynamic MRI might then be compared to the supposed standard. A very common graphical device is to plot a regression line of EF’s derived from the system under study to ones derived from the standard and to report the slope, intercept and correlation coefficient for this regression. Even a cursory inspection of papers in this genre reveals major inconsistencies. In reality, no present modality can lay claim to the status of gold standard for quantitative cardiac studies. Indeed, if there were such a modality, there would be little point in trying to develop new modalities for this task. Because of the lack of a convincing gold standard for either classification or estimation tasks, simulation studies are often substituted for clinical studies, but there is always a concern with how realistic the simulations are. Researchers who seek to improve the performance of medical imaging systems must ultimately demonstrate success on real patients. A breakthrough on the gold-standard problem was the 1990 paper by Henkelman, Kay and Bronskill on ROC analysis without knowing the true diagnosis [1]. They showed, quite surprisingly, that ROC parameters could be estimated by using two or more diagnostic tests, neither of which was accepted as the gold standard, on the same patients. Recent work by Beiden, Campbell, Meier and Wagner has clarified the statistical basis for this approach and studied its errors as a function of number of patients and modalities as well as the true ROC parameters [2]. The goal of this paper is to examine the corresponding problem for estimation tasks. For definiteness, we cast the problem in terms of estimation of cardiac ejection fraction, and we pose the following question: If a group of patients of unknown state of cardiac health is imaged by two or more modalities, and an estimate of EF is extracted for each patient for each modality, can we estimate
14
John Hoppin et al.
the bias and variance of the estimates from each modality without regarding any modality as intrinsically better than any other. Stated differently, can we plot a regression line of estimated EF vs. true EF without knowing the truth?
2
Approach
We begin with the assumption that there exists a linear relationship between the true EF and its estimated value. We will describe this relationship for a given modality m and a patient p using a regression line with a slope am , intercept bm , and noise term m . We represent the true EF for a given patient with Θp and an estimate of the EF made using modality m with θpm . The linear model is thus represented by θpm = am Θp + bm + m .
(1)
We make the following assumptions: 1. Θp does not vary for a given patient across modalities and is statistically independent from patient to patient. 2. The parameters am and bm are characteristic of the modality and independent of the patient. 3. The error terms, m , are statistically independent and normally distributed 2 . with zero mean and variance σm We write the probability density function for the noise m as 1 1 2 pr({m }) = exp − 2 m , 2 2σm 2πσm m=1 M
(2)
using assumption number 3 above, where M is the total number of imaging modalities. Using equ. 1, we rewrite equ. 2 as the probability of the estimated EF’s for multiple modalities and a specific patient given the linear model parameters and the true EF as M 1 1 2 }, Θp ) = exp − 2 (θpm − am Θp − bm )2 . pr({θpm }p |{am , bm , σm 2 2σm 2πσm m=1 (3) The notation {θpm }p represents the estimated ejection fractions for a given patient p over M modalities. Using the following property of conditional probability pr(x1 , x2 ) = pr(x1 |x2 )pr(x2 ), as well as the marginal probability law, pr(x1 ) = dx2 pr(x1 , x2 ),
(4)
(5)
Objective Comparison of Quantitative Imaging Modalities
15
we write the probability of the estimated EF for a specific patient across all modalities given the linear model parameters as 2 pr({θpm }p |{am , bm , σm }) = M 1 dΘp pr(Θp )S exp − 2 (θpm − am Θp − bm )2 , 2σm m=1
(6)
where S=
M
1 . 2 2πσ m m=1
(7)
From assumption number 1 above, the likelihood of the linear model parameters can be expressed as M P 1 2 L= (− 2 (θpm − am Θp − bm ) ) , (8) S dΘp pr(Θp ) exp 2σm p=1 m=1 where P is the total number of patients. Upon taking the log and rewriting products as sums we obtain, λ = ln(L) = P ln(S)+ M P 1 ln dΘp pr(Θp ) exp (− 2 (θpm − am Θp − bm )2 ) . 2σm p=1 m=1
(9)
It is this scalar λ, the log-likelihood, which we seek to maximize to obtain our 2 estimates of am , bm , and σm . These estimates will be maximum likelihood estimates for our parameters. Although pr(Θp ) may appear to be a prior term, we are not using a maximum a posteriori approach; we are simply marginalizing over the unknown parameter Θp . Thus we have derived an expression for the log-likelihood of the model parameters which does not require knowledge of the true EF Θp , i.e. without the use of a gold standard. This is analogous to curve fitting lines without the use of the x-axis. Although the expression for the log-likelihood in equ. 9 does not require the true EF Θp , it does require some knowledge of their distribution pr(Θp ). We will refer to this distribution as the assumed distribution (pra (Θp )) of the EFs. In this paper we will investigate the effect different choices of the assumed distributions have on estimating the linear model parameters. We first sample parameters from a true distribution (prt (Θp )) and generate different estimated EF’s for the different modalities by linearly mapping these values using known am ’s and bm ’s, then add normal noise to these values with known σm ’s. These EF estimates form the values θpm , which will be used in the process of determining the estimates of the linear model parameters by optimizing equ. 9. We will look at cases in which the assumed and true distributions match, as well as cases in which they do not match.
16
John Hoppin et al.
For our experiments we will investigate beta distributions and truncated normal distributions as our choices for both the assumed and true distributions. The beta distribution is limited to the interval [0,1] with probability density function given by pr(θ) =
θν−1 (1 − θ)ω−1 , B(ν, ω)
(10)
where B(ν,ω) is a normalizing constant. The truncated normal distribution is given by 1 pr(θ) = A(µ, σ) exp − 2 (θ − µ)2 Π(x), (11) 2σ where A(µ, σ) is the normalizing constant and Π(x) is a rect function which truncates the normal from 0 to 1. It should be noted that µ and σ are the mean and standard deviation for the normal distribution, not necessarily the mean and standard deviation of the truncated normal. Our choice of distributions bounded between 0 and 1 stems from our desire to apply these methods to the specific problem of evaluating modalities which estimate EF, a parameter which is bounded between 0 and 1. Using a truncated normal for the assumed distribution in equ. 9, we find the following closed-form solution for the log-likelihood: 2 β − 4αγ β 2α + β A(µ, σ) π √ √ exp − erf erf λ = P ln(S) + 2 α 4α 2 α 2 α (12) where α=
M 1 a2m + , 2 2 2σ 2σm m=1
β=− γ=
M µ am (θpm − bm ) − , 2 2 σ σm m=1
M µ2 (θpm − bm )2 + . 2 2σ 2 m=1 2σm
The expression for the log-likelihood with a beta assumed distribution does not easily simplify to a closed form solution, and thus we used numerical integration techniques to evaluate equ. 9. We used a quasi-Newton optimization method in Matlab on a Dell Precision 620 running Linux to maximize the log-likelihood as a function of our parameters[3]. For each experiment we generated EF data for 100 patients using one of the aforementioned distributions. We then ran the optimization routine to estimate the parameters and repeated this entire process 100 times in order to compute sample means and variances for the parameter estimates. The tables below consist of the true parameters used to create the patient data as well as the sample means and standard deviations attained through the simulations.
Objective Comparison of Quantitative Imaging Modalities
3
17
Results
3.1
Estimating the Linear Model Parameters for a Given Assumed Distribution
We first investigated the results of choosing the assumed distribution to be the same as the true distribution. The asymptotic properties of maximum likelihood estimates would predict that in the limit of large patient populations the estimated linear model parameters would converge to the true values[4]. The results, shown in Table 1, are consistent with this prediction. For the experiment below we have chosen ν = 1.5 and ω = 2 for the beta distribution and µ = 0.5 and σ = 0.2 for the truncated normal distribution. Figure 1 illustrates the results of an individual experiment using the truncated normal distribution. Table 1. Values of the estimated linear model parameters using matching assumed and true distributions. a1
a2
True Values 0.6 0.7 pr(Θ)=Beta 0.59±.03 0.69±.03 pr(Θ)=Normal 0.58±.04 0.68±.04 σ1 True Values 0.05 pr(Θ)=Beta 0.048±.005 pr(Θ)=Normal 0.048±.006
a3 0.8 0.79±.05 0.78±.06 σ2 0.03 0.029±.009 0.028±.010
b1
b2
b3
-0.1 0.0 0.1 -0.10±.02 0.00±.02 0.11±.03 -0.09±.02 0.01±.02 0.11±.03 σ3 0.08 0.079±.007 0.080±.007
In an attempt to understand the impact of the assumed distribution on the method we next used a flat assumed distribution, which is in fact a special case of the beta distribution (ν = 1,ω = 1). We used the same beta and truncated normal distributions for the true distribution as was chosen in the previous experiment, namely ν = 1.5, ω = 2, µ = 0.5 and σ = 0.2. As shown in Table 2, the parameters estimated using a flat assumed distribution are clearly not as accurate as those in the experiment with matching assumed and true distributions. However, the systematic underestimation on the am ’s and the systematic overestimation on the bm ’s has not affected the ordering of these parameters. In fact, the estimated parameters have been shifted roughly the same amount. It should also be noted that the estimates of the σm ’s are still accurate. We will return to this point later in the paper. 3.2
Estimating the Linear Model Parameters and the Parameters of the Assumed Distribution
After noting the impact of the choice of the assumed distribution on the estimated parameters it occurred to us to investigate the effect of varying this distribution. In the case of the beta distribution this was simply a case of adding
18
John Hoppin et al.
0.8
0.6
0.7 0.6
0.4
0.4
θ
θ
0.5 0.2
0.3 0.2
0
0.1 −0.2 0
0.2
0.4
0.6
Θ
0.8
0 0
1
0.2
(a)
0.4
Θ
0.6
0.8
1
(b)
1.4 1.2 1
θ
0.8 0.6 0.4 0.2 0 0
0.2
0.4
Θ
0.6
0.8
1
(c)
Fig. 1. The results of an experiment using 100 patients, 3 modalities, and the same true parameters as shown in Table 1. In each graph we have plotted the true ejection fraction against the estimates of the EF for three different modalities ((a), (b) and (c)). The solid line was generated using the estimated linear model parameters for each modality. The dashed lines denote the estimated standard deviations for each modality. The estimated a, b and σ for each graph are (a)0.59, -0.07, 0.06, (b)0.69, 0.03, 0.025 and (c)0.83, 0.12, 0.082. Note that although we have plotted the true EF on the x-axis of each graph, this information was not used in computing the linear model parameters.
Objective Comparison of Quantitative Imaging Modalities
19
Table 2. Values of estimated linear model parameters using a flat assumed distribution (pra (Θ) = 1). a1 True Values 0.6 0.53±.03 prt (Θ)=Beta prt (Θ)=Normal 0.50±.01 True Values prt (Θ)=Beta prt (Θ)=Normal
a2
a3
b1
b2
b3
0.7 0.8 -0.1 0.0 0.1 0.61±.03 0.70±0.05 -0.09±.02 0.02±.02 0.13±.03 0.56±.03 0.64±.08 -0.05±.02 0.07±.03 0.18±.04 σ1 σ2 σ3 0.05 0.03 0.08 0.049±0.005 0.031±0.009 0.079±0.007 0.048±0.005 0.033±0.008 0.080±0.007
ν and ω to the list of parameters over which we were attempting to maximize the likelihood. In similar fashion, we added µ and σ to the list of parameters for the truncated normal distribution. In the case of the beta distributions, we limited the search in the region 1≤ ν,ω≤5, since values of ν and ω between 0 and 1 create singularites at the boundaries, an impossibility considering the nature of EF. In the case of the truncated normal distributions we limited the search in the region 0≤ µ ≤1 and 0.1≤ σ ≤10. We began by choosing the form of the assumed distribution and the true distribution to be the same, i.e. we estimated the parameters of the beta distribution while using beta distributed data. We found that the method successfully approximated the values of all parameters, including those on the assumed distribution, as displayed in Table 3. The results of an individual experiment is displayed graphically in Fig. 2.
Table 3. Values of estimated linear model and distribution parameters with the assumed distribution and the fixed true distribution having the same form. a1 True Values pr(Θ)=Normal pr(Θ)=Beta
0.6 0.59±.03 0.60±.09 b1 True Values -0.1 pr(Θ)=Normal -0.09±.03 pr(Θ)=Beta -0.10±.03 σ1 True Values 0.05 pr(Θ)=Normal 0.050±.002 pr(Θ)=Beta 0.048±.006 Distribution True Values µ = 0.5, ν = 1.5 pr(Θ)=Normal µ = 0.50±.03 pr(Θ)=Beta ν = 1.50±.53
a2 0.7 0.69±.04 0.70±.09 b2 0.0 0.01±.03 0.01±.03 σ2 0.03 0.029±.004 0.030±.011 Parameters σ = 0.2, ω = 2.0 σ = 0.20±.02 ω = 2.08±.99
a3 0.8 0.79±.04 0.79±.11 b3 0.1 0.11±.04 0.11±.04 σ3 0.08 0.080±.003 0.080±.006
20
John Hoppin et al.
In the previous experiment the estimated parameters associated with both the beta and truncated normal distributions were very close to their true values. We now show the results when the assumed distribution differs from the true distribution in Table 4. We know from our previous experiment that when the form of the assumed and true distributions match, the correct distribution parameters are estimated (on average). However, it remains to be seen what distribution parameters will be estimated when the forms of the two distributions differ. Thus in Fig. 3 we display the true distribution as well as the assumed distribution with the mean estimates of the distribution parameters. Note that although the assumed distribution cannot equal the true distribution, it does take on a form which approximates the true distribution in an attempt to maximize the likelihood. Table 4. Values of estimated linear model parameters using different forms of the varying assumed distribution and the fixed true distribution. a1 True Values pra (Θ)=Normal/prt (Θ)=Beta pra (Θ)=Beta/prt (Θ)=Normal True Values pra (Θ)=Normal/prt (Θ)=Beta pra (Θ)=Beta/prt (Θ)=Normal True Values pra (Θ)=Normal/prt (Θ)=Beta pra (Θ)=Beta/prt (Θ)=Normal
4
a2
a3
0.6 0.7 0.8 0.56±.04 0.65±.05 0.74±.06 0.66±.10 0.78±.09 0.89±.12 b1 b2 b3 -0.1 0.0 0.1 -0.09±.02 0.01±.02 0.12±.03 -0.14±.06 -0.06±.06 0.03±.07 σ1 σ2 σ3 0.05 0.03 0.08 0.050±.005 0.029±.004 0.080±.007 0.050±.007 0.025±.011 0.079±.009
Discussion and Conclusions
We have developed a method for characterizing an observers’ performance in estimation tasks without the use of a gold standard. Although a gold standard is not required for this method, it is necessary to make some assumptions on the distribution of the parameter of interest (i.e., EF). We have found that when the assumed distribution matches the true distribution, the estimates of the linear model parameters are both accurate and precise. Conversely, when the assumed and true distributions do not match, we find that our linear model parameters are no longer as accurate. This led us to investigate the role of the assumed distribution in the accuracy of the linear model parameters. By optimizing both the distribution parameters and the model parameters we found that one can effectively find both the model parameters and the form of the assumed distribution.
Objective Comparison of Quantitative Imaging Modalities
0.6
21
0.8 0.7 0.6
0.4
0.5
θ
θ
0.4 0.2
0.3 0.2 0
0.1 0
−0.2 0
0.2
0.4
0.6
Θ
0.8
−0.1 0
1
0.2
(a)
0.4
Θ
0.6
0.8
1
(b)
1.4 1.2 1
θ
0.8 0.6 0.4 0.2 0 0
0.2
0.4
Θ
0.6
0.8
1
(c)
Fig. 2. The results of an experiment using 100 patients, 3 modalities, and the same true parameters as shown in Table 3. In each graph we have plotted the true ejection fraction against the estimates of the EF for three different modalities ((a), (b) and (c)). The solid line was generated using the estimated linear model parameters for each modality. The dashed lines denote the estimated standard deviations for each modality. The estimated a, b and σ for each graph are (a)0.66,-0.11,0.050, (b)0.75,0.01,0.035 and (c)0.86,0.07,0.073. Note in this study the parameters of the beta distribution were estimated along with the linear model parameters.
22
John Hoppin et al.
2.5
Probability Density
2
1.5
1
Estimated Density True Density
0.5
0 0
0.2
0.4
0.6
Ejection Fraction
0.8
1
0.8
1
(a)
Probability Density
1.5
1
Estimated Density True Density
0.5
0 0
0.2
0.4
0.6
Ejection Fraction
(b)
Fig. 3. When the form of the assumed distribution does not match that of the true distribution, we see that the optimal distribution parameters are such that the form of the assumed distribution approximates the true distribution. In (a), the true distribution is a truncated normal which is approximated automatically by the method using a beta distribution (ν = 3.93, ω = 3.47). In (b), the roles are reversed, as a truncated normal automatically approximates a beta distribution (µ = 0.33, σ = 0.42).
Objective Comparison of Quantitative Imaging Modalities
23
When comparing different imaging modalities one would typically prefer the modality with the most reproducible estimates, i.e. the smallest σ. While the estimates of the slope and intercept of our linear model change according to the assumed distribution, the estimates of the σ values remain accurate. This facilitates modality comparisons without knowledge of a gold standard. While the σ’s serve as a description of a modality’s reproducibility, the slope and intercept values describe the systematic error (or bias) of the modality. If one is confident in these estimates they could be employed to adjust and correct systematic error for each modality. Another interesting result of the experiments is the successful estimation of the distribution parameters to fit the form of the true distribution. This could serve as an insight into the distribution of the true parameter for the population studied, i.e., the patient distribution of EF’s. A major underlying assumption of the method proposed in this paper is that the true parameter of interest does not vary according to modality. This assumption may not be accurate in the context of estimating EF, which may vary moment to moment with a patient’s mood and breathing pattern. This assumption may be valid, however, for other estimation tasks. Another assumption we have made is the linear relationship between the true and estimated parameters of interest. More complicated non-linear models can easily be accommodated by this method. However, the integration in equ. 8 could become more costly. In the future, we would like to investigate varying true parameters, i.e. the true EF for a patient varies with modality. We would also like to study the robustness of the technique to different underlying true distributions. In addition, we plan to study the fundamental mathematical properties of the method.
5
Acknowledgements
The authors thank Dr. Dennis Patton from the University of Arizona for his helpful discussions on the various modalities used to estimate ejection fractions. This work was supported by NSF grant 9977116 and NIH grants P41 RR14304, KO1 CA87017-01, and RO1 CA 52643.
References 1. Henkelman, R.M., Kay, I., Bronskill, M.J.: Receiver Operator Characteristic (ROC) Analysis without Truth. Medical Decision Making. 10 (1990) 24–29. 2. Beiden, S.V., Campbell, G., Meier, K.L., Wagner, R.F.: On the Problem of ROC Analysis without Truth: The EM Algorithm and the Information Matrix. In Medical Imaging 2000: Image Perception and Performance Proceedings of SPIE Vol. 3981 (2000) 126–134. 3. Press, W.H.,Teukolsky, S.A.,Vetterling, W.T.,Flannery, B.P.: Numerical Recipes in C: The Art of Scientific Computing. Cambridge University Press, New York, New York. 1995. 4. Kullback, S: Information Theory and Statistics. Dover Publications, Mineola, New York. 1968.
Theory for Estimating Human-Observer Templates in Two-Alternative Forced-Choice Experiments Craig K. Abbey1 and Miguel P. Eckstein12 1
2
Cedars-Sinai Medical Center, Los Angeles, CA 90048 USA Dept. of Psychology, University of California, Santa Barbara 93106
[email protected] and
[email protected]
Abstract. This paper presents detailed derivations of an unbiased estimate for an observer template (a set of linear pixel weights an observer uses to perform a visual task) in two-alternative forced-choice experiments. Two derivations of the covariance matrix associated with the error present in this estimation method are also derived and compared in human-observer data.
1
Introduction
In medical imaging applications, an optimal imaging system produces images that allow a clinician or other observer to best perform a diagnostic task of interest.[1] Optimizing imaging systems by this principle must therefore imply numerous evaluations of human-observer performance. Unfortunately, such studies are costly and time consuming. These difficulties with human-observer studies have motivated the search for models of human-observer performance in diagnostic tasks. A predictive model of human-observer performance could be used in place of human observers as part of a general system optimization.[2] In many tasks, the goal is to detect or discriminate a spatially compact signal – such as a focal lesion – embedded in image noise. There is good evidence in this case that observers adopt a linear decision strategy described by an observer template.[3,4,5] The observer template can be thought of as the set of pixel weights that determine the observer’s visual strategy for performing the task. There has been a considerable effort to find observer templates that predict human observer performance and to understand how the templates change with the statistical properties of the images. Recently, Ahumada and coworkers have described a new approach to the problem of determining the observer template (called a “classification image” in the vision literature) in yes-no visual detection and discrimination tasks. [6,7] In this approach, the observer template is estimated directly from a human-observer study using both the trial-to-trial decisions and the noisy image stimuli that produced the decisions. The approach has been extended to the two-alternative forced-choice (2AFC) experimental paradigm by Abbey et al[8] and used to evaluate the effects of correlated noise on the observer template.[9] M.F. Insana, R.M. Leahy (Eds.): IPMI 2001, LNCS 2082, pp. 24–35, 2001. c Springer-Verlag Berlin Heidelberg 2001
Theory for Estimating Human-Observer Templates
25
However, there is currently very little theoretical analysis of the estimation problem at the core of direct methods to obtain observer templates. The goal of this paper is to begin filling this gap. We present a detailed derivation demonstrating that the 2AFC estimation procedure suggested by Abbey et al is an unbiased estimate of the observer template under the assumptions of Gaussian noise and a linear detection strategy. In addition, we derive an analytical approximation for evaluating the statistical error in the estimation procedure, and we show that this approximation accurately predicts the sample error in humanobserver data.
2
Theory
In a 2AFC detection task, an observer is presented with two images and asked to identify the image that contains the signal. We will denote an image generically by the vector g. We will refer to the signal present image as g+ , and to the signal-absent image as g− . The two images can be decomposed into components g+ = b + s + n+ ,
and
g− = b + n− ,
(1)
where b is a common background, s is the signal profile, and the noise vector, n+ and n− , correspond to the noise in the signal-present and signal-absent images respectively. We will restrict our attention to simple detection and discrimination tasks, and hence the signal profile, s, is presumed to be a fixed (nonrandom) vector. The two noise vectors are assumed to be independent multivariate Gaussian random vectors with zero mean, and known correlation structure described by the covariance matrix, Kn . In this section, we describe how a decision is made in a given trial of a 2AFC experiment under a linear decision model described by an observer template. We then derive an unbiased estimate of the observer template, and the error associated with this estimate. The estimation procedure has been used previously to analyze 2AFC detection tasks. The derivation of the estimate and its error covariance matrix are novel to this work. 2.1
Modelling Decisions in a 2AFC Detection Task
Linear Internal Response Variables An observer is presumed to make a decision in each trial of a 2AFC experiment by forming a scalar-valued decision variable (sometimes called a test statistic or observer response variable) to each of the two images in a given trial. The image that produces the larger decision variable is chosen by the observer as the image containing the signal. For a linear observer, a decision variable, λ, is a linear function of the image defined by λ = wt g + ε,
(2)
where ε is a stochastic internal noise component. The internal noise component is often assumed to be a zero-mean Gaussian random variable, independent of
26
Craig K. Abbey and Miguel P. Eckstein
g. We will follow that assumption here and denote the variance of the internal noise component by σε2 . Note that ε is intended to be a composite internal noise variable and can have elements arising from multiple sources, such as the intrinsic and induced internal noise components suggested by Burgess. [10] The superscript t in Eqn (2) indicates the transpose operation, and hence wt g is the scalar product of w and g. The vector w is the observer template that determines how the image influences the decision variable. The goal of the procedure described in this paper is to estimate this vector. In a given trial of a 2AFC experiment, an observer is presented with both g+ and g− . From these two images, the observer forms decision variables λ+ and λ− according to Eqn (2) with corresponding internal noise components ε+ and ε− . The observer makes a correct decision if λ+ > λ− , and an incorrect decision otherwise. The Trial Score and Figures of Merit for Performance We will define the trial score, o (to indicate the outcome), to be 1 when a correct decision is made and 0 when an incorrect decision is made. In terms of the decision variables, λ+ and λ− , the score can be defined as o = step λ+ − λ− , where the step function is 1 for arguments greater than 0, and 0 otherwise. The trial score can be related to the components of (2) and (1) by o = step wt b + s + n+ + ε+ − wt b + n− − ε− = step wt (s + ∆n) + ∆ε , (3) where ∆n = n+ − n− , and ∆ε = ε+ − ε− . For consistency with prior work,[9] we note that ∆n is equivalent to ∆g − s. For the multivariate distribution ascribed to n+ and n− , ∆n is a zero-mean Gaussian random vector with covariance matrix given by 2Kn . Likewise, ∆ε is a Gaussian random variable with variance 2σε2 The most general measure of performance in a 2AFC experiment is the proportion of correct responses, PC . In terms of the trial score from (3), the proportion correct is defined by an expectation over both ∆n and ∆ε as PC = o = step wt (s + ∆n) + ∆ε ∆n,∆ε , (4) where the angular brackets, · · · , indicate a mathematical expectation. Subscripts on these brackets are used to emphasize the variables involved in the expectation. For Gaussian distributions governing ∆n and ∆ε, the proportion correct is directly related to the detectability index d . In the notation defined above, the observer detectability index is defined by wt s dw = . wt Kn w + σε2
(5)
Theory for Estimating Human-Observer Templates
27
Note that the subscript on the detectability index indicates the dependence on the observer template w. The relationship between proportion correct and the observer detectability under the Gaussian assumptions used here is given by d PC = Φ √w , (6) 2 where Φ is the standard Gaussian cumulative distribution function (CDF), x 1 1 dz √ exp − z 2 . (7) Φ (x) = 2 2π −∞ 2.2
Derivation of Basic Quantities for Template Estimation
Section 2A described how the observer template fits into a mathematical model of how decisions are made in a 2AFC task. We now turn to deriving the basic quantities needed for an unbiased estimate of the observer template within the framework of this decision model. Abbey et al[8] analyze the vector quantity q = (2o − 1) ∆n, and state (without proof) that its mean value is related to the observer template, w. Note that the 2o − 1 term effectively weights ∆n by 1 if a correct decision is made on the trial (trial score of 1), and by −1 if an incorrect decision is made (trial score of 0). Hence we can think of q as the difference between the noise field of the image chosen by observer as containing the signal and the noise field of the rejected image. The goal of this section is to derive the relation between q and the observer template, w. Preliminary Results Three scalar expectations will be necessary for the subsequent derivations. We state them here without derivations. Let z be a standardnormal random variable (zero-mean and variance of unity) and let a and b be independent of z, then b , (8) step (az + b) = Φ |a| b Φ (az + b) = Φ √ , (9) 2 a +1 and
b2 1 zΦ (az + b) = exp − , 2 (a2 + 1) 2π (a2 + 1) a
(10)
where Φ is the standard Gaussian CDF defined in (7). The expectation in (8) is straightforward to derive using the definition of the CDF. Expectations (9) and (10) can be obtained using integration by parts.
28
Craig K. Abbey and Miguel P. Eckstein
We also use the following basic result from probability theory. Let x and y be two random variables such that x and T (y) have the same distribution. Then the expectation of some function of x, say g (x), is equivalent to an expectation in y according to g (x)x = g (T (y))y .
(11)
This result can be found proved in many probability theory texts. Expectation of q Let us consider the expectation of q = (2o − 1) ∆n. The quantity of interest is defined as q = (2o − 1) ∆n = 2 o∆n − ∆n = 2 o∆n . Note that o∆n is a vector quantity, and hence its vector-valued expectation can be thought of as a scalar expectation in each element. The second line follows from the first because ∆n is a zero-mean random vector, and hence ∆n = 0. It is tempting to assume that the product, o∆n, will likewise be zero-mean. However, the trial score is dependent on ∆n, and hence the product will not necessarily have a mean of 0. If we substitute (3) for o, we can rewrite the expectation as q = 2 step wt (s + ∆n) + ∆ε ∆n ∆n,∆ε which explicitly includes the observer template and the internal-noise component. Since ∆ε is presumed to be independent of ∆n, we√can compute the expectation in ∆ε from (8) (equating ∆ε with az where a = 2σε ) as step wt (s + ∆n) + ∆ε ∆ε ∆n ∆n t
w (s + ∆n) √ =2 Φ . ∆n 2σε ∆n
q = 2
The remaining expectation is only over ∆n. We compute this quantity by making two changes of variables as described below. Since ∆n is presumed to be a zero-mean multivariate Gaussian with covari√ 1/2 ance matrix 2Kn , it has exactly the same distribution as 2Kn ˜ z, where ˜ z is a vector of independent standard normal random variables. Each vector element is independent, zero-mean, and has variance equal to one (i.e. K˜z = I). Note 1/2 that Kn should be interpreted in the sense of a matrix square root [11] which 1/2 acts on the eigenvalue spectrum of Kn . Note that Kn is also a symmetric non1/2 1/2 negative matrix and Kn Kn = Kn From (11), we can write the expectation in ∆n as an expectation in ˜ z,
Theory for Estimating Human-Observer Templates
√ 1/2 wt s + 2Kn ˜ z √ 1/2 √ q = 2 Φ 2Kn ˜ z 2σε ˜ z √ t 1/2 t z w s + 2w Kn ˜ √ = 23/2 K1/2 Φ ˜ z n 2σε ˜ z
29
(12)
A useful property of ˜ z in (12) is that its distribution is invariant to unitary transformations. As a result, we can appeal to (11) and replace ˜ z by Uz in (12). Let us consider a unitary matrix U such that w t Kn w e1 , (13) Ut K1/2 n w = t
where e1 = [1, 0, 0, 0, · · · ] . A unitary matrix that satisfies this relation can be constructed by setting the first column of the matrix equal to the unit vector u1 = √
1 wt K
nw
K1/2 n w,
(14)
and then choosing the remaining column vectors to be an ortho-normal basis for the orthogonal complement of u1 . Note that u1 = Ue1 . Applying (13) to (12) yields √ 1/2 wt s + 2wt Kn Uz 3/2 1/2 √ q = 2 Kn Φ Uz 2σε z √ t tK w z w s + 2w n 1 √ = 23/2 K1/2 z . n U Φ 2σε z
Note that the argument of Φ is only dependent on z1 , the first element of z (this is the result of et1 z). Since the elements of z are statistically independent, every element of the remaining vector-valued expectation will be zero except for the first. We can thus write q = 23/2 K1/2 n Uce1 ,
(15)
where c=
Φ
wt s +
2wt Kn w z1 √ . z1 2σε
√
z1
√ √ √ Application of (10) (with a = 2wt Kn w/ 2σε and b = wt s/ 2σε ) yields an expression for the value of c given by √ 2 w t Kn w (wt s) c= exp − 4 (wt Kn w + σε2 ) 2π (wt Kn w + σε2 )
30
Craig K. Abbey and Miguel P. Eckstein
Plugging this expression for c back into Eqn (15), and using the definition of u1 in (14) for Ue1 yields 2 (wt s) 2 exp − q = Kn w. 4 (wt Kn w + σε2 ) π (wt Kn w + σε2 ) Using the definition of d found in (5), we obtain 2 2 exp − (dw /2) q = Kn w. π (wt Kn w + σε2 )
(16)
Hence, the expectation of q – the difference in the noise fields weighted by the trial score – is seen to be the product of the noise covariance matrix, the observer template, and a complicated positive scalar. We have argued previously[8] that the scalar magnitude in Eqn (16) is essentially irrelevant since multiplying w by a positive constant (and adjusting the internal-noise standard deviation by the same constant) yields an equivalent decision strategy. To make this point another way, consider the elements of w in terms of their units. Elements of w take the image intensities in g+ and g− and turn them into units of the internal response variable. Since the internal response variable is unobservable, it is not clear what these units are, and hence the magnitude of w is somewhat arbitrary. To get a scaled version of w, we need to remove the dependence on Kn from the right side of the equation. To do this, we can consider the expected value of K−1 n q. The resulting expectation, −1 Kn q = K−1 n q 2 2 exp − (dw /2) = w, (17) π (wt Kn w + σε2 ) is seen to be a scaled version of the observer template. Covariance Matrix Associated with q In order to get an analytic expression for the error associated with the template estimate described in the next section, we will need to know the covariance of q. We define this quantity to be t Kq = (q − q) (q − q) = qqt − q qt , (18) where the expectation is computed with respect to a matrix quantity qqt . From the definition of q, we see that t 2 qq = (2o − 1) ∆n∆nt = ∆n∆nt = 2Kn , (19)
Theory for Estimating Human-Observer Templates
31
where the second line is a result of the fact that 2o − 1 can only assume the 2 values 1 or −1, and hence (2o − 1) must always be 1. The third line follows from the definition of a covariance matrix for a zero-mean random variable. We can use (19) and the expression for q in (16) to write the covariance matrix of q in Eqn (18) as 4 exp −2 (dw /2)2 Kn wwt Kn . (20) Kq = 2Kn − π (wt Kn w + σε2 ) 2.3
Estimation Procedures
Template Estimation The analytic results of section 2B suggest procedures for estimating the observer template and obtaining the error associated with that estimate. We can replace the expected value in (17) by its sample average to form an estimate of the observer template. In the ith trial of an experiment (i = 1, · · · , NT ), we can compute qi from the trial score, oi , and the noise-field difference, ∆ni . The estimated observer template is ¯, w ˆ = K−1 n q
(21)
where q ¯ is the sample average NT 1 q ¯= qi NT i=1
=
NT 1 (2oi − 1) ∆ni . NT i=1
Since a sample average is an unbiased estimate of its mean, w ˆ is therefore an unbiased estimate of the observer template, scaled according to Eqn (17). Estimation Error Covariance Matrix From Eqn (21) we see that the error covariance associated with w ˆ is defined −1 −1 Kw ˆ = Kn Kq ¯ Kn ,
¯. The covariance where Kq¯ is the covariance matrix of the sample average q matrix of q ¯ is related to the covariance matrix for q by 1 Kq . NT
(22)
1 −1 K Kq K−1 n . NT n
(23)
Kq¯ = The resulting error covariance is thus Kw ˆ =
32
Craig K. Abbey and Miguel P. Eckstein
Two approaches present themselves for determining the error covariance matrix associated with w. ˆ In the sample approach, we substitute the sample estimate for Kq in (22). The sample covariance is defined N
ˆq = K
T 1 t (qi − q ¯) (qi − q ¯) . NT − 1 i=1
The resulting covariance matrix for w ˆ is given by 1 −1 ˆ ˆw K Kq K−1 K ˆ = n . NT n
(24)
A second approach can be derived from the analytic expression for Kq given in Eqn (20). In this case we find that 2 4 exp −2 (dw /2) 1 −1 Kn wwt Kn K−1 Kw K 2Kn − ˆ = n NT n π (wt Kn w + σε2 ) 2 2 exp −2 (d /2) w 2 −1 wwt . = Kn − NT π (wt Kn w + σε2 ) If the second term in this expression is small enough, it may be conveniently neglected yielding the approximation Kw ˆ
2 −1 K . NT n
(25)
The attraction of this approximation is that it is simple and independent of w.
3
Testing Sample and Approximate Errors in the Estimated Template
The template estimation procedure given in (21) has been validated previously by Abbey et al.[8] However, in that work the estimation error was not considered. As a preliminary test of the approximate error given in (25), we use the human observer data of Abbey et al. to compare the sample and approximate methods for obtaining errors in the estimated template. 3.1
Experimental Data
The psychophysical data used here are the results of a 2, 000-trial 2AFC detection task. The images are 32 × 32 pixels with a mean intensity of 128 grey-levels (GL) corresponding to a mean luminance of 16.0 cd/m2 on a linearized display. Because of the small size of the images (32×32 pixels) the images are upsampled by a factor of 2 for display yielding an effective pixel size of 0.60mm. The signal
Theory for Estimating Human-Observer Templates
33
in these experiments is a centered Gaussian “bump” with a standard deviation of 3.0 pixels and an amplitude of 10.0 GL. Gaussian white noise (independent from pixel to pixel) with a common standard deviation (σpix ) of 25.0 GL is added to each image. The images are intended to resemble those used in early work by Burgess[3]the signal profile used in the experiments and the uniform background of a nonsignal image. An example of a noisy signal-present and signal-absent image can be found in the middle of the figure.
Fig. 1. Images and estimated templates. A and D: Mean signal-present and signal-absent profiles. B and E: Example signal-present and signal-absent images. C and F: Template estimates for the two participating observers.
Two observers participated in the psychophysical study. Observer CKA is an author of this paper and fully aware of research goals motivating the experiment (or so he believes). Observer ECF is naive to the goals of the study and compensated for performing the experiments. Both observers have fairly extensive experience in detection tasks of this sort, and task-specific training was completed before starting the experiment. The two images of the right side of Figure 1 are the estimated observer templates – computed according to Eqn (21) – for observers ECF and CKA. A
34
Craig K. Abbey and Miguel P. Eckstein
feature of interest in these two templates is the mild negative region surrounding a bright central region. This negative surround is indicative of suppression at low spatial frequencies. 3.2
Template Error Comparison
Figure 2 shows comparisons of the sample and approximation methods for obtaining the estimator error. The plots show the variance of the elements on a horizontal slice through the middle of the estimated templates. Because the noise is white with a common variance, the approximate method in (25) predicts a common variance for the elements of the template estimate variance to 2 = 1.6 × 10−6 . As seen in the figure, this prediction is in good be 2/NT σpix agreement with the sample estimate of variance obtained from (24). Hence, it appears that Eqn (25) is a good analytic method for obtaining the estimation error covariance matrix.
Fig. 2. Sample estimate and analytic approximation to the variance on a slice through the center of an observer template.
4
Summary
We have presented a detailed derivation of a method for estimating a linear observer template in 2AFC experiments. In particular, we have explicitly derived the expected value of the template estimate, as well as two methods for obtaining the error associated with this estimate. Our preliminary comparison of the two methods indicates a high degree of agreement in human observer data. It is our hope that the mathematical framework described here will serve as a starting point for further investigations into direct methods for understanding how human observers perform visual tasks relevant to medical imaging. For example, we
Theory for Estimating Human-Observer Templates
35
have begun work to extended the approach presented here to general multiplealternative forced-choice experiments. We have also been using the analysis of estimator error presented here to derive statistical hypothesis tests on estimated templates.
Acknowledgements The authors wish to thank Francois Bochud for helpful discussions. Support: NIH RO1-53455
References 1. H.H. Barrett, “Objective assessment of image quality: Effects of quantum noise and object variability.” J. Opt. Soc. Am. A, Vol. 7, pp. 1266-1278 1990. 2. K.J. Myers et al., “A systematic approach to the design of diagnostic systems for nuclear medicine,” in Information Processing in Medical Imaging (S.L. Bacharach, ed.). Martinus Nijhoff, Dordrecht, Netherlands: 431-444, 1986. 3. A.E. Burgess, and H. Ghandeharian, “Visual signal detection. II. Signal-location identification,” J Opt Soc Am A, 1:900-905, 1984. 4. H.H. Barrett, T. Gooley, K. Girodias, J. Rolland, T. White, and J. Yao, “Linear discriminants and image quality,” in Information Processing in Medical Imaging, (A.C.F. Cholchester and D.J. Hawkes, Eds.), Springer-Verlag, Berlin, 458-473, 1991. 5. C.K. Abbey, and H.H. Barrett, “Linear iterative reconstruction algorithms: Study of observer performance,” in Information Processing in Medical Imaging, (Y. Bizais, C. Barillot, and R. Di Paola, Eds.), Kluwer Academic, Dordrecht, 65-76, 1995 6. A.J. Ahumada, “Perceptual classification images from vernier acuity masked by noise,” Perception 26, pp. 18, 1996. 7. B.L. Beard and A.J. Ahumada, Jr., “A technique to extract relevant image features for visual tasks,” Proc. SPIE Vol. 3299, pp. 79-85, 1998. 8. C.K. Abbey and M.P. Eckstein, “Estimation of human-observer templates for 2 alternative forced choice tasks”, Proc. SPIE 3663, pp. 284-295 1999. 9. C.K. Abbey, M.P. Eckstein, and F.O. Bochud, “Estimates of human observer templates for a simple detection task in correlated noise”, Proc. SPIE 3981, 2000. 10. A.E. Burgess and B. Colborne, “Visual signal detection. IV. Observer inconsistency,” J Opt Soc Am A, 5:617-627, 1988. 11. K.V. Mardia, J.T. Kent, and J.M. Bibby, Multivariate Analysis. Academic press, San Diego., 1979.
The Active Elastic Model Xenophon Papademetris1 , E. Turan Onat2 , Albert J. Sinusas3,4 , Donald P. Dione4 , R. Todd Constable1 , and James S. Duncan1,3 1
4
Departments of Diagnostic Radiology, 2 Mechanical Engineering, 3 Electrical Engineering, and Medicine, Yale University New Haven, CT 06520-8042
[email protected]
Abstract. Continuum mechanical models have been used to regularize ill-posed problems in many applications in medical imaging analysis such as image registration and left ventricular motion estimation. In this work, we present a significant extension to the common elastic model which we call the active elastic model. The active elastic model is designed to reduce bias in deformation estimation and to allow the imposition of proper priors on deformation estimation problems that contain information regarding both the expected magnitude and the expected variability of the deformation to be estimated. We test this model on the problem of left ventricular deformation estimation, and present ideas for its application in image registration and brain deformation during neurosurgery.
Continuum mechanical models have been extensively used in medical imaging applications over the last ten years, particularly within the contexts of image registration and cardiac motion estimation. More recently, similar models have been applied to the problem of brain deformation during neurosurgery. The models used have been selected either (i) because of their mathematical properties (e.g. [3,9]) or (ii) as an attempt to model the underlying physics of the situation (e.g. [11,17,21]). Such models are a specific case of the quadratic regularizers used in many computer vision applications, such as in the work of Horn[12] or in the deformable models used for segmentation (see McInerney and Terzopoulos [16] for a review). The classical elastic model is derived from the properties of elastic solids such as metals. In cases of small deformations, the linear elastic model may also be applied to model biological tissue which is more hyperelastic in nature. All linear elastic models so far used in medical imaging work are passive models. These models will produce no deformation of their own and are essentially used for smoothing and/or interpolation. Using an elastic model results in an underestimation of the deformation as the model itself biases the estimates towards zero deformation. In this paper we present work to extend these elastic models to allow for non-zero bias. We call this model the ‘active elastic model’. The active elastic model is designed to be used to solve a problem of the following form: ‘Given an input of noisy, possibly sparse, displacements find a M.F. Insana, R.M. Leahy (Eds.): IPMI 2001, LNCS 2082, pp. 36–49, 2001. c Springer-Verlag Berlin Heidelberg 2001
The Active Elastic Model
37
dense smooth displacement field which results in a deformation which is close to a desired/expected deformation.’ This new method allows us to construct a proper prior model on the deformation that includes both a mean (the desired magnitude of the deformation) and a covariance (derived from the desired degree of smoothness). The rest of this paper reads as follows: In section 1, we review the basic mathematics of the general energy minimization framework and we compare the use of a passive and an active elastic model for estimation purposes. In section 2, we examine the problem of bias in deformation estimation and demonstrate how the active model can be used to reduce this bias. We present some preliminary results of the application of an active model to reduce the bias in left ventricular deformation estimation in section 3.1 and we conclude by discussing potential applications of this methodology in other areas such as image registration and brain deformation during neurosurgery in section 4.
1
The Energy Minimization Framework
In this section we describe a framework in which the goal is to estimate a displacement field u which is a smooth approximation of a noisy displacement field um . We will assume that um is derived from some image-based algorithm, such as the shape-based tracking algorithm[20,17], MR tagging measurements (e.g. [11]) or optical flow estimates (e.g. [12]). We can pose this problem as an approximation problem whose solution is a least-squares fit of u to um subject to some smoothness constraints and takes the form: arg min m 2 c(x)|u (x) − u(x)| dv + W (α, u, x)dV (1) u ˆ= u V where: u(x) = (u1 , u2 , u3 ) is the vector valued displacement field defined in the region of interest V and x is the position in space, c(x) is the spatially varying confidence in the measurements um and W (α, u, x) is a positive semi-definite regularization functional. W is solely a function of u, a model parameter vector α and the spatial position x. This approach also generalizes to the case where the input displacement field um is sparse. At those locations where no measurement um (x) exists the confidence c(x) can be set equal to zero. 1.1
The Linear Elastic Model
In the early computer vision literature (e.g. [12]) the model W was generated using a regularization functional which penalized a weighted sum of the squared derivatives of the displacement field to impose a smoothness constraint. In medical imaging analysis work the classical linear elastic model is often used, especially in those cases where the problem is the estimation of a real deformation (e.g. left ventricular motion estimation [11,17].)
38
Xenophon Papademetris et al.
A common way to define solid elastic models is in terms of an internal energy function. This internal energy function must be invariant to rigid translation and rotation in order to satisfy certain theoretical guidelines (see Eringen[8] for more details.) Hence the use of any elastic model provides no constraints on the rigid component of the displacement. Additional constraints must be employed to take advantage of any other prior information regarding the magnitude of the overall translation and rotation. The classical linear elastic model[22] captures the mechanical properties of a deforming solid in terms of an internal, or strain energy function of the form:
W =
1 t
C
2
(2)
where C is a 6 × 6 matrix representing the elastic properties of the material and is the strain vector. In the most commonly used case, that of isotropic, infinitesimal linear elasticity these can be written as:
C −1
1 −ν 1 −ν = E 0 0 0
−ν 1 −ν 0 0 0
−ν 0 0 0 −ν 0 0 0 1 0 0 0 , 0 2(1 + ν) 0 0 0 0 2(1 + ν) 0 0 0 0 2(1 + ν)
∂u1 ∂x1 ∂u2 ∂x3 ∂u3 ∂x3
= ∂u1 ∂x2 + ∂u1 ∂x + 3 ∂u2 ∂x3 +
∂u2 ∂x1 ∂u3 ∂x1 ∂u3 ∂x2
(3)
where u(x) = (u1 (x), u2 (x), u3 (x)) is the displacement at point x = (x1 , x2 , x3 ). E is the Young’s modulus which is a measure of the stiffness of the material and ν is the Poisson’s ratio which is a measure of the incompressibility. In the rest of the paper we will refer to the classical linear elastic model as the passive model to distinguish it from the active linear elastic model described in the next section. 1.2
The Active Linear Elastic Model
The classical linear elastic model described in equation (2) is a passive model. In the absence of any external force, the material will do nothing. Given no external work, equilibrium is reached at the lowest energy state where the strain vector is identically equal to zero. Such a material model is not accurate in the case of actively deforming objects such as the left ventricle of the heart. In this case, a substantial part of the deformation is actively generated by the muscle and is clearly not a result of external forces. This active deformation does not produce a change in the strain energy of the material and to account for this factor we need to modify the elastic model appropriately. With this in mind we propose the active elastic model which takes the form: W =
1 ( − a )t C( − a ) 2
(4)
The Active Elastic Model
39
where a is the active strain component. The active strain component represents the deformation that is not a product of external forces and hence should not be penalized by the model. In the absence of external forces, the active elastic model results in a deformation equal to the one actively generated by the object. So in this sense it can deform itself and hence it justifies the label active. Given a prior model of the active contraction, the active elastic model can also be used to generate a prediction of the position of the deforming object. This model is also appropriate in the case where it is used to regularize an image registration problem where there is no such physical notion of active deformation. Here, the active component a can be thought of as the expected magnitude of the deformation. 1.3
The Elastic Model as a Prior Probability Density Function
The energy minimization problem described in equation (1) can also be expressed as a Bayesian maximum a-posteriori estimation problem[17]. In this case, the solution vector u ˆ is the u that maximizes a posterior probability density p(u|um ). Using Bayes’ rule, we can pose this problem (at each point x) as:
p(um |u)p(u) u ˆ = argumax p(u|um ) = p(um ) arg max m {log p(u |u) + log p(u)} (5) = u by noting that p(um ) is a constant once the measurements have been made. The measurement probability p(um |u) can be obtained by using a white noise model for the noise in the measurements um . The prior probability density function p(u) can be derived using an energy function (such as W ) using a probability density function of the Gibbs form [10]. We note that this approach has been previously used in medical imaging problems (e.g. Christensen [3], Gee [9] and others). In the cases of the passive and the active model, this prior distribution has the form: − t C
2 −( − a )t C( − a ) Active: log p(u) = k2 + 2
Passive: log p(u) = k1 +
(6) (7)
where k1 and k2 are normalization constants. Note further that the standard multivariate normal distribution (mean=µ, covariance =Σ) has the form (k3 is similarly a normalization constant): log p(u) = k3 +
−(u − µ)t Σ −1 (u − µ) 2
(8)
By comparing equations (6) and (7) to equation (8), we can see that in both cases the material matrix C plays a similar role to the inverse of the covariance matrix (the stiffer the material is, the greater the coupling between the displacements of neighboring points and hence the smaller the effective component of
40
Xenophon Papademetris et al. q 1
A
p 1
K
p 2
A
q 2
L
Fig. 1. A one-dimensional example. Consider a one-dimensional object consisting of two points p1 and p2 originally a distance L apart. The body is modeled using an elastic spring of stiffness of K. The body is then somehow deformed (stretched). In the deformed state, we have initial estimates of the positions of p1 and p2 shown as q1 and q2 respectively, and the confidence in these estimates is given by A. The problem can be visualized by connecting point pairs (p1 ,q1 ) and (p2 ,q2 ) with zero length springs of effective stiffness A and points (p1 ,p2 ) with a spring of stiffness K and length L. In this case, the initial displacements are given by um = [q1 − p1 , q2 − p2 ]t and the strain 2 . is equal to u1 −u L the covariance matrix), and that in the case of the active model, the active strain
a acts like the mean of the distribution. In the case of the passive model, the mean is effectively zero. Hence we can explicitly see that the active elastic model is a generalization of the passive model, by adding the possibility of having a non-zero mean.
2
Bias Reduction Using the Active Elastic Model
The passive elastic model will likely underestimate the real deformation as a result of its penalization of all deformations. We proceed to illustrate the problem by means of a simple example and demonstrate how the active model can be used to reduce the bias. We also describe how the problem (or more precisely its symptoms) have been dealt with in the literature and point out some of the shortcomings in those approaches. 2.1
A Simple Example
To illustrate the concept of the active elastic model more concretely we will use the simple one-dimensional case described in figure 1. In this case the approximation functional (see equation 1) takes the form: arg min m (9) u ˆ= A |u (p1 ) − u(p1 )|2 + |um (p2 ) − u(p2 )|2 + W u We will consider two forms of W , a passive model Wpassive and an active model Wactive which have the form: K u1 − u2 2 Wpassive = 2 L 2 K u1 − u2 − a Wactive = 2 L
(10)
The Active Elastic Model
41
Note here that the active model reduces to the passive model if the value of the active strain a is set to zero. Substituting for the models defined in equation (10) into equation (9), and differentiating with respect to u we obtain the following matrix equations (in the active case): m −K u1 Au1 + K a A+ K L L = (11) a u2 Aum −K A+ K 2 + K
L L To simplify the math in order to make the illustration clearer, we set um (p2 ) = 0, u(p2 ) = 0. This results in the following two solutions for u(p1 ):1 Aum (p1 ) A+ K L Aum (p1 ) + K a Active Model: u(p1 ) = A+ K L
Passive Model: u(p1 ) =
(12) (13)
Further we can write the expected value of u(p1 ), E(u(p1 )) in terms of the expected value of um (p1 ), E(um (p1 )) as: A E(um (p1 )) A+ K L
A
K a m E(u Active Model : E(u(p1 )) = (p )) + 1 A+ K A+ K L L
Passive Model : E(u(p1 )) =
2.2
(14) (15)
Bias Estimation and Reduction
In the solution produced by the passive model, the expected value of u(p1 ) (see equation 14) will be smaller than the expected value of the measurements um (p1 ) as long as K > 0. Hence any estimation using the passive elastic model is biased, and will underestimate the actual deformation. Consider the case where L = 1, A = 3K. In this case by substitution into equations (14) and (15) we get the following expressions: Passive: E(u(p1 )) =
3 E(um (p1 )), 4
Active: E(u(p1 )) =
3 1 E(um (p1 )) + a 4 4
So by an appropriate choice of a derived from knowledge of the specific problem the bias in the estimation can be significantly reduced. For example, if we had 1
As an aside, we also note that the expressions of equations (12) and (13) can be . For example, rewritten so that the constants K and A appear only as the ratio K A um (p1 ) equation (12) can be rewritten as u(p1 ) = 1+ K . Hence the absolute value of the AL
stiffness K or the data confidence A do not enter into the problem. This can be a problem in the case of the estimation of real deformation (such as in the case of the left ventricle) as the two are measured in different units and hence make the equation inconsistent from a dimensionality viewpoint.
42
Xenophon Papademetris et al.
1) prior knowledge of the expected strain in this case (where = u(p2 )−u(p ), we L a could use such information to set the active strain so as to reduce the bias. We note further that the effect of the bias is more significant where the relative confidence of the measurements (A) is low as a result of noisy data.
2.3
Alternative Methods of Bias Reduction
We also note that the problem of bias has been dealt with in a number of different ways in the literature (often without being actually recognized as such). Zero Stiffness: This ‘solution’ is used by Park et al[19] where the Young’s Modulus is set to zero. In this case, temporal filtering is used for noise reduction. This eliminates the problems associated with bias; it also forfeits all the usefulness of exploiting the spatial relationships between different points in the model. The method is successful in part because the input data are very clean. Direct Bias Correction: Sometimes further knowledge about the problem can be used to correct for some of the bias. In our earlier work [18,17] on left ventricular deformation estimation we solved the problem in a two step fashion, for each frame in the image sequence. At each time t the problem was solved first using a formulation like that of equation (1) to produce an estimate of the position of all the points at time t + 1. Then all points that were on the endo- and epi-cardial surfaces of the heart at time t were mapped to the (pre-segmented) endo- and epi-cardial surfaces at time t + 1, using a modified nearest neighbor approach. In this approach the bias in the radial and circumferential directions is largely accounted for but there remains bias in the longitudinal direction (which lies parallel to the ‘major’ axis of the surface). Other methods which constrain the tracked tokens to lies on a given curve or surface fall into this category of bias correction (e.g. [14]). The Incremental Approach: In this case the estimation problem is broken into a number of small (algorithmic) steps. This has the effect of reducing the bias which is directly related to the magnitude of um . Consider again the simple example of figure 1 with L = 1, A = 3K as before. If the displacement um (p1 ) is applied in one step, we get an estimate of u(p1 ) = 0.75um(p1 ) and a bias of 0.25um(p1 ). The incremental approach is best explained algorithmically. At each increment i ∈ (0, N ) the estimate of u(p1 ) is defined as di (p1 ). Then, for any increment i we calculate di (p1 ) as: i = 0 : d0 (p1 ) = 0 i > 0 : di (p1 ) = di−1 (p1 ) + 0.75
i um (pi ) − di−1 (p1 ) N
The Active Elastic Model
43
This essentially is a history-free approach as in each step the model is only used to regularize the difference between the current input and the last step as opposed to the whole of the input. This approach results in smaller input displacements which are closer to zero, thus resulting in a reduction of the bias. The reduction of the bias is directly related to the number of steps. In this specific case when N = 2 the total bias is 0.16um(p1 ), when N = 4 it is 0.08um(p1 ), and for N = 8, it is reduced to 0.04um(p1 ). The Fluid Model: This is essentially the limiting case of the incremental approach. In the work of Christensen[3], it takes the differential form: µ∇2 v + (λ + µ)∇(∇.v) = F
(16)
where F is the image derived forcing function and v is the local velocity vector. The isotropic linear elasticity model can also be written in differential form by differentiating the energy functional posed in equation (1) and generating a force F by grouping together all external displacements um . This takes the form (as derived in Christensen [2]): µ∇2 u + (µ + λ)∇(∇.u) = F
(17)
where λ and µ are the Lam`e constants which are defined in terms of the Young’s Eν E modulus E and the Poisson’s ratio ν as[22]: λ = (1+ν)(1−2ν) , and µ = 2(1+ν) . If we compare equations (16) with (17) we see that they have essentially the same form, with the one being in terms of the velocity v and the other in terms of the displacement u. The fluid model can be seen to be the limiting case of the incremental approach of the previous section as the step size goes to zero. This approach has the advantage of explicitly stating its assumptions properly and possibly some numerical advantages. 2 Disadvantages of the Incremental/Fluid Approach: The incremental/fluid approach substantially reduces the bias, but the history of the deformation is lost at each (algorithmic) step. Hence in this way we cannot capture aspects of real materials such as progressive hardening with increased deformation (using nonlinear elastic models) as at each step the deformation is assumed to be zero. Also the fact that the analysis is reset at the end of each step makes incorporation of temporal smoothness constraints in problems such as left ventricular motion estimation very difficult. Perhaps more fundamental in certain cases is the lack of the ability of either of these approaches to encapsulate any prior information available as to the expected magnitude of the deformation, as opposed to simply its relative smoothness. 2
This is perhaps the answer to the ‘controversy’ as to whether the linear elastic model is useful in the case of large deformations. If the (passive) linear elastic model is applied using the incremental approach, as is often the case, it is really a fluid model in disguise hence it has similar large deformation capabilities.
44
2.4
Xenophon Papademetris et al.
Relation of the Active Elastic Model to Other Methods
In this section we clarify the relationship of certain other methods in the literature which relate or appear to relate to the active elastic model. Any criticism of these methods is simply with respect to its application in the problem of interest of our own work. (We do note that these methods were mostly designed to solve different problems.) The thin-plate spline: A common regularization function is the thin-plate spline model[1] which in two dimensions has the form (using u = (u1 , u2 ) and x = (x1 , x2 ) : W (u) =
∂ 2 u 2 ∂ 2 u 2 ∂ 2 u 2 ∂ 2 u 2 ∂ 2 u 2 ∂ 2 u 2 1 1 1 2 2 2 + + + + + ∂x21 ∂x22 ∂x1 ∂x2 ∂x21 ∂x22 ∂x1 ∂x2
It can easily be shown that this function would qualify as a solid elastic model as it is invariant to rigid translation and rotation. In fact this function is invariant to all affine transformations. Hence, the bias in the estimate of the deformation in methods which utilize the thin-plate spline as a regularizer (e.g. [4]), is limited to only that component of the deformation which is not captured by an affine transform. In this respect the thin-plate spline is superior to the standard (passive) elastic regularizers, but a bias problem still remains which in certain cases could be substantial. The Active Shape Model: In a series of papers Cootes et al (e.g. [6,7]) presented a methodology for segmentation and registration using a point-based shape model. While this is interesting work, it does not directly relate to the active elastic model presented in this paper. The goal of the active shape model is to capture the statistical variation of the shape of a given structure/object across a number of images, whereas the goal of our work is to be able to include information regarding the expected deformation of a given object across a sequence of images. The balloon variation of the active contour: In the balloon model of Cohen et al[5], an additional force is added to the standard snake[13] algorithm to provide for a constant expansion or contraction force. While this force does reduce the bias towards zero deformation of the underlying snake, it does so as an additional force and not as a change in the regularization model. Hence it cannot be used to capture prior information regarding the expected magnitude of the deformation, as can the elastic model. Non-Rigid Registration of Brain Images with Tumor Pathology: Kyriacou et al [15] presented some interesting work relating to the registration of pre- and posttumor brain images. To achieve an accurate registration a uniform contraction of the tumor is first used to estimate the shape of the post-tumor brain prior to the growth of the tumor. Unlike the balloon approach of Cohen[5], this uniform contraction procedure is very close in spirit to our work on the active elastic model, as in this case the tumor is shrinking under the influence of internal contraction and not as a result of an external force.
The Active Elastic Model
3 3.1
45
Experimental Results Methodology
In this section we present some preliminary results of the application of this algorithm to left ventricular deformation estimation. The active elastic model is used to do two things: (i) Isovolumic Bias Correction and (ii) Imposition of a temporal smoothness constraint alongside the Isovolumic Bias Correction. We bootstrap the algorithm by using the output produced by our previous work [18,17]. We label this algorithm as the ‘passive’ algorithm. In the passive algorithm, the images are segmented interactively and then initial correspondence is established using a shape-tracking approach. A dense motion field is then estimated using a passive, transversely linear elastic model, which accounts for the fiber directions in the left ventricle. The dense motion field is in turn used to calculate the deformation of the heart wall in terms of strains. We note that, although we apply bias correction in the passive algorithm (see section 2.3) bias remains in the estimate of the strain in the longitudinal direction (which lies parallel to the ‘major’ axis of the surface). The output of the ‘passive’ algorithm consists of a set of vectors p (xi , tj ) representing the strain estimated by the passive algorithm at position xi and time tj . Typically we divide the heart into about 800-1000 (i.e. i ∈ 1 : 1000) elements and use 6-9 time frames (j ∈ 1 : 9) resulting in a total of approximately 7000 6 × 1 vectors p = [ prr , pcc , pll , prc , prl , plc ]t . The components of p are the normal strains in the radial (rr), circumferential (cc) and longitudinal (ll) directions as well as the shears between these direction (e.g. prc is the radial-circumferential shear strain). These vectors p are then used to generate an estimate of the active strain a
(in one of two different methods as discussed below) and then a new set of output strains is estimated using the new ‘active’ algorithm. In this case we do not employ any additional bias correction. A. Isovolumic Bias Correction: In this bias correction procedure at each discrete element position xi and time tj we generate an output vector a (xi , tj ) by adjusting the longitudinal strain to create a new set of strain estimates a that result in an incompressible deformation. The fractional change in volume produced under strain p can be approximated as: δV p = (1 + prr ) × (1 + pcc ) × (1 + pll ) If we assume that most of the bias is in the longitudinal direction and that in reality the volume is preserved we can generate an estimate of the active strain
a (xi , tj ) by simply (i) setting a (xi , tj ) = p (xi , tj ) and (ii) adjusting the longitudinal component of a to correct for any divergence from the incompressibility constraint i.e. 1
all = p (1 + rr ) × (1 + pcc )
46
Xenophon Papademetris et al.
These estimates a are used as the mean value for the active elastic model. The variance is determined by the stiffness matrix and is the same as it was for the passive model. We label the results produced by this procedure as Active. B. Temporal Smoothing and Isovolumic Bias Correction: In this case, before estimating the active strain component a as above the strain vectors p (xi , tj ) are smoothed by performing a temporal convolution with a one-dimensional Gaussian kernel of standard deviation σ = 1.0 in the time direction to produce a temporally smooth set of vectors s . The s vectors are then used instead of the un-smoothed vectors p as the input to isovolumic bias correction procedure described above. This combined temporal smoothing and isovolumic bias correction procedure is used to generate an estimate of the active strain a to be used with the active elastic model. We label the results produced by this procedure as ActiveT. 3.2
Experiments
Data: We tested the new algorithm(s) by comparing its output to those obtained using MR tagging[14] and implanted markers[18]. In the MR tagging case we used one human image sequence provided to us by Dr Jerry Prince from John Hopkins University. The images were acquired using 3 orthogonal MR tagging acquisitions and the displacements estimated using an algorithm presented in Kerwin[14]. From these displacements we estimate the MR tagging derived strains. Images from one of the three acquisitions had the evidence of the tag lines removed using morphological operators, was segmented interactively and the strains were estimated using our previous approach (Passive)[18]. In the case of implanted markers we used 8 canine image sequences with implanted markers as was described in [18]. Tests: We tested two permutations of the active algorithm. For the algorithm labeled Active in figure 3, we used as input the output of the passive algorithm after isovolumic bias correction, without any temporal smoothing. The algorithm labeled as ActiveT used the output of the passive algorithm with both temporal smoothing and isovolumic bias correction. Figure 2 illustrates the output of algorithm ActiveT at four points in the cardiac cycle as applied to the MR tagging sequence. The output of the tagging method[14] at End-systole is presented for comparison. Figure 3 shows the error between the estimates of our old algorithm labeled passive and the two variations of the new active algorithm (Active and ActiveT ), as compared to the output of the tagging algorithm[14] and to the estimates obtained using the MR markers. In the case of the tagging algorithm we observe an overall reduction in mean strain error from 9.9% (passive) to 8.1% (active) at end-systole (frame 10). In the case of the implanted markers we observe a similar reduction from 7.2% to 6.3%. It is also interesting to note that the MR tagging algorithm [14] produces a reduction of myocardial volume of 12% between end-diastole and end-systole, our
The Active Elastic Model
47
Longit.
Radial
Circum.
passive algorithm an increase of approximately 14% and all both versions of the active algorithm produced small increases (< 2%) showing that the isovolumic bias correction was effective.
Fig. 2. Leftmost four columns: Circumferential, Radial and Longitudinal strain outputs of our active (Active 2T ) algorithm at four points in the systolic half of the cardiac cycle. Far right column: Output of MR tagging based algorithm[14] on the same image sequence.
(End-Systole)
End-Systole
Fig. 3. Absolute Strain Error vs Tag Data or Implanter Markers. Passive – passive model from [18], Active and ActiveT represent two versions of the active algorithm without and with temporal smoothing. We note that both the active algorithms result in error reduction as compared to the passive algorithm. In the case of the tagging data we plot the absolute error in the cardiac-specific strains whereas in the case of implanted markers we use the principal strains instead (see [18].)
48
4
Xenophon Papademetris et al.
Conclusions
The active elastic model is a generalization of the original elastic model which penalizes deformations away from a preset value as opposed to simply all deformations. This model can be used as a prior to solve problems where we have prior information regarding the magnitude and the variability of the expected deformation, hence it can be used to construct a proper prior probability density function for the displacement field having both a mean and a covariance, as opposed to the more traditional elastic model which has a fixed mean of zero. The cardiac deformation example is an obvious application of this model as the active strain component can be used to model the active contraction of the left ventricle in the systolic phase of the cardiac cycle. In the case of image registration such an active model could be used to good effect in cases where even a gross sense of the magnitude of the deformation exists a priori. For example, in Wang et al[23] where statistical shape-based segmentation information is used to constrain an elastic model, information from the segmentation regarding the relative deformation of different structures can be used with an active elastic model to drive the elastic model towards the expected solution, thus applying ‘forces’ to the elastic model from within as opposed to from ‘the outside’. Another example is the case of cerebro-spinal fluid loss in neurosurgery which results in large deformations in the ventricles not accounted for by gravitational forces[21]. In this case an active elastic model could be used to account for the expected large deformation of the ventricles (based perhaps on population statistics from inter-operative images) and hence reduce the bias in the final displacement field.
References 1. F. L. Bookstein. Principal warps: Thin-plate splines and the decomposition of deformations. IEEE Transactions on Pattern Analysis and Machine Intelligence, pages 567–585, 1989. 2. G. E. Christensen. Deformable Shape Models for Anatomy. Ph. D. dissertation, Washington University, Saint Louis, MI, August 1994. 3. G. E. Christensen, R. D. Rabbitt, and M. I. Miller M. I. Deformable templates using large deformation kinematics. IEEE Transactions on Image Processing, 5(10):1435–1447, 1996. 4. H. Chui, J. Rambo, R. Schultz, L. Win, J. Duncan, and A. Rangarajan. Registration of cortical anatomical structures via 3d robust point matching. In Information Processing in Medical Imaging, pages 168–181, Visegrad, Hungary, June 1999. 5. L. D. Cohen and I. Cohen. Finite element methods for active contour models and balloons for 2D and 3D images. IEEE Trans. Pattern Analysis and Machine Intelligence, 15(11):1131–1147, November 1993. 6. T. Cootes, A. Hill, C. Taylor, and J. Haslam. The use of active shape models for locating structures in medical images. In H. H. Barrett and A. F. Gmitro, editors, Information Processing in Medical Imaging, pages 33–47. LNCS 687, SpringerVerlag, Berlin, 1993.
The Active Elastic Model
49
7. T. F. Cootes, C. J. Taylor, D. H. Cooper, and J. Graham. Active shape models – their training and application. Comp. Vision and Image Understanding, 61(1):38– 59, 1995. 8. A. C. Eringen. Mechanics of Continua. Krieger, New York, NY, 1980. 9. J. C. Gee, D. R. Haynor, L. Le Briquer, and R. K. Bajcsy. Advances in elastic matching theory and its implementation. In CVRMed-MRCAS, Grenoble, France, March 1997. 10. D. Geman and S. Geman. Stochastic relaxation, Gibbs distribution and Bayesian restoration of images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 6:721–741, 1984. 11. E. Haber, D. N. Metaxas, and L. Axel. Motion analysis of the right ventricle from MRI images. In Medical Image Computing and Computer Aided Intervention (MICCAI), pages 177–188, Cambridge, MA, October 1998. 12. B. K. P. Horn and B. G. Schunk. Determining optical flow. Artificial Intelligence, 17:185–203, 1981. 13. M. Kass, A. Witkin, and D. Terzopoulus. Snakes: Active contour models. International Journal of Computer Vision, 1:312–331, 1988. 14. W. S. Kerwin and J. L. Prince. Cardiac material markers from tagged MR images. Medical Image Analysis, 2(4):339–353, 1998. 15. S. Kyriakou and C. Davatzikos. A biomechanical model of soft tissue deformation with applications to non-rigid registration of brain image with tumor pathology. In Medical Image Computing and Computer Assisted Intervention, pages 531–538. Springer, Berlin, 1998. LNCS 1496. 16. T. McInerney and D. Terzopoulos. Deformable models in medical image analysis: a survey. Medical Image Analysis, 1(2):91–108, 1996. 17. X. Papademetris, A. J. Sinusas, D. P. Dione, and J. S. Duncan. Estimation 3D left ventricular deformation from echocardiography. Medical Image Analysis, in-press (March 2001). 18. X. Papademetris, A. J. Sinusas, D. P. Dione, and J. S. Duncan R. T. Constable. Estimating 3D strain from 4D cine-MRI and echocardiography: In-vivo validation. In Medical Image Computing and Computer Aided Intervention (MICCAI), Pittsburgh, U.S.A., October 2000. 19. J. Park, D. N. Metaxas, and L. Axel. Analysis of left ventricular wall motion based on volumetric deformable models and MRI-SPAMM. Medical Image Analysis, 1(1):53–71, 1996. 20. P. Shi, A. J. Sinusas, R. T. Constable, E. Ritman, and J. S. Duncan. Point-tracked quantitative analysis of left ventricular motion from 3D image sequences. IEEE Transactions on Medical Imaging,, 19(1):36–50, January 2000. 21. O. Skrinjar and J. Duncan. Real time 3D brain shift compensation. In Information Processing in Medical Imaging (IPMI 99), pages 42–55, 1999. 22. A. Spencer. Continuum Mechanics. Longman, London, 1980. 23. Y. Wang and L. H. Staib. Elastic model based non-rigid registration incorporating statistical shape information. In Medical Image Computing and Computer Aided Intervention (MICCAI), pages 1162–1173. Springer, Berlin, 1998. LNCS 1496.
A Minimum Description Length Approach to Statistical Shape Modelling Rhodri H. Davies, Tim F. Cootes, and Chris J. Taylor Division of Imaging Science, Stopford Building, Oxford Road, University of Manchester, Manchester, M13 9PT, UK.
[email protected]
Abstract. Statistical shape models show considerable promise as a basis for segmenting and interpreting images. One of the drawbacks of the approach is, however, the need to establish a set of dense correspondences between examples of similar structures, across a training set of images. Often this is achieved by locating a set of ‘landmarks’ manually on each of the training images, which is time-consuming and subjective for 2D images, and almost impossible for 3D images. This has led to considerable interest in the problem of building a model automatically from a set of training shapes. We extend previous work that has posed this problem as one of optimising a measure of model ‘quality’ with respect to the set of correspondences. We define model ‘quality’ in terms of the information required to code the whole set of training shapes and aim to minimise this description length. We describe a scheme for representing the dense correspondence maps between the training examples and show that a minimum description length model can be obtained by stochastic optimisation. Results are given for several different training sets of 2D boundaries, showing that the automatic method constructs better models than the manual landmarking approach. We also show that the method can be extended straightforwardly to 3D.
1
Introduction
Statistical models of shape show considerable promise as a basis for segmenting and interpreting images [5]. The basic idea is to establish, from a training set, the pattern of ‘legal’ variation in the shapes and spatial relationships of structures in a given class of images. Statistical analysis is used to give an efficient parameterisation of this variability, providing a compact representation of shape and allowing shape constraints to be applied effectively during image interpretation [6]. One of the main drawbacks of the approach is, however, the need during training - to establish dense correspondence between shape boundaries over a reasonably large set of example images. It is important to establish the ‘correct’ correspondence, otherwise an inefficient parameterisation of shape can result, leading to difficulty in defining shape constraints. In practice, correspondence has often been established using manually defined ‘landmarks’; this is both time-consuming and subjective. The problems are exacerbated when the approach is applied to 3D images. M.F. Insana, R.M. Leahy (Eds.): IPMI 2001, LNCS 2082, pp. 50–63, 2001. c Springer-Verlag Berlin Heidelberg 2001
A Minimum Description Length Approach to Statistical Shape Modelling
51
Several previous attempts have been made to automate model building [10, 11, 16]. The problem of establishing dense correspondence over a set of training boundaries can be posed as that of defining a parameterisation for each of the training set, leading to implicit correspondence between equivalently parameterised boundary points. Two different but equally arbitrary parameterisations of the training boundaries have been proposed [2, 14] , but neither of these addresses the issue of optimality. Shape ‘features’ (e.g. regions of high curvature) have been used to establish point correspondences, with boundary length interpolation between these points . Although this approach corresponds with human intuition, it is still not clear that it is in any sense optimal. A third approach, and that followed in this paper, is to treat finding the correct parameterisation of the training shape boundaries as an explicit optimisation problem. The optimisation approach has been described by several authors [10, 16, 4] and is discussed in more detail in Section 3. The basic idea is to find the parameterisation of the training set that yields, in some sense, the ‘best’ model. We have previously described an approach in which the best model is defined in terms of ‘compactness’, as measured by the determinant of its covariance matrix [16]. We represented the parameterisation of each of a set of training shapes explicitly, and used genetic algorithm search to optimise the model with respect to the parameterisation. Although this work showed promise, there were several problems: the objective function, although reasonably intuitive, could not be rigorously justified; the method was described for 2D shapes and could not easily be extended to 3D; and it was sometimes difficult to make the optimisation converge. In this paper we define a new objective function with a rigorous theoretical basis and describe a new representation of correspondence/parameterisation that extends to 3D and also results in improved convergence. Our objective function is defined in an information theoretic framework. The key insight is that the ‘best’ model is that which describes the entire training set as efficiently as possible, thus we adopt a minimum description length criterion. In the remainder of the paper we outline the model-building problem, review previous attempts to automate the process, describe in detail how we construct our objective function, describe our representation of correspondence, and present experimental results for automatic model building, using genetic algorithm search to optimise the objective function.
2
Statistical Shape Models
A 2D statistical shape model is built from a training set of example outlines. Each shape, Si , can (without loss of generality) be represented by a set of (n/2) points sampled along the boundary at equal intervals, as defined by some parameterisation Φi of the boundary path. Using Procrustes analysis [9] the sets of points can be rigidly aligned to minimise the sum of squared differences between corresponding points. This allows each shape Si to be represented by an n-dimensional shape vector xi , formed by concatenating the coordinates of its
52
Rhodri H. Davies, Tim F. Cootes, and Chris J. Taylor
sample points, measured in a standard frame of reference. Using Principal Component analysis, each shape vector can be approximated by a linear model of the form x=x ¯ + Pb
(1)
where x ¯ is the mean shape vector, the columns of P describe a set of orthogonal modes of shape variation and b is a vector of shape parameters. New examples of the class of shapes can be generated by choosing values of b within the range found in the training set. This approach can be extended easily to deal with continuous boundary functions [16], but for clarity we limit our discussion here to the discrete case. The utility of the linear model of shape shown in (1) depends on the appropriateness of the set of boundary parameterisations {Φi } that are chosen. An inappropriate choice can result in the need for a large set of modes (and corresponding shape parameters) to approximate the training shapes to a given accuracy and may lead to ‘legal’ values of b generating ‘illegal’ shape instances. For example, consider two models generated from a set of 17 hand outlines. Model A uses a set of parameterisations of the outlines that cause ‘natural’ landmarks such as the tips of the fingers to correspond. Model B uses one such correspondence but then uses a simple path length parameterisation to position the other sample points. The variance of the three most significant modes of models A and B are (1.06, 0.58, 0.30) and (2.19, 0.78, 0.54) respectively. This suggests that model A is more compact than model B. All the example shapes generated by model A using values of b within the range found in the training set are ‘legal’ examples of hands, whilst model B generates implausible examples - this is illustrated in Fig. 1 . model A
model B
Fig. 1. The first three modes of variation (±2σ) of models A and B
The set of parameterisations used for model A were obtained by marking the ‘natural’ landmarks manually on each training example, then using simple
A Minimum Description Length Approach to Statistical Shape Modelling
53
path length parameterisation to sample a fixed number of equally spaced points between them. This manual mark-up is a time-consuming and subjective process. In principle, the modelling approach extends to 3D, but in practice, manual landmarking becomes impractical.
3
Previous Work
Various authors have described attempts to automate the construction of statistical shape models from a set of training shapes. The simplest approach is to select a starting point and equally space landmarks along the boundary of each shape. This is advocated by Baumberg and Hogg [2] but, as shown in the previous section, it does not generally result in a satisfactory model. Kelemen et al [14] use spherical harmonic descriptors to parameterise their training shapes. Although it is independent of origin, this is still an arbitrary parameterisation of the boundary, which is in no obvious sense optimal. Benayoun et al [3], Kambhamettu and Goldgof [13] and Wang et al [19] all use curvature information to select landmark points. It is not, however, clear that corresponding points will always lie on regions that have the same curvature. Also, since these methods only consider pairwise correspondences, they may not find the best global solution. A more robust approach to automatic model building is to treat the task as an optimisation problem. Hill and Taylor [10] attempt this by minimising the total variance of a shape model. They choose to iteratively perform a series of local optimisations, re-building the model at each stage. This makes the approach prone to becoming trapped in local minima and consequently depends on a good initial estimate of the correct landmark positions. Rangarajan et al [17] describe a method of shape correspondence that also minimises the total model variance by simultaneously determining a set of correspondences and the similarity transformation required to register pairs of contours. Bookstein [4] describes an algorithm for landmarking sets of continuous contours represented as polygons. Points are allowed to move along the contours to minimise a bending energy term. Again, it is not obvious that the resulting model is in any useful sense optimal. Kotcheff and Taylor [16] describe an objective function, based on the determinant of the model covariance. This favours compact models with a small number of significant modes of variation, though no rigorous theoretical justification for this formulation is offered. They use an explicit representation of the set of shape parameterisations {Φi } and optimise the model directly with respect to {Φi } using genetic algorithm search. Their representation of {Φi } is, however, problematic and does not guarantee a diffeomorphic mapping. They correct the problem when it arises by reordering correspondences, which is workable for 2D shapes but does not extend to 3D. Although some of the results produced by their method are better than hand-generated models, the algorithm did not always converge.
54
4
Rhodri H. Davies, Tim F. Cootes, and Chris J. Taylor
An Information Theoretic Objective Function
We wish to define a criterion for choosing the set of parameterisations {Φi } that are used to construct a statistical shape model from a set of training boundaries {Si }. Our aim is to choose {Φi } so as to obtain the ‘best possible’ model. Since we wish to obtain a compact model with good generalisation properties we define the ‘best’ model as that which can account for the observations (the training boundaries) in as simple a way as possible. We formalise this by stating that we wish to find {Φi } that minimises the information required to code the whole training set to some accuracy δ on each of the elements of {xi }. Note that to describe {xi } to arbitrary accuracy would require infinite information; δ should be chosen to reflect the measurement errors involved in acquiring the training boundaries. 4.1
Description Length for a Set of Shape Vectors
Suppose we have a set {Si } of s training shapes that are parameterised using {Φi } and sampled to give a set of n-dimensional shape vectors {xi }. Following (1) we can approximate {xi } to an accuracy of δ in each of its elements using a linear shape model of the form xi = x ¯ + Pbi +ri
(2)
Where x ¯ is the mean of {xi }, P has t columns which are the t eigenvectors of the covariance matrix of {xi } corresponding to the t largest eigenvalues λj , bi is a vector of shape parameters, and ri is a vector of residuals. The elements n λj over the of ri can be shown to have zero mean and a variance of λr = n1 j=t+1
training set. The total information required to code the complete training set using this encoding is given by IT otal = IModel + sIb + sIr
(3)
Where IModel is the information required to code the model (the mean vector, x ¯, and the eigenvectors of the covariance matrix P), Ib is the average information required to code each parameter vector bi , and Ir the average information required to code each residual vector, ri . For simplicity, we assume that the elements of the mean x ¯ and the matrix P are uniformly distributed in the range [-1,1], and that we use km bits per element for the mean and kj bits per element for the j th column of P giving quantisation errors δm = 2−km and δj = 2−kj respectively. Thus
IModel = nkm + n
t j=1
kj
(4)
A Minimum Description Length Approach to Statistical Shape Modelling
55
The elements of bi are assumed to be normally distributed over the training set with zero mean and variance λj . To code them to an accuracy δb , we require on average
Ib =
t
[kb + 0.5 log(2πeλj )]
(5)
j=1
Where kb = −log(δb ), see Appendix A for details. All logs are base 2. Similarly, to code the n elements of ri to an accuracy of δr = 2−kr we require on average Ir = n[kr + 0.5 log(2πeλr )]
(6)
Substituting (4), (5) and (6) into (3) we obtain
IT otal = nkm + n
t
kj + s
j=1
t
[kb + 0.5 log(2πeλj )] + sn[kr + 0.5 log(2πeλr )]
j=1
(7) 4.2
Minimum Description Length
IT otal is a function of the quantisation parameters km , kj , kb , and kr , which are related to δ, the overall approximation error. Since we wish ultimately to minimise IT otal with respect to {Φi } we need first to find the minimum with respect to the quantisation parameters. This can be found analytically, leading to an expression in terms of s, n, k, t, {λj } and λr . IT otal = −0.5(n + nt + st) log(12αλr /s) + snk t log(λj ) + 0.5ns log(αλr ) +0.5(n + s) j=1
+0.5s(n + t) log(2πe) − 0.5st log(s)
(8)
ns where α = ( n(s−1)−t(n−s) ) The details of this derivation are given in Appendix B. Thus, for a fixed number of modes, t, to optimise IT otal we need to minimise
F = (n + s)
t
log(λj ) + [n(s − 1) − t(n + s)] log(λr )
(9)
j=1
Note that this is independent of δ. Finally, the number of modes, t, should be chosen to minimise IT otal . Since t must be an integer, this can be achieved
56
Rhodri H. Davies, Tim F. Cootes, and Chris J. Taylor
using a simple, exhaustive search. Note, however, that the average information required to code bj , the j th element of the shape vector bi , is kb +0.5 log(2πeλj ). This must be greater than zero, which imposes an upper bound on t such that λt > 12αλr /(2πe).
5
Representation and Optimisation of {Φi }
We wish to find the global optimum of F in (9) with respect to the set of shape parameterisations {Φi }. Our approach is to use an explicit representation of {Φi } coupled with a stochastic search to find the optimum. We require a representation of {Φi } that ensures a diffeomorphic mapping between each pair of training shapes. In 2D this can be achieved by enforcing the ordering of corresponding points around the training shapes. In 3D, however, no such ordering exists. We have developed a new method of representation that guarantees diffeomorphic mapping without using an explicit ordering constraint. Here we describe the method for 2D shapes; Appendix C explains how it can be extended to 3D. We define a piecewise linear parameterisation for each training shape by recursively subdividing boundary intervals by inserting nodes. The position of each new node is coded as the fraction of the boundary path length between neighbouring nodes - thus by constraining the subdivision parameters to the range [0,1] we can enforce a hierarchical ordering where, at each level of the hierarchy, nodes are positioned between those already present. This is illustrated by the example in Fig. 2 which demonstrates the parameterisation of a circle.
Fig. 2. A diagram that demonstrates the parameterisation of a circle. The squares represent the landmarks that are already in place. The parameter values are: Φi = (Origin, 0.65(0.65(0.4, 0.8), 0.8(0.5, 0.2)) Recursive subdivision is continued until an arbitrarily exact parameterisation is achieved. Correspondence is assumed across the whole training set between equivalent nodes in the subdivision tree. We can manipulate a set of these parameterisations {Φi } in order to optimise our objective function F . In practice the search space is high-dimensional with many local minima leading us to prefer a stochastic optimisation method such as simulated annealing [15] or genetic algorithm search [8]. We chose to use a genetic algorithm to perform the experiments reported below.
A Minimum Description Length Approach to Statistical Shape Modelling
6
57
Results
We present qualitative and quantitative results of applying our method to several sets of outlines of 2D biomedical objects. We also investigate how our objective function behaves around the minimum and how it selects the correct number of modes to use. 6.1
Results on 2D Outlines
We tested our method on a set of 17 hand outlines, 38 left ventricles of the heart, 24 hip prostheses and 15 outlines of the femoral articular cartilage.
←− ±2σ −→
←− ±2σ −→
←− ±2σ −→
←− ±2σ −→
Fig. 3. The first three modes of variation of the automatically generated models. Each row shows the variation (±2σ) of a mode. The top rows are the principal modes with the largest variation.
58
Rhodri H. Davies, Tim F. Cootes, and Chris J. Taylor
In figure 3, we show qualitative results by displaying the variation captured by the first three modes of each model (first three elements of b varied by (2±[standard deviations over training set ]). We also give quantitative results in figure 4, tabulating the value of F , the total variance and variance explained by each mode for each of the models, comparing the automatic result with those for models built using manual landmarking and equally spaced points. The quantitative results in figure 4 show that the automatically generated models are significantly more compact than both the models built by hand and by equally-spacing points. It is interesting to note that the model produced by equally spacing landmarks on the hip prostheses is more compact than the manual model. This is because equally-spaced points suffice as there is little variation, but errors in the manual annotation adds extra noise that is captured as statistical variation. Hip Prostheses
Hands Mode 1 2 3 4 5 6 VT F
Hand Built 9.34 5.12 2.41 1.38 0.67 0.49 20.68 43020
Equally-spaced 20.74 7.4 5.13 3.15 1.71 1.21 41.21 44114
Automatic 8.44 4.61 2.1 1.36 0.44 0.34 18.64 41304
Mode 1 2 3 4 5 6 VT F
Hand Built 4.01 1.24 0.71 0.63 0.60 0.51 8.21 30401
Hand Built 1.89 1.36 0.57 0.49 0.17 0.14 4.98 10541
Equally-spaced 2.48 1.21 0.70 0.49 0.32 0.17 6.15 11014
Automatic 3.73 0.98 0.7 0.55 0.51 0.48 7.1 27989
Knee Cartilage
Heart Ventricles Mode 1 2 3 4 5 6 VT F
Equally-spaced 3.81 1.04 0.61 0.54 0.5 0.48 7.88 28123
Automatic 1.97 1.13 0.66 0.34 0.18 0.13 4.68 7348
Mode 1 2 3 4 5 6 VT F
Hand Built 6.29 4.10 2.01 1.86 1.65 1.28 20.23 18023
Equally-spaced 6.54 4.42 2.33 1.87 1.66 1.24 21.01 18514
Automatic 5.91 4.82 2.37 1.65 1.58 1.43 19.04 17494
Fig. 4. A quantitative comparison of each model showing the variance explained by each mode. F is the value of the objective function and VT is the total variance.
6.2
The Behaviour of F
To demonstrate the behaviour of our objective function we took some landmarks from the automatically generated hand model and added random noise to each one. Figure 5 shows a plot of F against the standard deviation of the noise. The plot shows that as the landmarks are moved further away from their original positions, the value of F increases - as expected.
A Minimum Description Length Approach to Statistical Shape Modelling
6.3
59
Selecting the Number of Modes
We used the automatically generated heart model to show how the number of modes affects the value of the objective function. Figure 6 shows a plot of F against the number of modes used in the model. The values form a quadratic with a minimum at nine modes which captures approximately 93% of the total variation. 4
5.8
2
x 10
5.6 1.8
5.4 1.6
5.2 1.4 F
F
5
4.8
1.2
4.6 1
4.4 0.8
4.2
4
0
0.5
1
1.5
2 2.5 3 Std. dev on each point
3.5
4
4.5
Fig. 5. How noise on the landmarks affects the value of the objective function.
7
0.6
5
0
5
10 15 Number of Modes
20
25
Fig. 6. The values of F for a model built with a different number of modes.
Discussion and Conclusions
We have derived an objective function that can be used to evaluate the quality of a statistical shape model. The expression we use has a theoretical grounding in Information Theory, is independent of quantisation error and unlike other approaches [10, 16],does not involve any arbitrary parameters. The objective function includes a log(λi ) term which is equivalent to the product of the λi ’s, (and thus the determinant of the covariance matrix) as used by Kotcheff and Taylor [16], but the more complete treatment here shows that other terms are also important. As well as providing good results when used as an objective function for automatically building statistical shape models, the function may also be used to calculate the correct number of modes to use for a given model. If we implicitly optimise the number of modes, however, the number of false minima increases which means the genetic algorithm requires a larger population to find a satisfactory solution. We have described a novel representation of correspondence that enforces a diffeomorphic mapping and is applicable in 2D and 3D. Although a formal proof
60
Rhodri H. Davies, Tim F. Cootes, and Chris J. Taylor
is beyond the scope of this paper, the representation reduces the search space, which improves the convergence properties of the search. As with any stochastic optimisation technique, our search requires a large number of function evaluations. The results in this paper typically took several hours to produce. Although this is a one-off, off-line process, it is likely to become impractical when a larger set of training shapes is used because its complexity is at least O(s2 ). We are currently working on finding faster ways of locating the optimum. Although we have not yet implemented the method in 3D, the results on 2D objects offer a significant improvement over those from a hand-built model. The method we have described in this paper only optimises the shape parameterisation. We also intend to consider the pose of each shape in the search, as this is likely to affect the information content of the model.
A Information Content of a Scalar Suppose a variable x can take n discrete values xi (i = 1 . . . n), with a corresponding probability pi . The entropy, or information needed to transmit a value of x is given by
Hd = −
n
pi log(pi )
(10)
i=1
If the log is base 2, then Hd has units of bits. Now, suppose x is a continuous variable with a p.d.f. p(x). Now the entropy is defined as Hc = −
∞
p(x)log(p(x))dx
(11)
−∞
(10) and (11) are not, however, directly comparable since, to transmit a real number to an arbitrary accuracy may require an infinite number of bits. Suppose now what we approximate the continuous case with discrete x’s that can take on the values xi = iδ(i = 1 . . . n), with a probability given by the continuous p.d.f p(x), thus pi ≈ δp(iδ). Now Hd ≈ Hc − logδ
(12)
Therefore, if we discretise x in steps of 2−k , then H d ≈ Hc + k
(13)
A Minimum Description Length Approach to Statistical Shape Modelling
61
For example, if all values are in the range [0, R], then p(x) = 1/R and Hc = log(R) so Hd = k + log(R). This agrees with the value obatained from (10), as there are 2−k R possible states, all equall likley. Alternatively, if x is distributed as a Gaussian with variance σ 2 , it can be shown that Hc = 0.5log(2πeσ 2) [18], so the number of bits required to transmit a value of x, disretised in steps of 2−k , is [k + 0.5log(2πeσ 2 )].
B Quantisation Effects IT otal in (7) is a function of the quantisation parameters δm , {δj }, δb and δr . Since we wish ultimately to minimise IT otal with respect to {Φi } we need first to find the minimum with respect to these parameters. First, we need to determine what quantisations δm , {δj }, δb and δr are required to achieve a quantisation error δ = 2−k in the final reconstruction. We assume that by quantising a parameter, we effectively add noise to that parameter. We have used error propagation to estimate the effects of noise on the final reconstruction. ¯ induces In our linear model (2), noise of variance σ 2 on the elements of x noise of variance σ 2 on xi . Similarly, noise of variance of σ 2 on the elements of bi can be shown to induce an average noise of variance σ 2 /2 on the elements of xi . Noise of variance σ 2 on the elements of the j th column of P induces an average noise of variance λj σ 2 on each element of xi . Quantising a value to δ induces noise with a flat distribution in [−δ/2, δ/2] ¯ , P, and bi , causes an additional and thus a variance of δ 2 /12. Thus quantising x error that must be corrected by the residual term, ri . In effect, the variance of the residual is increased from the original λr . Considering this, the variance on the elements of the residual is given by
λ/r = λr +
t 1 2 t 2 (δm + δb + λj δj2 ) 12 2n j=1
(14)
Using the central limit theorem we assume that the residuals are normally distributed. λr is substituted for λr in (7) giving IT otal = nkm + n
t
kj
j=1
+ s
t
[kb + 0.5 log(2πeλj )] + sn[k + 0.5 log(2πeλ/r )]
(15)
j=1
We can now find the minimum of IT otal with respect to δm , {δj }, δb and δr . By equating the differentials to zero, we can show that at the optimum δj2
=2
−2kj
δb2
2 = 2−2km = 12λr /s δm
= =
12λr /(sλj ) = 2−2kb = 12λr
2 δm /λj 2 = sδm
(16) (17) (18)
62
Rhodri H. Davies, Tim F. Cootes, and Chris J. Taylor
Substituting (16), (17) and (18) into (14) gives ns ) λ/r = αλr where α = ( n(s − 1) − t(n − s)
C Extension to 3D In this appendix, we describe how our representation of the parameterisation can be extended to surfaces. Our ultimate goal is to build 3D statistical shape models of biomedical objects. Our training data are parallel stacks of shape outlines, segmented from slices of 3D Magnetic Resonance Images (MRI). We can interpolate these outlines to form a triangulated surface using the algorithm of Geiger [7]. As in 2D, we define an explicit representation of the parameterisation by recursively subdividing the surface by inserting nodes. The position of each new node is defined as its position inside the triangle formed by its three neighbouring nodes. As these triangles edges are ambiguous on the original surface, we can flatten the surface to that of a sphere using the methods of Angenent at al or Hurdal at al [1, 12]. These methods flatten each surface using conformal mappings. As these mapping are diffeomorphisms, each point on the original surface has a unique, corresponding point on the sphere. This allows us to subdivide the surface using spherical triangles. An example is shown in figure 7. We iterate until we have a sufficient number of landmarks on the sphere. They can then be projected onto the shape’s surface using the inverse of the conformal mapping and evaluated using our objective function.
Fig. 7. A diagram to demonstrate how each point is added to the surface. The hollow points are those that are already fixed on the surface. This is a simplification, as on the sphere, the triple of points would form a spherical triangle.
Acknowledgements The authors would like to thank Dr. Alan Brett for his contribution to the ideas for the work in this paper. Tim Cootes is funded under an EPSRC Advanced Fellowship Grant. Rhodri Davies would like to thank the BBSRC and AstraZeneca Pharmaceuticals 1 for their financial support. 1
AstraZeneca Pharmaceuticals, Alderley Park, Macclesfield, Cheshire, UK
A Minimum Description Length Approach to Statistical Shape Modelling
63
References [1] Angenent, S., S. Haker, A. Tannenbaum and R. Kikinis: On the laplace-beltrami operator and brain surface flattening. IEEE Trans. Medical Imaging, 1999. 18: p. 700-711. [2] Baumberg, A. and D. Hogg, Learning Flexible Models from Image Sequences, in European Conference on Computer Vision, Stockholm, Sweden. 1994. p. 299-308. [3] Benayoun, A., N. Ayache, and I. Cohen. Adaptive meshes and nonrigid motion computation. in International Conference on Pattern Recognition. 1994. Jerusalem, Israel. [4] Bookstein, F.L., Landmark methods for forms without landmarks: morphometrics of group differences in outline shape. Medical Image Analysis, 1997. 1(3): p. 225243. [5] Cootes, T., A. Hill, C. Taylor, and J. Haslam, The use of Active shape models for locating structures in medical images. Image and Vision Computing, 1994. 12: p. 355-366. [6] Cootes, T., C. Taylor, D. Cooper and J. Graham, Active shape models - their training and application. Computer Vision and Image Understanding, 1995. 61: p. 38-59. [7] Geiger, B., Three-dimensional modelling of human organs and its application to diagnosis and surgical planning, . 1993, Technical Report, INRIA, France. [8] Goldberg, D.E., Genetic Algorithms in Search, Optimisation and Machine Learning. 1989: Addison Wesley. [9] Goodall, C., Procrustes Methods in the Statistical Analysis of Shape. Journal of the Royal Statistical Society, 1991. 53(2): p. 285-339. [10] Hill, A. and C. Taylor. Automatic landmark generation for point distribution models. in British Machine Vision Conference. 1994. Birmingham, England: BMVA Press. [11] Hill, A. and C.J. Taylor, A framework for automatic landmark identification using a new method of non-rigid correspondence. IEEE Transactions on Pattern Analysis and Machine Intelligence, April, 2000. [12] Hurdal, M. K, P. L. Bowers, K. Stephenson, D. W. L. Sumners, K. Rehm, K. Schaper, D. A. Rottenberg, Quasi-conformally flat mapping the human cerebellum, MICCAI’99, p. 279-286, [13] Kambhamettu, C. and D.B. Goldgof, Point Correspondence Recovery in Non-rigid Motion, in IEEE Conference on Computer Vision and Pattern Recognition. 1992. p. 222-227. [14] Kelemen, A., G. Szekely, and G. Gerig, Elastic model-based segmentation of 3-D neuroradiological data sets. IEEE Transactions On Medical Imaging, 1999. 18(10): p. 828-839. [15] Kirkpatrick, S., C. Gelatt, and M. Vecchi, Optimization by Simulated Annealing. Science, 1983. 220: p. 671-680. [16] Kotcheff, A.C.W. and C.J. Taylor, Automatic Construction of Eigenshape Models by Direct Optimisation. Medical Image Analysis, 1998. 2: p. 303-314. [17] Rangarajan, A., H. Chui and F. L. Bookstein,The Softassign Procrustes Matching Algorithm, in 15th IPMI 1997. p. 29-42. [18] Therrien, C.W., Decision Estimation and Classification. 1989: John Whiley and Sons. [19] Wang, Y., B. S. Peterson, and L. H. Staib. Shape-based 3D surface correspondence using geodesics and local geometry. CVPR 2000, v. 2: p. 644-51.
Multi-scale 3-D Deformable Model Segmentation Based on Medial Description Sarang Joshi, Stephen Pizer, P. Thomas Fletcher, Andrew Thall, and Gregg Tracton Medical Image Display & Analysis Group, University of North Carolina at Chapel Hill, Chapel Hill NC 27514
[email protected]
Abstract. This paper presents a Bayesian multi-scale three dimensional deformable template approach based on a medial representation for the segmentation and shape characterization of anatomical objects in medical imagery. Prior information about the geometry and shape of the anatomical objects under study is incorporated via the construction of exemplary templates. The anatomical variability is accommodated in the Bayesian framework by defining probabilistic transformations on these templates. The modeling approach taken in this paper for building exemplary templates and associated transformations is based on a multi-scale medial representation. The transformations defined in this framework are parameterized directly in terms of natural shape operations, such as thickening and bending, and their location. Quantitative validation results are presented on the automatic segmentation procedure developed for the extraction of the kidney parenchyma-including the renal pelvis-in subjects undergoing radiation treatment for cancer. We show that the segmentation procedure developed in this paper is efficient and accurate to within the voxel resolution of the imaging modality.
1
Introduction
Modern anatomic imaging technologies are enabling extremely detailed study of anatomy, while the development of functional imaging modalities are providing detailed in vivo associated information regarding the physiological function. While modern imaging modalities provide exquisite imagery of the anatomy and its function, automatic segmentation of these images and the precise quantitative study of the biological variability exhibited in these images continues to pose a challenge. In this paper we present a multi-scale medial framework based on deformable templates[7],[5],[16] for the automatic extraction and analysis of the shape of anatomical objects from the brain and abdomen, imaged respectively via MRI and CT. The multi-scale deformable template approach is based on the medial axis representation of objects first proposed by Blum [3] for studying shape. The approach presented herein is an extension of the early work by Pizer[13] and Firtsch[6] in 2D on deformable medial representation of objects. We adopt a Bayesian approach of incorporating prior knowledge of the anatomical variations and the variation of the imaging modalities. Following the M.F. Insana, R.M. Leahy (Eds.): IPMI 2001, LNCS 2082, pp. 64–77, 2001. c Springer-Verlag Berlin Heidelberg 2001
Multi-scale 3-D Deformable Model Segmentation
65
deformable templates paradigm, we incorporate prior information about the geometry and shape of the anatomical objects under study via the construction of exemplary templates. The infinite anatomical variability is accommodated in the Bayesian framework by defining probabilistic transformations on these templates[7]. The segmentation problem in this paradigm is that of finding the transformation S of the template, that maximizes the posterior, P (S|data) ∝ P (data|S)P (S) , where P (S) is the prior probability function capturing prior knowledge of the anatomy and its variability, and P (data|S) is the data likelihood function capturing the image data-to-geometry relationship. For efficiency of implementation we equivalently maximize the log-posterior given by LogP (S|data) = LogP (data|S) + LogP (S|data) , up to an additive constant. The modeling approach taken in this paper for building exemplary templates and associated transformations is based on a multi-scale medial representation. The transformations defined in this framework are parameterized directly in terms of natural shape operations, such as thickening and bending, and their location. This multi-scale approach has many stages of scale, at each of which the geometric primitives are intuitive for that scale and have the property that their spacing is comparable to the linear measure of the size of space (modeling aperture) that they summarize. This leads to a spatial tolerance that successively decreases with scale level. A Markov Random Field approach, described in detail in [14] is used to defining the energetics of the log probabilities needed for the posterior. The log probabilities at a given scale are not only conditioned on a neighborhood at that scale, but conditioned on the result of the next larger scale. The posterior at each scale can then be separately optimized successively decreasing the scale. The multi-scale nature of our approach allows for the investigation of these properties at various scales from the coarse scale of entire body sections to the fine scale on the order of the resolution of the imaging modality. The intuitiveness derives from the ability to have many of the levels of scale describe medial properties. In addition, the size properties derived from medial description allow the creation of natural levels of scale each suited for shape description at that scale level. The next two sections discuss the medial representation of objects. Section 3 discusses the deformation of models to fit image data and the geometric measures used in the log prior term measuring geometric typicality. Section 4 discusses the log likelihood term measuring the match of a deformed model to a target image, and Section 5 gives segmentation results using this method.
2
Medial Representation of Objects
Many authors in image analysis, geometry, human vision, computer graphics, and mechanical modeling have come to the understanding that the medial relationship between points on opposite sides of a figure is an important factor in
66
Sarang Joshi et al.
the objects shape description. Biederman [1], Marr [11], Burbeck [4], Leyton [9], and others have produced psychophysical and neurophysiological evidence for the importance of medial relationships (in 2D projection) in human vision. The medial geometry has also been explored in 3D by Nackman [12], and Siddiqi [15], and medial axis modeling techniques have been applied by many researchers, including Bloomenthal [2], Igarashi [8] and Markosian [10]. Of these, Bloomenthal skeletal-based soft-objects; Igarashi used a medial spine in 2D to generate 3D surfaces from sketched outlines; and Markosian used implicit surfaces generated by skeletal polyhedra. Our representation, described in [Pizer 1999], expands the notion of medial relations from that of a medial atom implying boundaries by including of a widthproportional tolerance and by using a width-proportional sampling of the medial manifold in place of a continuous representation. The advantages, relative to the ideas of medial axis descended from Blum [1967], are in representational and computational efficiency and in stability with respect to boundary perturbation. Associating a tolerance with the boundary position provides opportunities for stages of the representation with successively smaller tolerance. Representations with large tolerance can ignore detail and focus on gross shape, and in these large-tolerance stages, discrete sampling can be coarse, resulting in considerable efficiency of manipulation and presentation. Smaller-tolerance stages can focus on retirements of the larger-tolerance stages and thus more local aspects. The medial representation used in this paper called m-rep, is based on a hierarchical representation of linked figural models, defined at coarse scale by a hierarchy of figures protrusions, indentations, neighboring figures, and included figures which represent solid regions and their boundaries simultaneously. The linked collection of figural components imply a fuzzy, i.e., probabilistically described boundary position with a width-proportional tolerance. At small scale these figural boundaries are made precise by displacing a dense sampling of the m-rep implied boundary. A model for a single figure is made from a net, (a mesh or a chain) of medial atoms; each atom describing not only a position and width, but also a local figural frame giving figural directions, and an object angle between opposing, corresponding positions (medial involutes) on the implied boundary. A figure can be expressed as a sequence over scale of nets, implying successively refined (smaller tolerance) versions of the figural boundary. 2.1
Single Figure Description via m-rep
We now describe the representation of single figural forms. Our representation is based on the notion of medial involutes of Blum [1967] and starts with a parameterization of a medial atom m that locally implies opposing figural boundaries as illustrated in Fig. 1. The medial atom m by itself not only implies two opposing sections of boundary, but as well the solid region between them. Medial atoms on the interior of the medial manifold are defined as a four tuple m = {x, r, F , θ}, consisting of:
Multi-scale 3-D Deformable Model Segmentation
67
1. x ∈ IR3 , the skeletal position, 2. r ∈ IR+ , the local width defined as the distance from the skeletal position of two or more implied boundary positions, 3. F ∈ SO(3) the local frame parameterized by (n, b, b⊥ ), where n is the normal to the medial manifold , b is the direction in the tangent plane of the fastest narrowing of the implied boundary sections, 4. θ ∈ [0, π2 ] the object angle determining the angulation of the implied sections of boundary relative to b. The two opposing boundary points implied by the medial atom are given by y = x + p and y = x + s. The vectors p and s are given by
p = rR(b,n) (θ)b
,
s = rR(b,n) (−θ)b ,
where R(b,n) (θ) is a rotation by θ in the (b, n) plane.
Fig. 1. A medial atom defined by the 4-tuple {x, r, F, θ} with involutes P and S perpendicular to the implied surface.
For stability at the ends in image matching, medial atoms on the boundary of the medial manifold also include an extra parameter η that captures the elongation of the edge away from a spherical end cap. The end section of the medially implied boundary is as a parametric curve form one involute to the other passing through the point x + ηrb and orthogonal to b. The curve c(t) parametrized by t ∈ [−1, 1] is defined by c(t) = x + rη(t)R(b,n) ((1 − t)θ))p , where η(t) = (cos(tπ) + 1) with θ being the object angle.
(η − 1) +1 , 2
68
Sarang Joshi et al.
In the above representation x gives the central location of the solid section of figure that is being represented by the atom m. The scalar r gives the local scale and size of the solid section of figure that is being represented by the atom. The object angle θ and the direction b also define the gradient of the scalar field r via ∇r = −b cos θ . The scalar field r also provides a local ruler for the precise statistical analysis of the object. There are three basic types of medially defined figural segments with corresponding medial manifolds M of dimension 0, 1, 2 respectively. Figural segments with two dimensional medial manifolds represent slab-like segments, tube-like segments, where the medial manifold is an one dimensional space curve, and spherical segments, where the medial manifold consists of a single point. Shown in Fig. 2 are examples of slab like and tubular figures. In this paper we will focus on slab-like segments having 2-dimensional medial manifolds discretized into a net of medial atoms. For easy of implementation we have been using a quadrilateral mesh of discretized medial atoms mki,j ∈ M , (i, j) ∈ [1, N ] × [1, M ] for approximating the continuous medial manifold at particular scale k with tolerance and the level of discretization inversely proportional to scale with the final scale having tolerance on the order of the resolution of the imaging modality. We define a medial scale space by a sequence of successive refinement of medial nets defined via offsets from a spline interpolation of medial atoms from the scale above. 2.2
Spline Interpolation of Medial Atoms
Given a quadrilateral mesh of medial atoms mi,j , (i, j) ∈ [1, · · · , N ] × [1, · · · , M ] we define a continuous medial surface via a B´ezier interpolation of the discretely sampled medial atoms. The medial position x(u, v), u ∈ [i, i + 1], v ∈ [j, j + 1] is defined via a bicubic polynomial interpolation of the form x(u, v) =
3
dm,n um v n
m,n=0
with dm,n are chosen to satisfy the known normal/tangency and continuity conditions at the sample points xi,j . Given the interpolation of the medial positions the radius function r(u, v) is also interpolated as a bicubic scalar field on the above interpolated medial manifold given r and ∇r at the mesh points points xi,j . Having interpolated r and its gradient, the frame F and the object angle θ are defined via the relation ship ∇r = −b cos θ
Multi-scale 3-D Deformable Model Segmentation
69
Fig. 2. Top rows shows an example of a slab like figure with 2 dimensional medial manifold. Shown in the bottom row is tubular figure with 1 dimensional medial manifold. 2.3
Figural Coordinate System
The prior (geometric typicality) measure requires geometrically consistent correspondence between boundary points in the model and those in a deformed model. The likelihood (deformed model to target image match) measure requires correspondence between template intensities at positions in 3-space relative to the model and target image intensities at positions in 3-space relative to the deformed model. Both of these correspondences are made via the medial geometry. The continuous medial manifold of a figure, defined via the spline interpolation describe above, is parameterized by (u, v), with u and v taking the atom index numbers at the discreet mesh positions. A parameter t ∈ {−1, 1} designates the side of the medial manifold on which an implied boundary point lies. As described in section 2.1, t varies continually between −1 and 1 as the implied boundary point moves around the crest of the object from one side of the medial axis to another. For single figures boundary correspondences are defined via the common parameterization (u, v, t). Positions in the image in the neighborhood of the implied boundary are inˆ where (u, v, t) is the parameterization of the closest point on dexed by (u, v, t, d), the medially implied boundary and dˆ is the signed distance (interior = negative, exterior = positive) from the boundary in multiples of the local radius r of the medial point at (u, v).
70
2.4
Sarang Joshi et al.
Connecting m-reps Figures into Objects
As illustrated in Fig. 3, protrusion and indentation figures combine into objects in a hierarchical fashion, with the same Boolean operators of union and difference as with Constructive Solid Geometry models, but here recognizing the tolerance of the figures. A figure may be separated from all other figures, or it may be the parent of one or more attached sub-figures: protrusion and/or indentation. A sub-figure on a slab or tube or sphere may be a slab or tube. The interior of a protrusion sub-figure is combined with the parent by union of their interiors with the modification that the boundaries may smoothly blend. An indentation subfigure subtracts its interior from its parent, in the set theoretic sense, again with smooth blending. As illustrated in Fig. 3, a slab protrusion or indentation on a figure has a segment of its medial meshs end atoms that are at the open end of the figure and on the implied boundary of the parent, where the subfigure attaches to its parent. If the subfigure is a tube, it has a single open-end atom where the tube is attached to its parent, and a closed end atom at the other end. We call these the hinge atoms. The remaining end atoms form the closure of that figure. We intersect the subfigures interpolated medial mesh with the implied boundary of the parent figure. In what is presented herein we will concentrate on single figure objects.
Fig. 3. Fig. showing the medial mesh of protrusion sub figure with hinge atoms and the resulting blended implied surface.
2.5
Construction of m-rep Figures
Using the visualization and computer aided design techniques developed, we have built numerous models of anatomical objects. In this paper we focus on the automatic segmentation of the kidney as imaged in CT for radiation treatment for cancer. Shown in Fig. 4 is the template m-rep model of the kidney built from a CT of the abdomen.
Multi-scale 3-D Deformable Model Segmentation
71
Fig. 4. Fig. showing the m-rep model of the template kidney. The left panel shown the medial atoms and the implied surface. The right panel shows the model overlaid on the associated CT imagery.
3
Transformation of m-reps Figures
Having defined the construction of typical anatomical objects via m-rep figures, anatomical variability is accommodated by defining a cascade of transformations S k , k = 0, · · · , N increasing in dimensionality. These transformations are applied globally to the entire object as well as locally to individual atoms at various scales. Each transformation is applied at its own level of locality to each of the primitives appearing at that level. At each level of locality by the Markov random field framework the primitive is related only to immediately neighboring primitives at that level. Each level’s result provides both a initial value and a prior for the primitives at the next smaller scale level. The transformation at the last (smallest) scale level is finally a dense displacement field applied to the boundary of the figure on the scale of the voxel resolution of the imaging modality. 3.1
Object-Level Similarity Transformation
To begin with, a similarity transformation S 0 = (α, O, t) ∈ [(IR+ ×SO(3)) n IR3 ] is defined on the scale of the entire object and is applied to the whole medial manifold M. The similarity transformation S 0 scales, translates and rotates equally all the medial atoms of the object, that is m1i,j = S 0 ◦ mi,j = {αOxi,j + t, αr, O ◦ F , θ} . Notice that the similarity transformation does not affect the object angle. As the medial representation is invariant under the similarity transformation, this is
72
Sarang Joshi et al.
equivalent to applying the similarity transformation S 0 to the implied boundary B of the medial mesh to yield the transformed boundary B 1 . A prior is induced on the above defined transformation based on the displacement of the implied boundary of the objects. Throughout, an independent Gaussian prior on boundary displacement is used with variance proportional to the local radius r. For the whole object similarity transformation S 0 the log-prior becomes ||y − S 0 ◦ y||2 dy . LogP (S 0 ) = − 2(σr(y))2 B 3.2
Atom Level Transformation
Having accomplished the gross placement of the figure, attention is now focused on the sub-sections of the figure defined by each of the medial atoms. At this stage local similarity transformations as well as rotations of the local angulation, 1 = (α, O, t, β)i,j ∈ [(IR+ × SO(3)) n IR3 ] × [− π2 , π2 ] are applied to the medial Si,j atom, that is, 1 1 1 m2i,j = Si,j ◦ m1i,j = (αi,j Oi,j x1i,j + ti,j , αi,j ri,j , Oi,j ◦ F1i,j , θi,j + βi,j ) .
(1)
The resulting implied boundary is defined as B 2 . A prior on the local atom 1 transformations Si,j is also induced based on the displacement of the implied boundary with an additional Markov random field prior on the translations, guaranteeing the smoothness of the medial manifold. In keeping with the level 1 be the portion of the implied boundary affected by the atom of locality Let Bij 1 1 of the atom m1i,j becomes mi,j . The prior energy on the local transformation Si,j 2 n,m=1 ||ti,j − ti+n,j+m ||2 ||y − y || , dy − LogP (S 1 ) = − 1 − x1 1 (σr(y))2 ||x || Bi,j i,j i+n,j+m i,j n,m=−1 where y is the corresponding position on the figural boundary implied by the transformed atom m2 , and ti,j is the translation component of the local trans1 formation Si,j . Good association between points on the boundary y and the deformed boundary y is made using the figural coordinate system describe in section 2.3. The point y is the point on the deformed model having the same (u, v, t) coordinates as that of the original point y. The integral in the above prior is implemented as a discrete sum over a set of boundary points by defining a sampling of the (u, v, t) coordinate space and calculating the associated implied boundary before and after an atom deformation. 3.3
Dense Boundary Displacement Field Transformation
At the final stage the implied boundary of the figure is displaced in the normal direction using a dense displacement field defined on the implied boundary B 2 , y ∈ B 3 = y + n(y)d(y), y ∈ B 2 , where n(y) is the normal to the implied boundary at y ∈ B 2 .
Multi-scale 3-D Deformable Model Segmentation
73
As with the local atom transformations the prior is induced on the dense displacement field using a Markov random field prior derived from energetics associated with thin elastic membranes to guarantee smoothness. The log-prior on the displacement field d(y) becomes |d(y)|2 2 − |∇d(y)| dy (2) LogP (d(x)) = − 2 B2 (σr(y)) B2 The above above prior is implements via a discrete implementation as follows. Let yi ∈ B2 , i = 1, · · · , N be the set of discrete boundary points on the implied boundary B 2 . Let N (yi ) be the set of neighbors of the point yi . The discrete approximation of equation 2 becomes −
N |d(yi )|2 i=1
4
(σr(yi )
−
N i=1 j∈N (yi )
|d(yj ) − d(yi )|2 . ||yj − yi ||
Image Data Log-Likelihood
Having defined the transformation and the associated prior energetics, we now define the data likelihood function needed for defining the posterior. We have been defining the data likelihood functions, using the object centered coordinate system developed in section 2.3, by defining correlation functions between a predefine template image Itemp and the data Idata in the neighborhood of the boundary of the medially define object B. Leting δ be the size of the collar around the object, in multiples of r the local radius, the data log likelihood function becomes δ ˆ data (y , d)dyd ˆ Itemp (y, d)I dˆ , (3) −δ
B
ˆ ∈ IR3 is the point in the template image at distance rdˆ away from where (y, d) ˆ is the point in the data image at distance rdˆ the boundary point y, and (y , d) away from the boundary point y in the transformed object B . This association between points in the template image and the data image is made using the object coordinate system described in section 2.3. The image positions in the ˆ where (u, v, t) neighborhood of the implied boundary are indexed by (u, v, t, d), is the parameterization in the object centered coordinate system of the closest point on the medially implied boundary B, and dˆ is the signed distance (interior = negative, exterior = positive) from the boundary in multiples of the local radius r of the medial point at (u, v). In implementing the correlation defined in Eqn. 3 care must be taken in implementing the surface integral by a discrete voxel summation. The template image needs to be normalized by the determinant of the Jacobian associated with the implied model surface B. At model building time intensities in the template image Itemp are associated with their positions’ ˆ values. As the model deforms, a target image position is calculated (u, v, t, d)
74
Sarang Joshi et al.
ˆ value, using the deformed model, and the intensity for each template (u, v, t, d) interpolated at that target image position is associated with the corresponding template intensity. We have have been using two basic types of templates: an analytical template derived from the derivative of the Gaussian and an empirical template learned from an example image from which the template medial model was built. Using the data likelihood defined above and the prior defined in previous section, the log posterior is defined as a weighted sum of the two terms with weights chosen by the user. For optimizing the log-posterior with respect to the global object similarity transformation and the local atom-by-atom transformation, we have been using a genetic optimization algorithm. Genetic algorithms have the advantages of not being susceptible to local minimum and not requiring the computation of the derivative of the posterior with respective to the transformation parameters. For optimizing the posterior with respect to the dense displacement field d(bf y) we have been using a simple gradient decent algorithm.
5
Results
We have been using the automatic segmentation procedure for extracting the kidney parenchyma-including the renal pelvis-in subjects undergoing radiation treatment for cancer. Results from a series of three data sets are presented. Using a few seconds, the user rigidly place the template model in the subject data set. This initialization stage of the algorithm is followed by the hierarchical automatic segmentation which takes on the order of 5 minutes for convergence depending on the data set. At the first scale level, a object similarity transformation is estimated accommodating gross size and orientation differences between the template model kidney and the subject’s kidney. Fig. 5 compares the results of the similarity transformation to the clinical hand segmentation in the axial, coronal, and sagittal views through the kidney. The yellow contour of the resulting implied boundary is overlaid, for comparison with the clinical hand segmentation shown in red. Note that the clinical hand segmentation did not include the renal pelvis, while our single figure model of the kidney used in this study includes the renal pelvis. initial hand placement of similarity Fig. 5, shows the improvement in the segmentation as a result of the atom deformation process, thus accommodating more local object shape changes. The arrow in Fig. 6 highlights the improvement due to the final stage of the deformation, as the dense displacement field accommodates the fine featured variation in the shapes of the kidney. For quantitative comparisons of the segmentations of the method with manual segmentations, we have used two metrics from a geometric scoring package developed by Guido Gerig and Matthieu Jomier called VALMET : relative overlap and mean surface distance. The relative overlap measure is defined as the ratio of the intersection of the two segmentations divided by the union. Although the relative overlap is commonly used in the literature for scoring
Multi-scale 3-D Deformable Model Segmentation
75
Fig. 5. Axial (left), coronal (middle) and sagittal (right) slices through the subject kidney CT data set. The contours show the results of the object similarity transformation and the atom deformation. Notice the improvements in the results at the places marked.
Fig. 6. The improvement in the segmentation of the kidney after the dense displacement field deformation. The contours shows the results of the atom transformation of the dense displacement field deformation.
segmentations it is sensitive to the size of the object and not very effective in characterizing shape differences between two segmentations. The symmetric, mean surface distance Ds between the boundary of the two segmentations using Euclidean distance transforms of the segmentations is defined as follows. Let yi1 , i = 1, · · · , N ∈ B 1 and yj2 , j = 1, · · · , M ∈ B 2 be the boundary points of two segmenattions B 1 , B 2 ; the mean surface distance then is N M 1 1 1 Ds (B 1 , B 2 ) = min ||y 1 − yj2 || + min ||y 1 − yj2 || . 2 N i=1 j=1···M i M j=1 i=1···N i Shown in table 1 is the summary of the results from the study for the three data sets. The results shown above are typical of the three data sets and are form Data set 613. The segmentation improves at each stage of the algorithm for all three data sets. The accuracy of the segmentation as measured via the
76
Sarang Joshi et al.
mean surface distance is on the order of the resolution of the data set and on average within one pixel of the hand segmentation. Table 1. Table showing the relative overlaps and the mean surface distance between the manual segmentations and the automatic segmentations at the different stages of the hierarchical procedure for the three data sets processed. Data Set (cm)
Scale Level Relative Overlap Surface Distance (cm) Similarity Transformation 0.85 0.26 613 Atom deformation 0.86 0.23 0.15 × 0.15 × 0.5 Field deformation 0.90 0.16 Similarity Transformation 0.88 0.22 608 Atom deformation 0.89 0.19 0.2 × 0.2 × 0.4 Field deformation 0.93 0.14 Similarity Transformation 0.77 0.65 1402 Atom deformation 0.86 0.38 0.15 × 0.15 × 0.3 Field deformation 0.90 0.38
6
Discussion and Conclusion
It can be seen from the quantitative analysis of the segmentations that the accuracy of the automatic segmentation as measured via the average surface distance is on the order of the resolution of the imaging modality. Although these results show that our current methodology can segment structures in the abdomen such as the kidney with high level of accuracy, improvement can be expected from the change in the image template used in the data likelihood. All the results shown in this paper were generated using a Gaussian derivative template for the data-likelihood. We expect that the results would be substantially improved by the use of our already implemented but not yet tested training image template in place of the Gaussian derivative template that would allow a spatially varying template capturing the different gray scale characteristics of the kidney boundaries. This model to image match would be further improved a statistical model reflecting image intensity variations across a population of subjects. We have also been working on extending this frame work to the deformation of objects with multiple attached sub-figures and multiple objects with priors induced on the transformations that reflect the knowledge of the associated relative typical geometry.
7
Acknowledgement
We thank Prof. Gerig and Matthieu Jomier for the use of their scoring tool for the comparison of segmentation as well as for the many insightful discussions and comments. We would like to also thank Dr. Zhi Chen for the generating
Multi-scale 3-D Deformable Model Segmentation
77
the table comparing the segmentations. We also thank Prof. Ed. Chaney for providing us the data sets and invaluable insights. This work was supported by NIH Grants P01 CA47982 R01 CA67183 This research was carried out on computers donated by Intel.
References 1. Irving Biederman. Recognition-by-Components: A Theory of Human Image Understanding. Psychological Review, 94(2):115–147, 1987. 2. Jules Bloomenthal and Chek Lim. Skeletal methods of shape manipulation. In Proc. Shape Modeling and Applications, pages 44–47. IEEE, 1999. 3. H. Blum. A transformation for extracting new descriptors of shape. In Models for the Perception of Speech and Visual Form. MIT Press, 1967. 4. A. C. Burbeck, S M Pizer, B. S. Morse, D. Ariely, G. Zauberman, and J. Rolland. Linking object boundaries at scale: a common mechanism for size and shape judgments. In Computer Science Department technical report TR94-041, page 361:372, Chapel Hill, 1996. University of North Carolina. 5. T. Cootes, C. Taylor, D. Cooper, and J. Graham. Active shape models - their training and application. Computer Vision, Graphics, and Image Processing: Image Understanding, 1(61):38–59, 1994. 6. D. Fritsch, S. Pizer, L. Yu, V. Johnson, and E. Chaney. Segmentation of Medical Image Objects using Deformable Shape Loci. In International Conference on Information Processing in Medical Imaging, pages 127–140, Berlin, Germany, 1997. Springer-Verlag. 7. U. Grenander. General Pattern Theory. Oxford Univ. Press, 1994. 8. Takeo Igarashi, Satoshi Matsuoka, and Hidehiko Tanaka. Teddy: A sketching interface for 3d freeform design. Proceedings of SIGGRAPH 99, pages 409–416, August 1999. 9. M. Leyton. Symmetry, Causality, Mind. MIT Press, Boston, 1992. 620 pages. 10. Lee Markosian, Jonathan M. Cohen, Thomas Crulli, and John F. Hughes. Skin: A constructive approach to modeling free-form shapes. Proceedings of SIGGRAPH 99, pages 393–400, August 1999. 11. David Marr and H. K. Nishihara. Representation and recognition of the spatial organization of three-dimensional shapes. Proc. Roy. Soc. London Ser. B, 200:269– 294, 1978. 12. Lee R. Nackman. Three-Dimensional Shape Description Using the Symmetric Axis Transform. PhD thesis, UNC Chapel Hill, 1982. under the direction of Stephen M. Pizer. 13. S. Pizer, D. Fritsch, P. Yushkevich, V. Johnson, and E. Chaney. Segmentation, registration, and measurement of shape variation via image object shape. IEEE Transactions on Medical Imaging, 18:851–865, October 1999. 14. S.M. Pizer, T. Fletcher, Y. Fridman, D.S. Fritsch, A.G. Gash, J.M. Glotzer, S. Joshi, A. Thall, G Tracton, P. Yushkevich, and E.L. Chaney. Deformable M-Reps for 3D Medical Image Segmentation. In Review, ftp://ftp.cs.unc.edu/pub/users/nicole/defmrep3d.final.pdf, 2000. 15. Kaleem Siddiqi, Sylvain Bouix, Allen Tannenbaum, and Steven W. Zucker. The hamilton-jacobi skeleton. In Proc. Computer Vision, volume 2, pages 828–834. IEEE, 1999. 16. Alan Yuille and Peter Hallinan. Active Vision, chapter Deformable Templates. MIT Press, Cambridge, MA, 1992.
Automatic 3D ASM Construction via Atlas-Based Landmarking and Volumetric Elastic Registration Alejandro F. Frangi1 , Daniel Rueckert2 , Julia A. Schnabel3 , and Wiro J. Niessen1 1
3
Image Sciences Institute, University Medical Center Utrecht (UMC) Room E.01.334, Heidelberglaan 100, 3584 CX Utrecht, The Netherlands {alex,wiro}@isi.uu.nl 2 VIPG, Department of Computing, Imperial College, London, UK
[email protected] CISG, Radiological Sciences, Guy’s Hospital, King’s College London, UK
[email protected]
Abstract. A novel method is introduced that allows for the generation of landmarks for three-dimensional shapes and the construction of the corresponding 3D Active Shape Models (ASM). Landmarking of a set of examples from a class of shapes is achieved by (i) construction of an atlas of the class, (ii) automatic extraction of the landmarks from the atlas, and (iii) subsequent propagation of these landmarks to each example shape via a volumetric elastic deformation procedure. This paper describes in detail the method to generate the atlas, and the landmark extraction and propagation procedures. This technique presents some advantages over previously published methods: it can treat multiple-part structures, and it requires less restrictive assumptions on the structure’s topology. The applicability of the developed technique is demonstrated with two examples: CT bone data and MR brain data.
1
Introduction
Statistical models of shape variability [5] or Active Shape Models (ASM) have been successfully applied to perform segmentation and recognition tasks in twodimensional images. In building those statistical models, a set of segmentations of the shape of interest is required as well as a set of landmarks that can be defined in each sample shape. Manual segmentation and determining point correspondences are time consuming and tedious tasks. This is particularly true for three-dimensional applications where the number of slices to analyze and the amount of landmarks required to describe the shape increases dramatically with respect to two-dimensional applications. This work aims at automating the landmarking procedure while we still rely on the existence of a manual segmentation of the shapes. Several authors have proposed techniques to find point (landmark) correspondences but only a few of them have indicate or investigated their applicability in M.F. Insana, R.M. Leahy (Eds.): IPMI 2001, LNCS 2082, pp. 78–91, 2001. c Springer-Verlag Berlin Heidelberg 2001
Automatic 3D ASM Construction
79
the field of statistical shape models. Wang et al. [16] use a surface registration technique to find 3D point correspondences based on a metric matching surfaceto-surface distance, and surface normals and curvature. The authors suggest that this technique could be used to build 3D ASMs but they do not report any results on statistical model building. Kelemen et al. [9] report on the construction of 3D ASMs of neuroradiological anatomical structures. In this method the authors used a correspondence-by-parameterization approach to establish surface landmarks. The landmark correspondence is defined in the parameter domain of an underlying spherical harmonic parameterization. Although this approach has been used to build 3D ASMs, no explicit volumetric or surface registration between shapes takes place. To our knowledge, little work has been done on the automatic construction of 3D ASM using elastic registration [7,8,4,3]. The frameworks proposed by Brett and Taylor [4,3] are most closely related to this paper. In these approaches, each shape is first converted into a polyhedral representation. In the first approach [4], shape pairs are matched using a symmetric version of the Iterative Closest Point (ICP) algorithm of Besl and McKay [2]. Using this method, the authors were able to build 3D ASMs by automatically finding corresponding landmarks between surfaces. Surfaces are represented by means of dense triangulations that are matched via sparse triangulations (obtained by triangle decimation from the dense triangulations). The nodes of this sparse triangulation become the final landmarks. One problem acknowledged by the authors is the possibility of obtaining shape models with surface folding due to some landmark groups (triples) matched in different order between training examples. This is a consequence of the use of the ICP technique which does not incorporate connectivity constraints (purely local registration). In Brett and Taylor [3] this problem is overcome by transforming the surface to a planar domain by means of harmonic maps where connectivity constraints can be explicitly enforced. This technique avoids invalid cross-correspondences but is only applicable to single-part shapes that are topologically isomorphic to a disk. The work by Fleute and Lavall´ee [7,8] is also closely related to our work. They use a multi-resolution elastic registration technique based on octree-splines. This approach is a surface-based technique that registers shapes by minimization of a distance measure. In contrast to this, in this work we use a free-form elastic registration technique based on maximization of normalized mutual information (volume-based technique). In addition, we provide experiments giving empirical evidence of the convergence of the atlas generation procedure that is not analyzed in [7,8]. In this work a technique is introduced that addresses the shortcomings of point-based registration where no overall connectivity constraints are imposed. It uses a free-form elastic registration technique based on maximization of normalized mutual information (volume-based technique). Our method introduces global constraints by modifying the pairwise shape corresponder from a pointbased registration technique into a volume-based elastic registration technique. By construction, the deformation field is enforced to be smooth and the regu-
80
Alejandro F. Frangi et al.
larization term of the deformation will further penalize folding. In addition, our method can be applied to multiple-part shapes. The paper is organized as follows. In Section 2, our approach is described. In Section 3, results are presented that show the applicability of the method to modeling the radius in volumetric Computed Tomography (CT) data and the caudate nucleus in Magnetic Resonance Imaging (MRI); empirical evidence is given on convergence properties and reconstruction errors. Finally, Section 4 closes the paper with some conclusions and directions for future research.
2 2.1
Method Background
Suppose that we have n shapes described as vectors, {xi ; i = 1 · · · n}. Each shape consists of l 3-D landmarks, {pj = (p1j , p2j , p3j ); j = 1 · · · l} that represent the nodes of a surface triangulation. How to obtain those l 3-D landmarks is not a trivial issue and is precisely the topic of this paper. Each vector is of dimension 3l and is made up of the concatenation of the landmarks, i.e. xi = (p11 , p21 , p31 , p12 , p22 , p32 , · · · , p1l , p2l , p3l ). Moreover, it is assumed that the positions of the landmarks of all shapes are in the same coordinate system. These vectors form a distribution in a 3l-dimensional space. The goal is to approximate this distribution with a linear model of the form x=x ˆ + Φb
(1)
n where x ˆ = n1 i=1 xi is the average landmark vector, b is the shape parameter vector of the model, and Φ is a matrix whose columns are the principal n 1 ˆ)(xi − x ˆ)T . The princomponents of the covariance matrix S = n−1 i=1 (xi − x cipal components of S are calculated as its eigenvectors, φi , with corresponding eigenvalues, λi (sorted so that λi ≥ λi+1 ). If Φ contains the t eigenvectors corresponding to the largest eigenvalues, then we can approximate any shape of the training set, x, using Eqn. (1) where Φ = (φ1 |φ2 | · · · |φt ) and b is a t dimensional vector given by b = ΦT (x − x ˆ). The vector b defines the shape parameters of the ASM. By varying these parameters we can generate different instances of the shape class under analysis using Eqn. (1). Under the assumption that the cloud of landmark vectors follows a multi-dimensional Gaussian distribution, the variance of the i-th parameter, bi , across the training set √ is given by λi . By applying limits to the variation of bi , for instance |bi | ≤ ±3 λi , it can be ensured that a generated shape is similar to the shapes contained in the training class. 2.2
Overview
Ideally, a landmark is an anatomically characteristic point that can be uniquely identified on a set of shapes. However, anatomical landmarks are usually too
Automatic 3D ASM Construction
81
sparse to accurately describe a 3D shape. Therefore, we will consider pseudolandmarks, i.e. landmarks lying on the shape’s surface and determining its geometry. In our framework, automatic landmarking is carried out by mapping the landmarks of an atlas that is representative of a set of training shapes. Let us assume that n segmented shapes (3D binary images) are available, Tn = {Bi } where i = 1 · · · n. To generate the landmarks for the n shapes, the task is to build an atlas A, landmark it, and propagate its landmarks to the n shapes (Fig. 1). In the following we will describe these three steps in detail. Patient Coordinates
1
Atlas−aligned Coordinates
Atlas Coordinates
Atlas Coordinates
Ta
Te
Copy Landmarks
Ta
Te
Copy Landmarks
n
Atlas−aligned Coordinates
Te−1 −1
Te
PCA
Landmark
Atlas Fig. 1. Overview of the automatic landmarking framework. All individual data sets are matched to an atlas via an quasi-affine transformation (Ta ) and an elastic transformation (Te ). The landmarks in the atlas can then be copied to the individual patients. The elastic deformation is subsequently reversed. Thus, Principal Component Analysis (PCA) is carried out in a space where all shapes are aligned with the atlas (atlasaligned coordinates). The principal modes of variation will therefore account for elastic deformations and not for pose or size changes.
Atlas Building. In the context of this paper, an atlas is an average representation of the shape of a structure inferred from a set of training shapes Tn . In order to build the atlas, three issues have to be addressed: the selection of a pairwise corresponder to match two different shapes, a strategy to blend shapes that are represented as binary volumes in a common coordinate frame, and a scheme to obtain an average or mean shape with marginal bias towards a particular individual. Pairwise shape corresponder. Given a shape Bi , it is matched to the atlas, A, using an quasi-affine registration algorithm with nine degrees of freedom (rigid transformation plus anisotropic scaling) adapted from [14]. This algorithm matches shapes using a criterion based on normalized mutual informa-
82
Alejandro F. Frangi et al.
tion [15]. Since the shapes are binary images, we have experimented with several other registration measures (sum of squared differences and cross-correlation) but normalized mutual information was found to be superior. After registration, the shape Bi is expressed in the coordinate system of A. The coordinate system of A will be referred to as the atlas-aligned coordinate system. Shape blending. Once we have found the quasi-affine transformations that map each of the Bi shapes into atlas-aligned coordinates, these shapes have to be combined to form an average shape (binary image). Let Bi and DT (Bi ) denote the shape in atlas coordinates and its Euclidean distance transform [6] respectively, with the convention that inner points have a negative distance while outer points have a positive distance. Then, an average shape can be obtained in the distance transformed domain by computing n DT (Bav ) = n1 i=1 DT (Bi ). A binary representation of the shape Bav can be obtained by thresholding the distance transform map to its zero-level set (Figure 2(a)). Mean shape. To generate the mean shape it is necessary to register all Tn shapes into a common reference frame (atlas-aligned coordinates). However, the atlas is not initially known. To solve this problem an iterative algorithm was developed. One training shape is randomly selected as the initial atlas, A0 , and all remaining shapes are registered to it using the pairwise shape corresponder. After this step, all shapes Tn are expressed in the canonical system of A0 and can be blended to generate a new atlas A1 . This procedure is iterated I times to reduce the effect of the initial shape. Any metric of similarity between the atlases of two consecutive iterations can be used to monitor the convergence of the procedure. The final atlas is AI . This iterative algorithm is summarized in the flow diagram of Figure 2(b). To check for the influence of the randomly selected training shape, atlases with different start shapes have been quantitatively compared. Atlas Landmarking. By means of the iterative procedure of the previous subsection a binary atlas, A, has been obtained. In order to landmark this atlas the marching cubes [11] algorithm is used which generates a dense triangulation of the boundary surface. This triangulation can be decimated to obtain a sparse set of nodes that keeps the geometry of the original triangulation to a desired degree of accuracy. The number of nodes in this decimated triangulation corresponds to the number of landmarks. The use of different triangle densities (decimation ratios) has been investigated to observe their influence in the statistical models generated with our technique (see results section). The decimation strategy applied in this paper is the one proposed by Schroeder et al. [13]. Note that, as an alternative to marching cubes, an expert could manually pinpoint anatomical landmarks in the atlas. Anatomical landmarks, however, may be too sparse to accurately represent the shape of the structure. By using marching cubes, a dense and approximately even distribution of landmarks is obtained.
Automatic 3D ASM Construction
B’1
DT
DT 1 Threshold
B’n
DT
83
Bav
DT n
(a)
(b)
Fig. 2. (a) Shape-based blending of n registered binary shapes based on distance transforms (DT). By convention, the inside of the shape has negative distance and the outside positive distance. (b) Flow diagram of the iterative atlas construction algorithm.
Landmark Propagation. Once the atlas is constructed and landmarked, its landmarks can be propagated to the individual shapes. This is carried out by warping each sample binary volume into the atlas with a transformation, T = Ta + Te , that is composed of an quasi-affine (Ta ) and an elastic (Te ) transformation. The transformation Ta accounts for pose and size differences between the atlas and each sample volume while the transformation Te accounts for shape differences. The global transformation is obtained using an quasi-affine registration algorithm adapted from [14]. Registration of binary volumes was carried out using normalized mutual information [15]. The elastic transformation is expressed as a volumetric free-form deformation field computed with the method of Rueckert et al. [12] that also uses normalized mutual information as a registration measure. Once the global transformation T has been found, the landmarks of the atlas could be propagated to the atlas-aligned coordinate system by applying the inverse of the elastic transformation (Te−1 ). This process is repeated for each sample shape. As a result, a set of landmarks is obtained that describes shape variations with respect to the atlas. Since these landmarks are now in atlas-aligned coordinates, pose and size variations are explicitly eliminated from further analysis. These transformed landmarks are subsequently used as input for Principal Component Analysis (PCA) as indicated in Figure 1. Figure 1 suggests that each sample shape is warped to the atlas. In this case, the inverse of the deformation field has to be computed to propagate the landmarks. However, this mapping does not necessarily exist. This was illustrated for the sake of conceptual simplicity only. From a computational point of view
84
Alejandro F. Frangi et al.
it is more convenient to warp the atlas to each sample shape and use the direct deformation field for landmark propagation.
3 3.1
Results Data Sets
In order to exemplify the methodology, two case studies were analyzed. The first case study consists of a set of 14 manual segmentations of the head of the radius, a bone of the wrist, extracted from CT scans (voxel dimensions 1 × 1 × 2 mm3 ). The second is a set of 20 manual segmentations of the caudate nucleus, a deep structure of the brain, from MR scans (voxel dimensions 1 × 1 × 1.2 mm3 ). In building the model of the caudate nucleus each hemisphere of the structure was treated independently. This was done because this particular two-part structure has an almost specular symmetry with respect to the sagittal plane separating the left and right brain hemispheres. Such symmetry would be difficult to capture with a single quasi-affine transformation. After the landmarks of each side (sub-atlas) are extracted and propagated, Principal Component Analysis (PCA) is applied to the concatenation of the landmarks of both sides. In this way, inter-hemisphere relationships are included in the statistical analysis. 3.2
Atlas Construction
Convergence Properties. As a metric to measure convergence we have used the κ statistic [1]. This statistic measures the similarity between two binary images, κ(Am , Am−1 ), in a way that is independent of the structure’s volume. Figure 3 shows the evolution of the κ statistic, κ(m), as a function of the iteration number, m. This statistic ranges between 0.0 and 1.0 and a value above 0.9 is usually regarded as an excellent agreement [1]. The κ(m) statistic compares the similarity between the atlases Am and Am−1 . Figure 3(a) corresponds to the atlas of the radius. Two curves are shown for two different initial shapes used in the initialization procedure. Similar curves are drawn in Figure 3(b) for the left and right caudate nucleus atlases. The atlas of each subpart (left/right caudate nucleus) was obtained independently. The trend of these plots is similar to that observed in the atlas of the radius. Figure 3 indicates that after five iterations the shape of the atlas stabilizes (κ > 0.97). Effect of Initial Shape. We investigated whether the atlases generated with the two different initializations are comparable in shape, i.e. similar up to an quasi-affine registration. This was done in the following way. For each individual shape, two quasi-affine transformations can be found that map it to each of the two atlases, A and B. Let us call these transformations TAi and TBi , respectively. Let TAB be the quasi-affine transformation that maps the atlas A into the atlas −1 B. In this situation, the transformation Ti = TBi TAB TAi should be equal to the identity transformation, TI . It is possible now to measure the average and the
Automatic 3D ASM Construction Iterative atlas computation (radius)
Iterative atlas computation (nucleus caudate)
1
1
0.99
0.98
0.98
0.96 κ(m)
κ(m)
0.97 0.96
0.94 Reference A (left) Reference A (right) Reference B (left) Reference B (right)
0.92
0.95
0.9
0.94 0.93 0.92
85
0.88
Reference A Reference B 1
2
3
4
5 6 Iterations, m
7
8
9
10
0.86
1
2
3
(a)
4
5 6 Iterations, m
7
8
9
10
(b)
Fig. 3. Convergence of the atlas construction algorithm. The κ statistic between two consecutive atlases as a function of the iteration number. Iteration zero corresponds to the reference (initial) shape used in the iterative algorithm. The κ(m) statistic compares the agreement between the atlases Am and Am−1 . Curves for different initial shapes (A and B) are shown.
standard deviation of the difference Ti − TI . These two measures will provide the bias and dispersion introduced by using two different initial shapes to build the atlas. The results of this analysis are shown in Table 1 for each atlas and each transformation parameter. This table indicates that the deviation from an identity transformation depends on the type of shape. For the very elongated and thin structure of the caudate nucleus the error standard deviations (SDs) are larger compared to the radius. As a consequence, the influence of the initial shape on the final atlas will depend on the shape itself. Translation and rotation error SDs are below 3.3 mm and 0.1◦ , respectively. Scaling error SDs are below 14.5%. From a practical point of view Table 1 indicates that the atlas does indeed depend on the initial shape and that the effect is has to do with the class of shapes being modeled. In the applications presented in this chapter, this effect is not critical. After performing an quasi-affine registration of the atlases
Table 1. Mean (standard deviation) of the error in each transformation parameter (translation, rotation and scaling) of the transformation Ti with respect to the identity transformation for three different atlases. Parameter tx ty tz rx ry rz sx sy sz
Units [mm] [mm] [mm] [◦ ] [◦ ] [◦ ] [%] [%] [%]
Radius -0.72 (1.68) -1.20 (1.32) +0.64 (1.99) +0.01 (0.02) -0.01 (0.02) -0.01 (0.02) -0.57 (1.99) -1.48 (1.78) +1.57 (6.08)
Caudate (L) +1.25 (3.28) -0.20 (0.71) -0.25 (0.54) -0.01 (0.03) -0.04 (0.09) +0.01 (0.08) +3.45 (14.51) -2.12 (6.28) -3.22 (7.23)
Caudate (R) +0.62 (1.42) -0.14 (0.57) +0.06 (0.17) +0.02 (0.03) +0.10 (0.05) -0.02 (0.06) -5.60 (8.20) -1.47 (3.92) -1.98 (4.12)
86
Alejandro F. Frangi et al.
1st mode
2nd mode
3rd mode
√ −3 λi
mean
√ +3 λi
Fig. 4. Shape instances generated using the 3D model from 14 data sets of the radius. The instances are generated by varying a single shape parameter, fixing all others constant at zero standard deviations from the mean shape. Each instance of the model consists of 2500 nodes.
generated with two different initializations, the average boundary-to-boundary distance between the two atlases was 1.3 mm and 0.6 mm for the radius and the two caudate nucleus atlases, respectively. These errors are on the order of, and slightly smaller than the voxel dimensions, respectively.
3.3
Point Distribution Models
Figures 4 and 5 show the mean shape models and the first three modes of variation obtained from PCA for the radius and caudate nucleus test cases, respectively. The number of mesh nodes is 2500 and 1000, respectively. In both cases there are√no visible surface foldings neither in the mean shape nor in the models for ±3 λi .
Automatic 3D ASM Construction
87
1st mode
2nd mode
3rd mode
√ −3 λi
mean
√ +3 λi
Fig. 5. Shape instances generated using the 3D model from 20 data sets of the caudate nucleus. The instances are generated by varying a single shape parameter, fixing all others constant at zero standard deviations from the mean shape. Each instance of the model consists of 1000 nodes. 3.4
Reconstruction Error
Figure 6 illustrates the relative shape variance explained with an increasing number of modes. Similar curves for different decimation ratios (number of model triangles) are provided. These curves are only marginally dependent on this factor. From ten modes onwards, the model captures more than 90% of the shape variance. Note the steeper slope of the curves corresponding to the caudate nucleus. Over the training set there is apparently less variability in the shape of the caudate nucleus than in the shape of the radius. As a consequence, with fewer modes a larger amount of shape variation can be explained. In order to assess the ability of these models to recover shapes not used in the training set we carried out the following experiment. Reconstruction errors were computed by reconstructing the landmarks of one shape of the training set with the ASM built from the remaining shapes (leave-one-out experiment).
88
Alejandro F. Frangi et al. Cumulative relative variance (radius)
Cumulative relative variance (nucleus caudate)
100
100
80
80
40
40
20
0
0.25 0.50 0.75 0.90 0.95
60 %
0.25 0.50 0.75 0.90 0.95
%
60
20
0
2
4
6
8 Modes
10
12
14
(a)
0
0
5
10 Modes
15
20
(b)
Fig. 6. Percentage of total shape variance versus the number of modes used in the 3D ASM and for various decimation ratios. The number of landmarks before decimation was 15519 for the radius, and 2320 for the caudate nucleus. The decimation ratio represents the ratio between the nodes eliminated from the triangulation of the atlas and its initial number. Note that the number of modes is at most the number of sample shapes minus one.
The errors reported in Figure 7 are the average of the reconstruction errors over all shapes taking out one in turn. The same experiment was repeated for different decimation ratios and increasing number of modes of shape variation taken into the reconstruction. The reconstruction errors were computed in millimeters. For the caudate nucleus, the reconstruction error is below the voxel dimensions (10 modes). In the case of the radius, the reconstruction error is slightly larger than the slice thickness. One possible explanation to this higher error could be the fact that no image resampling was used during registration. On the other hand, in comparison to the shape of the caudate nucleus, the radius represents a more complex structure with larger shape variability in the training set. This could explain the poorer reconstruction performance in the leave-one-out experiments of the radius. The plots of Figure 7 also indicate that the reconstruction error is slightly dependent on the decimation ratio and, as expected, inversely proportional to the number of modes of variation.
4
Discussion and Conclusion
This paper presents a method for the automatic construction of 3D active shape models. The technique is based on the automatic extraction of a dense mesh of landmarks in an atlas constructed from the training shapes which are propagated through an elastic deformation field to each shape of the training set. The method is able to treat single and multiple-part shapes. The first part of the proposed technique involves the building of an atlas from a set of example shapes. In Section 3 we showed experimental results indicating that this procedure is convergent. Moreover, different initial shapes seem to contribute only marginally to the final atlas. That is, the final atlases are similar
Automatic 3D ASM Construction Leave-one-out reconstruction error (radius) 2.9
0.25 0.50 0.75 0.90 0.95
1.1 Error (mm)
2.8 Error (mm)
Leave-one-out reconstruction error (nucleus caudate) 1.15
0.25 0.50 0.75 0.90 0.95
2.85
89
2.75 2.7
1.05
1
2.65 0.95
2.6 2.55
0
2
4
6
8 Modes
(a)
10
12
14
0.9
0
5
10 Modes
15
20
(b)
Fig. 7. Reconstruction error in the leave-one-out experiments. The number of landmarks before decimation was 15519 for the radius, and 2320 for the caudate nucleus. The decimation ratio represents the ratio between the nodes eliminated from the triangulation of the atlas and its initial number.
up to an quasi-affine transformation. However, we note that the influence of the initial shape depends on the class of shapes being modeled and has to be assessed on a case-by-case basis. In the work by Fleute and Lavall´ee [7,8] a similar algorithm was used to build the average model (atlas). However, no experimental evidence was reported with respect to the convergence of the atlas construction algorithm. An alternative to our iterative method of atlas construction is the tree-based approach presented by Brett and Taylor [4]. This hierarchical strategy is attractive since it gives a unique (non-iterative) way to build an atlas from a given set of examples. However, one problem of Brett’s method is that the training shapes have to be ranked according to a pairwise match quality. This requires that all possible pairs have to be matched and scored before the tree is built. Brett presented results with only eight shapes [4] but ordering the examples according to the matching quality would be cumbersome for a realistic amount of training shapes. For a total number of n shapes it is necessary to compute N = (n − 1)2 ≈ O(n2 ) pairwise matches to build the average shape. Our approach obtains the average shape in N = nI ≈ O(n) matches where I is the total number of iterations required for convergence. Section 3 shows experimental evidence that after about five iterations the atlas shape stabilizes. Our method for building the mean shape model is based on averaging shapes in the domain of their distance transforms. A similar strategy was proposed by Leventon et al. [10] to incorporate statistical constraints into the level-set approach to image segmentation. However, in that work, PCA is applied on the distance transform domain and not on a surface representation. As a consequence, the number of degrees of freedom is considerably larger than in our method. There is an intrinsic limitation in both our method and that of Leventon et al. Averaging distance transforms of several shapes does not necessarily yield a valid mean shape representation. It is easy to show, for instance, that
90
Alejandro F. Frangi et al.
in case of a large misalignment between the averaged shapes, this procedure can introduce topological changes. Although we did not observe this problem in our experiments this can be a potential source of failure of the technique when building models of complex structures. The proposed technique could be used with any elastic registration algorithm. In this sense, the method is a generic framework open to future research. Currently, the volumetric elastic registration of Rueckert et al. [12] is used to match binary images. The use of elastic registration as a method to establish shape correspondences imposes a constraint on the type of shapes that can be handled. It is assumed that the class of shapes has a well-defined topology. If there are sub-structures in one image not represented in the other image to be matched, the transformation would have to destroy those parts. This situation could arise when building a model of normal and abnormal medical structures where some parts in the latter are missing because of a diseased state or surgery. However, establishing correspondences in these mixed models also remains an ill-defined problem with any of the previously published approaches [7,8,3]. Results of the construction of models of two anatomical structures have been presented. Experiments were carried out to establish the ability of the models to generalize to shapes not present in the training set. The average reconstruction error was below 2.65 mm (radius) and 0.95 mm (caudate nucleus) when the number of nodes used was sufficient to explain 90% of the shape variability. These errors are on the order of, and slightly smaller than the voxel dimensions, respectively. In our experiments we have not observed problems of wrong correspondences leading to flipping of triangles and surface folding. This is an important improvement compared to the initial method of Brett and Taylor [4]. Also, our method is less restrictive in terms of the shapes that can be modeled. This is an important feature with respect to the improved method of Brett and Taylor [3] that is based on harmonic maps and therefore limited to shapes that are isomorphic to a disc. Finally, it would be interesting to perform a comparison between the models built with different methods. In order to carry out a quantitative comparison it is necessary to define a measure of model quality. The definition of such a measure is in itself an interesting issue. Obviously, different methods will yield different sets of landmarks which precludes a landmark-based comparison. If one defines a given segmentation task, a comparison could be established on the basis of the segmentation accuracy. Although these measures can have a prominent practical value to determine the best model-building technique for a given problem, the conclusions will remain task-dependent. Possibly, other more task-independent criteria related to the compactness and generalizability of the built models could be within the interesting candidate measures to explore.
Acknowledgements This research was sponsored by the Dutch Ministry of Economic Affairs (IOP Beeldverwerking IBV97009) and EasyVision Advanced Development, Philips
Automatic 3D ASM Construction
91
Medical Systems BV, Best, The Netherlands. Dr. Maarten Hoogbergen provided us with the radius segmentations, and the Department of Psychiatry of the University Medical Center Utrecht with the caudate nucleus segmentations.
References 1. D.G. Altman. Practical Statistics for Medical Research. Chapman & Hall, 1991. 2. P.J. Besl and N.D. McKay. A method for registration of 3D shapes. IEEE Trans Pattern Anal Machine Intell, 14(2):239–55, February 1992. 3. A.D. Brett and C.J. Taylor. Automated construction of 3D shape models using harmonic maps. In S. Arridge and A. Todd-Pokropek, editors, Medical Image Understanding and Analysis, pages 175–78, London, July 2000. 4. A.D. Brett and C.J. Taylor. A method of automated landmark generation for automated 3D PDM construction. Imag Vis Comp, 18(9):739–48, 2000. 5. T.F. Cootes, C.J. Taylor, D.H. Cooper, and J. Graham. Active Shape Models their training and application. Comp Vis Image Underst, 61(1):38–59, 1995. 6. P.E. Danielsson. Euclidean distance mapping. Comp Graph Imag Proces, 14:227– 48, 1980. 7. M. Fleute and S. Lavall´ee. Building a complete surface model from sparse data using statistical shape models: applications to computer assisted knee surgery. In W.M. Wells, A. Colchester, and S. Delp, editors, Medical Imaging Computing & Computer-Assisted Intervention, volume 1496 of Lect Notes Comp Science, pages 879–87, Boston, USA, September 1998. Springer Verlag. 8. M. Fleute and S. Lavall´ee. Incorporating a statistically based shape model into a system for computer-assisted anterior cruciate ligament surgery. Med Image Anal, 3(3):209–22, 1999. 9. A. Kelemen, G. Sz´ekely, and G. Guerig. Elastic model-based segmentation of 3-D neuroradiological data sets. IEEE Trans Med Imaging, 18(10):828–39, October 1999. 10. M. Leventon, W.E.L. Grimsom, and O. Faugeras. Shape-based 3D surface correspondence using geodesics and local geometry. In Comp Vis Patt Recogn, volume 1, pages 316–23, South Carolina, USA, June 2000. IEEE Computer Society. 11. W.E. Lorensen and H.E. Cline. Marching cubes: a high resolution 3D surface reconstruction algorithm. Computer Graphics: SIGGRAPH’87 Conference Proceeding, 21:163–69, July 1987. 12. D. Rueckert, L.I. Sonoda, C. Hayes, D.L.G. Hill, M.O. Leach, and D.J. Hawkes. Non-rigid registration using free-form deformations: Application to breast MR images. IEEE Trans Med Imaging, 18(8):712–21, August 1999. 13. W.J. Schroeder, J.A. Zarge, and W.E. Lorensen. Decimation of triangle meshes. Comp Graphics, 26(2):65–70, 1992. 14. C. Studholme, D.L.G. Hill, and D.J. Hawkes. Automated 3D registration of MR and PET brain images by multiresolution optimization of voxel similarity measures. Med Phys, 24(1):25–35, 1997. 15. C. Studholme, D.L.G. Hill, and D.J. Hawkes. An overlap invariant entropy measure of 3D medical image alignment. Pattern Recogn, 32(1):71–86, 1998. 16. Y. Wang, B.S. Peterson, and L.W. Staib. Shape-based 3D surface correspondence using geodesics and local geometry. In Comp Vis Patt Recogn, volume 2, pages 644–51, South Carolina, USA, June 2000. IEEE Computer Society.
A Regularization Scheme for Diffusion Tensor Magnetic Resonance Images Olivier Coulon, Daniel C. Alexander, and Simon R. Arridge Department of Computer Sciences, University College London, Gower Street, London WC1E 6BT, United-Kingdom
[email protected]
Abstract. A method for regularizing diffusion tensor magnetic resonance images (DT-MRI) is presented. The scheme is divided into two main parts: a restoration of the principal diffusion direction, and a regularization of the 3 eigenvalue maps. The former make use of recent variational methods for restoring direction maps, while the latter makes use of the strong structural information embedded in the diffusion tensor image to drive a non-linear anisotropic diffusion process. The whole process is illustrated on synthetic and real data, and possible improvements are discussed.
1
Introduction
Diffusion tensor magnetic resonance imaging (DT-MRI) is an image acquisition technique based on water diffusion characteristics, that allows the investigation in vivo of physiological and structural information of tissues [3]. Applications of DT-MRI cover research activities such as white matter fiber tracking and brain connectivity studies as well as clinical diagnostic of disruptions caused by multiple sclerosis or stroke. The measurement acquired at each voxel is a diffusion tensor (DT), D, represented by a symmetric positive definite matrix, that quantifies the amount of diffusion in every direction. This tensor can be expressed by a set of six coefficients, and is often decomposed into its eigensystem, consisting of three eigenvalues and three associated eigenvectors. As with many other MRI techniques, the level of image noise depends on the chosen voxel discretization and the acquisition time. A post-processing technique to reduce noise would relax the scanning time versus voxel size trade-off as well as improving analsyis methods. However DT-MRI is a fairly new technique and its nature requires new image processing methods (see for instance [1]). In particular, few regularization methods have been presented in the literature so far. In this paper we propose a regularization scheme for DT-MR images. The method uses variational and PDE-based techniques and relies on the separation of two types of information contained in the tensor: the principal diffusion direction (PDD), defined by the eigenvector associated with the largest eigenvalue, and the amount of diffusion along each eigenvector, defined by the eigenvalues. M.F. Insana, R.M. Leahy (Eds.): IPMI 2001, LNCS 2082, pp. 92–105, 2001. c Springer-Verlag Berlin Heidelberg 2001
A Regularization Scheme for Diffusion Tensor Magnetic Resonance Images
93
The PDD field is regularized using an adaptation of recent methods for restoring direction maps [16,7,20] , and the 3 eigenvalue maps are regularized subsequently using a non-linear anisotropic diffusion process. In the next section we introduce necessary concepts related to DT-MR images. Then the two following sections describe the regularization methods for the PDD field and the eigenvalue maps. Experiments and results are then presented.
2
Background
In the following, D = (Dij ) will stand for the diffusion tensor matrix, (λi )i=1,2,3 for its eigenvalues with λ1 ≥ λ2 ≥ λ3 , and (vi )i=1,2,3 for the associated eigenvectors. Several scalar measurements can be derived from the DT-MR images to describe properties of the underlying tissues (see for instance [4]). Here, we will make use of the fractional anisotropy, FA, a measure of anisotropy of the tensor, which varies between 0 and 1, and is defined as follows: (λ1 − λ2 )2 + (λ1 − λ3 )2 + (λ2 − λ3 )2 (1) FA = 2(λ21 + λ22 + λ23 ) The multi-dimensional nature of the information contained in each voxel makes these images difficult to visualize. Various techniques exist for visualisation of tensor volumes. In this paper, we represent each tensor as an ellipsoid whose axes are aligned with the (vi ) and scaled by the eigenvalues (see for example figure 1). Noise in DT-MR images comes from various sources. Partial volume effects caused by large voxel sizes result in a local averaging of the estimated tensor. This can lead to poor estimation of directions and values, and often creates oblate tensors (λ1 λ2 ) where white matter fibers of different orientation cross inside one voxel. Another source of noise is the motion of the ventricles, related to blood flow, that may cause a blur of the tensor estimation. There are also problems of “sorting bias” ([5]): in the presence of noise the ranking of eigenvalues may not be regionally consistent. This results in an overestimate of the local anisotropy and one can observe “switches” between principal directions of diffusion, leading to corrupted eigenvector fields. One method to address that problem is proposed in [5]. To reduce the effects of these sources of noise, a few methods can be found in the literature. Parker et al. [13] regularize DT-MR images using a non-linear smoothing of the diffusion-weighted images used to estimate the tensor. Even though it aims at regularizing the whole tensor information, this method does not take into account the strong structural information carried by the eigenvectors, and brings intrinsic edge-estimation problems. Poupon et al. [17] have proposed a PDD field regularization method, which we describe later in section 3. It is difficult to process the magnitude and direction information simultaneously. For instance, figure 1 shows that an interpolation of two anisotropic tensors based on linear interpolation of the coefficients results in a tensor with correctly
94
Olivier Coulon, Daniel C. Alexander, and Simon R. Arridge
interpolated PDD but lower anisotropy, which is an undesirable effect. A better approach would be to transform directions and eigenvalues separately. Moreover, we do not know any scalar measure that separates tissues equally on the basis of both pieces of information (e.g. one that distinguishes between anisotropic and isotropic regions as well as between anisotropic differently oriented regions). Therefore, we chose to process directions and eigenvalues separately. This is a similar approach to color image restoration methods that separate chromaticity and brightness (see for instance [19]), which led to recent research in direction map restoration presented in the next section. An important advantage of our approach is the fact that the DT eigenvalues can be regularized using directional information from the previously restored PDD field.
a)
b)
c)
Fig. 1. Tensor interpolation. A coefficient-based interpolation of (a) and (b) results in tensor (c) with lower anisotropy. Interpolation: Cij = 12 (Aij + Bij ). The next two sections describe our regularization techniques for both the PDD field and the eigenvalue maps.
3
Restoration of the Principal Diffusion Direction
To our knowledge, the only method presented so far for restoring the first eigenvector field can be found in [17]. In this paper, Poupon et al. propose a Markovian model associated with an energy function whose minimum corresponds to the regularized field. The model relies on a trade-off between the two following assumptions: – white matter tracts have a low curvature, – diffusion should be maximum in the direction of the first eigenvector. We adopt the same assumptions and propose a very simple iterative restoration scheme, which is inspired from the most recent advances in direction map restoration [16,7,20]. Direction maps are a particular case of vector fields in which each vector has a Euclidean norm equal to 1. Directions live, in the 3D case, on the unit sphere S2 . Perona first proposed a PDE-based model for the 2D case [16], and was followed by Chang and Shen [7] and Tang et al. [20] who devised more general models.
A Regularization Scheme for Diffusion Tensor Magnetic Resonance Images
3.1
95
Direction Regularization: The Original Model
Chang and Shen [7] presented variational models to restore features on non-flat manifolds and algorithms to implement them in the discrete case. Our problem fits into this framework, since the unit sphere S2 is a simple example of a non-flat manifold. Our method starts from the models in [7], which we briefly summarize here. Let f : Ω → M be a feature distribution on a m-dimensional Riemannian manifold M , where Ω ⊂ IRn is the image domain. Chang and Shen define the fitted total variation (TV) energy to be minimized: λ TV e(f, p)dp + d2 (f (0) , f )dp, (2) ε (f, λ) = 2 Ω Ω where e(f, p) is a strength function at pixel p, d is the metric on M induced by its Riemannian structure, and f (0) the original feature distribution. The variational problem is solved by studying the associated Euler-Lagrange equations, leading to a diffusion equation. A discrete model is then derived from this continuous approach. If α is a pixel on the discrete domain Ωn , let Nα be a neighborhood of α. If one defines a locally Riemannian distance dl (a distance that locally tends to the metric d), the strength function is defined as follow: 1 d2l (fα , fβ )] 2 . (3) e(f, α) = [ β∈Nα
The fitted TV energy then becomes: εTV (f, λ) =
e(f, α) + λ
α∈Ωn
1 d2 (f (0) , fα ), 2 l α
(4)
α∈Ωn
In the case of directions, i.e. M = S2 , if one chose dl to be the embedded Euclidean distance (the Euclidean distance in IR3 , which locally tends to the geodesic distance on S2 ), the corresponding diffusion equation is: dfα = FαTV (f ) = Πfα ( wαβ fβ + λfα(0) ), dt
(5)
β∈Nα
where Πfα is the orthogonal projection on the plane tangent to M at fα , and wαβ is a weight defined by: wαβ (f ) =
1 1 + . e(f, α) e(f, β)
(6)
The following iterative scheme then minimizes εTV : a) f˜αn = fαn−1 + ∆tFαTV (f n−1 ), b) fαn =
n f˜α . n f˜α
(7)
96
Olivier Coulon, Daniel C. Alexander, and Simon R. Arridge
This model has the great advantage that it does not require the computation of spatial derivatives, and it converges strictly to minimize εTV (f, λ). The flow FαT V is composed of two terms. The first term is a projection in the tangent plane of a weighted sum of the neighboring features at the considered pixel. The strength function can be interpreted as a local measure of smoothness of the direction map: the smaller it is, the smoother the map is, and the higher the influence of the voxel is. This weighting recalls more classical anisotropic diffusion schemes, a high strength function indicating presence of an “edge” and inducing slow feature diffusion. The second term is a data-driven term that constrains the regularized map to be close to the original one. 3.2
Modifications of the Model
Although it is non-linear and preserves singularities, the model presented above requires some modifications in order to restore our direction field properly. We therefore modified the scheme implemented by equation 7 in order to fit the particular requirement of the PDD maps. Although those changes loosen the strict variational theoretical framework, they are easily interpreted in terms of behavior, and prove to be efficient in practice. Chan and Shen’s TV model restores orientations, living on S2 , whereas the PPD field only carries axial information, so that v1 ≡ −v1 . To cope with this difference, we must map features to the same hemisphere before computing distances. At each iteration of equation 7, when we compute FαTV , if a vector fβ ∈ Nα does not belong to the hemisphere defined by fα (i.e. if fβ .fα < 0) it needs to be “flipped”. Therefore we propose the new definition: wαβ fβα + λfα(0) ). (8) FαTV (f ) = Πfα ( β∈Nα
with: fβα =
−f , if f .f < 0 β β α fβ , else
(9)
Another limitation of the original model lies in the definition of the weights wαβ . We have found empirically that neighboring features often influence each other even though they belong to two different fiber bundles (figure 4-(b)). We must reduce the diffusion flow between two neighboring features fα and fβ in two situations: 1. α belongs to a tract (anisotropic medium) and β belongs to an isotropic medium. In that situation the two corresponding tensors have very different anisotropy values. 2. α and β belong to separate anisotropic tracts whith different orientations. The first point above suggests the choice of a weighting function that takes into account the anisotropy. The second point is partly taken into account by
A Regularization Scheme for Diffusion Tensor Magnetic Resonance Images
97
the original model although the control of the diffusion flow is not strict enough. Therefore we propose to replace equation 6 with the following weighting function: wαβ (f ) =
aα + aβ (fα .fβ )2m , 2
(10)
where aα = FA(α) is the fractional anisotropy at node α, as defined in equation 1, and m a control parameter. Influence of neighbors is then weighted by their anisotropy, so that data from an isotropic medium have negligible influence on data from white matter fiber tracts, and the diffusion flow decreases with differences in direction. The higher the value of m, the more severe this “directional tuning” is. This particular model is shown to give excellent results experimentally, as shown in section 5. A quick analogy with the method of Poupon et al. [17] leads us to the following observations. Both methods have the same assumptions, i.e. fiber tracts have a low curvature and the regularised field must be close enough to the original data. We do not state that our model gives better results than that in [17] and, in fact, performances are probably similar. However, our model has the following advantages: it is certainly faster, it has a simpler implementation, and most of all it does not have to face the problem of discretisation of the directions since they are defined and transformed in a continuous fashion on S2 – in [17] the unit sphere is discretised in 162 directions.
3.3
Re-projection of the Other Eigenvectors
Once the first eigenvector has been restored, before re-constructing the whole tensor, the second and third eigenvectors must be reoriented. We use a similar approach to the preservation of principal directions algorithm presented in [1], which computes the reorientation of diffusion tensors after a non-rigid transformation has been applied to the whole image. The second eigenvector is projected on the plane orthogonal to the regularized first eigenvector in order to compute the new eigensystem: – let (v1 , v2 , v3 ) be the original set of eigenvectors, and v1r the regularized first eigenvector. – Define v2r = v2 − (v2 .v1r )v1r . – Define v3r = v1r × v2r The new tensor is then constructed using the new set of eigenvectors and the original eigenvalues: λ1 0 0 Dr = (v1r v2r v3r ) 0 λ2 0 (v1r v2r v3r )T 0 0 λ3
(11)
98
4
Olivier Coulon, Daniel C. Alexander, and Simon R. Arridge
Regularization of the Eigenvalues
The eigenvalues describe the amount of water diffusion along the principal axes of the DT. They are also subject to noise and reducing this noise would help the discrimination of tissues and quantitative analysis of DT images. Consistency of eigenvalues should be preserved along fiber tracts and we propose here a method that aims to regularize them in an anisotropic fashion using the previously restored PDD map. Various non-linear anisotropic smoothing methods have been proposed in the literature (see [10,21] for reviews). Most of them use a formulation related to the local image structure. Some are PDE-based using either a diffusion tensor [9,23] or an explicit flow formulation [11], others use a smoothing kernel whose shape adapts to the local image structure [2,12]. The diffusion approach, as studied by Weickert [22], is particularly interesting because of its general character and its scale-space, as well as pure regularization, point of view. We focus here on the coherence-enhancing scheme proposed in [23]. General nonlinear anisotropic diffusion filtering is defined as follows. Given the original image I0 (x), a one parameter family of images I(x, t) is built as the solution of the diffusion equation ∂I
∂t = div(M∇I), I(x, 0) = I0 (x),
(12)
where M is a flow tensor (usually called diffusion tensor, but we will avoid this denomination to prevent confusion with the diffusion tensor in the images), and is a function of the local image structure. In the continuous case, if M is symmetric positive definite and smooth enough, existence and uniqueness of f have been proven, as well as scale-space and regularization properties [22]. At every voxel, let us define the flow tensor with the following matrix: µ1 0 0 (13) M = E 0 µ2 0 ET , 0 0 µ3 where E = [v1r v2r v3r ] is the eigenvector matrix of Dr . With this formulation, one can explicitly define the diffusion flow along the three eigenvectors by means of the (µi )i=1,2,3 . In the case of DT-MR images, structural information is mostly carried by the first eigenvector v1r . In the white matter, the PDD generally points along the fiber bundle, and along that direction we can expect to find other points in the same fiber. In isotropic tissues, where A has a low value, v1r does not have any particular meaning and there should not be any diffusion flow along that direction. Therefore, in a similar way to coherence-enhancing diffusion [23], we define the following set of eigenvalues: µ1 =
0
α + (1 − α) exp(C (1 − µ2 = µ3 = α, 2
1 )) FA2σ
if FA2σ = 0, else,
(14)
A Regularization Scheme for Diffusion Tensor Magnetic Resonance Images
99
where α is a small parameter that guarantees a minimum diffusion and keeps M positive definite, and FAσ = FA ⊗ Gσ is the fractional anisotropy (eq. 1) smoothed with a Gaussian function. C is a control parameter that controls the acceptable magnitude of anisotropy. Smoothing the fractional anisotropy ensures the smoothness of M and helps the stability of the scheme. The scheme defined by equations 12-14 has the following behavior, illustrated in figure 2: – where FAσ C, i.e. in tissues with low anisotropy, we have µ1 ≈ α and µ2 = µ3 = α, there is almost no diffusion flow. – where FAσ C, i.e. in white matter, we have µ1 ≈ 1 and µ2 = µ3 = α, there is a diffusion flow along v1r .
Fig. 2. Function µ1 (FAσ ) for different values of C Instead of defining a semi-local measure of coherence using image derivatives [23], fractional anisotropy provides a natural characterisation of the underlying tissues. Moreover, values of anisotropy are characteristics of tissues and are expected to be stable across images. Therefore we could in the future expect to define a value of C based on a quantitative evaluation of fractional anisotropy for a population of subjects. Figure 3 shows an example of a µ1 map together with a simple anisotropy image and its smoothed version. If one defines Ii = ∇I.vir , the diffusion flow can be explicitly written as: J = ([α + (1 − α) exp(C 2 (1 −
1 ))]I1 )v1r + (αI2 )v2r + (αI3 )v3r FA2σ
(15)
with the following diffusion equation: ∂I = div(J). ∂t
(16)
100
Olivier Coulon, Daniel C. Alexander, and Simon R. Arridge
FA
FA σ
µ1
Fig. 3. fractional anisotropy map FA, FAσ with σ = 0.8, and the corresponding µ1 map with C = 0.8 The diffusion process described by equations 15 and 16 is applied simultaneously on the 3 eigenvalue maps. The discretization of the diffusion equation is performed using a rotation-invariant optimised scheme presented in [25]. We observe that, because we use the regularized PDD field, the flow direction does not need to be smoothed as is done in [24]. Because the diffusion process can only be driven along v1r there is no flow from white matter to an isotropic tissue, except where a tract terminates, and this particular problem will be addressed in further work. For the same reason, when two different bundles are neighbors, there is no intensity flow “mixing” them. Note that, as for the PDD field, regularization is essentially performed in white matter, where anisotropy is high enough. Further work will aim at defining a scheme that regularize grey matter and CSF isotropically while still having an anisotropic behavior in white matter.
5
Experiments
The process was run on synthetic and real data. In the following, the PDD regularization was performed until time t = 10, with λ = 0.5, and m = 2. The eigenvalue map regularization was performed until scale t = 100 with σ = 0.8, C = 0.8, and α = 0.001. Firstly, a synthetic dataset was created and is shown in figure 4. It is composed of two orthogonal straight bundles to which some uniform noise has been added. Outside the bundles, directions are chosen randomly, and anisotropy is set to 0.05. The original direction regularisation TV model was used, as well as our modified PDD regularisation model. Figures 4-(b) and 4-(c) show that both models regularize the directions inside each bundle properly, and have very small influence from the isotropic medium. At the interface, however, it is clear that with the original TV model the two bundles influence each other while our model overcomes this limitation.
A Regularization Scheme for Diffusion Tensor Magnetic Resonance Images
101
Fig. 4. (a) A noisy synthetic direction map; (b) the same, regularised with the TV model at t = 10 with λ = 0.5 with a close-up of the interface between the two bundles; (c) regularised with our modified model at t = 10 with λ = 0.5 and m = 2 and a close-up of the interface. Directions are scaled with anisotropy.
Secondly, the whole process was run on echo-planar DT-MR brain data acquired with cardiac gating and a 96x96 matrix, and reconstructed to 128x128, 1.875x1.875x3.0 mm3 voxels, with 42 axial slices. Figure 5 shows some results on this dataset. One can see in figure 5-(b,c) a representation of the tensor ellipsoids on a small part of the extremity of the splenium of the corpus callosum. Six tracts were built from six points, using “hyperstreamlines”, from the Visualization Toolkit library [18]. Those hyperstreamlines are generated using the PDD field, and their cross section is determined using the values of λ2 and λ3 . It is clear that the tracking is improved by the PDD regularization. The tracks appear to have a smoother curvature and their relative trajectories are more consistent. At the top-left corner of the image, the three tracts are no longer diverging, and at the junction of the two bundles, tracts keep a smoother and more consistent trajectory. As shown in figure 5-(b,c), it is more difficult to assess the regularity of the eigenvalues map, mainly because the tracts are not clearly visible in the scalar images.
102
Olivier Coulon, Daniel C. Alexander, and Simon R. Arridge
a)
b)
d)
c)
e)
Fig. 5. Fractional anisotropy map (a), λ1 map at the same slice before (b) and after regularisation (c), tracts and tensors along the extremity of the splenium of the corpus callosum before (d) and after regularization (e).
Figures 6-(a,b,c) show a close-up of a tract from the exact same part of the brain, in the original data, after PDD regularization, and after eigenvalue regularization. The effect of the eigenvalues regularization is more evident here: one can see on figure 6-(c) that the hyperstreamline has a more regular cross section, showing more regularity of λ2 and λ3 along the tract as well as an improvement in anisotropy, more regular along the tract. This shows that the model has the expected effect: eigenvalues are regularised consistently along tracts. Some problems still remain with the eigenvalue regularization scheme. For example, strong partial volume effects induce a regularisation bias and a loss of anisotropy. For instance on the upper part of the corpus callosum, the local shape of the tracts creates an important partial volume effect that reduces anisotropy at the inter-hemispheric plane. This low anisotropy is propagated by
A Regularization Scheme for Diffusion Tensor Magnetic Resonance Images
103
Fig. 6. (a) One tract in the original data, (b) after the PPD regularization, and (c) after eigenvalues regularization. the diffusion process and eigenvalues are locally corrupted. This issue is mainly related to the quality of the data and the original resolution. The choice of the parameter C also influencesthe quality of the diffusion process and appears to be problematic in areas where the relationship between the value of the anisotropy and the nature of the underlying tissues is not so clear (for instance white matter with relatively low anisotropy). This particular problem should be solved by controlling the flow using not only the anisotropy but also the variation of the eigenvalues (λi )i=1,2,3 along the principal directions. Another problem is the diffusion flow “going out” of the tract ends. This will be addressed in further work and is related to the use of a contour map to identify the location of tract terminations. The main issue is that the regularization process is effective only in highly anisotropic media. Smoothing in low anisotropy tissues would further improve tissue discrimination. Moreover, noise in fractional anisotropy maps is higher where tissues have low true anisotropy (grey matter) [13,15], and there is considerable need for eigenvalue regularization in those areas.
6
Conclusion
We have presented a regularization scheme for DT-MR images, that includes a restoration of the PDD field followed by non-linear anisotropic coherenceenhancing diffusion applied on the eigenvalue maps. The PDD restoration proves to be successful and will find direct application in white matter fiber tractography. Indeed, almost all methods for tractography use the PDD information [8,14,17,6]. Even though tractography results are difficult to assess they do provide a good environment in which to validate the PDD field restoration. Techniques for proper validation of our methods within this application are the focus of current considerations.Regularization of the eigenvalue maps needs improvement and is also under current investigations. A new definition of the µi functions (eq. 14) should allow a continuous change of behavior of the flow from strong
104
Olivier Coulon, Daniel C. Alexander, and Simon R. Arridge
anisotropic coherence-enhancing diffusion in white matter to isotropic smoothing in grey matter and CSF. Because the DT provides a natural description of the local tissue structure, it seems reasonable that it can be used to determine the type of behavior the flow should have. We plan to look further into the use of the DT to determine directly what the flow tensor at each point should be within a non-linear anisotropic diffusion process.
Acknowledgements OC is funded by the Wellcome Trust. Images were kindly provided by Geoff Parker, Imaging Science and Biomedical Engineering, University of Manchester, and the NMR research unit of the Institute of Neurology, London. All 3D renderings were done using the Visualisation Toolkit (VTK - http://www.kitware.com).
References 1. Alexander, D.C., Pierpaoli, C., Basser, P.J., Gee, J.C.: Spatial transformations of diffusion tensor images. To appear in IEEE Trans. on Medical Imaging (2001) 2. Almansa, A., Lindeberg, T.: Enhancement of fingerprint images using shapeadapted scale-space operators. In Sporring, J., Nielsen, M., Florack, L., Johansen, P., editors, Gaussian Scale-Space Theory, Kluwer-Academic (1997) 3–19 3. Basser, P.J., Matiello, J., LeBihan, D.: MR Diffusion tensor spectroscopy and imaging. Biophysical Journal 66 (1994) 259–267 4. Basser, P.J., Pierpaoli, C.: Microstructural and physiological features of tissues elucidated by quantitative-diffusion-tensor MRI. Journal of Magnetic Resonance, Series B 111 (1996) 209–219 5. Basser, P.J., Pajevic, S.: Statistical artefacts in diffusion tensor MRI (DT-MRI) caused by background noise. Magnetic Resonance in Medicine 44 (2000) 41–50 6. Basser, P.J., Pajevic, S., Pierpaoli, C., Duda, J., Aldroubi, A.: In vivo fiber tractography using DT-MRI data. Magnetic Resonance in Medicine 44 (2000) 625-632 7. Chan, T., Shen, J.: Variational restoration of non-flat image features: model and algorithm. Technical Report CAM-TR 99-20, UCLA (1999) 8. Conturo, T.E., Lori, N.F., Cull, T.S., Akbudak, E., Snyder, A.Z., Shimony, J.S., McKinstry, R.C., Burton, H., Raichle, M.E.: Tracking neuronal fiber pathways in the living human brain. Proc. Natl. Acad. Sci. USA 96 (1999) 10422–10427 9. Cottet, G.H., Germain, L.: Image processing through reaction combined with nonlinear diffusion. Mathematics of Computation 61:204 (1993) 659–673 10. Deriche, R., Faugeras, O.: Les EDP en traitement des images et vision par ordinateur. Technical Report 2697, INRIA (1995) 11. Krissian, K., Malandain, G., Ayache, N.: Directional anisotropic diffusion applied to segmentation of vessels in 3D images. In Scale-Space’97, LNCS 1252, SpringerVerlag (1997) 345–348 12. Lindeberg, T., Garding, J.: Shape-adpated smoothing in estimation of 3D depth cues from affine distortions of local 2D brightness structure. Image and Vision Computing 15 (1997) 415–434 13. Parker, G.J.M., Schnabel, J.A., Symms, M.R., Werring., D.J., Barker, G.J.: Nonlinear smoothing for reduction of systematic and random errors in diffusion tensor imaging. Journal of Magnetic Resonance Imaging 11 (2000) 702–710
A Regularization Scheme for Diffusion Tensor Magnetic Resonance Images
105
14. Parker, G.J.M., Wheeler-Kingshott, C.A., Barker, G.J.: Distributed anatomical brain connectivity derived from diffusion tensor imaging. In IPMI’2001, SpringerVerlag (2001). 15. Pierpaoli, C., Basser, P.J.: Toward a quantitative assessment of diffusion anisotropy. Magnetic Resonance in Medicine 36 (1996) 893–906 16. Perona, P.: Orientation diffusion. IEEE Trans. on Image Processing 7(3) (1998) 457–467 17. Poupon, C., Clark, C.A., Frouin, V., R´egis, J., Bloch, I., Le Bihan, D., Mangin, J.-F.: Regularization of diffusion-based direction maps for the tracking of brain white matter fascicles. Neuroimage 12 (2000) 184–195 18. Schroeder, W., Martin, K., Lorensen, B.: The Visualization Toolkit. An object oriented approach to 3D graphics, 2n d edition. Prentice Hall (1998) 19. Tang, B., Sapiro, G., Caselles, V.: Color image enhancement via chromaticity diffusion. Technical Report, ECE-University of Minensota (1999) 20. Tang, B., Sapiro, G., Caselles, V.: Diffusion of general data on non-flat manifolds via harmonic maps theory: the direction diffusion case. International Journal of Computer Vision 36(2) (2000) 149–161 21. Weickert, J.: A review of non-linear diffusion filtering. In ScaleSpace’97, LNCS 1252, Springer-Verlag (1997) 3–28 22. Weickert, J.: Anisotropic diffusion in image processing. B.G. Teubner, Stuttgart (1998) 23. Weickert, J.: Coherence-enhancing diffusion filtering. International Journal of Computer Vision 31(2/3) (1999) 111–127 24. Weickert, J.: Coherence-enhancing diffusion of colour images. Image and Vision Computing 17 (1999) 201–212 25. Weickert, J., Scharr, H.: A scheme for coherence-enhancing diffusion filtering with optimised rotation invariance. To appear in J. of Visual Communication and Image Representation (2000)
Distributed Anatomical Brain Connectivity Derived from Diffusion Tensor Imaging Geoffrey J.M. Parker1,2, Claudia A.M. Wheeler-Kingshott1, and Gareth J. Barker1 1
NMR Research Unit, University Department of Clinical Neurology, Institute of Neurology, University College London, Queen Square, London WC1N 3BG, UK 2 Imaging Science and Biomedical Engineering, University of Manchester, Oxford Road, Manchester M13 9PT, UK
[email protected]
Abstract. A method is presented for determining likely paths of anatomical connection between regions of the brain using MR diffusion tensor information. Level set theory, applied using fast marching methods, is used to generate 3-D time of arrival maps, from which connection paths between brain regions may be identified. The method is demonstrated in the normal brain and it is shown that major white matter tracts may be elucidated and that multiple connections and tract branching are allowed. Maps of the likelihood of connection between brain regions are also determined. Two metrics are described for estimating the (informal) likelihood of connection between regions.
1
Introduction
Diffusion tensor imaging (DTI) is an MRI technique developed to allow noninvasive quantification of the self-diffusion of water in vivo (see for example [1,2,3]). Diffusion is anisotropic in many tissues; in particular, brain white matter demonstrates significant anisotropy. High anisotropy reflects both the underlying highly directional arrangement of white matter fibre bundles forming white matter tracts and of their intrinsic microstructure. DTI is able to characterise this anisotropy and to distinguish the principal orientation of diffusion, corresponding to the dominant axis of the bundles of axons making up white matter tracts in any given voxel. However, although DTI provides directional information concerning microscopic tissue fibre orientation at the voxel scale, it provides no explicit connection information between voxels. Early work [4] into anatomical connectivity attempted to group together neighbouring DTI voxels based on a similarity measure reflecting their relative principal diffusion orientations and coincidence. While this approach allows voxels to be grouped into sets that correspond to anatomical tracts, or portions thereof, it does not provide information concerning the fibre directions within these regions, and is therefore poorly suited to determining the route of interregion connectivity. The classification into separate groupings is also binary; there is no attempt at determining the connection likelihood. Other work has M.F. Insana, R.M. Leahy (Eds.): IPMI 2001, LNCS 2082, pp. 106–120, 2001. c Springer-Verlag Berlin Heidelberg 2001
Distributed Anatomical Brain Connectivity
107
attempted to provide the route of connection between regions [5,6,7,8,9]. Each of these approaches follows the paths of white matter tracts by tracing from a start voxel in a point-to-point manner; a single path is produced (although the methods presented in [5] allow for separate paths to merge if they meet). The methods of [6,7,8] may be defined as ‘streamline-like’ approaches, due to their close analogy to standard methods for finding paths through vector fields. The approach of [5] relies on a voxel similarity measure to define a chain of voxels that represents a good path through the tensor field. A balance between the bending energy of putative traces and their faithful following of the directional information provided by the diffusion tensor is achieved using a Markovian approach. Each of the above methods suffers from two major disadvantages that the method presented herein attempts to overcome: firstly, there is no (or at best limited) natural mechanism to allow for the branching of tracts (an anatomically reasonable occurrence, seen, for example, in the corona radiata), meaning that connectivity is restricted to a representation as a one-to-one mapping between voxels in different regions; secondly, there is no attempt to determine how reasonable, or likely, any path is in representing a ‘true’ pathway of connection. The method presented here utilises the principles of level set theory and of the fast marching algorithm [10,11,12]. These techniques model the evolution over time of an interface or front. We hypothesise that the fast marching technique may be used in the context of the diffusion tensor field to propagate fronts that evolve at a rate governed by the directionality of the tensor. We control propagation using the principal eigenvector (1 ) of the tensor. As the 1 field provides a variable rate of propagation for the front, different regions in a volume dataset will be crossed by the front at different times after propagation begins. Maps showing the time of arrival from a start point may therefore be determined for the whole brain. From this information, paths of connection between brain regions may be determined [11,15]. We introduce associated ‘goodness’ metrics describing how likely a putative connection is, based on the information in the DTI data set. This paper describes the four major steps involved in determining anatomical brain connectivity using fast marching tractography: the evolution of a front from a seed point using a variant on the fast marching method; the generation of paths from all points in a given dataset to the seed point; the creation of connectivity maps using a goodness metric; and the selection of a subset of the paths as being reasonable pathways of connection. We also describe two putative goodness metrics. Examples of the results of this process in normal brains are presented.
2 2.1
Methods Data Acquisition
Two sets of DTI brain data were acquired using a GE Signa 1.5 Tesla scanner with a standard quadrature head coil. Diffusion-encoded images were obtained
108
Geoffrey J.M. Parker, Claudia A.M. Wheeler-Kingshott, and Gareth J. Barker
using an echo-planar acquisition with the following parameters: cardiac gating; 96 × 96 acquisition matrix, reconstructed to 128 × 128; 1.875 × 1.875 × 3.0 mm3 or 1.875 × 1.875 × 2.5 mm3 voxels; 42 axial slices; 240 mm FOV; TE 100 ms; 28 non-collinear diffusion-weighting directions [16]. The acquisition time for each dataset was approximately 20 minutes. The second order 3 × 3 symmetric diffusion tensor, its eigenvalues and eigenvectors, and the fractional anisotropy [2] of the tensor were calculated from these data using software developed in-house. 2.2
Front Evolution
The first step towards determining paths of connection to a given start, or seed, point, A ∈ 3 , involves the growth of a volume from this point. This is achieved using a modified version of the fast marching algorithm, a rapid implementation of boundary-value level sets methods. We are primarily interested in the behaviour of the surface of the volume (the front). The rate, F , at which the front propagates from the start point(s) is linked to the information contained in the 1 field (Fig. 1). A number of possibilities exists for the form of this function (for examples see [13,14]). Here we present a new definition of F as a measure of voxel similarity, related to the ideas of voxel linking presented in [5]. Each iteration, p, in the front evolution involves the determination of F (r), where r is the position of a voxel that is a candidate for being occupied by the front during the pth evolution step (a voxel belonging to the ‘narrow band’ (Fig. 1b)): F (r) = min( ( |1 (r) · nd (r)| ), ( |1 (r ) · nd (r)| ), ( |1 (r) · 1 (r )| ) ) .
(1)
r is the position of a voxel neighbouring r that has already been passed by the front, along the direction of the discretised normal to the front, nd (r) (Fig. 1), and F (r) is defined along nd (r). This formula ensures that front evolution will occur most rapidly if both 1 (r) and 1 (r ) are close to co-linear with nd (r), and close to co-linear with each other. Front evolution will be fastest along the white matter tracts, where strong coherence between the 1 in neighbouring voxels is observed (Fig. 1a). The definition of nd (r) involves the following steps: We define S(r) as the set of nearest neighbours, q, to r that have already been passed by the front (q ∈ S(r)) (Fig. 1b). We then define an approximation to the unit normal at r (using 26-neighbour connectivity), using f ∈ {0, 1} to describe whether a voxel, q, has already been passed by the front: n(r) =
∇f . |∇f |
(2)
nd (r) is defined as the unit vector connecting voxel centres most closely approximating n(r). r is then defined as the member of S(r) connected to r in the direction −nd (r). Figure 2a shows a 2-D map of 1 in an axial brain image. Also shown is a map of fractional anisotropy, showing white matter tracts as high signal intensity (Fig.
Distributed Anatomical Brain Connectivity
Front
Grey Matter Front 00 11 11 00 00 11
r’ r 11 00 00 11 00 11
109
Tract
r’ r n(r)
n ε1 (a)
(b)
Fig. 1. Vectors used in the calculation of the speed function, F . (a) The principal eigenvector of diffusion, 1 , is arranged in a directionally coherent manner in tracts, whilst in grey matter this coherence is largely lacking. (b) The relationship between the positions of the front, 1 (r), 1 (r ) (needles), grid points (voxels) passed by front (black circles), grid points in the narrow band (grey circles), and grid points not yet reached by front (white circles). S(r) highlighted by dotted region
2b), due to the highly directional nature of water diffusion in these structures, reflecting the directionally coherent organisation of the tissue microstructure. The evolution of a front using the fast marching algorithm allows a time of arrival, T , from the seed point to any point in the image volume to be determined. F and T are related by the Eikonal equation: |∇T |F = 1 .
(3)
Note that our definition of nd (r) is related to ∇T under the condition that T can be assumed to be approximately equal for all members of S(r): nd (r) ≈ n(r) ≈
∇T . |∇T |
(4)
To provide a value of T (r) we approximate Eq. 3 to T (r) = T (r ) +
|r − r | . F (r)
(5)
This construction ensures that information concerning the values of T (r) propagates only from the voxel with which the value of F (r) is determined (Eq. 1), along nd (r). Based on the methods of Sethian [10], we define a set of all points lying just outside the front (the narrow band), and which are candidates for inclusion within the front (Fig. 1b). The grid point into which the front propagates at iteration p is the member of the narrow band with the smallest value of T .
110
(a)
(c)
Geoffrey J.M. Parker, Claudia A.M. Wheeler-Kingshott, and Gareth J. Barker 1111111111111111111111111111111111111111111111 0000000000000000000000000000000000000000000000 0000000000000000000000000000000000000000000000 1111111111111111111111111111111111111111111111 0000000000000000000000000000000000000000000000 1111111111111111111111111111111111111111111111 0000000000000000000000000000000000000000000000 1111111111111111111111111111111111111111111111 0000000000000000000000000000000000000000000000 1111111111111111111111111111111111111111111111 0000000000000000000000000000000000000000000000 1111111111111111111111111111111111111111111111 0000000000000000000000000000000000000000000000 1111111111111111111111111111111111111111111111 0000000000000000000000000000000000000000000000 1111111111111111111111111111111111111111111111 0000000000000000000000000000000000000000000000 1111111111111111111111111111111111111111111111 0000000000000000000000000000000000000000000000 1111111111111111111111111111111111111111111111 0000000000000000000000000000000000000000000000 1111111111111111111111111111111111111111111111 0000000000000000000000000000000000000000000000 1111111111111111111111111111111111111111111111 0000000000000000000000000000000000000000000000 1111111111111111111111111111111111111111111111 0000000000000000000000000000000000000000000000 1111111111111111111111111111111111111111111111 0000000000000000000000000000000000000000000000 1111111111111111111111111111111111111111111111 0000000000000000000000000000000000000000000000 1111111111111111111111111111111111111111111111 0000000000000000000000000000000000000000000000 1111111111111111111111111111111111111111111111 0000000000000000000000000000000000000000000000 1111111111111111111111111111111111111111111111 0000000000000000000000000000000000000000000000 1111111111111111111111111111111111111111111111 0000000000000000000000000000000000000000000000 1111111111111111111111111111111111111111111111 0000000000000000000000000000000000000000000000 1111111111111111111111111111111111111111111111 0000000000000000000000000000000000000000000000 1111111111111111111111111111111111111111111111 0000000000000000000000000000000000000000000000 1111111111111111111111111111111111111111111111 0000000000000000000000000000000000000000000000 1111111111111111111111111111111111111111111111 0000000000000000000000000000000000000000000000 1111111111111111111111111111111111111111111111 0000000000000000000000000000000000000000000000 1111111111111111111111111111111111111111111111 0000000000000000000000000000000000000000000000 1111111111111111111111111111111111111111111111 0000000000000000000000000000000000000000000000 1111111111111111111111111111111111111111111111 0000000000000000000000000000000000000000000000 1111111111111111111111111111111111111111111111 0000000000000000000000000000000000000000000000 1111111111111111111111111111111111111111111111 0000000000000000000000000000000000000000000000 1111111111111111111111111111111111111111111111 0000000000000000000000000000000000000000000000 1111111111111111111111111111111111111111111111 0000000000000000000000000000000000000000000000 1111111111111111111111111111111111111111111111 0000000000000000000000000000000000000000000000 1111111111111111111111111111111111111111111111 0000000000000000000000000000000000000000000000 1111111111111111111111111111111111111111111111 0000000000000000000000000000000000000000000000 1111111111111111111111111111111111111111111111 0000000000000000000000000000000000000000000000 1111111111111111111111111111111111111111111111 0000000000000000000000000000000000000000000000 1111111111111111111111111111111111111111111111 0000000000000000000000000000000000000000000000 1111111111111111111111111111111111111111111111 0000000000000000000000000000000000000000000000 1111111111111111111111111111111111111111111111 0000000000000000000000000000000000000000000000 1111111111111111111111111111111111111111111111 0000000000000000000000000000000000000000000000 1111111111111111111111111111111111111111111111 0000000000000000000000000000000000000000000000 1111111111111111111111111111111111111111111111 0000000000000000000000000000000000000000000000 1111111111111111111111111111111111111111111111 0000000000000000000000000000000000000000000000 1111111111111111111111111111111111111111111111 0000000000000000000000000000000000000000000000 1111111111111111111111111111111111111111111111 0000000000000000000000000000000000000000000000 1111111111111111111111111111111111111111111111 0000000000000000000000000000000000000000000000 1111111111111111111111111111111111111111111111 0000000000000000000000000000000000000000000000 1111111111111111111111111111111111111111111111 0000000000000000000000000000000000000000000000 1111111111111111111111111111111111111111111111 0000000000000000000000000000000000000000000000 1111111111111111111111111111111111111111111111 0000000000000000000000000000000000000000000000 1111111111111111111111111111111111111111111111 0000000000000000000000000000000000000000000000 1111111111111111111111111111111111111111111111 0000000000000000000000000000000000000000000000 1111111111111111111111111111111111111111111111 0000000000000000000000000000000000000000000000 1111111111111111111111111111111111111111111111 0000000000000000000000000000000000000000000000 1111111111111111111111111111111111111111111111 0000000000000000000000000000000000000000000000 1111111111111111111111111111111111111111111111 0000000000000000000000000000000000000000000000 1111111111111111111111111111111111111111111111 0000000000000000000000000000000000000000000000 1111111111111111111111111111111111111111111111 0000000000000000000000000000000000000000000000 1111111111111111111111111111111111111111111111 0000000000000000000000000000000000000000000000 1111111111111111111111111111111111111111111111 0000000000000000000000000000000000000000000000 1111111111111111111111111111111111111111111111 0000000000000000000000000000000000000000000000 1111111111111111111111111111111111111111111111 0000000000000000000000000000000000000000000000 1111111111111111111111111111111111111111111111 0000000000000000000000000000000000000000000000 1111111111111111111111111111111111111111111111 0000000000000000000000000000000000000000000000 1111111111111111111111111111111111111111111111 0000000000000000000000000000000000000000000000 1111111111111111111111111111111111111111111111 0000000000000000000000000000000000000000000000 1111111111111111111111111111111111111111111111 0000000000000000000000000000000000000000000000 1111111111111111111111111111111111111111111111 0000000000000000000000000000000000000000000000 1111111111111111111111111111111111111111111111 0000000000000000000000000000000000000000000000 1111111111111111111111111111111111111111111111 0000000000000000000000000000000000000000000000 1111111111111111111111111111111111111111111111 0000000000000000000000000000000000000000000000 1111111111111111111111111111111111111111111111 0000000000000000000000000000000000000000000000 1111111111111111111111111111111111111111111111 0000000000000000000000000000000000000000000000 1111111111111111111111111111111111111111111111 0000000000000000000000000000000000000000000000 1111111111111111111111111111111111111111111111 0000000000000000000000000000000000000000000000 1111111111111111111111111111111111111111111111 0000000000000000000000000000000000000000000000 1111111111111111111111111111111111111111111111 0000000000000000000000000000000000000000000000 1111111111111111111111111111111111111111111111 0000000000000000000000000000000000000000000000 1111111111111111111111111111111111111111111111 0000000000000000000000000000000000000000000000 1111111111111111111111111111111111111111111111 0000000000000000000000000000000000000000000000 1111111111111111111111111111111111111111111111 0000000000000000000000000000000000000000000000 1111111111111111111111111111111111111111111111 0000000000000000000000000000000000000000000000 1111111111111111111111111111111111111111111111 0000000000000000000000000000000000000000000000 1111111111111111111111111111111111111111111111 0000000000000000000000000000000000000000000000 1111111111111111111111111111111111111111111111 0000000000000000000000000000000000000000000000 1111111111111111111111111111111111111111111111 0000000000000000000000000000000000000000000000 1111111111111111111111111111111111111111111111 0000000000000000000000000000000000000000000000 1111111111111111111111111111111111111111111111 0000000000000000000000000000000000000000000000 1111111111111111111111111111111111111111111111 0000000000000000000000000000000000000000000000 1111111111111111111111111111111111111111111111 0000000000000000000000000000000000000000000000 1111111111111111111111111111111111111111111111 0000000000000000000000000000000000000000000000 1111111111111111111111111111111111111111111111 0000000000000000000000000000000000000000000000 1111111111111111111111111111111111111111111111 0000000000000000000000000000000000000000000000 1111111111111111111111111111111111111111111111 0000000000000000000000000000000000000000000000 1111111111111111111111111111111111111111111111 0000000000000000000000000000000000000000000000 1111111111111111111111111111111111111111111111 0000000000000000000000000000000000000000000000 1111111111111111111111111111111111111111111111 0000000000000000000000000000000000000000000000 1111111111111111111111111111111111111111111111 0000000000000000000000000000000000000000000000 1111111111111111111111111111111111111111111111
(b)
(d)
Fig. 2. (a) Principal eigenvectors of the diffusion tensor (1 ), modulated with the principal eigenvalue. (b) Fractional anisotropy map. (c) T map. Dark ≡ low T values (early arrival); bright ≡ high T values (late arrival). Black pixels represent CSF or regions outside the brain parenchyma. (d) Traces from arbitrary points to start point (point of confluence of all paths is the start point for the front propagation process) using discrete gradient descent through T map (c)
Equivalently (Eq. 3), this ensures that the front will evolve at the point at which F is highest, ensuring rapid propagation along the direction of 1 , and slow propagation in other directions. Values of T in the narrow band are tested to see if they need to be updated at each iteration step. If u is the position of a voxel that was an unsuccessful candidate at the previous iteration step (p − 1), the value of T (u) is updated only if F (u)p > F (u)p−1 . Figure 2c shows an illustrative 2-D axial example (i.e. the z component of 1 is set to zero) of front propagation from a seed region placed in the splenium of the corpus callosum. The front propagates at a rate determined by the local
Distributed Anatomical Brain Connectivity
111
1 value. Fastest propagation occurs along the white matter tracts where 1 directional coherence is high. 2.3
Determining Paths of Connection
Level set and fast marching methods may be used to construct minimum cost paths through weighted domains [11,15]. We interpret the diffusion tensor field as providing this cost function. The time, T , at which the front reaches each point in the image is determined by the cumulative effect of the F values experienced by the front up to that point. |∇T | (Eq. 3) may therefore be interpreted as a cost function affecting the rate of front propagation. More specifically, given a cost function G(x1 , x2 , . . . , xn ) and a starting point A ∈ 3 , it is possible to define a path γ(τ ) : [0, ∞) → 3 from a seed point, A, to any point r ∈ 3 , that minimises the integral
r=γ(L)
G(γ(τ ))dτ ,
(6)
A=γ(0)
where L is the total length of γ and τ is the position on γ [11]. As we require the tensor field to provide the cost function, a natural definition of G is obtained by direct substitution of the |∇T | values calculated by the fast marching process [11], as this ensures that cost will be low when the 1 are being followed faithfully and be high for increments that do not follow 1 . The fast marching algorithm ensures that the minimum cost incurred in travelling from A to r is the time of arrival, T (r). This implies a path between A and r satisfying:
(r)
|∇T |(γ(τ ))dτ .
T (r) = min γ
(7)
(A)
The minimum cost path back to the seed point, A, may be found by gradient descent through T [11,15]. This process is shown from arbitrary points in the previous 2-D example in Fig. 2c,d, allowing any point within the data set to be connected to the original start point. Back propagation from different points may lead to paths that merge (or, equivalently, branch when viewed from the seed point outwards). Gradient descent is achieved either by using discrete steps between voxel centres (26-neighbour connectivity) or by a Euler approximation with constant time step. For the discrete approach ∇T is replaced by the discretised normal, nd , ensuring that the voxels involved in propagating information at each iteration of the fast marching process (Eqs. 1 and 5) are linked. For presentation purposes, the resulting discrete paths are smoothed using a moving average of width 3. 2.4
‘Goodness’ Metrics
Each point in T (i.e. every point within the brain) may be connected to the seed point using the gradient descent method (Fig. 2d). The likelihood that any
112
Geoffrey J.M. Parker, Claudia A.M. Wheeler-Kingshott, and Gareth J. Barker
given path is representative of a true anatomical connection may be estimated by determining a ’goodness metric’, φ. By mapping φ throughout the brain, all points with a high likelihood of connection to the seed point may be identified. By setting a threshold in φ, the most likely pathways may be extracted. A possible definition of φ for any putative pathway, γ, may be constructed using the speed function, F , as determined during the front evolution process: φ1 (γ) = min F ( γ(τ )) τ
= min τ
1 . |∇T |( γ(τ ))
(8)
This formulation uses the intrinsic cost function defined in the path generation process (Eq. 7), and may therefore be seen as a ‘natural’ choice for a goodness metric. An alternative metric may be defined by analysing the relationship between the path tangent and the underlying 1 direction. We employ the scalar product between the tangent, w, and 1 : φ2 (γ) = max ( 1 − |w( γ(τ )) · 1 ( γ(τ ))| ) . τ
(9)
This formulation may be interpreted as assessing how faithful the paths found using the above methods are to the underlying arrangement of the 1 field.
(a)
(b)
Fig. 3. (a) φ1 map. (b) φ2 map. Points with a high φ are bright, implying a high likelihood of connection to the start point (arrow ). 2-D front propagation as in Fig. 2
Both metrics assign an informal likelihood for a given pathway based on the worst case along the length of the path of the property they are sensitive to. Figure 3 shows application of the 2 metrics in the previous 2-D example. Both
Distributed Anatomical Brain Connectivity
113
show high connectivity between the seed point and regions in a crescent pattern posterior to the corpus callosum, as would be expected from inspection of the arrangement of the 1 field (Fig. 2).
3
Results
Figure 4 shows maximum intensity projections (MIPs) of φ1 and φ2 in a 3-D example in the motor pathway. The MIPs are normalised into the Talairach co-ordinate system [17] using SPM99 [Wellcome Dept. of Cognitive Neurology, Institute of Neurology, UCL, London, UK]. The seed point was placed in the middle portion of the cerebral peduncle. The form of the corticospinal tract is clearly visible.
(a)
(b)
Fig. 4. MIPs in Talairach co-ordinates (grid ). Bright implies high φ. (a) φ1 ; (b) φ2 . Paths determined using discrete gradient descent
Talairach-normalised maps of φ1 and φ2 at different axial levels reveal that the seed region is connected to the region of white matter immediately adjacent to the primary motor area and the supplementary motor area (approximate Talairach co-ordinates (-20,-20,65) and (-10,-10,65), respectively) (Fig. 5). The maps also show that the route taken passes through the posterior limb of the internal capsule, consistent with the expected anatomical pathway. The seed region is also connected to the pyramids in the medulla, again consistent with the known route of the corticospinal tract. It is evident from Figs. 4 and 5 that metric 2 provides an estimate of the likelihood of connection within the brain with higher contrast between wellconnected and less well-connected regions. Also, Fig. 5b shows that likelihoods based on φ1 produce more regions on the contralateral side of the brain to the
114
Geoffrey J.M. Parker, Claudia A.M. Wheeler-Kingshott, and Gareth J. Barker
i
ii
iii
(a)
iv
(b)
(c)
Fig. 5. Maps in Talairach co-ordinate system of (a) Fractional anisotropy, (b) φ1 , (c) φ2 . i) Left pyramid; ii) posterior limb of the internal capsule; iii) Talairach co-ordinates (-15,-10,52); iv) Talairach co-ordinates (-16,-12,52). From top to bottom, images are at approximate Talairach z co-ordinates -50, 10, and 52
seed point with relatively high φ values - a situation which is unlikely to represent true anatomical connectivity. For these reasons we judge φ2 as being superior to φ1 . Figure 6 shows the 1% most likely paths from the seed point used for Figs. 4 and 5, when using φ2 as the goodness metric (discrete gradient descent method). It is apparent that connections to the motor region of the brain have been found, even though a region of relatively low diffusion tensor anisotropy has had to have been traversed. The principal tract of interest is well described in Figs. 4–6, with other areas of the brain showing low connection likelihood. However, one notable ‘false-positive’ is apparent in Fig. 6; the algorithm assigns a region in the corpus callosum as having high connection likelihood, and this area is shown as connected, which is likely to be erroneous. However, when viewed in Fig. 5, this region has a somewhat lower likelihood than much of the rest of the ‘high φ region’, suggesting that the arbitrary threshold of 1% for selecting paths is inadequate. Further experiments were performed on a second dataset using φ2 and the Euler gradient descent method. The 1 % most likely paths were traced from start points in the mid-cerebral peduncle (as before) and in each of the optic radiations, from a position lateral to the lateral geniculate nucleus. Figure 7 shows the results in the corticospinal tract. Figure 8 shows the results in the optic radiations. No significant false-positives were observed in these experiments.
Distributed Anatomical Brain Connectivity
115
Fig. 6. Coronal (through the corticospinal tract) and axial (through the level of the pyramids) fractional anisotropy maps plus paths generated using discrete gradient descent method. Solid arrows show low anisotropy due to the presence of crossing fibres. Dashed arrow shows probable false-positive paths
4
Discussion
We have shown results of diffusion tensor tractography using fast marching tractography (FMT). We have shown that in the normal brain it is possible to use this approach to follow major white matter tracts. Examples have been presented in the motor pathway and the optic radiation in two individuals. The examples presented show that maps of connectivity may be obtained that are in agreement with known anatomical and functional connectivity. The use of the Talairach co-ordinate system provides a degree of validation of the pathways of connection found. Regions of gross functional anatomy may be identified, in particular the motor areas. This validation approach is therefore appropriate for the corticospinal tract. However, there is no general ’goldstandard’ of anatomical connectivity in the human brain; surprisingly little is known about human brain anatomical connectivity. This lack of information is due to the fact that the tract tracing methods successfully used for determining connections in animal brains cannot be applied to humans due to their invasive nature. Therefore, the only human connectivity data available so far stems from gross dissection of the human brain [17], histological staining techniques of major fibre bundles [18], and degeneration studies in patients [19,20,21]. These techniques cannot be applied generally in vivo but can only be used post mortem or under specific disease conditions. These constraints do not apply to many animal models. In particular, detailed knowledge of anatomical connectivity is available in the macaque brain, forming the closest well-described model to the human brain. A study is under way examining the relationship between DTI tractography using the fast marching method and the well-known cerebral anatomical
116
Geoffrey J.M. Parker, Claudia A.M. Wheeler-Kingshott, and Gareth J. Barker
Fig. 7. Coronal (through the corticospinal tract) and sagittal (through the brain midline) fractional anisotropy maps plus paths connected to the mid portion of the cerebral peduncle. Paths generated using Euler gradient descent method
connectivity of this animal [22]. Such validation studies are required to allow future studies into the relative merits of different tractography approaches. Figures 6 and 7 demonstrate the ability of this method to trace paths through relatively low anisotropy regions, and successfully identify the connection to the motor area. The region in question has low anisotropy values due to the presence of crossing fibres, demonstrating the limitations of the relatively coarsescale DTI technique in resolving small-scale pathways. Whilst the fast marching tractography approach is able to continue through such regions, and find the ‘true’ pathways of interest, the possibility of false positives must not be ignored. Likewise, although multiple points have been shown to have a high likelihood of connection to the start region, the possibility of false negatives cannot be ruled out. Previous preliminary work has included a measure of the tensor anisotropy in the definition of the speed function, F [13,14]. Here we deal only with 1 , allowing grey matter structures, such as the thalamus, to be included in the tractography process. This is also likely to contribute to the ability to follow tracts through the low anisotropy regions discussed above. A possible development to reduce false positive rates would be to use a quantitative threshold for the φ values, rather than a simple ‘best centile’ approach. The quantitative nature of the goodness metrics should lend themselves to such a thresholding approach; only a justification for setting a threshold without having to resort to arbitrary decisions is required. We have concluded that goodness metric 2 is superior to metric 1, even though φ1 may be a more natural choice. The reason for the better performance of φ2 may be that it is insensitive to the of curvature of a path. The definition of
Distributed Anatomical Brain Connectivity
117
Fig. 8. Axial fractional anisotropy map at the level of the optic radiations, showing bilateral top 1 % traces, determined using the Euler gradient descent method. Start point on each tract highlighted by black circles
φ1 involves the fast marching speed function, F , relating neighbouring eigenvector orientations and coincidence, and hence has an explicit curvature component. When assessing putative pathways, a significant penalty on curvature is not necessarily desirable, hence the better performance of φ2 , which depends only on the relationship between the path tangent and the underlying eigenvector field. This argument then raises the question as to whether the definition of F (Eq. 1) for the front propagation process is the most suitable choice, as the definition given here has an implicit curvature dependance. Further work will investigate alternative speed functions and goodness metrics. The incorporation into the speed function of diffusivity information reflecting the form of the whole tensor may yield benefits under some conditions. An obvious alternative to the use of ’worst case’ goodness metrics, such as φ1 and φ2 , is to define a metric with characteristics more globally related to the nature of the path. This approach appears attractive from the point of view of reducing the possible detrimental effect of noise on the estimation of path likelihood. However, the use of a ’worst case’ metric provides the benefit of reducing the risk of spurious paths being found due to the selection of separate, abutting or crossing paths as a single path - such occurrences may have a small effect on global estimates of goodness, whilst having a decisive effect when using a ’worst case’
118
Geoffrey J.M. Parker, Claudia A.M. Wheeler-Kingshott, and Gareth J. Barker
approach. Additionally, worst-case metrics are naturally pessimistic, and are therefore likely to provide some protection against false-positive results. Further work is also required in the development of more formal likelihood measures. Such metrics have been developed utilising the bending energy of individual paths [5], and are likely to be applicable to our approach. However, the use of path curvature is not necessarily appropriate, as discussed above. The methodology presented here is related in its philosophy to graph search methods, as suggested in [23] for non-diffusion MR brain images. However, the main aim of [23] was to measure brain morphometry, and connectivity based on diffusion information was not determined. Other workers have determined pathways of connection using other methods [4,5,6,7,8,9], but have not generally allowed for tract branching or generated measures of the likelihood of connection between regions. The time for the analysis of a 42-slice whole-brain DTI dataset using the fast marching tractography approach as outlined in this work, including each of the steps outlined in the methods section, is of the order of 10 minutes. Such an analysis allows all points in the brain to be assessed for their likelihood of connection to a seed point. This time may be reduced by restricting the analysis volume using prior knowledge of anatomical connectivity. The fast marching tractography approach allows the generation of both connectivity maps and connection paths. These appear reasonable in comparison with known normal anatomy. The principal advantages of this method are the possibility for branching of paths, and the estimation of the likelihood of connection of any brain region to a start region. These two benefits in combination make it unique in the study of connectivity using DTI data to date.
Acknowledgements This work was supported by the Multiple Sclerosis Society of Great Britain and Northern Ireland. The contributions of Klaas Stephan, Olga Ciccarelli, Sofia Eriksson, David Werring, and Olivier Coulon are gratefully acknowledged.
References 1. Basser, P.J., Mattiello, J., Le Bihan, D.: Estimation of the Effective Self-Diffusion Tensor from the NMR Spin Echo. J. Magn. Reson. B. 103 (1994) 247–254 2. Pierpaoli, C., Basser, P.J.: Toward a Quantitative Assessment of Diffusion Anisotropy. Magn. Reson. Med. 36 (1996) 893–906 3. Basser, P.J., Pierpaoli, C.: Microstructural and Physiological Features of Tissues Elucidated by Quantitative-Diffusion-Tensor MRI. J. Magn. Reson. B. 111 (1996) 209–219 4. Jones, D.K., Simmons, A., Williams, S.C.R., Horsfield, M.A.: Non-Invasive Assessment of Axonal Fiber Connectivity in the Human Brain via Diffusion Tensor MRI. Magn. Reson. Med. 42 (1999) 37–41
Distributed Anatomical Brain Connectivity
119
5. Poupon, C., Clark, C.A., Froulin, V., et al.: Regularization of Diffusion-Based Direction Maps for the Tracking of Brain White Matter Fascicles. NeuroImage 12 (2000) 184–195 6. Conturo, T.E., Lori, N.F., Cull, T.S., et al.: Tracking Neuronal Fiber Pathways in the Living Human Brain. Proc. Nat. Acad. Sci. USA 96 (1999) 10422–10427 7. Mori, S., Crain, B.J., Chacko, V.P., van Zijl, P.C.M.: Three-Dimensional Tracking of Axonal Projections in the Brain by Magnetic Resonance Imaging. Ann. Neurol. 45 (1999) 265–269 8. Basser, P.J., Pajevic, S., Pierpaoli, C., Duda, J., Aldroubi, A.: In Vivo Fiber Tractography Using DT-MRI Data. Magn. Reson. Med. 44 (2000) 625–632 9. Tuch, D.S., Belliveau, J.W., Wedeen, V.J.: A Path Integral Approach to White Matter Tractography. In: Proceedings of the 8th meeting of the International Society for Magnetic Resonance in Medicine. (2000) 791 10. Sethian, J.A.: A Fast Marching Level Set Method for Monotonically Advancing Fronts. Proc. Nat. Acad. Sci. USA 93 (1996) 1591–1595 11. Sethian, J.A.: Level Set Methods and Fast Marching Methods. 2nd edn. Cambridge University Press, Cambridge (1999) 12. Malladi, R. and Sethian, J.A.: An O(N logN ) Algorithm for Shape Modeling. Proc. Nat. Acad. Sci. USA 93 (1996) 9389–9392 13. Parker, G.J.M. and Dehmeshki, J.: A Fast Marching Analysis of MR Diffusion Tensor Imaging for Following White Matter Tracts. In: Medical Image Understanding and Analysis MIUA2000 (2000) 185–188 14. Parker, G.J.M. and Dehmeshki, J.: A Level Sets Approach to Determining Brain Region Connectivity. In: Proceedings of the 1st International Workshop on Image and Signal Processing and Analysis IWISPA 2000, 22nd International conference on Information Technology Interfaces (2000) 145–150 15. Kimmel, R. and Sethian, J.A.: Computing Geodesic Paths on Manifolds. Proc. Natl. Acad. Sci. USA 95 (1998) 8431–8435 16. Jones, D.K., Horsfield, M.A., Simmons, A.: Optimal Strategies for Measuring Diffusion in Anisotropic Systems by Magnetic Resonance Imaging. Magn. Reson. Med. 42 (1999) 515–525 17. Talairach, J. and Tournoux, P.: Co-planar Stereotaxic Atlas of the Human Brain. Georg Thieme Verlag, Stuttgart (1988) 18. B¨ urgel, U., Schormann, T., Schleicher, A., Zilles, K.: Mapping of Histologically Identified Long Fiber Tracts in Human Cerebral Hemispheres to the MRI Volume of a Reference Brain: Position and Spatial Variability of the Optic Radiation. NeuroImage 10 (1999) 489–499 19. Miklossy, J., van der Loos, H.: The Long-Distance Effects of Brain Lesions: Visualization of Myelinated Pathways in the Human Brain Using Polarizing and Fluorescence Microscopy. J. Neuropathol. Exp. Neurol. 50 (1991) 1–15 20. Pujol, R., Marti-Vilalta, J. L., Junque, C., Vendrell, P., Fernandez, J.,and Capdevilla, A.: Wallerian Degeneration of the Pyramidal Tract Studied by Magnetic Resonance Imaging. Stroke 21 (1990) 404–409 21. Werring, D.J., Toosey, A.T., Clark, C.A., Parker, G.J.M., Barker, G.J., Miller, D.H., Thompson, A.J.: Diffusion tensor imaging can detect and quantify corticospinal tract degeneration after stroke. J. Neurol. Neurosurg. Psychiatry 69 (2000) 269–272 22. Stephan, K.E., Parker, G.J.M., Barker, G.J., Rowe, J.B., MacManus, D.G., Passingham, R.E., Lemon, R.N., Turner, R.: In Vivo Tracing of Anatomical Fibre Tracts in the Macaque Monkey Brain by Diffusion Tensor Imaging (DTI). In: Proceedings Human Brain Mapping (2001) (In press)
120
Geoffrey J.M. Parker, Claudia A.M. Wheeler-Kingshott, and Gareth J. Barker
23. Styner, M., Coradi, T., Gerig, G.: Brain Morphometry by Distance Measurement in ˇ amal, M., Todd-Pokropek, A. a Non-Euclidian, Curvilinear Space. In: Kuba, A., S´ (eds.): Information Processing in Medical Imaging IPMI’99. Lecture Notes Computer Science, Vol. 1613. Springer-Verlag, Berlin Heidelberg New York (1999) 364– 369
Study of Connectivity in the Brain Using the Full Diffusion Tensor from MRI Philipp G. Batchelor1, Derek L.G. Hill1 , Fernando Calamante2 , and David Atkinson1 1
Division of Radiological Sciences, King’s College London Institute of Child Health, University College London
2
Abstract. In this paper we propose a novel technique for the analysis of diffusion tensor magnetic resonance images. This method involves solving the full diffusion equation over a finite element mesh derived from the MR data. It calculates connection probabilities between points of interest, which can be compared within or between subjects. Unlike traditional tractography, we use all the data in the diffusion tensor at each voxel which is likely to increase robustness and make intersubject comparisons easier.
1
Introduction
Water molecules in tissue move continuously, and this movement can be exploited to study diffusivity in the brain using MRI [1,2]. Such MR diffusion images define six independent values at each voxel. The six values define a symmetric tensor, which can equivalently be described using 3 eigenvalues and 3 eigenvectors of that tensor. The anisotropy of the diffusion tensor is of particular interest in brain images, as it is related to fibre tracts in white matter [3,4,5,6,7,8]. The technique of MR tractography has recently been proposed to study white matter connectivity in the brain [4,5,6,7] using MR diffusion images. Here, we want to propose a different approach, which uses the complete diffusion tensor. Essentially, we are solving a diffusion equation based on the measured diffusion tensor, the initial condition is a seed at a point which would be used as starting point in tractography. The seed diffuses through the brain, and the amount at some position is interpreted as a probability to reach that point, given the input data. The advantage of this approach is that it is not dependent on a point to point eigenvalue/eigenvector computation, thus in that sense hopefully more robust. It is also intuitively related to the underlying physico-chemical process1 [9,10].
1
which is actually more complicated, but some of the other physical quantities cannot be measured in vivo
M.F. Insana, R.M. Leahy (Eds.): IPMI 2001, LNCS 2082, pp. 121–133, 2001. c Springer-Verlag Berlin Heidelberg 2001
122
2
Philipp G. Batchelor et al.
Methods: Extracting Information
The images produced by Diffusion tensor MRI provide new information on the imaged tissue, not available with classical MR techniques [11]. The images produced, however, are not directly related to the tensor components, which must be computed algebraically [11]. Recent studies have addressed the optimisation of acquisition of diffusion tensor data[6,15]. In this study, we are concerned with the analysis of such data. 2.1
From Diffusion Imaging to Diffusion Tensor
The diffusion weighted images are acquired in different directions. As the tensor has at least six independent components, it is necessary to acquire diffusion weighted images with the diffusion sensitisation in at least six directions ei . We note these images βi , i = 1 . . . n, n ≥ 6. Furthermore, the non weighted image is noted β0 . Then βi log = −b eti Dei (1) β0 We acquired 25 slices using a single shot Echo Planar Images (EPI), with b = 1000 s/mm2 , and voxel size 2 × 2 × 2 mm, and six averages. Here b is related to the diffusion weighting gradient strength and timings. The √ seven directions √ were: the x,√ y, and z coordinate √ axis directions, plus [1, 1, 1]/ 3, [−1, −1, 1]/ 3, [1, −1, −1]/ 3, and [−1, 1, −1]/ 3. Positive Definiteness For the physics to make sense, one should have βi ≤ β0 in order for the left hand side to be negative, so as to make the directional measurements eti Dei positive, which a diffusion tensor should satisfy. Figure 1 shows in white regions where this is not satisfied for β1 /β0 .The tensor is not positive definite outside the head, and also at some voxels within the head. The number of voxels in the brain where the tensor is not positive definite decreases with data averaging, suggesting that the non-positive definite values are caused by measurement noise. We define li := − log ββ0i /(2b), so, for li > 0, we get the system of equations for the unknown diffusion tensor: et1 De1 = Et1 d = l1 .. .. . . etn Den = Etn d = ln where we have defined d = [Dxx , Dyy , Dzz , Dxy , Dxz , Dyz ]. Ei is constructed from the directions ei from the mapping [x, y, z] → [x2 , y 2 , z 2 , 2xy, 2xz, 2yz]. Summarised in matrix form: E := [E1 ; . . . ; E1 ] is the matrix whose rows are the Ei ’s. We get, Ed = [l1 , . . . , l7 ]t with the obvious condition that E has maximal rank. It is still possible for the computed tensor to have negative eigenvalues, as shown in figure 2
Study of Connectivity in the Brain
123
Fig. 1. White: positions where the ratio of the diffusion weighted image in direction e1 is larger than the value in the β0 image at the corresponding position. Middle, one acquisition, right, averaged six times. The β0 is diaplyed on the left
Fig. 2. In this axial slice through the head pixels where the diffusion tensor has a negative eigenvalue are marked as white, left smallest eigenvalue, middle middle eigenvalue
2.2
Classification of Diffusion Related Images
There are various ways of displaying and analysing the diffusion tensor images[16]. One straightforward possibility is to display the tensor components, giving six images in three dimensions. This can be dangerous, however, as the components of a tensor are not invariants of the coordinate system, and the handling of six images is cumbersome. A preferable approach is to use invariants of the tensor[3]. Scalars have the advantage of being easy to use. The eigenvalues are the essential scalar information contained in a tensor. The fundamental invariants are the homogeneous functions of the three eigenvalues: λ1 λ2 λ3 ,
λ1 λ2 + λ2 λ3 + λ1 λ3 ,
λ1 + λ2 + λ3
(2)
124
Philipp G. Batchelor et al.
to which one can add any function of the eigenvalues, for example λλ31 or the relative anisotropy (RA) [3,15]. One could generalise this to study the ratio of the largest eigenvalue to the average of the two other ones, to quantify how cylindrical the tissue is. Usually, the aim is to emphasise the anisotropy of the tensor, as the anisotropy of the measured tensor is supposed to be related to the anisotropy of the tissues. We use the convention that eigenvalues are sorted from 1 to 3 by increasing value. We must take care to avoid negative eigenvalues. If we take the relative anisotropy (RA), which is the ratio of the standard deviation to the mean for the three eigenvalues, the denominator is the mean diffusivity. A negative eigenvalue makes a low denominator, thus a larger anisotropy. This is in general true, negative eigenvalues create large anisotropy. For example, changing one sign of the totally isotropic tensor gives a fractional anisotropy larger than 1, namely 1.1547. Associated with the eigenvalues are the eigenvectors. These define vector fields on the image. Different techniques have been developed to display them. Usually, in what is called tractography, the direction corresponding to the maximal eigenvalue is followed. For example, [4] studies the flow associated with the largest eigenvector: r˙ (t) = emax (r(t))
(3)
One difficulty associated with eigenvalue/eigenvectors is the sorting [17], and particularly multiple eigenvalues, and associated singularities in the vector field. For example, Basser et al in [4] stop following the tract when it reaches the boundary, or a region with low anisotropy, or the radius of curvature is below two voxels, or the principal direction is not the most collinear one with the tract tangent. Because of all the problems associated with noise, such as ambiguities in direction and errors in sorting of the eigenvalues, it can be advantageous to study connection probabilities rather than absolute connections. One approach is to use the fast marching algorithm to propagate a front using the direction of the principal eigenvector [18]. The advantage of our approach is that we use the entire diffusion tensor in an intuitive way, and measuring diffusion of a seed placed in the tensor field allows an elegant definition of probability. 2.3
Diffusion Equation
Traditional tractography enforces a choice, based on sorting at every step. This means rejecting two thirds of the available information, even in cases where the difference between the largest and second largest eigenvalues might be extremely small. Also, even with a smooth family of tensors, the eigenvalues are not necessarily smooth [19]. We propose here a different technique, based on a simple model of diffusion: at a point which would be used as starting point for tractography, a seed is diffused.
Study of Connectivity in the Brain
f (x) =
1 0
in starting region elsewhere
125
(4)
is the seed, used as an initial condition in the partial differential equation (PDE) ∂ u(x, t) ∂t u(x, 0) = f (x)
∇ · D∇u(x, t) = −
(5) (6)
This uses directly the diffusion tensor, and thus avoids the problems related to the computation of the eigenvalues. We interpret it as a probabilistic tractography, which allows the starting value to follow simultaneously all the possible paths around it, with a certain probability. This is an idealised model of diffusion, and a more realistic equation would have to take into account intracellular and extracellular diffusion, convection terms, etc...[9]. To understand the effect of the anisotropy, we compare with the effect of homogeneous isotropic diffusion, we call u ˆ the isotropically diffused seed: ∂ u ˆ(x, t) ∂t u ˆ(x, 0) = u(x, 0) = f (x)
∇ · ∇ˆ u(x, t) = ∆ˆ u(x, t) = −
(7) (8)
In the computation, we use natural boundary conditions, i.e. in the variational formulation, we do not enforce a boundary value. This corresponds to D∇u · n = ∇ˆ u · n = 0 on the boundary. Under this constraint, from Green’s theorem, the mean value is a time constant: u(t) = uˆ(t) = f . This boundary condition means that the normal part of the gradient is zero, in other words, nothing escapes. In terms of heat flow, this would mean that the brain is insulated. Not only are these assumptions intuitive, but as they ensure that the total amount of seed is conserved, enabling us to interpret u(t)/ u a probability. 2.4
Discretisation-Numerics
We have written in equation 5 the continuous form of the equation, but to solve it, it needs to be discretised. As in [20], we note that the discrete equation can be as meaningful as the continuous one. We used a Crank-Nicholson scheme, with Galerkin finite element discretisation in space, and finite difference in time. The finite element method amounts to considering a weak formulation [21] of the equation on a finite dimensional subspace of the space of solutions. This subspace is normally the one generated by continuous, piecewise polynomial functions, whose support is a mesh neighbourhood of a vertex. The diffusion equation is a parabolic equation, and for further theoretical considerations on the method, we refer to [21]. Example Our approach can be clarified by considering a simple abstract two dimensional example. Consider the simplest possible square mesh in figure 3, with just one internal node (node 5), and eight boundary nodes.
126
Philipp G. Batchelor et al. 7
4
1
8
9
5
6
2
3
Fig. 3. A mesh for a square, with one internal node
On it, we will test some simple diffusion tensors, e.g. 1+0 0 1 with > 0. For any value of , small or large, the tractography will impose to move in the x direction. Using the numbering on the figure, we see that the nonzero matrix components affecting the inner node are (15), (25), (45), (55), (85), (95), and (65). Usually, the contribution at a node is computed by summing the integrals over elements containing that node. The discrete version
1
−→
Fig. 4. Left, the initial condition, Right, after one time step, for = 0.1.
of the integral using the finite elements is the sum of the values of the mass matrix times the vector of values at vertices of the solution. This is due to the fact that the finite element basis is not orthogonal, and the mass matrix is the
Study of Connectivity in the Brain
127
matrix of inner products. We give the values given by the discretised diffusion equation on this very small mesh. The values at different time steps is given by iteratively solving a linear system of the form AU = BUold , where A and B are computed from the stiffness, mass, and time step, and the vector U contains the nodal values of the function. At = 0, = 0.1, and = 1, and t = 0.1, in time steps of 0.01, i.e. ten iterations. (remember that the square has sides of length 1): 0.0868 0.1178 0.0206 0.1178 0.3140 0.1178 and 0.0206 0.1178 0.0868
0.0869 0.1168 0.0215 0.1196 0.3104 0.1196 and 0.0215 0.1168 0.0869
0.0859 0.1120 0.0274 0.1294 0.2908 0.1294 0.0274 0.1120 0.0859
This is also illustrated in figure 4. The numbers can be read as probabilities to reach the corresponding point, starting with probability one at node 5. As with all diffusion related techniques, there is still the problem of the stopping time. For the moment, we use an ad-hoc choice.
3
Application
3.1
Diffusion Tensor Components
The images in figure 5 show the diffusion components. It is difficult to grasp all the information contained in a tensor field, using this representation. We can get a very rough idea of what the diffusion tensor look like on average by selecting a ‘brain region’, and a CSF region, as shown in figure 6. In the region of interest, the mean diffusivity, i.e. one third of the trace of the diffusion tensor 2 2 was: 1.038 · 10−3 mm s .
3.2
Anisotropies
It is important to remember that high anisotropy doesn’t mean high diffusivity, and this doesn’t mean high consistency of direction of the eigenvector! In figure 7, we display some of the classical anisotropies [11,15], with the mean diffusivity image. The images displayed here are from axial slices of two volunteers. The mean relative anisotropy (RA) was 0.2841 in the brain, and 0.1472 in the CSF, i.e. the ratio of means anisotropies is approximately in ratio 2:1 whereas the corresponding diffusivity is in ratio 1:3. 2
The region used contains parts of CSF, which has a higher diffusivity and lower 2 anisotropy. The mean diffusivity in a white matter region is ∼ 0.86 · 10−3 mm , in s accordance with the literature
128
Philipp G. Batchelor et al. y
x
z
x
y
z
Fig. 5. Diffusion tensor components
3.3
Diffusion Equation
The first step in solving the diffusion equation is to extract a region of interest. For the purpose of demonstration, we use approximate white matter segmentations, obtained by thresholding the anisotropy. (For better results, a segmented tissue map from the structural MR of the same subject, or even from an atlas could be used in place of the diffusion anisotropy mask, though matching such data to the EPI diffusion images requires a non-rigid registration algorithm that can compensate for MR distortion [22,23]). This limits the solution to the region of interest, and also excludes region with greatest likelihood of neagtive eigenavlues (figure 2). Furthermore, the CSF has high diffusivity, but is not of interest for the connectivity. One of the meshes used is displayed in figure 8. After extracting the regions, we need to convert the diffusion tensor components from values on the voxel dataset to values on the finite element mesh. In figure 9 we display the results at different time steps. For the starting position as chosen (the points positions are shown by crosses in the figure showing the isotropic homogeneous diffusion), the values of the seeded function are shown in table 1.
Study of Connectivity in the Brain
129
Fig. 6. The region of interest, left, and the CSF region chosen for equation Table 1. Values of the solution of the diffusion equation at different points, t = 40[2b]. These should be interpreted as probabilities of reaching the locations xi Point
x0 x1 x2
4
x u(x, 40)
u ˆ(x, 40) 0.121119 0.173109 0.0970939 0.0695148 0.149793 0.00732437
Discussion
It is generally accepted in the diffusion MRI literature that tissue anisotropy should be described in terms of rotation invariant indices based on some function of the eigenvalues, as for example relative anisotropy. There is less agreement on how to efficiently use the diffusion tensor information for purposes such as studying connectivity within the brain. The recently proposed technique of tractography, involving following the principle eigenvector at each voxel, has been widely reported [4,5,6,7], but has some disadvantages. This technique, which we refer to as traditional tractography, ignores large amount of the data collected (the two other eigenvectors, and the size of the eigenvalues), and does not correctly take account of partial volume effects and noise in the data. With traditional tractography, if the wrong direction is chosen at any point (e.g. due to noise) then the rest of the trajectory is wrong. We have proposed an alternative strategy which can be described as probabilistic tractography. In this approach, we solve the full diffusion equation on a finite element grid derived from the MR data. Starting at a selected seed point, the signal diffuses throughout the brain, taking account of the diffusion tensor values at each location, and by comparing the amount of signal that has reached a target point of interest, compared to what would have reached that target in
130
Philipp G. Batchelor et al. FA
FA
RA
RA
D
D
Fig. 7. The anisotropy indices and the mean diffusivity, for two different subjects
an isotropic medium, we can assign probabilities to connections. This can be rerun from as many different seed points as desired. We believe that this method makes fewer assumptions about the data than more traditional approaches, as is illustrated when the two largest eigenvalues are almost identical, rather than having to make a decision at each stage as to the correct direction to follow. A similar finite element methodology has previously been used to study sucrose distribution in the cat brain based on MR diffusion measurements[24], however that study was not related to connectivity analysis. We have demonstrated our technique on a single slice taken from a 3D MR diffusion tensor data set of two subjects, but the approach can easily be extended to three dimensions. We suggest correcting for any diffusion tensors that are not positive definite prior to solving the diffusion equations. As we point out, these corrections should be done anyway, even if the tensor data is being processed merely for anisotropy images, as anisotropy will be erroneously overestimated if one of the eigenvalues is negative. An advantage of our approach is that it facilitates intersubject comparisons. For example, it might be desirable to study the relative strengths of connection between locations A and B compared with C and D in a cohort of subjects. This could be done by seeding points A and C in each subject, running the algorithm, and then comparing the probabilities calculated at points B and D. The locations A-D could be identified in the images in many ways. For some studies, these might be locations of functional activity identified using BOLD fMRI. In other cases, points A-D could be features on an atlas non-rigidly registered to each subject’s diffusion images [22,23]. It is worth noting that intersubject comparison using our approach does not require non-rigid transformation of the tensor values
Study of Connectivity in the Brain
131
Fig. 8. The xx, xy, and yy components of the tensor, sampled over the mesh representing the white matter. The contour of the brain is also displayed. Flat shading is used to show the underlying mesh structure . The gray values thus represent intensities, here mapped linearly from minimum (black) to maximum (white) to a grayscale table
themselves, thus avoiding the difficulties in doing this identified by Alexander and Gee [25].
Acknowledgements We would like to thank Laura Johnson, Donald Tournier, Dr A. Connelly and J. Schnabel, Prof. D. Hawkes, and the EPSRC (Gr/N04867) for funding.
References 1. D. Le Bihan, R. Turner, P. Douek, and N. Patronas. Diffusion MR Imaging: Clinical Applications. AJR, 159:591–599, 1992. 2. A. Szafer, J.Z. Zhong, and J.C. Gore. Theoretical Model for Water Diffusion in Tissues. Magn. Reson. in Med., 33:697–712, 1995. 3. P. Basser and C. Pierpaoli. Microstructural and Physiological Features of Tissues Elucidated by Quantitative-Diffusion-Tensor MRI. Med. Phys., Series B 111:209– 219, 1996. 4. P.J. Basser, S. Pajevic, C. Pierpaoli, J. Duda, and A. Aldroubi. In Vivo Fiber Tractography Using DT-MRI Data. Magn. Reson. in Med., 44:625–632, 2000. 5. T.E. Conturo, N.F. Lori, T.S. Cull, Akbudak E., Snyder A.Z., Shimony J.S., McKinstry R.C., Burton H., and Raichle M.E. Tracking neuronal fiber pathways in the living human brain. Proc. Natl. Acad. Sci. USA, 96:10422–10427, 1999. 6. D.K. Jones, A. Simmons, S.C. Williams, and M.A. Horsfield. Non-invaisce assessment of axonal fiber connectivity in the human brain via diffusion tensor MRI. Magn. Reson. Med., 42:37–41, 1999.
132
Philipp G. Batchelor et al.
Fig. 9. Top: Diffusion, at time t = 20 (left ), and t = 40 (middle), and right, homogeneous, isotropic diffusion at t = 40. Below, the positions used in the table are marked by a sphere, in clockwise order starting from bottom from x0 to x2 ; right a three dimensional view, incorporating the ventricleslabel
7. S. Mori, B.J. Crain, V.P. Chacko, and van Zijl P.C. Three-dimensional tracking of axonal projections in the brain by magnetic resonance imaging. Ann. Neurol., 45:265–269, 1999. 8. C. Pierpaoli and P.J. Basser. Toward a Quantitative Assessment of Diffusion Anisotropy. Magn. Reson. in Med., 36(6), 1996. 9. C. Nicholson and E. Sykov´ a. Extracellular space structure revealed by diffusion analysis. Trends in Neurosciences, 21(5):207–215, 1998. 10. I. Vorisek and E. Sykova. Evolution of Anisotropic Diffusion in the Developing Rat Corpus Callosum. J. Neurophysiol., 78:912–919, 1997.
Study of Connectivity in the Brain
133
11. P. Basser and C. Pierpaoli. A Simplified Method to Measure the Diffusion Tensor from Seven Images. Magn. Reson. in Med., 39:928–934, 1998. 12. D.K. Jones, M.A. Horsfield, and A. Simmons. Optimal Strategies for Measuring Diffusion in Anisotropic Systems by Magnetic Resonance Imaging. Magn. Reson. in Med., 42:515–525, 1999. 13. N.G. Papadakis, D. Xing, C.L.H. Huang, L.D. Hall, and Carpenter T.A. A comparative study of acquisition schemes for diffusion tensor imaging using MRI. J. Magn. Reson., 137(1):67–82, 1999. 14. N.G. Papadakis, C.D. Murrills, L.D. Hall, Huang C.L.H., and T.A. Carpenter. Minimal gradient encoding for robust estimation of diffusion anisotropy. Magn. Reson. Imag., 18(6):671–679, 2000. 15. N.G. Papadakis, D. Xing, G.C. Houston, J.M. Smith, M.I. Smith, M.F. James, A.A. Parsons, C. L.-H. Huang, L.D. Hall, and T.A. Carpenter. A Study of Rotationally Invariant and Symmetric Indices of Diffusion Anisotropy. Magn. Reson. in Med., 17(6):881–892, 1999. 16. G. Kindlmann, D. Weinstein, and D. Hart. Strategies for Direct Volume Rendering of Diffusion Tensor Fields. IEEE Trans. Vis. and Comput. Graphics, 6(2):124–138, 2000. 17. K.M. Martin, N.G. Papadakis, C. L.-H. Huang, L.D. Hall, and T.A. Carpenter. The Reduction of the Sorting Bias in the Eigenvalues of the Diffusion Tensor. Magn. Reson. Imag., 17(6):893–901, 1999. 18. G.J.M. Parker and J. Dehmeshki. A Fast Marching Analysis of MR Diffusion Tensor Imaging for Following White Matter Tracts. In Proceedings, MIUA 2000, pages 185–188, 2000. 19. T. Kato. Perturbation Theory for Linear Operators. Classics in Mathematics. Springer, 1995, reprint of 1980 edition. 20. Langtangen Hans Petter. Computational Partial Differential Equations. Number 2 in Lecture Notes in Computational Science and Engineering. Springer, 1999. 21. Thom´ee Vidar. Galerkin Finite Element Methods for Parabolic Problems. Springer Series in Computational Sciences. Springer, 1997. 22. D. L. G. Hill, C. R. Maurer Jr., A. J. Martin, S. Sabanathan, W. A. Hall, D. J. Hawkes, D. Rueckert, and C. L. Truwit. Assessment of intraoperative brain deformation using interventional MR imaging. In C. Taylor and A. Colchester, editors, Medical Image Computing and Computer Assisted Inter, volume 1679 of Lecture Notes in Computer Science, pages 910–919. Springer Verlag, 1999. 23. C. R. Maurer Jr., D. L. G. Hill, A. J. Martin, H. Liu, M. McCue, D. Rueckert, D. Lloret, W. A. Hall, R. E. Maxwell, D. J. Hawkes, and C. L. Truwit. Investigation of intraoperative brain deformation using a 1.5T interventional MR system: preliminary results. IEEE Transactions on Medial Imaging, 17(5):817–825, 1998. 24. P.G. Mc Queen, A.J. Jin, C. Pierpaoli, and P.J. Basser. A Finite Element Model of Molecular Diffusion in Brain Incorporating in vivo Diffusion Tensor MRI Data. In ISMRM, page 193, 1996. 25. D.C. Alexander and J.C. Gee. Elastic Matching of Diffusion Tensor Images. Computer Vision and Image Understanding, 77:233–250, 2000.
Incorporating Image Processing in a Clinical Decision Support System Paul Taylor, Eugenio Alberdi, Richard Lee, John Fox, Margarita Sordo, and Andrew Todd-Pokropek Centre for Health Informatics and Multiprofessional Education, University College London, Archway Campus, Highgate Hill, London, UK, N19 3UA
[email protected]
Abstract. A prototype system to assist radiologists in the differential diagnosis of mammographic calcifications is presented. Our approach is to incorporate image-processing operators within a knowledge-based decision support system. The work described in this paper involves three stages. The first is to identify a set of terms that can represent the knowledge required in an example of radiological decision-making. The next is to identify image processing operators to extract the required information from the image. The final stage is to provide links between the set of symbolic terms and the image processing operators.
1
Introduction
The intended application of our work is breast X-rays or mammograms. Mammography is used in the USA and in many European countries to screen for breast cancer. It is the investigation of choice throughout the diagnosis, management and followup of the disease. Work in computer aids for the interpretation of mammograms began in the 1980s and the volume of articles published in the field continues to grow. The main thrust of this work has been on the detection of abnormalities. There are a variety of clinical signs for which radiologists search when reporting mammograms. One of the most important is that of calcification. The chief difficulty is that calcifications are not necessarily indicative of cancer and the differentiation of benign and malignant, or potentially malignant, calcifications requires high-level radiological expertise. The best algorithms are able to detect calcifications with very high levels of sensitivity. These levels of sensitivity are, however, only achieved at some cost in specificity. Attempts to use computers in mammography have therefore tended to rely on the human film-reader to maintain specificity. The idea is that the computer can be used to prompt for abnormalities. The prompts will alert the user to signs he or she might have missed, thus increasing sensitivity without, it is hoped, an adverse impact on specificity[1]. Researchers in this paradigm have tried to establish what impact false positive prompts have on the human film-reader. Is there a threshold for the false M.F. Insana, R.M. Leahy (Eds.): IPMI 2001, LNCS 2082, pp. 134–140, 2001. c Springer-Verlag Berlin Heidelberg 2001
Incorporating Image Processing in a Clinical Decision Support System
135
positive rate, above which the prompts cease to be helpful? It seems that there is no simple answer to this question[2]. One of the factors that affects performance is the radiologists’ understanding of the basis for the false positive prompts[3]. A goal for our work is that the system should be able to generate an explanation of why a certain set of calcifications are considered to be potentially malignant. The United Kingdom screening programme is facing a severe and worsening manpower crisis[4]. It is already necessary to involve non-radiologists in the interpretation of films. The evidence suggests that radiographers can be trained to interpret screening films with similar performance to radiologists[5]. We are keen to develop a system which can support the training of non-radiologist film readers, and believe that such a system must include a representation of the knowledge radiologists acquire when learning to interpret screening images. One of the problems facing mammography today is that improvements in the image acquisition process mean that many more signs are detected than was the case a few years ago. A pressing goal of research in mammographic image analysis is therefore to improve the capacity of the radiologist accurately and consistently to assess the level of risk associated with calcifications. A third goal for our work is to to communicate useful information about the risk associated with calcifications. In the next section we give an overview of the prototype system we are developing. We then go on to describe in more detail the development of the knowledge base, the image-processing operators incorporated into the system and the basis for the mapping between the two.
2
Overview of the Design of CADMIUM II
The prototype developed here is based on an earlier system known as CADMIUM (Computer Assisted Decision Making for Image Understanding in Medicine) [6]. CADMIUM II has a different architecture based on a client-server model. There are three principal components to the architecture. The CADMIUM client handles interactions with the user and the display of all required images. The CADMIUM server provides longer-term storage of the image data and is responsible for all image processing. The third element in the design is called the Solo Server. The Solo Server is a decision support engine. It uses a language known as Proforma which has been designed to represent clinical guidelines[7]. CADMIUM Client The CADMIUM Client supports the user interface and controls interaction with the CADMIUM Server and the Solo Server. When the user requests a case, the relevant images are retrieved from the CADMIUM Server. If he or she then requests decision support for that case, the Client calls the Solo Server, which handles the reasoning, and obtains the results of image processing from the CADMIUM Server. CADMIUM Server The CADMIUM server stores all of the patient data and all of the images. All image processing operations are performed on the server. The images used in the
136
Paul Taylor et al.
prototype are mammograms digitised at a resolution of 50 microns and rescaled to 100 microns. The image processing algorithms are executed on the Server. The results of the image processing are then made available to the CADMIUM Client. Solo Server The Solo Server stores the Proforma protocols that represent the clinical decisions supported by the system and performs the symbolic reasoning. The representation of a decision includes the set of candidate options, the arguments that serve to increase or decrease the support for a candidate. The decision of interest in our application is the differential diagnosis of calcifications and the candidates are the three diagnostic classes corresponding to the three different management options: benign, malignant and indeterminate. The arguments are based on properties of the calcifications.
3
Development of the Knowledge Base
The aim is to develop a computer aid that provides support in the form of arguments that are expressed in terms that are familiar to radiologists and other trained film-readers. The development of these arguments involved three steps. First, working with radiologists we built a representation of the protocol followed in the UK breast screening programme. This included an element corresponding to the decision made in assessing calcifications. Next, we carried out studies to identify an appropriate set of terms to use in arguments based on the characteristics of calcifications. Finally we identified the published evidence that could provide a sound basis for these arguments. In this paper we are chiefly concerned with the second of the three steps. Two ‘knowledge elicitation’ studies were performed. In the first, eleven radiologists were asked to think out loud as they interpreted twenty cases of calcifications. The audiotapes of these sessions were analysed and 159 different terms for the description of calcifications were identified. Working with radiologists we removed synonyms and compound terms, reducing the set of descriptors to 50. A subset of 19 of these descriptors were useful in discriminating between benign and malignant calcifications. These discriminating descriptors were classsified as high or low certainty arguments. High certainty arguments were only ever used to describe calcifications with a clear diagnosis - either benign or malignant. Low certainty arguments are never used for both benign and malignant calcifications, but were sometimes used for calcifications considered indeterminate on the basis of radiological appearance. Both sets are presented in Table 1. A more detailed account of this investigation is presented elsewhere[8]. The second knowledge elicitation study had two aims: first to validate the descriptor set derived from the above study and second to obtain more data about the certainty associated with the arguments. Ten radiologists reported on 40 sets of calcifications. They used a form based on the descriptor set. They were allowed to suggest new descriptors where the existing set was felt to be inadequate, but only minor adjustments to the descriptor set were required. We
Incorporating Image Processing in a Clinical Decision Support System High Certainty Arguments Benign vascular distribution curvilinear shape large size contour with a rim lucent density centre isolated Malignant branching shape ill-defined contour orientation towards nipple
137
Lower Certainty Arguments Benign well defined contour homogeneous variation small AND scattered AND round associated opacity AND few (1-5) flecks low density AND small AND multiple AND scattered Malignant pleomorphism small AND low density AND assoc. opacity
Table 1. Arguments derived from the first study
then constructed a table with a row for each of 45 descriptors and a column for each of the 40 sets of calcifications. Each cell of the table was used to record the number of radiologists who used that descriptor for that set of calcifications. This number varied between 0 and 10. The number was assumed to correspond to the applicability of the descriptor in the case of the set. This data forms the basis for the mapping between the descriptor set and the image processing operations.
4
Image Processing
The decision support provided in CADMIUM II is based on arguments about how the characteristics of calcifications relate to the risk of malignancy. In order to provide appropriate advice, the system must determine which of the different arguments in the knowledge base apply in a given case. In CADMIUM II image processing is used to detect and characterise the calcifications in a mammogram. In the following two sub-sections the selection and implementation of operators for the detection and characterisation of calcifications are described. Detection Algorithms for the detection of calcifications have been published since the 1970s. The best of these achieve high level of sensitivity at what are thought to be acceptable levels of specificity. Although there are a number of such algorithms in the literature data comparing the performance of the different approaches is relatively scarce. We selected four quite different approaches to the detection of calcifications and reimplemented them according to a common scheme. The details of this work and of the subsequent comparison are published elsewhere[9]. Karssemeijer’s Markov random field approach was identified as the most appropriate for our purposes[10]. Characterisation We want to use a set of image processing measures to characterise the detected calcifications. By this we mean that we wish to identify the extent to
138
Paul Taylor et al.
which they exhibit the various properties that were identified in building the knowledge base. Starting with the properties used in the high and low certainty arguments of Table 1, we identified a set of underlieing dimensions we would have to measure. For a significant subset of these there are readily available image processing techniques that have been previously applied in imaging systems[11]. The remaining properties do not correspond very obviously to measurements that have been used by previous authors working in the application of image analysis to mammography. We have designed a set of image processing measures that allow us to characterise mammograms according to these properties. Both sets are listed in Table 2. Existing measures Properties of Calcs round vs linear small vs large lucent centre vs non high vs low density Contour well-defined vs ill-defined Number of flecks many vs few Distribution scattered vs clustered Variation between flecks variable vs uniform shape variable vs uniform size New measures Shape branching vs not branching curvilinear vs straight Contour with rim vs without Associated finding opacity vs no opacity Distribution ductal vs segmental Orientation towards nipple vs not Shape Size Density
Measures compactness no. of pixels contrast across diameter mean gray level mean border contrast no. of calcifications mean separation S.D. of shape measures S.D. of size nodes of skeleton local curvature of skeleton contrast over boundaries opacity detection cluster moment angle and tendency
Table 2. Measures for the characterisation of calcifications
Linking Image Processing and Symbolic Reasoning The final stage in the development of CADMIUM II is to map between the measures described in the above section and the symbolic descriptors used in the arguments. This involves two steps. First it is necessary to determine that the expected association between the measure and the property referred to in the knowledge base actually exists. Secondly it is necessary to establish which ranges of measurement values correspond to the appropriate qualifiers for the property. To give a concrete example, one of the properties used in the knowledge base is that of size. This is measured above as ‘area’, simply the number of pixels that call within the boundary of an identified calcification.
Incorporating Image Processing in a Clinical Decision Support System
139
To complete the required mapping between the knowledge base property ’size’ and the image processing measurement ’area’ we must first ensure that assessments of size correlate with measurements of area. Then we must identify the ranges of our measurement of area that correspond to the qualifiers used in the assessment of ’size’: namely large, medium and small. The basis for both of these steps is the data collected in the second of the knowledge elicitation studies described above. The radiologists’ assessments of a set of calcifications can be compared with the image processing measurements to test the validity of the measurement. For those measures that are deemed to be valid, we can plot the image processing measurement against the frequency with which radiologists applied each of the linguistic qualifiers for that property. We can then set thresholds to optimise the separation between the distributions of the qualifiers.
Conclusion The development of highly sensitive algorithms for the detection of calcifications has focussed attention on the need to make these algorithms more specific. Many teams have considered the use of neural nets, or other classification schemes, to discriminate between benign and malignant calcifications. The general approach is to detect calcifications, to derive a set of features from the detected calcifications and then use these, in combination with some kind of gold standard, to ’learn’ a classification rule[11]. Many of the approaches, including all of the neural network approaches, involve the derivation of a rule that remains implicit, that is to say which cannot be explained to the user. We believe that users will perform better if the rule or rules used in the classification of calcifications can be made explicit. Such a system would not only be of value as a decision support tool but could also have a role in supporting the training of non-radiologist film-readers. One of the important areas of further work the development of techniques to handle uncertainty. There are three distinct forms of uncertain information that are involved. The first is the uncertainty associated with the image data. The images contain noise in the form both of scattered radiation and background texture. The detection of calcifications in the image is inevitably associated with a degree of error. The second form of uncertainty concerns the mapping between the quantitative measurements of image properties and the more qualitative terms used in the knowledge base. The final form of uncertainty concerns the strength of the arguments that relate the characteristics of calcifications to provisional diagnoses and management options. We expect to use different approaches to handle the different forms of uncertain information. The lowest level, that of image data, we believe that a Bayesian approach is most appropriate. As regards the higher level, our belief is that users are more interested in a clear statement of rough levels of certainty than in precise estimates of probability and that the argumentation approach adopted here will be appropriate.
140
Paul Taylor et al.
Acknowledgements This work was supported by the United Kingdom’s Engineering and Physical Sciences Research Council and the Imperial Cancer Research Fund. The help of Drs Given-Wilson, Davies, Schneider, Cooke, Rankin, Nockler and their colleagues is gratefully acknowledged.
References 1. Roehrig, J., Doi, T., Hasegawa, A. et al. Clinical results with the R2 Imagechecker system. In: Karssemeijer, N., Thijssen, M., Hendriks, J., and van Erning,L. (eds.) Digital Mammography, Nijmegen. Kluwer Academic Publishers, Dordrecht (1998) 395–400 2. Astley, S., Zwiggelaar, R., Wostenholme, C. et al. Prompting in mammography: how good must prompt generators be? In: Karssemeijer, N., Thijssen, M., Hendriks, J., and van Erning,L. (eds.) Digital Mammography, Nijmegen. Kluwer Academic Publishers, Dordrecht (1998) 347–354–400 3. Hartswood, M., Procter, R., and Williams, L. Prompting in practice: How can we ensure radiologists make best use of computer-aided detection systems in screening mammography? In: Karssemeijer, N., Thijssen, M., Hendriks, J., and van Erning,L. (eds.) Digital Mammography, Nijmegen. Kluwer Academic Publishers, Dordrecht (1998) 363–370 4. Field, S. UK Radiologist workforce survey - Breast Imaging Service. Royal College of Radiologists Newsletter 54 (1998) 12–14 5. Cowley, H. and Gale, A. PERFORMS and mammographic film reading performance: radiographers, breast physicians and radiologists Tech. Rep., Institute of Behavioural Sciences, University of Derby. A report for the Workforce Issues in the Breast Screening Programme meeting (1999). 6. Taylor, P., Fox, J. and Todd-Pokropek, A. The development and evaluation of CADMIUM: a prototype system to assist in the interpretation of mammograms. Medical Image Analysis 3 (1999) 321–337 7. Fox J., Johns, N. and Rahmanzadeh, A. Dissemination of medical knowlege: the PROforma approach. Artificial Intelligence in Medicine 14 (1998) 157–181 8. Alberdi, E., Taylor, P., Lee, R. et al. CADMIUM II: Acquisition and representation of radiological knowledge for computerized decision support in mammography. In: Overhage, J. (ed.): Proceedings of the American Medical Informatics Association Symposium, American Medical Informatics Association, (2000) in press 9. Lee, R., Taylor, P. and Alberdi, E. A comparative study of four techniques for calcification detection. In: Yaffe M. (ed.) Proceedings of the Fifth International Workshop on Digital Mammography, Medical Physics Publishing (2000) in press 10. Karssemeijer, N. Adaptive noise equalisation and recognition of microcalcification clusters in mammograms. International Journal of Pattern Recognition and Artificial Intelligence 7 (1993) 1357–1376 11. Giger, M., Huo, Z., Kupinski, M. and Vyborny, C. Computer-aided diagnosis in mammography. In: Sonka M. and Fitzpatrick, J. (eds.) SPIE Handbook of Medical Imaging: Volume 2, International Society for Optical Engineering, (2000) 915–1004
Automated Estimation of Brain Volume in Multiple Sclerosis with BICCR D. Louis Collins, Johan Montagnat, Alex P. Zijdenbos, Alan C. Evans, and Douglas L. Arnold Montreal Neurological Institute McGill University, Montreal, Canada {louis,jmontagn,alex,alan,doug}@bic.mni.mcgill.ca http://www.bic.mni.mcgill.ca
Abstract. Neurodegenerative diseases are often associated with loss of brain tissue volume. Our objective was to develop and evaluate a fully automated method to estimate cerebral volume from magnetic resonance images (MRI) of patients with multiple sclerosis (MS). In this study, MRI data from 17 normal subjects and 68 untreated MS patients was used to test the method. Each MRI volume was corrected for image intensity non-uniformity, intensity normalized, brain masked and tissue classified. The classification results were used to compute a normalized metric of cerebral volume based on the Brain to IntraCranial Capacity Ratio (BICCR). This paper shows that the computation of BICCR using automated techniques provides a highly reproducible measurement of relative brain tissue volume that eliminates the need for precise repositioning. Initial results indicate that the measure is both robust and precise enough to monitor MS patients over time to estimate brain atrophy. In addition, brain atrophy may yield a more sensitive endpoint for treatment trials in MS and possibly for other neuro-degenerative diseases such as Huntington’s or Alzheimer’s disease.
1
Introduction and Previous Work
A number of neuro-degenerative diseases are characterized by brain tissue loss. For example, multiple sclerosis (MS) is a neurological disorder that predominately affects young adults and is associated with recurrent attacks of focal inflammatory demyelination (plaques) that cause neurological impairment, separated by periods of relative stability. It is difficult to evaluate the effect of therapy in clinical trials of MS since it is a complex disease with a high degree of variability in clinical signs and symptoms that vary over time and between individuals. The clinically accepted gold standard measure for burden of disease in MS is the Kurztke Expanded Disability Status Scale (EDSS) [1]. Unfortunately, this metric is highly variable between neurologists (large inter-rater variability), is dependent on the timing of the test with respect to the latest exacerbation of the disease and has a variable sensitivity to change depending on the degree M.F. Insana, R.M. Leahy (Eds.): IPMI 2001, LNCS 2082, pp. 141–147, 2001. c Springer-Verlag Berlin Heidelberg 2001
142
D. Louis Collins et al.
of clinical disability. Taken together, these factors make it difficult to precisely and accurately quantify the overall burden of disease. Therefore, large numbers of subjects (hundreds) are required to participate in clinical trials for new drug evaluation in order to have enough statistical power to detect oftentimes subtle differences between treatment arms. Our goal has been to develop an objective, automatic, robust image-based method to quantify disease burden in MS. Our interest has turned to central nervous system (CNS) atrophy since histopathological work has confirmed that substantial axonal loss occurs in MS plaques [2] and recent quantitative work has confirmed that CNS atrophy is greater in MS patients than in age-matched normals [3,4,5]. We propose a fully automated, head-size normalized, brain-volume estimation procedure. The BICCR metric is defined as ratio of brain tissue volume to the total volume enclosed within the intra-cranial cavity. The volumes are derived from the result of a tissue classification process. This metric is similar to the brain parenchymal fraction (BPF) of Fisher [6] where BPF is defined as the ratio of brain tissue volume to total volume enclosed by the brain surface. The main difference is that all extra-cerebral CSF (i.e., CSF between the cortex and dura, in addition to that in the sulci) is included in the BICCR measure. We will show that the BICCR measure is better correlated with disability, and thus may be a better surrogate for disease burden.
2 2.1
Methods Data
Controls: Seventeen normal healthy controls (age range of 25-61 years) were recruited from the staff, students and research fellows of the Montreal Neurological Institute and McGill community. Patients: Seventy patients with MS were selected from the population followed in the Montreal Neurological Hospital MS clinic. Forty-eight patients were classified as relapsing-remitting (RR), characterized by recurrent relapses with complete or partial remission (disease duration 0.5 to 24 years, EDSS range 0-5.0, age range 26-58). Twenty-two patients were classified as secondary progressive (SP), characterized by progression in the absence of discrete relapses after earlier RR disease (disease duration 4 to 36 years, EDSS range 3.5-9.0, age range 27-59 years). MRI acquisistion: All MR data was acquired on a Philips Gyroscan operating at 1.5 T (Philips Medical Systems, Best, The Netherlands) using a standard head coil and a transverse dual-echo, turbo spin-echo sequence, 256x256 matrix, 1 signal average, 250mm field of view, (TR/TE1/TE2 = 2075/32/90 ms) yielding proton density-weighted (PDW) and T2-weighted (T2W) images. Fifty contiguous 3mm slices were acquired approximately parallel to the line connecting the anterior and posterior commissures (AC-PC line).
Automated Estimation of Brain Volume in Multiple Sclerosis with BICCR Non−uniform Input intensity correction image
Filtered image
Bayesian classification
Corrected image
Tissue classes
Stereotaxic registration
Registered image
143
Normalization Normalized image
Atrophy computation Masking and volumes computation
Anisotropic diffusion
Fig. 1. Diagram of the atrophy computation method stages.
2.2
Data Analysis
Atrophy Estimation The fully automated method uses MR images to quantify brain atrophy and is based on estimation of the brain to intracranial capacity ratio (BICCR). The method estimates the intracranial, brain parenchymal and CSF volumes and uses these values in a ratio described below. The technique is voxel-based. Each image voxel is classified as a brain tissue, CSF or background. The number of voxels in each class multiplied by the elementary voxel volume gives an estimate of actual tissue and CSF volumes. As a voxel-based approach, this method requires preliminary processing stages that aim at correcting the image intensities by minimizing the bias and the noise due to the acquisition device. Images are also registered in a common brain-based coordinate space (Talairach) by a linear registration procedure. This ensures that the scale differences between individuals are compensated for and that the resulting atrophy measure is invariant to brain size. Figure 1 diagrams the atrophy measure stages. The processing stages involve: Intensity non-uniformity correction. The inhomogeneity of the MR acquisition device magnetic field introduces a bias perceptible in images as a continuous variation of gray-level intensities. The non-uniform intensity correction algorithm [7] iteratively proceeds by computing the image histogram and estimating a smooth intensity mapping function that tends to sharpen peaks in the histogram. The intensities for each tissue type thus have a tighter distribution and are relatively flat over the image volume. Application of this procedure improves the accuracy of the tissue classification stage described below [7]. Stereotaxic registration. Each image is linearly registered in a common Talairach space in order to compensate for size variations between individuals. Moreover, the Talairach-like brain-based coordinate system of stereotaxic space facilitates anatomically driven data manipulation in all processing steps. The target image for stereotaxic registration is a template image built from an earlier study [8] involving the averaging of more than 300 MR images. The registration algorithm proceeds with a coarse-to-fine approach by registering subsampled and blurred MRI volumes with the stereotaxic target [9]. The final data used for subsequent processing is only resampled once to minimize resampling/interpolation artefacts.
144
D. Louis Collins et al.
Intensity normalization. In preparation for intensity-based classification, each image is intensity normalized to an average PDW (PD-weighted) or T2W (T2weighted) target volume already in stereotaxic space. An affine intensity mapping is estimated that best maps the histogram of each image onto the template. After normalization, the histogram peaks corresponding to each tissue class have the same value in all images. In conjuction with intensity non-uniformity correction, this step permits data from all subjects to be classified using a single trained classifier (i.e., the classifier does not have to be retrained for each subject). Cropping. Since the entire cerebrum was not covered by the MRI acquisition in all subjects, the inferior (z < −22mm, in Talairach coordinates) and superior (z > 58mm) slices were cropped away from both PDW and T2W volumes, cutting off the very top of the brain (above the centrum semi-ovale) and the bottom of the brain (just above the pons). This yielded an anatomically equivalent 80mm thick volume across all subjects that contains most of the cerebrum. Anisotropic diffusion It has been shown that the application of an edgepreserving noise filter can improve the accuracy and reliability of quantitative measurements obtained from MRI [10,11]. We have selected anisotropic diffusion, a filter commonly used for the reduction of noise in MRI. This type of filter was pioneered by Perona and Malik [12] and generalized for multidimensional and multispectral MRI processing by Gerig et al. [13]. This stage reduces voxel misclassification due to noise and minimizes the speckled appearance sometimes apparent in the resulting classified images. Bayesian classification. A Bayesian classifier [14] is then used to identify all grey-matter (GM), white-matter (WM), cerebrospinal fluid (CSF), lesion (L) and background (BKG) voxels. Prior to classification, the Bayes classifier is trained manually by selecting a set of 20 volumes randomly among all volumes to be processed. From each sample volume, 50 voxels belonging to each class are selected by hand. The resulting 5000 samples (20 volumes × 50 samples × 5 classes) were used to compute each class mean intensity and the covariance matrices used in the Bayesian classifier. Brain masking. Mathematical morphology [15] was used to eliminate the scalp and meninges from further processing. A brain mask was created by applying an opening operator (i.e., erosion followed by dilation) to the PDW volume after thresholding at 40% of the mean PDW intensity value. Voxels remaining in the regions of the eyes and nasal sinus were removed using a standard mask in stereotaxic space. The resulting patient-specific brain mask was applied to both the PDW and T2W volumes leaving all voxels within the intracranial cavity. BICCR computation. After processing, the total volume of voxels in each class was used to define the BICCR metric: BICCR =
GM + WM + L . GM + WM + L + CSF
(1)
It is is important to note that the value of CSF contains all extra-cerebral cerebrospinal fluid within the cropped volume in addition to the ventricular and sulcal components. Similar to the brain parenchymal fraction (BPF) of Fisher
Automated Estimation of Brain Volume in Multiple Sclerosis with BICCR
(a)
(b)
145
(c)
Fig. 2. Results: (a) box & whisker plot for comparison of BICCR mean values (heavy circles) for NC, RR and SP groups; correlation of BICCR with age (b) and disease duration (c) (RR=black circles, SP=grey squares). and Rudick [6,4], the BICCR metric is a ratio and not only represents a sizenormalized index of brain atrophy but it also accounts for possible differences in voxel size between scans due to scanner drift. To determine the reproducibility of the method, 4 healthy volunteers were scanned on 2 separate occasions over a mean period of 222 days. BICCR was computed for each image set. Reproducibility was estimated by computing the coefficient of variation of the repeated measures.
3
Results
The BICCR value for the normal control (NC, n=17) subjects was 86.1 ± 2.8 (mean ± s.d.). The mean coefficient of variation estimated on scan-rescan tests of 4 normal controls was 0.21%. Comparison of the mean BICCR values for the NC, RR and SP groups is presented in Figure 2-a. An ANOVA showed a significant difference between groups (F = 8.885, p < 0.001). A post-hoc test (Tukey’s HSD) showed that BICCR was significantly lower in the secondary progressive group (81.3 ± 5.1) than either the NC group (p < 0.001) or the relapsing-remitting group (84.5 ± 4.3; p = 0.01). The Z-score (number of standard deviations from the mean of healthy controls) was -0.673 for RR (not significantly different from NC) and 1.864 (p < 0.001) for SP groups. The average absolute percentage of brain tissue lost (compared to normal controls) was 1.8% for RR and 5.6% for SP groups. We looked at the relationship between BICCR with respect to age, disease duration and EDSS. ANOVA showed no significant differences in age between the NC, RR and SP groups (F = 1.134, p = 0.327). As expected, the mean duration of disease of the SP group was significantly greater than that for the RR group (Student’s t = 3.88, p < 0.001). Also expected, disability (measured by EDSS) was greater for the SP group when compared to that of the RR group (t = 11.43, p < 0.001).
146
D. Louis Collins et al.
For the RR group, BICCR was correlated with disease duration (Spearman r = −0.523, p < 0.001), but not with age, nor disability as measured by EDSS (see Figs 2-b and -c). For the SP group, BICCR was correlated with disease duration (Spearman r = −0.661, p < 0.001) and EDSS (Spearman r = −0.649, p < 0.001) but not with age. When evaluated over all patients with MS (RR and SP combined), BICCR was correlated with EDSS (r = −0.409, p < 0.01) and duration (r = −0.593, p < 0.0001). The main difference between the BICCR and BPF metrics is the inclusion of extra-cerebral CSF in the denominator. In a simple test to compare the correlation of disability (measured by EDSS) with BICCR and a measure similar to BPF, we used morphological operators to remove the extra-cerebral CSF voxels from the BICCR metric. When evaluated on 20 SP MS patients, the magnitude of the Spearman’s correlation coefficient dropped from -0.638 (BICCR) to -0.574 (modified BICCR).
4
Discussion
We have presented a robust procedure to estimate brain atrophy using a fully automatic technique and have applied it to MRI data from normal controls and patients with MS. We have confirmed that the brains of patients with MS have greater atrophy when compared to normal controls, and that atrophy progresses with the severity and duration of the disease. Our procedure compares well to the BPF measure of Fisher [6]. The mean BPF and BICCR values are similar for normal controls. However, the BPF method is reported to have a very small intersubject variance when estimated on normal controls (approximately 0.7%). This value is much smaller than the variance for normal controls reported here. This may be due to subject selection and the greater age range for our normal controls. Another difference between the two techiques is that the classification procedure used in the BPF computation accounts for partial volumes effects between tissue classes, while the BICCR method uses a discrete classification result. While this method should yield an unbiased result for objects that are larger than the voxel size, the BICCR method may underestimate CSF volume in regions that have dimensions on the order of the voxel size, in sulci for example. The high precision of the BICCR method permits detection of small changes ( 0.5%) in brain volume (i.e., atrophy) in single subjects over a short period of time (< 1 year). Comparison of BICCR with a BPF-like measure shows that BICCR correlates better with disability, making it possibly a more sensitive surrogate for disease burden. These results have important implications for the design of clinical trials if atrophy is deemed an acceptable surrogate for burden of disease in MS. The fact that cerebral atrophy is generally correlated with irreversible neurological dysfunction make atrophy an important surrogate to evaluate in MS using state of the art image analysis techniques. Characterization of brain atrophy will yield information complementary to other MR-based measures of focal and dif-
Automated Estimation of Brain Volume in Multiple Sclerosis with BICCR
147
fuse abnormality with varying specificity for underlying pathological changes. Brain atrophy may yield a more sensitive endpoint for treatment trials in MS and possibly also for other neurdegenerative diseases such as Huntington’s or Alzheimer’s disease. Acknowledgements: Funding for this work was provided by the Medical Research Council of Canada.
References 1. J. F. Kurtzke, “Rating neurologic impairment in multiple sclerosis: An expanded disability status scale,” Neurology, vol. 33, pp. 1444–1452, 1983. 2. B. D. Trapp, J. Peterson, R. M. Ransohoff, R. Rudick, S. Mork, and B. Lars, “Axonal transection in the lesions of multiple sclerosis,” New England Journal of Medicine, vol. 338, pp. 278–85, 1998. 3. J. Simon, L. Jacobs, M. Campion, et al. “A longitudinal study of brain atrophy in relapsing multiple sclerosis. the multiple sclerosis collaborative research group (MSCRG).,” Neurology, vol. 53, no. 1, pp. 139–48, 1999. 4. R. Rudick, E. Fisher, J.-C. Lee, J. Simon, D. Miller, and L. Jacobs, “The effect of avonex (ifnβ-1a) on cerebral atrophy in relapsing multiple sclerosis,” Neurology, vol. 52, pp. A289–290, Apr 1999. 5. M. Filippi, G. Mastronardo, M. A. Rocca, C. Pereira, and G. Comi, “Quantitative volumetric analysis of brain magnetic resonance imaging from patients with multiple sclerosis,” J Neurol Sci, vol. 158, pp. 148–53, Jun 30 1998. 6. E. Fisher, R. Rudick, J. Tkach, J.-C. Lee, T. Masaryk, J. Simon, J. Cornhill, and J. Cohen, “Automated calculation of whole brain atrophy from magenetic resonance images for monitoring multiple sclerosis,” Neurology, vol. 52:A352, 1999. 7. J. G. Sled, A. P. Zijdenbos, and A. C. Evans, “A non-parametric method for automatic correction of intensity non-uniformity in MRI data,” IEEE Transactions on Medical Imaging, vol. 17, Feb. 1998. 8. A. C. Evans, D. L. Collins, and B. Milner, “An MRI-based stereotactic atlas from 250 young normal subjects,” Soc.Neurosci.Abstr., vol. 18, p. 408, 1992. 9. D. L. Collins, P. Neelin, T. M. Peters, and A. C. Evans, “Automatic 3D intersubject registration of MR volumetric data in standardized talairach space,” Journal of Computer Assisted Tomography, vol. 18, pp. 192–205, March/April 1994. 10. J. R. Mitchell, S. J. Karlik, D. H. Lee, M. Eliasziw, G. P. Rice, and A. Fenster, “Quantification of multiple sclerosis lesion volumes in 1.5 and 0.5T anisotropically filtered and unfiltered MR exams,” Medical Physics, vol. 23:115–126; 1996. 11. A. P. Zijdenbos, B. M. Dawant, R. A. Margolin, and A. C. Palmer, “Morphometric analysis of white matter lesions in MR images: Method and validation,” IEEE Transactions on Medical Imaging, vol. 13, pp. 716–724, Dec. 1994. 12. P. Perona and J. Malik, “Scale-space and edge detection using anisotropic diffusion,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 12, pp. 629–639, July 1990. 13. G. Gerig, O. K¨ ubler, R. Kikinis, and F. A. Jolesz, “Nonlinear anisotropic filtering of MRI data,” IEEE Transactions on Medical Imaging, vol. 11:221–232; 1992. 14. R. Duda and P. Hart, Pattern Recognition Scene Analysis. New York: Wiley, 1973. 15. J. Serra, Image Analysis, Mathematical Morphology. London: Academic Press, 1982.
Automatic Image Registration for MR and Ultrasound Cardiac Images Caterina M. Gallippi and Gregg E. Trahey Department of Biomedical Engineering, Duke University, Durham, NC 27708
[email protected]
Abstract. The Statistics Based Image Registration (SBR) method for automatic image registration is presented with application to magnetic resonance (MR)and ultrasound (US) cardiac time series images. SBR is demonstrated for MR myocardial perfusion assessment and US myocardial kinetics studies. The utility of the method for a range of other clinical applications is discussed.
1
Introduction
Accurate multi- and mono- modal image registration could enhance the diagnostic relevance of medical imaging by aligning information in a fashion conducive to disease assessment, motility estimation, or volume measurement. Fully automatic registration methods can expedite information alignment and obviate human error introduced by user selection of corresponding image information. Many techniques for image registration have been described. Extrinsic methods are sometimes inconvenient and can not be applied retrospectively. Landmark and segmentation based methods may require labor intensive human interaction. Pixel or voxel property based registration methods require no a priori information and can be applied to many modes of medical imaging with no data reduction or image segmentation [1]. Several paradigms are reported in literature for pixel property based image registration (Geimanet al,[2]; Hein et al,[3]; Maes et al, [4]; Mailloux et al,[5]; Meyer et al, [6]; Penney et al, [7]; Wells et al,[8]). One pixel property registration method, maximization of mutual information (MI), is based on the principle of identifying image regions that have high individual brightness entropy but low joint entropy. Wu et al outline another approach to image matching that uses a modified correlation measure instead of probability distributions to match pixels in high contrast regions with similar brightness statistics [9]. Developing a single registration algorithm that is substantially robust for aligning both MR to MR and US to US images is a challenging task given the inherent differences in the two imaging modalities. Contrast sensitivity is a major attribute of MR but is degraded by the presence of granular structure called speckle that results from the coherent interactions of echo reflections in US [10]). One of the authors (Gallippi) adapted Wu et al’s method and applied the technique to automatic image registration and warping of MR myocardial perfusion M.F. Insana, R.M. Leahy (Eds.): IPMI 2001, LNCS 2082, pp. 148–15