Developing Quality Dissertations in the Sciences: A Graduate Student's Guide to Achieving Excellence

17211-DevelQualDiss-Sciences

11/19/08

10:54 AM

Page b

COPYRIGHT © 2009 BY STYLUS PUBLISHING, LLC Published by Stylus Publishing, LLC 22883 Quicksilver Drive Sterling, Virginia 20166-2102 All rights reserved. No part of this book may be reprinted or reproduced in any form or by any electronic, mechanical or other means, now known or hereafter invented, including photocopying, recording and information storage and retrieval, without permission in writing from the publisher. Library of Congress Cataloging-in-Publication-Data Lovitts, Barbara E., 1960Developing quality dissertations in the sciences : a graduate student’s guide to achieving excellence / Barbara E. Lovitts and Ellen L. Wert.— 1st ed. p. cm. Includes bibliographical references and index. ISBN 978-1-57922-259-8 (pbk.) 1. Dissertations, Academic—Handbooks, manuals, etc. 2. Science—Study and teaching (Graduate)—Handbooks, manuals, etc. I. Wert, Ellen L., 1953– II. Title. LB2369.L683 2009 808'.0665—dc22 2008031315 13-digit ISBN: 978-1-57922-259-8 (paper) Printed in the United States of America All first editions printed on acid free paper that meets the American National Standards Institute Z39-48 Standard. First Edition, 2009

10 9 8 7 6 5 4 3 2 1

Acknowledgments Credit for the idea of translating the book version of this work into a handbook for graduate students and faculty goes to Louis Sherman, Purdue University, one of the members of my advisory committee. Ted Greenwood, of the Alfred P. Sloan Foundation, provided moral and financial support for the project and wholeheartedly endorsed creating not one generic handbook but three, one each for the sciences, social sciences, and humanities. My publisher and editor, John von Knorring, Stylus Publishing, ably orchestrated all the people and pieces in what turned out to be a longer than anticipated process. Many, many thanks go to Ellen Wert, my co-author, and the consultants who worked with her, Chris Golde, of Stanford University and a member of my advisory committee; Mary Huba, Iowa State University; and Dannelle Stevens, Portland State University. Last but not least is Karen Klomparens, of Michigan State University and an advisory committee member, who contributed intellectually to this project and provided an institutional home for the funds that supported it.

17211-DevelQualDiss-Sciences

12/8/08

2:20 PM

Page i

Developing Quality Dissertations in the Sciences A Graduate Student’s Guide to Achieving Excellence

Barbara E. Lovitts and Ellen L. Wert

S T E R L I N G , V I RG I N I A

17211-DevelQualDiss-Sciences

12/8/08

2:20 PM

Page ii

Contents

List of Tables

iii

Preface To the Faculty

iv

To the Student

vii

1 2 3 4 5

Identifying the Purpose of the Dissertation

1

Understanding Originality and Significance

4

Aiming for Excellence in the Dissertation

8

Maintaining Consistent Quality Within the Dissertation

14

Achieving Excellence

22

Appendix A: Tasks of the Sciences, Engineering, and Mathematics Dissertation

25

Appendix B: Advice for Writing a Science, Engineering, or Mathematics Dissertation

35

Making the Implicit Explicit: About the Study

39

About the Authors

40

ii

17211-DevelQualDiss-Sciences

12/8/08

2:20 PM

Page iii

Tables

Table 1.1. Table 1.2. Table 2.1. Table 2.2. Table 3.1. Table 3.2. Table 3.3. Table 3.4. Table 4.1. Table 4.2. Table 4.3. Table 4.4. Table 4.5. Table 4.6. Table 4.7. Table A.1. Table A.2. Table A.3. Table A.4. Table B.

The Purpose of the Dissertation: Descriptions From Faculty in the Sciences, Engineering, and Mathematics The Purpose of the Dissertation: Descriptions From Faculty in Biology, Physics, Electrical and Computer Engineering, and Mathematics Original Contribution: Characterizations From Faculty in Biology, Physics, Electrical and Computer Engineering, and Mathematics Significant Contribution: Characterizations From Faculty in Biology, Physics, and Electrical and Computer Engineering, and Mathematics The Biology Dissertation at Different Quality Levels The Physics Dissertation at Different Quality Levels The Electrical and Computer Engineering Dissertation at Different Quality Levels The Mathematics Dissertation at Different Quality Levels The Tasks of a Dissertation in the Sciences, Engineering, and Mathematics Introduction/Problem Statement in the Sciences, Engineering, and Mathematics Dissertation at Different Quality Levels Literature Review in the Sciences, Engineering, and Mathematics Dissertation at Different Quality Levels Theory in the Sciences, Engineering, and Mathematics Dissertation at Different Quality Levels Method or Approach in the Sciences, Engineering, and Mathematics Dissertation at Different Quality Levels Results or Data Analysis in the Sciences, Engineering, and Mathematics Dissertation at Different Quality Levels Discussion or Conclusion in the Sciences, Engineering, and Mathematics Dissertation at Different Quality Levels Tasks of the Biology Dissertation by Quality Level Tasks of the Physics Dissertation by Quality Level Tasks of the Electrical and Computer Engineering Dissertation by Quality Level Tasks of the Mathematics Dissertation by Quality Level Writing the Science, Engineering, or Mathematics Dissertation

iii

1 3 6 7 11 11 12 13 15 16 17 18 19 20 21 26 29 31 33 35

17211-DevelQualDiss-Sciences

12/8/08

2:20 PM

Page iv

To the Faculty

on your own dissertation, you probably now see it as a major exercise that demonstrated your expertise at the moment when you were ready to make the transition from being a student to being a professional. Most of your faculty colleagues would say the same thing, as I discovered through a study of graduate faculty across 10 disciplines. Again and again, the faculty described the dissertation as both an opportunity to develop and sharpen skills and knowledge and evidence of a student’s readiness to work independently at a professional level.1 This understanding of the dissertation, it seems, comes with some distance from the process. However, while students are anticipating or engaged in the process, the dissertation is a bit of a mystery and a source of great anxiety. Many students spend a great deal of time (perhaps too much time) wrestling with process questions such as how to identify a topic and how to organize time and material. They try to make sense of a vague demand that the dissertation be “original” and “significant.” Important questions about expectations of quality—What does it mean to make an original or significant contribution? What constitutes an outstanding or very good dissertation?—often go unasked because it is assumed, by both faculty and students alike, that students should simply know. Of course, they do not know. That they do not is evident from the varying levels of quality, both from student to student in a department and within individual dissertations. Some faculty spend a great deal of time coaching and guiding students through the process and do so for a longer period of time than should be necessary. Further, the dissertation is the point of attrition for nearly half of aspiring PhDs who make it this far. Yet it is the rare department that ex-

plicitly and publicly states its expectations about this capstone project.

L

OOKING BACK

Shared Expectations When I asked experienced dissertation advisors to describe their expectations for originality, significance, and quality in their students’ dissertations, the responses I heard were clear and consistent both within and across the disciplines and fields. The faculty offered similar views on the purpose of the dissertation and what it means to make an original and a significant contribution. They also expressed similar views on what constitutes outstanding, very good, acceptable, or unacceptable work. It seems, then, that students spend a great deal of valuable time and energy trying to guess something that the faculty implicitly agree upon but have not articulated in any formal way, either to the students or each other.

Why Explain the Dissertation? Although the dissertation represents the transition from student to professional and should display the capacity for self-directed work, students need and deserve the benefit of guidance about the process and clear expectations about quality. The department or program has a responsibility to communicate clearly both its standards of quality and those of the discipline. The faculty advising dissertations have a particular responsibility to explain the process, work with students to set firm but realistic deadlines, and provide timely, constructive feedback at every step.

iv

17211-DevelQualDiss-Sciences

12/8/08

2:20 PM

Page v

P R E FA C E There are ethical, professional, and practical reasons for making expectations about the dissertation explicit, the process of review transparent, and the production efficient: It is only right and fair to give students guidance about standards and expectations and then allow them to make choices about how to use it. Members of the profession who prepare doctoral students should communicate the norms and skills of the field to those who seek to enter it.2 The process of dissertation advising will move more quickly and efficiently if students know what is expected of them. Some faculty members may argue that this call for greater clarity and transparency is a plea for handholding or coddling. Actually, I am suggesting something quite the opposite. If advisors provide guidelines about standards and expectations from the start, students’ responsibilities for meeting them become immediately apparent. Students will also have benchmarks against which they can judge and revise their work, reducing the amount of work for advisors and committee members.

Making Expectations Explicit When it became clear that the findings of my study could be put to very specific and practical use, the Alfred P. Sloan Foundation, which funded the study, offered to underwrite a set of booklets for graduate students in the sciences, engineering, and mathematics; social sciences; and humanities. The booklets summarize faculty members’ descriptions of the purpose of the dissertation, the role and nature of an original and a significant contribution, and levels of performance. The specific examples throughout this booklet, many of them drawn from mathematics, physics, biology, and electrical and computer engineering, represent the observations of 101 faculty from 28 science, engineering, and mathematics departments.

Using This Booklet: Benefits to the Faculty and Program Although, as I discuss below, my primary hope is that you and your students use this booklet together, I also hope you and your colleagues will find it valuable in your ongoing conversation about guiding your pro-

v gram’s students. Over time, a collective effort to articulate expectations will help your program achieve a consistent level of quality. In addition, with a transparent process in place and a body of evidence around student achievement, you will also be better equipped to provide information about program quality as part of internal and external accountability processes.

Using This Booklet: Benefits to the Student This booklet is intended primarily as an entry point for discussion with the students you advise. I do not pretend to represent what you, as advisor, expect or what your program requires. Instead, you will find, throughout the booklet I urge the student to talk with you and your colleagues about the particular emphases and expectations of your discipline, subdiscipline, department and program. The information in this booklet is offered as a way to start the discussion. Indeed, the earlier students are introduced to the standards and expectations of their disciplines, the better. You might consider providing this booklet to students long before they start the dissertation, for example, in a research methods course or in a journal club. Whether you bring this booklet to your students’ attention or they bring it to yours, I hope that you will use it with them to plan, set goals, and provide feedback on work in progress. For example, the sections about the purposes of the dissertation, originality, and significance would be important to discuss with students who are preparing proposals and planning their projects. The descriptions of quality might be of use as you discuss the specific goals of a student’s project. I believe that you and your students will find the process of identifying expectations for the dissertation, as a whole and in its parts, very useful. Your students will be better able to assess their own work and address deficiencies before they submit drafts to you. When you receive drafts, you can use the expectations to provide focused feedback. This puts the conversation about quality directly in the context of professional standards and expectations. Finally, your efforts to articulate quality will help students learn to judge quality in their own and others’ work. They can share what they have learned with

17211-DevelQualDiss-Sciences

12/8/08

2:20 PM

Page vi

P R E FA C E

vi colleagues and peers; ultimately they will become better professionals and, for those who become faculty members, better advisors. I do not, however, suggest that these descriptions of quality be casually given quantitative values and used to “score” dissertations for decisions about passing or failing. In fact, I strongly resist this impulse. I hope that you will use the descriptors with students to set goals and monitor progress while they are in the process of researching and writing their dissertations so that they can make improvements along the way.

Focusing Students on Responsibility More clearly defining expectations is a way to inspire students and spur them to higher levels of achievement. I suggest offering a set of clear expectations about the dissertation, articulating goals, and providing feedback and guidance because I believe doing so challenges students to meet their responsibility as aspiring scholars

and researchers, to check their progress toward professional excellence, and to work more independently. Barbara E. Lovitts

Notes 1. The study and the findings are described in detail in Barbara E. Lovitts, Making the Implicit Explicit: Creating Performance Expectations for the Dissertation (Sterling, VA: Stylus Publishing, 2007). 2. On this topic, faculty and graduate students alike might be interested in the observations of senior faculty from mathematics, neuroscience, chemistry, and the history of science, along with others from the humanities and social science disciplines, found in C. M. Golde, G. E. Walker, and associates. Envisioning the Future of Doctoral Education: Preparing Stewards of the Discipline. Carnegie Essays on the Doctorate (San Francisco: Jossey-Bass, 2006).

17211-DevelQualDiss-Sciences

12/8/08

2:20 PM

Page vii

To the Student

A Question of Quality

just beginning your graduate studies or have advanced standing in your program, it is highly likely the dissertation is much on your mind—and with good reason. This capstone project of your graduate education is not like anything else you have written before. It is unlike anything you will write afterward. The dissertation is a unique, hybrid project—both a professional-level report on research and a crucial piece of evidence of your qualification for a credential. It is a point of exit from student status and a point of entry to professional and scientific life. In short, your dissertation is a process and a product. Writing a dissertation is a process of developing expert knowledge: the understanding, skills, and thought processes of a professional researcher. In this sense, you began your dissertation as soon as you entered your graduate program. Through classes, supervised research, participation in journal clubs, teaching assistantships, and conversations with fellow students, faculty members, mentors, and advisors, you are acquiring skills and sorting information, learning about issues, making connections, and developing ideas for work to be done in your field—all of which is necessary for writing the dissertation. As a product, the dissertation is an external representation that you have achieved the expertise necessary to be a professional in your field. It demonstrates that you know how to approach and master complex ideas and information: you have mastered your discipline’s knowledge base, have acquired its professional skills and competencies, and are capable of doing independent research. And so, as a product, the dissertation serves as a “union card” or credential for admission into the scientific and professional community.

W

HETHER YOU ARE

Faculty in the science disciplines, along with mathematics and engineering, are currently giving serious thought to the nature and meaning of the dissertation. Many departments and programs are moving away from single-study dissertations, long the traditional form, and toward a dissertation compiled from a set of articles or research studies. Your dissertation might focus on a single study, report on several studies, or consist of a collection of published or publishable papers. Judgments about the quality of a dissertation, however, are independent of its form. As you talk with your advisor and dissertation committee members about the form your dissertation will take and the departmental requirements you should meet, you should be asking questions about their expectations: What does it mean to make an original or significant contribution? What criteria will be used to judge the quality of my work?

Starting the Conversation About Quality Knowing that questions about quality and expectations often go unasked, and that few programs or departments have explicit expectations, I decided to ask a representative group of faculty who have extensive experience advising dissertations to describe what they expect in a dissertation: What is the purpose of the dissertation? What constitutes an original or significant contribution? What makes a dissertation outstanding, very good, or acceptable? Under what conditions is work unacceptable? The responses of 101 faculty from 28 science, engineering, and mathematics departments are the founda-

vii

17211-DevelQualDiss-Sciences

12/8/08

2:20 PM

Page viii

P R E FA C E

viii tion of this booklet. As a way to start the conversation about quality, I offer these advisors’ general descriptions of standards and expectations for the dissertation in the sciences, engineering, and mathematics. Specific examples are drawn from physics, biology, electrical and computer engineering, and mathematics. It is important to keep in mind that these descriptions are a starting place. After reading this booklet, talk with your advisors and other faculty members to understand the expectations specific to your field, department, and project. Ask them what in your department and field is considered to be an original or significant contribution and what constitutes an outstanding or very good dissertation. Read recent dissertations from your department. Ask your advisors to recommend an outstanding or very good dissertation in your field— and study it. Look at your own work critically and ask for feedback in terms of the descriptors of quality presented here. I also suggest that you discuss this booklet and its information with your fellow students. It is an opportunity to develop colleagues and a way to combat isolation and the feeling that you are the only person wrestling with tough questions. In fact, as you will see throughout this booklet, the key to quality in the dissertation is communication. If nothing else, I hope that reading this booklet will prompt you to have many conversations with your dissertation advisor, to seek more than one mentor among the faculty, and to talk with other students to compare

experiences. I hope you will map out your dissertation with your advisor and committee and check in frequently with them. Finally, I hope you will ask questions, ask for feedback, and follow up with your program’s administrators if you are not getting the help you need.

A Question of Professional Responsibility As someone pursuing advanced training, you are learning what constitutes credible work in the field. Your increasing understanding of the standards of the field is part of what makes you a professional. It also means that you have to take responsibility for your work. Whether your faculty mentors, dissertation advisor or advisors, committee members, or department heads take the initiative to make their expectations of quality explicit or you take the initiative to bring this booklet to their attention and ask for their thoughts, once their expectations are made clear, it is your responsibility to act on them. As someone who aspires to the PhD, you are responsible for setting goals for yourself, checking your progress, asking for feedback, and taking feedback in the context of professional standards. I wish you well in your work. Barbara E. Lovitts

17211-DevelQualDiss-Sciences

12/8/08

2:20 PM

Page 1

1 Identifying the Purpose of the Dissertation

on the title page of nearly every dissertation, “Submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy,” underscore that the dissertation is part of a process. The requirements of your program and the many informal opportunities for learning are designed to move you from student to professional, from someone largely dependent on others for guidance in learning to an independent, expert learner and producer of knowledge. Consider the observations of the faculty who participated in the study on which this booklet is based. The various purposes they ascribe to the dissertation, sum-

T

marized in Table 1.1, point to the dissertation as a product that provides evidence that you have mastered professional skills and knowledge: your discipline’s theories and methods and a vast array of facts, principles, concepts, theories, and paradigms. The dissertation is also evidence that you have developed informed opinions about various issues, learned how to approach problems, and how to judge others’ work. Whether you plan a career in academia, government, business, industry, or in the nonprofit sector or business, the successful completion of your dissertation will signify your ability to conduct high-level inquiry and to create new knowledge.

HE WORDS

Table 1.1 The Purpose of the Dissertation: Descriptions From Faculty in the Sciences, Engineering, and Mathematics The purpose of the dissertation is to prepare the student to be a professional in the discipline. Through this preparation the student learns and demonstrates the ability to conduct independent, original, and significant research. The dissertation thus shows that the student is able to • • • • • • • • • • •

identify/define problems, generate questions and hypotheses, review and summarize the literature, apply appropriate methods, collect data properly, analyze and judge evidence, discuss findings, produce publishable results, engage in a sustained piece of research or argument, think and write critically and coherently, be a professional in and contribute to the discipline.

The dissertation shows mastery of the field. The dissertation prepares the student for a career. It is the capstone of the graduate education and research experience, a rite of passage from student to professional. It is a “union card” or credential for admission to the profession.

1

17211-DevelQualDiss-Sciences

2

12/8/08

2:20 PM

Page 2

D E V E L O P I N G Q U A L I T Y D I S S E R T AT I O N S I N T H E S C I E N C E S Table 1.2 provides descriptions that emerged from the discussions with the faculty about the dissertation in their discipline. As you consider the discipline-specific examples in Table 1.2, think about your particular field and program. Ask your advisor and other faculty members what they consider the purpose of the dissertation to be. What aspects of the dissertation does your field or program emphasize? What are the expectations for the quality of writing? As you discuss your dissertation with your advisor and committee members, ask about ways your particular project best lends itself to serving the purposes of the dissertation in your field. What are the goals of the

process? What, specifically, must your dissertation demonstrate? What, exactly, will give evidence that you have mastered the expected knowledge and skills? What will demonstrate your capacity to independently produce professional-level work in the future? From these conversations, you might want to draw up a summary of the purposes you and your advisors agree on. Ask your advisor, committee, and other faculty mentors to suggest recent dissertations from your department that might serve as good examples. Put yourself in your committee’s shoes and consider how the students fulfilled the purposes of the dissertation that your advisors have described.

17211-DevelQualDiss-Sciences

12/8/08

2:20 PM

Page 3

Identifying the Purpose of the Dissertation Table 1.2 The Purpose of the Dissertation: Descriptions From Faculty in Biology, Physics, Electrical and Computer Engineering, and Mathematics Biology The purpose of the dissertation is to • make an original and significant contribution; • develop the skills of science/scientific research and prepare for a career in science by learning how to identify scientific problems, approach problems (methods), critically analyze and evaluate a large body of empirical data and judge the evidence, write coherently and argue a point in writing; • demonstrate that the student can do scientific research and is an authority in the area; • demonstrate that the student has knowledge of the field, is able to generate hypotheses, has the discipline to collect data and the resourcefulness to organize and analyze it, is able to put the work in a broader context of what is happening in the field, and is able to write up the work and prepare it for publication. Physics The dissertation is • a capstone on the student’s research and training; • an opportunity for the student to provide a coherent presentation of the body of work done while he or she is a graduate student; • a “union card” to become a member of the research community. Its purpose is for the student to • • • •

gain professional training for the future; learn and demonstrate the ability to work independently; learn how the field functions; demonstrate expertise in the area, mastery of the subject matter, and the ability to think critically and put a project in the context of the field; • demonstrate the completion of formal academic requirements. Electrical and Computer Engineering While the dissertation communicates results of research to colleagues and the field, it is an instructional tool for • learning how to think about process and write at a professional level; • learning how to define a problem, summarize the current state of the art, understand the problem in a broader context, propose a solution, perform research, and obtain significant results; • mastering a particular area sufficiently to make future contributions to the field. It is also a record that • provides evidence of the scholarship the student has accomplished during graduate school; • documents the student’s research project and provides a tutorial for the next student; • provides evidence of the students’ professional competence, such as thinking and writing skills and engineering competencies. Mathematics The purpose of the dissertation is to produce a fully formed professional, independent mathematician, or mathematics researcher. The process ensures that the student has acquired the skills necessary to learn and carry on professional activity as a mathematician, such as solving problems, organizing thoughts and writing papers, coming up with good problems, doing research, and writing mathematics. The dissertation demonstrates that the student can do original, independent research of some significance in some area of mathematics. It certifies that the person is qualified to have a PhD and that he or she can do certain kinds of work and can conduct research independently.

3

17211-DevelQualDiss-Sciences

12/8/08

2:20 PM

Page 4

2 Understanding Originality and Significance

the terms original and significant in the examples in chapter 1. Maybe you have heard or been told that you must make an original and significant contribution to the field through your dissertation. Graduate students across all disciplines spend a great deal of time worrying about these expectations: What, exactly, does it means to make an original or significant contribution in my field? Is my project or its results important enough to be considered original and significant? What, specifically, about my dissertation must be original or significant? In talking with the faculty who participated in the study, it became evident that the qualities of originality and significance are elusive and difficult to define. Moreover, the terms are often shorthand for “the capacity to make an original or significant contribution.”

It is important to understand that the original contribution is not necessarily your entire dissertation but something that is part of it. The faculty in the study explained that an original contribution may result from

Y

OU PROBABLY NOTICED

• asking or identifying new questions or problems; • applying new ideas, methods, approaches, or analyses to an old question or problem; • developing or applying new theories, theorems, theoretical descriptions, or theoretical frameworks; • inventing, developing, or applying new methods, approaches, applications, computations, devices, techniques, or technologies; • creating, finding, or using new data, data sets or information; • applying old ideas, methods, approaches, or analyses to new data; • developing or applying new analyses, analytic approaches, frameworks, models, or statistical procedures; • coming up with new ideas, connections, inferences, insights, interpretations, observations, perspectives; • producing new conclusions, answers, findings, or proofs; • combining or synthesizing things (experiments, facts, knowledge, problems, technologies, theoretical constructs) from other fields or disciplines.

Originality An original contribution offers a novel or new perspective. The faculty in the life and physical sciences, engineering, and mathematics who participated in the study offered the following characterization: An original contribution is something that has not been done, found, proved, or seen before. It is publishable because it adds to knowledge, changes the way people think, moves the field forward, or advances the state of the art. To achieve this goal, you might develop an original insight or advance, or you might borrow a contribution from another discipline and apply it to your field for the first time.

Your original contribution might appear in almost any part of the dissertation. It can be a question, theory, data, data source, method, analysis, or result.

4

17211-DevelQualDiss-Sciences

12/8/08

2:20 PM

Page 5

Understanding Originality and Significance Degrees of Originality There are, of course, levels or degrees of originality. At the lowest level, originality typically involves applying known methods or techniques to new data, materials, or the like. Originality in this sense might make a small or incremental change to the knowledge base and be considered to have little consequence. At the higher levels, making an original contribution typically involves asking new questions or applying new methods to old or new problems and achieving results that are immediately recognizable as having consequence. Contributions such as these advance the field, are publishable, and often appear in top-tier journals. At the very highest level, the contribution has the potential to change the field. It is rare for a graduate student to make an original contribution at the highest level. Indeed, few faculty make contributions at this level. Table 2.1 provides some examples, drawn from biology, physics, electrical and computer engineering, and mathematics, of what it means to make an original contribution. As you consider these descriptions, think about the questions you might ask your advisor and committee members about their expectations and the potential of your project to create new knowledge, take a new approach to inquiry, invent something, or make something work better or differently. The expectations for originality also vary by subdiscipline. In experimental fields, originality usually involves generating novel information or data. By contrast, in theoretical fields, it could be a new idea or algorithm. However, your role in making an original contribution might be difficult to identify, especially if you are in a field that relies on a high degree of teamwork, or if you are one of several students collaborating on an advisor’s funded research project. It is, therefore, important to talk with your advisor about your role in the project and how to identify your contribution to the field. Understand Expectations Because expectations for originality vary across disciplines, subdisciplines, and even advisors, it is important to ask your advisor and committee, from the start, for clarification about what is expected of you. Ask to see examples of what they mean when they—and others in the field—call something original. In your ongoing conversations, discuss specifically where, in your project, you are most able to make an original contribution. Is it through, for example, your research question, theory, data, data source, method, analysis, or

5 result? Are you using an existing concept or method in a new way? Discuss, as well, how you might best bring your original contribution to the reader’s attention.

Significance A significant contribution is defined primarily by its consequences. As suggested by the descriptions provided by the faculty from biology, physics, electrical and computer engineering, and mathematics who participated in the study (summarized in Table 2.2), a significant contribution is of interest and importance to the community and influences the field by changing the way people think. It is important to understand that significance is typically determined, over time, by the disciplinary community, not the individual advisor or committee. Moreover, understanding and appreciation of a contribution often emerges, sometimes many years after the fact. Sometimes a topic, issue, or approach is “before its time”; it does not immediately fit with prevailing knowledge or thought, but later it is recognized as significant. Although significance is frequently stated as a requirement for the PhD, graduate students rarely make significant contributions—or are rarely recognized at the time for making significant contributions. Faculty do not typically expect them to do so. They look, rather, for the capacity to produce a significant contribution in the future. The faculty who participated in the study described a significant contribution as something that is useful and will have an impact and is therefore publishable in top-tier journals because it • offers a nontrivial to very important original breakthrough at the empirical, conceptual, or theoretical level; • answers an important question or solves an important problem; • causes those inside, and possibly those outside, the community to see things differently; • influences others’ research; • has implications for and advances the field, the discipline, other disciplines, or society. As with originality, there are degrees of significance. At the highest level, significance is a function

17211-DevelQualDiss-Sciences

12/8/08

2:20 PM

Page 6

D E V E L O P I N G Q U A L I T Y D I S S E R T AT I O N S I N T H E S C I E N C E S

6

Table 2.1 Original Contribution: Characterizations From Faculty in Biology, Physics, Electrical and Computer Engineering, and Mathematics Biology An original contribution is something that has not been done or found before. It is something that moves the field forward. It might take the form of new questions, theory, methods, techniques, technologies, observations, data, information, inferences, or analytical approaches. It might involve applying a new method or analysis to an old question or problem, new answers to an old question, or combining a problem and a technology that had not been combined before. Physics An original contribution is something new in the state of the art of the field that is worthy of publication in the top-ranked journals; something that has not been solved, done, seen, or known before. It results from • coming up with new ways of thinking about old ideas; • inventing or developing new techniques, computations, statistical procedures, or theoretical descriptions; • improving mathematical formulations; • pushing the frontier in sensitivity and accuracy of the theory being tested; • building something new such as a small apparatus or a more complex one; • making new forms of matter; • making measurements that lead to an unexpected answer; • closing off an unproductive or incorrect line of research; • tying together different experiments or facts in one experiment. Electrical and Computer Engineering An original contribution is something that has not been done before. It comes from the student not the advisor. It might entail • • • • • • • • • •

presenting a new and innovative theory, method, or application; improving on a solved problem by developing a better, cheaper, or easier method; successfully solving new or existing problems; developing new algorithms, models, or computational methods; inventing new devices; making something work better or differently; taking a method or a result from another field and applying it to an engineering problem; making a conceptual contribution; elucidating a topic; advancing the knowledge base or the state of the art.

Mathematics An original contribution is something new, something not previously proved or done before. It would be a publishable result, a step beyond what is currently available in the literature that extends the frontiers of the discipline. It might offer a new idea, perspective, connection, approach, method, computation, theory, theorem or section of a theorem, counterexample, proof or shorter proof with more insight and motivation, result, or application; It might solve a big, open problem with a known technique or offer a new approach to an old problem or an improvement on known results. It might advance the body of knowledge in mathematics, including improving the presentation and understanding of how known things fit together. In applied mathematics, an original contribution might be made by formulating a new problem, making connections between fields that have not been made before, or obtaining new results.

17211-DevelQualDiss-Sciences

12/8/08

2:20 PM

Page 7

Understanding Originality and Significance of the field’s long-term interest in the problem or result, the difficulty involved in solving the problem, the influence of the result on further developments in the field, as well as the degree to which the result affects other fields, disciplines, and even society. At the lower levels, the contribution is a small or incremental improvement. A contribution is of little or no significance if people say, “So what?” However, the most significant contributions are often ones that initiate a new trend or destabilize a conservative area, thus creating new questions and new research agendas. Table 2.2 summarizes the descriptions of a significant contribution provided by the faculty from biol-

7 ogy, physics, electrical and computer engineering, and mathematics who participated in the study. Understand Expectations Talk to your advisor and committee members about their expectations for significance. Do so early in the process, and continue this discussion as you progress. Do they want to see a significant contribution in your dissertation or do they want you to demonstrate that you have the capacity—the knowledge and skills—to make a significant contribution in the future? What specific aspect of your project has the potential to make a significant contribution? How can you best present this contribution?

Table 2.2 Significant Contribution: Characterizations From Faculty in Biology, Physics, Electrical and Computer Engineering, and Mathematics Biology A significant contribution is something that is publishable or published in top journals, increases understanding of an underlying mechanism, answers a very important question, disproves prior work, introduces a new biological process, influences or changes the field, moves the field forward, or opens up a new field. Physics A significant contribution is one that is original and publishable, advances the understanding of the field, raises questions about current knowledge, makes predictions, answers existing questions, explains experimental facts, points to new directions for research, is of interest to people outside the community, or has an impact on other people’s research. Electrical and Computer Engineering A significant contribution is a useful breakthrough that will have impact, such as putting a folk theorem into a solid theoretical framework; combining existing techniques; coming up with a clever solution to a very important problem; solving a long-standing problem; or developing a product that is adopted by a company. It may have implications or lead to applications in other fields and areas. It could stimulate further work or open up a new field. Mathematics A significant contribution is something that is of interest to others that provides a better understanding of and advances the subject or field. It is a profound improvement in what is known. It is a publishable result that has depth and breadth, a new idea, a huge insight, or something that is not intuitively or mathematically obvious. It makes unexpected connections between things or fields of mathematics. It might accomplish something that other people have tried to do and failed, solve or make headway on an open problem, offer a unified proof of two disparate results, introduce a new methodological approach that can be applied to other things, or have implications that are of value to a wide community.

17211-DevelQualDiss-Sciences

12/8/08

2:20 PM

Page 8

3 Aiming for Excellence in the Dissertation

the scholarship of others to learn. However, in the process, you also make judgments about the quality of their work. In the same way that you make judgments about the scholarship of others, your advisor and committee members make holistic judgments about the quality of their students’ dissertations. However, your faculty advisors also read student work with another purpose: They read to teach. They must read carefully to see where they can suggest improvements in students’ conceptualization of the topic, in their methods, in their presentation of results, and in their writing. Moreover, your advisors and committee read to certify quality. That is, they must make sure their students’ dissertations demonstrate professional competence and capacity for future professional-quality contributions. In short, your advisors and committee are reading your drafts and final version to determine whether your dissertation is at a level of quality that demonstrates your readiness to make the transition from student to professional.

Y

quality. They are good enough. In rare instances, some dissertations are unacceptable. The faculty participating in the study provided descriptions of what makes a dissertation outstanding, very good, acceptable, or unacceptable. In the following sections, you will find summaries of what they said about quality at these different levels. Use these summaries as a way to start planning and, later, evaluating your own work. They are also useful guides as you discuss your project with your advisors and committee members: Am I making progress toward my goal for excellence? Where do I need to make a special effort to develop my dissertation? What might I do to improve the quality?

OU READ

Outstanding Outstanding dissertations are characterized by originality, high-quality writing, and compelling consequences. They show deep knowledge of the literature and mastery of the subject matter. They display a richness of thought and insight, and make an important breakthrough. The body of work in outstanding dissertations is deep and thorough. In experimental fields, the experiments are well designed and well executed. The quality and care put into the measurement techniques and analyses instill confidence in the results. Even though outstanding dissertations are rare— faculty see them once or twice a decade, if that often— the faculty in the study were able to provide a very consistent set of descriptors. They described an outstanding dissertation in the biological and physical sciences, engineering, and mathematics at the higher levels of originality or significance in that it

Degrees of Quality Like published articles, completed dissertations have been written and rewritten. The ideas and presentation have been subjected to expert criticism and honed through repeated drafts, feedback, and editing. And, like published research articles and books, most dissertations are very good. A few dissertations are remarkable or outstanding in some aspect. On the other hand, some dissertations are, for a variety of reasons, just within the boundaries of the profession’s standards of

• asks new questions or addresses an important question or problem;

8

17211-DevelQualDiss-Sciences

12/8/08

2:20 PM

Page 9

Aiming for Excellence • uses or develops new tools, methods, approaches, devices, or new types of analyses; • pushes the discipline’s boundaries and opens new areas for research; • is of interest to a larger community and changes the way people think. They explained that in its execution, the outstanding dissertation • • • • • • • • •

is very well written and very well organized; exhibits mature, independent thinking; is thoroughly researched; clearly states the problem and explains why it is important; has a brilliant research design; has well-planned and well-performed experiments; is theoretically sophisticated and shows a deep understanding of theory; has a complete and convincing analysis; ties the whole research project together in the conclusion.

They also described the outstanding dissertation in terms of less-tangible attributes: “compelling,” “concise,” “creative,” “elegant,” “insightful,” “persuasive,” “simple,” and “surprising.” The results or conclusions of an outstanding dissertation push the discipline’s boundaries and are publishable in the top-tier journals. Along with offering new and significant knowledge, an outstanding dissertation is a pleasure to read. Each part of the outstanding dissertation, from introduction through conclusion, is excellent, and the pieces are integrated seamlessly. The writing is clear and persuasive. The ideas are set out very clearly and concisely. The writer anticipates—and answers—the reader’s questions. Outstanding dissertations were described as “page turners,” surprising and edifying the reader. Readers often react with, “Wow! Why didn’t I think of that?” Very Good The very good dissertation is very good indeed. It fulfills the purposes of the dissertation requirement and establishes the student as a capable scientist, mathematician, or engineer. The majority of the dissertations that faculty see are very good, and this is the level they expect of most graduate students.

9 The faculty in the study explained that a very good dissertation displays the student’s mastery of the field, addresses a meaningful question or problem, and is executed competently. Although it might not hold the promise of altering the field, it has the potential to make a modest contribution to the field by expanding its knowledge and thinking. The dissertation contains material for two or three papers that could be published in top-tier professional journals. More specifically, the faculty described a very good dissertation as “original or significant,” making a “modest contribution to the field.” A very good dissertation has a good question or problem that is the next step in a research program, and, as such, the problem tends to be small and traditional. The dissertation shows understanding and mastery of the subject matter; uses appropriate, standard theory, methods, and techniques; includes well-executed research; demonstrates technical competence; and presents solid, expected results/answers. Although the very good dissertation is well written and well organized, it misses opportunities to completely explore interesting issues and connections, and has some loose ends. Acceptable A dissertation that meets the basic criteria for the award of the PhD is considered acceptable. Such a dissertation contains a sufficient amount of solid work to demonstrate that the student can do research. It might result in some conference papers, but it has little in the way of publishable material, and what is publishable is likely to be accepted by lower-tier journals. The faculty in the study explained that an acceptable dissertation demonstrates technical competence and shows the student’s ability to do research, use standard methods, and minimally to competently apply theory to a problem. However, they noted, a student might display a narrow understanding of the field or of theory, or review the literature adequately but not with insight. The analysis might be unsophisticated or limited; it does not explore all possibilities and it misses connections. The acceptable dissertation shows little promise of adding much to the field. It is not very original or significant because it is narrow in scope, focuses on a question or problem that is not interesting, or has predictable results. It might be a highly derivative, small extension of someone else’s work. The writing is often weak and may need strong editorial work.

17211-DevelQualDiss-Sciences

10

12/8/08

2:20 PM

Page 10

D E V E L O P I N G Q U A L I T Y D I S S E R T AT I O N S I N T H E S C I E N C E S Sometimes a dissertation that is adequate may reflect circumstances. In some experimental fields, the research simply might not have worked out as planned or expected. Sometimes an acceptable dissertation is the result of choices and compromises: The student has accepted a job or postdoc position and needs to sprint to finish. In such instances, the student has achieved a primary purpose of graduate education, which is getting a professional position. More typically, however, an acceptable dissertation is the product of poor communication between student and advisor, or inadequate advising. Because much of this guidance should take place before you begin to write the dissertation, it is important to talk soon and in detail with your advisor and other faculty members about your topic, your research question or problem, your plan for researching it, the methods you will use to collect and analyze data, the results you are getting, and your interpretation of the results. Get early feedback on your plan for organizing your dissertation and presenting your results. In addition, be sure to get feedback on the quality of your writing as you begin to draft the chapters. Because strong skills in organizing and writing are a critical professional attribute, even if your dissertation is very good or even outstanding in other respects, it will be considered only acceptable if you cannot communicate your ideas clearly and effectively. Just as excellent writing enhances a solid piece of scholarship, weak writing undermines otherwise excellent ideas and research. Unacceptable It is your responsibility to produce professional-quality work, and it is your advisor’s responsibility to prevent unacceptable work from advancing. As the faculty who participated in the study concurred, faculty advisors should provide the guidance necessary to ensure that the dissertation meets professional standards. The advisor should make sure that the student is working with a clearly defined question or problem and must make sure that the student is using proper methods. The advisor should provide prompt and constructive feedback. It is your responsibility to follow through on your advisor’s and committee’s guidance. Work that is poorly written and full of errors and

mistakes or has other serious flaws is not of adequate quality. The faculty in the study were clear that they would turn back a draft if the question or problem is trivial, weak, unoriginal, or has already been solved. Work that does not demonstrate that the student understands basic concepts, processes, or conventions of the discipline is unacceptable when • theory is missing, wrong, or not handled well. • methods are used inappropriately, or incorrect methods are used. • the data are flawed, wrong, false, “fudged,” or misinterpreted. • the results presented are obvious, already known, unexplained, or misinterpreted. • the analysis is wrong, inappropriate, incoherent, or confused. • the conclusions drawn from the data are invalid or oversold. The faculty also described the unacceptable dissertation in terms of its presentation: The organization is confusing and the writing is filled with spelling and grammatical errors. They said that a dissertation that violates standards of academic integrity through plagiarism, falsification of data, or misrepresentation of data is unacceptable. These problems should be caught early. Use the criteria in this booklet as a starting point for identifying problems or as a way to plan improvements. If your advisor and other members of your dissertation committee ask you to revise and rewrite, make sure you understand specifically what you need to do to improve—and make the improvements.

Examples from the Disciplines The faculty in biology, physics, electrical and computer engineering, and mathematics who participated in the study offered descriptions of the dissertation in their discipline at these four quality levels. As you consider the summaries of their descriptions in tables 3.1 through 3.4, develop questions that you might raise with your advisor and committee about their expectations for quality in general and for your particular project.

17211-DevelQualDiss-Sciences

12/8/08

2:20 PM

Page 11

Aiming for Excellence Table 3.1 The Biology Dissertation at Different Quality Levels Outstanding Very organized and well written; the writing is clear, concise, critical, persuasive, and compelling; focused, coherent, and organized around a major theme or question; original and significant; expresses new and independent ideas; addresses a very important issue or answers a long-standing question; shows a deep understanding of the literature and the gaps in the field; hypothesis driven; has well-planned and well-performed experiments; uses or develops new tools, methods, approaches, or new types of analyses; the experiments are brief and very well described; has a large quantity of high-quality data; the data are extremely clear; has a very significant new discovery; the conclusion ties the whole thing together; has an impact on theory; opens up a new area for research; will move the field in a new direction. Very Good Solid work; has an argument; well written, well organized, and broad in scope; original and significant; the quality of the science is good; demonstrates understanding of all aspects of the subject; has a novel, timely question or may look at an old question with a new approach or a new analytical method; makes a prediction; uses appropriate techniques and analyses; has all the right controls; the data are very well done; provides solid answers; may confirm an already known answer; will not necessarily have a huge impact on the field. Acceptable Workmanlike; a significant amount of solid work is done reasonably well; is a chore to read; original, or not very original and not very exciting; has a few innovative things but little in the way of publishable data; the science is acceptable but is not particularly good science; the concepts are derivative; sets up a problem and answers the question, but the question is not exciting; the literature review is adequate, shows acquaintance with the key papers but does not really discuss what is important about them; technically adequate; uses good scientific methods; the experiments are reasonably well done; has all the right controls; produces some novel data; adds data to an existing hypothesis; the results are useful but not exciting; may confirm what is already known; not a particularly meaningful contribution; not going to have a great impact on the field. Unacceptable The quality of the science is not good; lacks depth of understanding of the project; does not make an original contribution; the writing is bad, has no story line or argument, has spelling and grammatical errors; does not have a good question; the experiments are poorly done and poorly analyzed; the quality of the data collection and statistical analyses is poor; may have engaged in unethical behavior; the data are false or fudged; the data are not interpreted well; makes too much of the results; draws invalid conclusions from the data; does not (cannot) explain what has been done or what it all means.

Table 3.2 The Physics Dissertation at Different Quality Levels Outstanding Based on outstanding research; written beautifully and coherently; ideas flow at a high level of sophistication; puts things in context; addresses a scientific audience that is broader than that of the narrow discipline; exhibits very mature, high-quality, original, independent thinking; explains something everybody knows in a very insightful way; opens new vistas for understanding some element of the subject; uses a method from a different area; provides a good incremental solution or makes significant progress on a problem; solves a problem; contains some element of a breakthrough such as a new discovery or a new technique for collecting or analyzing data; develops a model that explains the data; synthesizes two or more separate things; makes connections to, or contains ideas or predictions that are transferable to other areas; makes a contribution to and advances the field or subfield. Very Good A large body of solid work with a lot of very nice results; original but does not contain a real breakthrough or open up the field; writing is organized and flows nicely; has a coherent argument; displays mastery of the subject matter but puts it in a less clear perspective; the style needs some polishing; may use a lot of buzzwords or terms that are not directly related to the project; may not understand all the possible connections and may not explain the most important ones as well as possible.

11

17211-DevelQualDiss-Sciences

12/8/08

2:20 PM

Page 12

D E V E L O P I N G Q U A L I T Y D I S S E R T AT I O N S I N T H E S C I E N C E S

12 Table 3.2

(continued)

Acceptable Acceptable amount of solid work that does not go much beyond what is required but shows that the student has mastered the apprentice physicist role; uninteresting and uninspiring; exhibits very little originality or imagination; presented clearly, but style and content may be far apart; the English is sometimes very bad; demonstrates that the student can take data, analyze them, and do calculations using the same techniques that have been used before; results are solid but not exciting; lacks broad understanding of the field; the student has not made all the connections he or she could; marginally advances the field. Unacceptable The body of work is inadequate and not of sufficient quality to be publishable; just stapling papers together without providing background; the problem is not original; the student completely misunderstands some very fundamental physics; arguments are wrong; uses the same methods to redo an experiment; the results are wrong, substandard, or already well known; does not advance the field.

Table 3.3 The Electrical or Computer Engineering Dissertation at Different Quality Levels Outstanding Well written; clearly states the problem and why it is new and important; writing demonstrates clear thinking; sets out ideas clearly and concisely; presents a convincing argument; includes many details; original, significant, insightful, and creative—puts things together in unique ways; shows intellectual effort, depth, and tenacity; takes knowledge to a new level; based on mathematical or physical science; addresses a new problem, large class of problems, or a problem that has been of great interest to the field; thoroughly researched; invents new methods or devices; results in an elegant solution or a general model that applies to a broad class of problems; obtains results that are of interest to the larger community; results in several publications in first-ranked journals in different areas; opens a new area for research. Very Good Very solid; well written; clear, comprehensive, and coherent; original but not very significant; lacks the sparkle of elegance; lacking in one of the key tasks (theory, methods, data analysis); problem is not broad, interesting, or significant; has solid theory, methods, and data analysis; data and results are described in detail; misses several opportunities; has some obvious loose ends; makes a modest contribution to the field. Acceptable Good work but feels incomplete; demonstrates the student can do research but does not demonstrate true mastery of the area; difficult to understand; may need strong editorial work; project is narrow in scope; original but not very significant; does not make the case for why the research is new or important; not particularly interesting or creative; introduction is sloppy; shows lack of understanding of how referenced papers fit together; does not place the work in context; the theory and methods are marginal; the experiments are not exciting or do not work; connections are missed, not fully explored, or made in a tenuous way; has some applications; results in some conference papers; a small, weak contribution. Unacceptable Has big holes; poorly written, difficult to follow; not original; problem is insignificant or has already been addressed; may have great idea or conceptualization but the student does not have the technical skills to carry it out or has the technical skills but not the conceptualization; the context is missing or the student misses the point of the context; big problems with the topic, theory, or methods; no new method or results; talks only about the mechanics of a certain engineering solution; poor presentation of correct results; the student understands what the solution does but does not know what problem it solves or how it relates to other people’s problems; does not advance the state of the art or make a new contribution.

17211-DevelQualDiss-Sciences

12/8/08

2:20 PM

Page 13

Aiming for Excellence Table 3.4 The Mathematics Dissertation at Different Quality Levels Outstanding Original, significant, and hard to do; short, simple, elegant, surprising, beautiful, and out of the ordinary; does something others have tried and failed to do, with a combination of good, original, artistic ideas, clever original arguments, and exceptional technical virtuosity; uses ideas or techniques from other areas that no one thought to use before; provides a very simple proof for a result or a conjecture thought to require a long proof; decisively solves a problem that many people have wanted to know about; develops new methods or approaches that other people will use; anticipates developments to come. Applied mathematics: intellectually deep, even if it does not create new mathematics; provides new insight into and understanding of the natural sciences. Very Good A very solid, moderately significant contribution that proves the student has mastered the methods and technologies of the field and can do mathematics; well written; somewhat surprising; contains a few new ideas; technically difficult; obtains the expected result with the expected method; uses standard methods to solve a new problem or prove a new theorem; solves a less well-known problem that fewer people have tried to solve; has a very good result; contains material for one to three papers; has a reasonable chance of having some impact. Acceptable Student has done enough work on a problem that is typically given by the advisor; has mastered the literature, figured out the techniques, and can think about and solve problems, often with some hints from the advisor; usually well written but may need better structure and organization and may contain typos and grammatical errors; the problem is not exciting; the proof is pretty standard but done correctly; result is not very interesting or surprising; has a few new approaches or ideas but shows little original thought or creativity; tends to be narrowly focused and not technically difficult; lacks elegance—contains long, very technical calculations without a lot of insight; adds to the common literature in mathematics; extends knowledge and moves the field forward a little; will not have a significant impact on the field. Unacceptable Not original or significant; original but too easy or trivial; has mistakes or errors; the result is already known, the student copies (plagiarizes) existing work or borrows parts of existing work and puts it in student’s own language; the theory or results are wrong; the proof is wrong because of technical things or the student does not understand the subject; the proof is incomplete or contains a serious gap or mistake; the proof needs to be simplified; the student missed some nontrivial details; a very easy extension of already known work; the student did not recognize that the result could be easily proven in a few lines; a compendium of existing examples.

13

17211-DevelQualDiss-Sciences

12/8/08

2:20 PM

Page 14

4 Maintaining Consistent Quality Within the Dissertation

N A SUCCESSFUL DISSERTATION ,

the parts work together to form an integrated whole. Each piece has a clear function and relates to the others. Taken together, all of the parts connect to form a coherent argument. Indeed, as the faculty members in the Making the Implicit Explicit study talked about quality in the dissertation, they consistently talked about the quality of the argument, using words like “coherent” and “convincing.” They described high-quality dissertations as “well organized”: the student uses all of the parts of the dissertation to build and support a clear, overarching idea. Because the expectations for the parts of the dissertation, their relationship, and importance vary, not only among disciplines but from department to department and among faculty, it is critical to understand what your committee expects or recommends regarding the tasks and organization of your dissertation. But first, understand the difference between the form of the dissertation and the tasks of the dissertation. The form is the structure, the parts or divisions. Think of the form as the presentation. You accomplish the tasks, regardless of the dissertation’s structure. Think of them as the substance.

lished. The student is expected to connect the articles in a meaningful way. Faculty in some fields or departments ask that the dissertation be organized specifically by tasks, with section titles such as introduction or problem statement, literature review, theory, methods, results or analysis, and discussion and conclusion. It is very important to understand what, exactly, your advisors and committee mean when they talk about the chapters, literature review, or the methods section. Not only do disciplines differ in their approaches, but each department and university has different requirements—that may be under review or in the process of being revised—about the structure of the dissertation.

I

Understand the Tasks While there is no set formula for organizing the dissertation, there are, however, essential tasks that must be completed no matter how the dissertation is organized. Through these tasks the student demonstrates his or her professional skills and knowledge. For example, at this point in your education you should know the important literature in the field and the state of current thinking about your topic. Your task is to present this knowledge in the context of your research question, to synthesize it, to do so succinctly, and to offer this synthesis in service of your argument. You might be required to present this information in a section called “literature review.” If your dissertation is a compilation of papers, you might not have a literature review section, but you will incorporate a review and synthesis of the literature at appropriate points within the papers.

Understand the Form In your early conversations with your advisors about your dissertation topic, it is important to be clear about the requirements for the form of your dissertation. In many of the science-related disciplines or departments, students present a collection of papers on their topic; some of these papers may even have been already pub-

14

17211-DevelQualDiss-Sciences

12/8/08

2:20 PM

Page 15

Maintaining Consistent Quality

15

Table 4.1 The Tasks of a Dissertation in Sciences, Engineering, and Mathematics Science and Engineering Introduction/problem statement Literature review Theory Method Results/data analysis Discussion and conclusion Mathematics Introduction/problem statement Discussion of the literature Statement of results/theorem Approach to the problem/techniques Proof of results Conclusion/future directions

Table 4.1 lists the tasks that the biology, physics, engineering, and mathematics faculty in the Making the Implicit Explicit were asked to discuss. Be aware, however, that ideas about and requirements for the various tasks of the dissertation vary not only by discipline, department, and university, but among faculty members in the same department. For example, while some faculty who participated in the study observed that they do not find literature reviews useful, others said they are a very important part of the dissertation, noting that the process of reading the literature helps the student learn to synthesize information and ideas and teaches the student how researchers communicate. Not only should all the parts of the dissertation fit together and each task be addressed, but they should also be of consistent quality. As we discuss each of the tasks, we provide in Tables 4.2 through 4.7 summaries of the descriptions of faculty from biology, physics, electrical and computer engineering, and mathematics. The Introduction In the sciences the introduction is often an extended abstract that functions as an executive summary. The student’s task is to state the problem, set the project in context, present the research question, and let the reader know the general strategy for the argument. As a distinct chapter, the introduction often includes the literature review and theory, and provides an overview of the entire dissertation. Many students write or rewrite the introduction after all else is completed. In the paper-

style dissertation, the introduction typically connects the papers and identifies a common theme. Literature Review In the traditional-style dissertation, the literature review is typically part of the introductory chapter. In the paper-style dissertation, each substantive chapter starts with an introduction and a literature review. The task of the literature review is not simply to summarize the literature but rather to connect the problem to the research on the topic, provide a context by synthesizing the history and controversies of the field, and build a story that leads to the hypotheses. The process of choosing the appropriate works to cite and of crafting an argument is not easy. However, doing so is an important professional skill. Theory For the theoretical dissertation, the theory has to be new and original and significant. The theory section is really the outcome of the dissertation, the central task. By contrast, the experimental dissertation might just present a summary of relevant theories and explain how they relate to the experiment. In general, the student’s task is to link theory to the research question and the research methods. Method or Approach An essential task in a science or engineering dissertation is providing enough detail so that the experiment can be understood and replicated. The write-up

17211-DevelQualDiss-Sciences

12/8/08

2:20 PM

Page 16

D E V E L O P I N G Q U A L I T Y D I S S E R T AT I O N S I N T H E S C I E N C E S

16

Table 4.2 Introduction/Problem Statement in the Science, Engineering, and Mathematics Dissertation at Different Quality Levels Outstanding • • • • • • • • •

well written brief, interesting, and compelling motivates the work, has a hook provides a clear statement of the problem explains why the problem is important and significant places the problem in context presents an overview of the theory, methods, results, and conclusions lays out the study’s implications provides a road map of the dissertation

Very good • • • • • • •

well written interesting has breadth, depth, and insight motivates the work poses a good question or problem explains why the problem is important and significant provides an overview of the dissertation

Acceptable • • • • • •

not well written or well organized lacks or provides minimal motivation for the work makes a case for a small problem does not do a good job of explaining why it is important provides minimum or poor context for the problem presents minimal overview of the work

Unacceptable • • • • • • •

poorly written and organized provides no motivation for the problem problem is not stated, is wrong, or trivial does not make the case for the importance of the topic does not provide or does not put problem in a clear context does not present an outline or overview of the research contains extraneous material

must be thorough and include justifications for each method employed, as well as a discussion of its strengths and weaknesses. However, while some dissertations discuss how methods are used, others are about building them and using them as tools. Discussion of methods might be the central outcome in a tool-building dissertation. For example, the engineering faculty in the study described the methods section as being the “core of the dissertation”: de-

scribing the work in detail is essential because it is how engineers persuade others of the validity of their work and also because other people may base their work on it. Results and Data Analysis In the empirical sciences and engineering, data analysis and results involve taking raw material and trying to make sense out of them. This task is the “bridge”

17211-DevelQualDiss-Sciences

12/8/08

2:20 PM

Page 17

Maintaining Consistent Quality Table 4.3 Literature Review in the Science, Engineering, and Mathematics Dissertation at Different Quality Levels Outstanding • • • • • • • •

comprehensive, thorough, complete, coherent, concise, up to date shows critical and analytical thinking about the literature synthesizes the literature integrates literature from other fields displays understanding of the history and context of the problem identifies problems and limitations like a good review article makes reader look at the literature differently

The student • • • •

is selective (discriminates between important and unimportant works). identifies and organizes analysis around themes or conceptual categories. adds own insights. uses the literature to build an argument and advance the field.

Very Good Comprehensive but not exhaustive The student • • • • •

provides a thoughtful, accurate critique of the literature. shows understanding of and command over the most relevant literature. selects literature wisely and judiciously. sets the problem in context. uses literature to build a case for the research.

Acceptable The student • • • • • • • •

provides adequate coverage of the literature. demonstrates that he or she has read and understood the literature. does not provide critical analysis and synthesis. is not selective (does not distinguish between more- and less-relevant works). misses some important works. cites some works that are not relevant. provides an undifferentiated list (“This person said this, this person said that.”). does not put problem in context.

Unacceptable Missing, inadequate, or incomplete The student • has not read enough and does not cite enough sources. • misinterprets or does not understand the literature. • misses, omits, or ignores important studies, whole areas or literature, or people who have done the same thing. • has not read the sources or has only read the abstracts. • cites articles that are out of date. • does not provide a context for the research.

17

17211-DevelQualDiss-Sciences

12/8/08

2:20 PM

Page 18

D E V E L O P I N G Q U A L I T Y D I S S E R T AT I O N S I N T H E S C I E N C E S

18

Table 4.4 Theory in the Science, Engineering, and Mathematics Dissertation at Different Quality Levels Outstanding • • • • • • • • • •

original, creative, insightful, innovative simple and elegant well conceived, logically consistent, and internally coherent identifies and critically analyzes strengths and weaknesses uses more than one theory compares or tests competing theories advances concepts develops, adds to, revises, or synthesizes theory(ies) aligns with research question, methods, and observations has broad applicability

Very Good Complete and correct • informs the research question and measures The student • uses existing theory well. • identifies where it works and where it does not work. Acceptable The student • understands theory. • uses theory appropriately. • does not specify or critically analyze the theory’s underlying assumptions. Unacceptable Absent, omitted, wrong, or used inappropriately The student • misunderstands or misinterprets theory. • cannot explain it or why theory is being used. • does not align theory with the research question, literature review, or methods.

between the theory and methods that allows the student to establish the conclusions. It should “bring it all together” and answer the question, “What has been done in this work?” As an independent section, it should have a coherent introduction to and a summary of the results. It should provide enough detail so the reader can understand the intermediate steps. In mathematics, the statement of results/theorems comes before the proof of results. The statement of results/theorems task is one of the most important tasks of a mathematics dissertation and “the main indica-

tor” of a student’s research potential. The statement of results/theorems needs to be clear, correct, and coherent. The proof of results task is similarly indispensable. The quality of the writing is also very important at this point of the dissertation. Poor writing makes it difficult to impossible for faculty to decide whether the proof is correct. Discussion or Conclusion The discussion or conclusion is typically the last section or chapter of a dissertation. Some faculty draw a

17211-DevelQualDiss-Sciences

12/8/08

2:20 PM

Page 19

Maintaining Consistent Quality

19

Table 4.5 Method or Approach in the Science, Engineering, and Mathematics Dissertation at Different Quality Levels Outstanding • • • • • • •

original, clear, creative, and innovative provides thorough and comprehensive description identifies strengths and weakness/advantages and disadvantages flows from question and theory uses state-of-the-art tools, techniques, or approaches applies or develops new methods, approaches, techniques, tools, devices, or instruments uses multiple methods

Very good Appropriate for the problem The student • uses existing methods, techniques, or approaches in correct and creative ways. • discusses why method was chosen. • describes advantages and disadvantages. Acceptable Appropriate for the problem The student • uses standard or less-sophisticated methods correctly. • provides minimum or sufficient documentation. • shows basic competence. Unacceptable Lacks a method • method is fatally flawed The student • • • •

uses wrong method for the problem. uses method incorrectly. uses methods that do not relate to question or theory. does not describe method or describes it poorly (insufficient detail).

distinction between the discussion and the conclusion: The conclusion summarizes and wraps things up, whereas the discussion, which is more important, should tie in to the introduction and put the work in a larger perspective. Some disciplines and faculty consider the discussion or conclusion task to be the creative part of the dissertation, the place where the student has a chance to draw independent conclusions

about the project and show what it means in the larger perspective of the discipline. The biology, physics, engineering, and mathematics faculty who participated in the Making the Implicit Explicit study had many insightful and specific comments about the quality of the tasks of the dissertation in their fields. You can find summaries of their descrip-

17211-DevelQualDiss-Sciences

12/8/08

2:20 PM

Page 20

D E V E L O P I N G Q U A L I T Y D I S S E R T AT I O N S I N T H E S C I E N C E S

20

Table 4.6 Results or Data Analysis in the Science, Engineering, and Mathematics Dissertation at Different Quality Levels Outstanding • • • •

original, insightful analysis is sophisticated, robust, and precise aligned with question and theory results are usable, meaningful, and unambiguous

The student • • • • • • • •

uses advanced, powerful, cutting-edge techniques. sees complex patterns in the data. iteratively explores questions raised by analyses. presents data clearly and cleverly. makes proper inferences. provides plausible interpretations. discusses limitations. refutes or disproves prior theories or findings.

Very good Thorough, appropriate, and correct The student • • • • •

uses standard methods. produces rich, high-quality data. links results to question and theory. substantiates the results. provides plausible arguments and explanations.

Acceptable Objective, routine, and correct The student • • • • • •

aligns the data and results with the question and theory. produces a small amount of thin data. has correct results, but they are not robust. includes extraneous information and material. has difficulty making sense of data. has an interpretation that is too simplistic.

Unacceptable • • • • •

analysis is wrong, inappropriate, or incompetent data are wrong, insufficient, fudged, fabricated, or falsified data or evidence do not support the theory or argument data do not answer the question interpretation is not objective, cogent, or correct

The student • • • •

does not distinguish between good data and bad data. does not discern what is important or explain the results. makes improper inferences. overstates the results.

17211-DevelQualDiss-Sciences

12/8/08

2:20 PM

Page 21

Maintaining Consistent Quality tions in Appendix A. Ask your advisors to be similarly specific, if not more specific, about what constitutes high-quality work in the tasks of the dissertation. Above all, understand the form you are required to use for your dissertation and the tasks you must address within it. Take time to map out your argument

21 and discuss in detail with your advisors and committee how you will make the dissertation work as a coherent whole. Discuss with them how each of the parts will contribute to the argument. Study recent dissertations from your department to see how the students made the pieces work together.

Table 4.7 Discussion or Conclusion in the Science, Engineering, and Mathematics Dissertation at Different Quality Levels Outstanding • • • • • • • • • • •

short, clear, and concise interesting, surprising, and insightful summarizes the work refers to the introduction ties everything together explains what has been accomplished underscores and explains major points and findings discusses strengths, weaknesses, and limitations identifies contributions, implications, applications, and significance places the work in a wider context raises new questions and discusses future directions

Very Good The student • • • • • • • •

provides a good summary of the results. refers to the introduction. states what has been done. ties everything together. states the dissertation’s contribution. identifies possible implications. discusses limitations. identifies some future directions.

Acceptable The student • • • • •

summarizes what has been accomplished. repeats or recasts the results or major points. does not address the significance or implications of the research. does not place the work in context. identifies a few, nonspecific next steps.

Unacceptable Inadequate or missing The student • • • • • •

summarizes what has already been said. repeats the introduction. does not tie things up. does not understand the results or what has been done. claims to have proved or accomplished things that have not been proved or accomplished. does not draw conclusions.

17211-DevelQualDiss-Sciences

12/8/08

2:20 PM

Page 22

5 Achieving Excellence

we have offered suggestions for achieving quality in your dissertation. The final table of the booklet, Appendix B, provides practical advice gathered from the science, engineering, and mathematics faculty in the study. We close with some frank recommendations about the work ahead of you.

T

HROUGHOUT THIS BOOKLET,

Why is writing so important? The quality of your ideas will not shine through poor or ineffective writing. Regardless of how brilliant the ideas and how outstanding your research, if you cannot convey your ideas and data clearly, concisely, and coherently, the reader will not be able to appreciate their import. Likewise, if your presentation is not well planned, if you cannot effectively map out and sustain an argument or make a case, you miss an opportunity to share your knowledge and contribute to the field. Moreover, the mechanics of grammar and the style of prose affect one’s perception of a work’s quality. Many faculty believe that unclear writing reflects unclear thinking, and that a good researcher is also a competent writer. Once you are out in the field, your writing will be an indicator of the quality of your thought and your attention to the details of research. If your work is error laden, or you did not take the time to plan and organize the presentation of the material, the reader will wonder, with good reason, how careful you were in your research. The faculty in the study spoke candidly about the surprising amount of poor writing they see among their graduate students, and some even suggested that writing should be among the essential tasks of the dissertation. For these reasons, learning to write and communicate well is one of your professional responsibilities. Moreover, as an expert, you are expected to be able to communicate effectively about your field to a variety of audiences. Your dissertation demonstrates your skill in communicating with your peers in the discipline.

Practice Academic Honesty By passing the dissertation, your advisor, committee, department, and university are certifying that you have the skills, knowledge, and disposition expected of a professional in your field, which includes a commitment to integrity. Honest use of data and sources is the fundamental expectation of academic work. None of the levels of quality described here matter if you plagiarize, deliberately misuse or misrepresent sources, or falsify data. Such serious betrayal of professional standards puts your degree in jeopardy. It could even be rescinded if the dissertation is passed and then found to have these abuses.

Develop Professional-Level Writing Skills You probably noticed that “writing” is a constant item in the summaries of faculty descriptions of quality. The quality of your writing plays a large role in faculty members’ assessment of the quality of your dissertation. They are looking at both the quality of your prose (word choice, structure of the sentences) and the quality of your argument.

22

17211-DevelQualDiss-Sciences

12/8/08

2:20 PM

Page 23

Achieving Excellence Take Action to Improve Your Writing Although some graduate students, domestic and international, have problems with grammar, many also have problems with composition—basic structure and rhetoric. Some faculty report spending a great deal of time working with students to improve the quality of writing in the dissertation—time that should be focused on the larger ideas. You have made a considerable investment of time and energy learning about your discipline. If the feedback on your papers for your course work, dissertation proposal, or chapter drafts suggests that you need to pay more attention to this skill, here is a plan of action: • Know good writing. Ask your advisor or other faculty members to recommend the work of a scholar whose writing is very clear; read it not for the subject but for how it is written. • Plan your dissertation. While you are working on the dissertation, it is important to be clear about what you want to say before you begin to shape chapter sections, paragraphs, and sentences. For example, make a map or outline of your dissertation and plot out the argument. Try it out by presenting it to other students and faculty mentors. Use the map as you draft the various pieces of the dissertation. • Plan the pieces of the dissertation. Decide how you will organize each chapter. What are the main points? In what order does the reader need to know the information? What supporting data will you use, including charts, graphs, tables, equations, and other illustrations? • Write and revise. Understand the process of writing. Approach the tasks of writing and revising as separate activities, for they each demand different skills. Although writing and polishing at the same time might seem efficient, it is not. After you have drafted a piece, set it aside for a while, and then review what you have written and reshape it. Then go back again, at another time, and polish the grammar and prose by reading aloud what you have written. You will be surprised at what you see. • Follow conventions. Find and use a handbook of grammar and style; your university’s bookstore and library will have many. Ask other

23 students and faculty members to recommend books about academic writing. In addition, know the style and formatting conventions of your discipline. Many disciplines publish style guides, with instructions for everything from charts and tables to citation format and manuscript preparation. Some of these guides include discussions of grammar and punctuation or suggest good sources for information. Major journals also provide guides for preparing manuscripts. • Get feedback. Although there are many things you can do as your own editor, it is important to seek and consider the feedback of others. Go to the campus writing center for tutoring. Join a writing group and work with your peers. Ask a faculty member you respect to work over a short piece of text with you; even a half an hour in which someone edits your work and explains his or her suggestions can be extremely enlightening. • Practice writing and presenting your research. Take advantage of opportunities to present at conferences or campus research colloquia. Seize opportunities to write and speak about your field or project. The more varied the audience the better.

Set the Bar Having a clear target makes it easier for you to ask questions, seek guidance, and make corrections. It also makes it easier for your advisors to answer questions, provide guidance, and suggest revisions. The descriptions of quality presented in this booklet are benchmarks of excellence. You and your advisor and committee might find it helpful to develop specific expectations of quality for your dissertation. Doing so will help them guide your work and will help you use their feedback. If you and your advisors are interested in putting expectations for your dissertation in writing, you might start with the general examples in this booklet and customize them for your discipline and project. A larger question, though, is where to set the bar. You and your advisors need to talk about finding the balance between challenging yourself and setting real-

17211-DevelQualDiss-Sciences

24

12/8/08

2:20 PM

Page 24

D E V E L O P I N G Q U A L I T Y D I S S E R T AT I O N S I N T H E S C I E N C E S istic goals. Discuss the level of quality they expect in the dissertation. Talk about your goals for quality and your professional aspirations (you might find it helpful to put these thoughts in writing first, before the conversations). As you set the bar together, keep in mind that the “very good” that the faculty in the study discussed is the equivalent of an A. Achieving that level of excellence is an accomplishment that you can be proud of.

Engage Your Advisors Above all, we recommend that you take the initiative to speak with your advisors, committee members, and other faculty members who mentor you, and that you do so early in the process and frequently throughout it. You are entitled to guidance about your dissertation and to prompt and constructive feedback throughout the process. Know what you are aiming for and what you need to do to meet your goals. Be clear about the tasks of the dissertation and the expectations for quality for each. Work with your advisor and committee to set a realistic schedule that includes time to respond to drafts, especially early in the process. Meet the deadlines you and your committee agree to. When you meet, use their time and expertise wisely; ask questions and make sure you understand their directions. Make the revisions your advisor and committee members ask you to make—or provide them with reasoned justifications for not making them. If their feedback seems contradictory or confusing, ask for clearer or more consistent guidance. Be proactive about solving any problems. That is an important professional skill. If you feel stuck or unsure about something, talk to your advisor, a faculty member you respect, or other students. If you are not getting the guidance or feedback that you need, bring

it to the attention of someone who can help you, such as the program head, department chair, or graduate dean. Don’t withdraw and hope the problem will go away. Don’t give up. Persistence is also an important professional attribute.

Engage Your Peers Use the process of writing the dissertation to develop colleagues for the future. Sharing work with peers in order to get honest and expert feedback is a habit— and skill—you should learn early. Form a group of peers and meet regularly to check drafts, discuss problems and questions, and savor accomplishments. Meeting regularly to discuss your work will help keep you on track as you prepare to meet interim deadlines.

Applaud Yourself No doubt, at every milestone in the process, you will want to thank the friends and family members who are supporting and encouraging you. They take pride in what you’ve accomplished and the role they have played in your success. But remember, there is someone else who should be congratulated: you. Even when you sometimes feel like you are taking small steps up a big mountain, pause to recognize how far you have come; you will be surprised at the distance you have covered. By pursuing the PhD, you are making an enormous investment in yourself. Take time along the way to appreciate what your commitment to this effort says about you, personally and professionally. The dissertation is a symbol of the hours and commitment that you have devoted to the process. And when you have finished it, and the dissertation is submitted and defended, be sure to celebrate!

17211-DevelQualDiss-Sciences

12/8/08

2:20 PM

Page 25

Appendix A Tasks of the Science, Engineering, and Mathematics Dissertation

25

Introduction Outstanding Not discussed

Shows some evidence that the student knows what he or she is supposed to be doing; makes a case for the problem and justifies the work but lacks depth; does not put the problem in proper context; does not fully comprehend the nuances of the various arguments and controversies that shaped the field; lacks perspective on the field; does not establish the boundaries of the field; does not build the case for the hypotheses; student has not told the reader what he or she needs to know

Hastily prepared and poorly written; bad grammar; confusing; no argument, just a bunch of paragraphs strung together; does not express any independent ideas; does not show sufficient knowledge of the area; student does not really understand why he or she is doing what the student said he or she is doing

Page 26

Unacceptable

2:20 PM

Acceptable

12/8/08

Thorough, interesting, well written in an engaging style; expresses independent ideas; identifies an important problem in the field; demonstrates good knowledge of relevant literature; creates a context for the question; provides a historical overview and a clear description of the cutting edge of the field; develops a compelling rationale for why the question is important; makes clear how the research fits in with what has happened before and how it will push the boundary; leads the reader directly to the hypotheses; lays out implications of the research in a clear and compelling fashion; ties everything together; lays out a road map for the rest of the dissertation

Very Good

17211-DevelQualDiss-Sciences

26

Table A.1 Tasks of the Biology Dissertation by Quality Level

Literature Review Very Good

Acceptable

Unacceptable

Comprehensive; thorough; up to date; shows superior, discriminating, evaluative knowledge of the literature; synthesizes the field; brings the literature together like a good review article; adds own insights; cites only important relevant information, student knows what to put in and what to leave out; makes it clear that the student is widely read and has read things that are not cited; deeply historical, covers the history of the concept and the controversies; connects the literature to the theme that is being developed in the dissertation; explains why the work is important and where it comes from; helps build the argument leading up to the hypotheses;

Knows what the problem is and what has been done; discriminates between important and informative papers and unimportant and uninformative ones

Not in full command of the literature; cites all the right papers but does not put them in the right context; misses important papers or includes a lot of irrelevant ones; no judgment or discrimination between key papers and poor papers, treats all papers as equal; discusses every paper in succession; does not really understand the question the paper was asking; does not use the literature to push boundaries

Not thorough, less synthetic, and limited in scope; nebulous and confused; lacks knowledge of the literature; does not cover all the different points of the field; neglects half the field; misses key papers; student does not seem to understand what the questions are and what is important and what is not; does not credit or appreciate the history that went before it; incomplete survey of the literature or the survey is adequate to thorough, but there is no clear statement of the bounds of the field; just description, almost a listing of the literature

APPENDIX A

Outstanding

Acceptable

Unacceptable

Acceptable

Unacceptable

may include papers the advisor has not seen or read; may include papers not written in English

Outstanding

Understands theory; uses theory properly and correctly but may need guidance

Not discussed

Absent; student is unaware of it; misinterprets it; does not understand it; gets it wrong; cannot explain why he or she is using it; methods do not flow from it; leaps to generalizations based on limited observations

2:20 PM Page 27

Superior presentation of how the theory relates to the problem; links the question with the theory; explains why the research is going in one direction and not another; innovative, comes up with new concepts or new mathematical or statistical applications, invents theoretical representations of the data; takes the observations way beyond the actual observations; has implications that go beyond the specific situation; makes a contribution to theory; develops or adds to existing theory, revises or comes up with a new variation of a theory; an insight; is completely inventive or original; changes the way people will do research and approach data

Very Good

12/8/08

Theory

Methods Outstanding

Very Good

Acceptable

Unacceptable

Appropriate, inventive, clear; connected to the predictions; laid out in excruciating detail; precisely describes how data were handled; has a lot of detail and attention to detail; provides pros and cons about how much to trust the data with precision and accuracy; applies new methods; comes up with a new method or technique

Complete; comes up with novel ways to use existing techniques

Appropriate to the study; used correctly; explained in sufficient detail so someone can figure out exactly what was done

Leaves things out; not enough detail; misapplied, not used correctly; student did not do things that should have been done

17211-DevelQualDiss-Sciences

Very Good

APPENDIX A

Outstanding

27

Results/Data Analysis Outstanding

Analysis is appropriate and done well at a sophisticated level; uses a relatively new analytical technique; results are associated with the problem; write-up goes beyond the data in the tables and is not cluttered with extraneous data

Uses the right procedures; analyzes the data correctly in a clear-cut fashion; produces a small amount of modest data; has difficulty making sense of the data; novel insights come from the mentor or someone else

No results; results are ambiguous with respect to the hypotheses; results do not reflect the methods; results come from failed experiments; results are not presented in a succinct and informative way; data dump; insufficient data; data do not answer the question asked; student is unable to distinguish between good and bad data and between important and unimportant data; student sees too much in the data; inappropriate or incompetent analysis of the data, data do not fit the assumptions underlying the statistical test; student is unable to justify the statistical analyses; analyses are wrong; obvious things are not analyzed; flat-out reporting with no context; mentor tells student how to interpret the data

Page 28

Unacceptable

2:20 PM

Acceptable

12/8/08

Novel insights; notices things and organizes them into patterns; sees things the advisor did not see; connected to and aligned with the hypotheses being tested and other parts of the study; done rigorously and with good controls; has a very strong statistical foundation for the analysis; creative, innovative analytical methods; sophisticated analysis of the data; gets as much out of the data as possible as opposed to just the obvious; sticks to statistically relevant interpretations; unambiguous and clearly presented; good figures and graphs

Very Good

Discussion and Conclusion Outstanding Not discussed

Acceptable

Unacceptable

Simple; workmanlike; technical; summarizes the dissertation, draws it all together; simply says whether the hypothesis was accepted or rejected; repeats or recasts the results; conclusions are consistent with the results; does not address the significance or implications of the research; no next steps or suggestions for other kinds of research that might continue this line of reasoning

Inadequate; does not have any conclusions; just a summary of what was already said; student does not understand the results; student does not understand or puts results in the broader context; student does not know its impact or thinks its impact was far greater than it actually was; does not discuss the implications of the work to the extent that is warranted

APPENDIX A

Creative; flawless writing; brings everything together; really sees what the data are telling him or her; provides new insights on what was studied and extrapolates on it; identifies new problems and asks more questions; discusses future directions, which path to go down, which hypotheses should be tested next; realizes the importance, significance, and implications of the work and the contribution made; shows what it all means in the larger perspective of science; puts it in a context that goes beyond the science, discusses the broader impacts, says something about the societal importance or what it means to the world at large; opens up a whole new way of thinking about the problem; pushes the discipline further

Very Good

17211-DevelQualDiss-Sciences

(continued)

28

Table A.1

Outstanding Not discussed

Acceptable

Unacceptable

Provides an understandable motivation for the work; does not convey a sense of excitement; provides the minimum context for the work; does not make connections to other areas

Displays a clear lack of mastery; provides no motivation for the problem; does not say why the problem is important; does not provide context; student fails to cite some very important papers; not understandable

12/8/08 2:20 PM Page 29

A microcosm of the whole thesis; motivates the work; clearly written so that a general physicist or a scientifically literate person can understand the significance of the project; states where the problem is with respect to the field and situates it in a broader context; shows a deep understanding of the project— briefly describes what has been done before, what techniques are going to be used, where the project is headed, and what the results are; discusses the significance of the research in a very honest and direct way; builds connections to other scientific issues

Very Good

Literature Review Outstanding Almost like a review article; comprehensive and critical; picks out the important key papers and synthesizes them; shows understanding of where each reference being evaluated fits; puts the problem in perspective and conveys its importance

Very Good Comprehensive

Acceptable

Unacceptable

Cites all the relevant references but the student is not aware of the context; not directly related to the exact work the student is doing

Does not cite key works; cites a limited number of articles; student is unaware of the broader context; totally ignores what has been going on in the area; lacks key references

Theory Outstanding Elegant, simple, original, significant, universally applicable; student is aware of the range of theories and their strengths and weaknesses; finds a problem in a theory; points out why the current understanding is wrong or irrelevant; student understands that one parameter cannot be tweaked without looking at the whole; leads to a very hard experiment that yields wonderful results; adapts theory to actual experiments

17211-DevelQualDiss-Sciences

Introduction

APPENDIX A

Table A.2 Tasks of the Physics Dissertation by Quality Level

Very Good Not discussed

Acceptable Uses it appropriately; uses it to derive the correct physical understanding from the data; experiment demonstrates student’s understanding of theory

Unacceptable Wrong; not relevant to the experiment; not understood; regurgitated; described at a nominal level

29

(continued)

Methods Outstanding

Acceptable Student has done just enough work; provides minimum information about the method

Not discussed

Unacceptable Uses the same equipment and same methods as previous students

12/8/08 2:20 PM

A nice, coherent description of method; shows student’s understanding of the advantages and disadvantages and strengths and weaknesses of the method used; elegantly conceived theoretical or new mathematical approach to a problem; new methods, techniques, instruments, or devices that make a quantum leap; develops an algorithm; codes data in a way that yields stable and efficient results

Very Good

Page 30

Results/Data Analysis Outstanding Provides a coherent introduction and summary of the results; has enough detail to provide understanding of intermediate steps; finds and analyzes challenging data; applies a known technique to a new context; analyzes the data in a new way and obtains better, more believable results or information thought impossible to obtain

Very Good

Acceptable

Contains a large amount of high-quality data; thoroughly investigates almost all sources of error.

Correct results; does not have obvious flaws; assesses the data objectively and with integrity; does not know when to present figures that make sense; someone else has better results

Unacceptable Fabricates data; student could not measure phenomenon with precision or accuracy; does not interpret data objectively; does not provide a context for the results

Discussion and Conclusion Outstanding

Very Good Summarizes the project; refers to the introduction and ties everything together; makes uninspired connections; points to some possible future work

Acceptable Gives the expected standard discussion without any further thought; establishes links with the introduction

Unacceptable Not discussed

APPENDIX A

A short, clear, concise summary that refers to the introduction; places the results in context; explains what has been done, what it means, and how it advances the field; is insightful and makes inspired connections; demonstrates the student’s understanding of the broader field; discusses applications and suggests new directions for future research

17211-DevelQualDiss-Sciences

30

Table A.2

Very Good

Acceptable

Well written; captivating; provides motivation and establishes the importance of the problem and places it in context; presents a very clear and concise statement of the problem, results, conclusions, and contributions; lays out the plan for the dissertation

Writing is good; motivates the work but does it less well and is not captivating; clearly describes what the problem is and why it is important; starts with the big picture and narrows it down to the point being made; indicates what the contributions are

Mediocre; not well written; the ideas seem to be there; is narrower in scope

Unacceptable Problem is not stated; includes a lot of extraneous material

12/8/08

Outstanding

2:20 PM

Literature Review Very Good

Acceptable

Complete, comprehensive, up to date, organized, and coherent; has a nice logical structure; provides a critical look at the problem; supports the statement of the problem and the statement of the contribution; puts the work in a context of what has been and is being done; shows a good understanding of the state of the art; discriminates between important and unimportant papers; identifies gaps in the literature; states limitations of previous work; simplifies and discusses very complex papers; summarizes and ties together all the different methods people have been employing, using a common notation; provides detailed examples of existing methods; is a contribution in and of itself; educates the reader

Written at the appropriate level of depth with the appropriate amount of references

Good enough; pretty comprehensive but may be missing a few important works; shows that there are holes in the literature with respect to the problem

Unacceptable Inadequate or missing; does not provide a context for or relate to what is being done; omits a lot of important material; cites works not read; plagiarizes

Page 31

Outstanding

Theory Outstanding

Very Good

Creative, insightful, elegant, significant; conceived and presented logically and correctly; takes theory beyond the literature; develops a new theory; provides a mathematically correct foundation for the research

Appropriate, complete, and correct; builds on existing theory; ties the project together; defines where the theory works and where it does not

17211-DevelQualDiss-Sciences

Introduction

APPENDIX A

Table A.3 Tasks of the Electrical and Computer Engineering Dissertation by Quality Level

Acceptable Presents a lot of theory that is never used; assumes away all the difficulties

Unacceptable Omitted; student does not understand or justify the theory; uses theory inappropriately

31

(continued)

Methods Acceptable

Unacceptable

Provides a comprehensive description; has a simple, complete, elegant approach; exhausts all possibilities; combines theory and methods; has a balanced use of theory, experiments, and simulations; uses existing theory to develop methods for useful applications; seizes new tools and applies them to the problem; demonstrates things through examples or simulations

Compares chosen method against existing methods; discusses its advantages and disadvantages; identifies why the method was chosen; states all the assumptions; indicates the range of parameters in which the method will work; tests hypotheses experimentally or with simulations

Precise and complete enough so others can replicate; states what is being established and how; sequential process: does all combinations randomly; may require major corrections

Shoddy; lacks clear scientific deductive thinking; does not identify what is being measured or why; just a bunch of simulations; makes improper generalizations

2:20 PM

Very Good

12/8/08

Outstanding

Well written; clear, simple, and appropriate presentation of unambiguous results, contributions, applications, limitations, and impact; insightful; very repeatable; measurements have a high degree of precision; documentation supports the precision, accuracy, and reliability of the results; has statistically and observably significant useable results; results, including unexpected results, match or support the theory; graphically displays carefully selected variables and results; draws proper conclusions and makes proper inferences

Very Good

Acceptable

Theory and results correspond; provides an explanation for the correspondence

Sufficient; measurements, theory, and analysis align; does not justify the claim it works better than something else; needs major revisions

Page 32

Results/Data Analysis Outstanding

Unacceptable Data are inaccurate, fudged, or falsified; selectively presents only supporting data; has lots of tables but no analysis or discussion of the contribution; provides evidence that it works in only one small, narrow situation; does not understand the results; draws conclusions based on very little data

Discussion and Conclusion Outstanding

Good summary of results; clearly states contributions; has possible applications and future directions

Acceptable Not well done; provides some considerations for future work based on shortcomings of current work

Unacceptable Not discussed

APPENDIX A

Short summary that brings out major points and ties in to the introduction; contains lucid insights; places work within the context of the field; identifies contributions and applications as well as limitations and shortcomings; anticipates criticism; discusses future directions

Very Good

17211-DevelQualDiss-Sciences

32

Table A.3

Acceptable

Well written but may not clearly delineate the difference between the background and the new results; introduces the reader to the subject in a friendly way; provides an overview of the methods; states clearly what has been proved

Provides a context for the work

Problem statement is wrong or trivial, or the problem has already been solved; one paragraph that simply states, “We will prove the following results”; not put into a clear context; the background is not explained very well

Page 33

Well written; provides motivation for the research; presents a clear, mathematically well-formulated statement of the problem; tells why the problem is important and how it fits into the world; places the problem in a historical context and connects it with the literature/previous research; indicates how the student will solve it; provides a synopsis of the main results and a road map for the rest of the dissertation

Unacceptable

2:20 PM

Very Good

12/8/08

Outstanding

Discussion of the Literature Outstanding Shows an understanding of the history and context of the problem and how this work is related to existing results and how it is different from them; uses original sources; presents the literature coherently and quotes sources correctly; shows that the student has very thoroughly digested and internalized a lot of mathematical work; indicates how it fits in the greater world of mathematics

Very Good The problem is placed in context but less clearly

Acceptable

Unacceptable

Places the problem in context; provides some discussion of the literature; mentions important things

Inadequate coverage of the literature; misses a lot of very relevant things; does not harmonize the conventions used in different articles

Statement of Results/Theorems Outstanding Provides a clear and correct statement of results; self-contained and easy to understand; surprising and of great interest

Very Good Shows student has clearly made progress; interesting but not surprising

17211-DevelQualDiss-Sciences

Introduction/Problem Statement

APPENDIX A

Table A.4 Tasks of the Mathematics Dissertation by Quality Level

Acceptable Statement is correct but not that good; student can speak the language of mathematics; not terribly interesting; has some minor discrepancies; terms are not well defined and may cause some confusion

Unacceptable Incorrect, something is mathematically wrong; the notation is so bad that it is impossible to understand

33

(continued)

Approach to the Problem (Techniques) Outstanding

Acceptable

Stylistic, not just formal; clear on what methods were used

Uses well-worn techniques; presented in a very complicated way

Unacceptable Unclear; the approach is wrong or so messy one cannot tell whether it is right or wrong

12/8/08

Original, insightful, clear, clean, unique, innovative, brilliant, beautiful, and unquestionable; uses a completely new group of ideas; synthesizes two totally dissimilar things; improves prior work; useful elsewhere

Very Good

17211-DevelQualDiss-Sciences

34

Table A.4

Very Good

Acceptable

Well written; original; has new ideas; short, slick, streamlined, precise, correct, elegant; difficult but easy to follow; unexpected— uses a theorem from another area; beautiful—as simple as can be; shows exactly how assumptions are used; does not make any unnecessary assumptions; bridges between or synthesizes areas

Clear, correct; skillful use of sophisticated techniques; proved rigorously and mathematically; notations are clear

Has some original thinking, some new ideas, but the innovation is confusing; contains extraneous material; does not get to the heart of the matter clearly; long, boring, routine; needs work

Unacceptable No proof, or the proof is not shown; the logical deductions are not correct; contains unnecessary assumptions; is difficult to follow

Page 34

Outstanding

2:20 PM

Proof of Results

Conclusion/Future Directions Outstanding

Very Good

Student really thinks about the problem in a new way; convincing; main result has applications and implications; makes conjectures, discusses future directions

Solid summary of results; discusses limitations, obstacles to overcome; points to new problems; specifies some possible future directions or implications

Acceptable States that the theorems are true; makes some conjectures

Unacceptable Not discussed

APPENDIX A

17211-DevelQualDiss-Sciences

12/8/08

2:20 PM

Page 35

Appendix B Advice for Writing a Science, Engineering, or Mathematics Dissertation Table B

Writing the Science, Engineering, or Mathematics Dissertation

Introduction • • • • •

Think of your introduction as an extended abstract or executive summary. State your question or problem clearly and concisely. Discuss the importance of your problem. Put your problem in context. Discuss your setup (experimental dissertations), how your results were obtained, and how broadly applicable they are. • Identify your contribution and its significance. • Provide an overview of your entire dissertation. • Review and rewrite your introduction as necessary after writing your conclusion, weeding out extraneous material. Literature Review • • • • • • • • • • • •

Model your literature review on a good review article. Be selective, critical, and comprehensive in your review of the literature. Include and cite the most important and relevant works. Bring in relevant literature from other fields and disciplines. Connect the literature to your problem. Provide a context for your problem by summarizing and synthesizing the history and controversies that surround your problem. Organize your literature review around themes. Present an analytic and synthetic discussion of the literature. Include your own insights on the literature. Use the literature to show what is missing in the literature and why your study needs to be done. Build a story that leads to your hypotheses, if you have them. Discuss how your study will advance the field.

Theory • • • •

Link your theory to your research question(s), methods, and observations. Select research methods that are appropriate for your theory. Understand and be able to explain the theory(ies) you are working with or developing. Use the theory(ies) appropriately.

35

17211-DevelQualDiss-Sciences

12/8/08

2:20 PM

Page 36

APPENDIX B

36 Methods • • • • •

Use the best, most appropriate cutting-edge methods or techniques for your problem. Align and integrate your methods with your theory and research question(s). Where possible and appropriate, use multiple methods to research your problem. Think of your methods section(s) as an instruction manual for a beginning graduate student. Provide enough detail so that your experiments can be understood and replicated (experimental dissertations). • Provide a justification for each method or technique you use. • Discuss the pros and cons, strengths and weaknesses of your methods. Results and Analysis • • • • • • • •

Be thorough and correct. Ask penetrating questions of your data. Engage in supplementary analyses. Explore questions raised by your analyses. Look for and explain complex patterns in your data. Provide a coherent introduction to and summary of your results. Provide enough detail so readers can understand intermediate steps. Provide plausible explanations and interpretations of your results.

Engineering • Explore your design space and exercise your theory. • Challenge your own assumptions. • Show where your assumptions hold up and where they do not. Mathematics • Provide a clear, correct, and coherent statement of your results. • Simplify your proof. • Demonstrate that your idea is original. Discussion and Conclusion • • • • • • • • • •

Take a deep breath and find the time and energy to write a solid conclusion. Connect your conclusion to your introduction. Tie all the pieces of your dissertation together. Highlight the major points and findings of your work. Draw independent conclusions about your research. Discuss the significance and implications of your work. Place and discuss your conclusion in the context of the larger perspective of the discipline. Identify the shortcomings of your research. Anticipate and respond to criticism. Identify new questions or next steps.

17211-DevelQualDiss-Sciences

12/8/08

2:20 PM

Page 37

APPENDIX B

37 DOS

DON’TS The Dissertation as a Whole

• • • • • • •

• • • • • •

Master your discipline’s subject matter. Learn and internalize the professional standards of your discipline. Come up with a clearly defined researchable question or problem whose answer/results will be important to your field. Know the literature in your field and in related fields, especially as it relates to your dissertation topic. Take the initiative to explore new ideas and new literatures. Look at other fields and other disciplines for ideas. Do what you say you are going to do in your dissertation proposal or something very close to it, but remember, ideas evolve and so should your research. Use proper research methods appropriately. Take the time necessary to fully develop your ideas. Make the average graduate student in your discipline the target audience for your writing. Present your ideas clearly, concisely, and persuasively. Anticipate and answer readers’ questions and criticisms. Make the revisions your advisor and committee members ask you to make or provide them with good justifications for not making them.

• •

• • • •

Think of your dissertation as the biggest or best thing you will ever do in your life. Set an unreasonable quality level for your dissertation. Outstanding dissertations are rare. Faculty expect most dissertations to be very good. You can always expand and polish your work after you receive your degree. Expect your advisor to do your research or write your dissertation for you. Submit a manuscript containing spelling and grammatical errors Misinterpret or oversell your results. Push for a defense if your advisor does not feel you are ready to defend.

Introduction • • • • • • • •

Think of your introduction as an extended abstract or executive summary. State your question or problem clearly and concisely. Discuss the importance of your problem. Put your problem in context. Discuss your set up (experimental dissertations), how your results were obtained, and how broadly applicable they are. Identify your contribution and its significance. Provide an overview of your entire dissertation. Review and rewrite your introduction as necessary after writing your conclusion.

•

Put in a lot of extraneous material.

Literature Review • • • • • •

• • • • • •

Model your literature review on a good review article. Be selective, critical, and comprehensive in your review of the literature. Include and cite the most important and relevant works. Bring in relevant literature from other fields and disciplines. Connect the literature to your problem. Provide a context for your problem by summarizing and synthesizing the history and controversies that surround your problem. Organize your literature review around themes. Present an analytic and synthetic discussion of the literature. Include your own insights on the literature. Use the literature to show what is missing in the literature and why your study needs to be done. Build a story that leads to your hypotheses, if you have them. Discuss how your study will advance the field.

• • • •

• •

Discuss everything that has been written on your topic. Overlook relevant parts of the literature. Cite literature that is out of date. Cite only books or review articles, go back to the original papers. Take the literature at face value. Present a descriptive summary of the literature devoid of critical analysis of it. Cite papers you have not read or do not understand. Plagiarize or misuse your sources.

• • • •

Omit theory. Use the wrong theory(ies). Use the theory(ies) or equations inappropriately. Build theory when there is no need for theory.

• •

Theory • • • •

Link your theory to your research question(s), methods, and observations. Select research methods that are appropriate for your theory. Understand and be able to explain the theory(ies) you are working with or developing. Use the theory(ies) appropriately.

17211-DevelQualDiss-Sciences

12/8/08

2:20 PM

Page 38

APPENDIX B

38 DOS

DON’TS Methods

• • • • • • •

Use the best, most appropriate cutting-edge methods or techniques for your problem. Align and integrate your methods with your theory and research question(s). Where possible and appropriate, use multiple methods to research your problem. Think of your methods section(s) as an instruction manual for a beginning graduate student. Provide enough detail so that your experiments can be understood and replicated (experimental dissertations). Provide a justification for each method or technique you use. Discuss the pros and cons, strengths and weaknesses of your methods.

• • •

Use the wrong method(s) for your problem. Use methods inappropriately. Omit necessary details.

Results and Analysis • • • • • • • •

Be thorough and correct. Ask penetrating questions of your data. Engage in supplementary analyses. Explore questions raised by your analyses. Look for and explain complex patterns in your data. Provide a coherent introduction to and summary of your results. Provide enough detail so readers can understand intermediate steps. Provide plausible explanations and interpretations of your results.

• • • •

Falsify, fudge, fabricate, or hide data. Present figures and tables without discussing them. Make improper inferences Overstate your results.

Engineering • • •

Explore your design space and exercise your theory. Challenge your own assumptions. Show where your assumptions hold up and where they do not.

Mathematics • • •

Provide a clear, correct, and coherent statement of your results. Simplify your proof. Demonstrate that your idea is original.

Discussion and Conclusion • • • • • • • • • •

Take a deep breath and find the time and energy to write a solid conclusion. Connect your conclusion to your introduction. Tie all the pieces of your dissertation together. Highlight the major points and findings of your work. Draw independent conclusions about your research. Discuss the significance and implications of your work. Place and discuss your conclusion in the context of the larger perspective of the discipline. Identify the shortcomings of your research. Anticipate and respond to criticism. Identify new questions or next steps.

• • •

Omit the conclusion. Blow it off and write an insufficient conclusion. Repeat your introduction.

17211-DevelQualDiss-Sciences

12/8/08

2:20 PM

Page 39

Making the Implicit Explicit: About the Study

is based in large part on a study conducted during 2003–2004 at nine doctoralgranting universities. The study included focus groups with 276 faculty in 74 departments across 10 disciplines representing the sciences (biology, electrical engineering and computer engineering, physics/physics and astronomy, mathematics), social sciences (economics, psychology, sociology), and humanities (English, history, philosophy). The faculty were selected because they had advised many doctoral students and served on many dissertation committees. The average faculty member chaired 13 dissertations and served on 36 dissertation committees. Faculty from the following universities participated in the study:

T

• • • • • • • • •

The faculty were asked to discuss dissertations in their disciplines, and, specifically, to describe dissertations and the tasks of dissertations at four different quality levels: outstanding, very good, acceptable, and unacceptable. They were also asked to explain the purpose of the dissertation and what it means to make an original and significant contribution in their discipline. At the time of the focus groups, as a group, the 101 faculty whose responses are featured in this booklet had chaired about 1,155 dissertations and served on about 3,755 dissertation committees, individually chairing, on average, 11 dissertations and serving on 37 dissertation committees. The biology faculty represented such diverse fields as anatomical sciences, botany, cell and molecular biology, developmental biology, ecology, evolution, genetics, population biology, and systematics. The engineering faculty represented computer and electrical engineering. Theoretical and experimental physics was represented by faculty from physics and astronomy departments. The mathematics faculty represented pure and applied mathematics. For a full analysis of their discussions, see chapters 7, 8, 9, and 10 of Barbara E. Lovitts., Making the Implicit Explicit: Creating Performance Expectations for the Dissertation (Sterling, VA: Stylus Publishing, 2007).

HIS BOOKLET

Duke University Michigan State University Northwestern University Stony Brook University Syracuse University University of Colorado at Boulder University of Illinois at Urbana-Champaign University of Kansas University of Southern California

39

17211-DevelQualDiss-Sciences

12/8/08

2:20 PM

Page 40

About the Authors

Barbara E. Lovitts is an independent higher education researcher. She was formerly Senior Program Officer in the Center for the Advancement of Scholarship on Engineering Education at the National Academy of Engineering, and is the author of Leaving the Ivory Tower: The Causes and Consequences of Departure from Doctoral Study. She has worked at the University of Maryland, the American Institutes for Research, the National Science Foundation, and the American Association for the Advancement of Science. Ellen L. Wert, a former program officer at The Pew Charitable Trusts, is a freelance writer and editor who has been involved for nearly two decades with national efforts to improve U.S. graduate education, including Preparing Future Faculty, the Carnegie Initiative on the Doctorate, and the Survey on Doctoral Education and Career Preparation. Past clients include the American Association for Higher Education, The Brookings Institution, the Carnegie Foundation for the Advancement of Teaching, and the Education Policy Institute.

40