The AssessmenT CenTer hAndbook for PoliCe And fire Personnel
AboUT The AUThor Charles d. hale is the President of Resource Management Associates, a multifaceted firm he created in 1981 to provide technical assistance and consulting services to municipal governments. Mr. Hale began his career in the public sector as a police officer in El Segundo, California, in 1965, where he served for seven years as a patrol officer, investigator, and field supervisor. While working full time as a police officer, he attended California State University at Long Beach where he attained his Bachelor’s Degree in 1970 and his Master’s Degree in 1972. From 1972 to 1974, Mr. Hale served as the Assistant Director of the Police Assaults Study conducted by the Oklahoma University Research Institute and funded by the Law Enforcement Assistance Administration. In 1973, Mr. Hale served on a team of consultants involved in a management and organization study of the Police Department in Amarillo, Texas. Since that time, Mr. Hale has conducted similar studies in more than 200 police departments in several states. He is the author of numerous publications in the field of law enforcement, including Police Patrol Operations and Management, which is currently in the third edition and is published by Prentice-Hall. As President of Resource Management Associates, Mr. Hale directs a small staff of personnel engaged in conducting management studies of police departments and designing and administering written promotional examinations, oral examinations and assessment centers for police and fire departments. Mr. Hale has personally designed and administered over 300 assessment centers for dozens of different positions in the police and fire services. His company has clients in more than 25 states, including Alabama, Arizona, Colorado, Connecticut, Florida, Georgia, Illinois, Indiana, Iowa, Kentucky, Maine, Massachusetts, Maryland, Michigan, Minnesota, Mississippi, Missouri, Nebraska, New Hampshire, Ohio, Pennsylvania, Rhode Island, South Carolina, Tennessee, Vermont, Virginia, Wisconsin, and Wyoming.
Third Edition
THE ASSESSMENT CENTER HANDBOOK FOR POLICE AND FIRE PERSONNEL By
CHARLES D. HALE President Resource Management Associates Tinley Park, Illinois
Published and Distributed Throughout the World by CHARLES C THOMAS • PUBLISHER, LTD. 2600 South First Street Springfield, Illinois 62794-9265
This book is protected by copyright. No part of it may be reproduced in any manner without written permission from the publisher. All rights reserved.
©2010 by CHARLES C THOMAS • PUBLISHER, LTD. ISBN 978-0-398-07948-2 (spiral) Library of Congress Catalog Card Number: 2010012351 With THOMAS BOOKS careful attention is given to all details of manufacturing and design. It is the Publisher’s desire to present books that are satisfactory as to their physical qualities and artistic possibilities and appropriate for their particular use. THOMAS BOOKS will be true to those laws of quality that assure a good name and good will.
Printed in the United States of America TS-R-3
Library of Congress Cataloging in Publication Data Hale, Charles D. The assessment center handbook for police and fire personnel / by Charles D. Hale.--3rd ed. p. cm. Includes index. ISBN 978-0-398-07948-2 (Spiral) 1. Assessment centers (Personnel management procedure)--Handbooks, manuals, etc. 2. Police--Personnel management--Handbooks, manuals, etc. 3. Fire departments--Personnel management--Handbooks, manuals, etc. I. Title. HF5549.5.A78H35 2010 363.2’2--dc22 2010012351
PrefACe To The Third ediTion
I
am gratified this book has helped people who are either interested in learning more about assessment centers or who hope to improve their performance as they face an assessment center. There are other sources of information about assessment centers and all of them have something important to contribute to the growing body of knowledge about this important selection method. In preparing the third edition I have attempted to expand upon the concepts and principles presented in the first two editions and have introduced a few new ideas. Several chapters have been extensively edited and reformatted. In addition, in Chapter 4, I have added sections of the Tactical EMS Problem as well as the Public Education Exercise, both of which have special applicability to emergency medical and fire prevention personnel. I have also included a new Chapter 9, entitled “Best Practices in Assessment Center Exercises,” which provides a number of examples on how candidates may achieve superior performance in many different kinds of assessment center exercises. I have included only a few of the best examples I have found and I know there are many others of which I am not aware. I have addressed the use of modern technology in a new section in Chapter 5 and have attempted to reinforce the notion that assessment center developers and administrators need to be continually alert for opportunities to improve their policies and incorporate advancing technology into their work products. I am confident that we will see new developments in this arena in the years ahead and I hope to be able to update this book accordingly. Regardless of how hard we work to design and administer successful assessment centers, our hard work can be easily unraveled by the thoughtless actions of a careless, lazy, or unprofessional assessor. I have been privileged to work with a great many thoughtful, dedicated and hard-working men and women who have served as assessors in the several hundred assessment centers I have administered over the last thirty years. Selecting good assessors does not happen by accident nor do they work effectively without sound principles of supervision and administration. I have provided some additional insight into the selection, preparation, and supervision of assessors in Chapter 6. v
vi
The Assessment Center Handbook for Police and Fire Personnel
The proper ways of evaluating the results of an assessment center, scoring candidates, and reporting the results of the process need not be a mystery, but it sometimes can be. The best practice, I believe, is to make the scoring process as transparent and uncomplicated as possible. Most candidates taking part in an assessment center are not mathematicians nor are they master statisticians and therefore prefer to receive the results of their efforts in the most practical, simple, and easy-to-understand format. I have expanded my discussion of candidate scoring in Chapter 7 as a means of addressing this issue. We learn by our mistakes and I continually preach to participants that they will learn more about their own strengths and weaknesses by the mistakes they make in the assessment center. In the second edition of this book, I included a new chapter on “Some of the Greatest Blunders,” and I continue to think that it is one of the more useful chapters of the book for the first-time participant. I have added some new material for this section, once again drawing upon my own experience as an assessment center administrator. I have no doubt that the future will provide additional examples for this section as well. I continue to believe that part of the success of an assessment center is the fact that candidates believe that they are a fair, realistic, and practical way for them to demonstrate their ability to perform the tasks of a position for which they are being evaluated. When they stop believing this, the process is doomed to failure. This is one reason that I have tried, as an assessment center administrator, to do whatever can reasonably be done to ensure that candidates feel satisfied with the results of the process. This belief is discussed in Chapter 11 where I have provided some additional thoughts to reinforce this view. Assessment centers continue to be widely used as the preferred and accepted manner of evaluating the potential of members of police and fire departments as a part of their standardized promotional examination process. As I suggested in the Preface to the previous edition, the art and practice of designing job-related and reliable assessment centers is a work in progress. As we continue to ply our trade, we will continue to advance the theory and success of the process. I hope this third edition will play some part in that advancement.
ACknoWledGmenTs
I
n previous editions I have acknowledged that this book is the result of a collective effort involving numerous people and organizations. This statement is no less true today. However, it is virtually impossible for me to list the many men and women with whom I have worked over the last 30 years who have contributed in one way or another to making this book a reality. Nevertheless, I owe them all a great debt of gratitude for their understanding, support and guidance. I am also indebted to the many police and fire chiefs, city managers, personnel directors, fire and police commissioners, civil service examiners, and others who have been gracious enough to allow me to assist them in designing and administering assessment centers for their agencies. I appreciate the trust they have placed in me and I hope that I have not disappointed them. I would also like to acknowledge the contributions of the hundreds of men and women who have participated in assessment centers under my direction. They have offered many comments and suggestions which, over the years, have helped us to improve the quality of our work. They deserve a great deal of credit for having the fortitude to go through the process which is indeed exhausting and challenging and I sincerely hope that they consider the experience a satisfying and rewarding one. Finally, I would like to acknowledge with the greatest respect and fond memory of several of my colleagues who have passed on to their eternal reward but who remain among my most treasured friends: Mr. Gary Konzak, former Chief of Police in Carol Stream, Illinois and Grand Junction, Colorado; Mr. Robert Sauer, retired Chief of Police of Hanover Park, Illinois; Mr. Pete Sanders, retired Chief of Police of the Riverdale, Illinois, Police Department; Mr. Ed Archer, retired District Chief of the Rockford, Illinois, Fire Department; Mr. William Sudbury, former Chief of Police of the Munster, Indiana, Police Department; Mr. Bennie Crane, retired District Chief of the Chicago Fire Department; and Mr. Robert Baird, retired Chief of Police of the Elgin, Illinois, PoliceDepartment. These gentlemen, professionals all, have served their respective profession well and nobly and I am honored to have served with them. vii
ConTenTs Page Preface to the Third Edition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . v Chapter 1. What Is an Assessment Center? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 2. Advantages of the Assessment Center . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 3. Why Some People Don’t Like Assessment Centers . . . . . . . . . . . . . . . . . 14 4. Typical Assessment Center Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 The In-Basket Exercise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 The Group Problem-Solving Exercise . . . . . . . . . . . . . . . . . . . . . . . . . 25 The Tactical Exercise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 The Tactical Fire Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 The Tactical EMS Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .32 The Tactical Police Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 The Employee Meeting Exercise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 The Citizen Interview Exercise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 The Fire Inspection Exercise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44 The Shift or Company Meeting Exercise . . . . . . . . . . . . . . . . . . . . . . . 46 The Shift or Company Training Exercise . . . . . . . . . . . . . . . . . . . . . . . 48 The Community Meeting or Presentation Exercise . . . . . . . . . . . . . . . 50 The Public Education Exercise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .53 The Staff Meeting Exercise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54 The News Media Exercise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56 The Program Development Exercise . . . . . . . . . . . . . . . . . . . . . . . . . . 59 The Criminal Investigation Exercise . . . . . . . . . . . . . . . . . . . . . . . . . . . 61 The Career Interview Exercise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63 The Accident Review Board Exercise . . . . . . . . . . . . . . . . . . . . . . . . . . 65 ix
x
The Assessment Center Handbook for Police and Fire Personnel 5. Assessment Center Design and Administration . . . . . . . . . . . . . . . . . . . . 67 The Importance of Good Planning . . . . . . . . . . . . . . . . . . . . . . . . . . . . .67 Exercise Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68 Candidate Orientation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71 Facilities and Equipment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73 Scheduling Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75 Maintaining Test Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77 Staff Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78 Use of Modern Technology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .80 Fire Simulators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .80 The Electonic In-Basket . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .81 The In-Car Camera . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .82 Using Videotape or DVDs to Simulate Reality . . . . . . . . . . . . . . . . . . . .84 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .84 6. Selection and Training of Assessors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85 Assessor Selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85 Assessor Training . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87 The Assessor Manual . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .88 Assessor Orientation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .88 Assessor Training . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .89 Care and Nurturing of Assessors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .90 Rules of Conduct for Assessors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .91 7. Candidate Evaluation, Scoring, and Feedback . . . . . . . . . . . . . . . . . . . . .92 Assessor Note-Taking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .92 Weighting of Exercise Components . . . . . . . . . . . . . . . . . . . . . . . . . . .93 Maintaining Consistency . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .94 Selecting the Evaluation Criteria . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .95 Performance Dimensions and Benchmarks . . . . . . . . . . . . . . . . . . . . .97 The Rating Scale . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .100 The Numerical Scale . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .100 The Raw Score Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .101 Reporting Candidate Scores . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .102 Assessor Commentary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .104 Providing Feedback to the Candidates . . . . . . . . . . . . . . . . . . . . . . . .105 8. Preparing for an Assessment Center . . . . . . . . . . . . . . . . . . . . . . . . . . . .108
Contents
xi
9. Best Practices in Assessment Center Exercises . . . . . . . . . . . . . . . . . . .115 In-Basket Exercise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .115 The Tactical Fire/EMS Problem Exercise . . . . . . . . . . . . . . . . . . . . . . .118 The Group Problem-Solving Exercise . . . . . . . . . . . . . . . . . . . . . . . . .121 The Employee Meeting Exercise . . . . . . . . . . . . . . . . . . . . . . . . . . . . .123 The Community Meeting Exercise . . . . . . . . . . . . . . . . . . . . . . . . . . . .125 The Citizen Interview Exercise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .126 The Company or Shift Training Exercise . . . . . . . . . . . . . . . . . . . . . . .127 10. Some of the Greatest Blunders . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .130 The Orientation Session . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .131 The In-Basket Exercise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .132 The Employee Meeting Exercise . . . . . . . . . . . . . . . . . . . . . . . . . . . . .133 The Tactical Fire Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .137 The Community Meeting Exercise . . . . . . . . . . . . . . . . . . . . . . . . . . . .138 The Program Development Exercise . . . . . . . . . . . . . . . . . . . . . . . . . .140 The Citizen Interview Exercise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .141 The Staff Meeting Exercise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .142 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .144 11. Ensuring Candidate Satisfaction with Assessment Center Methods . . .145 Candidate Orientation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .146 Customizing the Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .147 Keeping the Exercises Realistic and Relevant . . . . . . . . . . . . . . . . . .148 Selection and Qualifications of Assessors . . . . . . . . . . . . . . . . . . . . . .148 Obtaining Feedback from Candidates . . . . . . . . . . . . . . . . . . . . . . . . .149 Candidate Debriefing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .151 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .153 12. Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .154
Figures: 2.1 5.1 5.2 7.1 7.2 7.3
How Candidates View an Assessment Center . . . . . . . . . . . . . . . .10 Relationship Between Performance Dimensions And Job Tasks . .70 List of Facilities and Equipment . . . . . . . . . . . . . . . . . . . . . . . . . . .75 Example of Calculating Weighted Scores . . . . . . . . . . . . . . . . . . .102 Example of a Detailed Breakdown of an Assessment Center Score .103 Sample Group Comparison of In-Basket Scores . . . . . . . . . . . . .104
xii
The Assessment Center Handbook for Police and Fire Personnel Appendices: A. B. C. D. E. F. G. H. I. J. K. L. M. N. O. P. Q.
R. S. T.
Assessment Center Guidelines . . . . . . . . . . . . . . . . . . . . . . . . . . . 159 Typical In-Basket Items . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179 Guidelines for Scoring the In-Basket Exercise . . . . . . . . . . . . . . . 182 Group Problem-Solving Exercise . . . . . . . . . . . . . . . . . . . . . . . . . 184 Tactical Fire Problem Exercise . . . . . . . . . . . . . . . . . . . . . . . . . . . 186 Tactical Fire Problem Scenario . . . . . . . . . . . . . . . . . . . . . . . . . . . 189 Fire Tactical Problem: Evaluation of Tactical Considerations for Tactical Fire Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192 Employee Meeting Exercise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194 Citizen Interview Exercise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195 Role-Playing Instructions for the Citizen Interview Exercise . . . . . 196 Fire Inspection Exercise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 198 Shift Meeting Exercise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 200 Company Training Exercise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202 Community Meeting Exercise . . . . . . . . . . . . . . . . . . . . . . . . . . . 204 Staff Meeting Exercise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 206 Program Development Exercise (Police Lieutenant) . . . . . . . . . . 207 Room Configurations for Various Exercises . . . . . . . . . . . . . . . . . 209 Q-1 Group Problem-Solving . . . . . . . . . . . . . . . . . . . . . . . . . 209 Q-2 Shift or Company Training Exercise . . . . . . . . . . . . . . . 210 Q-3 Employee or Citizen Interview Exercise . . . . . . . . . . . . 210 Q-4 Police or Fire Tactical Exercise . . . . . . . . . . . . . . . . . . . 211 Typical Assessment Center Schedules . . . . . . . . . . . . . . . . . . . . 212 Candidate Feedback Questionnaire . . . . . . . . . . . . . . . . . . . . . . 214 Candidate Feedback Forms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 218
Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221
The AssessmenT CenTer hAndbook for PoliCe And fire Personnel
Chapter 1 WhAT is An AssessmenT CenTer?
A
n assessment center is not a place, but rather a process that is used in both private industry and in governmental agencies for the purpose of determining those persons who have the ability or potential to assume higher levels of supervisory, managerial, and administrative ability.
What’s this I hear about assessment centers?
Assessment centers are traditionally viewed as a part of a selection process, whereby candidates are evaluated on their ability to perform a particular job for which they are applying, but an assessment center can also serve other purposes as well. For example, assessment centers can be used to assist individuals in learning more about their strengths and weaknesses so that they can better prepare themselves to achieve whatever career goals they may have set for themselves. Carefully-designed assessment centers 3
4
The Assessment Center Handbook for Police and Fire Personnel
can also be used to evaluate deficiencies in department operations, management practices, and training programs. For example, the results of an assessment center may indicate the need for additional training in public speaking, cultural diversity awareness, or human relations. By being tested in a “real world” environment, candidates learn a great deal about their own strengths and weaknesses. Participating in an assessment center helps to prepare a candidate for the challenges that he or she will face in the position for which he or she is being evaluated. Assessment centers create a learning environment for participants. By taking part in an assessment center, participants can better understand the direction their organization is going and prepare for the “journey.”1 An assessment center can also be used to pinpoint problems with organizational performance and policy and procedure development. For example, the police or fire tactical problem exercise (described in Chapter 4) may yield tremendously valuable information about a police or fire department’s tactical procedures and how well (or poorly) they are understood by members of the organization. Similarly, a well-designed role playing exercise may reveal deficiencies in how members of the department deal with internal employee issues, external relationships or customer service issues. While these other uses are important, the assessment center is used primarily as a tool for evaluating candidates for promotion or appointment and it is in this context that the material contained in this book is presented. An assessment center consists of an organized, standardized, and comprehensive evaluation of behavior based on multiple inputs. Multiple trained observers and techniques are used. Judgments about behavior are made, in major part, from specifically developed assessment simulations. These judgements are pooled in a meeting among the assessors or by a statistical integration process. In an integration discussion, comprehensive accounts of behavior and often, ratings of it, are pooled. The discussion results in evaluations of the performance of the assessed on the dimensions or other variables which the assessment center is designed to measure. The assessment center process differs from more traditional examination methods in that it is a test of skill and ability rather than knowledge. In other words, it measures not how much someone knows about something but rather how well a person can function in an assigned role or task. Some people are known to be great at memorizing information from textbooks, procedural manuals, and statutes, but they fall short in such critical areas as 1 Patrick Oliver, “The Assessment Center Method: Not Just for Promotions Anymore.” in Subject to Debate: A Newsletter of the Police Executive Research Forum (March/April, 1998), p. 2.
What is an Assessment Center?
5
leadership, human relations, and decision-making skills which are difficult to measure in a written examination. Perhaps the greatest advantage of the assessment center is that it provides a higher degree of reliability and insight into supervisory or management potential than is possible with other examination methods. In addition, experience has shown that candidates feel that the assessment center is much more fair and job-related than other types of examinations. As a result, they are less inclined to challenge the results of an assessment center, even when they do poorly. Assessment centers are sometimes confused with oral interviews, or “oral assessments” as they are sometimes called, but they are quite different and the characteristics of an assessment center are quite distinct. Assessment centers must adhere to certain basic principles that have been developed over time and must be administered under carefully controlled conditions if they are to be considered valid and reliable (see Appendix A). These conditions are:2 1. A job analysis of relevant behaviors must be conducted to determine the dimensions or competencies important to job success in order to identify what should be evaluated by the assessment center. 2. Behaviors displayed by participants must be classified into meaningful and relevant categories such as behavioral dimensions, attributes, characteristics, aptitudes, qualities, skills, abilities, competencies or knowledge. 3. The techniques used in the assessment center must be designed to provide information for evaluating the dimensions previously determined by the job analysis. 4. Multiple assessment techniques must be used. 5. Assessment techniques must include a sufficient number of job-related simulations to allow opportunities to observe the candidate’s behavior related to each dimension/competency being assessed. 6. Multiple assessors must be used to observe and evaluate each assessee. 7. Assessors must receive thorough training and demonstrate performance that meets the guideline in the “Assessor Training” section of this document before participating in an assessment center. 8. A systematic procedure must be used by assessors to record specific behavioral observations accurately at the time of observation.
International Task Force on Assessment Center Guidelines, “Guidelines and Ethical Considerations for Assessment Center Operations,” International Journal of Selection and Assessment, Vol. 17, No. 3 (September, 2009), pp. 244–247.
2
6
The Assessment Center Handbook for Police and Fire Personnel
9. The integration of each individual’s behaviors must be based on pooled information from assessors or through a statistical integration process. The assessment center process has proven itself in more than fifty years of application in both government and in private enterprise. While assessment centers are usually viewed as tools for determining a candidate’s suitability for promotion in rank or assignment to a higher position in the police or fire service, this method is also used by some agencies for evaluating candidates for entry-level positions. No single method can or should be used to evaluate the skills and abilities of persons for new appointment, promotion, or career advancement and none of the many methods that are available is absolutely guaranteed to produce perfect results. Instead, any fire or police department personnel selection or promotion program should be multifaceted and designed to meet the unique needs of the employing agency. No one method will work well in all circumstances and no single method is infallible. Although there are other methods that can (and should) be used in evaluating the supervision, management, and administrative skills of candidates for various positions, there are none that provide the same level of insight into how a person will actually perform if eventually appointed to the position for which he or she is being evaluated. “Research has consistently demonstrated that Assessment Centers successfully predict a variety of important outcomes: Job Performance, Management Potential, Training Performance, Career Development.”3 This is due, in part, to the fact that, in the assessment center method, there is an attempt to place the candidate into situations and scenarios that are very similar to the situations and scenarios that the candidate will actually confront if appointed to the position. An assessment center attempts to capture the essential characteristics of the position for which the person is being considered. While the candidate understands that the exercise is merely a simulation, he or she also understands that every attempt has been made to make the simulation reflect the actual conditions of the position for which he or she is being considered. For example, in an In-Basket Exercise, the organization of the police or fire department used in the exercise will normally reflect or be similar to the actual organization of the police or fire department in which the candidate works or where the position is located for which the person is applying. Similarly, in a Community Meeting Exercise, the candidate will normally be asked questions by the role players based upon actual conditions in the community or neighborhood that is being represented in that exercise. Michael D. Blair, “Best Practices in Assessment Centers: Reducing Group Differences to a Phrase for the Past,” paper presented at the 27th annual IPMAAC Conference on Personnel Assessment, Baltimore, Maryland, June, 2003. 3
What is an Assessment Center?
7
Because candidates understand that the exercises which they will experience are intended to reflect reality, they are more likely to appreciate the fact that their reactions to these situations should also reflect reality. They must understand that they are expected to deal with each of the scenarios presented to them in the same manner they would if, in fact, the scenario were an actual situation. In truth, candidates rarely need to be told this at all. Experience has shown that candidates become so involved in the emotion of the situation that they often forget that they are role playing and do, in fact, react exactly the way they would in a real-life situation. This becomes quite important in the evaluation of candidates if it becomes clear to the assessors that a candidate may lose confidence under pressure or react belligerently when encountering opposition or hostility. One of the most unique characteristics of assessment centers is how quickly and accurately assessors are able to “size up” a candidate. In a relatively brief period, assessors are able to learn as much about a candidate’s personality and behavior as someone who has worked with them for years. I sometimes test this theory by asking assessors, after the candidates have been scored, to give the chief executive officer of the agency a brief thumbnail sketch of each of the candidates. They might respond, for example by saying that “candidate B has a lot of potential but lacks self confidence.” They might describe another candidate as very intelligent but inflexible and officious. Still another candidate might be described as “being able to talk a good game but lacking in substance.” In nine cases out of ten, the chief executive officer of that agency will know exactly who the assessors are describing! For me, this simply validates the process because it demonstrates that assessors are able to make accurate judgments of candidates in a relatively short period of time. It also helps to nullify the criticism we sometimes hear that “the assessors can’t really get to know me that well in such a short time” (see Chapter 3). The fact is, they can and they do!
Chapter 2 AdVAnTAGes of The AssessmenT CenTer
T
he assessment center method offers a number of advantages over more traditional selection methods, such as supervisory and peer evaluations, career aptitude tests, paper and pencil tests, and oral interview boards. First, assessment centers have been found to be much more reliable in evaluating supervisory, managerial and administrative ability than more traditional selection methods. Written examinations, for example, test a person’s knowledge (or, more often, his or her ability to memorize written material), but cannot indicate how well that person will be able to actually apply that knowledge. We all know someone who may be a good “test taker” but fails to function effectively in the position to which he or she was appointed. Oral interviews, on the other hand, are heavily influenced by how well the interviewers like what the person has to say, his or her appearance, and the nature of the questions being asked. Oral interviews also dwell on hypothetical issues and cannot actually place the candidate in the actual role for which he or she is being considered. Assessment centers, on the other hand, examine the person’s ability to actually perform the job by creating job simulations which closely parallel the actual job situation. In addition, the assessment center is flexible in that it can be adapted to a variety of situations and uses and has been successfully employed to evaluate persons for both entry-level positions as well as promotions. Assessment centers have been used to select police chiefs, fire chiefs, city managers, and office managers. Assessment centers can be adapted to evaluating candidates for an infinite number of positions in both the public and the private sectors. Importantly, the assessment center method has been shown to be a highly reliable method of evaluating candidates for a particular position. Practical experience has demonstrated that there is a very high probability that a person who performs well in an assessment center will eventually go on to perform well in the position for which he or she is being evaluated. 8
Advantages of the Assessment Center
9
There are obvious exceptions. While an assessment center can demonstrate that a person has the aptitude to capably perform a particular task, it cannot always determine that a person will be the “best fit” for the job. Other factors, such as attitude, emotional maturity, and personality characteristics, may need to be factored into the selection decision. Sometimes the “most qualified” person is not the “best” person for the job. In addition, successful performance in an assessment center is not a guarantee that a person will perform well if appointed to the position. Job performance is strongly linked to attitude, motivation, and other characteristics that may affect how well a person will perform in a position, regardless of his or her ability. The assessment center can accurately measure the capacity of a person to do the job for which he or she is being considered, but other factors will ultimately determine whether that person will work up to his or her full potential. Does this mean that no other selection devices have value or purpose? Clearly, this is not the case. Good selection decisions were made long before anyone ever thought of using assessment centers, and no one would suggest that no other selection devices should be considered. There is clearly room for more than one approach to making a selection decision, and other selection devices have use in many situations. For example, a written test is a good way to test a person’s knowledge of the job, but is incapable of predicting how well that person can apply the job knowledge in a real-world situation or under pressure. By contrast, an assessment center is a test of ability, not knowledge. In an assessment center, candidates are evaluated under conditions that closely approximate the actual conditions of the job, and experience has demonstrated that the way a candidate reacts in a simulated condition is often indicative of the way he or she will react in an actual situation. Another advantage of the assessment center method is the high degree of acceptance usually associated with the process by the candidates themselves. This is particularly important in the case of a promotional examination in a police or fire department. Police officers and firefighters often are skeptical of any promotional process based on anything other than seniority on the job and can be expected to complain that just about any other kind of promotional process is biased, unfair, not job-related, or too subjective. Experience has shown that an assessment center may yield entirely different results. Most candidates who participate in an assessment center describe the process as fair, objective, job-related and a good test of their ability to do the job for which they are being considered. For example, Figure 2.1, shown below, illustrates how one group of candidates viewed the assessment center in which they participated.
10
The Assessment Center Handbook for Police and Fire Personnel 16
14
12
10
8
6
4
2
0 Definitely Yes
Yes
Somewhat
Very Little
No
Figure 2.1. Were the exercises in which you participated a fair test of your ability to perform the duties of the position?
Many municipalities use assessment centers for evaluating candidates for top-level positions, such as police chief and fire chief. While these are usually not promotions in the true sense of the word, it is nevertheless important that candidates have confidence in the fairness of the process. Having gone through an assessment center process, which is typically one in which the candidates are placed under a great deal of pressure, even the unsuccessful candidate can appreciate the fact that the person who is eventually selected for the position was undoubtedly well-suited for the job. The assessment center is more than simply a selection tool. It may also be used as a means of self-evaluation. There are few candidates (usually those who already know it all) who do not gain insight into their own strengths and weaknesses as a result of participating in the assessment center process. Thus, it is a process of learning and self-evaluation by which candidates help themselves grow and develop and to be better prepared for the next selection process. An important part of the assessment process—especially those designed for promotions within the ranks of a police or fire department—is, or ought to be, that of candidate feedback. In some fashion, candidates should be provided with some information that tells them what they did right and what they did wrong and how they can improve their job skills in the future (see “Candidate Debriefing” in Chapter 11). Candidate feedback can be most effective using videotape summaries of the candidates’ performance by the assessors. Rather than providing the candidates with written narratives, which often do not convey the sense of
Advantages of the Assessment Center
11
emotion that is intended in them, it may be preferable to have the assessors record their comments on videotape (or on DVD or similar device) so that the candidates have a chance to hear, eyeball to eyeball, how their performance was rated by the assessors. While a candidate does not always want to hear the cold truth about how poorly he or she acquitted themselves in a particular exercise, they can and should learn by the comments offered by the assessors. If they want to, they will learn a lot by going through the process. Those who are smart enough to know that they do not know it all stand to gain quite a bit by participating in an assessment center. In those communities and agencies where assessment centers have been used over a long period of time for the same positions, it is a gratifying experience to observe the development of the candidates over that span of time. There is no question in my mind that this development has been due, in a large part, to the experience and feedback provided by the assessment center process. Most candidates, if they are honest with themselves, concur with the evaluations provided by the assessors. In debriefing sessions conducted with individual candidates, they are asked to describe how they felt about their performance in the process. Not surprisingly, the candidates’ own evaluation of their performance closely approximate that of the assessors. This helps to reinforce the validity of the process and the level of credibility associated with it in the minds of the candidates themselves. There are those, of course, who, if they do not succeed in any kind of promotional process, will find a way to blame everyone but themselves. They are called the “two percenters” and there is not much anyone can do for them because they are always
Careful note-taking is required.
12
The Assessment Center Handbook for Police and Fire Personnel
right and everyone else is always wrong. While the assessment center has been sometimes cloaked in mystery, and is viewed by some as actually a form of psychoanalysis, the assessment center method need not be either complicated or technically cumbersome. While many of the designers and administrators of assessment centers have backgrounds in psychology and psychometrics, the methods used in evaluating candidates and developing job-related and practical exercises require common sense, a working knowledge of the position for which candidates are being considered, and a healthy dose of fairness and objectivity by the assessors. Assessors need to be practical observers with a thorough knowledge of what is required to fill a particular position. After it is all said and done, the assessors are expected to answer this question: Given what you have observed in this scenario, what can you tell us about his or her ability to perform the particular task being evaluated? Assessors should not be expected to read into the minds of the candidates or to be armchair psychologists. The evaluation process should be straightforward, not technically-complicated or cumbersome. It is important that the candidates feel comfortable about the process in which they are about to participate. The process is difficult and challenging enough without playing mind games with them by keeping them guessing about what is in store for them. Whenever possible, it is a good idea to schedule an orientation session for the candidates a few days before the process begins so that they will have a better understanding of what is expected of them. In the orientation session (see Chapter 5), it may be desirable to discuss with the candidates the actual exercises in which they will participate as well as the manner in which the assessors will rate their performance. It is useful to give the candidates as much insight as possible into the process so that the assessment center will be seen by the candidate as an opportunity to excel rather than as an obstacle placed in the path of their success. While some people are opposed to telling candidates too much before the process begins and believe that they should be expected to “figure it out for themselves,” those who have the ability and can do the job for which they are being tested will emerge in the process. Telling people in advance what to expect is the only fair thing to do and it works to the advantage of all candidates, not just those who have never been through the process before. It is important for the candidates to know that there are no “tricks” to be learned or hidden obstacles to overcome. To the extent possible, candidates should be given insight into all aspects of the process so that they can be
Advantages of the Assessment Center
13
as prepared as possible. The only thing that is gained by keeping candidates confused and in the dark about the process is to reinforce the view that many candidates have that the process is unfair, unrealistic, and cloaked in mystery. This is not an accurate perception, nor is it what we want candidates to think about the process.
Chapter 3 WhY some PeoPle don’T like AssessmenT CenTers1
A
ssessment centers are not without their critics, but most of the common complaints about assessment centers can be easily countered. For example, I have listed below some of the typical complaints you might hear about assessment centers, along with my own counterargument. Complaint number 1. The “wrong” people come out on top and the “good guys” finish last. explanation: Because assessment centers measure future job performance rather than past performance, people too often lose sight of the fact that people who have done well in previous positions may not be capable of doing well in more demanding or complex assignments. Good patrol officers don’t necessarily make good sergeants, and an excellent Deputy Fire Chief may not perform well as the Chief of the Department. Assessment centers focus on skills and abilities rather than on past performance. Complaint number 2. i am not a good “role player” and don’t do well in “make believe”exercises. explanation: Candidates are asked to step into the “role” of the position for which they are being considered (e.g., police lieutenant, fire captain, etc.). They are asked to perform the duties of the position just as they would in real life. What assessors actually see in these situations is the 1 I have borrowed some ideas for this section from an article by Dr. Walter S. Booth entitled “Ten Complaints About Assessment Centers” that appeared in the October, 1997 issue of Law and Order magazine, pp. 87–93.
14
Why Some People Don’t Like Assessment Centers
15
“real” person, acting just the way they really would in “real” life if they were to be promoted. People who complain that they are not good “role players” are really saying that they don’t have the tools to do the job for which they are being tested. Complaint number 3. The assessors don’t get to know the “real me.” explanation: As a matter of fact, the assessors get to know the “real” candidate better than he or she might like. They learn quickly who can make a decision under intense pressure, who can relate well to people, who is able to make a decision even when it may not be the popular thing to do, who can manage their time well, who can inspire others to get things done, who can communicate effectively, both orally and in writing, and who has the ability to plan and organize complex events. In a very short time, assessors are able to make accurate and sometimes uncanny predictions about who is able to do the job and who is not. Ultimately, the “real person” does emerge, usually in vivid detail.
How do they get to know the “real” me?
Complaint number 4. The scenarios are not realistic. explanation: Regardless of how much effort is put into the process, virtually every part of an assessment center is a simulation, meaning that it
16
The Assessment Center Handbook for Police and Fire Personnel
is not real but that it is intended to be as real-like as possible. If the consultant or person designing the assessment center has done his or her job properly, this complaint can be easily countered. For every person who says, after an employee interview exercise, “We don’t have people like that working here,” there are two or three others who say “I have a guy just like that on my shift!” A good job analysis, combined with solid interaction between the assessment center designer and a local Subject Matter Expert, will ensure that the exercises properly reflect local working conditions and organizational policies and procedures (see Chapter 5). Often, the ones who complain about the exercises not being “realistic” are those who are not realistic about their own abilities. Moreover, candidates need to accept the fact that the exercises are not real, but rather simulations, and therefore must depart from reality in some aspects. Complaint number 5. The assessors can’t judge me after seeing me for only an hour. explanation: Actually, if the process is well-designed and the evaluation criteria and rating procedures are properly developed, the assessors are able to get a very good understanding of the candidates’ skills and abilities in a very short time. Each exercise should measure different attributes and collectively allow the assessors to form a very reliable and accurate image of each candidate and to judge them against the standards that have been established for the position for which the candidates are being considered. This assumes, of course, that the assessors are qualified by virtue of training and experience and that they have been properly trained in the assessment center method. Complaint number 6. They are too subjective. explanation: While assessment centers are not as objective as some tests, such as multiple-choice and true-false examinations, they can be highly objective if properly designed and administered. Trained assessors, using consensus rating techniques and working with carefully-designed evaluating procedures (see Chapter 8), can be nearly as accurate in evaluating performance and measuring attributes as any paper-andpencil test. Where subjectivity does exist, it usually works in favor of the candidate, since assessors are inclined to give candidates the benefit of the doubt in nearly all cases. In addition, assessment centers usually are free from the adverse impact against protected classes that affect many paper-and-pencil tests.
Why Some People Don’t Like Assessment Centers
17
Assessors achieve a high degree of reliability in their ratings.
Complaint number 7. They are too expensive. explanation: There is no question that assessment centers are much more expensive than other forms of tests. This is because they are highly labor-intensive due to the time involved in designing and administering them. There is very little that can be done to minimize the costs of an assessment center without sacrificing the integrity and outcome of the process. On the other hand, those who have used assessment centers know that the results are much more satisfactory than other forms of testing and that the costs can be easily justified when balanced against the future of the organization. Ultimately, how much does it cost the police or fire department to promote a mediocre or marginally-qualified sergeant or lieutenant? How much can we afford to invest in the future of the organization? The real question is “How much does it cost the organization to promote or appoint someone to a position for which he or she is not qualified?” Successful selection devices such as assessment centers are a wise investment in the future of the organization. Complaint number 8. i would rather be judged by my own peers than by “outsiders” who don’t know me. explanation: Oddly enough, many people feel just the opposite. Most people recognize that being judged by your own peers or superior officers will almost always be impacted by past allegiances, personal animosity, and biases and prejudices. Outside assessors, on the other hand, are not allowed to know anything about the candidates and are expected to judge them solely based upon what they say and do in the process,
18
The Assessment Center Handbook for Police and Fire Personnel
not upon some preconceived belief or judgement about the candidate. Candidates sometimes feel frustrated because assessors do not get to know the “real me.” In effect, they are saying that they should be credited with what they have accomplished in the past, when in fact they are being judged on what they are capable of doing in the future. In fact, it is usually a good idea to keep any background information about the candidates, including their length of service, educational level and past job performance data, away from the assessors so that they can concentrate on evaluating the candidates strictly on what they see and hear them do in the assessment center.
Assessors work hard to render a fair evaluation of candidates.
Complaint number 9. The assessors will do whatever the Chief tells them to. They all stick together. explanation: Not if the process is properly controlled by the assessment center manager. Indeed, most people involved in administering assessment centers are careful to avoid even the appearance of conflict of interest, or in being overly-friendly with the chief executive of the department. Moreover, most police and fire chiefs are careful to keep a reasonable distance between themselves and the assessment center process and the assessors. If a chief executive officer is committed to the assessment center process, he or she must allow the process to unfold without his or her involvement one way or another. Assessors must be guided by their own perceptions of the candidates, not by their relationship with the chief of the department. This makes the impartiality of the assessors and their ability to observe and evaluate candidates with total objectivity an essential consideration.
Why Some People Don’t Like Assessment Centers
19
Complaint number 10. The role player treated me differently than mike or sally. explanation: Role players should be carefully trained and instructed to treat candidates exactly the same way. A script should be created for each role playing situation so that the role-player will say the same things in the same manner and in the same sequence to each candidate (see Appendix J). What cannot be controlled, however, is the way the candidate reacts to the role player. Each candidate will handle a role-playing scenario differently, and the role player will be expected to react realistically to the behavior of the candidate. Some candidates, who know how to deal effectively with people, will find role-playing situations relatively easy, while others may encounter many problems in the same scenario. This is a reflection of the individual candidate’s ability, not of the process itself. This is one reason that it is a good idea to record all parts of the process on videotape, DVD or other recording media so that, if challenged, the ultimate fairness, objectivity, and consistency of the process can be demonstrated. In summary, assessment centers are not for everyone and will not fit perfectly into the testing program of every police or fire department. They are not a panacea and, while believed by many to be superior to other kinds of testing strategies, may not be right for all agencies. They do, however, offer an attractive and useful alternative to most traditional testing programs.
Chapter 4 TYPiCAl AssessmenT CenTer eXerCises
O
ne of the unique characteristics of the assessment center method is that it can be adapted to meet an infinite variety of needs and can be tailored to a multitude of positions and situations. Imagination and the practical limitations of time and resources are the only constraints on what can be included in an assessment center. The first step in designing an assessment center is to conduct a formal job analysis, wherein the duties of the position are examined and the skills and abilities needed to perform successfully in the position are identified. The job analysis may be conducted on a very informal basis, involving simply a brief interview with selected agency representatives, or it may be much more formal and sophisticated, involving both interviews with position incumbents and the completion of detailed questionnaires by incumbents and their supervisors. If any part of the assessment process is later challenged in court, the job analysis data linking actual job tasks performed by position incumbents and the performance dimensions measured in an assessment center will be vitally important. The more time and effort that goes into the design of the process, the better the chance of its success. At the very least, the exercises selected to be included in the assessment center should correlate with the actual duties of the position. For example, one would not usually include an exercise requiring candidates to develop a program budget for a particular operation unless this was a requirement of the job for which the candidates were being considered. As a practical matter, the In-Basket exercise is not recommended for first-level supervisors in the police or fire service because this Exercise is designed to test a higher level of administrative ability than is usually required for persons in a first-level supervisor’s position. Similarly, it is not recommended that candidates for a Fire Chief position be given a fire problem as a practical exer20
Typical Assessment Center Exercises
21
cise, since the technical fire-fighting skills of the candidates would be less important than their administrative, managerial, and organizational skills. The key point to remember is that there should be a strong relationship between the actual exercises contained in the assessment center and the duties of the position for which the candidate is being evaluated. The remainder of this chapter describes a number of different assessment center exercises and the manner in which they can be used to evaluate candidates for various positions in police and fire departments. The in-bAskeT eXerCise
The In-Basket Exercise is an excellent way to evaluate a candidate’s time management ability, decision-making, problem-solving, and written communication ability. The object of the In-Basket Exercise is to test the candidate’s ability to (1) assign appropriate priorities; (2) delegate to the proper persons; (3) take required and appropriate action; (4) plan ahead; (5) keep necessary personnel informed of events; and (6) communicate effectively. The ability of the candidate to work and to make good decisions under pressure is also an important consideration in this exercise. An example of the type of items obtained in a typical In-Basket is shown in Appendix B. The In-Basket Exercise is often used in an assessment center for highranking positions in the police or fire department or other executive-level positions. The In-Basket Exercise is not recommended for first-level supervisory positions (e.g., police corporal or police sergeant or fire lieutenant or company officer) since the nature of these jobs do not usually lend themselves to the kind of administrative matters that are typically found in an InBasket. For example, a first-level supervisor would not normally deal with executives from other police or fire departments, department heads within the same governmental unit, municipal officials, and the like. Typically, their focus is on their subordinates, rather than on dealing with higher administrative officials. The In-Basket is a timed exercise in which candidates are confronted with a series of written memoranda, letters, telephone messages, and other types of correspondence in which they are expected to take some form of action. They are usually placed in a situation which requires them to be away from the office for several days, thus requiring them to deal with as many matters as possible before they leave. They are usually graded on the number of items that they successfully complete within the time provided. In some cases, items may be assigned higher or lower priority, so that a failure to take action on one item could be more or less critical than a failure to complete another item.
22
The Assessment Center Handbook for Police and Fire Personnel
Good time management is essential in the In-Basket Exercise.
Candidates are usually provided with a personnel roster or organizational chart that lists the key individuals with whom he or she will need to interact in order to successfully complete the items in the In-Basket. In some cases, this may be a fictitious organization, while in others, it may reflect the actual organization of the agency in which the process is being conducted. In all cases, the identities of actual persons are changed. While many of the items in an In-Basket may be “stock” items used time and time again, it is a good idea to change the items from time to time in order to keep the material fresh and current. In addition, it is always recommended that the items fit the actual position for which the person is competing as well as the job for which the candidate is being considered. For example, it would not be appropriate for a candidate for a police chief position to receive a memorandum from the city manager regarding an upcoming budget meeting if the community involved does not have a city manager but instead operates under a strong mayor form of government. To the extent possible, the situations presented in the In-Basket should reflect the actual conditions and characteristics of the agency in which the assessment center is being conducted. It is always recommended that the In-Basket Exercise be reviewed by a Subject Matter Expert before it is given to the candidates to ensure that the items contained in the exercise are relevant and pertinent to the position for which candidates are competing. The Subject Matter Expert may also be asked to provide recommended actions and priorities to be assigned to each of the items in the In-Basket (see Appendix C). This provides a standard against which the candidates’ actions may be judged by the assessors. This further helps to improve the objectivity of the process since assessors are not making arbitrary decisions about how correct an action is, but are rather
Typical Assessment Center Exercises
23
comparing the candidates’ actions against those recommended by the Subject Matter Expert.
A ranking officer of the department should always review draft materials to ensure their relevancy and accuracy. Photograph courtesy of Bartlett Fire Protection District, Bartlett, Illinois.
The Subject Matter Expert is usually a ranking member of the organization in which the assessment center is being conducted. In a police or fire department, this may be the chief or deputy chief or a division commander having direct responsibility over the position for which candidates are being evaluated. Candidates should be provided with writing tools, such as pens, pencils, and writing tablets to use in preparing their responses to the items in the InBasket. In some cases, prepared forms may be used to help insure uniformity of responses and to ease evaluation by the assessors. A blank calendar pad for the month in which the particular situation is taking place may also be included as a way of assisting candidates in planning follow-up meetings and activities. The In-Basket should be a timed exercise. The amount of time allowed will usually depend upon the number of items to be included in the In-Bas-
24
The Assessment Center Handbook for Police and Fire Personnel
ket. This number may vary according to the nature of the position and the complexity of the duties performed. The items should be sufficient in number and difficulty to test the candidates’ ability to deal with a variety of issues and to schedule and coordinate different and conflicting activities. The optimum number of items is between 15 and 20 and between ninety minutes and two hours should be allowed for completing the In-Basket. There should be just enough time to place most candidates under pressure to complete all of the items and some candidates may find themselves running out of time before finishing all items. If a candidate finds that he or she is running out of time, he or she should have completed all of the high-priority items and as many of the medium- and low-priority items as possible. Failing to complete a low-priority item will not count as heavily against a candidate as will failure to complete an item of considerable importance. For example, an item regarding a cancellation or re-scheduling of a meeting the week following the candidate’s return to the office would be less important than an item from a female employee complaining of sexual harassment and demanding to be assigned to a different shift or division. The In-Basket may contain items that are linked together by event or persons involved. It is important that the candidate look for these items so that the action taken on one item can be interrelated with the action on the other item. For example, one item may be a memorandum from an employee who is suddenly announcing his plans to retire and move to a South American country. This may be related to a different item regarding drugs missing from the evidence locker and indicating that the employee now planning on retiring in South America was the last person to sign off on the missing drugs. These two items are probably connected and need to be handled together rather than separately. Some items may be given to candidates at planned intervals during the administration of the exercise to simulate telephone calls or other interruptions. For example, one item may be a telephone call received late in the exercise that informs the candidate that a ranking member of the department has been taken to the hospital with a possible heart attack. This may require the candidate to alter some of the actions already taken if they involve the heart attack victim. The items contained in the In-Basket should be straightforward, practical, and relevant to the position for which candidates are being considered. There is no purpose to be served by trying to purposely confuse the candidate by inserting items in the In-Basket that would not actually be found in the in-basket of an incumbent, or by purposely inserting erroneous data or misinformation as a means of adding to the stress placed on the candidate. The In-Basket is a difficult exercise for most people and it is important that they treat it as a real experience rather than as a contest of wills or a psycho-
Typical Assessment Center Exercises
25
logical game. While the standard in-basket exercise is a paper and pencil test, electronic in-baskets are also available in which candidates receive and prepare responses to items by keyboard rather than in written form (see Chapter 5). If there is a key to success in the In-Basket Exercise, it is to read or scan everything contained in the In-Basket before taking any action. Most candidates like to take a few minutes to quickly look over everything in the InBasket and place them in two or three piles according to their relative importance. Then they proceed to complete the most important items first, followed by those of lesser importance, and finally those of least importance, if they have time. This is a logical and common sense approach that will usually yield satisfactory results. One common error that many first-time candidates make in the In-Basket is that they fail to use their imagination and do only what is specifically required in the instructions. For example, there is no instruction that a candidate place someone in charge during his or her absence, but this is something they would normally do if they planned on being away for several days. Similarly, it is unlikely that anyone would be away from the job for several days and not notify their immediate supervisor. However, since this requirement is not included in their instructions, many first-time candidates overlook this necessary action. We live in an age of technology, and even though an executive may be away from his or her office for several days, they are never out of touch. Modern technology offers us instant communication by way of facsimile machines, cellular phones, pagers, e-mail, the Internet and other media. These can and should be used by a candidate when working an In-Basket Exercise, if appropriate. In addition, few executives ever leave to go on a trip with an empty briefcase. “I’ll call you Monday from Memphis,” or “I’ll fax you my instructions from the hotel in Philadelphia” and similar instructions might be appropriate when completing items in an In-Basket Exercise. This may not be possible, though, if the candidate is headed for Tasmania or a similar remote location. The GroUP Problem-solVinG eXerCise
The Group Problem-Solving Exercise, sometimes called the Leaderless Group Discussion, is an exercise in which a group of candidates (usually not less than three nor more than six), is assigned to work together for up to an hour to solve a problem or to complete an assignment. If the candidates are competing for ranking positions within the police or fire departments, they may be told to assume that they have been assigned to a
26
The Assessment Center Handbook for Police and Fire Personnel
committee by the fire chief or police chief. The committee has been asked to meet to discuss how a particular issue, or series of issues, might be resolved. An example of this exercise is shown in Appendix D. The topics assigned to the group should be something with which they can be expected to be familiar. For example, it would not be appropriate to assign a group of sergeants a problem involving the departmental budget unless they are expected to have a working knowledge of the budget and how it is formulated. In other words, the assignment should be one that is relevant to the position for which they are being considered.
Group consensus is a good way of making key decisions.Photograph courtesy of Joplin Police Department, Joplin, Missouri.
If candidates are competing for a chief’s position in a fire or police department, they may be told to assume that they are all fire chiefs or police chiefs from a particular region, or that they are serving on a committee of the local or state police chiefs’ or fire chiefs’ association. Once again, the topic
Typical Assessment Center Exercises
27
assigned to the group should be one with which they can be expected to be familiar. In some cases, candidates may be assigned specific roles to play or tasks to perform. In the assigned role format, the group may represent, for example, various community organizations or interest groups such as senior citizens, the youth, social services, downtown merchants, schools, or neighborhood alliances. Candidates may be assigned to represent one of these groups and can be given information about the composition of that group and its particular interests. They may be assigned to work together to solve a particular problem of interest to the particular agencies or organizations represented by the group. This is an excellent way to see how well candidates can appreciate the views and concerns of others and how they can apply themselves to understanding and attempting to see problems from another person’s point of view. In the assigned task format, candidates may be assigned the task of “selling” a particular topic or project to the other members of the group. For example, a typical assignment might tell the candidates to assume that the fire department has received a federal grant of $50,000 to improve the level and quality of fire department services in their community. Each member of the group can be assigned a different project and will be expected to try to earn the support of the other members of the group for that project. This format often puts the candidates in a very competitive posture since they feel that their success in the exercise depends upon getting the other members of the group to go along with their project. In any group exercise, assessors are looking for a candidate to demonstrate strong leadership ability. At the same time, they are looking for a candidate to demonstrate his or her ability to work within the framework of a group to solve a problem. A strong candidate is often one who participates throughout the exercise, and who encourages others to participate as well. A strong candidate in this exercise is one who listens to what others have to say rather than one who is more concerned about having his or her own way or who tries to dominate the group. Candidates in this exercise must understand that there is a very sharp difference between leadership and domination. Some people incorrectly believe that if they speak loud enough and long enough and if they drown out everyone else in the group they are the winner. The truth is just the opposite. Listening to what others have to say, encouraging others to participate, and helping to bring the group to consensus are the most important characteristics a candidate can exhibit in this exercise. Candidates should be given between 15 and 30 minutes to study their instructions before beginning the exercise. Candidates should be told to prepare independently and not to discuss the assignment with each other
28
The Assessment Center Handbook for Police and Fire Personnel
before the exercise begins. This helps to preclude candidates from conspiring among themselves by establishing certain ground rules, such as who will take notes, how much time will be allotted to each person to speak, and so on. Whatever organizational issues that need to be decided by the members of the group should be done before the assessors, since this is part of the evaluation process. The room may be equipped with a flip chart, easel, and marking pens or a dry erase board. These are placed in the room for the group to use as tools in the discussion. A candidate who seizes the initiative to use these tools to facilitate the group discussion will probably be evaluated favorably by the assessors for recognizing and seizing this opportunity. Participation is clearly an important ingredient for success in the Group Problem-Solving Exercise. Candidates whose participation is minimal — who say very little and who contribute almost nothing to the final product of the group’s efforts – will surely not be rated highly by the assessors. But the quality of a person’s participation is just as important, if not more so, as the quantity of his or her participation. What a candidate says or does to help the group solve the problem is just as important as how much time a candidate spends saying or doing it. Some persons may try to place themselves in a leadership role in this exercise by organizing the group, assigning tasks, setting dates for future meetings, and the like. This is a good strategy, but it can backfire if the other members of the group resist such efforts. If it is done, it must be done in a way that will not threaten or challenge other members of the group. In addition, some persons may volunteer to keep the minutes of the meeting. This is an excellent idea, but it is one that can lead to disaster if the note taker becomes so involved in taking notes that his or her participation in the discussion suffers. On the other hand, some candidates have effectively used the note-taking role to become the informal leader of the group by keeping the discussion on track, by reinforcing significant ideas, and organizing the discussion. The Group Problem-Solving Exercise can be used for just about any position, but it is probably most useful when testing candidates who are not acquainted with one another. In small organizations where candidates all know each other quite well, we sometimes find that the results of the group exercise may be influenced by seniority and other organizational variables. Younger candidates may feel compelled to defer to older candidates who are informal leaders in their agency. In addition, organizational alliances and interpersonal biases, rivalries or disputes may intrude into the group process. This is not to say that a group exercise should not be used in those agencies where the candidates are known to each other, but rather to simply point out that organizational dynamics may affect the results.
Typical Assessment Center Exercises
29
The Group Problem-Solving Exercise, or Leaderless Group Discussion, is perhaps one of the better exercises available to us to test a person’s leadership ability, particularly in a group setting. Jansen and Stoop (2001), for example, found that the Leaderless Group Discussion (LGD) had some of the highest validity of any of the predictors they examined in their research.1 While strong leadership skills may be manifested in a one-on-one role playing scenario, such as the Employee Meeting Exercise, or “Problem Employee” Exercise, displaying leadership among peers would appear to be a greater challenge and it is one that can ably be demonstrated in a group setting. The TACTiCAl eXerCise
First-line supervisors in both police and fire departments need to be able to think and act quickly and to make decisions under pressure. They need to be able to see “the big picture” when dealing with tactical problems and to use available resources effectively. These abilities can be measured very successfully in a tactical problem designed to simulate conditions in the field. The kinds of tactical problems used in the police and fire departments are quite different, but they are also quite similar in the manner in which they simulate actual field conditions. While the kinds of tactical problems presented to candidates in these exercises may not be something that they will be faced with on a regular basis, their failure to respond to them properly can have disastrous consequences for the agency. The TACTiCAl fire Problem
While fighting actual working fires is not something that a firefighter will do every day, it is one of the most important things he or she is trained to do. Firefighters spend countless hours drilling in a variety of situations designed to improve their ability in a live fire situation. On the fire ground, there is no time to refer to a textbook or written procedure before taking action. A Tactical Fire Problem is a useful exercise for both a company officer or chief officer (e.g., battalion chief or district chief) position in the fire service, although the kinds of problems presented to candidates for these two positions would be substantially different. In the case of a company officer exercise, the fire problem should usually be fairly simple involving one, two, or 1 P. G. Jansen and B. A. M. Stoop, “Dynamics of Assessment Center Validity: Results of a 7-year study,” Journal of Applied Psychology (2001), 86, pp. 741–753.
30
The Assessment Center Handbook for Police and Fire Personnel
three companies, since it would be unlikely that a company officer would be in charge of a fire problem involving much more than this (see Appendix E for an example of a Fire Tactical Problem). In the case of a chief officer exercise, the problem would be substantially more complicated and would probably involve several companies as well as a more complex situation, such as multiple injuries, explosion, hazardous materials, etc. A typical fire problem for a company officer exercise would involve a small fire in a single building such as a residence or commercial structure. A typical fire problem for a chief officer exercise might involve a much larger structure, such as a multifloor apartment complex or nursing home, a large warehouse containing explosives or hazardous materials, or a fire in a large retail establishment with significant exposure problems.
Tactical fire problems endeavor to replicate reality as much as possible. Photograph courtesy of Greater Brighton Fire Protection District, Greater Brighton, Colorado.
Locations selected for the Tactical Fire Exercise can be either actual locations in the jurisdiction in which the candidates are employed or they may be fictitious structures. In some cases, it may be possible to use actual struc-
Typical Assessment Center Exercises
31
tures located in a different jurisdiction so that none of the candidates would have prior knowledge of the building that might give them an advantage over another candidate. The advantage of using an actual structure is that preplan information is often available which can be used by the candidate to prepare his or her approach to the problem. In addition, if a diagram of the building is available, it can be projected onto a chalkboard or similar surface by use of an overhead projector. This can then be used by the candidate to illustrate placement of apparatus, hose lines, water supply, etc. In the Tactical Fire Problem, candidates will usually be given information regarding the nature of the incident, to include building location, time of day, weather conditions, and status of incident upon arrival. They will usually be given a few minutes to study this information and then be asked to explain to the assessors their actions upon arriving at the scene. They will normally be told to assume that they are responding on the first (or second) arriving engine and that they will assume responsibility as Incident Commander for the duration of the incident.2 They will usually be instructed to assume that they have all normally available resources at their disposal in this situation. It is important that a Subject Matter Expert meet with the assessors before this exercise begins to discuss operating strategy and normal operating procedures during this incident. In this way, candidates may be evaluated on the basis of the actions that would normally be expected in this kind of situation in that particular community. Assessors need to have detailed information regarding available apparatus, type of equipment, hydrant placement, alarm procedures, Incident Command strategies, and the like when evaluating candidates in this exercise. In some cases, a Resource Person may be assigned to work with the candidate in this exercise. The Resource Person may be one of the assessors, a staff person, or a ranking member of the fire department in which the exercise is taking place. The role of the Resource Person is to simulate communications between the candidate and central communications as well as other units at the scene of the incident. The Resource Person may also help to control the exercise by providing the candidate with progress reports on the condition of the incident at specific intervals. In this way, the candidate may have a mental picture of the incident scene and can thereby make decisions based upon up-to-date information on what is happening at that time. It is a good idea to develop a scenario providing for different problems to be encountered during the exercise (see Appendix F). These problems might include, for example, a victim trapped in a third-floor apartment, the loss of water pressure due to a faulty hydrant, a structural collapse, or an
2 Some local policies provide for the first engine on scene to make a quick attack, thereby automatically passing “command” to the second company arriving. This issue needs to be resolved in advance so that the candidates are not confused on this point.
32
The Assessment Center Handbook for Police and Fire Personnel
explosion from a ruptured gas line. These can be planned at predetermined intervals to further test the candidate’s problem-solving skills. These events should all be rehearsed in advance with a Subject Matter Expert to ensure realism and so that assessors will understand the proper procedures for dealing with them. Another way to simulate reality is through the use of videotaped pictures of the actual building involved. Several slides of the building, taken from different angles, can be projected on a screen and then videotaped. By using very basic tools, flame and smoke can be imposed on the slides. This is done in such a way that the fire problem changes every thirty seconds or so, giving the candidate a simulated view of the building and the fire situation. One of the problems that this poses is that the fire problem will continue to escalate or change regardless of what actions are taken by the candidate. However, if they are informed in advance that this will occur, it will help to ease their frustration. A further enhancement of this problem is the use of portable radios by both the candidate and the Resource Person to simulate the company officer speaking with communications on the radio. Another staff person from the agency may be assigned to play the role of other companies at the scene and may also communicate with the candidate by way of the portable radio. This helps to add to the realism of the exercise and allows the candidates to more easily inject themselves into the role they are expected to play. At the conclusion of the Tactical Fire Problem, which will normally run for about 10 minutes, time should be allowed for assessors to ask the candidates questions about their approach to the problem. In this way, the assessors can get a better idea of why certain actions were or were not taken by the candidates. Candidates often appreciate the opportunity to explain their rationale to the assessors, even though the assessors might not agree with their strategy. Among other things, the assessors should be looking for the candidate’s ability to set reasonable priorities, to develop a workable plan of action, and to effectively coordinate the activities of units at the fire scene. The TACTiCAl ems Problem
Fire department personnel do more than fight fires. Indeed, most fire departments spend far more time responding to emergency medical calls than they do fighting fires. It is therefore appropriate that they be confronted with an emergency medical problem as a part of the assessment center process. The set up for the tactical emergency medical problem is very much like
Typical Assessment Center Exercises
33
that for the tactical fire problem with the obvious exception that the incident involves a medical incident rather than a fire incident. Digital photos can be used, for example, to show actual or simulated emergency medical incidents, such as serious automobile accidents, train derailments, building explosions, and the like.
Emergency medical care is an important element of a firefighter’s daily work. Photograph courtesy of Bartlett Fire Protection District, Bartlett, Illinois.
Candidates are placed into the situation as the Incident Commander and are expected to act out the manner in which they would handle the incident, to include making assignments, obtaining status reports, calling for additional resources, and so on. Like in the tactical fire problem, assessors are provided with a checklist of items that candidates would be expected to deal with or provide for during the emergency medical incident. This checklist should be reviewed by a Subject Matter Expert to ensure that the benchmarks used to evaluate candidates are relevant and appropriate for the position for which candidates are being evaluated and that they are consistent
34
The Assessment Center Handbook for Police and Fire Personnel
with the Standard Operating Policies and Procedures of the department. The TACTiCAl PoliCe Problem
One of the unique things about police work is that an officer never knows what to expect when he or she leaves for work. No two days are alike, and each situation the officer encounters offers different challenges. For the most part, this is an advantage in that it helps to maintain a high level of interest in the job and reduces boredom and monotony. Most successful police officers like the fact that they never know what to expect when they report for duty each day. On the other hand, this constant shift in the environment in which the police officer works, and the uncertainty behind each call to which an officer responds, can also create a level of anxiety which does not always lend itself to job satisfaction or good job performance. Police officers and their supervisors often work in situations where they must be able to make quick decisions based on sketchy information. Some of these decisions must be made in life or death situations and a poor decision can have disastrous consequences. Working under these conditions can be quite stressful. The Tacti-
Officers in tactical Deployment. Photograph courtesy of Joplin Police Department, Joplin, Missouri.
Typical Assessment Center Exercises
35
cal Police Problem is a good tool to measure how well a candidate can work under pressure and his or her ability to make sound decisions in such situations. In the Tactical Police Problem, candidates are usually placed in the position of a patrol supervisor and are given information concerning the day of week and shift on which they are working, the number of officers available to them and their assignments, and any other information that will assist them in becoming familiar with the situation in which they are being placed. They are then provided with information regarding some kind of tactical problem that has just occurred in their zone of responsibility. This could be a barricaded subject, an escaped prisoner, a lost child, a school shooting, a major disturbance, or some other event. They are informed that they have been requested to respond to the scene of the incident and assume command of the situation. They are also informed that, for the duration of the exercise, they will be the senior officer on the scene and will be responsible for making all decisions regarding the incident and related incidents. Candidates are usually given 10 or 15 minutes to study the material that has been provided to them and then are asked to report to the exercise room where they may find a large map of the area in question projected onto a chalkboard or similar surface by means of an overhead projector. They are asked to place themselves at the scene of the incident and to tell the assessors what they are doing upon their arrival. As in the Tactical Fire Problem, a Resource Person is assigned to communicate with the candidate. The Resource Person will assume the role of the central communications officer and controls the movement of units into the incident. One or more of the assessors may be assigned to play the role of officers and units involved in the situation. For example, if the candidate is assigned four district cars, one Community Service Officer, a traffic car and one investigator, two of the assessors may be assigned the role of those units. The Resource Person will also have additional information and resources that will be available to the candidate, but only if the candidate specifically asks for them. For example, the Resource Person may be given a list of other units and resources as well as resources from other agencies that may be available to assist if requested. However, these are only provided to the candidate if he or she specifically asks for them. The Resource Person will also read from a prepared script and will announce new events occurring at specific intervals. These events may relate to the incident in which the candidate is involved, or may be unrelated to the incident, but may require some action on the part of the candidate. For example, the candidate may be informed of a bank alarm activated in his or her sector which may require him or her to have a unit dispatched to inves-
36
The Assessment Center Handbook for Police and Fire Personnel
tigate. Some incidents may be relatively minor and may be “held” by the candidate until sufficient resources are available. Failure to take appropriate action on significant incidents, as well as the inappropriate use of resources on incidents of little consequence may adversely affect a candidate’s score in this exercise.
Police officers are often called upon to engage in high-risk tactical situations. Photograph courtesy of Bloomingdale Police Department, Bloomingdale, Illinois.
The Tactical Police Problem will usually run 15 minutes, including a few minutes reserved at the end of the exercise for questions from the assessors. Assessors should be provided with a check list of actions that the candidate should be expected to take, such as calling for a K-9 unit (in the case of a lost child); calling for an area-wide radio broadcast for information on a possible suspect; requesting assistance from neighboring zones of police agencies to have sufficient personnel at the scene; and the like. This list should be reviewed in advance with the Subject Matter Expert to ensure that the actions of candidates are consistent with local policies and procedures. The Tactical Police Problem is an excellent way to test a candidate’s abil-
Typical Assessment Center Exercises
37
ity to make sound decisions under stress, to anticipate events, and to see the “big picture” rather than to be narrowly focused on one event at a time. It is also one of the most difficult exercises in an assessment center, particularly for someone who has not participated in one previously. Among the common errors made by the first-time candidate in this exercise are the following: • The candidate may simply minimize the significance of the problem and therefore not take appropriate action as soon as the situation actually dictates. For example, what may initially appear to be “only” a missing child may later prove to be a child abduction with extremely adverse consequences. • Perhaps because a candidate fails to appreciate the potential seriousness of a problem, he or she will often fail to utilize a sufficient number of personnel and resources to deal with it. Underutilization of resources is a common problem and will often impede the candidate’s ability to successfully solve the problem. Overutilization of resources in this kind of exercise is a sin easily forgiven. • Candidates often fail to keep track of their units and what is happening around them. A successful candidate will keep a running log of all involved units and what they are doing. This can be done by using a writing pad or by writing the location and assignment of all units on the map that is projected on the chalk board. This helps the candidate to keep a record of all personnel involved and to make changes in assignments quite easily. In constructing the Tactical Police Problem, it is important that the scenarios be as realistic as possible and be based on actual locations in the community. Here again, a Subject Matter Expert from the department should be asked to aid in providing actual street locations, addresses, and other information such as number of units to be assigned, additional resources available, and the like. In addition, assessors should be thoroughly familiar with the usual operating procedures of the department in similar situations. The Tactical Police Problem should be reviewed by the assessors and the Resource Person with a member of the staff prior to the assessment center so that candidates will feel as comfortable as possible once the exercise begins. It is also helpful to conduct a “dry run” of the problem in advance so that the assessors can become familiar with the timing and sequencing of events, radio procedures, and unanticipated difficulties that may be encountered when the ex-
38
The Assessment Center Handbook for Police and Fire Personnel
ercise begins. The emPloYee meeTinG eXerCise
One of the most frequent and important tasks of any first-line supervisor in either the police or fire service is to supervise his or her immediate subordinates, monitor their job performance, counsel them regarding poor job performance, encourage and reward them when they excel, and recommend disciplinary action when they commit serious breaches of rules and regulations. A first-line supervisor is only as good as the subordinates he or she supervises, and the performance of subordinates is often a direct reflection on the ability of the supervisor to get the most out of his or her employees. The Employee Meeting Exercise is a very common exercise in just about any assessment center for a first-line supervisor or a middle manager. In some cases, it is also appropriate to use for an executive-level position. There are few supervisory or management jobs that do not require a person to deal with other employees and to work with their subordinates in resolving problems. See Appendix H for an example of this exercise. In this exercise, candidates are given information regarding a particular problem that requires the candidate to speak with one of his or her subordinates. Candidates may be given basic information about the subordinate, such as how long they have been employed, their family status, and their previous job performance. Candidates may be told that they have just been promoted to the position and have recently transferred to this particular unit, so they have not had the opportunity to work with this employee in the past and may only have background information on the employee. In most cases, the “employee,” who will be a role player, will react rather negatively or with a lack of interest or enthusiasm to the information and advice being offered by the candidate. The role player may try to place the blame for the problem on someone else or simply suggest that the new supervisor is picking on him or her and may not know enough about being a supervisor to make an informed judgement about the problem. This places the supervisor in the unenviable position of defending himself or herself while trying to let the employee know who is in charge. A strong role player may succeed in dominating the interview if the candidate does not have a sound plan of action before the interview begins. The role player may also try to place blame on the administration or other staff personnel as a means of avoiding responsibility for his or her own poor job performance. The candidate must be quick to defend the administration as well as other supervisors and to make it clear to the employee that the
Typical Assessment Center Exercises
39
real problem is his or her own poor job performance and to accept responsibility for his or her own actions. This can be a difficult exercise for candidates who have little experience in dealing with interpersonal problems. It may be difficult for a candidate to accept the role player as a real person, or to place himself or herself into a superior-subordinate relationship with the role player. It is important that the candidate have the ability to step into the role and to accept the role player
Counseling a subordinate is an important element of a supervisor’s job. Photograph courtesy of Joplin Police Department, Joplin, Missouri.
as a real person and not simply a role player. As in all simulations, the success of a candidate depends, in part, on his or her ability to accept the situation as given and to handle each and every scenario as if it were the real thing. It is important that the role player not overplay the role or refuse to allow the candidate to solve the problem. While the role player should not make the task too easy for the candidate, he or she should also not make it impossible for the candidate to solve the problem. If a candidate has a reasonably
40
The Assessment Center Handbook for Police and Fire Personnel
good approach, is able to apply sound logic in attempting to solve the problem, and is able to motivate the role player to work together in solving the problem, the role player should allow the problem to be solved. On the other hand, a candidate who simply insists upon forcing his or her authority on the role player and makes no effort to appeal to the role player’s intellect or common sense is not solving the problem in a satisfactory manner. It is important for candidates to approach this exercise with an open mind and to be a good listener as well as a good speaker. Some candidates make the mistake of entering into this exercise with the perspective that they know what the problem is and that they have the perfect solution for it. A successful candidate will be able to enter into the exercise with a good plan of action, but that plan of action should include obtaining all pertinent facts before making a decision on a definite course of action. It is also important that the candidate keep a positive attitude throughout the exercise, as the role player may very well try to provoke the candidate by using certain words or actions. It is clearly inappropriate for the candidate to lose his or her temper, regardless of the level of provocation offered by the role player. This exercise may be considered as having three tasks. The first task is problem identification, in which the candidate sets out to identify the problem. This is usually easy enough, since the instructions often describe why the interview is being held, and thus has already identified the problem. However, it is up to the candidate to obtain additional information in order to fully understand the scope and depth of the problem. The second task is called problem resolution, in which the candidate attempts to devise a plan to solve the problem. It is wise to avoid having preconceived ideas going into the exercise. The plan of action needs to be based upon a full understanding of the true scope and nature of the problem. This requires obtaining additional information from the employee during the interview. The third task, and the most difficult in some cases, is closure. It is not good enough for the candidate to merely devise a plan of action if the employee refuses to buy into the plan. The candidate must be able to “sell” the plan to the employee by gaining the employee’s agreement and acceptance. This is not always easy to do and requires the candidate to employ logic as well as persuasion in gaining the confidence and voluntary cooperation of the employee. Some candidates never achieve this goal and thus fail to receive a satisfactory score from the assessors in this exercise. Successful closure also requires the candidate to provide for a follow-up meeting to review progress in meeting the goal(s) set forth during the initial interview. The employee should have a clear idea of what is expected of
Typical Assessment Center Exercises
41
him or her at the conclusion of the interview as well as the consequences if those goals are not met. The CiTizen inTerVieW eXerCise
While the Citizen Interview Exercise may appear more germane to police than to fire personnel, it is used extensively in assessment centers for both services. Although handling citizen complaints may seem like a task more appropriate for police personnel, there are many instances in which fire company officers and chiefs have to deal with an angry or frustrated citizen. The bottom line is that a way needs to be found to solve the citizen’s problem, if that is possible.
Good human relations skills are an essential characteristic of police work. Photograph courtesy of LaGrange Police Department, LaGrange, Illinois.
The Citizen Interview Exercise usually places the candidate in the position of a patrol supervisor, company officer, or shift commander who is informed that a citizen has asked to speak with “someone in charge” about a problem. An example of this exercise is shown in Appendix I. It is important that the role player play the assigned role consistently from one candidate to the next and not attempt to improve each time. It is also important that the role player allow the candidate to solve the problem, but
42
The Assessment Center Handbook for Police and Fire Personnel
not without effort. In other words, the role player should place enough obstacles in the path of the candidate to make it difficult to succeed, but not impossible. A role player will usually play the role of the citizen and will be provided with a script describing some kind of problem in which the candidate is asked to seek a solution (see Appendix J). This will often involve some complaint or allegation regarding a specific police officer or firefighter or may be about poor service of some type. For example, the role player may complain that an officer was rude or discourteous during a routine traffic stop, or that the officer declined to take a report or initiate an investigation of the theft or loss of a bicycle. The responsibility of the candidate in this exercise is to take the required information and formulate a plan for dealing with the problem. Candidates frequently neglect to ask pertinent questions, or to record important information. In addition, they sometimes are defensive when the allegation is revealed to them and may say something like “Well, there are two sides to every story,” which amounts to saying, “I am not sure I believe everything you are telling me.” Even though there are two sides to every story, and even though the candidate must wait to obtain all the pertinent information before making a decision, this is not what the citizen wants to hear. A better way of saying it might be, “Well, this is certainly a serious matter, and I will talk to Officer Jones (or Firefighter Green) immediately to find out what he (or she) has to say about this.” The time allotted for this exercise is usually 15 minutes, which is not a great deal of time, but it is enough to get the job done if the candidate has a good plan and follows the plan. It is important to greet the citizen warmly and show genuine interest in his or her problem. A candidate may offer the citizen a cup of coffee, a glass of water, or a soft drink as a way of making the citizen comfortable. It is always a good idea to let the citizen speak freely for the first few minutes to get an idea of the nature of the problem. A candidate should not rush the citizen or make the citizen feel as if they are in a hurry, or that they have something more important to do than speak with them. Rather, the candidate should make the citizen believe that dealing with them is the most important thing they have done all day. If the candidate plans on taking notes (and they should), they should do this in an unobtrusive manner. They should first ask the citizen if he or she objects. It is important that the candidate not interrupt the citizen, but that the candidate keep them headed in a straight direction. He or she can do this by asking a few clarifying questions, such as “Did you get the officer’s name?” or “What did the firefighter say then?” Candidates should employ active listening techniques, such as nodding their head, saying “I see” or “yes, I understand.” This lets the citizen know that his or her message is being received and understood and gives them greater comfort in their time of
Typical Assessment Center Exercises
43
need. Even though the candidate may not be the person in the organizational hierarchy who will eventually be asked to resolve the problem (for example, some complaints may be referred to Internal Affairs Investigations for resolution), it is important that the citizen feel that some action is being taken to resolve the problem for them. Candidates should avoid acting as if they are trying to pass the buck or let someone else make the decision. “You really need to see Lt. Green in Internal Affairs about this” is not telling the citizen what they want to hear. “I will speak with Lt. Green in Internal Affairs about this immediately, and one of us will get back to you in a few days” is a much better response. Citizens like to feel that the person to whom they are speaking is going to take the initiative to get the problem resolved, even though that person may lack the authority to ultimately resolve the problem. When the citizen leaves the interview, he or she should feel as if they came to the right place and they should feel confident that something will be done regarding their concern. It is usually a good technique to ask the citizen, “What would you like me to do?” if that is not already obvious, or to explain to the citizen what action a candidate plans to take and ask, “Is this satisfactory?” Candidates must be careful not to promise too much or to build up undue expectations in the mind of the citizen. Candidates should not promise to do something that is clearly beyond their authority or make promises that they cannot deliver. It is a good idea to let the citizen know what will happen in the future. Candidates may let the citizen know that they or someone else will get back to them in a few days, if that is satisfactory. Candidates should leave no doubt in the citizen’s mind (and in the minds of the assessors) what action they plan to take to resolve the problem that has been presented. Dealing with citizens is one of the more common exercises in an assessment center. It is not a terribly difficult exercise for most candidates, but it does require good interviewing skills, a logical plan of action, and the ability to be sensitive to the problem being presented. In this and other assessment center exercises, candidates should always remember that half of communicating is listening, and that they need to be a good listener as well as a good problem solver. It may be desirable to link the Citizen Interview Exercise together with the Employee Meeting Exercise. For example, after a candidate has interviewed a citizen and obtained the necessary information about a problem the citizen has with the way he or she was treated by a police officer or firefighter, the candidate may then be asked to interview the officer or firefighter about whom the allegation was made. This creates a complex but realistic scenario in which the candidate’s fate in the second exercise may be linked to how well he or she does in the first one. If the candidate fails to obtain essential
44
The Assessment Center Handbook for Police and Fire Personnel
details about the encounter during the interview with the citizen, he or she will have a more difficult time in explaining the problem when speaking with the employee. Even though these two exercises may be linked together, they should still be treated as separate exercises and should be scored separately by the assessors. It is sometimes the case that candidates will do quite well in one of the exercises but rather poorly in the other. Each requires a sightly different set of skills. Assessors need to be able to separate these two exercises in their mind so that they do not allow a candidate’s performance in one of the exercises to cloud their evaluation of the candidate’s performance in the other. The fire insPeCTion eXerCise
Fire inspections are one of the more important non-emergency tasks performed by members of a fire department. These may be performed by a specialized unit of fire inspectors working within the Fire Marshal’s Office, or by designated fire companies within the suppression division. Most fire departments involve their fire companies in some aspect of fire prevention inspections, and thus it becomes important that the company officer be thoroughly familiar with fire prevention principles and building codes as well as electrical and safety codes. It is also important that the officer be capable of explaining these codes to the homeowner, business owner, and company management in order to gain voluntary compliance and to achieve an optimum level of community safety. In this exercise, candidates are told to assume that they have been directed by a superior officer to meet with the officials of a local business to discuss several problems that have been discovered during a recent fire inspection. Typically, the company in question would be a warehouse, retail store, manufacturing plant, or some other type of occupancy in which there could be code violations that pose a fire safety problem. A Subject Matter Expert should be asked to prepare a fictitious inspection report on the company in question. The candidate will be given a few minutes to familiarize himself or herself with the report. It is helpful if the building in question is an actual location in the community and one with which the candidates would be familiar. See Appendix K for an example of this exercise. Candidates are instructed that they are to meet with one, two or three company representatives to discuss the violation discussed in the report. The purpose of this meeting is to gain the company official’s voluntary and willing compliance with the code requirements. This requires that the candidate be able to explain the reasons for the code requirements and the
Typical Assessment Center Exercises
45
benefit to be gained by the company by voluntary compliance. The role of company officials may be played by the assessors or by role players furnished either by the assessment center administrator or by the
Fire Inspections may be a necessary function of the fire company. Photograph courtesy of Bartlett Fire Protection District, Bartlett, Illinois.
client agency. Role players should be familiar with the location, construction, contents, and characteristics of the company so that they can play the role assigned to them in a realistic manner. They should also be familiar with the code sections which will be discussed in the inspection report. Role players will be expected to pose certain obstacles in the path of the candidates in gaining voluntary compliance with the code provisions. They might point out, for example, the costs involved in making the necessary changes, or the fact that the chance of ever having a fire or other incident in the building is so slight as to negate the advantage of compliance. However, the role players should not be unmovable or totally obstinate in their objections so that the candidate is faced with a no-win situation. While the role players should not be a passive audience, they should allow themselves to be convinced by the candidate of the merits of voluntary compliance if
46
The Assessment Center Handbook for Police and Fire Personnel
the candidate is able to mount a logical and persuasive argument for compliance. In this exercise, candidates are asked to demonstrate their ability to deal with citizens in a somewhat or potentially confrontational situation while maintaining their poise and self control. They are also asked to explain code requirements that may be somewhat technical in nature to persons who may have no technical background in this field and to help them understand the need for compliance. Successful performance in this exercise requires excellent interpersonal skills as well as the ability to communicate effectively. Technical knowledge of building code requirements is also helpful. An important objective of this exercise is to gain the voluntary cooperation of company management in making the required corrections. This requires tact and “salesmanship” on the part of the candidate, as well as the ability to objectively and rationally explain the basis for the requirements. It is also helpful if the candidate can devise ways in which the city or the fire department can assist the building manager in gaining compliance. The shifT or ComPAnY meeTinG eXerCise
One of the basic tasks of any first-line supervisor in either a police department or a fire department is to meet regularly with members of their shift, squad, company, or unit to discuss changes in procedure or policy, to assign work, to review work in progress, and to exchange information to ensure coordination of activities. The shift or company meeting exercise simulates a meeting between a first-line supervisor and his or her subordinates to review and discuss assignments as well as new policies or procedures. In a police department, this meeting would be comparable to a roll-call briefing. In a fire department, it would be called a company meeting. In this exercise, the candidate will usually be provided with instructions informing him or her that he or she is about to have a meeting with two or three members of their company, shift, or unit for the purpose of discussing new policies or procedures or information that has been handed down from the administration. In most cases, some of the items to be discussed may meet with resistance by one or more of the “subordinates.” For example, the information to be discussed may include a new performance evaluation system, or changes in the city’s insurance or benefits program, or a change in shift scheduling that may adversely affect some members of the group. An example of this exercise is shown in Appendix L. Meeting with subordinate personnel to exchange information and to discuss changes in policy or procedure is not a terribly difficult assignment, but
Typical Assessment Center Exercises
47
it is one that can sometimes test the leadership ability of the supervisor. This is particularly true when the information the supervisor has to disseminate may not be well received by his or her subordinates. The subordinates may
The shift briefing is an important responsibility of the shift supervisor. Photograph courtesy of LaGrange Police Department, LaGrange, Illinois.
attempt to test the supervisor’s leadership ability or his or her loyalty to the organization or its administration by making disparaging comments about the new policy or procedure or about the person (the chief or deputy chief) who initiated the change. The supervisor must try to explain the nature of the changes and to explain their purpose as well. In effect, the supervisor is being asked to not only defend the changes but to gain the subordinate’s willing acceptance of them. This is not always an easy task and requires a high level of leadership ability as well as human relations skills. The supervisor must be quick to defend the actions of the administration, even though he or she may not always agree with the instructions they receive. Subordinates may look for an opportunity to have the supervisor reveal his or her own lack of support for something the administration has done. This serves to strengthen the bond between a supervisor and his or her subordinates, but it may also undermine the supervisor’s ability to command or maintain the respect of his
48
The Assessment Center Handbook for Police and Fire Personnel
or her subordinates. The role of the shift or company members will usually be played by the assessors, who will look for ways to test the candidate’s leadership ability. A candidate must be able to confront things that are said or done, to challenge his or her leadership ability or authority, and must take decisive action when such things occur. For example, a candidate should not allow a subordinate to refer to the supervisor or to anyone else in a demeaning or disparaging way. If this does occur, the subordinate should be admonished and put on notice that such conduct will not be tolerated. Candidates sometimes do poorly in this exercise because they are unable to gain or retain control over the subordinates in what is clearly a somewhat chaotic situation. Candidates need to enter into the exercise with a positive attitude and with confidence and poise. A candidate cannot allow the role players to sidetrack him or her into meaningless arguments or to become angry or frustrated. In some cases, it may be necessary to call a role player aside and explain to them that their conduct is disruptive and disrespectful and that further violations will not be tolerated. This will usually be seen by the disruptive person as a signal to pay attention and go along with the rest of the program. It is important that candidates not overreact to remarks made by the role players either. This can be just as bad as not reacting at all. Candidates should be firmly in charge of the situation and set forth ground rules early in the meeting. Continuous violations of those ground rules may be cause for disciplinary action. The shifT or ComPAnY TrAininG eXerCise
Providing in-service training to subordinates is an important and frequent task of a first-line supervisor in any police or fire department. Both police and fire departments are unique in that they are constantly being updated in new techniques, laws, and procedures. Both police officers and firefighters operate in environments that are constantly evolving and developing, and it is important that they be kept abreast of these changes. The Shift or Company Training Exercise is similar to the Shift or Company Meeting Exercise except that it focuses on the delivery of a brief training program for members of the supervisor’s unit, shift or company. As in the Shift or Company Meeting Exercises, candidates in the Shift or Company Training Exercise are likely to encounter some degree of resistance from those persons who are playing the role of their subordinates. As in the Shift or Company Meeting Exercise, candidates in the Shift or Company Training Exercise are expected to gain and retain control of the group and to gain the
Typical Assessment Center Exercises
49
subordinate’s understanding and compliance with the material being presented to them. In this exercise, candidates are usually given 10 or 15 minutes to prepare a 15-minute training program to be presented to members of their company. This may be an assigned topic or an unassigned topic, in which case they are free to develop the training on any topic they wish. In the assigned topic, the candidate will be given both the topic to be presented as well as suitable materials to be used in preparing the training program. See Appendix M for an example of this exercise. In a fire department training program, for example, a candidate may be assigned one of several basic topics such as use of knots, ventilation procedures, use of Self-Contained Breathing Apparatus, or other operations. They may be given a reference book, such as the IFSTA Manual on Essentials, or some other reference book to use in preparing for their presentation. They may also be provided with a flip chart and marking pens or other suitable materials to use in making their presentation. In a police department, the topic to be assigned may be an existing General Order or Standard Operating Procedure on such topics as high speed pursuits, domestic violence, use of firearms, interviews and interrogations,
Staff meetings are often conducted on an informal basis. Photograph courtesy of Joplin Police Department, Joplin, Missouri.
50
The Assessment Center Handbook for Police and Fire Personnel
and the like. While any one of these subjects may be quite complex, it is up to the candidate to develop a training program that can be presented in a 15minute period. Thus, this training will usually be seen as refresher training and the role players as veteran police officers rather than new recruits. It is often a good idea to use several different topics for this exercise and to assign them to candidates in a random fashion. The advantage of this technique is that it keeps candidates from informing other candidates what they should prepare for. This is particularly important if the assessment center takes place over several days and candidates in the latter days have the opportunity to prepare themselves on the basis of what they are told by persons who have already been through the process. This requires the subjects assigned to the candidates to be generally similar in terms of level of difficulty and complexity. Otherwise, some candidates may later claim that the topic they were assigned was more difficult than that given to someone else. Assessors should be given copies of the materials that will be assigned to the candidates in advance of the exercise. This will allow them to familiarize themselves with the material so that they may ask questions of the candidates designed to test both the candidate’s knowledge of the subject matter as well as their ability to answer questions under pressure. In this exercise, candidates will be evaluated on their instructional abilities and their use of instructional techniques, such as providing an overview of the subject to be presented, providing learning objectives, involving members of the “class” in the presentation, use of visual aids, organization of ideas, time management, and the like. In addition, candidates may be evaluated on their ability to digest and explain complex and technical information in a precise and timely manner and to think and act quickly under pressure. Successful performance in this exercise requires a candidate to have good organizational skills as well as the ability to communicate effectively and to deal successfully with opposition and confrontation. The CommUniTY meeTinG or PresenTATion eXerCise
Members of both police and fire departments are frequently called upon to go before community groups to give public education programs or to answer questions regarding the level or quality of police or fire services. While this is a task often reserved for the chief of the department or other ranking members of the department, lower-level supervisors may also be called upon to perform this function. In the Community Presentation or Community Meeting Exercise (see Appendix N), the candidate is placed in the position of appearing before a group of community members to either present an educational program on
Typical Assessment Center Exercises
51
behalf of the police or fire department (e.g., crime prevention, fire safety education, etc.) or to meet with the community members to discuss problems and concerns they have regarding the police or fire department. Role players may be used as community members or this function may be performed by the assessors themselves, although it is difficult for the assessors to take
Careful planning is essential if desired goals are to be achieved. Photograph courtesy of Bartlett Fire Protection District, Bartlett, Illinois.
good notes when they are actively playing a role. In some cases, community representatives may be asked to play themselves in this exercise. This technique works extremely well, since no one else can speak better for the community than people who reside there. In most cases, the candidate will be given a topic on which to speak to the group, and will be given a short time (15 minutes or so) to prepare a brief address to the group. It is usually a good idea to allow the candidate to set the stage for the meeting and to deliver the presentation he or she has prepared, unless it appears that the candidate does not intend to allow time for questions from the audience. Responding to questions from those in at-
52
The Assessment Center Handbook for Police and Fire Personnel
tendance is a key component of this exercise and candidates should allow for this. It is important that candidates understand that, regardless of the topic on which they plan to speak. Nothing is more important than the concerns of the audience. Shutting down the audience or ignoring their questions or even minimizing their importance is a fatal mistake in this exercise. Role players will usually be expected to raise issues designed to test the candidate’s ability to think on his or her feet and to respond effectively to “hot” issues. In addition, they may be placed in the position of defending the administration of the department, particularly if they are told that the chief had intended to attend the meeting but, at the last moment, found something more important to do. If candidates encounter a hostile audience, they should allow those persons the opportunity to vent their anger and frustration and express their concerns. Candidates should not appear defensive nor should they try to minimize the importance of the members’ concerns or frustrations. They need to be empathetic and place themselves in the place of those in the audience. Candidates should be open and flexible and should let the members of the audience know that they are sincerely interested in working with them and aiding them in solving their problems. They must let these persons know that the police or fire department considers their problems important and are willing to work with them to find solutions. Candidates may not always have the answers community members want or expect. They may be asked questions on topics with which they may not be familiar. When this occurs they should honestly admit that they do not have the answer but that they will be glad to look into the subject and get back to the person later. It is always a good idea to identify people in the group by their last name (Mr. or Mrs. Jones, etc.) and to take a brief note on their question. Candidates may ask the person for their telephone number and inquire whether they can get back to them or have someone else get back to them in a day or two with information on their question. Role players should be allowed to place obstacles in the path of the candidates by asking them questions for which they may not be prepared, but they should not be unreasonable in their expectations of the candidates. In many cases, the candidate may be appearing in a situation with which he or she is not familiar and may be asked questions about which they are not well informed. When this happens, role players should be tolerant and respect the candidate’s lack of experience. On the other hand, a candidate who appears to attempt to bluff the audience and try to act better informed than he or she really is will usually not impress the assessors favorably. If role players are being used in this exercise, they should be coached by someone who is familiar with the community to raise issues that are pertinent to the community. They may wish to assume fictitious names or iden-
Typical Assessment Center Exercises
53
tities (“I’m Sam Brown and I own the hardware store on North Main Street”) to inject a greater sense of realism into the exercise. It serves little purpose for the role players to ask questions of candidates that are not relevant to the community. This may be a difficult exercise for candidates who are not comfortable appearing before groups or who are not good public speakers. The ability to communicate effectively, to think under pressure, and to deal sensitively and effectively with the problems of others are important qualities for a successful performance in this exercise. Candidates who have difficulty in this type of exercise should be encouraged to take courses in public speaking or to seek opportunities to speak before groups both inside the department and in the community. The PUbliC edUCATion eXerCise
Fire departments take their responsibility to inform and educate the public about fire prevention methods and safety procedures during fires and other emergencies very seriously. Typically, public education is a responsibility of the Fire Marshal’s Office if the department is large enough to have one, but in smaller departments company officers and chief officers from suppression units may often be called upon to make presentations to the
Public speaking is an important skill in an assessment center.
public on a variety of topics relating to fire prevention and public safety. The audience for these presentations can be as varied as the topics from which
54
The Assessment Center Handbook for Police and Fire Personnel
the speaker may wish to choose and include senior citizens, elementary school children, business owners and managers, homeowners, and other stakeholders in the community. In the Public Education Exercise, candidates are assigned to make an impromptu presentation to a target audience and will usually be given 15 minutes or so to prepare for the presentation. Candidates may also be provided with information pamphlets, reference materials, flip charts and marking pens, and other resources that may prove useful in preparing for and delivering the presentation. Assessors usually play the role of the target audience and will behave and ask questions typical of that audience. If the audience is a third grade elementary school class, assessors will need to adjust their questions to those typical of children of this age (i.e., where is your dog?). If the audience is a group of local business people and the topic is fire prevention in the work place, questions from the assessors would obviously be more directly related to the topic. Every interest group in the community has a different set of concerns that may need to be addressed in an exercise such as this and it is important that assessors, acting as role players, reflect the actual concerns that exist in that community and that might be voiced by that particular interest group. Homeowners and senior citizens, for example, will usually be concerned about rising taxes while local merchants will be concerned about regulations which may impact their profit margin. A successful candidate in this exercise will be one who can respond sympathetically and with understanding to the needs and concerns of the target audience while at the same time deliver a clear and cogent message about the importance of fire prevention in the community. Moreover, candidates in this exercise need to conduct themselves in a professional manner that will reflect positively on the image and reputation of the fire department. This is an excellent exercise in which to test the public speaking skills of candidates as well as their ability to think quickly and to respond effectively to questions while under pressure. The sTAff meeTinG eXerCise
The Staff Meeting Exercise is a variation of the Shift or Company Meeting Exercise in that it involves the candidate in a meeting situation with two or three subordinate personnel. This exercise is often used in an assessment center for a middle management or executive-level position to simulate the interaction that might occur between a recently appointed command officer and his or her immediate subordinate officers.
Typical Assessment Center Exercises
55
In this exercise, candidates are usually informed that they have recently been appointed to the position for which they are being considered and that they are about to have their first formal staff meeting with members of their immediate staff. In a police department, this could be a lieutenant or commander in charge of patrol having a meeting with two or three shift supervisors. In a fire department, this could be a chief officer meeting with two or three company officers. See Appendix O for an example of this exercise. The purpose of the meeting may be for the newly-appointed commander or chief officer to discuss his or her management philosophy with his or her immediate subordinates, or to discuss other matters relating to the administration of the particular unit or division. Assessors or role players may play the role of the subordinate officers and will usually be expected to react in different ways to the information being discussed. For example, one of the role players may respond to the information being presented with indifference while another may be altogether hostile in his or her attitude. Another role player may be either noncommittal or passive, or may attempt to win the favor of the commander by feigning praise, admiration or support for the information being discussed.
Chief officers need to know how to get the most out of their staff. Photograph courtesy of Bartlett Fire Protection District, Bartlett, Illinois.
56
The Assessment Center Handbook for Police and Fire Personnel
In some cases, the role players may demonstrate their dislike for or distrust of each other by making rude or inappropriate comments or engaging in other forms of unacceptable behavior. When this occurs, the candidate must be quick to intercede and take action to ensure that this kind of behavior is not allowed to continue. The objective of this exercise is to allow the candidates to demonstrate their ability to develop an effective working relationship with subordinate personnel. This requires that they be sensitive to the interests of others, be an effective listener and communicator, and be a good problem-solver as well. Candidates in this exercise should be aware that their leadership ability will probably be tested by the role players and they should be alert for any attempts to challenge their authority. Such attempts should be dealt with firmly but not in a confrontational manner. On the other hand, they should not overanticipate what may happen during the meeting, nor should they enter into the exercise in a confrontational manner. They should have a positive attitude going into the exercise and should remain positive and open throughout the meeting. Even if sanctions may be required against members of the group, they should be done in such a way as to not alienate the other members of the meeting. For example, it is a good idea for the candidate to take a particularly unruly subordinate aside and speak to that person in private to appeal to his or her sense of fairness. In this way, the candidate may agree to work with the subordinate once the meeting has ended. This is an effective way of handling a disruptive person and will usually allow the candidate to regain control of the meeting and bring it to a successful conclusion. The Staff Meeting Exercise can be a difficult one for someone who has problems dealing with a challenge to his or her authority. The best preparation for this kind of exercise is to develop techniques that will enable candidates to deal effectively with a disruptive participant in a way that will help them maintain control over the meeting while at the same time recognize that even the most disruptive participant may have a valid point to raise and should be recognized. The neWs mediA eXerCise
One of the characteristics of police departments and fire departments is that the work they do is not merely important, but is newsworthy as well. People are always interested in what police officers and firefighters do, whether it be apprehending a fugitive or rescuing an infant in harm’s way. There are few things that police and fire departments do that are not of some interest to the news media. As a result, dealing with the news media is an
Typical Assessment Center Exercises
57
everyday responsibility of someone in both the police department and the fire department. This will often be a designated Public Information Officer, but even in departments where such a position does exist, the ranking officer in the department cannot avoid dealing with the press on a regular basis. In addition, any officer in command of a newsworthy event will usually have to deal with the news media at least until the designated Public Information Officer arrives. Anyone who has had much exposure to the news media knows that media representatives can often be difficult and sometimes impossible to deal with. The media have a responsibility to get “the story” to their viewers, listeners, and readers. Sometimes, in their quest to be the first with the story,
Most news reports can be a valuable ally to the police and fire service.
they use what we may consider to be ruthless or unethical tactics in getting the information they need for their story. In addition, they may use pressure tactics to get the information they are seeking. Also, they sometimes are not too careful about verifying the accuracy or validity of the information they obtain.3 Unscrupulous reporters can be a difficult problem for the police or fire administrator or command officer. In their eagerness to obtain information that may be translated into a sensational story, they sometimes go to any ends and seem not to care about the people who may be adversely affected by what they write in their story. Police administrators are especially vulnerable to reporters who seem more interested in how sensational a story is than they are about how truthful it may be.
Most news media representatives are responsible and professional journalists. The criticism made here applies to those few who create a “bad rap” for the entire industry.
3
58
The Assessment Center Handbook for Police and Fire Personnel
The News Media Exercise is one in which the candidate is placed in a situation of preparing for an impromptu news conference dealing with a situation which may prove embarrassing or damaging to the police or fire department. The candidate is provided with a description of the event in the form of an incident report and then is given a few minutes to prepare a brief news release. He or she is then asked to meet with members of the news media to give the news release and to attempt to answer any questions they may have. The information that is being related to the press may involve, for example, a shooting by a police officer under questionable circumstances, a person killed in a traffic accident involving a fire truck responding to a false alarm, or a similar situation. The circumstances should be such that a person might question whether the actions of the police or fire department in the situation were appropriate, which might lead the news media to place an inappropriate interpretation on the events being reported. The task of the candidate in this case is to release the essential information and to place it in the context that the department was acting properly and that the incident is currently under investigation. Role players will usually be assigned to play the role of local media representatives (television, radio, or newspapers). Role players may be provided with props such as camera, clipboards, microphones, press credentials, and tape recorders to add to the realism of the exercise. In some cases, actual media representatives may volunteer to play themselves in this exercise. In one case, a local newspaper reporter played herself in a news media exercise and then wrote a very favorable article of her experience in the local newspaper. Role players will be provided with the same information that is given to the candidates and will be ready to ask leading questions designed to place the candidate under pressure or to defend the actions of members of their department. If a candidate denies that some action took place, the role player may respond by saying that “according to our sources . . .” as a way of injecting controversial facts into the situation. A role player may purposely place the most unfavorable interpretation on a set of facts as a way of placing the candidate in a defensive position. It is usually good practice in any matter under investigation to limit the amount of information released to the news media. Otherwise, the investigation itself may be compromised. In addition, there are certain privacy issues that must be considered. On the other hand, police and fire departments have a responsibility to keep the public informed of information that is important to them and have no right to hold back information that may be of interest to them unless there is a sound reason for withholding such information.
Typical Assessment Center Exercises
59
Candidates who unnecessarily withhold information from the news media in this kind of situation will usually regret that decision. Experience has shown that the more open a candidate is in dealing with the press, the less pressure will be exerted by the press to obtain still more information. An open and cooperative attitude by the candidate in this exercise will gain much more support than secrecy and any attempt to unnecessarily withhold information. Unless there are sound investigative reasons for not disclosing the information, candidates would be well advised to give the information to the press. They will find the press much easier to deal with if they do so. Role players should not make this exercise a “no-win” situation. While it is appropriate to place pressure on the candidates, they should temper the pressure according to the level of openness and cooperation expressed by the candidate. This exercise requires the candidate to be able to communicate effectively and to make sound decisions under pressure. Moreover, this is an exercise in which the candidate must be able to deal with confrontation and opposition with an even temper. The ProGrAm deVeloPmenT eXerCise
The Program Development Exercise (see Appendix P) is an exercise that is usually used for middle managers or higher-ranking members of a police or fire department. In this exercise, candidates are given an assigned topic and are asked to prepare a written report on that topic. The topic will usually be one of a fairly technical and complex nature. In some cases, the candidate may be given reference materials to use in preparing his or her paper, while in other cases the candidate may be left to conduct whatever research he or she may be able to in preparing his or her paper. This is an excellent exercise to test a candidate’s ability to formulate a plan for dealing with a specific problem, or to come up with creative ideas for consideration by staff, as well as their written and oral communication skills. The time allotted for the preparation of the paper will vary depending upon the nature of the topic and the degree of sophistication expected in the final product. In some cases, candidates may be given several days to research the topic and turn in their completed paper. In other cases, candidates may have overnight to prepare their paper and turn it in at the beginning of the next day. In still other cases, candidates may be given their assignment at the beginning of the day and may be required to turn it in later that same day. The more time candidates have to work on their papers, the better the end result
60
The Assessment Center Handbook for Police and Fire Personnel
Good planning is the key to writing a good report.
should be. Candidates may be given the option of submitting a typewritten or computer-produced paper or one that is handwritten. It is recommended that candidates not be rated heavily on which option they choose, unless all candidates have access to computer terminals and printers and are expected to be computer literate on the job, in which case a computer-generated paper should be expected. The nature of the topic to be assigned to the candidates should be relevant to the position for which they are being evaluated and should be a subject with which all candidates are familiar. It would not be appropriate, for example, to ask candidates to submit a budget proposal for upgrading the department’s canine program, or DARE program, if only one or two candidates are familiar with those programs. There are many topics that might be assigned to candidates in this kind of exercise, including such things as: • Enlisting community support in the investigation and eradication of illegal drugs. • Expanding the use of civilians and volunteers in police programs and operations. • Expanding the role and scope of the fire department’s public education and fire prevention program. • Gaining wider departmental acceptance for the Community Policing Program. • Finding a way to increase the frequency of inspections of commercial occupancies. • Implementing a productivity improvement program in the patrol division.
Typical Assessment Center Exercises
61
Candidates are usually instructed to limit their papers to five or six pages which should be double-spaced if typewritten or computer-generated. Assessors are given a brief time to review the papers and then candidates are scheduled to appear before the assessors to make an oral presentation to them regarding their paper. They are instructed not to simply read the paper, since the assessors have already read it, but rather to summarize the high points of the paper. They are also instructed to leave ample time for questions from the assessors. Questions from the assessors are intended to clarify or reinforce the points made by the candidate. In some cases, the question and answer period will reveal that the candidate had not completely thought out his or her proposal. In other instances, follow-up questions from assessors may even reveal that the candidate’s paper was actually the work of someone else. In this exercise, candidates are usually rated on both written and oral communication skills as well as their program analysis and organization and planning skills. Relatively equal weight should be given to the effort that went into preparing the paper as well as the candidate’s ability to orally present and defend the ideas presented in the paper. The CriminAl inVesTiGATion eXerCise
The Criminal Investigation Exercise, as its name implies, is an excellent exercise to use for candidates who are being considered for positions in criminal investigation or in command of investigators. Due to the fact that it is specialized in nature, this exercise is not recommended for patrol supervisors or for command officers or other persons who are not directly involved in the field of criminal investigation. In this exercise, candidates are given a “crime report” which contains information about a crime that has been recently reported. The report would include several clues that, if properly interpreted, might lead to the identification of the person(s) responsible for the offense. In addition, the report will list several “witnesses,” who have additional information about the crime. Each witness possesses different information regarding the crime. Pieced together, the witnesses’ statements, along with the information contained in the crime report, can lead to the successful identification of the offender(s). However, the challenge for the candidate is to gain all the pertinent information that is available and make the correct deduction from that information. Two or three role players are used as the witnesses in this exercise. Each
62
The Assessment Center Handbook for Police and Fire Personnel
Automation is an important tool of the modern police officer. Photograph courtesy of LaGrange Police Department, LaGrange, Illinois.
“witness” is given a script that contains the information regarding the crime scene that this witness can provide to the candidate, but the information is provided only if the candidate asks for it specifically. In other words, the witness will not volunteer any information, but will only respond to the inquiries made by the candidate. This requires candidates to ask the right questions in order to get the correct information. They must also draw the correct conclusions from the information they receive. Candidates are usually given 15 minutes or so to review the “crime report” and then are given another 30 minutes or so to interview the three witnesses. The candidate may spend as little or as much time as he or she wishes with each witness, so long as the maximum allotted time is not exceeded. One or more assessors will be assigned to observe and evaluate the witness interview by the candidate. Following the last interview, the candidate will be given 30 minutes to prepare a written supplemental report detailing the results of the follow-up investigation; recommendations as to additional investigation to be conducted as well as evidence to be obtained and processed; and conclusions regarding the probable identification of the person(s) responsible. The assessor assigned to observe the interviews will also be assigned to review the follow-up report and will evaluate the candidate on the skill demonstrated in the interviews themselves as well as the
Typical Assessment Center Exercises
63
candidate’s conclusions contained in the written report. This is an excellent exercise to evaluate personnel being considered for investigative positions. However, one drawback is the length of time required to complete the exercise, and the amount of time needed by the assessor to observe and evaluate the candidate. For this reason, it is recommended that only one assessor be assigned to this exercise, and that one hour be allowed for each candidate participating in this exercise. To do well in this exercise, candidates need to have good interviewing skills as well as the ability to organize and plan their work. They also need to have good analytical ability to aid them in arriving at reasonable conclusions from complex or ambiguous information. Finally, good written skills are required in this exercise. The CAreer inTerVieW eXerCise
Most assessment center exercises place the candidate in the position for which he or she is being considered so as to evaluate how the candidate might react when confronted with situations that are common to that position. In this way, candidates may be evaluated on their ability to do a job, rather than on their past performance in a particular position. The Career Interview Exercise differs from most other assessment center exercises since the candidate is simply being asked to tell the assessors about himself or herself and to describe in their own words why they believe they are qualified for the position for which they are being evaluated. In effect, the Career Interview Exercise is similar to a structured oral interview in which candi-
The Career Interview Exercise gives candidates a chance to sell themselves.
64
The Assessment Center Handbook for Police and Fire Personnel
dates are asked a series of questions designed to test their job knowledge and decision-making ability. In the Career Interview Exercise, however, the emphasis is placed on allowing the candidate to speak in his or her words about why he or she is qualified for the position. In this exercise, candidates should be given 15 to 30 minutes to prepare a 15-minute presentation to the assessors focusing on their qualifications to be promoted. They may be given flip charts to aid them in making their presentation. Candidates should not be given the topic of the presentation until they are to begin preparing for it. Otherwise, some candidates will have a greater advantage over others by having more time to prepare their presentation. In addition, candidates should not be allowed to bring prepared resumes, college transcripts, copies of papers they have written, awards, certificates, or other materials with them to give to the assessors. The only thing they should be allowed to present to their assessors is the material they prepare during the preparation period immediately before the exercise. Candidates should be admonished to allow time for questions from the assessors, and their failure to do so should go against them in the assessors’ evaluation of them. Assessors should be allowed to ask questions limited to the information contained in their presentation and should not be allowed to ask questions of an unrelated nature. However, one or two “stock” questions may be asked, such as “Describe your strengths and your weaknesses” or “What have you done to prepare yourself for promotion?” as long as these same questions are asked of all candidates. There are three distinct advantages to this exercise. One is that candidates sometimes feel that the assessors did not really get to know them. “I would like to have been able to tell them something about myself” is a common complaint. In addition, assessors are often curious, after seeing a candidate in several role-playing exercises, about a candidate’s background, experience, education, or training. This exercise sometimes helps to give the assessors a more rounded and complete picture of the candidate. Finally, this exercise may be an advantage to the candidate who has truly prepared himself or herself for promotion through study and various job experiences. The candidate who is simply going through the process and who has not really made an effort to prepare himself or herself for promotion will find themselves at a disadvantage in this exercise. In this exercise, candidates should keep the following points in mind. First, they should never hesitate to talk about their strengths, but they should also not try to hide their weaknesses. The candidate who says, “I can’t really think of any weakness” is not being honest with the assessors, and they will know this. Second, since time is limited, candidates should not try to list every specific accomplishment of their career, but rather hit only the highlights. Third, they should keep in mind that they are being considered for
Typical Assessment Center Exercises
65
promotion to a higher position. They should therefore frame their experiences in terms of how they have prepared themselves for a position of higher responsibility in the organization. The fact that they made more arrests for driving while intoxicated last year than anyone else speaks to their initiative, but not necessarily to their ability to be a good supervisor. Finally, they should keep in mind that the assessors are not their enemies, but are simply people who have been selected to evaluate them and their potential for promotion. Most of them have gone through the same thing they are going through and understand their anxiety, fear, and frustration. Rely on the assessors to be fair in their evaluation and the candidate will not be disappointed.
Good planning is the key to writing a good report.
The ACCidenT reVieW boArd eXerCise
Police supervisors as well as fire company officers are often called upon to investigate accidents involving their subordinates and to make recommendations to higher authority concerning corrective action as well as discipline. In the Accident Review Board Exercise, candidates are placed in the position of investigating an accident involving one of their subordinates and making both a written and an oral report to an Accident Review Board (the assessors). This exercise is somewhat time-consuming and complicated to administer since it involves several role players as well as time for the candidates to write their report.
66
The Assessment Center Handbook for Police and Fire Personnel
Candidates are first given a copy of an accident report submitted by one of their subordinates. After they have had the opportunity to review the report thoroughly (usually 10 minutes), they are then allowed 10 minutes to interview the officer or firefighter who was involved in the accident. They are allowed an additional 20 minutes to interview two different witnesses to the accident, each of whom saw the accident from a different vantage point and presumably may have different information about the accident. After the candidate has interviewed the officer or firefighter and both witnesses, he or she has an additional 30 minutes to complete their written report. This report is then reviewed by the assessors, sitting as an Accident Review Board. Finally, the candidate is expected to make an oral presentation to the Accident Review Board to explain and justify the conclusions reached and recommendations made in the written report. This is an excellent exercise to test a candidate’s problem-solving skills as well as his or her ability to conduct interviews and to draw reasonable and logical conclusions from the information obtained as a result of these interviews. However, time considerations make it difficult to administer this exercise to more than a few candidates at one time.
Chapter 5 AssessmenT CenTer desiGn And AdminisTrATion
T
he design and administration of an assessment center are tasks that are normally assigned to Personnel Specialists or consultants who are trained and have extensive experience in this field. Due to the complex nature of the process and the importance attached to the results to be achieved, it is always recommended that someone who is familiar with the assessment center process and who has had experience in designing and administering assessment centers be assigned this responsibility. In addition, there is usually more credibility attached to a process if it is designed and administered by someone from outside the police or fire department and who has no vested interests in the outcome. In most cases, however, it will be desirable to have available assistance from one or more staff persons from the police or fire department to assist in administrating the process. The imPorTAnCe of Good PlAnninG
Few good things happen accidentally and a successful assessment center is virtually impossible to achieve without good planning, Well-designed plans are necessary to ensure that the assessment center is content-valid, job-related, and properly executed and that the results can be defended if they are legally challenged. A number of things must be considered in the planning process, including the following: 1. The scope, nature and method of the job analysis that will be needed to support the design of the exercises.
67
68
The Assessment Center Handbook for Police and Fire Personnel
2. The number of candidates to be evaluated and the time frame in which the process will be completed. 3. The facilities that will be required for the administration of the assessment center. 4. The selection and training of the assessors. 5. Methods of candidate preparation and feedback.
Good planning is the key or a successful assessment center.
These are but some of the issues that must be considered well in advance of the design of the assessment center process. Failure to adequately take these issues into consideration will almost surely result in problems either while the process is being administered or subsequent to the process.1 eXerCise desiGn
The exercises included in the assessment center should be based upon either a thorough job analysis or a firm understanding of the duties and responsibilities of the position for which candidates are being evaluated. If no job analysis has been conducted for this position within the last five years, it is recommended that a formal job analysis be conducted. This can be performed by members of the Human Resources Division staff or by the consultant who has been retained to administer the assessment center. The job analysis should produce information about the typical tasks performed by persons in the position as well as the critical skills and abilities needed to perform the duties of the position in a satisfactory manner. For example, the typical tasks that may be identified in a job analysis for Police Sergeant might include counseling subordinates on their performance, inter-
See Cam Caldwell, George C. Thompson and Melissa L. Gruys, “Ten Classical Assessment Center Errors: Challenges to Selection Validity,” Public Personnel Management (Spring, 2003), Vol. 32, No. 1., pp. 73–88.
1
Assessment Center Design and Administration
69
viewing crime victims and witnesses, training new officers, disseminating information and policies to subordinates, and preparing written reports. Similarly, the typical skills and abilities needed to perform in this position might be human relations ability, judgement and reasoning, decision-making, oral communications, and problem analysis. These tasks, skills and abilities form the basis for an assessment center for the position of Police Sergeant. These same skills might be needed by persons competing for the position of Fire Lieutenant or Captain. It is important to keep in mind that if the promotional process is ever challenged, it will be necessary to demonstrate a link between the exercises included in the assessment center and the criteria upon which they are being evaluated and the tasks actually performed by persons in that position.
A job analysis helps us know what the job is all about.
It is important that a relationship can be demonstrated between the tasks performed by persons in the position for whom candidates are being tested and the performance dimensions or evaluation criteria used in evaluating candidates.2 For example, if one of the performance dimensions used in evaluating candidates in an assessment center for police sergeant is “problem analysis,” it must be demonstrated empirically that there is a relationship between that dimension and specific job tasks. A well-designed Job Analysis Questionnaire can provide the information needed to establish this relationship. Figure 5.1 illustrates how a relationship between job tasks and performance dimensions can be demonstrated through job analysis data. 2
Ibid, p. 75.
70
The Assessment Center Handbook for Police and Fire Personnel
The number of exercises to be included in an assessment center should be based upon two factors. First, the exercises should be sufficient to represent generally the typical tasks performed by a person in the position for which candidates are being evaluated. The more exercises that are included, the better will assessors be able to evaluate the suitability of the candidates for the position. The second factor to take into consideration is the number of candidates to be evaluated, the length of time available to evaluate them, and the costs involved. Time is money and the more time that is devoted to the process, the more costly the process becomes.
Figure 5.1. Relationships between Performance Dimensions and Job Tasks.
Assessment Center Design and Administration
71
As a general rule, no less than three exercises, and no more than five, should be included in the assessment center. More than five exercises may be desired in some cases for executive-level positions such as Fire Chief, Police Chief, or City Manager. Four exercises are about the optimum number of exercises to use in most promotional processes. Every effort should be made to design exercises around the actual organization and the community in which the position is being performed. It is important candidates feel that they are dealing with things, persons and situations that are pertinent to their own organization. Candidates for the position of Captain in a Sheriff’s department, for example, should not be given an In-Basket Exercise in which their immediate superior is shown as the Chief of Police or City Manager, since these positions would not be pertinent to a Sheriff’s department. If a candidate is being asked to have a shift or company meeting with subordinate personnel, he or she should be given information and directives that are relevant to the actual police or fire department, as opposed to topics that are merely generic in nature. This reinforces the value of the job analysis as a means of ensuring that the process is as realistic and job-related as possible. It also reinforces the need to have all exercise materials reviewed by a Subject Matter Expert before they are used. If the candidate is assigned to go over an unsatisfactory performance evaluation report with a subordinate, a staff person should be asked to prepare an actual performance evaluation report on a fictitious employee. The performance evaluation report would be identical to that actually used by the organization. On the other hand, if the Department does not use a standardized performance evaluation report, the candidate may simply be given a narrative description of an unsatisfactory employee and told to speak with the employee about his or her poor job performance. Candidates need to feel that the process was a true and accurate reflection of the requirements of the job for which they were being evaluated. Good planning and thorough preparation can help to make sure this happens. CAndidATe orienTATion
An assessment center is a difficult and challenging process and there is no need to go out of the way to make the process even more difficult than it already is. If anything, the assessment center administrator should try to provide as smooth a road as possible for the candidates who are about to participate in the assessment center. This is not to say that the process should be made too easy or that there should be no pressure applied, but rather the process should be designed in such a way as to allow each and
72
The Assessment Center Handbook for Police and Fire Personnel
every candidate the opportunity to do his or her best in the process. One way to do this is to have an orientation session for the candidates before the process begins.
The orientation session helps candidates get ready for what lies ahead.
During the orientation session, it is a good idea to have a copy of the exercise schedule available and review it with the candidates so that they will know exactly what will be expected of them, where they need to be, for how long, and at what time. It is also a good idea to review each of the exercises with the candidates in order to let them know something about the exercise itself so that they will have a better understanding of what is expected of them before they arrive. If possible, candidates should also be provided with information on the criteria upon which they will be evaluated so that they will have a good understanding of what dimensions of their performance will be evaluated by the assessors. Copies of recent articles about the assessment center process may also be provided to the candidates. As a general rule, the more information about the process that can be given to the candidates during the orientation session, the greater degree of acceptance and satisfaction with the results of the process can be expected from the candidates. However, it should be pointed out to the candidates that it is just as bad to overprepare for an assessment center as it is to fail to prepare at all. Candidates who enter into an assessment center process thinking they know all the answers and anticipating the likely outcome of an exercise may often fail to listen to or read instructions carefully. As a result, they may fare very poorly. Thus, while
Assessment Center Design and Administration
73
preparation is important, candidates should be instructed to keep an open mind as they enter into the process and not assume anything. There are those who feel that candidates should be told as little as possible about the process before it begins because they believe that “battling the unknown” is one of the real challenges in an assessment center. Candidates who have been through the process often share this view because they do not want to lose the edge they may have over other candidates with no experience with assessment centers. Both of these views seem shortsighted, however, because an assessment center is a test of what a person can do, not what they know or do not know about the process. Even when very specific suggestions are provided to candidates beforehand, experience has shown that some of them will perform well and that others will not. This is more a reflection of their skills and ability rather than anything provided to them in an orientation session. Should the orientation session be optional or mandatory? This is a good question and the answer depends on your own point of view. It seems rather foolish to require a person to attend a session that is intended to help that candidate prepare for a very important event in his or her career. Candidates with good potential and initiative and who are career-oriented probably will not need to be required to attend an orientation session just as candidates preparing for a written examination do not usually wait until someone provides them the books they need to study for the examination. They do it because they are highly motivated and are looking for every advantage they can get in preparing for the examination. Candidates who attend an orientation session for an assessment center solely because they are required to will probably realize little benefit from the information provided to them. As a part of the orientation session, candidates should be instructed on what tools they may need to bring with them to the assessment center. At a minimum, these would include writing tablets and pens or pencils, unless these will be provided for them. A calendar, a spell-checker, and a dictionary or thesaurus may also be helpful. fACiliTies And eqUiPmenT
The success of any assessment center depends, in part, upon having suitable facilities in which to conduct the assessment exercises. In some cases, these facilities may be found within the police or fire department or in the city hall. If not, it may be necessary to procure suitable space in other community buildings, such as the public library, school classrooms, parks and recreation buildings, community centers, and the like. In still other cases, private organizations, such as churches or church-affiliated confer-
74
The Assessment Center Handbook for Police and Fire Personnel
ence centers, may provide the necessary space. In still other cases, the desired space may need to be procured for a reasonable fee from local restaurants, hotels, or banquet halls. It is important that the assessment center administrator determine the desired number and type of rooms needed for the assessment center well in advance so that the necessary arrangements can be made without a problem. If possible, it is always a good idea for the assessment center administrator to personally inspect the proposed facilities in advance so that there will be no unpleasant last minute surprises. The kinds of facilities needed to successfully conduct an assessment center will vary with the number of candidates involved and the number and type of exercises to be included in the process. In general, however, the following would be considered the minimum necessary requirements: • The Exercise Room is where all or most of the exercises are conducted. As such, it should be ample in size and configuration to accommodate a variety of purposes. A large conference room or small classroom measuring approximately 20 by 40 feet in dimension would normally be suitable for this purpose. If more than one exercise is taking place at the same time, such as under the “dual panel” approach described elsewhere in this chapter, two rooms of the same size would be required for this purpose. This room should be furnished with a suitable number of chairs and tables for the assessors and the candidates, as well as with other needed equipment, such as a podium, a dry erase board, an overhead projector, and the like. There should also be a workplace for the assessment center administrator and a chair for the video camera operator, if someone other than the assessment center administrator is assigned to operate the video camera. Appendix Q contains recommended room configurations for several of the exercises contained in this book. • The Preparation Room is a room where two or more candidates may be seated during the time they are preparing for their next exercise. A medium-sized conference room would usually be suitable for this purpose. This room should be equipped with individual tables and chairs or desks and chairs for two or three candidates. Flip charts, marking pens, and other materials that may be needed by the candidates in preparing for the next exercise should also be provided. • The Break Room is a place where candidates may gather while they are not actually engaged in or preparing for an exercise. This room should normally be comfortably furnished to allow candidates to relax as much as possible before they are needed to begin preparing for their next exercise. Coffee, soft drinks, rolls and other refreshments may be provided in the Break Room.
Assessment Center Design and Administration
75
Figure 5.2 provides an example of a simple checklist that may be used to identify the facility and equipment needed for an assessment center.
Figure 5.2. List of Facilities and Equipment.
sChedUlinG meThods
One of the disadvantages of the assessment center method is that it is time-consuming and therefore costly. The more people there are to be in-
76
The Assessment Center Handbook for Police and Fire Personnel
cluded in an assessment center, the greater the cost will be. There are, however, ways to increase the number of candidates to be evaluated without substantially increasing the total costs. A typical assessment center will usually involve three or four different exercises and can normally accommodate five or six candidates in a single day. Between 15 and 20 minutes should be allocated for each candidate in any individual exercise. An additional 10 to 15 minutes need to be allowed for candidate evaluation and scoring by the assessors. As a result, one individual exercise involving six candidates would require between two and one-half and three hours to complete. In this way, three or four individual exercises, or three individual and one group exercise, could be accommodated in a single day (realizing that a full day for the assessors is usually more than eight hours). Appendix R provides an example of a typical oneday assessment center for five candidates involved in four exercises.
Plan ahead when preparing for an assessment center.
Additional candidates can be included in an assessment center either by increasing the time allowed, by reducing the number of exercises, or by increasing the number of assessors assigned. Most assessor panels include three or four assessors, but assessor panels involving two assessors or even one is not unusual, depending upon the nature of the exercise. For example, one assessor might be assigned to evaluate and score written papers or in-baskets, while another panel of two or three assessors might be evaluating candidates in other exercises at the same time. This would increase the number of facilities required, but that is a minor logistics problem.
Assessment Center Design and Administration
77
It is also possible to double the number of candidates by having two panels of two or three assessors each and assigning each panel to observe and score candidates in one or two exercises. For example, eight candidates could be evaluated in a single day by assigning one panel to evaluate them on two exercises and another panel to evaluate them on two other exercises. Since each panel sees all candidates on the same two exercises, consistency of scoring is maintained. Scores for the four exercises are then combined to produce a final overall score. The number of candidates can be increased by a magnitude of two, once again, by doubling the number of assessor panels and assigning each panel to a single exercise. In this way, 16 candidates can be evaluated in a single day using four different panels of two or three assessors each. Once again, this creates a problem of having suitable facilities, since four different Exercise Rooms would be required and two or more Preparation Rooms would also be required. mAinTAininG TesT seCUriTY
As in any other kind of examination, test security is important in an assessment center. Due to the fact that candidates in an assessment center are free to mingle between exercises, there is an opportunity for them to share their experiences with each other. This may lead to one candidate revealing to another candidate what to expect in a particular exercise. This is undesirable because it may allow one candidate to have an unfair advantage over another, even when it is the candidates themselves who create this problem.
Test security is very important when administering an assessment center.
During the candidate orientation session, the importance of test security should be stressed. It should be pointed out to the candidates that they are
78
The Assessment Center Handbook for Police and Fire Personnel
only creating problems for themselves if they discuss the exercises with other candidates during the process. It is also not a good idea to change exercises from one group to the next. It is therefore to everyone’s advantage if the candidates do not reveal details of the exercises to other candidates until the process has been completed. In some agencies, candidates may be issued a direct order by the Chief of the Department not to discuss the exercises with any candidate who has not completed the process. It may also be a good idea to have each candidate sign a statement agreeing not to discuss the results of the process with anyone else until after the last candidate has been evaluated. Violation of this agreement may result in charges being filed against someone and could lead to that candidate being eliminated from consideration for promotion. This is a rather forceful way of trying to get candidates to do something that is for their own good. When large numbers of candidates are being tested over two or three days, it may be possible to change certain aspects of the exercise so that one group of candidates will not have more time to prepare than another. For example, in the case of a Company or Shift Training Exercise, different topics may be randomly assigned to candidates so that they will have no way of knowing what topic they may be assigned. It is important, though, that all topics are equally difficult. sTAff sUPPorT
Local staff support is absolutely essential to the successful administration of any assessment center. In some cases, local staff may be unable or unwilling to assist in the administration of the exercises because they desire to keep the process “untainted” by local involvement, or because they lack the personnel necessary. In such instances, the assessment center administrator will need to provide the support staff, which will usually result in greater costs to the client agency. In a small assessment center involving only a few candidates, the requirements for support staff will be minimal. However, there are several functions in which the assessment center administrator may not wish to or be able to become involved. Video Camera operators. If the exercises are to be videotaped, someone will be needed to operate the video camera. Since the assessment center administrator has many other duties to perform, it is always good to have someone else assigned to operate the video camera and maintain custody of the videotapes.
Assessment Center Design and Administration
79
It is a good idea to record all parts of the assessment center.
resource Persons. If a tactical fire or police problem is to be included, one person will need to serve as the Resource Person. This person should be knowledgeable about the local department’s standard operating procedures and radio protocols. In some cases, the assessment center administrator or one of the assessors may perform this duty, but it is preferable to have someone else to be assigned to this function. For example, it may be possible to use a retired police officer or firefighter or communications operator from the same department to serve in this capacity. In any event, it should be someone who is thoroughly familiar with the manner in which that department operates so that they may respond accurately to requests for information and assistance from the candidate during the exercise. In addition, the Resource Person can assist the assessors in determining if the actions taken by the candidates during the exercise were logical and appropriate. role Players. Role players are absolutely essential in nearly all assessment centers. Role Players may be used to play the role of police officers or firefighters in a Shift or Company Meeting or Training Exercise; as news media representatives in a News Media Exercise; as community representatives in a Community Meeting or Presentation Exercise; as business owners and managers in a Fire Prevention Inspection Exercise; or as staff members in a Staff Meeting Exercise. While assessors may be used as role players, it is difficult for them to remain objective and impartial if they are participants in an exercise. In addition, it is difficult for them to take accurate notes when they are involved as role players. It is important that role players not “overact” their assigned role to the point that the candidate finds it impossible to achieve the desired goal of the exercise. In a Shift or Company Meeting Exercise, for example, role players should not be so belligerent or obstinate that the candidate is not able to maintain control of the meeting no matter how hard he or she tries. While it is permissible to place obstacles in the path of the candidate, it is
80
The Assessment Center Handbook for Police and Fire Personnel
important that these obstacles not be so difficult as to be insurmountable. While role players should not be passive in their interactions with the candidates, they should not be so difficult as to make it impossible for the candidate to perform satisfactorily in the exercise. The process is difficult enough on the candidates without making it impossible for them to perform satisfactorily regardless of what they do.
Role players are often used to inject realism into the exercises.
Use of modern TeChnoloGY
It is important that assessment centers, to the extent possible, accurately and realistically reflect the actual working conditions surrounding the position or positions for which candidates are being evaluated. While it is not possible to test the firefighting skills of candidates for Fire Lieutenant, Captain, or Battalion Chief by setting fire to an actual structure, it is possible to simulate this by using computer software. This is but one example of how modern technology can be used to enhance the realism of assessment center exercises. This and other examples will be discussed below. fire simulators Computers are often used to create realistic simulations of fire-related and EMS-related problems. There are several different commercial software programs available, averaging about $1,000 each, that can be used for this purpose. Most of these programs come with a series of photographs of residential, commercial, and retail structures, but most departments opt to import photographs of local structures using digital photography. The software allows the computer operator to display a photograph of the building on a large screen and to place fire, smoke, victims, and other objects on the
Assessment Center Design and Administration
81
screen. Both fire and smoke can be either animated or static images. In the former case, they can be made to grow larger or diminish, based upon the candidate’s tactical decisions. Audible features, such as radio traffic, sirens, cries for help, or power saws, can also be employed. The computerized simulation offers an advantage over other forms of tactical problems inasmuch as the condition of the fire can be changed to fit the candidate’s actions.
Computerized fire simulators enhance the tactical fire problem.
The electronic in-basket Although most In-Basket exercises are of the paper-and-pencil variety, it is possible to create a computerized In-Basket whereby candidates receive all or some items in the form of e-mail messages and are required to respond to them in the same way. Since most police and fire departments are computer-oriented and utilize electronic mail extensively, candidates may be expected to be computer-proficient and thus it would not be unreasonable to expect them to respond to all messages by way of the computer. This would require a sufficient number of computer work stations for candidates to use simultaneously. If there were a large number of candidates, this might pose a problem unless the exercises are administered at different times for different groups of candidates. Setting up and administering the electronic in-basket will usually require the assistance of an Information Technology (IT) staff person. This person may be assigned to the City’s Information Technology staff or may be a staff person assigned to the police or fire department. In any event, this person will be needed to convert in-basket items into e-mails and to design the method by which the electronic in-basket can be administered. For example, it may be necessary to set up “dummy” email accounts for candidates
82
The Assessment Center Handbook for Police and Fire Personnel
to use during the In-Basket Exercise. In addition, an IT staff person will usually be networked with the candidates’ work stations so that during the exercise the IT staff person can send additional e-mail messages to the candidates and can receive and store email messages received from the candidates so that they can later be retrieved and evaluated by the assessors. The obvious advantage of the electronic in-basket is that it allows the candidates to work in an environment that is more like the way they actually do business. With the traditional pencil and paper in-basket, candidates often complain that they spend too much time writing out lengthy memoranda which is “not really like the way we work.” An added advantage of the electronic in-basket is that, since most or all of the messages prepared by the candidates are typewritten, they are much easier to read and assessors spend relatively little time trying to figure out what a candidate wrote or what they meant. Candidates also report that they are able to work quicker with the electronic in-basket, thus helping to ensure that they complete the exercise in the allotted time. The in-Car Camera Many police departments have by now recognized the value of installing audio-video recording devices in marked police cars so that critical encounters with the public can be recorded for later evaluation. While some employee groups have objected to these devices as an “invasion of privacy,” the reality is that these devices, more often than not, are responsible for authenticating and documenting an officer’s actions in a critical situation and have cleared many police officers of unjustified and malicious charges of misconduct and police brutality. These same devices can be useful in providing additional information for a candidate to view and evaluate during a role playing exercise in an assessment center. In an assessment center, candidates are often confronted with a roleplaying situation in which they are expected to interview a “citizen” who may have some kind of complaint about the level or quality of police services. In many cases, the officer, playing the role of a supervisor or command officer, will be told a story by the citizen in which the citizen alleges improper conduct by an officer (e.g., unprofessional, rudeness, name-calling, etc.). The candidate may then be expected to interview another role player playing the role of the officer in question. In the second interview, the candidate is expected to try to make an independent determination about whether the allegations made by the citizen are justified. One critical piece of evidence would be the recording made by the in-car camera during the police-citizen
Assessment Center Design and Administration
83
In-car cameras have become a valuable tool for the police officer today. Photograph courtesy of LaGrange Police Department, LaGrange, Illinois.
encounter. For this technology to be used successfully, it is necessary to first of all prepare a script that will outline the role, actions, and statements of the offending officer as well as the actions and statements of the citizen making the complaint. It is then necessary to find persons who will be willing to play the role of the offending officer as well as the citizen during the recording session. These same two role players must also be available during the actual assessment center because they will be expected to role play with the candidates during the assessment center. The host agency will be asked to provide a police uniform to be worn by the person playing the role of the offending police officer. In addition, the host agency must be willing to provide a marked patrol car equipped with an in-car recording device as well as a ranking officer to assist with the videotaping process. One typical scenario is for the person playing the role of the citizen to be stopped by the officer while driving his car in an apparently legal and proper manner. Once the emergency lights and equipment of the police car are activated, the audio and video recording device is activated and the entire encounter between the “officer” and the ”citizen” will be recorded. During the encounter, the person playing the role of the officer will engage in certain behavior that is clearly inconsistent with the policies and procedures of the police agency (e.g., making an illegal search of the car, using
84
The Assessment Center Handbook for Police and Fire Personnel
unnecessary force against the citizen, being rude and using derogatory language toward the citizen, etc). It may take several “takes” to get the desired effect and to ensure that the actions that will be complained of later by the citizen role player are clearly documented in the audio and video recoding. The candidate will first be expected to interview the “citizen,” who will allude to the candidate some, but not all, of the things the officer did during the encounter that upset the citizen to the point that he or she decided to make an official complaint against the officer. After interviewing the citizen, the candidate will be allowed to view the video and audio recording of the event and to make notes on what he or she sees and hears while watching the recording. Finally, the candidate will be expected to interview the person playing the role of the officer and will be expected to use the information provided by the citizen as well as the information provided by the video and audio recording to critique the officer’s performance and to formulate a plan for correcting the officer’s behavior. UsinG VideoTAPe or dVds To simUlATe reAliTY
As I have said before, an assessment center attempts to simulate reality, but none of the situations created in an assessment center are real. However, video and audio recording media such as videotape and DVD can and have been used successfully to enhance the realism of some of the exercises used for testing police and fire candidates for promotion. One way these recording media may be used is to record a role-playing situation that a candidate will later be asked to view and evaluate. For example, a candidate for fire captain may be shown a scenario recorded on videotape or DVD in which a male firefighter makes unwelcome advances toward a female firefighter. After watching the videotaped or DVD recording, the candidate may be asked to meet with the offending firefighter to discuss the department’s policy on sexual harassment and point out the firefighter’s misconduct. ConClUsion
If assessment centers are to continue to receive the support they have earned over the years, they must continue to be relevant and contemporary in nature. This can only happen by plan and by design, as is the case of just about anything worthwhile.
Chapter 6 seleCTion And TrAininG of Assessors
T
he selection, training, and supervision of assessors have a direct and important impact on the quality and results of the end product of the assessment center process. Regardless of how well an assessment center is designed, the assessors are the keys to ensuring that the process comes together. The single objective in any assessment center process is to render a fair and objective evaluation of the candidates. Only the assessors can make this happen. While other things, such as a well-designed job analysis, or the proper design of the exercises, contribute to an accurate evaluation of the candidates, only the assessors can assure that candidates are accurately and fairly evaluated. Assessor seleCTion
Typically, assessors are persons who have relevant experience and training in the field in which the candidates are employed. In most cases, an assessment center for a Police Sergeant or Police Lieutenant examination will employ persons who have served in those positions. It is usually desirable that assessors have experience in a position one step higher than the rank for which candidates are being considered. In a fire department, for example, an assessment center for a Fire Captain or Fire Lieutenant would usually employ assessors who are currently serving, or who have previously served, as Battalion, Division, or District Chief Officers. Whenever possible, assessors should come from outside the agency in which the assessment center is conducted, unless the agency is so large— such as a state police agency or a police or fire department having more than 1,000 commissioned personnel—that assessors do not personally 85
86
The Assessment Center Handbook for Police and Fire Personnel
know the candidates or have any direct working relationship with them. Police and fire departments that have used internal command personnel to serve as assessors for their own promotional processes have experienced the age-old problem of claims of bias and favoritism that tend to erode confidence in the process. This should be avoided at all costs.
Assessors should represent the diversity of the community.
If retired persons are used as assessors, it is important that they have maintained their knowledge of and familiarity with the field in which they were formerly employed either through continuing education programs, training seminars, or membership in professional organizations. A retired Deputy Chief who is out of touch with current trends and practices in either the police or fire service may not be the best person to evaluate the skills and abilities of a candidate for promotion in that field. The lack of current knowledge and familiarity with contemporary practices could very well become the basis for a challenge against the validity of assessment center results. It is not necessary that assessors be either trained or experienced, although this is always preferable. While mandatary certification of assessors is not yet widespread, it has begun in at least one state and may eventually become more common. Public Act 95–956, which became effective in 2008, made Illinois one of the first states to enact legislation requiring state certification of assessors used in fire department promotional assessment centers. This legislation, which amended the Illinois Fire Department Promotion Act (50 ILCS 742), now requires all assessors used in fire department promotional assessment centers to be certified through the Illinois Fire Marshal’s Office. In addition to certain minimum standards of experience and education, assessors must receive specialized training which meets the standards imposed by the Fire Marshal’s Office. In the long run, this kind of certification should serve to raise the standards governing the manner in which candidates are evaluated and will thus prove to be a benefit to the profession. The most important qualification for being an assessor is to be able to make sound and impartial judgements concerning the performance of can-
Selection and Training of Assessors
87
didates. Assessors need not be trained in psychology or any of the behavioral sciences, nor do they need to be experts in personnel evaluation. The most important qualification is that they have an understanding of and experience in the position for which candidates are being evaluated and that they are diligent in applying the evaluation techniques and procedures that are specified by the assessment center administrator. It is also important that the assessment panel reflect diversity, even if the pool of candidates does not. Diversity is a fact of life in the world in which we work and in the workplace, and it is important that the diversity of the community be reflected in the assessment center panel. Having diversity among assessors is not only the “right thing to do,” research has shown that it increases the validity of the results and decreases sub-group differences.1 Assessors may be either paid by the firm conducting the assessment center or by the client agency, or may serve without compensation. In the latter case, arrangement for assessors will usually be made by the client agency on a reciprocal basis. A police or fire department will sometimes agree to provide one or two personnel to serve as assessors if that jurisdiction will repay the favor sometime in the future. In many parts of the country, this is a common practice. From the point of view of the assessment center administrator, however, it is usually best if the arrangements for the assessors are made by the person conducting the assessment center and that they are compensated for their services. This is the best assurance that the assessment center administrator has that the assessors will be qualified and will be able to perform their assigned duties in a professional and competent manner. Assessors who are “volunteered” by their own agencies may or may not be qualified and may or may not be able to properly perform the duties assigned to them. Some assessment center administrators would prefer not to take this chance, since their reputation, as well as the integrity of the process, depends on this unknown quality of volunteer assessors. Assessor TrAininG
The length and type of training needed by assessors will depend upon their experience in the field as well as whether they have served previously as assessors and the number and type of exercises that will be involved in the assessment process. Some exercises, due to their technical nature, will require more assessor training than others. For example, a tactical police or 1 Michael D. Blair, “Best Practices in Assessment Centers: Reducing Group Differences to a Phrase for the Past,” paper presented at the 27th annual IPMAAC Conference on Personnel Assessment, Baltimore, Maryland, June, 2003.
88
The Assessment Center Handbook for Police and Fire Personnel
fire problem will require assessors to have a good working understanding of how that particular police or fire department responds to the kind of tactical situation portrayed in the exercise. Since police and fire departments vary considerably in the way they handle tactical problems, assessors should take nothing for granted in this regard. Similarly, how a candidate responds to items in an In-Basket Exercise, and the priorities the candidate assigns to those items, should be judged by the assessors on the basis of how such matters are handled by the local police or fire agency, not how the assessors might expect such matters to be handled in their own department. Generally speaking, the more technical the exercise, the greater the level of training of assessors that is required. Training can be provided by some combination of the following methods: • The Assessor manual. The Assessor Manual, as the name implies, is a manual given to the assessors several days in advance of the assessment center. It contains detailed descriptions of the exercises to be included in the assessment center as well as information on candidate rating procedures, to include the criteria upon which candidates will be evaluated. In addition, the Assessor Manual should contain background information about the police or fire department for which the process is being conducted and a job description of the position for which candidates are being evaluated. Any other technical matter to be made a part of the assessment center, such as General Orders, Employee Evaluation Forms, Inspection Reports, Incident Reports, and other documents, should be included as appendixes. Finally, copies of the rating forms that will be used in evaluating the candidates and other relevant forms and documents can also be included as appendixes. • Assessor orientation. An orientation session should be held for the
The Assessor Manual serves as the reference guide for the assessment center.
Selection and Training of Assessors
89
assessors prior to the assessment center. By this time the assessors should have reviewed the Assessor Manual and may have questions concerning one or more of the exercises. It is a good idea to have someone from the client agency on hand during the assessor orientation to answer any questions the assessor may have. If technical exercises, such as the tactical fire or police problem, are to be included in the assessment center, the orientation session affords the assessors the opportunity to review the problem with a member of the agency’s staff. This helps to ensure that the assessors will keep local operating procedures and practices in mind as they evaluate the candidates. • Assessor Training. Depending upon the expectations of the client agency, the skill and experience of the assessors, and the nature of the exercises, between two hours and eight hours may be set aside for assessor training. Assessor training is especially helpful when new assessors are involved, or when experienced assessors are involved but have never worked with the assessment center administrator. The following areas should be addressed during the assessor training sessions: • Organization and structure of the Agency • Duties and responsibilities of the position(s) for which candidates are being assessed • A review and discussion of each of the exercises • A review and discussion of evaluation criteria and rating methods • Instructions on assessor feedback methods2 • A review of the “Guidelines and Ethical Considerations for Assessment Center Operations” (see Appendix A). One author has offered the following as a model for assessor training:3 • Focus on dimensions and behaviors related to each dimension • Define behaviors along the continuum within each dimension • Focus on recording behavioral observations for feedback • Conduct practice sessions using “live” mock candidates • Provide feedback to the assessors regarding rating accuracy • Debrief practice exercises If role players are to be used in the exercises, they should attend the training session to rehearse their roles in exercises in which they will be in-
2 It may be helpful to show assessors a videotape of a previous candidate feedback session so that they will know what is expected of them.
90
The Assessment Center Handbook for Police and Fire Personnel
volved. If possible, agency staff should be available to answer questions and to participate in mock sessions allowing assessors to observe each of the exercises and to practice using the evaluation criteria and the scoring instruments. This will help to ensure consistency in the candidate evaluation process. Assessor training should be documented either by videotape or a written record of the training provided, the persons attending, and the amount of time devoted to different parts of the training program. This is helpful in setting up future training programs and may also be useful in the unlikely event that the validity of the assessment center is later challenged. CAre And nUrTUrinG of Assessors
The care and nurturing of assessors is an important responsibility of either the Exercise Coordinator or the host agency. Assessors may or may not be compensated for their services, but the work they perform is very important and the manner in which they perform their duties is directly related to the success of the final outcome. Every effort should be taken to treat the assessors with courtesy and helpfulness throughout the process. Assessors should be provided with writing tablets, pens and pencils, staplers, calculators, stop watches, paper clips, pencil sharpeners, and other supplies that will be needed through the day, along with candidate rating forms and instructions for their preparation. Depending upon the exercise, it may also be useful to provide assessors with copies of the department operating policies and procedures (e.g., the Fire Department’s Incident Command System, the Police Department’s Citizen Complaint Policy) as well as applicable rules, regulations, and labor contract provisions. The exercise schedule should not be so demanding that the assessors are physically and emotionally exhausted at the end of the day. If possible assessors should not be required to work longer than a normal working day (8–10 hours), nor should they be required to work after hours preparing narrative reports recording their observations and evaluations of candidates. Mid-day meals should be provided for assessors as well as coffee, tea, and rolls in the morning and water, soft drinks, and a light snack at the end of the day. Assessors may be expected to provide their own dinner but may be reimbursed for this cost by the host agency unless this provision is al-
3.
Blair, op. cit.
Selection and Training of Assessors
91
ready included in the vendor’s contract. If assessors are to commute from a long distance (an hour or more), hotel rooms should also be provided. A two-hour commute on top of a ten-hour work day plus meal breaks makes for a very long day, especially if the process is to run for several days. rUles of CondUCT for Assessors
The professional conduct of assessors is to be assumed and most assessors who have served in this capacity know that their conduct throughout the entire process must be above reproach. One false move by an assessor is enough to cast doubt and criticism on the entire process and no one wants this to happen. As a result, it is a good idea to issue “rules of conduct” for assessors. If they are experienced assessors they will not mind being reminded of what is expected of them, and if they are not experienced they will appreciate knowing what is expected of them. Assessors should remember at all times that the credibility of the assessment center will depend, in part, on the professional conduct and deportment of the assessors. Here are some general rules for assessors: • Personal Breaks. The schedule we follow is often a busy one and does not allow a great deal of time for personal breaks. Always check with the exercise coordinator to see if there is enough time for a break before taking one. Failure to do so may cause the process to become unnecessarily delayed. • Appearance of the workplace. You are expected to keep the workplace free from newspapers, magazines, personal correspondence, food containers and other material that leaves the candidate or the client with the impression that we are anything less than neat and orderly in our approach to our work. • Cell phones. We recognize that cell phones, pagers, and beepers are your lifeline to your office and home, but it is very distracting to have repeated interruptions of this nature during an assessment center. We must insist that during the administration of the assessment center you remain focused on your role and that you deal with these distractions in a manner that does not interfere with your work as an assessor.
Chapter 7 CAndidATe eVAlUATion, sCorinG, And feedbACk
T
here are several methods that can be used to score candidates in an assessment center. The method of choice depends upon the experience and preference of the assessment center administrator as well as the needs of the client organization. For example, some local jurisdictions may have civil service or local personnel rules that specify that candidates are ranked on an eligibility list for promotion, while others simply employ a “pass-fail” method whereby all candidates who receive a passing score are placed on an unranked list and are eligible for promotion based upon the preference of the appointing authority. In still other cases, the assessment center score may be a weighted part of the total examination score and thus may be only one component of the overall promotional process. Regardless of the end product of the evaluation process, it is important that assessors use some standardized and logical method of evaluating, scoring or ranking the candidates. To this end, candidates must be evaluated on some objective criteria which can be linked to the basic dimensions of the job for which they are being evaluated. There must also be a standardized rating scale to use in scoring candidates. Assessor noTe-TAkinG
It is important that assessors take accurate and detailed notes concerning what they see, hear and surmise during the assessment center exercises. Assessor notes become the foundation upon which the assessors’ evaluations are based. Assessors may use lined pads to take “free form” notes, or they may be provided with standardized forms or checklists to as92
Candidate Evaluation, Scoring, and Feedback
93
sist them in their note taking. The evaluation criteria that will be used in scoring the candidates should be available to the assessors so that they will have a clear understanding of what is expected of the candidates in each exercise.
Assessors do not need a crystal ball to learn about the candidates.
Assessor notes are intended to aid the assessors in recalling pertinent indicators of a candidate’s performance. It is not necessary that assessors record verbatim everything a candidate says or does, but rather to record those significant indicators that relate directly to the candidate’s overall performance. The benchmark characteristics described later in this chapter are a good way to focus on these significant performance indicators. Assessor notes should be retained by the assessment center administrator as a permanent part of the test record. They may be affixed to the examination record and should be retained as long as may be statutorily required. WeiGhTinG of eXerCise ComPonenTs
Should all components of the assessment center be weighted equally? The answer to this question will come from the results of the job analysis, which should be able to link exercises to job tasks and required skills and abilities. If the job analysis clearly shows a difference in the importance between some job tasks and others, and between some job dimensions (e.g., skills and abilities) and others, this would indicate that some exercise components may be weighted heavier than others. Assuming that no such differentiation is indicated by the job analysis, an equal weighting of exercise components would be preferred.
94
The Assessment Center Handbook for Police and Fire Personnel
For example, it might be determined that, since a Fire Captain spends a great deal more time in quarters supervising personnel than on the fire ground, an Employee Counseling Exercise should be given greater weight than a Tactical Fire Problem. On the other hand, it could be argued that there is greater potential liability associated with a fire than with a “problem employee.” Other reasons for weighting separate parts of the assessment center differently could include provisions in the labor contract, civil service rules and regulations, or court-ordered provisions governing the process. However, unless there are compelling reasons to do otherwise, and unless disparate weighting can be supported by a reliable job analysis, equal weighting of all components of the process is advised. mAinTAininG ConsisTenCY
It is essential that assessors maintain consistency in the evaluation of candidates. This is especially critical when multiple panels of assessors are used, or when a large group of candidates is being assessed by the same panel. Assessors cannot be too harsh or critical in their rating one day and more easy-going and considerate the next day. The very first candidate the assessors score in a process, in effect, sets the standard for the remainder of the process. While it is not necessary that assessors compare one candidate against another, they should remember the standards they used in the beginning of the process as they are nearing the end of the process. To this end, it is useful, if time permits, to have mock sessions or “dry runs” using position incumbents in exercises evaluated and scored by the assessors during the assessor orientation. This allows assessors to “get the feel” of the scoring system and rating scale and become comfortable with each other. In this way, if they see that their initial ratings are too high or too low, they can make adjustments before the actual exercises begin. When multiple assessors are employed, they have the advantage of multiple sets of eyes and ears and can pool their observations. What one assessor sees as “too rigid,” another may see as “firm.” What one assessor sees as “cold and dispassionate,” another may see as “cool and reserved.” In the end, they come up with the right assessment! The exercise coordinator should also carefully observe the scoring process and inform assessors if they appear to be deviating from the standard that has been established for the process. For example, if the assessors remark that a candidate did “very good” or “excellent” in an exercise, and then assign scores which are in the “average” or “acceptable” range, the assessment center administrator should point out this apparent discrepancy. While the assessors are expected to exercise their own judgment in scor-
Candidate Evaluation, Scoring, and Feedback
95
ing the candidates, it is the role of the exercise coordinator to ensure that they are evaluating the candidates in a fair and consistent manner and in the manner required by the established rating scale. In this way, the assessment center administrator functions as a quality control manager, ensuring that the standards that have been established for the process are met. seleCTinG The eVAlUATion CriTeriA
The criteria by which candidates are being evaluated should relate to the skills, abilities and characteristics that have been determined to be necessary for successful performance in the position for which the candidates are being considered. In most cases, these criteria will be determined through a job analysis of the position. If no formal job analysis is conducted, essential skills and abilities can usually be determined by reviewing the job description for the position, or by interviewing position incumbents or their supervisors. Still another way is to identify the most important tasks performed by position incumbents and determine the skills and abilities needed to perform those tasks. For example, a first-line supervisor in a police department or a fire department is often expected to counsel subordinate personnel regarding their job performance and to attempt to obtain their voluntary compliance in improving their job performance. Skills in problem-solving, human relations, and interpersonal communication would seem to be necessary for a person to be able to successfully perform this task. There must be a logical, if not documented, relationship between the criteria upon which candidates will be evaluated and the tasks performed by persons in the position. In addition, criteria must be clear and unambiguous and must be defined in such a way as to leave no misunderstanding by the assessors of what is meant by them. In addition, there must be a way to differentiate between superior, average, and unacceptable performance. In some cases, it may be desirable to design evaluation criteria, performance benchmarks and rating forms specifically based upon particular tasks. In this way, the evaluation criteria and performance benchmarks can be customized to meet specific local conditions and operating policies. An example of this is in the design of evaluation guidelines for the In-Basket Exercise (see Appendix C), discussed in Chapter 4. Here, a Subject Matter Expert (a ranking member of the agency) has been asked to review each item of the In-Basket and to prepare recommended actions and priorities for them. These recommended actions and priorities are used as guides by the assessors when evaluating the completed In-Baskets. Another example of how these evaluation criteria and performance benchmarks can be customized is in the Tactical Fire Problem. In this case,
96
The Assessment Center Handbook for Police and Fire Personnel
candidates may be judged on such criteria as organization and planning, problem analysis, and decision-making, but these are generic criteria. To make them more reflective of local conditions and expectations, a Subject Matter Expert might be asked to rate specific tactical considerations that may be essential to the particular problem. These will almost always vary from one agency to the next. After a Subject Matter Expert has been asked to evaluate each of the possible tactical considerations (see Appendix G), weights can be assigned to them on the basis of this evaluation. These weights can then be factored into the candidate evaluation form. The following are examples of criteria that might be used in evaluating candidates in an assessment center, along with a list of typical exercises in which these criteria might be used. Performance dimension
Typical exercises
leadership: Skill in motivating others toward the accomplishment of goals and in exerting a positive influence on their behavior.
• Group Problem-Solving • Employee Meeting • Shift or Company Meeting • Staff Meeting
human relations: Skill in establishing and maintaining effective working relationships with subordinates, superior officers and other persons
• Group Problem-Solving • Employee Meeting • Shift or Company Meeting • Citizen Interview • Community Presentation
oral Communication: The ability to express ideas and to communicate orally in a clear and understandable manner and in correct grammatical style.
• News Media • Fire Inspection • Shift or Company Training • Citizen Interview • Community Presentation
Written Communication: The ability to express ideas and to convey information in written form in a clear and understandable manner using correct grammar, sentence structure and punctuation.
• In-Basket • Program Development • Criminal Investigation • Employee Meeting (with a report detailing the results of the interview)
decision-making: The ability to make sound decisions and to commit oneself to a defined course of action.
• Tactical Fire or Police Problem • Citizen Interview • News Media • Criminal Investigation • Employee Meeting
Candidate Evaluation, Scoring, and Feedback Problem Analysis: The ability to correctly recognize and identify a problem and to develop reasonable and effective ways of solving the problem.
• Tactical Fire or Police Problem • Citizen Interview • In-Basket • Criminal Investigation • Group Problem-Solving • Program Development
initiative: The ability to actively influence events, to initiate required action, and to think and act independently.
• Tactical Fire or Police Problem • Citizen Interview • Community Presentation • Criminal Investigation • Group Problem-Solving • Program Development
Judgment and reasoning: The ability to reach logical conclusions from incomplete or ambiguous information and to use sound judgment in formulating plans, making decisions or taking action.
• Tactical Fire or Police Problem • Employee Meeting • Community Presentation • Criminal Investigation • Program Development
stress Tolerance: The ability to maintain an acceptable level of performance under pressure and/or opposition.
• Tactical Fire or Police Problem • Shift or Company Meeting or Training • Staff Meeting • News Media
97
This list is obviously not all-inclusive but it does serve to reflect the tremendous versatility of the assessment center process. PerformAnCe dimensions And benChmArks
If assessors are to reliably and consistently evaluate candidates on these evaluation criteria, they must be given guidelines so that they will know what to look for. It is essential that assessors have a thorough understanding of the criteria upon which candidates are to be evaluated and that they know what those criteria mean. As Maher and Michelson point out, different people may have different ideas of what “leadership” means to them.1 For this 1 Patrick T. Maher and Richard S. Michelson, Preparing for Fire Service Assessment Centers (Bellflower, Ca: Fire Publications, Inc., 1992), p. 21. .
98
The Assessment Center Handbook for Police and Fire Personnel
reason, it is necessary to provide “benchmarks” to indicate what to look for in evaluating the candidates. The use of benchmarks, or “behavioral checklists,” has been shown to increase the accuracy, reliability and validity of ratings and allows assessors more time to observe actual behavior.2 The following are examples of such benchmarks.
leadership: Skill in motivating others toward the accomplishment of goals and in exerting a positive influence on their behavior.
benchmarks: In evaluating performance in this dimension, assessors should determine whether the candidate: 1. Maintained a positive attitude throughout the exercise. 2. Exerted a positive influence over others. 3. Maintained an attitude of poise and self-confidence during the exercise. 4. Offered positive and constructive suggestions for dealing with the problem. 5. Took steps to maintain control of the exercise.
human relations: Skill in establishing and maintaining effective working relationships with subordinates, superior officers and other persons.
benchmarks: In evaluating performance in this dimension, assessors should determine whether the candidate: 1. Listened thoughtfully to what others had to say. 2. Solicited ideas and suggestions from others. 3. Demonstrated a concern for the views, opinions, and feelings of others. 4. Maintained a friendly and cooperative attitude during the exercise. 5. Remained tactful when presenting his own point of view or when arguing against the views of others.
2 Michael D. Blair, “Best Practices in Assessment Centers: Reducing Group Differences to a Phrase for the Past,” paper presented at the 27th annual IPMAAC Conference on Personnel Assessment, Baltimore, Maryland, June, 2003.
Candidate Evaluation, Scoring, and Feedback
99
oral Communication: The ability to express ideas and to communicate orally in a clear and understandable manner and in correct grammatical style.
benchmarks: In evaluating performance in this dimension, assessors should determine whether the candidate: 1. Spoke clearly and distinctly. 2. Used proper grammar, vocabulary, and syntax. 3. Expressed ideas clearly, completely, and intelligently. 4. Conveyed thoughts in a logical and well-organized manner. 5. Maintained good eye contact with others.
Written Communication: The ability to express ideas and to convey information in written form in a clear and understandable manner using correct grammar, sentence structure, and punctuation.
benchmarks: In evaluating performance in this dimension, assessors should determine whether the candidate: 1. Used proper grammar, spelling, and sentence construction in his/her written communications. 2. Wrote neatly and legibly. 3. Conveyed information in a clear and understandable manner. 4. Presented his/her ideas in a logical and well-organized manner. 5. Used proper format in conveying information.
Problem Analysis: The ability to correctly recognize and identify a problem and to develop reasonable and effective ways of solving the problem.
benchmarks: In evaluating performance in this dimension, assessors should determine whether the candidate: 1. Indicated a thorough and complete understanding of the problem. 2. Identified components of the problem as well as its symptoms. 3. Explored various alternatives for dealing with the problem. 4. Made logical and rational decisions based on a full understanding of the problem. 5. Offered unique or innovative ways for dealing with the problem.
100
The Assessment Center Handbook for Police and Fire Personnel The rATinG sCAle
In order to ensure uniformity in rating candidates, it is important that assessors use a structured method of scoring the performance of candidates. This can be done in several ways. The assessment center administrator must design a uniform method for scoring candidates on the evaluation criteria, however. The numerical scale Perhaps the simplest scale to use, the numerical scale simply assigns numerical values to evaluation criteria ranging from “Not Qualified” to “Outstanding” or “Superior.” A simple four-point scale, such as the one shown below, serves this purpose quite well. • 1.1 to 2.0—not qualified: The candidate has failed to demonstrate the minimum level of skill or ability believed to be necessary to successfully perform the duties of this position. • 2.1 to 3.0—minimally qualified: The candidate has demonstrated a level of skill or ability that is consistent with minimally acceptable standards of job performance in this position. • 3.1 to 4.0—Well qualified: The candidate has displayed a level of skill or ability consistent with an average to above-average level of job performance in this position. • 4.1 to 5.0—superior: This candidate has displayed a level of skill or ability that is consistent with an excellent or superior level of job performance in this position. Using this scale, assessors should be free to assign scores within each of the categories according to their judgements concerning the level of skill or ability demonstrated by the candidate. For example, assessors may assign a score of 4.1 to a candidate who is clearly better than well qualified, but who has not achieved the top of the superior range. Assessors should also be free to rate a candidate somewhere between 0.0 and 0.9 if they believe that his or her performance is clearly less than acceptable. They must use their own judgment in deciding just where, in that range, to score the candidate.
Candidate Evaluation, Scoring, and Feedback
101
It may be desirable to provide assessors a greater range of choices than is possible with a three-point scale. For example, a five-point scale, such as that shown below, allows assessors greater specificity in evaluating candidates. 5
The candidate’s performance clearly exceeds the requirements for this position. This candidate has displayed superior knowledge and/or ability in this area and would be able to perform all of the duties of the position ably and without exception.
4
The candidate’s performance somewhat exceeds the requirements for this position. This candidate has displayed above average knowledge and/or ability in this area and would be able to perform all or most of the duties of the position in an acceptable manner and without major difficulty.
3
The candidate’s performance meets the minimum requirements for this position. This candidate has average knowledge and/or ability in this area and should be able to perform most or all of the duties of the position in an acceptable manner and with minimum difficulty.
2
The candidate’s performance fails to meet the minimum requirements for this position. This candidate has below-average knowledge and/or ability in this area and might not be able to perform some or all of the duties of the position in an acceptable manner and without minimum difficulty.
1
The candidate’s answer or performance fails to meet the minimum requirements for this position. This candidate has substantially below-average knowledge and/or ability in this area and would not be able to perform most or all of the duties of the position in an acceptable manner and without minimum difficulty.
The rating scale must be designed in such a way that assessors are offered clear choices in rating the performance of the candidates. The raw score method Another way to score the assessment center is to use the raw scores achieved by a candidate in any one exercise divided by the maximum raw score for that exercise expressed as a percentage score. For example, let’s say that there are four dimensions in a single exercise and that there are five
102
The Assessment Center Handbook for Police and Fire Personnel
benchmarks for each dimension. Using a two-point scale in which candidates are given 2.0 points for each benchmark they achieve, 1.0 point for each benchmark they partially achieve, and 0.0 points for each benchmark they failed to achieve, the maximum raw score for the exercise would be 40 (4 performance dimensions times 5 benchmarks times 2 points for each benchmark). A candidate who receives a total raw score of 30 points in this exercise would receive a final score of 75 percent in this exercise (30 divided by 40 equals 75%). Let’s assume that there are four exercises in the assessment center and that each exercise is worth 40 points each and that each of the four exercises is to be weighted equally. A final raw score of 120 in the four exercises (an average of 30 points on each exercise) would result in a final assessment center score of 75 percent (120 divided by 160 equals 75%). Let’s now say that there are four exercises in the assessment center but that they are not weighted equally. The results might look something like Figure 7.1.
Figure 7.1. Example of Calculating Weighted Scores.
rePorTinG CAndidATe sCores
The Assessment Center Administrator is responsible for tabulating candidate scores, checking them for accuracy, and reporting them to the agency where they will presumably be posted along with other promotional criteria in the form of a final eligibility list for promotion. Each agency will have different requirements in terms of how the assessment center scores are posted, but in most cases, they will be reported as a final percent or raw score that can be combined with other elements such as written score, seniority, and performance evaluation to create a final promotional score.
Candidate Evaluation, Scoring, and Feedback
103
Figure 7.2. Example of a Detailed Breakdown of an Assessment Center Score.
Depending upon the agency, it may be either necessary or desirable to provide additional details about how the candidate’s final assessment center score is calculated. Figure 7.2 provides one example of how this information may be presented, but there are many other possibilities as well. It may also be useful to provide candidates with an indication of how their scores compare to others in their group or to the average of the group as a whole. Figure 7.3 shows how such a comparison might look.
104
The Assessment Center Handbook for Police and Fire Personnel
Figure 7.3. Sample Group Comparison of In-Basket Scores.
Assessor CommenTArY
Assessor comments are intended to provide candidates insight into how they were scored as well as to provide them with important information concerning how they may improve in the future. It is important that these oral comments be consistent with the candidates’ scores. If, for example, assessors gave a candidate an average score, or one that is below average, assessors should not describe the candidate’s performance as “very good” or “quite good.” Such a comment would imply a better than average score, which was not the case. Instead, assessors should use phrases such as “needs more work,” “minimally acceptable,” or “relatively weak” to describe performance that is only average or slightly above average. Here are some additional examples:
Candidate Evaluation, Scoring, and Feedback
105
overall Performance
Typical Comments
not qualified: This person demonstrates a level of skill or ability that is below average for this position. Further development of this skill or ability is required in order to achieve satisfactory job performance.
• Failed to perform at an acceptable level. • Well below the standard expected for this position. • Failed to adequately recognize and deal with the problem.
minimally qualified: This person has demonstrated an average to above average level of skill or ability for this position. The level of skill or ability exhibited indicates that the person should be able to perform in this position in an acceptable, but not highly proficient, manner.
• Performed at an acceptable level for this position. • Performance was minimally acceptable for someone in this position. • The candidate performed at a level that was only average for someone in this position.
Well qualified: This person has demonstrated an above average to very good level of skill or ability for this position. This person should be able to perform the duties of this position in a proficient manner.
• The candidate performed at a level well above the minimum standard for this position. • The candidate’s performance was well above average in this area. • The candidate did a very good job of dealing with the issues.
superior: This person has displayed an excellent level of skill or ability in this area. The person’s performance revealed very few flaws in this behavioral dimension. The candidate should be expected to perform the duties of the position in an exceptional manner.
• The candidate did an exceptional job of dealing with the issues. • The candidate displayed a rare ability in handling the problem. • The candidate performed at a level much higher than the standard for this position.
ProVidinG feedbACk To The CAndidATes
Candidate feedback is an important element in any assessment center. Candidates have a great stake in the outcome of any assessment center and they deserve to be informed of how well or how poorly they performed. In addition, they should be given some indication by the assessors of their
106
The Assessment Center Handbook for Police and Fire Personnel
strengths and weaknesses and how their performance may be improved in the future.
Candidates appreciate receiving feedback on their performance in an assessment center. Photograph courtesy of LaGrange Police Department, LaGrange, Illinois.
There are several ways to provide feedback to candidates. One way is to simply allow candidates to review the notes and scoring sheets prepared by the assessors. However, this method is not recommended since candidates would probably not be able to properly interpret assessors’ raw notes, nor are such notes intended for this purpose. A better way of providing candidate feedback is to have assessors complete standardized forms which describe significant characteristics of the candidate’s performance in each exercise. Such forms may contain a series of checklists to show whether the candidate did or did not do something that was to be expected, or they may simply ask the assessors to respond to specific questions about the candidate’s performance. Examples of these forms are shown in Appendix T. One disadvantage of this method is that checklists are not terribly descriptive and may leave questions in candidates’ minds about “what does this mean?” or “how can I improve?” A much better method of letting candidates know how and why they were rated in the assessment center process is to have assessors videotape their comments and later allow candidates to view these comments during scheduled debriefing sessions. In this way, assessors can speak directly to the
Candidate Evaluation, Scoring, and Feedback
107
candidates and provide constructive criticism about their performance in the exercises. These comments may be on the same videotape used to record the candidate in the exercises. In any videotaped candidate feedback session, assessors should be asked to review and comment on specific aspects of performance in each of the exercises, referring specifically to the evaluation criteria used in evaluating the candidates. For example, if the assessors are evaluating leadership, oral communication, and human relations skill in one of the exercises, they should be asked to provide feedback about particular things the candidate did or did not do that influenced their evaluation of the candidate. Assessors should ensure that their videotaped comments agree with the scores assigned to the candidate. If candidate debriefing sessions are used, they should be scheduled as soon after the assessment process as possible so that the experience is still fresh in the minds of the candidates. Debriefing sessions should be conducted by either the assessment center administrator or a member of his or her immediate staff who was involved in the design and administration of the assessment center. This helps to ensure that candidates understand that the person who designed and administered the assessment center is prepared to accept responsibility for the results and is willing to meet with the candidates and discuss the process with them. Candidate debriefing sessions are an excellent way to help candidates learn from the process and to reinforce their understanding of and support for the process. It provides them with insight into how their performance was rated and gives them excellent feedback to enhance their opportunity for future growth and development. In addition, it gives them the opportunity to vent their concerns or complaints about the process to the person who was directly involved in the design and administration of the process. Even though a candidate may not have fared well in the process, he or she will usually feel good about the opportunity to see and hear for themselves why the assessors rated them the way they did. It is important that debriefing sessions not be allowed to turn into gripe sessions. The purpose of these sessions is to allow candidates to know how and why the were scored by the assessors so that they may gain from this experience. It should be made clear to the candidates that these debriefing sessions are not intended to allow candidates to challenge the results of the process or to argue against the views of the assessors. While the agency may provide other opportunities to challenge the process, this is not the purpose of the debriefing session and candidates should be made aware of that in advance so that they will have no false expectations about what they are to derive from the session.
Chapter 8 PrePArinG for An AssessmenT CenTer
A
ssessment centers have gained a great deal of credibility in recent years as a valid, useful and fair means of evaluating the qualifications, skills, and abilities of persons for advancement within the police and fire service. A growing number of communities have come to the realization that advancing someone in rank represents a significant investment in the future of the agency, and that the costs associated with designing and implementing a reliable method of predicting who will do well in a higher position is a sound investment in the future of the agency. Today, many communities have made the assessment center an integral part of their promotional process in the police and fire service. Assessment centers are also widely used in selecting police chiefs, fire chiefs and other municipal department heads.
There is no substitute for good preparation.
Even though the assessment center process differs considerably from other kinds of testing methods, candidates should not be intimidated by what they have heard about the process from someone else. The process is de108
Preparing for an Assessment Center
109
signed to be challenging, difficult, and demanding, but it is also intended to represent a fair, objective, and unbiased test of a candidate’s ability to perform the job for which he or she is being considered. There are no magic formulas or mystic potions to assist candidates in the process. The most important tools are simple common sense and the native ability they have developed over the years as they have studied and prepared themselves for advancement. Those seeking promotion in the police or fire service know also that, if they have not participated in an assessment center in the past, the chances are good that they may have the opportunity sometime in the future. As a result, they are naturally curious and anxious to know as much as they can about the assessment center process so that they can better prepare themselves for the time that they may be a participant. To meet this demand, a number of consultants, training institutions, and others have offered classes on “how to prepare yourself for an assessment center,” and a number of articles have appeared in professional magazines on the same subject. Indeed, there is no limit on the amount of background information that can be obtained, or the instructional material that is available, for those who want to do their best when their time comes to participate in an assessment center. Unfortunately, much of the time and money spent by a prospective candidate to prepare himself or herself to participate in an assessment center is of little value and may even be detrimental. While the desire of persons to prepare themselves to perform at their best in an assessment center is certainly understandable, a few words of caution are in order. It should be remembered, first of all, that an assessment center is not like any other kind of test for which someone can truly prepare themselves. It is not like a written examination, where long hours of studying books and reference materials may help to assure a passing grade. Nor is an assessment center like an athletic competition where weeks and months of long, grueling workouts and physical conditioning will assure a participant of earning a first place medal. Instead, an assessment center is not really a test at all, but rather a process of evaluating the extent to which candidates possess (or do not possess) certain skills and abilities and how well they can (or cannot) perform certain tasks. There are no “right” or “wrong” answers in an assessment center although some behaviors are more acceptable than others. For example, it is always ”wrong” to lose your temper in any assessment center exercise, because such behavior indicates an inability to think and act effectively under stressful conditions. Similarly, it is equally “wrong” to fail to participate fully in any Group Problem-Solving Exercise, since this exercise is designed to see how well candidates can work with others to solve a problem. Performing well in an assessment center usually means choosing behaviors that are appro-
110
The Assessment Center Handbook for Police and Fire Personnel
priate for the situation. Therefore, understanding the scenario and the expected outcome is much more important than trying to second guess the assessors.1
An assessment center is hard but rewarding work.
While the kind of skill or ability being evaluated in an assessment center can be developed and refined, it cannot be acquired from reading a book or watching a videotape or even from attending classroom lectures. The kinds of skills and abilities measured in an assessment center are those that are developed over a period of time through experience and hard work. Some individuals possess some of these skills naturally, and refine them to a fine edge over time, while others simply never master them. Just as many of us were never destined to be big league baseball stars, astronauts, or worldfamous entertainers, not all persons have what it takes to be a supervisor, middle-manager or chief executive officer in a police or fire department. There are also many different versions of what constitutes an “assessment center.” While a true assessment center must meet certain exacting standards (see Appendix A), no two designers of assessment centers do things exactly the same way. Some assessment centers, for example, are very generic and consist of a standard set of exercises that are used almost without alteration from one place to another, regardless of the unique characteristics of the position or the organization. I like to call this the “one size fits all” type. Others, however, are much more position-oriented and are carefully designed so as to emulate the duties and responsibilities of the poSee Patrick T. Maher and Richard S. Michelson, Preparing for Fire Service Assessment Centers (Bellflower, CA: Fire Publications, Inc., 1992), pp., 28–29.
1
Preparing for an Assessment Center
111
sition and the characteristics of the organization. As a result, the exact nature of an assessment center will vary from one place to another and will depend largely upon the experience, training, and bias of the person or persons responsible for its design and administration. Accordingly, someone who wants to learn more about an assessment center will be heavily influenced in his or her beliefs about what constitutes successful performance in an assessment center by the person imparting the information, whether it is from an article in a professional journal, book, videotape, or classroom presentation. This information, instead of helping the candidate prepare for the process, may cause him or her to develop certain biases or misconceptions which may eventually prove fatal to his or her performance. For example, a “Leaderless Group Discussion” is a common part of many assessment centers. In this exercise, candidates are gathered into small groups for the purpose of discussing a topic or issue and trying to come up with an acceptable solution. There are many ways to “play” this exercise. Some candidates may try to jump out in front early on and establish a strong leadership position and channel the discussion toward a specific goal. Another person may be much more cautious and wait to see how the discussion is going before making a move. In the end, the more successful candidate is the one who has the most to do with how the group eventually solves the problem. It is not usually the person who dominates the group or who talks the loudest or the longest. In this exercise, what a candidate does is often more important than what (or how much) they say. People often do poorly in an assessment center because they enter into an exercise with a preconceived notion about what is expected of them. Preconceived ideas about what is expected, or what kind of behavior is desired, or how the problem can best be solved, often run contrary to the beliefs and philosophy of those responsible for designing and managing the process. Ultimately, such false beliefs may lead to a negative or poor evaluation by the assessors. For example, a candidate who comes into an Employee Counseling Session with a copy of the department’s Employee Assistance Program (EAP) is well-prepared, since this program may be exactly what is necessary to get the employee back on track. However, when the problem turns out to be something quite different, such as being disappointed over the last promotional process, the candidate must be prepared to put aside the Employee Assistance Program and think of other ways to get the employee motivated. This being the case, then, what, if anything, can a candidate do to prepare to participate in an assessment center in a manner that will ensure his or her best performance? Is it impossible to prepare for an assessment center? Is there nothing a hopeful candidate can do to ensure that he or she
112
The Assessment Center Handbook for Police and Fire Personnel
does the best they can do? Actually, there are several things that someone can do to improve his or her performance in an assessment center.
Active participation is required in the Group Discussion Exercise. Photograph courtesy of Bloomingdale Police Department, Bloomingdale, Illinois.
First, candidates should learn as much about the idea behind assessment centers as possible. Read up on how they were first developed, how they were used, and the reasons they are used. There are many useful articles on assessment centers in professional journals and these can be very helpful in learning more about the process and becoming more familiar with the various methods that are used in assessment centers. At the same time, they should stay away from attempting to learn the “ins” and “outs” of assessment centers, since these often depend upon who designs and administers the assessment center. Second, candidates must carefully study the criteria which are frequently used in assessing performance in an assessment center. They will probably find that they tend to include such things as leadership, communication skills (both written and oral), organization and planning, human relations, judgment and reasoning, problem analysis, decision-making, and similar dimensions that have been found through a job analysis to determine or influence successful performance in a position. These traits will not be the same for all positions but there is a great deal of crossover among posi-
Preparing for an Assessment Center
113
tions. Once a candidate has studied those criteria commonly used in an assessment center, they should evaluate themselves on each of these criteria. They should honestly and completely assess their own strengths and weaknesses. Are they a good public speaker? (Nearly all assessment centers feature some form of public speaking). Do they have strong leadership ability or are they easily manipulated by peers or subordinates? (The “Problem Employee Exercise” or “Employee Counseling Exercise” is a common scenario in many assessment centers and will be used to test their leadership ability). Do they organize their time and work well, and can they use available resources effectively? (The In-Basket Exercise is an excellent method of evaluating how well one can solve problems with limited time and resources.) Third, candidates should learn as much as possible about the position for which they are competing. In the case of internal promotions, they may think they know all they need to know about the position, but they may be wrong. What does someone in this position do? What distinguishes between superior or only average performance in this position? What skills are needed to perform in this position? What are the characteristics of people who do well in this position? These are all important questions that candidates should be able to answer if they are to be a viable candidate for the position. Knowing the answers to these questions will enhance their performance in an assessment center. Fourth, candidates should maintain a positive attitude and be self-confident (not cocky or overbearing) as they enter into each of the exercises. Candidates should not have a chip on their shoulder or dare someone to knock it off. It is said in life that attitude is everything, and this is nowhere more important than when participating in an assessment center. Candidates should not condemn or criticize the process or blame someone else or the process for their failure. If they err, they should accept responsibility for their error and not try to cover up their mistakes. Assessors will spot this immediately and will recognize it as a sign of weakness on the candidate’s part. The ability to recognize and accept responsibility for their own errors and failures, on the other hand, is a sign of strength to most assessors. Fifth, candidates should keep an open mind going into the process. They should not “over prepare” or anticipate what may lie in store for them. They should eliminate any preconceived notions or biases that they may have picked up in their research about what will or will not happen to them or about what the assessors are looking for as they evaluate their performance. Instead, they should pay close attention to the instructions that are given to them, approach each and every exercise as if it were their one chance to show that they have what it takes to do the job, and handle each
114
The Assessment Center Handbook for Police and Fire Personnel
and every scenario as if it were a real live situation. In addition, candidates should not confuse play acting with role playing. They should not try to be something that they are not. Being themselves is the easiest thing in the world to do and will enhance their performance. Trying to be something that they are not often results in failure, and never increases their chances of success. Sixth, candidates should come to the process on time and prepared. Know where you are supposed to be and get there in plenty of time so that you will not be rushed. Get a good night’s sleep (if possible) before the process begins so that you will be rested physically, emotionally, and mentally. You cannot do your best when you are fatigued or worried about something else. Be sure to bring with you any necessary supplies such as a notepad and pens and pencils. You may be required to take notes and these materials are not always provided to the candidates. Finally, and most important, candidates must be able to learn from their mistakes. Whether they are participating in their first, second, or tenth assessment center, the chances are good that another one will be lurking somewhere in the future. They may make other mistakes in the future (we all do!), but they should try to not make the same ones again. They should learn as much as possible about what they failed to do in this assessment center so that they don’t make the same mistakes in the next one. Even though the exercises may be completely different, chances are good that some of the same characteristics will be evaluated by the assessors, and they need to know how they stack up in these areas. If a debriefing session or other feedback process is offered, candidates should take advantage of it and learn as much from it as possible. Even if they do not agree with what the assessors say, they should bear in mind that these are independent evaluators whose only mission is to give them an impartial, objective evaluation. The assessor’s perception of the candidate’s strengths and weaknesses are vitally important if they are to improve their performance in the future. In summary, it pays to know what to expect in an assessment center, but it does not pay to assume anything. Candidates may or may not be successful their first time around but there is no question that they will learn and profit from their experience, and that they will be a better candidate for having been through the process.
Chapter 9 besT PrACTiCes in AssessmenT CenTer eXerCises
E
xperience is the best way to learn how to do well in an assessment center, but there are some basic common-sense principles that can be learned and utilized by candidates to help them achieve good performance in an assessment center. Creativity and originality are also important. Candidates should remember that they are looking for ways to set themselves apart from the rest of the candidates so they should never be content with just doing well. Their goal should be to excel! This chapter describes some of the best practices that candidates can and should not do in presenting themselves in the most favorable light in an assessment center. Only a few suggestions are listed for each exercise discussed, but candidates can find many more ways to perform well in an assessment center if they just think about what they are doing and ask themselves, “How do I separate myself from the rest of the candidates in this exercise?” There is no single, sure-fire way to prepare nor is there a single best strategy to use in the exercises. The best course of action is for candidates to be themselves and to handle things just the way they would do in real life. Trying to be something they are not almost always works against them. in-bAskeT eXerCise
The In-Basket Exercise is time-oriented and therefore requires a candidate to read and analyze a great deal of information in a limited period of time. Effective time-management, the ability to properly prioritize, effective delegation and sound organization and planning skills are absolutely essential for success in this exercise. 115
116
The Assessment Center Handbook for Police and Fire Personnel
The following suggestions are offered to assist you in competing effectively in this exercise: scan items before taking action. The In-Basket Exercise is designed to put candidates under pressure and one way of doing this is to include more items in the In-Basket that an average candidate can reasonably be expected to complete in the allotted time, thereby forcing them to turn their attention to the most important items and leave routine items undone. A successful candidate will quickly scan all items in the In-Basket and sort them into three piles: (1) those they must do immediately; (2) those they should do as soon as possible; and (3) those they can afford to do later or leave undone if time expires before getting them all done. Simply working from the top down in an In-Basket Exercise is a clear path to disaster.
Scan items before you act.
establish reasonable priorities. In the real world, some things are more important than others. In an InBasket Exercise, those items which, by their nature, may somehow bring discredit to the agency or to the municipality if not handled at once, deserve top priority and must be handled immediately. Other, less important matters, such as those with deadlines that must be met, should be completed next while other items, routine in nature, may be left to be completed last, if time permits. Routine items are those things in which delayed action will not adversely impact their outcome.
Best Practices in Assessment Center Exercises
117
keep other people informed about what you are doing. Bosses don‘t like surprises so it’s important to keep them informed about what you are doing. If, in the In-Basket Exercise, you are informed that you are going to be leaving town for a week to attend to a family emergency, make sure to let your immediate superior (e.g., police chief, fire chief, mayor or city manager) know where you are going, how long you will be gone and how you can be reached during your absence. You should also appoint someone to take your place while you are gone, even if this is done automatically in your absence. Everyone needs to know who is in charge while you are away. In addition, if you receive a memorandum or email from your boss telling you to do something during your absence, you must do two things: (1) find someone else to do what you have been instructed to do (e.g., delegate) and (2) let your boss know that someone else will be doing what you were instructed to do. The rule of thumb is to keep everyone who needs to know informed of what you are doing before you leave and let them know when you expect to return. delegate appropriately. The art of effective delegation is an essential management tool and successful managers and supervisors are those who know how to delegate effectively. The In-Basket Exercise is usually filled with more items than a person can reasonably be expected to complete within the time permitted so it is important that you be able to hand off some of those items to others to complete in your absence. It is important, though, that you delegate the right things to the right people. Do not, for instance, delegate something to someone beyond their level of competency or authority. If you need another ranking officer or chief officer to do something, go directly to that officer, not through an intermediary such as a secretary. While it is acceptable to ask a secretary to send a letter in your name or to convey information to someone or to arrange for a meeting to be rescheduled, do not ask the secretary to exercise authority that he or she does not have. look for linkages between items. A common characteristic of the In-Basket Exercise is to have items that are, in one way or another, linked together in such a way that they may pose some sort of conflict or common theme. For example, you may receive a telephone call from a local news reporter wanting to know something about an internal Investigation that you have just initiated in response to an alle-
118
The Assessment Center Handbook for Police and Fire Personnel
gation of sexual harassment or some other breach of conduct. It may be possible to combine these two items so that they can be handled with a single response rather than separately, thereby reinforcing the linkage between the two items. In other cases, related items may pose a scheduling conflict, in which case you will need to recognize and find a way of avoiding the conflict. This will often be in the form of one item informing you that someone has requested a day off for personal reasons and another item informing you that this same person is required to be in court or in training or doing something else that same day, thus requiring you to deny the time off request so that the other requirement may be fulfilled. The TACTiCAl fire/ems Problem eXerCise
It is probably more difficult to demonstrate your true firefighting skills and knowledge of incident command procedures during a simulated fire incident than on a real fire ground incident. In a simulated fire incident, you do not have the advantage of dealing with real live people and you do not have the same powers of observation and deduction that might be possible at a real fire. In a fire simulation, you can take nothing for granted and things just don’t happen quite the same way they do at a real fire. Nevertheless, there are some ways you can overcome these obstacles. Utilize all available resources in a timely manner. Due to the fact that the fire simulation usually runs no more than ten minutes or so, it is important that you act promptly and decisively. It is also important that you have sufficient personnel and apparatus on the scene to handle the incident effectively. Candidates will almost never be criticized for having too many people and too much equipment on the scene (you can always turn them around and send them back), but you will frequently be criticized for trying to accomplish too much with too few resources. Once you make your initial size-up, determine what level of response will be necessary to handle the problem at hand and ask for additional resources if necessary and within reason.
Best Practices in Assessment Center Exercises
119
The Incident Commander will be in charge of the emergency fire or EMS situation. Photograph courtesy of Bartlett Fire Protection District, Bartlett, Illinois.
Take nothing for granted and assume nothing. You need to remember that the assessors can only give you credit for what you say and do in this exercise. They cannot give you credit for what you think or what you meant to do. Even if something ordinarily happens “automatically” in real life (e.g., command page, notification of Public Information Officer, designation of Rapid Intervention Team, etc.), you must make sure that these things get done if you expect the assessors to give you credit for them. Remember that if you did not say it, you did not do it, and if you did not do it, the assessors cannot give you credit for it. Nothing happens automatically in this exercise. follow your incident Command Procedures. You can be sure that assessors will be thoroughly briefed on your department’s Incident Command Procedures and will probably have read them over so you should expect them to know what actions are required by those procedures and you should act accordingly. These procedures are an important part of your department’s operating policy and you will be expected to demonstrate your understanding of and compliance with those procedures.
120
The Assessment Center Handbook for Police and Fire Personnel
Good teamwork is essential in conducting firefighting operations. Photograph courtesy of Bartlett Fire Protection District, Bartlett, Illinois.
remember to do a 360. It is usually standard procedure for the Incident Commander to do a walkaround or to have someone check all four sides of a structure to get a full view of what is occurring. Without seeing all sides of the structure, you cannot have a full understanding of the fire situation. If a fire simulator is used, requesting a 360 will usually allow you to see all four sides (A, B, C, and D) to gain a better understanding of the fire situation. be proactive, not reactive. In this exercise you must be continually focused on the future and must be able to anticipate what might lie ahead. Your actions should be designed to deal with the immediate situation in such a way that it does not escalate or that, if it does escalate (often it will, regardless of what you do), you have taken all reasonable and necessary actions to mitigate the impact of further escalation. Candidates should be continually probing, asking questions, checking on the progress and status of companies, and planning ahead for future actions that may be required. A good candidate in this exercise will stay ahead of the action, not behind it.
Best Practices in Assessment Center Exercises
121
The GroUP Problem-solVinG eXerCise
The purpose of the Group Problem-Solving Exercise (also known as the Leaderless Group Discussion) is to see how well candidates can work within the framework of a group to solve common problems. Consensus and teambuilding are important considerations. Active participation is an essential requirement of this exercise. Candidates who are unable or unwilling to take an active role in the group exercise will usually not fare well. Some candidates find this to be a very challenging exercise and may struggle to keep up with the discussion while other candidates may find this exercise an excellent way to showcase their leadership skills. The following suggestions are offered to assist you in competing effectively in this exercise: establish an effective working relationship with group members. You must remember that this is a group exercise and you will be evaluated by the assessors on how well you work with other members of the group. Your entire focus in this exercise should be to find ways to contribute to the success of the group rather than to bring credit to yourself. Teamwork, cohesiveness, and consensus-building are important objectives in this exercise.
It is important to maintain effective working relationship with others.
develop consensus of ideas and decisions. Each action you take in this exercise should be calculated to bring the group to consensus. This sometimes requires compromise and a bit of give
122
The Assessment Center Handbook for Police and Fire Personnel
and take. Even though the final outcome of the discussion may not be exactly as you would like to have it, the important thing is that the group has arrived at a decision that all members of the group agree with and can support. This may mean that you will need to subordinate your own views and expectations so that group consensus can be reached. Work to get the group heading in a single direction. Some groups seem to stray from the objectives of the discussion and get bogged down in meaningless trivia that contributes nothing toward the solution of the problem under discussion. One of the things the assessors will be looking for is someone who will keep the group heading in the right direction. This can be done by volunteering to serve as the note-taker (an assignment than may become overwhelming at times) or as the group moderator or time-keeper. Others do this by tactfully reminding the group of the purpose and objectives of the assignment and offering suggestions that will help the group remain focused on its central mission. Anything you can do to contribute to keeping the group on track and focused on its principal assignment will bring credit to you in the eyes of the assessors. have someone take notes and keep track of the time. In a group problem-solving exercise, a candidate may offer to serve as the chairperson or the timekeeper or the note-taker as a means of facilitating the group’s accomplishment of its assigned task. Each of these is an important role and each can be used to the advantage of the candidate if done properly. However, volunteering to serve as note-taker can sometimes be problematic in that the candidate gets so deeply involved in note-taking that he or she fails to remain active in the discussion. On the other hand, some candidates can perform this role well and help lead the group to consensus at the same time. You must decide whether you are capable of being a notetaker without negatively impacting your group participation. lead, don’t push. Some candidates are so anxious to impress others with their vast knowledge of a particular subject that they cannot wait their turn to speak. Instead, they continually cut off others just to get their own point across. This is not a trait that assessors appreciate and it is something to be avoided. More important, however, is the need to listen intently to what others have to say. Candidates who spend more time speaking than they do listening usually
Best Practices in Assessment Center Exercises
123
have very little to say and spend too much time saying it. You can demonstrate much more leadership by engaging in active listening skills than by speaking a lot and saying very little. The emPloYee meeTinG eXerCise
The Employee Meeting Exercise is an exercise requiring sensitivity to the needs of employees as well as strong leadership ability and a commitment to the goals of the organization. Candidates are expected to meet with a role player to discuss some kind of performance issue (e.g., poor attendance, poor attitude, lack of motivation or productivity, etc.) and find a way to get the employee “back on track.” The following suggestions are offered to assist you in competing effectively in this exercise: begin and end the exercise on a positive note. It is important to get the discussion started on a positive note in order to develop an open frame of mind by the role player so that he or she will be willing to accept the constructive criticism that will come during the meeting. Candidates should open the meeting with a warm greeting and positive statement to gain the confidence of the employee, thereby making it easier to get into the core purpose of the discussion. By the same token, candidates should end the meeting on a positive note in order to reinforce what was discussed during the meeting. make your expectations very clear. In this exercise, you will be confronted by a role player whose job performance, behavior, or conduct is creating a problem of such magnitude that you have decided to meet with the employee to discuss the situation. One of your responsibilities will be to factually and completely explain the nature of the problem being discussed and to make your expectations clear to the employee. Vague or poorly-defined statements such as “some improvement” or “just a little better” accomplish nothing and may leave the employee uncertain about what you expect in the future. You should leave nothing to the imagination of the employee in this exercise.
124
The Assessment Center Handbook for Police and Fire Personnel
Make your message very clear.
do not negotiate. Role players in this exercise will sometimes attempt to negotiate a solution by saying “I’ll do this for you if you’ll do this for me.” This is something a candidate must avoid because it weakens the candidate’s power base and makes it more difficult for the candidate to impose his or her will over the subordinate. Candidates need to work from a position of authority when dealing with “problem” employees and they cannot afford to allow the employee to manipulate them by attempting to get the candidate to enter into a negotiation session. be a boss, not a buddy. The Employee Meeting Exercise (or Employee Counseling Exercise) is designed to test a candidate’s ability to meet with an employee, discuss a serious performance problem, and devise a solution that will eventually solve the problem. Becoming an effective supervisor is often difficult for a person who is reluctant to impose his or her authority over persons with whom he or she has worked for some time. Employees will often play upon a supervisor’s past allegiances, loyalties, and relationships to get them to relinquish their authority and to avoid criticism or correction. This exercise is sometimes like a psychological tug of war or arm wrestling contest and there can be only one winner.
Best Practices in Assessment Center Exercises
125
The CommUniTY meeTinG eXerCise
The Community Meeting Exercise is often used in an assessment center to test a candidate’s ability to speak extemporaneously before community groups, to make a favorable impression on the public and to communicate orally in an affective and intelligent manner. Role players or assessors may be used to simulate community groups and the candidate is usually given information about the specific problems or issues that he or she will be expected to address during the meeting. Assessors or role players are briefed ahead of time on the demographic characteristics of the neighborhood in which the community meeting is taking place and on the types of problems and issues that might be raised during such a meeting (e.g., speeding cars, drug sales, loud noises, parking problems, etc.). Here are some things a candidate might want to do to enhance their performance in this type of exercise: listen to what the residents have to say. The purpose of this meeting is not to hear candidates make long-winded speeches but to see how effectively they can deal with real-life community issues. Candidates should spend most of their time listening and less time speaking. They should listen intently to what the residents have to say, process the information being received, formulate a plan for dealing with the issues presented and then tell the residents what they can or will do to solve the problem. offer to get back to the residents with answers or solutions. It’s a good idea for a candidate to let residents know that he or she will get back to them with answers to their questions. A candidate may wish to give residents his or her business card, telephone number, or e-mail address so that they may contact the candidate in the future for further assistance. A candidate may wish to simulate bringing someone along as a note-taker and promising to send everyone in attendance copies of the minutes of the meeting. One way or another, it is important that candidates let the audience know that they will make a concerted effort to solve their problems to the best of their ability. be willing to accept responsibility. Community residents expect representatives of their police and fire departments to be accountable for their own actions as well as the actions of
126
The Assessment Center Handbook for Police and Fire Personnel
their respective agencies. Sometimes mistakes are made despite our best efforts to deliver quality services in a prompt and efficient manner. Sometimes members of our agencies don’t always do what we expect them to and the reputation of the department may be brought into question. If a candidate is confronted with a situation in which something was not done that should have been done or that was not done the way it should have been, the best course of action is to admit the error and promise to do better in the future. The worst thing a candidate can do is to try to justify the actions or to make feeble excuses that only make the candidate or the department look worse. The CiTizen inTerVieW eXerCise
Dealing with citizen complaints about the level or quality of police or fire services is a basic responsibility of any police or fire supervisor. The basic mission of a police or fire department is public service and first-line supervisors have an important role to play in seeing to it that citizens are wellserved by their agencies. This is an exercise in which candidates are evaluated on their ability to be sensitive to the needs of others, to obtain essential information, to identify and analyze a problem, and to reach logical and appropriate solutions to problems. The following suggestions are offered to assist a candidate in competing effectively in this exercise: Greet the citizen warmly and make him or her feel welcome We sometimes forget that a citizen lodging a complaint about the level or quality of police or fire services may feel awkward and ill at ease in the surroundings in which the meeting is conducted. It is important that the candidate make every effort to make the citizen feel at ease and welcome so that the citizen will know that the candidate is clearly concerned about the problem(s) presented by the citizen. If, at the conclusion of the meeting, the citizen feels good about the experience and is comfortable with the solution offered by the candidate, the candidate will have accomplished a great deal.
Best Practices in Assessment Center Exercises
127
Be sure to greet the citizen warmly.
remember to sell yourself and your department. Any citizen contact is an opportunity for a police or fire supervisor or chief officer to sell the programs and services of their department. In this exercise, candidates are something like sales people and the product they are selling is community service. Candidates need to make every effort to convince the citizen that the department is concerned about his or her problem and that it will do whatever can be done, within reason, to see that the citizen’s problem is satisfactorily resolved. don’t make promises you can’t keep. There is nothing worse than telling someone you will do something and then not keeping your promise. While it is important that citizens feel that a police or fire supervisor or chief officer is truly concerned about their problem, candidates should never make promises that they can’t keep because that is the same thing as lying to the public. Let the citizen know what you can and will do for them, but be careful not to promise more than you can deliver. False promises raise false expectations which will eventually come back to harm the cooperation and support police and fire departments need from their constituencies. The ComPAnY or shifT TrAininG eXerCise
The object of this exercise is to put candidates under a certain amount of pressure to see how they handle it and to determine how well they can think and act “on their feet.” Candidates will usually be given a topic to be pre-
128
The Assessment Center Handbook for Police and Fire Personnel
sented to the company or shift (the assessors) and will be expected to present information on the topic in a training format. Role players will often engage in behavior designed to either get the candidate sidetracked or to test his or her patience and composure. There are several things a candidate should remember when participating in this exercise.
Effective police work requires a strong partnership between the police and the community. Photograph courtesy of Joplin Police Department, Joplin, Missouri.
maintain your composure. Assessors may deliberately say and do things to provoke you, such as reading newspapers, talking among themselves, or being otherwise unruly and obnoxious, and they do these things to see whether you can work effectively under pressure. You should not take these actions personally and remember that they do what they do for a reason and that all candidates will be treated the same way. Don’t lose your temper and don’t allow them to sidetrack you from your course of instruction.
Best Practices in Assessment Center Exercises
129
Get the role players involved. A good way to counter the interruptions and distractions of the role players is to get them involved in the class. Ask them questions on the subject matter, have them come to the front of the room and write something on the board or use them to demonstrate a technique (e.g., proper pat down and handcuffing procedures). Getting them involved in the class takes them out of their role playing persona and makes them easier to deal with. It also helps to increase their interest in the subject matter. don’t ignore criticism or unprofessional conduct. If a role player makes a disparaging remark about you or a senior officer or about the department administration, you cannot afford to ignore the remark because it may appear to the assessors that either you agree with the comment or simply do not know how to deal with it. When these kinds of negative remarks are made, you must be quick to put a stop to them and let the person who made the remark know that such comments are off limits and will not be tolerated. begin and end the session on a positive note. It is important that you demonstrate a positive attitude during the meeting and that you convey to the members of the audience a positive and upbeat attitude about the topic being discussed. Positive attitudes can be contagious and so can negative ones. If you tell the audience, “You probably will not like what I have to say,” you can be assured that they will definitely not like what you have to say. If, on the other hand, you can find a way to get them interested and motivated in the topic being discussed, they will have fewer reasons to object and more reasons to go along with the program.
Chapter 10 some of The GreATesT blUnders
P
articipating in an assessment center is, or should be, a learning experience. How much candidates learn from the process depends upon their attitude going in and their willingness to admit errors and to accept constructive criticism. A small number of those who participate in the assessment center process will never learn because they will never admit that they erred. In their mind, it was the process that was flawed, not what they did (or failed to do). These same people will always find something or someone to blame for their mistakes and will never learn from the process. There are a few others who simply, because of their own inadequacies, never get better and are what they are. This does not make them poor performers, but it does indicate that they will probably never move ahead or progress in their careers except by good fortune or attrition.
An assessment center can be very stressful.
130
Some of the Greatest Blunders
131
For the most part, however, participating in an assessment center can be an invaluable experience, even if they do not measure up or don’t do as well as they had hoped. Candidates should endeavor to make it a learning experience. They should try to profit by their mistakes and seek out and consider constructive criticism. For most candidates, there will be a next time, and they will be better prepared the second or even third time around. Chapter 9 includes a suggested list of things to avoid and things to try to accomplish in several different exercises. This chapter provides a discussion of some of the biggest blunders I have witnessed in more than 25 years of designing and administering assessment centers. Some of these are classic and some may seem even ridiculous, but each blunder described here actually occurred. I have attempted to record the details as accurately as possible and not embellish the description in any way. Each of these blunders contains a lesson to be learned, and I hope that these examples serve this purpose. The orienTATion session
I have stressed the importance of the orientation session for candidates several times in different chapters of this book, but I want to reinforce what I have said previously. Candidate orientation sessions have but one purpose: to provide candidates with necessary insight into the process and to allow them to perform to the best of their ability. No one else gains from the orientation session, and it is only the candidate who loses by not attending or by not paying attention. Later on, they make the very mistake they were warned about during the orientation session. What’s worse, they will blame that mistake on someone other than themselves. Example One: On two separate occasions (one for police and one for fire), one candidate out of all those eligible to participate in the process failed to attend the orientation session. Since it was not mandatory, no one made much of this fact. One fellow was in a class and made a decision that completing the class was more important than attending the orientation session for the Police Sergeant assessment center. Another fellow had made other plans for that day and decided that it was more important to keep the plans he had made than attend the 90-minute orientation session for Fire Lieutenant candidates. By the time these two individuals arrived for the assessment center, they discovered that they were several paces behind the rest of the candidates, all of whom knew what to expect when the process began. These two situations have something else in common. In addition to the
132
The Assessment Center Handbook for Police and Fire Personnel
fact that these two individuals made a decision that something else was more important to them than finding out what they could about the assessment center process, both of these individuals scored lower than all other candidates and failed the process! Did the orientation session make a difference? Draw your own conclusions! The lesson candidates can learn in this example is that they should be sure to attend the orientation session (if there is one), to take notes, pay attention, and ask questions. No one else will do this for them.
The in-bAskeT eXerCise
The In-Basket Exercise is particularly difficult for the first timer, and this is especially true if a candidate is not accustomed to handling a lot of paperwork, setting priorities and working under time constraints. First timers rarely do well in this exercise, and some experienced administrators sometimes have problems due to the time constraints. Example One: In one case, a relatively bright individual fared poorly in the In-Basket Exercise primarily because he failed to complete half the items. When he was asked why he had been able to complete so few of the items, he said that he had read a book on assessment centers in which the author advised that the first thing to do in an In-Basket Exercise was to prepare a detailed outline of the contents before proceeding. “I spent so much time working on the outline that I did not have time to complete all the items!” I suppose preparing an outline before beginning the In-Basket Exercise is a good idea if time constraints allow. But in this case, the candidate had only 90 minutes to complete 18 items. This averages out to five minutes for each item. Preparing an outline probably took him 20 or 30 minutes, which reduced his available time by one-third. He would have been better off simply doing a preliminary sort to determine priorities and then to begin working on the highest priorities and working his way down to the lowest ones. The lesson candidates can learn in this example is that they should be mindful of their time limitations and don’t be confused by what others have suggested.
Some of the Greatest Blunders
133
Your first in-basket may be somewhat overwhelming.
Example Two: There was another case in which a person got through most of the items in the In-Basket but failed to complete some of the most important items. When asked about his strategy in doing the In-Basket, he said that he decided to start with all the “easy” items (e.g., those having the least importance) to get them out of the way and then go on to complete the more difficult items. It was explained to him that the easiest items are also the least important and should therefore be left to do after the most difficult items (e.g., the most important) have been completed. Failing to complete the most important items cost this person dearly in this exercise. The lesson candidates can learn in this example is that they should think about their priorities and get the most important ones done first and then finish with the least important ones if they have time left.
The emPloYee meeTinG eXerCise
The Employee Meeting (or Counseling) Exercise is a basic exercise and one that may be encountered in any assessment center. It falls into the category of what I call “Supervision 101,” meaning that it is intended to simulate a fundamental responsibility of every police or fire supervisor. This does
134
The Assessment Center Handbook for Police and Fire Personnel
not mean that it is an easy exercise, because experience has proven that it is not. There are many ways to do poorly in this exercise. Some of these are described below. Example one: I recall one case in which a candidate for Police Sergeant was told that he was about to conduct an interview with an officer who was performing poorly. This officer was coming in late for work, failing to get his reports done on time, leaving critical information out of those reports, and doing very little in the way of self-initiated activity. The candidate, who was to assume the role of a new shift sergeant, was asked to speak with the officer, find out why these problems were occurring, and attempt to bring about an improvement in the officer’s performance. As the candidate was brought into the exercise room, the role player was standing near the desk and was greeted by the candidate saying, “Sit down and be quiet. I have something to say to you.” The candidate went on to tell the officer, as if he were a threeyear-old child, what was expected of him. There was no dialogue, no probing for facts, no attempt to discover if there were reasons for the officer’s behavior. Instead, the candidate chose to adopt a dictatorial posture when dealing with the officer, which merely resulted in hostility by the officer and the creation of even more serious problems in the future. Candidates sometime seem to anticipate that they will be given a rough time by role players and therefore want to try to gain the upper hand early by letting the role player know who is in charge. This is a tactical error since it suggests that the candidate already knows the problem before beginning the interview and has chosen a solution before knowing all the facts. It is much better to keep an open mind until the candidate knows the entire situation and then choose a course of action appropriate for the facts they have uncovered.
The lesson candidates can learn in this example is that they should keep an open mind when beginning the exercise. They should not form preconceived opinions before they know the facts. They should ask good questions, find out what is going on, and then choose an appropriate course of action.
Example Two: In another case, a candidate for Fire Lieutenant was instructed to speak with a firefighter on his crew who had a bad attitude about women. The firefighter was supposedly known to have a poor regard for women in the fire service and to make a number of derogatory remarks about the ability of women to do the job. The candidate was instructed to
Some of the Greatest Blunders
135
speak with the firefighter to let him know that this attitude has no place in the fire station and that future acts of this kind might lead to disciplinary action. The candidate began to explain the purpose of the interview to the role player, but within two or three minutes the role player had completely taken over the exercise. The candidate listened quietly as the role player managed to steer the conversation away from what the candidate was supposed to be discussing. By the time the exercise ended, the role player had totally dominated the conversation and the original purpose of the meeting had been lost somewhere along the way. Strong role players can be very skilled in gaining control of this exercise and getting the conversation to go in a different direction from that intended. Candidates need to be alert to this possibility and to find a way to gain control of the exercise early on and to retain control so that the purpose of the meeting is eventually satisfied. At the same time, it is important to allow the role player the opportunity to say what is on his or her mind and to participate in the conversation. Being in control of the exercise is not the same thing as dominating it. Candidates must be able to recognize the difference. The lesson candidates can learn in this example is that they should find a way to establish control of the exercise early on and to make sure that the role player does not get them to deviate from the original purpose of the meeting. Candidates should remember that on the highway of life, there are drivers and passengers, and in this exercise, they need to be behind the wheel.
Example Three: Problem-solving is a key component of the Employee Meeting Exercise. A candidate is given a problem and is expected to resolve it during the exercise. The manner in which a candidate intends to solve the problem is manifested by explaining to the employee (and thereby, to the assessors) what actions they plan to take in dealing with the problem. Failure to make this clear to the employee (and to the assessors) will inevitably cause them many problems in this exercise. In one case, a candidate did a very good job of explaining his concerns to the role player, in asking questions designed to find out what the employee knew about the problem and in establishing a good rapport with the employee. Unfortunately, after about seven minutes of conversation, the candidate closed his notebook and terminated the meeting, telling the employee he would get back to him later. After leaving the exercise room, the candidate asked me when he would be allowed to go back and explain to
136
The Assessment Center Handbook for Police and Fire Personnel
the assessors how he planned to deal with the problem. “That was your chance,” I said as the color drained from his face and he realized his error. The lesson candidates can learn in this example is that they should make it perfectly clear to the employee what course of action they intend to follow when dealing with the problems presented. In most cases they will not be allowed to go back later and tell the assessors what they plan on doing.
Example Four: A favorite strategy of the role player in the Employee Meeting Exercise is to say and do certain things during the interview that will challenge the authority of the candidate and perhaps provoke him or her to say or do something which they may later regret. A good role player can be very successful in using certain terms and engaging in certain types of behavior that will provoke the candidate into losing his or her temper, which would be a critical error in this exercise. In one case, a candidate for Police Lieutenant was dealing with a recalcitrant sergeant who had failed to do a proper job of handling a citizen complaint the previous day. Candidates were expected to explain to the sergeant the proper way to deal with citizens and to let him know that he had mishandled the incident in question. Finally, candidates were expected to gain acceptance by the role player of a course of action leading to a more acceptable way of dealing with such complaints. The role player in this exercise was very consistent among all candidates in his statements and behavior. At one point during the interview, he said to the candidate, “Well, now, lieutenant, I’ve been taking these complaints since you were in the academy!” The obvious intention of this remark was to provoke the candidate into losing his or her temper. For the most part, candidates found an acceptable way of dealing with this remark, although some handled It much better than others. One candidate, however, who was obviously getting more and more frustrated as the interview progressed, slammed his fist down on the table when the role player made his insulting remark and yelled something very unflattering to the role player and pointed his finger at him in a threatening manner. At that very moment, the tone and tenor of the exercise changed dramatically. The interview went from one of counseling an employee to one of threat and intimidation, and the candidate fared poorly at the hands of the assessors.
Some of the Greatest Blunders
137
The lesson candidates can learn in this example is that they should maintain control of their temper and not allow themselves to become provoked. When they lose control of their temper, they lose control of the situation. Candidates must remain in control, not by threat or intimidation, but by their sheer command presence and attitude of confidence.
The TACTiCAl fire Problem
In my mind, fighting a real fire, where you have the advantage of sight, sound, touch, smell, and dealing with real, live, tangible objects, is easier in some ways than trying to fight a fire under the artificially controlled conditions imposed within the framework of an assessment center exercise. Experienced firefighters at the scene of a real fire know instinctively what to do and what to expect from other crews at the scene. Unfortunately, things don’t often work that way in a simulated fire problem. One of the ways to do very poorly in this exercise is to fail to get enough help on the scene as quickly as possible. Because the scenario moves so quickly, many candidates don’t realize until it is too late that they simply don’t have enough personnel and equipment to do the job expected of them. Few candidates have ever been criticized for having too many personnel and pieces of apparatus at a fire scene, but many have been criticized for not having enough. As they often say, “You can always turn them around!” Example One: In one case, a candidate was assigned to handle a fire in a first-floor apartment unit in a three-floor, twelve-unit apartment complex. The first alarm to this location provided him with two engines, one truck and one squad. No ambulance was assigned on the initial alarm. The time of the fire was 6:00 a.m. on a weekday, which meant that there was a strong possibility of life safety as an issue. Most strategies would call for one engine to make a quick attack and a second to provide a backup line while the truck crew conducts primary search and rescue. This leaves one squad to either begin ventilation once the seat of the fire has been discovered, or to begin a primary search on the second and third floor or to be assigned as a Rapid Intervention Team (RIT). In short, all companies are committed almost immediately upon their arrival at the fire scene. In this particular case, the candidate waited more than seven minutes to call for a second alarm, which would have brought in three additional en-
138
The Assessment Center Handbook for Police and Fire Personnel
gines, one truck, and one ambulance. An even smarter move would have been to call for an ambulance due to the potential number of casualties. This candidate did neither and was faced with an escalating fire problem, causing him to split his companies, assign them multiple tasks, and wear them out in a hurry. The lesson candidates can learn in this example is that they should call for help early in the process and anticipate all possible contingencies so that they can be proactive in their tactical approach rather than reactive.
Example Two: In another case, a candidate was assigned to respond to a fire in a two-story wood frame house having balloon frame construction. Upon arrival, the candidate saw smoke showing from the roof in the rear of the building. He established command at the front of the building and never did a walk-around or “360,” nor did he assign anyone to go to the rear of the building to get a better view of the fire. In actuality, the fire was in the basement of the building and the smoke was venting through a chase in the walls and into the roof. A quick walk-around by the candidate would have provided him with a view of the rear of the building, which would have revealed the true location and nature of the fire. In this case, most of his tactical decisions were poor because he was making an incorrect assumption about the location and nature of the fire. The lesson candidates can learn in this example is that they should do a walk-around very early in the exercise. This will normally give them additional views of the fire building which in turn will provide them with a much better idea of the scope and location of the fire.
The CommUniTY meeTinG eXerCise
The Community Meeting Exercise can be difficult for those persons who are unaccustomed to speaking before groups or who have a hard time thinking on their feet. While formal training in public speaking is not required, it would be helpful to anyone who is expected to participate in this kind of exercise.
Some of the Greatest Blunders
139
One of the keys to this exercise is being able to listen to what others have to say and to answer questions asked by members of the audience. In most cases, the questions will be developed in advance of the exercise and will be geared to actual issues in the community. For example, role players acting as community members may be prompted to ask questions about gang problems in the area, speeding cars, or juveniles creating a disturbance in the parks and playgrounds. Candidates would be expected to respond with answers which are appropriate for the issues presented to them. It is important that candidates keep an open mind when participating in this kind of exercise and that they do not attempt to impose their own agenda or ideas upon those in the audience. Otherwise, they may very well lose the interest of the audience and find that what they have to say has fallen on deaf ears. Example One: In one situation, a candidate for Police Lieutenant was asked to meet with a group of community residents to discuss a number of issues that they had previously complained about. The candidate was expected to listen to what the residents had to say and attempt to formulate a plan of action that would address the problems and concerns of the residents. In this particular case, a candidate “over prepared” for the exercise by preparing a detailed presentation covering all the major points that he thought needed to be addressed during the meeting. The problem was that he was so intent on telling the residents what he had come to talk about that he failed to listen to their questions. In fact, he refused to allow them to ask any questions, telling them instead that he would address their questions when he was finished giving his presentation. Even when the role players persisted and threatened to walk out of the meeting, the candidate would not capitulate and insisted upon completing his presentation. By the time he had finished, the audience had left and he was speaking to a row of empty chairs. The lesson candidates can learn in this example is that they should not attempt to impose their own agenda on the audience. Even though they might have something important to say, it is not nearly as important as what the people in the audience want to know. Candidates must listen to what the audience has to say and tailor their presentation accordingly.
140
The Assessment Center Handbook for Police and Fire Personnel The ProGrAm deVeloPmenT eXerCise
The Program Development Exercise is an excellent way to test a candidate’s ability to conduct original research, to tackle tough and complex technical issues, to prepare a well-written and properly-documented written report, and to make an impressive oral presentation to an audience of critical assessors. In this exercise, candidates are often given a topic to research and then are asked to prepare a written report to the “City Council,” or the “Ways and Means Committee” or to a “Program Review Committee” consisting of senior members of the police or fire department command staff.
Present your ideas in a skillful manner.
Candidates are sometimes given several days to work on their paper, which allows them sufficient time to conduct original research, collect and analyze data, and prepare one or more drafts before putting the final touches on a final work product. The problem lies in the fact that there are no controls to ensure that a candidate does his or her own work. Candidates are, in effect, on the “honor system” and are expected to do their own work and not rely upon a family member, a friend or acquaintance, or even another candidate to do the work for them. Experience has shown, however, that a candidate who relies upon another person to do the work for them will usually be discovered and will pay dearly for his or her transgression. Example One: In one case, a candidate was given several hours to develop a proposal for the adoption of a new policy dealing with referral of juvenile offenders by the police department. One particular candidate
Some of the Greatest Blunders
141
submitted a very well-done paper, but during the oral presentation, the candidate had to continually refer to his paper and it became clear that he was not familiar with the material. When asked by the assessors, the candidate eventually admitted that he had not written the paper but that he had asked his secretary to copy the paper from material that he had found on the topic. His excuse was that the assignment was too difficult and that it was not realistic to expect a completed work product in the time available. In view of the fact that the other five candidates were given the same assignment and were able to prepare their own papers, the assessors were not particularly sympathetic to this candidate’s rationale. He failed the exercise. The lesson that can be learned from this example is that you should always do your own work. You may be pleasantly surprised with the results, and you always enjoy the results of your own efforts more than if someone else did the work for you.
The CiTizen inTerVieW eXerCise
For the average police officer, handling citizen complaints is about as basic as anything he or she will ever be called upon to do. To complete this exercise successfully, however, candidates need to have good listening skills. While it is not necessary that candidates be trained interviewers, they do need to be able to listen thoughtfully to what people have to say and then to formulate a plan of action that will solve whatever problem has been presented to them. One of the errors that candidates sometime make in this exercise is to treat the matter as “just another complaint,” when in reality, to the person making the complaint, it is a very serious problem. Candidates need to show the role player (and the assessors) that they are truly interested in what the citizen has to say and that their complaint is the most important thing they have to do at that very moment. Example One: In one situation, a candidate showed very little sensitivity to the citizen and very little interest in her complaint. Instead, the candidate treated the scenario as an exercise in how to fill out a police report. He made a point of asking all the questions necessary to complete the complaint form. He showed absolutely no interest in what the citizen had to say. It was as if the candidate was emulating Detective Joe Friday in the old “Dragnet” television show. “Just the facts, ma’am, just the facts.” When the exercise was over, the candidate had a perfectly completed report and a very angry citizen!
142
The Assessment Center Handbook for Police and Fire Personnel
The lesson candidates can learn in this example is that they should practice active listening skills and let the citizen know they are concerned about their problem. Candidates should not write anything down—not even the citizen’s name—until the citizen has had a chance to tell his or her story. Then the candidate can go back and retrace the story and begin to capture the essential points for their report.
Example Two: In another case, a candidate came to the Citizen Interview Exercise with no paper or pen, even though he had been told that he would be conducting a citizen interview and that he should “come prepared.” Those instructions should have been sufficient, but apparently they were not. Midway through the interview, the role player looked skeptically at the candidate and asked him, “Aren’t you going to write this down?” She was obviously very concerned about his failure to take any notes, as anyone in that situation would be. The candidate replied, “No, I’ll remember everything.” This was totally unbelievable and the candidate lost a great deal in the eyes of the assessors, and the role player later reported that she felt the candidate was insincere and uncaring. She was probably right! The lesson candidates can learn in this example is that they should come prepared for the exercise. If they know they will be conducting an interview, they should bring a pad and pen or pencil with them. It is their responsibility to come prepared for the exercise.
The STaff MeeTing exerciSe
In nearly all assessment centers, there will almost certainly be one exercise designed to test a candidate’s self-control as well as his or her ability to deal effectively with thorny interpersonal problems. The Staff Meeting Exercise is one that often contains these elements. Typically, the candidate is meeting with two or three staff personnel to talk about current issues or problems and one of the staff personnel engages in certain behavior designed to provoke the candidate. How the candidate handles such a situation will often reveal much about his or her character or style of leadership.
Some of the Greatest Blunders
143
Group exercises are often included in an assessment center.
Example One: In one situation, a candidate for Chief of Police was meeting with his senior management people during his first week on the job. One of the senior commanders was also the president of the police union. During the meeting, the union president made it a point to let the new chief know that the union was a force to be reckoned with in the department and he expected to have a voice in major policy matters. After about five minutes of hearing more than he wanted to from the union president, the chief ejected him from the meeting with a stern warning not to attempt to threaten or intimidate him again! While the behavior of the union president was clearly out of line and needed to be handled firmly by the Chief, there were many other ways that this situation could have been handled. Instead, this candidate chose to alienate the union president (and thus, a large majority of the department) during his first week on the job. It would not take many more meetings like this for the Chief to alienate just about every other member of the department! The lesson candidates can learn in this example is that they should seek conciliation rather than retribution when dealing with provocative issues. Making enemies of respected leaders is not the way to earn respect.
144
The Assessment Center Handbook for Police and Fire Personnel ConClUsion
While participating in an assessment center can be a challenging and arduous process, it can also be a valuable learning experience, but only if you are willing to admit your mistakes and have the capacity to learn from those mistakes. There is no perfect candidate and it is by making mistakes that we learn, develop, improve, and sharpen our skills. The only person who makes no mistakes is the person who does nothing. The only person who cannot learn from his mistakes is the person who is unwilling to admit and accept his or her own imperfections.
Chapter 11 ensUrinG CAndidATe sATisfACTion WiTh AssessmenT CenTer meThods
D
espite some arguments to the contrary, assessment center methods have proven to be a reliable, cost-effective, and valid means of predicting future job performance. They are, in many municipalities, an integral component of the public safety promotion or selection process. This is particularly important in view of the fact that police and fire promotional examinations are often troublesome, owing to rather rigorous civil service requirements and the rather strong unions representing members of these departments. Getting candidates to believe in the fairness and validity of the process is an important part of ensuring the success of any promotional examination. Experience has shown that candidates who believe that they have been fairly treated and that the process in which they participated was a reasonable test of their ability to perform the required duties of the position for which they were being examined are much less likely to challenge the results of the examination, regardless of what score they received. There are, of course, exceptions to this rule, and there are those persons who will never be satisfied with a promotional examination unless they receive the top score. There are six basic ways in which to ensure a reasonable level of candidate satisfaction with a promotional examination based upon assessment center methods. These should be considered basic to any assessment center in which there is a belief that the results might be challenged, although there is no guarantee that these safeguards, in and of themselves, will guarantee against a challenge by a candidate. The experience of the author in conducting several hundred assessment centers involving more than a thousand candidates has demonstrated the value of the methods described below.
145
146
The Assessment Center Handbook for Police and Fire Personnel CAndidATe orienTATion
Although assessment centers are more common today than they were ten years ago, many people find themselves participating in an assessment center for the first time. This can be a very traumatic experience, particularly when compared with other, more traditional, promotional examination methods. Chances are, a candidate has read about assessment centers or has talked with a friend who has participated in an assessment center. The information received by these means may only increase a candidate’s fear of the process. This fear can be lessened, but not altogether eliminated, by conducting a brief orientation session prior to the assessment center. Generally, this session should take place no more than one week before the actual assessment center.
Make sure to attend the orientation session if there is one.
During the orientation session, the assessment center administrator should explain the basic purposes of the assessment center, how it compares with more traditional testing methods, and the advantages of the assessment center over other methods. Generally, it is good to explain to the candidate exactly how their performance will be evaluated, to include the dimensions upon which they will be rated. The nature of the exercises may also be discussed, although it is important not to provide too much detail about the particular exercises. It is a good idea to videotape the orientation for the benefit of those who, through no fault of their own, are unable to attend. Some agencies make it mandatory that candidates attend the orientation session. My own belief is
Ensuring Candidate Satisfaction with Assessment Center Methods
147
that it should not be necessary to coerce someone to do something that is for their own benefit. If they choose not to attend, they have no one to blame but themselves if they are poorly prepared for the process. Some prefer to keep the orientation simple and basic without providing candidates much insight into the scope and nature of the exercises or the manner in which their performance will be evaluated in those exercises. This attitude seems to grow out of the “shock philosophy” in which outwitting the candidates seems to take precedence over evaluating candidates’ skills and abilities. This notion is rather outmoded today and has been replaced by a philosophy of openness and transparency in favor of fairness to the candidates. An attitude of openness and honesty concerning the process will do much to lessen candidates’ fear about what they are about to encounter and will help them to prepare for what lies ahead. In the long run, this will also help to avoid any complaints about unfair treatment. CUsTomizinG The ProCess
One of the complaints most often made by fire or police personnel about promotional testing is that it “wasn’t practical” or that it “had nothing to do with my job.” This often occurs when agencies attempt to purchase generic assessment center scenarios as a way of saving money or when they hire a consultant to design a process who does not take the time to carefully study the agency. When this occurs, it is often discovered that exercises have little relevance to the agency or to the position for which a candidate is being evaluated. For example, in one actual case, candidates for Sheriff’s Captain were given an in-basket designed for a police chief who reported to a city manager, mayor, and city council. As another example, it would not be appropriate to assign candidates to present a speech to a local “community group” on fire prevention or crime prevention if persons in that position are not regularly expected to make such presentations. A thorough job analysis should always precede the development of any assessment center. This job analysis will help to ensure that the exercises selected for the process are both agency-specific and position-relevant. The job analysis should also provide sufficient information concerning the skills and abilities needed for successful job performance which can be evaluated during the assessment center exercises. Without a thorough job analysis, the validity of any assessment center may be questionable. Those interested in implementing the assessment center as a part of their promotional process are urged to avoid the “cheap and easy” approach in which “canned” exercises are used. Customized exercises which bring realism and relevance to the process will cost more in the long run but are
148
The Assessment Center Handbook for Police and Fire Personnel
worth every penny spent on them in terms of good results and satisfied candidates. On the other hand, nothing can be more frustrating and disappointing to a candidate than the realization that the process that will determine their future in the department was done with an eye toward saving money rather than finding the most qualified candidate for the position. keePinG The eXerCises reAlisTiC And releVAnT
Candidates appreciate the fact that the exercises in which they participate are realistic, practical and related to the position for which they are being evaluated. They do not appreciate a process in which they feel that they have been taken advantage of by the assessors or in which they feel that the odds are stacked against them. Unnecessarily adversarial role playing, unrealistic time limits, and uncaring or obnoxious assessors all contribute to an “I can’t win” feeling by a candidate. Realistic and relevant exercises, on the other hand, help ensure a greater degree of comfort by candidates and stimulates their interest in and satisfaction with the process and the end results of that process. Conducting a thorough job analysis, carefully studying job descriptions and other pertinent data, observing position incumbents as they preform their duties, and interviewing subject matter experts and other department staff are the best ways to ensure that the exercises selected for the process are relevant and appropriate. Although this requires an investment of time and effort, the additional expense is well worth the effort. A satisfied candidate who believes that the process was practical and relevant, who is treated fairly and believes that it was an open and competitive process is one of the best ways to sell the program to others who may have not yet made up their mind about the process. seleCTion And qUAlifiCATions of Assessors
Police and fire personnel feel—rightly so—that their jobs are unique and different and that only those who have walked in their shoes can properly understand what they do and how the job should be done. Although the principles of supervision and management may not differ a great deal between public and private sector organizations, it is difficult to convince an applicant for Police Sergeant or Fire Lieutenant that someone who has not held that position can properly evaluate someone who is being considered for the position. As a general rule, it is important that assessors be experienced in the po-
Ensuring Candidate Satisfaction with Assessment Center Methods
149
Assessors must be independent, qualified and impartial.
sition for which the candidates are being considered. It is usually preferred that assessors, if currently employed in a police or fire agency, hold a position or rank superior to that for which the candidates are being evaluated. Whether active or retired, assessors should have no direct link with the agency in which the candidates are employed or with any member of the agency who may have a direct interest in the results of the process. The credibility of the process can be seriously undermined if it is later discovered that one of the assessors knew one of the candidates or might have been unduly influenced by a previous relationship with someone in the department. It is also useful, but not essential, that assessors have previous experience with the assessment center method. If an inexperienced assessor is to be used, it would certainly be a good idea to team that person up with an experienced assessor—particularly one who is familiar with the methods to be employed in evaluating candidates in that particular process. As discussed in Chapter 6, at least one state has mandated minimum training and experience standards for assessors and it is very likely that this may be the beginning of a trend in other states. obTAininG feedbACk from CAndidATes
It is important that those responsible for designing and administering assessment centers be continually on the alert for deficiencies in the process and for ways in which they can improve the product they deliver. Methods of evaluating candidates, the scope and nature of the exercises themselves,
150
The Assessment Center Handbook for Police and Fire Personnel
and methods for administering the assessment center should be continually evolving and improving so that the results achieved are as good as they can be. Simply doing the same thing each time in the same manner and using the same assessors and the same candidate evaluation methods is not good enough. Like all things in life, change is inevitable and in this case, change is one way to improve what we do.
Candidates appreciate getting feedback on their performance.
Candidates have an investment in the outcome of any assessment center and how they feel about the process is an important consideration when evaluating the results of the process. Candidates have a right to feel that they were properly and fairly treated and that the exercises in which they participated were appropriate and relevant to the position for which they were being evaluated. In a sense, the candidates are just as much a “client” as is the jurisdiction for which the assessment center is being conducted. As a result, it is important that candidates have some means by which they can express their feelings about the assessment center. In addition, the developers and administrators of assessment centers have a responsibility to constantly be alert for shortcomings and for ways in which to improve what they do. Candidates often have excellent suggestions for improving the quality of the process, and the assessment center developer or administrator should be receptive to these suggestions. A good way of obtaining feedback from candidates regarding the fairness, job relatedness, and objectivity of the assessment center process is to ask them to complete a brief questionnaire following the process. It is usually a good idea to do this immediately after they have completed the process when everything is still fresh in their minds. Even though they may not know
Ensuring Candidate Satisfaction with Assessment Center Methods
151
how they have been rated by the assessors, they usually have a good idea of how they fared. Candidates should not be asked to sign the questionnaire and their responses should remain anonymous, but it is often useful to ask them something about themselves (e.g., length of service, level of education, how many other assessment centers they have participated in, etc.). An example of a candidate feedback questionnaire appears in Appendix S. The purpose of the questionnaire should be made clear to the candidates. They should also be advised that their responses, whether positive or negative, will be summarized and reported back to the client organization. In this way, if they want to express a concern or feel that they have a legitimate complaint about the way the process was conducted, they will know that local officials are made aware of their concerns. Candidates should not be required to complete the questionnaire, but rather encouraged to do so. It is a good idea to let them know that their responses are reviewed and evaluated by the assessment center administrator and that changes in the process are often the result of these responses. On the other hand, positive responses from the candidates, which should be the rule rather than the exception, help to validate the fairness and objectivity of the process and to reinforce the value of the process to the organization. CAndidATe debriefinG
The assessment center method is more than just a promotional tool. It is a method of evaluating candidate capabilities and potential and this can be used as a learning mechanism by the agency. In most cases, candidates learn a great deal about their strengths and weaknesses simply by going through the process. Most will profit by this experience and become better
Assessors may be asked to prepare written evaluations of a candidate’s performance.
152
The Assessment Center Handbook for Police and Fire Personnel
candidates in the future. A means of providing feedback to the candidates is a useful adjunct to the assessment center process and it may help to alleviate any complaints of unfairness that might otherwise be leveled against the process. Feedback can be provided orally, in writing, or by videotape, or some combination of these written narrative comments can be prepared by each assessor after the conclusion of the process and these can then be combined into a single document by the exercise administrator. In some cases, the exercise administrator may wish to simply prepare a written narrative report paraphrasing the comments of the assessors. These will usually include statements concerning how the candidate might correct any deficiencies in performance noted by the assessors. It is usually desirable to schedule individual debriefing sessions as soon as possible after the conclusion of the assessment center. At this time, candidates can be given a copy of the written narrative report and allowed to discuss the conclusions and recommendations contained in it with the exercise administrator. This “one-on-one” level of personal involvement helps to assure the candidate that his or her interests are being protected and that their Agency is totally committed to openness and fairness in the process. It is important that candidates understand that the debriefing process is intended to provide them with useful feedback and construction suggestions on how they can improve their performance in the future. It is not intended to be used as a “gripe session” in which candidates complain about the assessors and the process in general. If the agency has a process whereby candidates are allowed to challenge the results of the assessment center it should be separate and apart from the candidate debriefing session. Those candidates who are open to constructive criticism and who are willing to admit their own mistakes will profit a great deal from these sessions, while those few candidates who are simply looking for an excuse for their own inadequacies will likely find these sessions relatively useless. Videotaped assessor comments can be invaluable in allowing candidates to hear and see, in living color, the assessors’ evaluations of their performance. These comments are usually recorded at the conclusion of the process and may or may not include a recitation of how the candidate was scored by the assessors in each exercise.1 In this way, candidates will have the benefit of knowing how the assessors viewed their performance. This may provide them with a unique insight into their skills and capabilities that may not be possible with traditional promotional testing procedures. Indeed, most testing procedures reveal only a candidate’s score and do not reflect how that score was achieved nor how it might be improved in future testing Local civil service rules might dictate that this information be withheld altogether or until final scores are posted. 1
Ensuring Candidate Satisfaction with Assessment Center Methods
153
processes. If it is neither possible nor desirable to provide videotaped or electronically recorded feedback, written feedback can also be provided. This can be done either by requiring assessors to provide narrative accounts of their observations of the candidates’ strengths and weaknesses, which may prove to be a very laborious task and which will also depend on the ability of the individual assessor to provide such narrative, or by using standardized forms such as shown in Appendix T, which can be produced relatively easily and with satisfactory results. ConClUsion
Like other testing methods, there are no guarantees that persons who score well in an assessment center will perform exactly as predicted once promoted to a new position. However, the predictability of this method is high, and it is for this reason that assessment centers have enjoyed a long history of success in police and fire testing. Contrary to what some may have believed a decade ago, assessment centers are not a thing of the past, but rather have been established as a legitimate and reliable method of assessing future job performance. Despite their reliability and success, however, experience has shown that one poor experience with an assessment center may ruin any chance that might have existed to incorporate this method into promotional procedures. The simple and practical steps outlined above can help to ensure that candidates accept the validity of the process as well as the results produced by it.
Chapter 12 ConClUsion
T
he assessment center method is a valid and reliable means of evaluating a person for promotion or advancement in police departments, fire departments and just about any other occupational field. It is not a fad, but rather a well-established, carefully thought out process that attempts to place the right person in the right job. However, the assessment center process may not be right for all organizations and may not always be the sole or best method of evaluating personnel for promotion or assignment. Written examinations, oral interviews, psychological examinations, on the job performance evaluations, and other methods should not be ruled out as useful and valid means by which to judge a person’s ability to perform successfully in another position. Any attempt to implement an assessment center process for the first time should be approached carefully and with full consideration of what is involved. Each process needs to be tailored to meet the particular needs of the individual organization and the requirements of the position for which candidates are being evaluated. Short cuts and attempts to implement this kind of a process cheaply and easily should be avoided because this can often lead to disastrous results that may make it impossible to “sell” the process in the future. While it may not be necessary to engage an independent consultant to design and administer an assessment center, it is wise to have someone on the staff who has assessment center experience and who has worked as an assessor and who will be responsible for the design and administration of the process. This book has attempted to provide an overview of the assessment center process for both the future candidate and for the municipal administrator who may be evaluating whether or not to implement an assessment center process in the future. For the latter, the best teacher is experience. If you are 154
Conclusion
155
seriously interested in implementing this kind of process in your organization, talk to colleagues who have experience with assessment centers. You will probably find a wide range of views among these persons, but the general consensus will no doubt be favorable. At the same time, you should be aware of the negative comments about assessment centers that often arise (see Chapter 3), and you should be prepared to answer them. I would not have been involved in the design and administration of several hundred assessment centers, involving several thousand candidates, for several hundred positions, if I did not believe in the process and if I could not convince others in the value of the process. I would not have taken the time to write this book if I did not believe that the assessment center process is a valid, reliable, and valuable method of evaluating candidates for future positions. But experience is the best teacher, and the only way the reader will learn whether what I have said here is truth or fiction is to experience the process for themselves.
APPendiCes
Appendix A AssessmenT CenTer GUidelines GUidelines And eThiCAl ConsiderATions for AssessmenT CenTer oPerATions1 international Task force on Assessment Center Guidelines2 This document is an update of several prior editions of guidelines and ethical considerations for assessment center operations dating back to 1975. Each set of guidelines was developed and endorsed by specialists in the research, development, and implementation of assessment centers. The guidelines are a statement of the considerations believed to be most important for all users of the assessment center method. For instance, the use of job-related simulations is a core concept when using the method. Job simulation exercises allow individuals to demonstrate their abilities in situations that are important on the job. As stressed in these guidelines, a procedure should not be represented as an assessment center unless it includes at least one, and usually several, job-related simulations that require the assessee to demonstrate a constructed behavioral response. Other important areas include assessor selection and training, using ‘competencies’ as dimensions to be assessed, validation, participants’ rights, and the incorporation of technology into assessment center programs. The current guidelines discuss a number of considerations in developing and using assessment centers in diverse cultural settings.
International Task Force on Assessment Center Guidelines, “Guidelines and Ethical Considerations for Assessment Center Operations,” International Journal of Selection and Assessment, Vol 17, No 3 (September, 2009). 1 Endorsed by the 34th International Congress on Assessment Center Methods, 24 September 2008, Washington, DC, USA. 2Task Force Members are listed in Appendix A. Inquiries may be sent to Taskforce Co-Chairs Deborah E. Rupp (
[email protected]) and Doug Reynolds (
[email protected]), or any Taskforce member.
159
160
The Assessment Center Handbook for Police and Fire Personnel
1. Purpose This document’s intended purpose is to establish professional guidelines and ethical considerations for users of the assessment center method. These guidelines are designed to cover both existing and future applications. The title ‘assessment center’ is restricted to those methods that follow these guidelines. These guidelines will provide: (1) guidance to industrial/ organizational psychologists, organizational consultants, human resource management (HRM) specialists and generalists, and others who design and conduct assessment centers; (2) information to managers deciding whether or not to institute assessment center methods; (3) instruction to assessors serving on the staff of an assessment center; and (4) guidance on the use of technology in assessments.
2. history of Guidelines The growth in the use of the assessment center method over the last several decades has resulted in a proliferation of applications in a variety of organizations. Assessment centers currently are being used in industrial, educational, military, government, law enforcement, and other organizational settings. From the beginning, practitioners began to raise concerns that reflected a need for standards or guidelines for users of the method. The 3rd International Congress on the Assessment Center Method, which met in Quebec (May 1975), endorsed the first set of guidelines. These were based on the observations and experience of a group of professionals representing many of the largest users of the method. Developments in the period 1975–1979 concerning federal guidelines related to testing, as well as professional experience with the original guidelines, suggested that the guidelines should be evaluated and revised. Therefore, the 1979 guidelines included essential items from the original guidelines, but also addressed the recognized need for: (1) further definitions, (2) clarification of impact on organizations and participants, (3) expanded guidelines on training, and (4) additional information on validation. Since 1979, the use of assessment centers has spread to many different organizations that are assessing individuals representing diverse types of jobs. During this period, pressures to modify the assessment center method came from three different sources. First, there had been attempts to streamline the procedures to make them less time consuming and expensive. Second, new theoretical arguments and evidence from empirical research had been interpreted to mean that the assessment center method does not work exactly as its proponents originally had believed, suggesting that the method should be modified. Third, many procedures purporting to be assessment centers had not complied with previous guidelines, because the guidelines may have been too ambiguous. Revisions in the 1989 third edition were designed to incorporate needed changes and to respond to some of the concerns raised since 1979. The 1989 revision of these guidelines was begun at the 15th International Congress on the Assessment Center Method in Boston (April 1987) when Dr. Douglas Bray held discussions with many attendees. Subsequently, Dr. Bray and Dr. George Thornton solicited additional com-
Appendix A
161
ments from a group of assessment center practitioners. The 1989 Task Force provided comments on drafts of a revision prepared by Bray and Thornton. A later draft was circulated and discussed at the 16th International Congress held in May 1988 in Tampa. The 1989 guidelines were written in response to comments obtained at the 1988 Congress and from members of the Task Force. The 1989 guidelines were endorsed by a majority of the Task Force and by participants at the 17th International Congress held in May 1989 in Pittsburgh. Changes in the 1989 guidelines from prior editions included: (1) specification of the role of job analysis; (2) clarification of the types of attributes/dimensions to be assessed and whether or not attributes/dimensions must be used; (3) delineation of the processes of observing, recording, evaluating, and aggregating information; and (4) further specification of assessor training. The 2000 revision of these guidelines was initiated at the 27th International Congress on Assessment Center Methods in Orlando (June 1999) when Dr. David R. MacDonald conducted discussions with a number of assessment center experts in attendance and also solicited input at a general session regarding aspects of the guidelines that needed to be (re)addressed. A primary factor driving the revision was the passage of a full decade since the 1989 edition. Other factors included a desire to integrate technology into assessment center methods and recognition of the need for more specific definitions of several concepts and terms. Input from members of the Task Force for the 2000 edition was synthesized into a final draft that was presented and endorsed at the 28th International Congress held in May 2000 in San Francisco, attended by 150 participants representing Australia, Belgium, Brazil, Canada, Columbia, Germany, India, Indonesia, Italy, Japan, Mexico, the Netherlands, the Philippines, Singapore, Sweden, Switzerland, Taiwan, the United Arab Emirates, the United Kingdom, and the United States of America. The current revision of these guidelines was initiated at the 32nd International Congress on Assessment Center Methods, which was held in Las Vegas in October 2004. A roundtable discussion addressed contemporary assessment center issues on which there had been little previous guidance. Subsequently, this Congress decided that additions and revisions were needed in two areas: (1) Because of the proliferation of multinational organizations using assessment centers across geographic regions, more guidance was needed on global assessment center practices. The 32nd Congress established a separate task force to examine the issue. A report from this task force served as the foundation for a new section of the guidelines. (2) Given recent research on the effectiveness of various assessor training components, the Congress suggested an expansion of guidelines in this area as well. A second round of discussions on these issues was held in 2006 at the 33rd International Congress in London. These discussions suggested additional guidance in two areas: (1) the use of technology in assessment center practices, and (2) recognition of methodological differences among assessment centers used for different purposes. This revision, co-chaired by Deborah Rupp and Doug Reynolds, was unanimously endorsed by the 34th International Congress (2008, Washington, DC),
162
The Assessment Center Handbook for Police and Fire Personnel
which was attended by delegates representing Austria, Belgium, Canada, China, Germany, India, Indonesia, Mexico, the Netherlands, Romania, Russia, Singapore, South Africa, South Korea, Spain, Sweden, the United Arab Emirates, the United Kingdom, and the United States of America. 3. Assessment Center defined An assessment center consists of a standardized evaluation of behavior based on multiple inputs. Several trained observers and techniques are used. Judgments about behavior are made, in major part, from specifically developed assessment simulations. These judgments are pooled in a meeting among the assessors or by a statistical integration process. In an integration discussion, comprehensive accounts of behavior – and often ratings of it – are pooled. The discussion results in evaluations of the assessees’ performance on the dimensions or other variables that the assessment center is designed to measure. Statistical combination methods should be validated in accordance with professionally accepted standards. Technology may be used to facilitate the writing of reports, presentation of exercise materials, scoring of dimensions, classification of behavior, etc., as long as the essential elements described below are not compromised and validation standards are upheld. There is a difference between an assessment center and assessment center methodology. Various features of the assessment center methodology are used in procedures that do not meet all the guidelines set forth herein, such as when a psychologist or human resource professional, acting alone, uses a simulation as part of an individual’s evaluation. Such personnel assessment procedures are not covered by these guidelines; each should be judged on its own merits. Procedures that do not conform to all the guidelines herein should not be represented as assessment centers or imply that they are assessment centers by using the term ‘assessment center’ as part of the title. The following are the essential elements for a process to be considered an assessment center: 1. Job analysis/competency modeling – A job analysis of relevant behaviors must be conducted to determine the dimensions or competencies important to job success in order to identify what should be evaluated by the assessment center. Throughout this document the terms ‘dimensions’ and ‘competencies’ are used interchangeably. The type and extent of the job analysis depend on the purpose of the assessment, the complexity of the job, the adequacy and appropriateness of prior information about the job, and the similarity of the new job to jobs that have been studied previously. If past job analyses and research are used to select dimensions and exercises for a new job, evidence of the comparability or generalizability of the jobs must be provided. When the job does not currently exist, analyses can be done of actual or projected tasks or roles that will comprise the new job, position, job level, or job family. Target dimensions can also be identified from an analysis of the organization’s vision, values, strategies, or key objectives.
Appendix A
163
Competency-modeling procedures may be used to determine the dimensions to be assessed by the assessment center, if such procedures are conducted with the same rigor as traditional job analysis methods. Rigor in this regard is defined as the involvement of subject matter experts who are knowledgeable about job requirements, the collection and quantitative evaluation of essential job elements, and the production of evidence of reliable results. Any job analysis or competencymodeling process must result in clearly specified categories of behavior that can be observed in assessment procedures. A “competency” may or may not be amenable to behavioral assessment as defined herein. A competency, as used in various contemporary sources, refers to an organizational strength, an organizational goal, a valued objective, a construct, or a grouping of related behaviors or attributes. A competency may be considered a behavioral dimension for the purposes of assessment in an assessment center provided it can be defined precisely and expressed in terms of behaviors observable on the job or in a job family and in simulation exercises. A competency also must be shown to be related to success in the target job or position or job family. 2. Behavioral classification – Behaviors displayed by participants must be classified into meaningful and relevant categories such as behavioral dimensions, attributes, characteristics, aptitudes, qualities, skills, abilities, competencies, or knowledge. In these guidelines, the term ‘dimension’ is used as a general descriptor for each type of behavior category. Note that other classification schemes also may be used. For example, categories may reflect components of the target jobs or the assessment itself. 3. Assessment techniques – The techniques used in the assessment center must be designed to provide information for evaluating the dimensions previously determined by the job analysis. Assessment center developers should establish a link from behaviors to dimensions to assessment techniques. This linkage should be documented in a dimension assessment technique matrix. 4. Multiple assessments–Multiple assessment techniques must be used. These can include tests, interviews, questionnaires, and simulations. The assessment techniques are developed or chosen to elicit a variety of behaviors and information relevant to the selected dimensions. Self-assessment and 3601 assessment data may be gathered as assessment information. The assessment techniques will be pretested to ensure that they provide reliable, objective, and relevant behavioral information for the organization in question. Pretesting might entail trial administration with participants similar to assessment center candidates, thorough review by subject matter experts as to the accuracy and representativeness of behavioral sampling, and/or evidence from the use of these techniques for similar jobs in similar organizations. 5. Simulations–The assessment techniques must include a sufficient number of job-related simulations to allow opportunities to observe the candidate’s behavior related to each dimension/competency being assessed. At least one – and usually several – job-related simulations must be included in each assessment center. A simulation is an exercise or technique designed to elicit behaviors related to dimensions of performance on the job requiring the participants to respond behaviorally to situational stimuli. Examples of simulations include, but are not limited to,
164
The Assessment Center Handbook for Police and Fire Personnel
group exercises, in-basket exercises, interaction (interview) simulations, presentations, and fact-finding exercises. Stimuli also may be presented through video, audio, or virtual simulations delivered via computer, telephone, video, the Internet, or an intranet. For simple jobs, one or two job-related simulations may be used if the job analysis clearly indicates that one or two simulations alone sufficiently simulate a substantial portion of the job being evaluated. If a single comprehensive assessment technique is used, then it must include distinct job-related segments. Assessment center designers should also be careful to design exercises that reliably elicit a large number of dimension-related behaviors. This should provide assessors with sufficient opportunities to observe dimension-related behavior. The stimuli contained in a simulation should parallel or resemble stimuli in the work situation, although they may be in different settings. The desirable degree of fidelity is a function of the assessment center’s purpose. Fidelity may be relatively low for early identification and selection programs for non-managerial personnel and may be relatively high for programs designed to diagnose the training needs of experienced managers. Assessment center designers should be careful that the exercise content does not favor certain assessees (e.g., those in certain racial, ethnic, age, or sex groups) for irrelevant reasons. To qualify as a behavioral simulation for an assessment center as defined herein, the assessment method must require the assessee to overtly display certain behaviors. The assessee must be required to demonstrate a constructed response. Assessment procedures that require the assessee to select only among provided alternative responses, such as seen in multiple-choice tests or multiple-choice computerized in-baskets, do not conform to this requirement. Likewise, a situational interview that calls for only an expression of behavioral intentions does not conform. Neither do ‘low-fidelity’ simulations and situational interviews. Although they may yield highly reliable and valid assessment ratings, they do not constitute the behavioral assessment required in assessment centers. Assessment center materials (e.g., simulations and other exercises, rating scales, assessor training materials) are often intellectual property protected by international copyright laws. Respect for copyrights and the intellectual property of others must be maintained under all circumstances. 6. Assessors–Multiple assessors must be used to observe and evaluate each assessee. When selecting assessors, where appropriate, the program should strive to have diverse assessors, both in terms of demographics (e.g., race, ethnicity, age, sex) and experience (e.g., organizational level, functional work area, managers, psychologists, etc.). The maximum ratio of assessees to assessors is a function of several variables, including the type of exercises used, the dimensions to be evaluated, the roles of the assessors, the type of integration carried out, the amount of assessor training, the experience of the assessors, and the purpose of the assessment center.
Appendix A
165
A participant’s current supervisor should not be involved in the assessment of a direct subordinate when the resulting data will be used for selection or promotional purposes. 7. Assessor training–Assessors must receive thorough training and demonstrate performance that meets the guidelines in the “Assessor Training” section of this document before participating in an assessment center. 8. Recording behavior and scoring – A systematic procedure must be used by assessors to record specific behavioral observations accurately at the time of observation. This procedure might include techniques such as handwritten notes, behavioral observation scales, or behavioral checklists. Audio and video recordings of behavior may be made and analyzed at a later date. When technology-based tools are used during the scoring process, these approaches should be evaluated for reliability and accuracy. Assessors must prepare a record of the observations made during each exercise before the integration discussion or statistical integration. 9. Data integration–The integration of each individual’s behaviors (individual dimension scores aggregated across exercises; exercise-specific scores; or potentially, depending on the purpose of the assessment center, across-exercise scores aggregated into an overall assessment rating) must be based on pooled information from assessors or through a statistical integration process. The process used should be carried out in accordance with professionally accepted standards. If an integration discussion is used, assessors should consider information derived from the assessment techniques for each dimension, but should not consider information irrelevant to the dimensions or the purpose of the assessment process. Methods of combining assessors’ evaluations of information discussed in their integration sessions must be based on reliable individual assessors’ ratings. Computer technology may also be used to support the data integration process, provided the conditions of this section are met. 4. nonassessment Center Activities The following kinds of activities do not constitute an assessment center: 1. Assessment procedures that do not require the assessee to demonstrate overt behavioral responses are not behavioral simulations; thus, any assessment program that consists solely of such procedures is not an assessment center as defined herein. Examples of these are computerized in-baskets calling only for multiple choice responses, situation interviews calling only for behavioral intentions, and written competency tests. Procedures not requiring an assessee to demonstrate overt behavioral responses may be used within an assessment center, but must be coupled with at least one simulation requiring the overt display of behaviors. 2. Panel interviews or a series of sequential interviews as the sole technique. 3. Reliance on a single technique (regardless of whether it is a simulation) as the sole basis for evaluation. However, a single comprehensive assessment technique
166
The Assessment Center Handbook for Police and Fire Personnel
that includes distinct job-related segments (e.g., large, complex simulations or virtual assessment centers with several definable components and with multiple opportunities for observation in different situations) is not precluded by this restriction. 4. Using only a test battery composed of a number of paper-and-pencil measures, regardless of whether the judgments are made by a statistical or judgmental pooling of scores. 5. Single-assessor evaluation (i.e., measurement by one individual using a variety of techniques, such as paper-and-pencil tests, interviews, personality measures, or simulations). 6. The use of several simulations with more than one assessor but with no pooling of data (i.e., each assessor prepares a report on performance in an exercise, and then the individual, un-integrated reports are used as the final product of the center). 7. A physical location labeled as an “assessment center” that does not conform to the methodological requirements noted above. 5. Assessment Center Policy statement Assessment centers operate more effectively as part of an integrated human resource system. Before the introduction of an assessment center into an organization, a policy statement should be prepared and approved by the organization. This policy statement should address the following areas: 1. Objective–An assessment center may be used for a variety of purposes. Falling into the broad categories of selection vs diagnosis vs development, such purposes might include prescreening, hiring, early identification and evaluation of potential, performance appraisal, succession planning, and professional development. An assessment center participant should be told, before the assessment, what decision(s) will or might be made with assessment center data. If the organization desires to make decisions with the data other than those previously communicated to the participant, the decision(s) should be clearly described to the participant and consent obtained. In addition, the assessment center developer must remain cognizant of the assessment center’s purpose when designing, developing, implementing, and validating the program. Decisions about the choice of dimensions, content of simulations, selection and training of assessors, scoring, feedback, and evaluation all should be made with the objective in mind. 2. Assessees–The population to be assessed, the method for selecting assessees from this population, the procedure for notification, and the assessment process should be specified. 3. Assessors–The assessor population (including sex, age, race, and ethnic mix), limitations on the use of assessors, assessor experience, and evaluation of assessor performance and certification requirements, where applicable, should be specified. 4. Use of data–The process flow of assessment records within the organization, individuals who receive reports, restrictions on access to information, procedures
Appendix A
167
and controls for research and program evaluation purposes, feedback procedures to management and employees, and the length of time data will be maintained in files should all be specified. Particularly for a selection application, it is recommended that the data be used within 2 years of the date of administration because of the likelihood of change in the participant or the organization. 5. Qualifications of consultant(s) or assessment center developer(s)–The internal or external consultant( s) responsible for the development of the center or of the exercises/simulations for the center should be identified and his or her professional qualifications and related training specified. 6. Validation–The statement should specify the validation model being used. If a content-oriented validation strategy is used, the documentation of the relationship of the job/job family content to the dimensions and exercises should be presented along with evidence of the reliability of the observations and rating of behavior. If evidence is being taken from prior validation research, which may have been summarized in meta-analyses, the organization must document that the current job/job family and assessment center are comparable and generalized to the jobs and assessment centers studied elsewhere. If local criterion-related validation has been carried out, full documentation of the study should be provided. If validation studies are under way, there should be a schedule indicating when a validation report will be available. Finally, the validation technique employed should be appropriate given the purpose/objective of the assessment center program (e.g., selection vs development). 7. Legal context – Laws existing both in an organization’s/ agency’s home state, province, or nation – as well as in the state, province, or nation where the assessment center program is being carried out – may have implications for program design, validation, implementation, and documentation. Most nations have disadvantaged and protected groups (such as native/ aboriginal people, recent immigrants, racial groups, religious groups, and groups defined by age, political opinion, sexual orientation, etc.) with accompanying regulations for legal protections for their members. The policy statement should articulate the particular laws and policies that are relevant for the assessment center program and state how legal compliance will be ensured. 6. Assessor Training Assessor training is an integral part of the assessment center program. Assessor training should have clearly stated training objectives, performance guidelines, and quality standards. The following issues related to training must be considered: 1. Training content–Whatever the approach to assessor training, the objective is to obtain reliable and accurate assessor judgments. A variety of training approaches may be used (e.g., lectures, discussion, observation of practice candidates, video demonstrations, observation of other assessors) as long as it can be shown that reliable, accurate assessor judgments are obtained. At a general level, all assessor training programs should include training on:
168
The Assessment Center Handbook for Police and Fire Personnel • The dimensions to be assessed, including their behavioral definitions. • The observation, recording, classification, and evaluation of behaviors relevant to the dimensions to be assessed. • The content of the exercises as well as which dimensions are targeted in which exercises. • The avoidance of common observational and rating errors (including how to distinguish behaviors from inferences).
Depending on the purpose of the assessment center, the training might include additional components such as professionalism, knowledge of the organization, knowledge of the target job, the ability to give accurate oral or written feedback, and consistency in role playing. The following minimum training goals are required: a. Knowledge of the organization and job/job family or normative group being assessed to provide an effective context for assessor judgments where appropriate. b. Thorough knowledge and understanding of the assessment dimensions, their definitions, their relationship to job performance, and examples of effective and ineffective performance. c. Thorough knowledge and understanding of the assessment techniques, exercise content, relevant dimensions to be observed in each portion of the assessment center, expected or typical behavior, examples or samples of actual behaviors, etc. d. Demonstrated ability to observe, record, and classify behavior in dimensions, including knowledge of the protocol for documenting behavior. e. Thorough knowledge and understanding of evaluation and rating procedures, including how data are integrated. f. Thorough knowledge and understanding of assessment policies and practices of the organization, including restrictions on how assessment data are to be used, when this is a requirement of assessors. g. Thorough knowledge and understanding of feedback procedures and strategies, where appropriate, to maximize assessees’ acceptance of feedback and behavior change. h. Demonstrated ability to give accurate oral and written feedback, when feedback is given by the assessors. i. Demonstrated knowledge and ability to play objectively and consistently the role called for in interactive exercises (e.g., one-on-one simulations or factfinding exercises) when role playing is required of assessors. Non-assessor role players also may be used if their training results in their ability to play the role objectively and consistently. 2. Training length–The length of assessor training may vary due to a variety of considerations that can be categorized into three major areas: a. Trainer and instructional design considerations:
Appendix A
169
• The instructional mode(s) utilized. • The qualifications and expertise of the trainer. • The training and instructional sequence. b. Assessor considerations: • Previous knowledge and experience with similar assessment techniques. • Type of assessors used (e.g., professional psychologists vs managers). • Experience and familiarity with the organization and the target position(s)/ job(s)/job families/target level. • The frequency of assessor participation. • Other related qualifications and expertise (e.g., testing and assessment, executive coaching). c. Assessment program considerations: • The target position’s level of difficulty. • The number of dimensions/competencies to be rated. • The anticipated use of the assessment information (e.g., immediate selection, broad placement considerations, diagnosis, development). • The number and complexity of the exercises. • The division of roles and responsibilities between assessors and others on the assessment staff (e.g., administrator and other support staff). • The degree of support provided to assessors in the form of observation and evaluation guides. It should be noted that length and quality of training are not synonymous. Precise guidelines for the minimum number of hours or days required for assessor training are difficult to specify. One day of training may be sufficient for a well-structured assessment center using a small number of exercises, a qualified trainer, and carefully selected assessors. However, extensive experience has shown that, for the initial training of assessors who have no experience in an assessment center that conforms to the guidelines in this document, more training may be needed (e.g., 2 days of assessor training for each day of assessment center exercises). Assessors who have experience with similar assessment techniques in other programs may require less training. More complex assessment centers with varied formats of simulation exercises may require additional training; simple assessment centers may require less. In any event, assessor training is an essential aspect of an assessment program. The true test of training quality should be assessor competence as described below. 3. Performance guidelines and certification – Each assessment center should have clearly stated performance guidelines for assessors contingent on the purpose of the assessment center and the various assessor roles. These performance guidelines should include, at a minimum, the ability to:
170
The Assessment Center Handbook for Police and Fire Personnel
a. Rate behavior in a standardized fashion. b. Recognize, observe, and report behavior into the appropriate dimensions, etc. c. Administer an exercise if the assessor serves as an exercise administrator. d. Play objectively and consistently the role called for in those interactive exercises where role playing is required of assessors. e. If applicable, deliver positive and negative behavioral feedback, if applicable, with supporting evidence in a manner that conveys concern/empathy and maintains or enhances the assessee’s self-esteem. f. Motivate assessees and engage in coaching, action planning, and goal setting. Some measurement is needed to indicate that the individual being trained is capable of functioning as an assessor. This measurement may vary and could include data in terms of (1) accuracy and reliability of rating performance (defined with regard to either an ‘expert’ standard or convergence with other assessors), (2) critiques of assessor reports, and (3) observation or shadowing of assessors in training by the assessment center staff. It is important that, before carrying out their actual duties, assessors’ performance is evaluated to ensure that they are sufficiently trained to function as assessors and that such performance is periodically monitored to ensure that the skills learned in training are applied. Each organization must be able to demonstrate that its assessors can meet minimum performance standards. This may require the development of additional training or other prescribed actions for assessors not meeting these performance guidelines. The trainer of assessors should be competent to enable individuals to develop the assessor skills stated above and to evaluate the acquisition of these skills. 4. Currency of training and experience – The time between assessor training and initial service as an assessor should be short (e.g., not to exceed 6 months). If a longer period has elapsed, prospective assessors should attend a refresher course or receive special coaching from a trained assessment center administrator. Assessors who do not have recent experience as an assessor (i.e., fewer than two assessment centers over 2 consecutive years) should attend a refresher course before they serve again or receive special coaching from a trained assessment center administrator. 7. informed Participation The organization is obligated to make an announcement before the assessment so that participants will be fully informed about the program. Ideally, this information should be made available in writing before the center. A second option is to present the information at the start of the program. While the information provided will vary across organizations, the following basic information should be given to all participants:
Appendix A
171
1. Objective(s)–The objective(s) of the program and the purpose of the assessment center. The organization may choose to disclose the dimensions measured and the general nature of the exercises before the assessment. 2. Selection–How individuals are selected to participate in the center. 3. Choice(s)–Any options the individual has regarding the choice of participating in the assessment center as a condition of employment, advancement, development, etc. 4. Staff–General information on the assessor staff and the role of the observer, including composition, relevant experience, and assessor training. 5. Materials–What assessment center materials completed by the individual are collected and maintained by the organization. 6. Results–How the assessment center results will be used, what recommendations will be made, and how long the assessment results will be maintained on file. 7. Feedback–When, how (e.g., written, face-to-face, technology-aided), and what kind of feedback (e.g., by dimension, by exercise, by a combination) will be given to the participants. 8. Development–Mechanisms for follow-up support and monitoring, if any (e.g., external coaching, training, mentoring; top management/supervisory support). 9. Alignment–How the assessment center results will be aligned with organizational strategy and culture, and how the results will be integrated with other HRM systems. 10. Reassessment–The procedure for reassessment (if any). 11. Access–Who will have access to the assessment center reports (and audio and/or video files, if applicable) and under what conditions. 12. Contact–Who will be the contact person responsible for the records and where the results will be stored or archived. 8. Validation issues A major factor in the widespread acceptance and use of assessment centers is related directly to an emphasis on sound validation research. Numerous studies demonstrating the predictive validity of individual assessment center programs have been conducted in a variety of organizational settings and reported in the professional literature. However, the historical record of this process’s validity cannot be taken as a guarantee that a given assessment program (or new application of an existing program) will or will not be valid. Ascertaining the validity of an assessment center program is a complicated technical process, and it is important that validation research meets both professional and legal standards. Research should be conducted by individuals knowledgeable in the technical and legal issues pertinent to validation procedures. In evaluating the validity of assessment center programs, it is particularly important to document the selection of the dimensions assessed in the center. The relationship of assessment exercises to the dimensions assessed should be documented as well.
172
The Assessment Center Handbook for Police and Fire Personnel
Validity generalization studies of assessment center research suggest that overall assessment ratings derived in a manner conforming to these guidelines show considerable predictive validity. Such findings support the use of a new assessment center in a different setting if the job, exercises, assessors, and assessees in the new situation are similar to those studied in the validation research and if similar procedures are used to observe, report, and integrate the information. The validity generalization studies of the predictive validity of the overall assessment rating do not necessarily establish the validity of the procedure for other purposes such as diagnosis of training needs, accurate assessment of skill level in separate dimensions, or the developmental influence of participation in an assessment center. The standards and principles for validation appear in Principles for the Validation and Use of Personnel Selection Procedures (Society for Industrial and Organizational Psychology Inc., 2003) and Standards for Educational and Psychological Testing (American Educational Research Association, American Psychological Association, & National Council on Measurements in Education, 1999). In principle, technology can be used for writing reports, presenting exercises, scoring dimensions, classifying behavior, or creating behavioral checklists, as long as the assessment process remains valid. 9. Assessment Centers for different Purposes Assessment centers are generally used for three major purposes: (1) to predict future behavior for decision making, (2) to diagnose development needs, and (3) to develop candidates on dimensions of interest. However, additional purposes for the assessment center method currently exist and will continue to evolve with further use. The design and operation of an assessment center may vary, depending on its intended purpose. For example, with assessments designed purely to support a personnel decision (e.g., promotion), the emphasis may be on a reliable and valid overall assessment rating. There may be little reason to generate individual dimension scores in this context. Alternatively, diagnostic assessment centers may require the generation of reliable and valid dimension scores only. Developmental assessment centers (DACs) seek to both assess and develop dimensions that may be learned (i.e., skills and competencies that can be improved upon in a reasonable amount of time). DACs involve multiple points of feedback and repeated practice, and may repeat exercises of the same type(s) as a way to track improvement on the dimensions over time. As a result, such programs may be longer than assessment centers for prediction and diagnosis. Feedback is an essential component of a DAC program, and in order to foster learning, this feedback needs to be immediate. Often, the role of assessors in DACs is not only to observe and evaluate behavior, but also to facilitate learning and development by delivering feedback and facilitating development planning during the assessment process. If the focus is purely on learning, DACs may be less standardized than assessment centers used for personnel decision making, and they may be customized to meet participants’ developmental needs. When validating or otherwise evaluating DACs, the appropriate criterion is change in participants’ understanding,
Appendix A
173
behavior, and proficiency on targeted dimensions. Because of the differences outlined above, assessment centers must be designed and implemented appropriately for their intended purpose. 10. rights of the Participant Assessment center activities typically generate a large volume of data on an individual who has gone through a center. These assessment data come in many forms and may include observer notes, reports on performance in the exercises, assessor ratings, peer ratings, paper-andpencil or computerized tests, video files, and final assessment center reports. This list, while not exhaustive, does indicate the extent of information about an individual that may be collected. The following guidelines for use of these data are suggested: 1. Assessees should receive feedback on their assessment center performance and should be informed of any recommendations made. 2. Assessees who are members of the organization have a right to read any formal summary written reports concerning their own performance and recommendations that are prepared and made available to management. Applicants to an organization should be provided with, at a minimum, what the final recommendation is and, if possible and if requested by the applicant, the reason for the recommendation. 3. To ensure test security, assessment center exercises and assessor reports on performance in particular exercises are exempted from disclosure, but the rationale and validity data concerning ratings of dimensions and the resulting recommendations should be made available on request of the individual assessee. 4. The organization should inform the assessee what records and data are being collected, maintained, used, and disseminated. Assessees should be informed if their activities in the assessment center are being recorded as well as if such recordings or other personal datawill be transferred across national borders or over the Internet. The organization must take precautions to ensure the security of data transferred over the Internet. Organizations that collect personal data during the assessment process should comply with applicable data protection regulations, such as the European Union Directive on Data Protection and the US Safe Harbor Privacy Principles. 5. If the organization decides to use assessment results for purposes other than those originally announced and that can have an impact on the assessee, then the assessee must be informed and consent obtained. 11. Conducting Assessment Centers Across Cultural Contexts It is common for single assessment center programs to cross both cultural and national boundaries. In some situations it may be necessary to adapt many assessment center practices to the local culture in which an assessment center is deployed. In other situations it may be decided that an assessment center program should be standardized across all regions to which it is being deployed. Practitioners using assessment center methods beyond the boundaries of the country/re-
174
The Assessment Center Handbook for Police and Fire Personnel
gion from which the assessment center program originated should determine the extent to which cultural accommodations may be necessary. The analysis should include evidence that the validity and applicability of the assessment center have not been compromised at either the design or implementation phase. A range of contextual factors should be considered during such a process, including: • Extent of commonality in the cultural, business, legal, and socio-political environments between countries (e.g., cultural beliefs and behaviors, local business laws). • Differences in national guidelines set by local professional associations. • Commonality of skills/dimensions critical for job success. • Commonality in performance standards/behaviors required for job success. • Extent of commonality of the business models between the organizations across which the assessment center/method is being adapted (i.e., overall business strategy, vision, values, and practices). • Degree of centralized vs decentralized (i.e., local) control across branches of the organization. • Whether comparison statistics (e.g., normative comparisons) are required to interpret the results across locations. • Extent to which personnel need to be transferred across countries/cultures. When assessment center programs are designed to be culturally specific, the following aspects should be considered for modification based on each culture in which the assessment center is used: • • • • •
Selection of performance criteria. Criteria for occupational success. Selection of exercises. Assessor training. Feedback process.
In contrast, several aspects of the assessment center process should remain standardized, even when the process has been culturally adapted. Features that should remain the same across cultures include: • • • •
Inclusion of behavioral observation. Training of assessors in the process of behavioral observation. Classification and rating of behavior. A systematic process of integrating evaluations across exercises, dimensions, and assessors.
Appendix A
175
12. national Assessment Center Guidelines In addition to the international guidelines presented herein, some countries have developed local guidelines to govern (in parallel with the international guidelines) assessment center practices in their specific national contexts. Examples of national standards include: • South Africa: Assessment Centre Study Group. (2007). Guidelines for assessment and development centres in South Africa (4th ed.). • Germany: Standards der assessment-center-technik. (2004). U¨ berblick und Hintergrundinformationen Hamburg, Germany: Arbeitskreis Assessment Center e.V. • Indonesia: Indonesian Task Force on Assessment Center Guidelines. (2002). Daya Dimensi Indonesia. Ethical guidelines for assessment center operations.
references American Educational Research Association, American Psychological Association, & National Council on Measurements in Education. (1999). Standards for Educational and Psychological Testing. Washington, DC: American Psychological Association. Available at http://www.apa. org/science/standards.html (accessed 1 August 2008). American Psychological Association Council of Representatives. (1990). APA Guidelines for Providers of Psychological Services to Ethnic, Linguistic, and Culturally Diverse Populations. Boston: American Psychological Association. Available at http://www.apa.org/ pi/guide.html (accessed 1 August 2008). American Psychological Association Public Interest Directorate and Council of Representatives. (2002). Guidelines on Multicultural Education, Training, Research, Practice, and Organizational Change for Psychologists. Washington, DC: American Psychological Association. Available at http://www.apa. org/pi/multiculturalguidelines.pdf (accessed 1 August 2008). International Test Commission. (2000). International Test Commission Test Adaptation Guidelines. International Test Commission. Available at http://www.intestcom.org/test_ adaptation.htm (accessed 1 August 2008). Society for Industrial and Organizational Psychology & American Psychological Association. (2003). Principles for the Validation and Use of Personnel Selection Procedures (4th ed). Bowling Green, OH: Society for Industrial and Organizational Psychology. Available at http://www.siop. org/_Principles/principles.pdf (accessed 1 August 2008). U.S. Department of Commerce. (2000). Safe harbor privacy principles. Available at http://www.export.gov/safeharbor (accessed 1 August 2008).
Table A1. Task force members 1975 Edition Albert Alon Douglas W. Bray, PhD William C. Byham, PhD Lois A. Crooks
Miracle Food Mart (Canada) AT&T Development Dimensions International, Inc. Educational Testing Service
176
The Assessment Center Handbook for Police and Fire Personnel
Donald L. Grant, PhD Lowell W. Hellervik, PhD James R. Huck, PhD Cabot L. Jaffee, PhD Alan I. Kraut, PhD John H. McConnell Leonard W. Slivinski, PhD Thomas E. Standing, PhD Edwin Yager 1979 Edition Albert Alon Dale Baker Douglas W. Bray, PhD William C. Byham, PhD Steven L. Cohen, PhD Lois A. Crooks Donald L. Grant, PhD Milton D. Hakel, PhD Lowell W. Hellervik, PhD James R. Huck, PhD Cabot L. Jaffee, PhD Frank M. McIntyre, PhD Joseph L. Moses, PhD (Chair) Nicky B. Schnarr Leonard W. Slivinski, PhD Thomas E. Standing, PhD Edwin Yager 1989 Edition Virginia R. Boehm, PhD Douglas W. Bray, PhD (Co-Chair) William C. Byham, PhD Anne Marie Carlisi, PhD John J. Clancy Reginald Ellis Joep Esser Fred Frank, PhD Ann C. Gowdey Dennis A. Joiner Rhonda Miller Marilyn Quaintance-Gowing, PhD Robert F. Silzer, PhD George C. Thornton III, PhD (Co-Chair)
AT&T and University of Georgia University of Minnesota AT&T–Michigan Bell Telephone Company Assessment Designs, Inc. International Business Machines American Management Association Public Service Commission (Canada) The Standard Oil Company–Ohio Consulting Associates
Miracle Food Mart (Canada) US Civil Service Commission AT&T Development Dimensions International, Inc. Assessment Designs Inc. Educational Testing Service University of Georgia Ohio State University University of Minnesota Human Resources International Assessment Designs Inc. Consulting Associates AT&T International Business Machines Public Service Commission (Canada) Standard Oil of Ohio Consulting Associates Assessment & Development Associates Development Dimensions International, Inc. Development Dimensions International, Inc. BellSouth Clancy & Associates Canadian National Railway Mars B.V. (the Netherlands) Electronic Selection Systems Corporation Connecticut Mutual Joiner & Associates New York Power Authority US Office of Personnel Management Personnel Decisions Inc. Colorado State University
Appendix A 2000 Edition William C. Byham, PhD Richard Flanary Marilyn K. Gowing, PhD James R. Huck, PhD Jeffrey D. Kudisch, PhD David R. MacDonald, PhD (Chair) Patrick T. Maher, DCrim Jeroen J. J. L. Seegers George C. Thornton III, PhD 2008 Edition William C. Byham, PhD Anuradha Chawla, PhD Alyssa Mitchell Gibbons, PhD Sebastien Houde, MSc Dennis Joiner, MS Myungjoon Kim, PhD Diana Krause, PhD Jeffrey D. Kudisch, PhD Cara Lundquist, MA David R. MacDonald, PhD Patrick T. Maher, DCrim Doug Reynolds, PhD (Co-Chair) Deborah E. Rupp, PhD (Co-Chair) Deidra J. Schleicher, PhD Jeroen J. J. L. Seegers, PhD George C. Thornton III, PhD
177
Development Dimensions International Inc. National Association of Secondary School Principals US Office of Personnel Management Human Resources International University of Southern Mississippi Steelcase, Inc. Personnel & Organization Development Consultants, Inc. Assessment & Development Consult (the Netherlands) Colorado State University
Development Dimensions International, Inc. RHR International Colorado State University University of Guelph & Royal Military College of Canada Dennis A. Joiner & Associates Korean Psychological Testing Institute DHV Speyer University of Maryland Southern California Edison Steelcase, Inc. Personnel & Organization Development Consultants, Inc. Development Dimensions International Inc. University of Illinois at Urbana-Champaign Purdue University Right Management Benelux (the Netherlands) Colorado State University
Glossary Assessee: An individual whose competencies are measured by an assessment center. Assessment center: A process employing multiple techniques and multiple assessors to produce judgments regarding the extent to which a participant displays selected behavioral dimensions. Assessor: An individual trained to observe, record, classify, and make reliable judgments about the behaviors of assessees.
178
The Assessment Center Handbook for Police and Fire Personnel
Competency: See Dimension. Developmental assessment center: An assessment center designed for the purpose of directly developing/improving assessees on the dimensions of interest. Dimension: Also called competency. A constellation or group of behaviors that are specific, observable, and verifiable and that can be reliably and logically classified together and that relate to job success. Feedback: Information comparing actual performance to a standard or desired level of performance. High (or low) fidelity: The extent to which an assessment center simulation requires the assessee to actually display jobrelevant behaviors related to one or more select dimensions. Fidelity is related to the realism of the simulation as compared with an actual job situation, task, etc. Job analysis: The process used to determine the behavioral dimensions linked to success or failure in a job, job role, or job grouping. The process typically consists of a combination of techniques to collect job information, such as interviews with and observations of incumbents, job checklists, interviews with upper-level managers/executives, and review of existing job documentation (job descriptions, training manuals, etc.). Reliability: The extent to which a measurement process yields the same results (given identical conditions) across repeated measurements. Simulation: An exercise or technique designed to elicit behaviors related to dimensions of performance on the job requiring the participants to respond behaviorally to situational stimuli. Validity: The extent to which a measurement tool or process, such as an assessment center, yields useful results. Multiple validities might be measured (e.g., ‘construct,’ ‘content,’ ‘face,’ ‘predictive,’ ‘social’) depending on the questions being explored and the tool or process being investigated.
Appendix b TYPiCAl in-bAskeT iTems PAnhAndle PoliCe dePArTmenT Telephone message To:
Lieutenant Collins
From:
Mary Reed
Phone: (413) 222-1234
Date:
January 10, 2010
Time:
Subject:
Rude Conduct
(
) Returned your call
( XX ) Please call
4:45PM
(
) Will call back
Message: Mrs. Reed is very upset because one of your officers was rude to her and acted in a very callous and unprofessional manner when trying to get information from her regarding the whereabouts of her son, Emanuel, who is wanted by this agency for armed robbery. She does not feel she should be treated this way and she wants to speak with you personally about this matter or she will file charges of harassment with the District Attorney's office. explanation: This is an item that can be delegated to a subordinate supervisor or another shift commander to investigate. even though the complainant refused to be identified, this is still a valid complaint and needs to be investigated.
179
180
The Assessment Center Handbook for Police and Fire Personnel hollY Glen fire dePArTmenT
e-mail message To:
Chief Jackson
From:
Lieutenant Sue Casey
Date:
January 13, 2010
Time:
1330 hours
Subject:
Grievance:
Please be advised that I wish to file a grievance over the score I received in the oral examination for Fire Captain last month. Despite the fact that I have more than 15 years of service and a good work record with no discipline, I was rated as “fails to meet standards” by the assessors. I believe this is due to the fact that one of the assessors, Deputy Chief Mel Brooks of the Fair Oaks Fire Department, is a good friend of my ex husband, who recently retired from the Fair Oaks Fire Department. Deputy Chief Brooks and my husband have been close for years and I know that Brooks would do anything to help my husband if asked. I also know that my husband was overheard telling someone else that he would be sure that I never get promoted if he has anything to do with it. There is no question in my mind that Deputy Chief Brooks was acting in my husband’s interest in blackballing me in this process and getting the other assessors to rate me as “fails to meet standards”. I would like to bring this matter to the Police and Fire Commission and am asking you for assistance in this matter. explanation: This is a relatively routine matter that, while the accusation is serious, should probably be handled in accordance with the department’s grievance procedure. item At 6:00PM, Officer Patsy Newsom, a young probationary officer on your shift, comes into your office and asks to speak with you in private. When you agree to speak with her she informs you that she believes that Officer John Mayberry, her Field Training Officer, may be drinking on duty. She tells you that she is sure that Officer Mayberry has a bottle in his locker and she has seen him take a drink from it on two different occasions. He told her “not to worry about it” and that it would be
Appendix B
181
“bad for your career” if she told anyone about what she saw. She does not want to make a formal complaint because she does not want to jeopardize her career, but she is afraid to go on the road with him when he is drinking. She asks you for assistance in this matter. explanation: This is a serious matter and needs to be looked into without delay. An internal investigation should be launched and appropriate personnel informed of the situation.
Appendix C GUidelines for sCorinG The in-bAskeT eXerCise ToWn of PeliCAn bAY
Police Chief Assessment Center Assessor Guidelines for scoring the in-basket exercise introduction: The following guidelines are intended to assist you in scoring the In-Basket Exercise. While there are rarely any “perfect” or absolutely correct ways of handling any of the items in the In-Basket, some ways of handling them are obviously preferred over others. item
recommended Action
explanation
recommended Priority
A: Telephone This is an important item, message since improper conduct from Mary by an officer is alleged. Reed regarding rude conduct.
Refer to patrol commander to speak with Mrs. Reed and the officer(s) involved and pre- Important pare a report for his/her review within one
b: E-Mail message from Lt. Casey regarding a grievance.
Have Lt. Casey submit a formal grievance in accordance with the department’s grievance Routine procedure and remind her to follow the chain of command.
Explanation: This is a relatively routine matter that, while the accusation is serious, should probably be handled in accordance with the department’s grievance procedure.
182
Appendix C C: Information from Officer Newsom about a senior officer drinking on duty.
This is a very serious matter and needs to be looked into without delay. An internal investigation should be launched and the appropriate personnel informed of the situation.
Contact proper authority to initiate an internal investigation and inform all personnel as required by local policy. Urgent
183
Appendix d GroUP Problem-solVinG eXerCise CiTY of eAsT VAlleY
Police sergeant Assessment Center Group Problem-solving exercise instructions: In this exercise, you are to assume that you are a Sergeant in the East Valley Police Department. The Chief of Police has assigned you to a committee consisting of other Sergeants in the Department. Your committee has been asked to discuss the problem described below and make a recommendation to the Chief concerning how it may be solved. Your group will have one hour to discuss the problem and to reach an agreement concerning what recommendation to make to the Chief of Police. Each person in the group may have a different view concerning the issue discussed, and it is important that all views be thoroughly discussed before a decision is reached. You will be evaluated in this exercise on how well you work within the group to reach a decision, your communication skills, and your ability to actively influence the group in reaching a decision. statement of Problem: The Chief of Police has asked your committee to identify ways in which police services in East Valley can be more appropriately tailored to the needs and expectations of the community. In essence, what the chief is interested in is how to improve the East Valley version of the community-oriented/problem-oriented style of policing that has been adopted by a number of other communities in the United States. The Chief would like your committee to discuss various ways in which police services 184
Appendix D
185
can be more readily adapted to the unique needs of local residents. In particular, the Chief is interested in ways in which to: 1. Involve local residents more in police department policy-making and establishing police priorities. 2. Obtain citizen feedback regarding the provision of police services. 3. Make police officers more sensitive to citizen needs and concerns. 4. Bring about a closer working relationship between citizens and individual police officers. The Chief would like your committee to study these issues and report back to him with specific recommendations, not just general or vague suggestions. He wants recommendations that, if approved, can be put into action with a minimum of delay and difficulty. Your committee will have one hour to discuss these issues and formulate recommendations to the Chief.
Appendix e TACTiCAl fire Problem eXerCise CiTY of freePorT
fire Captain Assessment Center Tactical exercise instructions: Note: This same situation is being presented to all candidates in this assessment center. Therefore, it is important that you refrain from discussing the exercise with your colleagues until after all candidates have completed the exercise. Otherwise, it is possible that candidates later in the process may be able to use information obtained from other candidates to their advantage. You alone will be responsible for ensuring that this does not happen! In this exercise, you will play the role of a Captain in the Freeport Fire Department. You are going to be presented with a tactical situation at a fictitious structure that has been inserted into an actual location in the city. The location of the fire and conditions surrounding the fire will be shown to you using simulation software developed for this purpose. This simulation is meant to be interactive so that the intensity of the incident will be dictated by the quality of your decision-making process. In other words, if you make the proper tactical decisions in a timely manner, the fire scene can be effectively controlled and the simulator will display that fact accordingly. Conversely, if you do not take the appropriate actions in a timely manner, the fire scene will gain intensity and the simulator will reveal that circumstance. You will also be provided with a map of the surrounding area. For the purpose of this exercise, you will serve as the Incident Commander and will retain that role throughout the exercise.
186
Appendix E
187
When you enter the exercise room, you will also find a projection of the map on a wall board which should help you orient yourself to the location. You are urged to use the marking pens that will be provided to illustrate your tactical decisions, such as placement of apparatus, command post, location of hose lines, etc. This will help you keep track of what you are doing and will make it easier for the assessors to understand the tactical decisions you are making. You will communicate during the exercise with the Resource Person, who will play the role of incoming companies as well as communications. All communications with the Resource Person should be in the form of standard radio transmission protocol. The Resource Person, acting as the on-site companies, may relay additional information to you which may be valuable to you in your role as Incident Commander. After familiarizing yourself with the fire situation, begin at once to explain to the assessors your strategy of fighting the fire upon your arrival at the scene. You will be asked to demonstrate your deployment of personnel and apparatus at the scene by writing on the wall board with one of the markers that will be provided. This will provide the assessors with a visual understanding of your overall strategy at the fire scene. The simulation will last approximately 10 minutes during which time you will explain your strategy to the assessors and deal with any new contingencies as they may arise. This will be followed by a period of approximately five minutes during which time the assessors will be allowed to ask you questions concerning your tactical decisions. You will be given 10 minutes to study the background information on the chosen tactical location so that you will be prepared to handle the situation presented to you. NOTE: This same assignment will be given to other candidates in this process. You are encouraged not to discuss this assignment with anyone who has not yet completed the exercise.
188
The Assessment Center Handbook for Police and Fire Personnel TACTiCAl eXerCise informATion
BUSINESS NAME:
Acme Finishing and Plating
OCCUPANCY TYPE:
Metal plating and heat treating plant
ADDRESS:
2200 Commerce Street
OCCUPANCY:
Two-story residential duplex
BUILDING CONSTRUCTION:
Ordinary and metal clad
DATE:
April 11, 2010
TIME:
1132 hours
WEATHER CONDITIONS:
Cloudy, 420 F
WINDS:
Out of the south @ 6 mph
Initial Information:
Passing motorist reports seeing smoke coming from upstairs window
Alarm
response
First
Engine 1, Engine 3, Truck 1, Car 1
Second
Engine 4, Squad 1
Third
Engine 2, Truck 2, Car 2
Appendix f TACTiCAl fire Problem sCenArio fAir hAVen fire dePArTmenT
fire lieutenant Assessment Center Tactical exercise scenario The following is a general sequence of events for this exercise: 1. Ten minutes before the exercise begins the candidate is provided written exercise instructions, a description of the building, occupancy, type of construction, weather conditions, responding units, and a diagram showing the building schematic and street location. 2. When the candidate is brought into the room, he/she will see a blank screen for the LCD projector; the LCD projector will be covered. He/she will also see a white board with an overhead projector showing the building schematic and street diagram. 3. The Resource Person will be seated facing the candidate, the screen, and the white board. The Resource Person will be introduced as the dispatcher and all incoming units, other than the unit on which the candidate is responding. 4. The simulator operator will be seated facing the screen and the white board and will control the fire situation. 5. One or two assessors will be seated together in a position to observe and hear the candidate. 6. The Exercise Coordinator will introduce the participants and their roles.
189
190
The Assessment Center Handbook for Police and Fire Personnel
7. The candidate will be asked if he/she has any questions. 8. The simulator operator will remove the cover from the LCD projector and the fire building will be visible with fire and smoke showing. 9. Upon beginning the exercise, the Resource Person will inform the candidate that he or she will arrive on the scene as the third-due engine on Engine 25. The captain will have the initial attack line and the lieutenant on Truck 22 is conducting search operations on the second floor of the building. 10. Upon arriving on-scene with Engine 25, the candidate will be expected to give a size-up and assume command. 11. Within two minutes after Engine 25 arrives on scene the Resource Person will announce that one mutual aid engine company from West Shore FD with three personnel is on the scene. 12. Within three minutes after Engine 25 arrives on scene the Resource Person will announce that a private ambulance from A.M.R. will arrive on the scene. 13. At brief intervals, the Resource Person will announce the arrival of volunteers who will arrive with their personal gear in their personal vehicles. 14. If the candidate asks for a 360 or assigns a company to the rear, he/she will be shown the rear view of the structure. 15. The candidate will begin to assign responding units to tasks such as laying a line, conducting an initial search, ventilation, etc. 16. As these assignments are made, the Resource Person will acknowledge their receipt. 17. At five minutes into the exercise, the interior crew will inform the candidate that they have a firefighter who has been injured by falling debris and who requires medical attention. This should prompt the candidate to call for a PAR and to send the RIT crew in to remove the injured firefighter. 18. From time to time the candidate may ask the Resource Person for status or progress reports from the assigned units. 19. If a second alarm or additional response is requested, the Resource Person will acknowledge and will indicate what kind of response will be arriving.
Appendix F
191
20. At brief intervals, the Resource Person will acknowledge the response of additional units or personnel. 21. As additional units or personnel arrive, the candidate should assign them to specific tasks or to staging. 22. Depending upon what the candidate does in terms of tactical priorities and assignments, the computer simulation will change the fire condition. 23. The Lead Assessor, using the Tactical Exercise Rating Form, will record all actions taken or assignments made by the candidate. 24. The Exercise Coordinator will be responsible for keeping the time. 25. At eight minutes into the exercise, the Resource Person will announce that he has a report from a neighbor that there may be an elderly victim trapped on the second floor of the building. This should prompt the candidate to call for a secondary search of the second floor. 26. At approximately ten minutes into the exercise the Resource Person, as the Deputy Chief, will announce that he is on the scene and request a face to face with the candidate. 27. Following the candidate’s report to the Deputy Chief time will be called and assessors will be allowed three minutes to ask questions of the candidate. 28. The total elapsed time should not exceed 15 minutes. 29. Assessors will have ten minutes to complete their consensus scoring of the candidate.
Appendix G fire TACTiCAl Problem eVAlUATion of TACTiCAl ConsiderATions for TACTiCAl fire Problem fire CAPTAin AssessmenT CenTer
Tactical Consideration evaluation Instructions to Subject Matter Expert: Please indicate, by placing a check in the appropriate column, whether each of these tasks, in your opinion, is (3) Essential; (2) Very Important; (1) Somewhat Important; or (0) Not Important in evaluating candidates in the tactical exercise. Please list any other considerations that you believe are important in the spaces provided.
192
193
Appendix G Tactical Consideration 1. Establish Incident Command? 2. Employ proper Incident Command Procedures? 3. Give an Initial Report? 4. Do an adequate size-up? 5. Establish a formal Command Post? 6. Make proper tactical assignments based upon a logical strategy? 7. Do a walk-around? 8. Develop an appropriate strategy? 9. Perform primary and secondary searches? 10. Provide for rehabilitation of personnel and victims? 11. Give due consideration to exposures? 12. Ask for Personnel Accountability Report? 13. Establish an Interior Sector? 14. Establish second line (backup) assignments? 15. Establish ventilation or roof sector? 16. Establish Rapid Intervention Team? 17. Call for additional apparatus in a timely manner? 18. Establish floor sector? 19. Establish, assign and communicate staging area? 20. Others? Please describe.
3
2
1
0
Appendix h emPloYee meeTinG eXerCise VillAGe of riVerVieW
Police sergeant Assessment Center employee meeting exercise instructions: In this exercise, you are to assume that you are a Sergeant in the Riverview Police Department. You are currently assigned to the patrol division as a field supervisor. You have been contacted by Officer John Doe, who wishes to see you about his performance evaluation report that he received earlier today (see attached). This is a report that you prepared for him based upon his job performance for the last 12 months. Your assignment is to meet with Officer Doe and go over the report with him and to answer any questions he has about the manner in which you rated his performance. You will have 15 minutes to complete this assignment. Use the remaining time to prepare for your meeting.
194
Appendix i CiTizen inTerVieW eXerCise CiTY of GrAnd VieW
Police sergeant Assessment Center Citizen interview exercise instructions: In this exercise you are to assume that you are a Sergeant in the Grand View Police Department. You are serving as a field supervisor in the patrol division. It is 6:30 PM on a Friday and you are serving as the shift supervisor in the absence of Lt. John Mullins who is home with a cold. You have been called to the station by communications to meet with a woman who is very upset and insists upon seeing the “Officer in Charge.” You will have 15 minutes to conduct your interview with the woman. During the interview, you should determine the nature of the women’s problem and attempt to identify a course of action that will solve the problem for her.
195
Appendix J role-PlAYinG insTrUCTions for The CiTizen inTerVieW eXerCise CiTY of blUe ridGe
fire Captain Assessment Center Citizen interview exercise role Playing instructions: In this exercise you are to play the role of a resident of Blue Ridge. You will be expected to assume a fictitious identify and use an actual address in the City. You will be meeting with a candidate who will be playing the role of a Captain in the Blue Ridge Fire Department. When you meet with the Captain, you will tell him that you were informed by your next-door neighbor that earlier today, around 3:00 PM, a unit from the Blue Ridge Fire Department had been on an emergency medical call across the street from your house. While leaving that call, the fire truck backed into your elderly mother’s car that was parked on the street in front of your house. You will explain that your mother is in intensive care at the hospital and you are taking care of her car while she is in the hospital. You were visiting her at the hospital when the accident occurred. When you arrived back home, your neighbor, Mrs. Cynthia Snoop, told you about the accident. Mrs. Snoop keeps a careful eye on everything that goes on in your neighborhood. She also informed you that after the accident, one of the men on the fire truck got off the truck, inspected the damage to the complainant’s car, said something to the driver, and then the truck left the scene.
196
Appendix J
197
You will tell the Captain that you have reported the damage to your insurance company and were advised that there would have to be a police report taken. The damage is estimated to be in excess of $1,000. The candidate will be expected to try to deal effectively with you by obtaining all necessary information and taking whatever action would be appropriate in a case such as this. You will need to be very upset about the damage and demand that the city pay for it. If something is not done, you may go see the Mayor and ask him to look into it.
Appendix k fire insPeCTion eXerCise norTh ridGe fire dePArTmenT
fire lieutenant Assessment Center fire inspection exercise instructions: In this exercise you are to assume that you are a Lieutenant in the North Ridge Fire Department. You are about to attend a meeting with the owners and operators of Pile High Paper Company, located in North Ridge. The purpose of this meeting is to discuss several potential fire prevention problems with the management of the firm. Pile High Paper Company is a 150,000 square foot warehouse building. The building is exterior masonry block with a built up roof. The building is classified 5b construction and is fully sprinklered. The business is a Chapter 54 occupancy. The Pile High Paper Company is a shipping and receiving warehouse which handles household office and paper supplies. Recent fire prevention inspections have revealed several potentially serious violations which pose a threat to personal safety as well as to the structure and its contents. Specifically, the following violations have been detected: 1. A new warehouse office was built (approximately 10’ X 10’) on the lower level that is not sprinklered. This area must be sprinklered or removed. 2. The sprinkler riser must be kept accessible for inspection and fire department access. A clearance of 3 feet must be maintained around sprinkler riser and 3 foot access aisle. 198
Appendix K
199
3. The plastic curtain wall must be removed between the loading dock and warehouse area. 4. Fire extinguishers (near loading dock) must be properly hung. 5. Rack storage is 20 feet high. Please resubmit data from sprinkler contractor showing system will handle this volume of product. 6. Extension cords used to power packaging machines may not be used in place of permanent wiring. The Fire Chief has asked you to meet with the management of the company to outline the Department’s concerns and to explain why corrections need to be made. You will be meeting with the following persons: Mr. J. B. Hardcastle, Owner of the Company Mr. E. L. McElroy, Building Manager Mr. R. D. Copp, Security/Safety Director The purpose of this meeting is to obtain the cooperation of company officials in taking the necessary corrective measures to ensure adequate building safety. You will have 15 minutes to complete this assignment. Use the remaining time to prepare for your meeting.
Appendix l shifT meeTinG eXerCise CiTY of JohnsToWn
Police sergeant Assessment Center shift meeting exercise instructions: In this exercise, you are to assume that you are a Sergeant in the Johnstown Police Department. You are currently assigned as the afternoon shift commander in the patrol division. Your immediate supervisor is Lieutenant Henry Houser, the patrol division commander. You have just come from a supervisor’s meeting in which several important issues were discussed. You have been instructed by Lieutenant Howser to review these issues with officers on your shift at the next shift briefing. The issues he has asked you to discuss with the members of your shift are as follows: 1. Lieutenant Howser reported that some officers continue to make unacceptable errors while preparing their reports or are late in getting them turned in. A growing number of reports are being sent back for correction, indicating a lack of attention and diligence on the part of the officers. Officers are instructed to pay more attention to this important part of their job. 2. Lieutenant Howser has pointed out that officers on some shifts have a tendency to question dispatchers on calls and to try to dictate to the dispatchers call assignments and priorities. Officers are to be reminded that the dispatchers are following the guidelines set for them and any questions on calls should be directed to your own supervisor. 200
Appendix L
201
3. The Principal of South Middle School called and is requesting additional patrol in the parking lot between 10:00 AM and 12:00 PM. He reports that a latemodel Toyota, blue in color, unknown license, has been seen driving through the lot with no apparent reason several times this past week. He has no further information on the driver or the vehicle. 4. The Department is seeking volunteers to serve as instructors for the new Citizen’s Police Academy that the Chief will be implementing next year. In order to minimize costs and enable us to continue this program, officers are asked to volunteer their time to serve as instructors for this worthwhile program. The total time involved will amount to only a few hours every three or four months. While there is no compensation provided for this effort, this is an important program and the Chief is hoping for an enthusiastic response. 5. The Chief is concerned at seeing more than one car at a time parked at local convenience stores and mini-marts. This presents a bad image to the public and suggests that the officers are “goofing off.” Please remind your officers that only one car should be out at such locations at one time. 6. Nominations for the Officer of the Year Award will be open several more days. There have been very few nominations this year and the Chief is concerned that more officers are not taking this program seriously. The Lieutenant would like to have at least one nomination from each shift by the end of the week. 7. The Chief announced that he has been informed by the City Manager’s Office that it appears that the pay increases scheduled for next month will now be delayed indefinitely due to a budget shortfall. It is not known whether the pay increase, when it does come, will be retroactive to the original date it was to become effective. You will be given 15 minutes to discuss these issues with your shift. Please use the remaining time to prepare for this assignment.
Appendix m ComPAnY TrAininG eXerCise CiTY of freedom
fire lieutenant Assessment Center Company Training exercise instructions: In this exercise, you are to assume that you are a Lieutenant in the Freedom Fire Department. You have recently been promoted and are currently serving as the Company Officer on Engine 4, third shift. You are about to conduct a15-minute drill for members of your company. You will have 15 minutes to prepare a 15-minute training drill for members of your company. This drill will involve some items of equipment used in firefighting. The subject of your training will be the following chapter from the IFSTA Manual, Essentials of Firefighting, 5th edition: Chapter 3. fire behavior A copy of this manual has been provided to you to assist you in preparing for your class. However, you will not be allowed to take the manual with you into the training room. You have been provided with a flip chart and marking pens to use in the event you wish to use visual aids during your presentation. There will be two members of your shift in this training session. They are: Firefighter Frank Jackson Firefighter Herb Green 202
Appendix M
203
Both of these individuals are seasoned firefighters and should already be familiar with the material that you will be covering during the drill. You will have 15 minutes to complete this assignment. Use the remaining time to prepare for your training session.
Appendix n CommUniTY meeTinG eXerCise VillAGe of Cool sTreAm
Police sergeant Assessment Center Community meeting exercise instructions: In this exercise, you are to assume the role of a Sergeant in the Cool Harbor Police Department. You are currently assigned as the shift supervisor on the afternoon shift. Your immediate supervisor is Captain David Johnson, the Patrol Division Commander. You are about to attend a meeting of neighborhood residents who reside in the Newport Village area. You have been asked to meet with them by Captain Johnson because of a number of complaints he has received from the residents of the area regarding juveniles loitering in the streets after dark, cars driving at high speeds on residential streets, abandoned automobiles left in yards and alleys, boarded up homes being used as hangouts for gangs and drug dealers, and a general lack of police visibility in the area. Captain Johnson had intended to come with you to the meeting but was called away to go with the Chief to a meeting in the Village Manager’s Office at the last minute. As a result, he has asked you to attend the meeting and attempt to help the residents of the area resolve their problems. Your assignment is to meet with the community residents and outline any specific police programs and activities the Department may have that will address their problems as well as to answer any questions they may have and generally assure them that the police department is interested in working with them. 204
Appendix N
205
You will have 15 minutes to complete this assignment. Use the remaining time to prepare for your meeting.
Appendix o sTAff meeTinG eXerCise CiTY of bellVUe
Police Chief Assessment Center staff meeting exercise instructions: For this exercise, you are to assume that you have recently been appointed to the position of Chief of Police in the City of Bellvue following a grueling competition in which you narrowly edged out two other very good candidates. This is your first full week on the job since accepting the position and you are about to have your first staff meeting with your two key Commanders: Deputy Chief Harvey Kidwell, who is in charge of Community Initiatives, and Deputy Chief Sharon Glass, who is in charge of Business and Finance. The purpose of this meeting is for you to lay out your management philosophy to your two Deputy Chiefs and to give them a sense of your goals, objectives, and priorities for the Department for the immediate future. In addition, you are interested in obtaining their views about what they see as the strengths and weaknesses of the Department and particular problems that need to be dealt with in the weeks and months ahead. Unfortunately, you have just received a telephone call from the City Manager’s Office and have been told to report there in 15 minutes. As a result, you will need to cut this meeting short. You will have 15 minutes to complete this assignment. You may use the remaining time to silently prepare for your meeting.
206
Appendix P ProGrAm deVeloPmenT eXerCise (PoliCe lieUTenAnT) CiTY of PUrPle sAGe
Police lieutenant Assessment Center Program development exercise instructions: In this exercise, you are to assume that you are a Lieutenant in the Purple Sage Police Department. You have been temporarily assigned to the Administrative Division as a Special Projects Coordinator. In this position, your primary duties are to serve as staff support to the Chief of Police and the command staff for the purpose of researching and developing special projects which may be designed and implemented to improve operational efficiency and effectiveness. While you are able to use your own discretion in researching and developing concepts and ideas, most of the projects you are working on are at the direction of the Chief of Police and the command staff. You have been asked by the staff to develop a preliminary proposal for consideration by the staff that will have the ultimate goal of enhancing the level and quality of police services in the City of Purple Sage. Your preliminary proposal should address the following topics: 1. Developing a cost-effective method of reducing the incidence of theft and burglary to motor vehicles in and around major retail shopping areas (you may specify particular areas in which your plan is to be implemented). 2. Enhancing patrol productivity and effectiveness in both residential and business areas of the City while at the same time stimulating employee morale. 207
208
The Assessment Center Handbook for Police and Fire Personnel
3. Identifying ways and methods by which to generate additional revenues through legitimate enforcement efforts in such a way that will not meet with adverse public reaction or political opposition. 4. Developing methods by which to enlist and enhance public support and cooperation in the implementation and maintenance of specific crime reduction programs. You are scheduled to meet with the Program Review Committee, consisting of the two Assistant Chiefs, tomorrow. The purpose of this meeting is to give them your preliminary ideas concerning your assigned topics. For the purpose of this exercise you are required to provide two copies of your report at 8:00 AM tomorrow. This report must be no more than five double-spaced pages, either typewritten or handwritten. Your written report need not be typewritten, but it must be legible and presented in a neat and orderly fashion. Your paper will not be scored by the assessors on written communication skills (e.g., grammar, punctuation, spelling, etc.) but may be evaluated in terms of its content, clarity of expression, organization, and logic. Tomorrow, you will be expected to address the Program Review Committee to orally present your report and to answer any questions they may have concerning your proposal. This presentation will last no more than 15 minutes and should include time for questions from the assessors.
Appendix q
OR
AS
SS
SE
SE
SS
AS
OR
room ConfiGUrATions for VArioUs eXerCises
CANDIDATES
VIDEO CAMERA
ASSESSOR
Room Set-up for Group Problem-Solving Exercise
209
210
The Assessment Center Handbook for Police and Fire Personnel
DRY ERASE BOARD
RESOURCE PERSON
CANDIDATE
OVERHEAD PROJECTOR
VIDEO CAMERA ASSESSORS
Room Set-up for Tactical Exercise
CANDIDATE
DESK
VIDEO CAMERA
AS
SE
SS
OR
S
Room Set-up for Citizen/Employee Interview Exercise
ROLE PLAYER
Appendix Q
211
CHALKBOARD FLIP CHART CANDIDATE
ASSESSORS
Room Set-up for Shift/Company Training Exercise
VIDEO CAMERA
Appendix r TYPiCAl AssessmenT CenTer sChedUle fire lieUTenAnT AssessmenT CenTer
exercise schedule Monday, December 7, 2009 1:00PM - 2:30PM
Candidate Orientation (All Candidates)
Wednesday, December 9, 20091 8:00 AM – 10:05 AM
Tactical Exercise 8:00 AM 8:50 AM 9:40 AM
10:05 AM – 12:10 PM
1-A 1-C 1-E
1-B 1-D
10:30 AM 11:20 AM
1-C 1-E
Fire Prevention Exercise 10:05 AM 10:55 AM 11:45 AM
1-B 1-D 1-A
12:10 PM – 1:00 PM
Lunch for Assessors
1:00 PM – 3:05 PM
Company Training Exercise
1
8:25 AM 9:15 AM
Candidates should report 10 minutes early to each exercise to receive their instructions.
212
213
Appendix R 1:00 PM 1:50 PM 2:40 PM 3:05 PM – 5:10 PM
1-C 1-E 1-B
1:25 PM 2:15 PM
1-D 1-A
3:30 PM 4:20 PM
1-E 1-B
8:25 AM 9:15 AM
2-B 2-D
10:05 AM 10:55 AM
2-C 2-A
12:55 PM 1:45 PM
2-D 2-B
2:35 PM 3:25 PM
2-A 2-C
Employee Interview Exercise 3:05 PM 3:55 PM 4:45 PM
1-D 1-A 1-C
Thursday, December 10, 20091 8:00 AM – 9:40 AM
Tactical Exercise 8:00 AM 8:50 AM
9:40 AM - 11:20 AM
2-A 2-C
Fire Prevention Exercise 9:40 AM 10:30 AM
2-B 2-D
11:20 AM – 12:30 PM
Lunch for Assessors
12:30 PM – 2:10 PM
Company Training Exercise 12:30 PM 1:20 PM
2:10 PM – 3:50 PM
Employee Interview Exercise 2:10 PM 3:00 PM
1
2-C 2-A
2-D 2-B
Candidates should report 10 minutes early to each exercise to receive their instructions.
Appendix s CAndidATe feedbACk qUesTionnAire Congratulations! You have recently finished participating in the assessment center for the position of Police Sergeant. We are always seeking ways in which we can improve the services we provide. As a result, we are interested in any comments you may have about the assessment center in which you participated. Your responses will help us to evaluate our own work and to improve the quality of our work in the future. Please take a few minutes to complete this questionnaire and return it to the Exercise Coordinator. Your responses will remain confidential. 1. Was the orientation session helpful in preparing you for the exercises? (circle one) 5 Very Much
4
3 Somewhat
2
1 Not at all
Comments: ______________________________________________________ ________________________________________________________________
214
215
Appendix S
2. Overall, were the exercises in which you participated appropriate and relevant for the position for which you were being evaluated? (circle one) 5
4
Very Much
3
2
Somewhat
1 Not at all
Comments: ______________________________________________________ ________________________________________________________________
3. Was this assessment center a fair test of your ability to perform the duties of the position for which you were being evaluated? ( circle one) 5
4
Very Much
3
2
Somewhat
1 Not at all
Comments: ______________________________________________________ ________________________________________________________________
4. Compared with other promotional examinations you have taken, how would you rate the assessment center? 5 Very Much
4
3 Somewhat
2
1 Not at all
Comments: ______________________________________________________ ________________________________________________________________
216
The Assessment Center Handbook for Police and Fire Personnel
5. Of all the exercises in which you participated, which was the most difficult for you personally? (check one) __
Group Problem-Solving
____
Citizen/Employee Interview
__
Community Presentation
____
Tactical Problem
Comments: ______________________________________________________ ________________________________________________________________
6. Of all the exercises in which you participated, which was the least difficult for you personally? __
Group Problem-Solving
____
Citizen/Employee Interview
__
Community Presentation
____
Tactical Problem
Comments: ______________________________________________________ ________________________________________________________________
7. Overall, what did you like most about the assessment center? Comments: ______________________________________________________ ________________________________________________________________
8. Overall, what did you like least about the assessment center? Comments: ______________________________________________________ ________________________________________________________________
217
Appendix S
9. How many other assessment centers have you participated in during the last 5 years? (check one) __ none
____ 1 or 2
____ 3 or more
10. Any other comments or suggestions? Comments: ______________________________________________________ ________________________________________________________________
Thank you for taking the time to answer these questions. Your comments and suggestions mean a great deal to us.
Appendix T CAndidATe feedbACk forms The following information is intended to provide feedback to the candidate in terms of how he or she performed in the assessment center process. Candidates should be allowed to review these comments and take notes, if they desire, to assist them in preparing for future assessment centers. In-Basket Candidate: ________________ Exercise: __________________ Evaluation Criteria:
Planning and organizing did the candidate
Yes
1. Use all available resources in handing the items in the In- Basket? 2. Delegate and assign tasks to the appropriate personnel? 3. Make proper notifications of his/her impending absence? 4. Use the calendar to schedule future meetings and follow-up activities? 5. Keep all necessary personnel informed of situations in which they have an interest?
218
some what
no
219
Appendix T
Comments: ______________________________________________________ ________________________________________________________________ ________________________________________________________________ Signed: ____________________________
Date: __________________
Evaluation Criteria: Problem Analysis did the candidate
Yes
some what
no
1. Properly recognize and identify the problem and its causes? 2. Demonstrate a thorough understanding of the problem and its causes? 3. Develop appropriate and reasonable solutions to the problem? 4. Show an appreciation for “the big picture” as opposed to having tunnel vision? 5. Consider all possible alternatives when developing solutions to the problem?
Comments: ______________________________________________________ ________________________________________________________________ ________________________________________________________________
220
The Assessment Center Handbook for Police and Fire Personnel did the candidate
Yes
some what
no
1. Demonstrate the ability to make sound decisions?
2. Establish reasonable and sound priorities in handling situations? 3. Consider all available options before making a decision? 4. Indicate the ability to stand by a decision once made? 5. Use sound logic in analyzing all available information before making a decision?
Comments: ______________________________________________________ ________________________________________________________________ ________________________________________________________________ Signed: ____________________________
Date: __________________
indeX A Accident Review Board Exercise, 65-66 oral presentation in, 66 oral report in, 65 written report in, 65 Accurate judgement of candidates by assessors, 7 Active listening in the Citizen Interview Exercise, 42 Administration of assessment centers, 67 Advantages of the assessment center, 8 Assessment centers accurate judgement of candidates by, 7 advantages of, 8 as a learning environment, 4 as a test of ability, 9 defined, 162 design and administration, 67 different purposes for, 172–173 differences from traditional testing methods, 4 expense of, 17 flexibility of, 8 high degree of acceptance by candidates, 9 how viewed by candidates, 10 policy statements, 166 predictive power of, 6 preparing for, 107 rights of participants, scheduling, App T, 218 subjectivity of, 16 use of, 3, 4, 8, 10 validation issues, 171–172 Assessor Manual, 88 Assessors care and nurturing of, 90 certification of, 86 commentary, 104 compensation, 87 impartiality of, 18 note-taking by, 92–93
orientation, 88 outside verses inside, 17 qualification of, 86, 148–149 rules of conduct for, 91 selection, 68, 85–87, 148–149 training, 5, 68, 167–170 Assigned roles in Group Problem-Solving Exercise, 27 b Benchmarks, 96–99 Best practices in assessment center exercise, 115 Blunders - see some of the greatest blunders, Break Room, 74 C Calculating weighted scores, 102 Cameras, in-car, 82–84 Candidate evaluation, 95 feedback, 10, 11, 68, 95, 105–107, 151–153 orientation, 12, 71–73, 131-132, 146–147 preparation, 68 satisfaction with assessment centers, 145 scoring, 100 Care and nurturing of assessors, 90 Career Interview Exercise, 63–65 points to consider in, 64 Certification of assessors, 86 Citizen Interview Exercise, 41–44 best practices in, 26–127 don’t make promises you cannot keep, 127 greet the citizen warmly, 126 remember to sell yourself and your department, 127 example of, App I, 195 greatest blunders in, 141–142
221
222
The Assessment Center Handbook for Police and Fire Personnel
linked to the Employee Meeting Exercise, 43 note-taking in, 42 performance dimensions in, 96, 97 role player in, 41, 42 role playing instructions for, App J, 196 Closure in Employee Meeting Exercise, 39 Coming on time and prepared, 114 Commentary assessor, 104 Common errors in Tactical Police Problem, 37 Community Meeting or Presentation Exercise, 50–53 best practices in, 125–126 be willing to accept responsibility, 125 listen to what the residents have to say, 125 offer to get back to the residents, 125 example of, App N, 204 greatest blunders in, 138–139 note-taking by assessors in, 51 performance dimensions in, 96, 97 role players in, 50–52 responding to questions from the audience in, 51 Compensation of assessors, 87 Company Meeting Exercise - see Shift or Company Meeting Exercise Company Officer in Tactical Fire Problem, 30 Company Training Exercise example of, App M, 202 Consistency in scoring, 94–95 Criminal Investigation Exercise, 61–63 drawback of, 62 performance dimensions in, 96, 97 written report in, 62 Criteria, evaluation selection of, 95 Customizing the assessment center, 147–148 d Debriefing, candidate, 107, 151–153 Decision-Making ability in Citizen Interview Exercise, 96 in Criminal Investigation Exercise, 96 in Employee Meeting Exercise, 96 in News Media Exercise, 96 in Tactical Fire Problem, 96 in Tactical Police Problem, 96 Design of assessment centers, 67 Detailed breakdown of assessment center scores, 103 DVDs, use of in an assessment center, 84
e Electronic In-Basket, 81 Employee Assistance Program, 111 Employee Meeting Exercise, 38–40 best practices, 123–124 be a boss, not a buddy, 124 begin and end on a positive note, 123 do not negotiate, 124 make your expectations very clear, 123 closure in, 39 example of, App H, 194 greatest blunders in, 133–127 linked to the Citizen Interview Exercise, 43 performance dimensions in, 96, 97 problem identification in, 39 problem resolution in, 39 role playing in, 38, 39 three tasks in, 39 Equipment and facilities required in an assessment center, 73–75 Errors - see common errors Evaluation criteria selection of, 95 Evaluation of candidates, 92 Examinations, written, 8 Examples of assessment centers, 17 Exercise Coordinator, 94 Exercise design, 68–71 Exercise Room, 74 f Facilities required in an assessment center, 68, 73–75 Feedback from candidates, 149–15, App S, 214 Feedback to candidates, 10, 11, 68, 92, 105–107, App T, 218 Fire Inspection Exercise, 44–46 example of, App k, 198 performance dimensions in, 96, 97 role players in, 44 successful performance in, 46 Fire Marshal’s Office - see Public Education Exercise Fire simulators, 80 Flexibility of assessment centers, 8 G Group Problem-Solving Exercise, 25–29 assigned roles in, 27 best practices, 121–123 developing consensus, 121
223
Index establishing effective working relationships, 121 having someone take notes, 122 lead, don ‘t push, 122 work to get the group headed in a single direction, 122 example of, App D, 184 leadership ability in, 27, 29 organizing the group, 28 performance dimensions in, 96, 97 topics assigned to, 26 Guidelines for assessment center administration, 5, 89, App. A, 159 Guidelines for scoring in-basket exercise, App C, 182
in Tactical Fire Problem, 97 in Tactical Police Problem, 97 J Job analysis, 5, 67, 68 Job tasks relationship with performance dimensions, 70 Judgment and Reasoning in Community Presentation Exercise, 97 in Criminal Investigation Exercise, 97 in Employee Meeting Exercise, 97 in Program Development Exercise, 97 in Tactical Fire Problem, 97 in Tactical Police Problem, 97
h k How candidates view assessment centers, 10 Human Relations benchmarks, 98 in Citizen Interview Exercise, 96 in Community Meeting Exercise, 96 in Employee Meeting Exercise, 96 in Group Problem-Solving Exercise, 96 in Shift or Company Meeting Exercise, 96 i Importance of good planning in an assessment center, 67-68 In-Basket Exercise, 6, 21–25 best practices, 115–118 delegating appropriately in, 117 establishing reasonable priorities in, 116 keeping other people informed, 117 looking for linkages in, 117 electronic, 81 greatest blunders in, 132–133 key to success in, 25 performance dimensions in, 96, 97 scanning items in, 116 use of technology in, 25 In-car cameras, 82 Incident Commander in Tactical Fire Problem, 31 in Tactical EMS Problem, 33 Information pooled from assessors, 6 Initiative in Citizen Interview Exercise, 97 in Criminal Investigation Exercise, 97 in Group Problem-Solving Exercise, 97 in Program Development Exercise, 97
Keeping exercises realistic and relevant, 148 Key to success in In-Basket Exercise, 25 l Leaderless Group Discussion, 111 see also Group Problem-Solving Exercise Leadership benchmarks, 98 in Employee Meeting Exercise, 96 in Group Problem-Solving Exercise, 27–29, 96 in Shift or Company Meeting Exercise, 96 in Staff Meeting Exercise, 96 Learning from mistakes, 114 Linking the Citizen Interview Exercise to the Employee Meeting Exercise, 43 List of facilities and equipment, 75 Locations for Tactical Fire Problem, 30 m Maintaining consistency in scoring, 94–95 Maintaining test security, 77–78 Methods of scheduling, 75–77 Minimally qualified, 100 typical assessor comments, 105 Modern technology use of in an assessment center, 80 Multiple assessment techniques, 5 n News Media Exercise, 56–59
224
The Assessment Center Handbook for Police and Fire Personnel
performance dimensions in, 96, 97 role players in, 58 Not qualified, 100 typical assessor comments, 105 Note taking by assessors generally, 92–93 by assessors in Community Meeting or Presentation Exercise, 51 in Citizen Interview Exercise, 42 Number of candidates to be evaluated, 68 Number of exercise, 71 Numerical scale, 100 o Objective of the Staff Meeting Exercise, 56 Open mind, 113 Oral assessment, 5 Oral communication skills benchmarks, 99 in Citizen Interview Exercise, 96 in Community Presentation Exercise in Fire Inspection Exercise in News Media Exercise, 96 in Program Development Exercise, 59 in Staff Meeting Exercise, 96 Oral presentation in Accident Review Board Exercise, 66 Oral report in Accident Review Board Exercise, 65 Organizing the group in the Group ProblemSolving Exercise, 28 Orientation session, candidate, 12, 71–73, 131–132 P Performance dimensions relationships with job tasks, 70 generally, 96–99 Planning importance of, 67–68 Positive attitude, 113 Predictive power of assessment centers, 6 Preparation of candidates, 68 Preparation Room, 74 Preparing for an assessment center, 108 Problem Analysis benchmarks, 99 in Citizen Interview Exercise, 96 in Criminal Investigation Exercise, 97 in Group Problem-Solving Exercise, 97 in In-Basket Exercise, 97 in Program Development Exercise, 97
in Tactical Fire Problem, 97 in Tactical Police Problem, 97 Problem identification in Employee Meeting Exercise, 39 Problem resolution in Employee Meeting Exercise, 39 Program Development Exercise, 59–61 example of, APP P, 207 greatest blunders in, 149–141 oral communication skills in, 59 performance dimensions in, 96, 97 questions from assessors in, 61 topics to be assigned in, 60 written communication skills in, 59 Public Education Exercise, 53–54 role players in, 54 successful performance in, 54 q Qualifications of assessors, 86, 148–149 Qualified, 100 typical assessor comments, 105 Questionnaire for candidate feedback, App S, 214 Questions from assessors in Program Development Exercise, 61 r Rating scale, 100 Raw score method, 101,102 “Real me,” 18 “Real person,” 15 Realism of exercises, 16, 148 Recording candidate behaviors, 5 Relationship between performance dimensions and job tasks, 70 Reporting candidate scores, 102–104 Resource Person in Tactical Fire Problems, 31, 79 in Tactical Police Problem, 35, 79 Responding to questions from the audience in the Community Meeting or Presentation Exercise, 51 Rights of participants in assessment centers, 173 Role players generally, 19, 79, 89 in Citizen Interview Exercise, 41, 52 in Community Meeting or Presentation Exercise,, 50–52 in Employee Meeting Exercise, 38, 39 in Fire Inspection Exercise, 44
225
Index in News Media Exercise, 58 in Public Education Exercise, 54 in Shift or Company Meeting Exercise, 47 in Staff Meeting Exercise, 55 in Tactical Fire Problem, 32 Role playing - see role players Room confirmation for various exercises, App Q, 209 Rules of conduct for assessors, 91 s Sample group comparisons of In-Basket scores, 104 Scheduling methods, 75–77, App R, 212 Scoring of candidates, 92 Security, test, 77–78 Selecting evaluation criteria, 95 Selection and qualification of assessors, 148–149 Selection and training of assessors, 68, 85 Shift or Company Meeting Exercise, 46–48 role players in, 47 Shift or Company Training Exercise, 48–50 best practices in, 127–129 begin and end the session on a positive note, 127 don’t ignore criticism or unprofessional conduct, 127 get the role players involved, 127 maintain your composure, 128 example of, App L, 200 similar to the Shift or Company Meeting Exercise, 48 successful performance in, 50 topics for, 50 Simulating reality, 16, 148 Simulators, fire, 80–81 Some of the greatest blunders, 130 Staff Meeting Exercise, 54–56 example of, App O, 206 greatest blunders in, 142–144 objective of, 56 performance dimensions in, 96, 97 role playing in, 55 Staff Support, 78 Standard Operating Policies and Procedures, 33 Stress tolerance in News Media Exercise, 97 in Shift or Company Meeting Exercise, 97 in Staff Meeting Exercise, 97 in Tactical Fire Problem, 97 in Tactical Police Problem, 97
Subject Matter Expert, 16, 95, 96 in design and selection of exercises, 71 in In-Basket Exercise, 22, 23 in Tactical Fire Problem, 31, 32 in Tactical Police Problem, 36, 37 Subjectivity of assessment centers, 16 Successful performance in Fire Inspection Exercise, 46 in Public Education Exercise, 54 in Shift or Company Training Exercise, 50 Superior, 100 typical assessor comments, 105 T Tactical Problem, 29 Tactical Fire/EMS Problem, 29–32 best practices, 118–120 be proactive, not reactive in, 120 follow Incident Command Procedures in, 119 take nothing for granted in, 119 utilize all available resources in a timely manner, 118 company officer in, 30 evaluation of, App G, 192 example of, App E, 186 greatest blunders in, 137–138 locations for, 30 Incident Commander in, 31 performance dimensions in, 96, 97 Resource Person in, 31, 32 scenario, App F, 189 simulating reality in, 32 Subject Matter Expert in, 31. 32 Tactical Police Problem, 34–37 common errors in, 37 performance dimensions in, 96, 97 Resource Person in, 35 Subject Matter Expert in, 36, 37 Technology use of in an assessment center, 80 use of in In-Basket Exercise, 25 Test security, 77–78 Topics to be assigned in Group Problem-Solving Exercise, 27 to be asisgned in Program Development Exercise, 60 to be assigned in Shift or Company Training Exercise, 50 Training of assessors, 68, 85, 167–170 Typical Assessment Center Schedule, App R, 212 Typical In-Basket Items, App B, 179
226
The Assessment Center Handbook for Police and Fire Personnel
U Uses of assessment centers, 3 Use of modern technology in an assessment center, 80 Use of technology in In-Basket Exercise, 25 V Validation issues in assessment centers, 171–172 Videotaped candidate feedback, 107 Videotapes use of in an assessment center, 84 Video camera operator, 78 W Weighted scores calculation of, 102 Well qualified, 100 typical assessor comments, 105 Witness statements in Criminal Investigation Exercise, 61 Written communication skills benchmarks, 99 in Criminal Investigation Exercise, 96 in Employee Meeting Exercise, 96 in In-Basket Exercise, 96 in Program Development Exercise, 59 Written examinations, 8 Written report in Accident Review Board Exercise, 65 in Criminal Investigation Exercise, 62 in Program Development Exercise, 60
