Educational Evaluation in East Asia: Emerging Issues and Challenges

EDUCATIONAL EVALUATION IN EAST ASIA: EMERGING ISSUES AND CHALLENGES

No part of this digital document may be reproduced, stored in a retrieval system or transmitted in any form or by any means. The publisher has taken reasonable care in the preparation of this digital document, but makes no expressed or implied warranty of any kind and assumes no responsibility for any errors or omissions. No liability is assumed for incidental or consequential damages in connection with or arising out of information contained herein. This digital document is sold with the clear understanding that the publisher is not engaged in rendering legal, medical or any other professional services.

EDUCATIONAL EVALUATION IN EAST ASIA: EMERGING ISSUES AND CHALLENGES

SAMUEL S. PENG AND

JOHN CHI-KIN LEE EDITORS

Nova Science Publishers, Inc. New York

Copyright © 2009 by Nova Science Publishers, Inc. All rights reserved. No part of this book may be reproduced, stored in a retrieval system or transmitted in any form or by any means: electronic, electrostatic, magnetic, tape, mechanical photocopying, recording or otherwise without the written permission of the Publisher. For permission to use material from this book please contact us: Telephone 631-231-7269; Fax 631-231-8175 Web Site: http://www.novapublishers.com NOTICE TO THE READER The Publisher has taken reasonable care in the preparation of this book, but makes no expressed or implied warranty of any kind and assumes no responsibility for any errors or omissions. No liability is assumed for incidental or consequential damages in connection with or arising out of information contained in this book. The Publisher shall not be liable for any special, consequential, or exemplary damages resulting, in whole or in part, from the readers’ use of, or reliance upon, this material. Any parts of this book based on government reports are so indicated and copyright is claimed for those parts to the extent applicable to compilations of such works. Independent verification should be sought for any data, advice or recommendations contained in this book. In addition, no responsibility is assumed by the publisher for any injury and/or damage to persons or property arising from any methods, products, instructions, ideas or otherwise contained in this publication. This publication is designed to provide accurate and authoritative information with regard to the subject matter covered herein. It is sold with the clear understanding that the Publisher is not engaged in rendering legal or any other professional services. If legal or any other expert assistance is required, the services of a competent person should be sought. FROM A DECLARATION OF PARTICIPANTS JOINTLY ADOPTED BY A COMMITTEE OF THE AMERICAN BAR ASSOCIATION AND A COMMITTEE OF PUBLISHERS. LIBRARY OF CONGRESS CATALOGING-IN-PUBLICATION DATA Avaailable upon request ISBN 9781606928875

Published by Nova Science Publishers, Inc.    New York

CONTENTS Introduction

1 Samuel S. Peng and John C. K. Lee

Part I. Practices and Challenges Chapter 1

Chapter 2

Chapter 3

Chapter 4

Chapter 5

Education Evaluation and Assessment in Taiwan: Status, Innovations and Challenges Mu-lin Lu

13

Educational Evaluation in Singapore: Current Status and Future Challenges R. Subramaniam

23

Educational Evaluation in South Korea: Current Issues and Future Prospects Sun-Geun Baek

35

Development of Educational Evaluation and Challenges of Practicing Educational Evaluation in Thailand Sirichai Kanjanawasee

45

Educational Evaluation in Hong Kong: Status and Challenges John Chi-Kin Lee

61

Part II. Variations and Evoluation Chapter 6

Chapter 7

Chapter 8

11

73

Educational Evaluation and Quality Assurance: The Case of Hong Kong Nicholas Sun-keung Pang

75

School-Based Assessments (SBA): Implementation and Challenges in the Malaysian Context Suan Yoong

93

Current Issues Regarding the School Evaluation System in Japan Yasuhiko Washiyama

111

vi Chapter 9

Chapter 10

Contents Teacher Evaluation in Thailand: Current Issues and Future Prospects Pruet Siribanpitak

117

PISA as a Catalyst for Capacity Building for Educational Evaluation in Macao Kwok-cheung Cheung

129

Contributors

143

Index

147

In: Educational Evaluation in East Asia Editor: Samuel S. Peng and John Ch-Kin Lee

ISBN 978-1-60692-887-5 © 2009 Nova Science Publishers, Inc.

INTRODUCTION Samuel S. Peng1 and John C. K. Lee2 1

Center for Research on Educational Evaluation and Development, National Taiwan Normal University, Taipei, Taiwan 2 Department of Curriculum and Instruction, The Chinese University of Hong Kong, Shatin, Hong Kong

BACKGROUND Educational evaluation plays an important role in the pursuit of educational excellence (Kennedy and Lee, 2008). At one level, it has been used in the evaluation of students' individual differences and performance. At another level, it has been used to contribute to an understanding of the overall effectiveness and efficiency of institutions and academic programs, as well as such matters as the specific qualities of school management, curriculum, teaching staff, pedagogical practices and educational policies. Educational evaluation is not a simple look in the mirror, a vain exercise; it is an effective way to provide evidence for all decision-making pertaining to resource allocation, program improvement, personnel hiring and promotion, selection of candidates, and educational policy-making. Evaluation is, in fact, an integral part of the multi-faceted process of enhancing quality and attaining excellence. Educational evaluation, as concept and as a set of practices, is not new in East Asia. However, in recent years it has received much greater attention from the general public as well as the educational community because of changes in the political and economic conditions in society. First, as the demand for educational resources increases, both the government and the schools as a whole need to know how well their resources have been effectively allocated and used; they also need to know the extent to which their objectives have been achieved. Evaluation is thus a necessary mechanism for monitoring the educational quality of institutions and programs. Secondly, as education in general and postsecondary education in particular have been expanded in response to global competition and the generally rapid rate of economic

2

Samuel S. Peng and John C. K. Lee

development in society, there are concerns about the equity and quality of education on the part of all stakeholders, including students, parents, teachers, school administrators and policy-makers. Subsequently, there have been strong demands for an effective evaluation system to ensure equity and quality, on the one hand, and proficiency and productivity on the other. Moreover, constant reforms of the educational system increase the demand for input, process and outcome evaluation. The great attention given to accountability and quality assurance makes the practice of evaluation more complex and innovative, with ever more diversified paradigms, methods and approaches. Thirdly, globalization increases awareness of the importance of a country’s international competitiveness and stimulates interest in international comparisons of student ability and competence. For example, the International Association for the Evaluation of Educational Achievement (IEA) and the Center for Educational Research and Innovation of the Organization for Economic and Development (CERI/OECD) have been carrying out international surveys of student achievement in science, mathematics, language and civic education. The results of their surveys have motivated the participating countries to consider how best to evaluate and improve their educational systems to ensure that their students are well prepared and competent. Consequently, the need for effective educational evaluation systems and methods is an important issue, one that is frequently studied and discussed in East Asian countries. However, the development of an effective evaluation system is not a simple task. Many factors need to be considered. Aside from measurement techniques and methodological perspectives, an effective system needs to be cognizant of the cultural, political, economic and social contexts in which an evaluation system is implemented. Consideration of such contexts is critical in East Asia for one simple reason – education is highly valued in our society due to its traditional Confucian culture. All high-stakes educational evaluations, such as senior high school entrance examinations, college entrance examinations and teacher qualification examinations, as well as school performance evaluations, often receive a great deal of public attention and scrutiny. Any official evaluation system, in terms of its contents, standards, and implementation procedures, may set the guidelines for teaching and student learning; schools tend to operate on the basis of, teachers tend to teach and students are likely to learn mainly what is to be assessed and evaluated. What is not tested or otherwise judged or assessed for “official” purposes usually gets less attention or is simply ignored. Thus, implementing a national evaluation system that is objective and fair without having any negative impact on schooling is a major challenge in most places in East Asia. Scholars in the field of educational evaluation are actively engaged in the search for an Asian model that will work well within all East Asian social and cultural contexts. East Asian countries serve as an ideal testing ground for judging the effectiveness of various evaluation models and programs because they share many things in common. Aside from the high value placed on positive attitudes toward education, educational governance at the systems level tends to be centralized and controlled by the central government. Given the similar cultural and social contexts, educators may be able to identify theories and practices

Introduction

3

that are most suitable for their students and educational systems by comparing the different evaluation systems or programs developed by other countries or special administrative regions. It was for this reason that the “Forum for Educational Evaluation in East Asia: Emerging Issues and Challenges” was initiated and held on October 2007 at National Taiwan Normal University. It brought together a group of distinguished scholars from Hong Kong SAR (China), Japan, Macao SAR (China), Malaysia, Singapore, South Korea, Taiwan and Thailand for scholarly discussion of the problem of developing, implementing and improving educational evaluation systems that are sensitive to the local cultures and social contexts. The breadth of issues discussed, free and open expression of ideas and opinions, and general sense of the success of this forum prompted the idea of publishing the presented papers in order to share the participants’ knowledge, ideas and experience with those who did not have an opportunity to participate directly in the forum. This book was therefore developed with the intention of providing a foundation for further discussion of educational evaluation models within the context of East Asian cultural and social conditions. It aims not only at identifying challenges of and strategies for educational evaluation but also at facilitating advanced research on educational evaluation in East Asia. Through an examination of the current principles, policies and practices of educational evaluation in this region, researchers will hopefully be able to further construct and refine evaluation theories and models rooted in the traditional cultures and educational values of East Asia. Of course, it is hoped that such theories and models will also be applicable to other regions of the world.

ORGANIZATION OF THE BOOK We have divided this book into two parts. Part I includes five case reports that center on the current status of educational evaluation programs and policies in Hong Kong SAR (China), Singapore, South Korea, Taiwan and Thailand. Specific policies, implementation procedures and the recent or ongoing development of educational evaluation programs are explored. In Part II, variations in educational evaluation policies and practices in the region are highlighted with chapters drawing on case studies of changes and strategies of educational evaluation in Hong Kong SAR (China), Japan, Macao SAR (China), Malaysia and Thailand. By providing case studies of the principles and practices of educational evaluation systems, the authors identify the current innovations, as well as challenges, associated with significant issues concerning the organization, management and implementation of educational evaluation in East Asia. The range of topics covered in the various chapters is summarized in Table 1.

4

Samuel S. Peng and John C. K. Lee Table 1. Topics covered by each author Topical area Evaluation policies

M.L. R.S. S.B. S.K. C.L. S.P. S.Y. W.Y. P.S. K.C. ©

Evaluation of educational progress Evaluation of basic education

©

Evaluation of the learning of reading, science and mathematics Evaluation of K-12 teachers

©

Evaluation of K-12 schools

©

Evaluation of higher educational institutions Evaluation of academic programs Evaluation of college candidates Evaluation methods

©

©

©

©

©

© ©

©

©

©

©

© ©

©

©

© ©

© © ©

© ©

Note: M.L. -- Mu-lin Lu; R.S. -- Ramanathan Subramaniam; S.B. -- Sun-Geun Baek; S.K. -- Sirichai Kanjanawasee; C.L. -- Chi-Kin John Lee; S.P. – Nicholas Sun-Keung Pang; S.Y. – Suan Yoong; W.Y. – Washiyama Yasuhiko; P.S. – Pruet Siribanpitak; K.C. – Kwok-cheung Cheung.

CHAPTER SYNOPSIS 1. Evaluation of Higher Education Mu-lin Lu, in the first of the case reports, focuses on the connection between higher education evaluation systems and the trend towards globalization. In Taiwan, under the revised University Act promulgated in 2005, the Minister of Education (MOE) took responsibility for organizing an evaluation committee and formulating evaluation policies and guidelines. Subsequently, in December 2005, the MOE and Taiwan’s universities and colleges jointly established the Higher Education Evaluation and Accreditation Council of Taiwan (HEEACT). Subsidized annually by the MOE, HEEACT is responsible for the general operation and administration of university evaluation, and is expected to safeguard the objectivity and accuracy of evaluations: the government’s financial support and the enrollment size of each institution are directly dependent on these evaluations. The MOE also expects the implementation of a systematic evaluation mechanism to lead universities toward greater autonomy and management efficiency. However, as a newly-founded public organization HEEACT confronts several technical problems, including the shortage of professional evaluators, the dependence on governmental funding, and the lack of consensus regarding evaluation indicators and methods and procedures for implementing evaluation. These challenges have made the goal of achieving fairness and objectivity in evaluation more difficult to accomplish. Dr. Lu argues that setting up a long-term evaluation mechanism to

Introduction

5

monitor the performance of universities is needed in order to assure teaching and learning quality, but also that universities still need time to accommodate and assimilate the “culture of evaluation.”

2. Evaluation of Mathematics and Science Learning Ramanathan Subramaniam focuses on educational evaluation in two areas: the learning of science and mathematics and the institutional evaluation of schools. He reports that Singapore places tremendous emphasis on the development of its human resources through education and its evaluation, which is necessary for improvement and innovation, because of the lack of indigenous natural resources. He also explains why the science and mathematics curricula in Singapore continue to set a very high standard. Dr. Subramaniam starts with the current status of and future challenges confronting educational evaluation in Singapore. He argues that reliance on centralized testing by an independent agency at the end of the last year of primary school, and the collaboration of this agency with an international examinations syndicate for the examinations coming at the end of junior and senior high school, have been major reasons why Singapore’s educational system is international in its practices and standards. However, he also points out that the present system of evaluation of science and mathematics learning, though now very efficient, may have to be fine-tuned in the future. For example, it may require such reforms as the use of 2-tier multiple choice questions, which overcome the problem of random guessing of correct answers by asking students to justify their answers on another level, i.e. from another choice of options (Tregust, 1988).

3. Evaluation for School Improvement Sun-Geun Baek describes the extent to which educational evaluation has been used as one of the main tools for the development and improvement of primary, secondary and higher education in South Korea. He refers to five educational evaluation programs in South Korea: • • • • •

the School Activities Records (SAR); the National Assessment of Educational Achievement (NAEA); the College Scholastic Ability Test (CSAT); the Teacher Competence Development Evaluation (TCDE); the Teacher Education Institute Accreditation System (TEIAS).

Baek points out that there have been ongoing conflicts among stakeholders on educational evaluation policies in Korea. For example, one of the teacher unions is strongly against the implementation of TCDE. Some teacher educators and institutes are against the implementation of TEIAS; furthermore, some teachers and professors disagree with changes in the educational evaluation system, and they also disagree with the implementation of the competition system. Nevertheless, Baek asserts that the future prospects of education in South Korea will be bright because Koreans not only acknowledge the importance of education but also are making substantial investments in education in order to build a powerful state.

6

Samuel S. Peng and John C. K. Lee

4. Evaluation for Quality Assurance Sirichai Kanjanawasee presents Thailand’s evaluation practices in relation to curriculum reform and quality assurance issues in that country. Based on the 1999 National Education Act, the establishment of educational standards, curricular revision and quality assurance are all part of the educational reform program. Kanjanawasee has introduced four levels of basic education in Thailand, and has indicated the need for effective in-service teacher training and for prototypes for integrated reading, thinking and writing assessment tools and procedure manuals. The national education policy should guarantee high-quality management by linking external and internal quality assurance mechanisms. In addition, there are challenges facing the development of the O-NET (Ordinary National Educational Test), which embodies content standards for different school curricula, as well as the A-NET (Advanced National Educational Tests), which are correlated with well-defined professional disciplines. Kanjanawasee also points out that much further research is needed in order to support system designs that are both meaningful and fair for all concerned.

5. Evaluation for Mixed Purposes and Educational Reform John Chi-Kin Lee emphasizes the problem of keeping the right balance between decentralization and accountability of educational evaluation, and invites readers to think of evaluation as being essentially an “accompaniment” rather than a “monitoring” device. Lee’s chapter provides a brief overview of current educational evaluation measures and issues in Hong Kong. With regard to such quality assurance mechanisms as school self-evaluation (SSE), external school review (ESR) and Territory-wide System Assessment (TSA), he leaves three questions open for discussion. First, how can we maximize the coherence, in both rhetoric and practice, between and among educational reform, educational supervision and evaluation policies? Second, how can we promote a systematic linkage between centralized and localized systems so as to drive effective school improvement and teacher development? Third, how can teachers’ workloads be reduced and teachers be relieved of their anxieties regarding educational evaluation, and how can we foster a harmonious educational-evaluation “community” of government agencies and teachers?

6. School Self-Evaluation Part II of the volume contains five studies of recent innovations in educational evaluation in East Asia. Nicholas Sun-Keung Pang introduces the details of a series of school-university partnership projects on school self-evaluation in Hong Kong SAR. He also reports on an ongoing investigation into the effectiveness of implementing self-evaluation in Hong Kong schools and the factors that have hindered and /or facilitated this self-evaluation movement. Pang suggests that there are two approaches to assuring the quality of education in Hong Kong: an external mechanism via inspection and an internal framework via self-evaluation. He concludes that, in order to facilitate change in schools, administrators should have the power and leadership capacity to unify a given school’s goals and specify the technology for achieving them, to promote consensus and the sharing of values among all members, and to

Introduction

7

focus attention by carefully selecting targets, controlling resources, and acting forcefully. Pang further suggests that effective leaders are those who can be flexible in adopting approaches to change that can cope with the challenges created by the ever-changing external environment, and in leading their organizations toward excellence.

7. Evaluation of Educational Progress Suan Yoong provides a critical review of the student assessment system currently used in Malaysia. Professor Yoong outlines the effects of the national assessment system, provides an overview of the strengths and weakness of an over-obsession with examinations, and explains the dominant forces shaping educational assessments in Malaysia, carefully taking into account the cultural contexts. In presenting the challenges faced by the implementation of school-based assessment (SBA) in Malaysia, Yoong points out that the examination-oriented tradition imposes a great pressure on schools as well as students to excel on examinations. Such pressure has led schools to attempt to control what students are to study, narrowing the school curriculum to include only those subjects and topics that are expected to be tested on exams and adopting policies that are geared solely toward helping students pass the highstakes examinations. He introduces the serious problems that have arisen from this narrowminded approach, including the failure to measure higher cognitive skills, the deprofessionalization of teachers and the taking of all the fun out of going to school. Yet Yoong also expresses concern that the Ministry of Education’s latest effort in examination system reform might find it difficult to overcome Malaysia’s deeply entrenched, examinationoriented culture, a culture that places the gaining of high test scores as the top priority of teaching and learning.

8. Evaluation of School Performance The chapter by Yasuhiko Washiyama deals with the notion of “autonomous school administration” in Japan and that country’s subsequent evaluation practices in response to the neoliberal ideas of deregulation and decentralization. Initially, Washiyama provides an historical overview of the school evaluation system in Japan. In the contexts of deregulation and decentralization, two evaluation-related policies have been developed: the Regulative Standards for School Establishment, revised in 2002, include new provisions on selfinspection and self-assessment; the Basic Policy of Economic Financial Administration and Structural Reform of 2005 mandates that all schools providing compulsory education be subjected to external evaluation. Professor Washiyama then introduces the Evaluation Guidelines for Schools of Compulsory Education. The purpose of these guidelines is to establish a Plan-Do-Check- Action management cycle for evaluation, to improve the quality of education, to provide for self-evaluation and external evaluation, and to create a more open school atmosphere. These guidelines cover the transition from the traditional quality control system to a system focusing on outcomes, various criteria for evaluation, more focused evaluations, and the establishment of a third-party evaluation organization. He also raises concerns about school evaluation, including the problem of a too-heavy evaluation workload for teachers and the fact that evaluation may become a form of routine paper work.

8

Samuel S. Peng and John C. K. Lee

Washiyama argues that these problems could be solved by the implementation of shadowing, the transition from school evaluation to school diagnosis, and the facilitation of collaboration among teachers. He further emphasizes that evaluation results must be communicated to teachers, students, parents and community members as favorable forces for improvement.

9. Evaluation of Teacher Performance Pruet Siribapitak first describes the number and qualifications of teachers in Thailand. He then discusses current issues in teacher evaluation, and explores the future prospects of teacher evaluation in that country’s public schools. Siribapitak discusses the various types of teacher evaluation for public school teachers, including induction program evaluation, annual performance evaluation, and academic status evaluation. He also points out that the Teacher Civil Service Commission of the Ministry of Education had set up a taskforce to review the teacher evaluation criteria in terms of three new dimensions: teacher discipline, morality and ethical conduct; the quality of teaching and student development; and the results of teaching and student development. Professor Siribapitak concludes that it might be more productive to design systems, not for their formative or summative orientations but in terms of the decisions that need to be made at different levels, e.g. by teacher, the principal, the school district, and the MOE.

10. Evaluation of Student Achievement In the final chapter, Kwok-Cheung Cheung offers an insightful analysis of educational evaluation and learning improvement in Macao. He first states that Macao’s educational system is characterized by a majority of privately-operated schools, and that the schools enjoy full administrative autonomy and complete freedom of instruction. Thus, there is not yet any form of external evaluation from kindergarten up to the terminal senior secondary level. The general public basically does not have a clear understanding of the educational achievement of schools, and educational evaluation is an area gravely in need of further scrutiny. PISA (the Program for International Student Assessment) was adopted as a flagship research program conducted by the Educational Testing and Assessment Research Centre (ETARC) of the University of Macau. Professor Cheung claims that PISA “serves as a catalyst for capacity building for educational evaluation in Macao.” There are five benchmarks: • • • •

Overall performance of the 15-year-olds in reading, mathematical and scientific literacy; Equity in the distribution of learning opportunities as measured by the impact of social, cultural and economic status of the home on literacy performance; Consistency of literacy performance standards across schools; Differential gender differences demonstrated in reading as well as in mathematical and scientific literacy;

Introduction •

9

Learning environments that can facilitate the life-long learning goals of 15-year-olds. Macao’s PISA then exemplifies a unique and effective international evaluation program

PERSISTING CHALLENGES AND CONDITIONS CONDUCIVE TO POSITIVE CHANGES In short, the authors of the essays in this book provide comprehensive information about their countries’ evaluation systems, programs and policies at various educational levels, and show how these are responding—or trying to respond—to the unique needs of their specific educational contexts. They also point out that there are still many problems to be solved. First of all, the evaluation of student achievement for the purpose of allowing entrance to higher levels of education, including senior high schools and colleges and universities, remains a challenge. Unlike the United States, most East Asian countries have a centralized system that requires students entering senior high schools and colleges and universities to take a national examination. Admissions are solely based on students' performance on a test which covers only selected subjects. “As a result, students tend to pay less attention to learning in classrooms and concentrate more on private tutorials in particular subjects tested in the entrance examination” (Kanjanawasee, p. 75). Such a system often hampers the normal developmental process of a pre-college education. Many individuals’ true potential cannot be fully expressed and developed because of the lack of opportunities and appropriate evaluation mechanisms. Thus, how to screen and identify studetns’ talents and abilities, and how to assure the high quality of education deserved by these students, remain serious questions. Thailand’s new program, which requires the use of high school grades and students’ performance outside of school, offers an alternative model for college admission practices. However, its success remains to be tested. The second challenge is how to evaluate teacher quality and performance in the context of fundamental educational reform. In particular, it remains hard to determine which characteristics of teachers are critical for effective teaching and other professional duties. From the perspective of East Asia’s Confucian heritage, a teacher should be able to fulfill three missions: cultivating a virtuous person, teaching life skills, and removing students’ doubts. This implies that teachers will not only need professional skills for effective teaching: they must also be able to serve as good role models for their students. In this age of specialization and professionalism, the traditional image of a teacher-saint has gradually faded away. Rather, there is a call for teacher-scholars who, as in China and Japan, are expected to do original research as well as teach effectively. Things have become more complicated in an era when more and more educators seem to view their students as mere “consumers.” Is it possible to talk about professional ethics when the pedagogical relationship has become a pedagogical contract that places more emphasis on responsibilities and rights than on the potential development of individuals? There may be two ways to approach the problem of teacher evaluation in our current society. One is normative, and seeks to define a qualified, professional teacher. The other is technical, and seeks to evaluate teachers to see if they are qualified and professional. Of course, we must have the definition or the standard before we can have evaluation. Thailand’s teacher evaluation program includes the evaluation

10

Samuel S. Peng and John C. K. Lee

of teachers’ ethics and moral behavior, and this approach is worthy of the consideration of other East Asian educators. Moreover, a national evaluation program needs to evaluate both for accountability and for innovation (summative and formative evaluation). Yet can we really rank schools and colleges only for the sake of accountability? And how can we lower the pressure placed on students competing for the best schools and colleges based solely on achievement test scores? And should qualitative evaluation be used, in order to improve both instruction and learning, rather than or together with the giving of merely quantitative ranks and grades? These are some of the several key issues that still need to be resolved. Finally, as shown in some of the essays in this book, successful implementation of educational evaluation measures necessitates supportive school leadership (Chapter 6), reduction of teacher anxieties and workloads (Chapter 5), enhanced communication among stakeholders (Chapter 8), and the fostering of shared decision-making across all levels, including government, school, parents and the community (Chapters 5 and 9). But to achieve these prerequisite goals will take the collective wisdom of scholars in the field of educational evaluation. In conclusion, evaluation is an integral part of the educational reform and innovation process. Assessment has the potential to act as a driving force for achieving educational excellence, but it must be carefully designed and implemented. Inappropriately designed educational evaluation programs or systems—those which cannot fit their contexts—may have a negative impact that could be very harmful to education. For example, in many East Asian countries where high-stakes examinations are prevalent, most students cram for exams and are good at taking tests but they have little time to pursue their own personal development, explore their talents, or reflect on the meaning of life. In other words, the pursuit of high scores on tests becomes in itself the central theme or purpose of their life. Yet as Lu in Chapter 1 points out, effective educational evaluation—with the ultimate goal of reforming or improving the educational system—needs the cooperation of the whole community: the parents (and larger public), students, teachers, school administrators and government officials. It will be above all this common or communal will that can take us to where we want to be.

REFERENCES De Grauwe, A. and Naidoo, J. P. (eds.). (2004). School evaluation for quality improvement: An ANTRIEP report. Paris: International Institute for Educational Planning, UNESCO. Kennedy, K. J. and Lee, J. C. K. (2008). The changing roles of schools in Asian societies: Schools for the knowledge society. London and New York: Routledge.

PART I. PRACTICES AND CHALLENGES

In: Educational Evaluation in East Asia Editor: Samuel S. Peng and John Ch-Kin Lee

ISBN 978-1-60692-887-5 © 2009 Nova Science Publishers, Inc.

Chapter 1

EDUCATION EVALUATION AND ASSESSMENT IN TAIWAN: STATUS, INNOVATIONS AND CHALLENGES Mu-lin Lu Ministry of Education, Taiwan, ROC

I. PREFACE Today, in the 21st century - the age of knowledge economy - a country’s competitiveness is measured by its ability to apply knowledge towards economic progress and the pursuit of excellence. In this light, higher education is regarded as the means to cultivating a nation’s high caliber workforce. As such, governments worldwide are performing evaluations of universities to ensure high quality within institutions of higher education, as well as to measure administrative performance and to determine how to allocate funding and resources. The number of Taiwan’s higher education institutions has expanded rapidly over the past decade, more than 70% of Taiwan’s high school and vocational school graduates advance to college or university. Therefore, the importance of controlling and safeguarding the quality of higher education continues to grow. Establishing and implementing an evaluation system is essential for improving the quality of university education. To this end, the Ministry of Education (MOE) authorized the independent Taiwan Assessment and Evaluation Association in 2004 to conduct evaluations of all universities in Taiwan. These evaluations focused primarily on university administration and secondly on academic disciplines, with the universities evaluated in groups. The evaluation results were announced as ratings. This round of evaluations motivated universities and colleges to make a greater effort toward selfimprovement, and it also triggered new thinking regarding the development of university evaluations in Taiwan. Based on the experiences of advanced countries, the MOE established the Higher Education Evaluation and Accreditation Council of Taiwan (HEEACT) in December 2005 to serve as an independent, dedicated agency handling all matters relating to the evaluation and assessment of higher education institutions. The goal of establishing of HEEACT aimed to make it easier to share evaluation experiences, accumulate data, develop

14

Mu-lin Lu

professional and objective evaluation criteria, train professional evaluators, and build a talent pool and database. In the following, we explain the historical development of Taiwan’s university evaluation system, its future development and current challenges.

II. STATUS A. Legal Basis – The University Act: The MOE’s Responsiblities in Evaluating Universities The revised University Act was promulgated in December 2005. Article 5, Paragraph 1, of the Act states that “Universities shall regularly carry out self-evaluations of teaching, research, services, academic affairs, administration, student participation and other proceedings; regulations for evaluations shall be formulated by the universities.” Article 5, Paragraph 2 stipulates that “In order to promote the development of universities, the Ministry of Education shall organize an Evaluation Committee, commission academic organizations or professional evaluators to carry out regular evaluation on the universities, and publish the results as a reference for educational subsidies from the government and the scale of adjustment and development of universities; the methods of evaluation shall be formulated by the Ministry of Education.” These two passages clearly stipulate that carrying out university evaluations is the joint responsibility of the MOE and the universities. We can only attain the goal of improving the quality of higher education if the two parties work together in a concerted effort to plan and promote university evaluations with a sense of direction.

B. Developing a Higher Education Evaluation Agency Suitable for Taiwan 1. The Establishment of Higher Education Evaluation Agencies a. Foreign Aapproaches. Evaluation agencies in other countries can be divided into three categories: “official” organizations such as in France; institutions that are independent from government agencies such as in Britain and Australia; and private-sector organizations such as in the United States. These three types of agencies share in common an emphasis on safeguarding objectivity and fairness, and thus their goal is to create and maintain impartial and unbiased institutions for performing professional evaluations. Their sources of funding can be roughly divided into government subsidies (Britain and France), membership fees (U.S.) and a combination of government subsidies and membership fees (Netherlands, Australia). In terms of function, these organizations carry out both institutional evaluations and academic discipline evaluations (also called specialized field evaluations). Depending on their focus, academic discipline evaluations can again be further divided into teaching evaluations and research evaluations. b. Taiwan’s Approach. Taiwan lacked an evaluation mechanism and a dedicated evaluation agency until only recently, when the Ministry of the Interior approved the establishment of the Taiwan Assessment and Evaluation Association (TWAEA) in August 2003. TWAEA is a private-sector agency that has carried various evaluation work on behalf of the MOE: These projects included the University Evaluation Planning and Implementation

Education Evaluation and Assessment in Taiwan

15

Plan of 2004, the University and College Faculty Development Center Evaluation Planning and Implementation Plan of 2005, the Evaluation of Universities of Science and Technology of 2005 and 2006, and the University Administration Follow-up Evaluation Plan of 2006. In addition, the MOE and Taiwan’s universities and colleges came together in December 2005 to jointly establish the Higher Education Evaluation and Accreditation Council of Taiwan (HEEACT), a public organization with the goal of serving in a capacity similar to the evaluation agencies of Britain and Australia, that are known for their “independence from official agencies.” However, as a sound evaluation institution cannot be created overnight, the MOE is currently supervising and assisting HEEACT to ensure that its development proceeds smoothly and efficiently. Presently, the MOE subsidizes funding that HEEACT requires for planning and evaluations, as it gradually works to establish a systematized, institutionalized dedicated agency for university evaluation. 2. Stage-by-Stage Establishment of a Higher Education Evaluation System a. Present Goals. Present goals for HEEACT include: to have HEEACT plan and implement all types of evaluation work so as to accumulate practical experience; to compile and study information on higher education evaluation in advanced countries, produce objective, professional evaluation criteria, and provide courses for the training of university evaluators; to develop the council into an evaluation planning institution; and to set up a diversified evaluation mechanism. b. Medium-to-Long Term Goals. Medium to long-term goals for HEEACT include: to gradually develop the Council into a dedicated, professional evaluation institution with the task of researching and planning the evaluation of Taiwan’s universities; to set up databases and talent pools; and to pass on these evaluations to other specialized evaluation institutions. 3. Present University Evaluations Conduct a. Putting into Practice University Self Evaluation. The University Act stipulates that universities shall regularly conduct self evaluations. Moreover for the sake of sustainable development, universities shall safeguard the quality of their administrative affairs and assume responsibility for administrative performance and improving their image and reputation. b. Assisting TEEAC in Implementing University Evaluation Work. Current Major Tasks: i. Planning and researching Taiwan’s university evaluation system. ii. Compiling and studying information on the evaluation of universities overseas. iii. Assisting universities in applying for international accreditation in all academic disciplines. iv. Developing and establishing qualification criteria for domestic and foreign professional evaluation institutions. v. Establishing talent pools and databases for the evaluation of universities in Taiwan. vi. Providing courses for the training of university evaluators. vii. Other evaluation-related matters. c. Implementing the University Department and Graduate School Evaluation Plan (20062010) based on a Five-Year Cycle i.

Goals of university department and graduate school evaluation

16

Mu-lin Lu a.

To gain an understanding of the current quality of university departments and graduate schools b. To determine and recommend status and deadlines for the quality accreditation of university departments and graduate schools. c. To encourage universities to establish quality improvement mechanisms for their departments and graduate schools. d. To assist university departments and graduate schools in developing unique characteristics and the pursuit of excellence. e. To use evaluation results as reference in formulating higher education policies for the government ii. Unique features of university department and graduate school evaluations a. b.

c. d. e. f.

Characteristic differences of university departments and graduate schools are respected; fixed quantitative indicators are not used University departments and graduate schools must explain on their own initiative to what extent they have progressed in fulfilling their founding mission and goals, in line with evaluation criteria. Evaluation content focuses on the assurance of teaching quality, but also pays attention to research quality. Evaluation process emphasizes that university departments and graduate schools should establish and implement self improvement mechanisms. Criteria-based self comparison is emphasized, rather than comparison with other universities or university departments and graduate schools. Evaluation results are reflected in an accreditation system, consisting of three categories: accreditation granted, accreditation on hold, and accreditation denied. Applicants that do not meet the criteria shall withdraw. Those whose accreditations that are on hold or have been denied shall accept a follow-up evaluation in the following year.

iii. Items covered in university department and graduate school evaluations Criteria 1: Goals, special characteristics and self improvement Criteria 2: Course design and teaching Criteria 3: Student learning and student affairs Criteria 4: Research and professional performance Criteria 5: Graduate performance iv. Dealing with evaluation results: Accreditations for university departments and graduate schools that are put “on hold” mainly for the following reasons: insufficient number of full-time teachers; insufficient learning space and resources; unclear goals; or course design does not reflect the goals of the university department/graduate school and lacks self improvement mechanisms. Accreditations are denied if the above-mentioned problems are severe enough that student learning and teaching quality are seriously affected. University departments and graduate schools whose accreditations have been granted shall submit a self improvement plan and

Education Evaluation and Assessment in Taiwan

17

implementation objectives based on the suggestions in the evaluation result report. They will not have to undergo another evaluation for the following five years. For those whose accreditations have been put on hold, HEEACT will conduct a follow-up evaluation regarding their shortcomings. Meanwhile, schools ‘on hold’ may not expand student recruitment or apply for the establishment of new graduate schools. For those whose accreditations have been denied, HEEACT will carry out a reevaluation of all items and the MOE will consider a reduction of student quotas. Should accreditation be denied after a follow-up evaluation or a reevaluation, student recruitment will be stopped. 4. Continuing with follow-up evaluations of university administrations 5. Encouraging universities to participate in international accreditation As Taiwan’s universities seek to develop unique characteristics and reach global standards, we cannot only use evaluations and assessments to strengthen the quality and development direction of academic disciplines. Universities and colleges are being encouraged to actively participate in international accreditation schemes as a way of obtaining international recognition. University departments and graduate schools that have already obtained international accreditation will be exempt from further evaluations during the accreditation period. Participation in international accreditation schemes will raise the quality of higher education and speed up the internationalization of universities in Taiwan, while also winning them the recognition in international academia. Universities and colleges in Taiwan currently participate in the following international accreditation schemes: a.

Medical school evaluation:

Since medicine is a specialized field, medicine department evaluations are planned and conducted by the Taiwan Medical Accreditation Council (TMAC). In 2002, TMAC’s evaluation system was approved by the National Committee on Foreign Medical Education and Accreditation in the United States. b. Engineering education accreditation: In 2005, engineering education accreditation system and schemes established by the Institute of Engineering Education Taiwan (IEET) were approved by the Washington Accord (WA) signatories. Engineering departments that have gained IEET accreditation can directly obtain WA recognition. Twelve engineering departments gained IEET accreditation in 2004, followed by 35 departments in 2005, and 41 departments in 2006, which shows that Taiwan’s engineering departments are actively participating in international accreditation. c.

Accreditation by the Association to Advance Collegiate Schools of Business (AACSB) in the United States:

The business management departments of Fu Jen Catholic University and National Sun Yat-sen University gained AACSB accreditation in April 2005, while National Chengchi University followed suit in December 2006. The business management departments of other universities including Asia University, Feng Chia University, National Cheng Kung University, National Chiao Tung University, National Chung Cheng University, National

18

Mu-lin Lu

Taiwan University, Tamkang University, and Yuan Ze University have applied for AACSB accreditation.

III. INNOVATIVE ACTIONS To truly achieve the goal of raising the quality of higher education, we must shed longheld thinking and approaches to university evaluations, and bring in new energy to create new possibilities for the future. In the following, we explain innovative actions being taken with regard to the university evaluation system, the role of universities and the application of evaluation results.

A. Creating a Well-Designed Evaluation System Evaluation work in Taiwan needs to develop toward systematization and institutionalization. We aim to create a well-rounded and sound evaluation system, building upon every single evaluation process. In order to do this, we must establish a team of evaluators and build a database, while properly integrating research, planning, implementation and auditing. Article 4 of the Regulations Governing University Evaluations stipulates that university administrations, departments and graduate schools, as well as course curriculums shall be evaluated every five to seven years. By putting into practice such evaluation work and building an evaluation system, we are bringing more effective management to our universities.

B. Leading Universities Towards Greater Autonomy Presently, Taiwan’s evaluation system is led by the government, with passive participation from universities. However, as the evaluation system evolves and universities mature with the evaluation process, we hope that all schools will understand the need for long-term self improvement. We also hope that evaluation results released by a professional, fair agency will serve as a reference for schools seeking to make improvements and develop unique characteristics. As soon as this practice takes root and a dedicated evaluation agency is able to function in its full capacity, we could consider establishing an evaluation system under which universities proactively apply for evaluation.

C. Improving the Quality of University Management, Efficiently Allocating Education Funding In Britain, higher education evaluation covers the two areas: teaching and research. The Quality Assurance Agency for Higher Education (QAA) is in charge of institutional and teaching evaluations, while the Research Assessment Exercise (RAE) is mainly in charge of evaluating research. Both agencies submit their evaluation results to the Higher Education

Education Evaluation and Assessment in Taiwan

19

Funding Council for England (HEFCE), which distributes grants and funding to higher education institutions based on these results. While the establishment of a systematized university evaluation system in Taiwan is still in the initial stages, we hope that the implementation of university evaluations will compel universities to attach greater importance to administrative performance. Moreover, the MOE will refer to evaluation results when deciding on matters relating to university development including funding, grants, and subsidies, the increases or changes to colleges, departments, graduate schools, and student quotas. The MOE’s overall aim in this regard is to achieve more efficient use of higher education resources, as well as their proper allocation.

IV. CHALLENGES In comparison with many advanced countries, the implementation of a university evaluation system in Taiwan is still in the initial stage. Numerous systemic or technical problems still need to be solved. Eight major challenges facing Taiwan’s university evaluation are listed below:

A. Training Professional Evaluators Evaluators presently selected by the dedicated evaluation agency can be divided into three categories based on the type of evaluation: 1. Veteran professors with research or practical experience in higher education administration; 2. Business or industry representatives familiar with specialized academic fields or university affairs; 3. Experts and scholars specializing in certain specialized fields. TEEAC’s evaluators hold an abundance of experience and knowledge in university administration or teaching in a specialized field. However, as university evaluation is a highly specialized undertaking, we need to address how the evaluation expertise of the evaluators can be raised through relevant courses, seminars and workshops.

B. Increasing Evaluation Funding In addition to TWAEA, which as a private-sector, nongovernmental organization, is independently funded and operated, the MOE and 153 universities and colleges have donated money to fund the Higher Education Evaluation and Accreditation Council of Taiwan (HEEACT). Since the council is a public organization, the MOE is fully subsidizing its operating costs during its initial stages. In the future, the MOE hopes that HEEACT will be able to engage in its own funding and create other revenue once operations are running smoothly and the entire system starts to take shape.

20

Mu-lin Lu

C. Designing Evaluation Indicators and Methods One example of the effort to design evaluation indicators and methods are the evaluations currently used for Taiwan’s university administrations, departments and graduate schools and colleges of medicine, where evaluation results come in the form of suggestions for improvement submitted by evaluators. This presents a new challenge to the schools and the public, who are accustomed to thinking in quantitative terms and are likely to note that evaluators evaluating different universities, departments and graduate schools are not the same. Our next step is to convince the schools and the public of the completeness of these indicators, the scientific soundness of the evaluation methods and the truth of the evaluation results.

D. Implementing Self Evaluation As the sanctuary of academia and high-level professional training, universities must assume responsibility for their own administrative efficiency, their research performance and social services. Moreover, universities should have a full understanding of their current strengths and weaknesses, to be gained continued and random self evaluations. Given this fact, universities can only achieve the goal of sustainable development if evaluations are incorporated into their organizational culture. However, Taiwan currently lacks a strong consensus regarding the essence and methods of university self evaluation. As a result, universities in Taiwan remain passive about accepting evaluation models based on outside performance requirements, which has not been helpful in their long-term development. Article 5, Paragraph 1 of the revised University Act stipulates that “universities shall regularly carry out self evaluation on teaching, research, services, academic affairs, administration, student participation and other proceedings; regulations for the evaluation shall be formulated by the universities.” In the spirit of this passage, the MOE will continue to encourage universities to create regulations for self-evaluation and to regularly carry out self-evaluations.

E. The Fairness of Evaluators In choosing the members of an evaluation committee for the evaluation of a university department or graduate school, HEEACT first submits a list of recommended evaluators to the department or the graduate school that is to be evaluated. The department or graduate school in question may reject committee members for any substantial reason. In principle, we already meet the general public’s expectations for an open and transparent evaluation procedure. Still, since Taiwan’s academic circles are rather small, it is difficult to completely draw a line between professors teaching at a department or graduate school that is being evaluated and the members of its evaluation committee. We still need to deliberate further on how we can truly ensure fairness and objectivity, and avoid conflicts of interest, when selecting evaluation committee members, as well as whether or not we need to bring in foreign scholars and experts.

Education Evaluation and Assessment in Taiwan

21

F. Procedure for Generating Evaluation Results A number of universities in Taiwan began department and graduate school evaluations in 2006. The review process for the evaluation results was mainly divided into the following three steps: committee members submit an evaluation report following an onsite visit, then the Academic Discipline Preliminary Review Committee reviews the report, and lastly a decision is made by the Accreditation Review Board (under the Council). If there are excessive discrepancies between the “Onsite Visit Draft Report” that is sent to all institutions under evaluation and the second-stage “Suggestions on Evaluation Results” submitted by the Academic Discipline Preliminary Review Committee or the third-stage “Report on Accreditation Results” by the Accreditation Review Board, the evaluated institutions will easily question the fairness of the evaluation process and reject the results. Therefore, we must consider how the procedure that generates the evaluation results can be designed with even more precision so that the institutions undergoing evaluation are not shocked when they obtain the results; that the evaluation results truly reflect the situation of the evaluated institution; and that there are no large discrepancies between the results and the institution’s situation during the onsite visit period.

G. Ways of Presenting the Evaluation Results Evaluation results can be presented as rankings, ratings, or the selected best. In the case of university administrations, and departments and graduate schools in Taiwan, the evaluation results are announced in the form of ratings, which complies with the spirit of the accreditation system. Furthermore, the criteria for deciding in favor or against accreditation include whether relevant measures taken by the institutions match their stated goals and whether they have implemented self improvement mechanisms. Feedback from the public shows that people ask themselves why the MOE does not announce university rankings to enable citizens to immediately clearly understand the strengths and weaknesses of each university and to help students make their choices. In actuality, the announcement of the evaluation results pertains to the goals and spirit of university evaluation. Therefore, there are certain differences in the way evaluation results are announced, depending on the type of evaluation and the evaluation goal. In the future, the MOE will hold more in-depth discussions with the evaluating agency regarding the form in which evaluation results will be announced.

H. Evaluation Follow-Up Mechanism The focus of evaluations is not based on grade or rank, but rather to gain an understanding of teaching quality and students’ learning performance in a systematic, scientific, and objective way. Thus, a long-term evaluation mechanism must be established so that the results of previous evaluations can serve as reference for monitoring a university’s performance. Only then will universities be able to keep improving their shortcomings and move toward excellence.

22

Mu-lin Lu

CONCLUSION In light of the trend towards globalization and internationalization, Taiwan’s higher education must be able to improve its quality, ensure performance, find its niche and develop unique strengths to gain a sure footing on the international stage and move towards excellence. University evaluation work plays a major role in this process. Our higher education evaluation work will succeed only if it is systematically and scientifically planned, implemented and reviewed. Moreover, we need to keep studying effective approaches used by advanced nations to serve as lessons for us. We hope that Taiwan’s higher education evaluation - with the assistance of an attentive public, active university participation, efficient government action and a well-rounded evaluation system-will be able to help our universities to achieve the vision of quality assurance and assume genuine responsibility for their performance.

In: Educational Evaluation in East Asia Editor: Samuel S. Peng and John Ch-Kin Lee

ISBN 978-1-60692-887-5 © 2009 Nova Science Publishers, Inc.

Chapter 2

EDUCATIONAL EVALUATION IN SINGAPORE: CURRENT STATUS AND FUTURE CHALLENGES R. Subramaniam National Institute of Education, Nanyang Technological University, Singapore

ABSTRACT As an island state, which is bereft of any natural resources, Singapore places tremendous emphasis on the development of its human resources through education. It has consistently ensured that its curricula are not only internationally competitive but its examination system is also benchmarked against international standards. Educational evaluation is a critical aspect of this quality control. This article focuses on educational evaluation in two areas: learning of science and mathematics at the primary, secondary and pre-university levels, and institutional evaluation of schools. It is shown that the use of external evaluation for assessing learning: centralized testing at the primary leaving level by an independent authority, a partnership model of this independent authority with an international examinations syndicate for testing at the secondary and pre-university leaving levels, and regular testing of science and mathematics achievement of students at the primary and secondary levels by an international evaluation association, have been factors which explain why the science and mathematics curricula in Singapore continue to be of a high standard. The use of academic attainment of students to band secondary schools and junior colleges in separate achievement tables, plus the inclusion of other academic and non-academic factors in these achievement tables is a key aspect of institutional evaluation, and explains why stakeholders in the country have been able to derive satisfaction from the government’s heavy investments in education. Some implications of these modes of educational evaluation are discussed.

Keywords: educational; institutional evaluation; Singapore; science and mathematics achievement; banding of schools.

24

R. Subramaniam

INTRODUCTION Education is probably the most important lever to empower a person in life as well as stimulate socio-economic development in a country. High levels of education of the citizenry have traditionally been correlated with enhanced developmental activities in a country. The challenges posed by globalization, new developments in science and technology, diffusion of information and communication technology in society, outsourcing, off-shoring and corporate downsizing have impacted tremendously on countries in the past ten years. Jobs have taken flight to countries which can provide best value for money. Since these developments affect the competitiveness of nations, they therefore impact not only on the employment of their workers but also on their education system since this prepares students for life in the workforce. Educational reform has been an important way in which many countries have been addressing these issues to varying extents, so that their students are better able to meet the challenge of the new economic landscape. Science and technology are potent agents for stimulating socio-economic development. The land-labor-capital model of economic development is a yesteryear paradigm that is increasingly giving way to a model based on science- and technology- driven economic development. The experiences of Western countries show that this development model has been very helpful in providing their people with a good standard of living. A number of countries in Asia have also successfully experimented with this model of economic development for nation building, for example, Japan and Singapore (Tan and Subramaniam, 1998). One of the key aspects of this model is that a country’s science and mathematics curricula need to be internationally competitive. Regular benchmarking of standards will help to ensure that educational reform is proceeding in the right direction and that the education system is well positioned for achieving the desired outcomes. Educational evaluation is a powerful tool to assess various aspects of the education process as it can produce performance data for the use of stakeholders. The origins of educational evaluation can be traced to the desire for quality control in the education process so that standards and accountability can be promoted. The large number of schools in a country, the diversity in their curricula and differences in assessment procedures at various levels pose issues of a systemic nature which, if not addressed properly, can affect standards. Educational evaluation affords directions in this regard as it can help to benchmark schools and subject offerings at the various levels, thus ensuring that they measure up to the desired norms. The epistemological foundations of educational evaluation have matured well over the past 40 years, and several evaluation instruments and methodologies are available to assess the efficacy of various aspects of the education process. Its importance has led to the formation of a number of professional bodies - for example, the International Association for the Evaluation of Educational Achievement (IEA), which has done yeoman service since its inception in 1959. The findings of this association have appeared in international journals and the media, and are highly regarded by the political establishment in several countries. The principal objective of this paper is to share some insights into the process of educational evaluation of science and mathematics learning at the primary, secondary and pre-university levels in Singapore. An additional objective is to describe an evaluation approach that has been used to appraise schools in the country. Whilst traditional modes of educational evaluation are deeply embedded in the education system in Singapore, the use of

Educational Evaluation in Singapore

25

educational evaluation of learning via leaving level examinations administered by an independent local authority or in conjunction with an international examinations syndicate provides the best assurance of quality control and academic rigor. Institutional evaluation must perforce be based predominantly on such results.

EDUCATION SYSTEM IN SINGAPORE To better appreciate the context of the ensuing discussion, a brief description of the education system in Singapore is given in this section (Lim and Tan, 1999). The medium of instruction in schools is English, except for mother tongue languages. Primary education is of six years duration. Students enter primary school at the age of seven. The core subjects are English, science, mathematics and mother tongue – mathematics is offered from the start but science is introduced only at the primary three level. At the end of primary four, there is a national streaming examination (Caleon and Subramaniam, 2005). Those who are channeled into EM1 and EM2 streams in Primary five are of above average abilities compared to those in the EM3 stream. Students in EM1 stream can take an extra subject (Higher Mother Tongue). However, from 2004, the EM3 stream was abolished. At the end of primary six, students sit for the Primary School Leaving Examinations. Based on the results of the Primary School Leaving Examinations, students are channeled into one of three streams at the secondary level: Special / Express, Normal or Technical. Such streaming allows students to learn at a pace that is best suited for their intellectual capabilities. Students in the special / express streams take the Singapore-Cambridge General Certificate of Education ‘Ordinary’ Level Examinations at the end of four years. Those in the latter two streams take this at the end of five years – prior to this, they would have taken the Singapore-Cambridge General Certificate of Education ‘Normal’ Level Examinations at the end of four years. The results of the Singapore-Cambridge General Certificate of Education ‘Ordinary’ Level Examinations determine how students progress further. Those with average grades can opt for polytechnic education to attain a diploma in a vocational discipline; those with weak grades can opt for technical education, which will equip them with a certificate in a technical specialty; and those who do well can opt for pre-university education at one of the junior colleges or a centralized institute – students enrolling in the former take the SingaporeCambridge General Certificate of Education ‘Advanced’ Level Examinations after two years while those who enroll in the latter take these after three years.. Those with good grades in the latter examinations can opt for university education to get a degree. The Ministry of Education is the agency responsible for overseeing education matters in the country. A particular source of strength is its implementation machinery, which ensures that policy matters are effectively translated into practice in the 350+ schools in the country.

EDUCATIONAL EVALUATION OF SCIENCE AND MATHEMATICS LEARNING AT THE PRIMARY LEVEL The curricula for science and mathematics at the primary level are prescribed by the Ministry of Education. At each level in the 6 years of education, individual schools

26

R. Subramaniam

administer their own examinations in all the subjects. Needless to say, there is bound to be some variability in the standards and difficulty levels for the various subject examinations. It is only at the Primary School Leaving Examinations (PSLE), which is held towards the end of primary six, that all students sit for the same examination in all the subjects and know where they stand at the national level. The PSLE is administered by the Singapore Examinations and Assessment Board (SEAB), the national independent agency for assessment and examinations. The PSLE is of a high standard for the following reasons: (a) The range of questions set in the subject examinations encompasses the different levels of Bloom’s taxonomy, with significant emphasis being placed on the higher levels. This is in view of the pronounced weightage given in the curricula on promoting thinking skills and problem-solving skills among students. (b) Even though the PSLE examination papers are not available in the public domain (as students are not allowed to take back the papers), anecdotal evidence based on the feedback provided by teachers and students as well as the attention occasionally given to these in the media indicate that the questions are intellectually challenging. (c) It is set by an independent authority, and there are rigorous quality checks on what appears in the test papers. Additional evidence that the science and mathematics standards at the primary level are internationally competitive can be drawn from the results of the Trends in Mathematics and Science Study (TIMSS), an examination set by the International Association for the Evaluation of Educational Achievement (IEA) every four years. These examinations are taken by students of many countries and afford a rigorous comparative benchmark of the science and mathematics curricula of a country. The findings from the TIMSS study reveal the following observations in relation to Singapore (Martin et al. 1997, Mullis et al. 1997; Toh et al. 2006): •

•

In TIMSS 1995, primary three students from Singapore were ranked 2nd in mathematics, while primary four students were ranked 1st in mathematics and 7th in science In TIMSS 2003, primary four students from Singapore were ranked 1st in mathematics and 1st in science.

Juxtaposing the observations made on the PSLE with the results of the TIMSS support the assertion that the curricula and assessment practices for gauging learning in science and mathematics at the primary level in Singapore are of a high standard.

EDUCATIONAL EVALUATION OF SCIENCE AND MATHEMATICS LEARNING AT THE POST PRIMARY LEVELS The post primary education system normally comprises 4 years of secondary education and 2 years of pre-university education. At all levels, examinations are conducted by

Educational Evaluation in Singapore

27

individual schools and junior colleges on a semester basis. There is bound to be some variability in the standards and difficulty levels of these examinations. However, the milestone examinations taken at the end of secondary and pre-university education are national examinations and, in effect, provide an evaluation of how well schools have prepared their students in the various subjects. What is significant is that these leaving level examinations are conducted jointly by two independent authorities: the Singapore Examinations and Assessment Board (SEAB) and the University of Cambridge Local Examinations Syndicate (UCLES), an international examinations syndicate. The external checks and validity provided in this manner constitute a reliable endorsement of the standards of the syllabus and curricula in the sciences and mathematics. The involvement of UCLES in educational evaluation of students’ learning, based on the syllabus prescribed by the Ministry of Education, offers a number of advantages: • •

• •

It is internationally recognized and has long years of experience in setting and marking examination papers. The examinations set and marked by it provide assurance that the standards are internationally benchmarked. (A few subject papers such as second language, are, however, marked locally). Involving an external agency to set and mark the examination papers ensures impartiality. Its co-branding of the leaving level examinations offers Singapore students recognizable entry level qualifications if they are to apply for placement in overseas schools or admission to foreign universities in the USA, UK, Australia and New Zealand, four of the popular destinations for higher education.

Additional support for the international competitiveness of the science and mathematics curricula at the secondary level is provided by data from the TIMSS studies done in 1995, 1997 and 2003. The findings are as follows for the Secondary two level: • • •

In TIMSS 1995, Singapore students were ranked 1st in science and 1st mathematics. In TIMSS 1999, Singapore students were ranked 2nd in science and 1st in mathematics. In TIMSS 2003, Singapore students were ranked 1st in science and 1st in mathematics.

Against the backdrop of the foregoing, it is clear that evaluation of science and mathematics learning at the secondary and pre-university levels are rigorous from an international standpoint.

INSTITUTIONAL EVALUATION OF SCHOOLS IN SINGAPORE Institutional evaluation is an integral aspect of the process of educational evaluation. Stakeholders and the public would like to know how effective schools have been in preparing students. A good evaluation system that cuts across all schools is indispensable from a

28

R. Subramaniam

national perspective as informed decisions can then be made on improvements to be made as well as lessons to be drawn from best practices. Since 1992, secondary schools and junior colleges in Singapore have been ranked on the basis of the academic standing of their students at the Singapore-Cambridge General Certificate of Education ‘Ordinary’ Level Examinations and Singapore-Cambridge General Certificate of Education ‘Advanced’ Level Examinations respectively. This has proved to be an effective tool, in effect producing a league table of schools and colleges to see where they stand nationally (http://www.moe.edu.sg). As the ranking is on the basis of leaving level examinations administered by SEAB and UCLES, they have the advantages of validity and reliability. Since ranking was introduced, it has been noticed that standards in schools and colleges have risen. The evaluation for schools works in this way. For the Normal stream, a composite index based on L1B4 is generated for each secondary school: average aggregate score of English (L1) plus the best four subjects taken by their students in the GCE ‘O” level examinations. Using the cut-off score of 21, the schools are distributed into 5 bands. For the Special /Express streams, a composite index (L1B5) weighted on the average aggregate scores of either English Language or Higher Mother Tongue (L1) plus the five best subjects taken by their students in the GCE ‘O” level examinations is assembled for each school. In this way, 9 bands below the cut off score of 18 points are generated. The lower the band score, the better is the performance of the school. For the year 2007, there were 58 schools in Special /Express stream as well as 51 schools in the Normal stream in the School Achievement Tables. The other important factors which are included in the Achievement Tables include: •

•

•

• •

Academic Value-added Performance: this is a measure of how well the schools are expected to perform in the GCE ‘O’ Level examinations based on the PSLE results of their students and how they actually performed. Physical and Aesthetic Achievement: the data for physical fitness is weighted on three scores – percentage of students who pass the National Physical Fitness Award test, percentage of overweight students, and fitness index. Best Practices: this focuses on effective practices and systems that lead to desirable outcomes in education – for example, organizational effectiveness, students’ allround development, staff well-being, teaching and learning. Character Development: this is to reiterate the point that values education and life skills are an integral aspect of students’ holistic education. National Education: this is to ensure that students are cognizant of Singapore’s history and of how the country has achieved much despite the absence of natural resources and the presence of other constraints.

A similar evaluation system holds at the pre-university level. For institutional evaluation, data on leaving level examinations are available with the Ministry of Education. Data on other aspects are obtained from schools via various means: for example, self-reporting by schools, interviews and external evaluation by teams from the Ministry of Education. Data on rankings (or banding) of schools provide the necessary information for schools to know where they stand with respect to others nationally and also provides the momentum for

Educational Evaluation in Singapore

29

them to improve their standings. The rankings done by the Ministry of Education have the advantages that the evaluation criteria are transparent and rigorous.

DISCUSSION Educational evaluation affords a systematic way of appraising the various processes in education. If carried out by a competent authority, the information it provides can offer an impartial perspective that would be of important value to stakeholders and the public. It would, in effect, contribute towards certification of the quality of education. In this chapter, we have focused on two aspects of educational evaluation: assessment of science and mathematics learning at the primary, secondary and pre-university levels, and evaluation of how schools meet their overall objectives. In a country with numerous schools and where there are bound to be variations in the standards and the difficulty levels of the examination papers set by these schools, a common leaving examination provides the best mode of evaluating how schools have prepared their students based on the syllabus prescribed by its Ministry of Education. This process of centralized testing evens out any differences in standards, difficulty levels and anomalies in content coverage that are inherently present in school-based examinations. This does not mean that school-based examinations have their drawbacks, it is just that the common denominator afforded by the same examination taken by all students provides more rigorous benchmarking of learning. More importantly, it is of interest to stakeholders to know how effective schools have been in their mission at a national level - a common mode of referencing provided by centralized testing offers validity and reliability in this regard. At the primary level, the examinations set by the SEAB, an independent agency, provides not only quality assurance but also third party evaluation of how schools have prepared their students for the examinations. Whist this mode of appraisal based only on the leaving level (primary six) to evaluate learning may be criticized for the absence of consideration of the results achieved by students at the other levels (primary one to five), it has to be borne in mind that the PSLE is not just an examination to assess what students have learnt in primary six but generally at the upper primary level (primary four to six) as well. That the examinations set at the primary leaving level are of a high standard can also be gauged from the fact that the national agency has been requested to provide testing services at the primary leaving level for other countries in the region such as Thailand, Philippines and Indonesia. On an international basis, the TIMSS results provided by the International Association for the Study of Educational Evaluation provide additional evidence that the science and mathematics curricula in Singapore are internationally competitive. At the secondary level, again a similar situation prevails – school-based examinations at all stages except towards the end of secondary education, where a nationwide examination for all subjects is taken. The examinations are administered jointly by SEAB and UCLES. The involvement of a renowned international examinations syndicate in national examinations provides strong support that international third party evaluation has its merits as it can ensure that high standards are maintained and that the examination results are internationally recognized.

30

R. Subramaniam

Additional support for the international competitiveness of the science and mathematics curricula at the primary and secondary levels is provided by the results of the TIMSS examinations administered by the International Association for the Study of Educational Evaluation. The situation at the pre-university level is also similar. Again, with the examinations jointly administered by the Singapore Examinations and Assessment Board and the University of Cambridge Local Examinations Syndicate, educational institutions know where exactly they stand at the national level in terms of the academic achievements of their students. A major reason why many countries participate in TIMSS is that it provides data for the evaluation of the effectiveness of their education system in science and mathematics, two subjects that count significantly in the science- and technology- driven world of today, with respect to those of other countries (Johnson, 1999). The testing is expensive and logisticsintensive but the data they provide is impartial and can catalyze much needed educational reform. Purves (1991, p34) has summarized this succinctly as follows: The world could be conceived as a huge educational laboratory where different national practices lend themselves to comparisons that could yield new insights into determinants of educational outcomes. The significance given to IEA surveys of science and mathematics achievement has led to a situation where rankings of countries have become a de-facto endorsement of educational indices of these countries (Johnson, 1999), not only in the international research literature but also in the popular media. Thus, the TIMSS results provide a solid endorsement of the educational standards in science and mathematics at the primary and secondary levels in Singapore. The system of streaming in schools, in Singapore has been criticized as being elitist and that it puts a stigma on those in the less desirable stream. It is a unique practice not found elsewhere. Critics of streaming tend to ignore the fact that streaming ensures that students are channeled into classes where they can study at a pace which suits their academic predisposition. Teachers are then in a position to provide better attention and support for their learning needs. One of the criticisms leveled against the ranking system is that it places too much emphasis on academic attainment. The process of education is more than just studying, it needs to develop students holistically through other means and these ought to be given weightage in the rankings – so say the critics! Also, as the premier schools get the better students (Tan and Gopinathan, 2000), the ranking is skewed in their favor. Moreover, with the excessive emphasis on examination results, schools may focus on producing examinationsmart students and not explore innovative approaches in their teaching. (Poh, 1999; Cheah, 1998). These concerns have led the Ministry of Education in 2004 to tweak the evaluation system by expanding the number of dimensions on which assessment is based. Instead of ranking lists, there are now School Achievement Tables, one for the Special / Express stream and another for the Normal stream. In the process, schools are now banded together – those with similar academic results are grouped together and are listed alphabetically, and it is not possible for the public to know the exact position of the schools in the tables. In the new evaluation system, predominant weightage is still given to the performance of the schools in the leaving level examinations as this is central to the mission of education but additionally other academic and non-academic factors are included in the achievement tables. This ensures that schools prepare students holistically, not just academically. In the new landscape,

Educational Evaluation in Singapore

31

significant emphasis is given to the promotion of creativity, innovation and enterprise among students (Tan and Subramaniam, 2002). The high stakes examination culture has put additional pressure on teachers to deliver and students to perform. Anecdotal evidence suggests that these may have implications on family life – for example, teachers putting in longer working hours and students clocking lengthier hours of study. There is, however, recognition that Singapore is unique – it is a small country with no natural resources and its survival is very much dependent on its linkages to the global economy and attracting of investments; education thus provides the best opportunity for its people to aspire for a better life and compete in the global economy. In such a scenario, it is inevitable that sacrifices have to be made. To a significant extent, the high salaries that teachers draw help to mitigate any such concerns they may have. Another aspect of the high stakes examination culture is the concern that late developers as well as those who are not academically inclined but who excel in pursuits such as the arts and sports are at a disadvantage. To address the former, multiple routes for entry to tertiary institutes are in place – for example, a student who has been in EM 3 stream in primary school and normal technical stream in secondary school can enter university after a stint in a technical institute and then at a polytechnic, provided that admission criteria to the university are met. To help meet the aspirations of the latter group of students, a School of the Arts as well as the Singapore Sports School has been established. In fact, the arts and sports scene in Singapore has undergone a renaissance in recent times, and job opportunities abound. The Singapore experience offers the following pointers to other East Asian countries which have less centralized system of education: •

• •

Ranking (or banding) of schools needs to be seriously considered so as to inject greater accountability in the system. Whilst there may be problems and constraints in introducing this in less centralized systems where there are differing examination systems, appropriate rubrics can be designed with suitable statistical normalizations to account for these differences. A national leaving level examination can be considered for the graduating years of schooling so that schools know how they have prepared their students. There needs to be pronounced emphasis on science and mathematics in the curricula as these subjects are crucial for the economy.

FUTURE SCENARIOS The 21st century presents new challenges to countries in various areas. No where is this more pronounced than in the area of education as it has the potential to influence developments in other sectors of the economy. Countries whose education systems are well positioned to address the new challenges can ensure that their students would be able to compete effectively in the new economy. Educational evaluation will thus continue to exert importance as it affords scope for enhancing institutional practice so that schools can position themselves for the future whilst evolving organically to the next level. In Singapore, the present system of educational evaluation of science and mathematics learning – use of national leaving examinations at the primary, secondary and pre-university

32

R. Subramaniam

levels, is not likely to change as it has served her well over the years. These are high stakes testing that have proved their utility value over the years. The examinations set by an independent authority at the primary leaving level affords third party validation of the quality and rigor of the educational standards prescribed by the Ministry of Education for schools. The links with UCLES at the post-primary levels are likely to be maintained as these have served Singapore well over the past few decades. In the author’s view, one aspect of testing that may be introduced in future examinations to gauge learning is the use of 2-tier multiple choice questions. Currently, multiple choice questions (MCQs) appear as part of most subject examinations at all leaving levels. The use of MCQs has the advantage that it can test a good number of concepts across the curricula in a limited time and also provides ease of administration and marking for teachers. One of the disadvantages of MCQs is the probabilistic dimension inherent in its format. In an MCQ with four options, the probability of getting the correct answer purely by guess work is 0.25. Also, by a process of elimination of options by intelligent guessing, it is possible to narrow down the answer to the correct option. A 2-tier MCQ addresses this significantly by asking students to justify their answers from another choice of options (Treagust, 1988). An example of a 2tier MCQ is given here:

Question Peter and Jane stand 300 meters apart in an open area where there are no buildings or trees. They shouted ‘Hello’ to each other at the same time. Given that Jane’s voice has a lower pitch compared to Peter’s and that there is no wind blowing, which one of the following statements is true:

Answer (a) (b) (c) (d)

Peter and Jane hear the word ‘Hello’ at the same time. Peter hears Jane’s voice first Jane hears Peter’s voice first None of the above

Reason (i) Since Peter’s voice has a higher pitch, it travels faster and Jane thus hears him first. (ii) A lower pitch sound carries less energy, and thus travels slower. (iii) The speed of sound is not affected by the pitch, hence Peter and Jane will hear at the same time. (iv) ____________________________________________________________ (Write your reason here if you do not agree with any of the above reasons.) Use of 2-tier MCQs would thus provide better indication of students’ understanding of a concept tested in a question. The use of 2-tier MCQs, however, is not without their

Educational Evaluation in Singapore

33

drawbacks, notwithstanding its acknowledged superiority over traditional MCQs. For example, the length of an examination paper will increase, considerable time and effort are needed to formulate 2-tier MCQs, and teachers need to be given training in formulating this kind of questions. To what extent this will impact on academic competencies of students is not clear but obviously they need to have an enhanced level of functional understanding of a topic. Lack or absence of testing instruments modeled on 2-tier MCQs in the various subject areas is another drawback. All these can come in the way of the wider acceptance of 2-tier MCQs in the examination system. With the pervasiveness of ICT in Singapore – high Internet penetration rates, high home PC ownership rates, broadband links in all schools and general cyber saviness of the younger generation, the author feels that there is some scope for reducing teachers’ workload by converting certain lessons into e-learning. This will allow students to take greater control of their learning needs and further immerse them in a media that they are comfortable with. Needless to say, this would have implications in the overall scheme of assessment. In more recent years, the term action research has appeared in the lexicon of teachers. Considerable encouragement is now given to teachers in schools in Singapore to engage in action research as a way of reflecting on their teaching as well as improving the effectiveness of various school programs. The reflective stance that action research provides as well as the critical dimension that it affords to re-look at school practices and programmes has implications on educational evaluation at the school level. In fact, Banfield and CayagoGicain (2006) argue that action research is a ‘powerful and empowering approach to educational evaluation” as it “has wide practical applicability from large scale evaluations that span the entire organization, to more modest evaluation of single programmes”. The infusion of action research in schools in Singapore is likely to lead to school-based educational evaluation contributing further to improved outcomes in the education process.

CONCLUSION Educational evaluation is a powerful approach to appraise various processes in the education system. It can provide important information to stakeholders. In this chapter, we have seen that the use of primary school leaving examinations conducted by an independent agency that is not connected to schools or the Ministry of Education provides the best form of educational evaluation of how schools prepare their students. In the case of leaving level examinations at the secondary and pre-univeristy levels, again the use of an international examinations syndicate in conjunction with an independent local agency provides valuable information on how effective schools are in preparing their students. Additionally, in the case of primary and secondary levels, the participation of students in the TIMSS examinations provides supporting evidence that the curricula in the science and mathematics in Singapore are internationally competitive. The high standing of Singapore students in science and mathematics in the TIMSS examinations in 1995, 1999 and 2003 shows that students are prepared to international standards. Moreover, the system of banding (previously ranking) of schools ensures that they are cognizant of the concepts of accountability and stakeholder value.

34

R. Subramaniam

Singapore has consistently sought international benchmarks for its education system as it provides the best assurance that the curricula and assessment are not only current but also follow international norms

REFERENCES Banfield, G. and Cayago-Gicain, S. (2006). Qualitative approaches to educational evaluation: A regional conference workshop, International Journal of Education, 7(4), 510-513. Caleon, I and Subramaniam, R. (2005). The impact of a cryogenics-based enrichment programme on attitude towards science and learning of science concepts, International Journal of Science Education, 27(6), 679-704. Cheah, Y. M. (1998). The examination culture and its impact on literary innovation: The case of Singapore, Language and Learning, 12, 192-209. Johnson, S. (1999). International Association for the evaluation of educational achievement: Science assessment in developing countries, Assessment in Education, 6(1), 57-73. Lim, E. P. Y. and Tan, A. (1999). Educational assessment in Singapore, Assessment in Education, 6(3), 391-404. Martin, M. O., Mullis, I. V. S., Beaton, A. E., Gonzales, E. J., Kelly, D. C. and Smith, T. A. (1999). Science achievement in the primary school years: IEA’s third international mathematics and science study (TIMSS). Chestnut Hill, MA: Boston College. Mullis, I. V. S., Martin, M. O., Beaton, A. E., Gonzales, E. J., Smith, T. A. and Kelly, D. C (1997). Mathematics Achievement in the Primary School Years: IEA’s third international mathematics and science study (TIMSS). Chestnut Hill, MA: Boston College. Poh, S. H. (1999). Assessment issues in Singapore, Educational Evaluation: Issues and Practice, 18, 31-32. Purves, A. C. (1991). Brief history of International Evaluation Association. In W. A. Hayes (Ed.), Activities Institutions and People, IEA Guidebook 1991, pp 34-48. The Hague: IEA. Tan, J. and Gopinathan, S. (2000). Education reform in Singapore: Towards greater creativity and innovation, NIRA Review, 7, 5-10. Tan, W. H. L. and Subramaniam, R. (1998). Developing countries need to popularise science, New Scientist, 2139, 52. Tan, W. H. L. and Subramaniam, R. (2002). Science and the student entrepreneur. Science, 298, 1556. Toh K. A., Riley J. P., Lourdusamy A., Subramaniam R. (2006). School and science achievement in Japan and Singapore: A tale of two cities. Educational Research for Policy and Practice, 5, 1–13. Treagust, D. F. (1988). Development and use of diagnostic tests to evaluate students’ misconceptions in science, International Journal of Science Education, 10, 159-170.

In: Educational Evaluation in East Asia Editor: Samuel S. Peng and John Ch-Kin Lee

ISBN 978-1-60692-887-5 © 2009 Nova Science Publishers, Inc.

Chapter 3

EDUCATIONAL EVALUATION IN SOUTH KOREA: CURRENT ISSUES AND FUTURE PROSPECTS Sun-Geun Baek Department of Education, Seoul National University, Republic of Korea

ABSTRACT This paper describes five significant features of educational evaluation in South Korea. The first feature is ‘the School Activities Records (SAR)’, which is designed to evaluate primary, middle, and high school students based on academic achievements, as well as the development of social behaviors. The second feature is ‘the National Assessment of Educational Achievement (NAEA)’, which is to control the quality of the nationwide educational system by providing information on the overall educational achievement and where improvement might be needed. The third feature is ‘the College Scholastic Ability Test (CSAT)’, which is administered at the national level to screen the eligible candidates for higher education. The fourth feature is ‘the Teacher Competence Development Evaluation (TCDE)’, which is designed to evaluate primary, middle, and high school teachers based on their teaching skills and management abilities. The final feature is ‘the Teacher Education Institute Accreditation System (TEIAS)’, which is designed to evaluate and accredit each teacher education program and institute. In conclusion, current issues and future prospects of educational evaluation in South Korea will be discussed.

Keywords: SAR, NAEA, CSAT, TCDE, TEIAS.

I. INTRODUCTION Located in the eastern part of the Asian Continent, South Korea is a democratic state and a constitutional republic that holds a five-thousand year history. South Korea has 16 administrative units: The metropolis of Seoul, six metropolitan cities, and nine provinces. South Korea has a population of around 48 million.

36

Sun-Geun Baek Table 1. The Number of Schools, Teachers and Students in South Korea (2006) Levels

Schools

Teachers

Students

Kindergarten

8,290

32,096

545,812

Primary School

5,733

163,645

3,925,043

Middle School

2,999

106,919

2,075,311

High School

2,144

117,933

1,775,857

Junior College

152

11,857

817,994

College and University

201

54,967

2,368,169

Others

274

8,928

406,835

19,793

496,345

11,915,021

Grand Total

More than 10 million people live in Seoul, which is the country’s capital. For your reference, the number of schools, teachers and students in South Korea are indicated in Table 1. In addition, the enrollment rates for primary, middle, and high schools have reached about 100 percent since 1994. The ratio of high school students, who go on to a higher stage of education, reached 82.8 percent in 2007, which was the one of the highest ratios in the world (KEDI, 2007). In this paper, five significant features of educational evaluation in South Korea will be described. In addition, current issues and future prospects of educational evaluation in South Korea will be discussed at the end.

II. SCHOOL ACTIVITIES RECORDS (SAR) The School Activities Records (SAR) is designed to evaluate primary, middle, and high school students based on academic achievements, as well as the development of social behaviors. The Presidential Commission on Educational Reform announced on May 31, 1995 that school teachers should construct his/her students’ SAR. The purpose of this evaluation system was to get not only summative information, but also diagnostic and formative information of students’ academic achievements, as well as the development of social behaviors. Although there was a trivial revision in 2005, both the content and the format of the SAR are almost the same as the old version. Eleven main categories’ names and their contents in the SAR are as follows (MOE and HRD, 2007a): (1) Personal Information: Student's name, gender, identification number, address, parent names and their occupation, and special comments. (2) Educational Background: Student's schooling (entrance, transfer, and graduation) records, and special comments.

Educational Evaluation in South Korea

37

(3) School Attendance: Number of instructional days per year, number of absent days, the reasons for absences (sickness, accident, and others), number of late days, the reasons for lateness (sickness, accident, and others), number of absent classes, the reasons for absences (sickness, accident, and others), and special comments. (4) Awards: Prize name, level of prize, award date, and provider. (5) Certificates: Type of certificate, registration number of certificate, registered date, and provider. (6) Career Guidance: Student's special talents and skills, interests, student's prospective occupation, parent expectations, and special comments. (7) School Optional Creative Courses: Areas of specific activities for optional creative courses and comments. (8) Extracurricular Activities: Class activities, school activities, club activities, group activities, and general comments for each sub-category. (9) Volunteer Work and Special Experiences: Contents, frequencies, places, and periods of time for both volunteer work and special experiences. (10) Academic Achievement: − − −

For primary school students: Special comments for each subject per year without grade and rank, and special talents and interests. For middle school students: Raw score, five scale grade and rank for each subject per semester, and special talents and interests. For high school students: Raw score and stanine score for each subject per semester, and special talents and interests.

(11) Behavioral Characteristics and Comprehensive Comments: General comments on behavioral characteristics and overall comments per year. Each student's information on the SAR has been used to improve individual teachinglearning processes. In addition, high schools have used each student’s information to select their new students among middle school graduates. Colleges and universities have also used each student’s information in SAR to select their new students among high school graduates. Recently, the portion of each student’s information within total entrance examination scores for higher education is gradually increased.

III. NATIONAL ASSESSMENT OF EDUCATIONAL ACHIEVEMENT (NAEA) The National Assessment of Educational Achievement (NAEA) is to control the quality of the nationwide educational system by providing information on the overall educational achievement and where improvements might be needed. The NAEA is administered generally once in a year at the national level. The purposes of the NAEA are as follows (KICE, 2007): (1) To diagnose the educational achievements of elementary, middle, and high school students and to check systematically the trends of their achievements.

38

Sun-Geun Baek (2) To provide basic reference data for improving the national curriculum by analyzing the students' achievements of the goals of the national curriculum and checking the problems of the national curriculum implementation at the school and classroom levels. (3) To improve teaching and learning methods and to produce data to set up a learning encouragement policy by analyzing the test items and the relationship between the students’ achievements and background variables. (4) To improve student assessment methods in the schools by developing and utilizing new and appropriate assessment tools. (5) To explore new research designs and methods, in order to attain the purposes of the study of the NAEA.

Until 1988, several educational research centers, such as the Korean Educational Development Institute (KEDI) or the Central Educational Research Center had intermittent conduct with the NAEA. However, the National Board of Educational Evaluation had regular conduct with the NAEA from 1989 to 1997. Since 1998, the Korea Institute of Curriculum and Evaluation (KICE), which was established as a government funded educational research and testing center on January 1 in 1998, has had annual conduct with the NAEA. The general procedures for the NAEA are as follows (KICE, 2007): (1) (2) (3) (4) (5)

KICE sets up the basic policy for the NAEA. KICE works out the basic action plan. KICE develops the test items with primary, middle, and high school teachers. KICE develops survey questionnaire for students, teachers, and principals. Municipal and provincial educational authorities administer these tests to their students. (6) KICE analyzes about 3% ~ 5%-sampled data from total populations, and reports the test results.

Table 2 shows 2007 NAEA's research subjects, assessment areas, item types, and others, such as the listening comprehension test or questionnaires. Specifically, both Korean Language and English tests included listening comprehension through the Educational Broadcasting Systems (EBS), which was a nationwide broadcasting system in South Korea. In addition, KICE analyzed about 3% ~ 5%-sampled data from the total population and reported the results to the public. Additionally, almost all municipal and provincial educational authorities have administered the NAEA tests to their all students and analyzed their whole data according to their own purposes.

Educational Evaluation in South Korea

39

Table 2. Structure of 2007 NAEA Research Subjects

6th Grade

Assessment Areas Korean Social Studies Mathematics Sciences English

9th Grade

Korean Social Studies Mathematics Sciences English

10th Grade

Korean Social Studies Mathematics Sciences English

Item Types and Others

- Multiple-choice items and constructed response items for each s ubject were developed for each subject. - Constructed response items took 20-40% of the total score depen ding on the features of the subject. - Listening comprehension test was included in Korean and Englis h. - Questionnaires for students, teachers, and principals were develo ped to investigate the relationship between the background variabl es and the students’ academic achievements. - 2007 NAEA implemented on October 16-17, 2007.

IV. COLLEGE SCHOLASTIC ABILITY TEST (CSAT) The College Scholastic Ability Test (CSAT) is administered at the national level to screen eligible candidates for higher education. CSAT is administered once in a year at the national level. The purposes of the CSAT are as follows (KICE, 2007): (1) To screen the eligible candidates for higher education. (2) To contribute to the improvement of high school education. (3) To provide valid, reliable, and objective data for selecting new students for colleges and universities. In addition, the general procedures for CSAT are as follows (KICE, 2007): (1) The Ministry of Education and Human Resources Development (MOE and HRD) has set up the basic policy for CSAT. (2) KICE works out the basic action plan. (3) KICE develops the test items. (4) Municipal and provincial educational authorities administer these tests to their students. (5) KICE analyzes and reports the test results. About 580,000 applicants (12th grade students, high school graduates, and those having equivalent certificates) took the CSAT on November 15, 2007. The stanine score for each content area appeared exclusively on the student report card of CSAT in 2007. For your reference, the stanine score is a type of standardized score with nine levels (see Table 3).

40

Sun-Geun Baek

However, the stanine score, percentile score, and one standardized score within two types (one type’s mean equals 100 with standard deviation equals 20 or another type’s mean equals 50 with standard deviation equals 10) will be appeared on the student report card of CSAT since 2008. Table 3. Structure of Stanine Score Stanine Scores

1

2

3

4

5

6

7

8

9

%

4

7

12

17

20

17

12

7

4

100-97

96-90

89-78

77-61

60-41

40-24

23-12

11-5

4-1

Percentile

Table 4. Structure of 2007 CSAT Content Areas Korean Mathematics A or B English Social Studies (4 out of 11 areas) or Sciences (4 out of 8 areas) or Vocational Education (3 out of 17 areas) Foreign Languages or Chinese Characters and Classics (1 out of 8 areas)

Times (Minutes)

Maximum Scores

Items

80

100

50

100

100

30

70

100

50

30 for each area

50 for each area

20 for each area

40

50

30

Table 4 shows the 2007 CSAT's content areas, periods of time for tests, maximum raw scores, the number of items, and item types. The CSAT score is one of the most important data for getting admission into colleges or universities. For your reference, higher educational institutes in South Korea generally use 'the total entrance examination scores' to select new students among high school graduates or those who have equivalent certificates. The total entrance examination scores are consisted differently among higher educational institutes or even are consisted differently among departments, within the same institute. Some institutes (or departments) use only (i) CSAT scores, whereas some institutes use (i) CSAT scores and (ii) the information on SAR, and others use (i) CSAT scores, (ii) the information on SAR, and (iii) institutes' own entrance examination scores such as interview scores, essay writing scores, recommendation letters' scores, awards, and so on.

Educational Evaluation in South Korea

41

In order to present more substantial information about the college and university admission system in South Korea, Table 5 shows the case of the Seoul National University, which is the best university in South Korea. Please note there are a few differences in the contents of Table 5 depending on departments or colleges within Seoul National University. Table 5. Seoul National University’s Admission System in 2008 Admission Type

Regular

Occasional

Special

No. of Students (Total: 3,277)

1,401 (42.8%)

1,761 (53.7%) Type A: 831 (24.8%) Type B: 930 (27.7%)

115 (3.5%) Type 1: 95 (2.9%) Type 2: 20 (0.6%)

Qualification

-

Higher than level 2 (top 11%) in CSAT at least 2 out of 4 areas.

- The residents of rural or fishing villages, or disabled students. - Higher than level 2 (top 11%) in CSAT at least 2 out of 4 areas.

Materials

CSAT, SAR, essay writing, interview, etc.

SAR, CSAT, interview, documents (recommendation letter, awards, etc.)

The same as regular type.

Selection Step I (selecting 1.5 - 3 times of the quorum)

Selection Step II (final stage)

SAR: 100% CSAT: 100% Documents: 100%

SAR: 50% Essay Writing: 30% Interview: 20%

SAR: 80% Documents: 10% Interview: 10% 1) Documents: 50% Interview: 30% Essay Writing: 20% 2) Documents: 50% Interview: 50%

CSAT: 50% SAR: 30% Interview: 20% CSAT: 40% SAR: 20% Interview: 40%

42

Sun-Geun Baek

V. TEACHER COMPETENCE DEVELOPMENT EVALUATION (TCDE) The Teacher Competence Development Evaluation (TCDE), which was designed to evaluate primary, middle, and high school teachers based on their teaching skills and management abilities. The main purpose of the TCDE is for the development of primary and secondary teacher’s competence in order to increase the competitive power of public education. It will be fully implemented at all schools by 2008 (MOE and HRD, 2007b). According to the results of the first nationwide TCDE trial operation at 67 primary and secondary schools in 2006, 74 percent of the teachers thought favorably of the TCDE system. It worked as a means of self-reflection, helping them to analyze the strengths and weaknesses of each individual teacher, and to identify the specific needs of both students and parents. Students and parents also positively responded to the TCDE system. They thought that it helped increase trust for teachers and ensured that student opinions were adequately reflected in class and school operations. The government implemented the second nationwide TCED trial operation in 506 primary and secondary schools on March, 2007. Following the ministry’s master plan, at the end of the trial period, principals, vice principals, and teachers in 237 primary schools, 189 middle schools, and 80 high schools will have been evaluated by other teachers, vice principles, and principals based on their teaching skills and management abilities. Students and parents will also be surveyed to measure their satisfaction with teachers’ performance, as well as the principal’s competence.

VI. TEACHER EDUCATION INSTITUTE ACCREDITATION SYSTEM (TEIAS) The Teacher Education Institute Accreditation System (TEIAS), which was designed to evaluate and accredit each teacher education program and institute. TEIAS can be defined as an official verification system to see whether or not each teacher education program and institute met the prescribed standards for quality assurance and quality improvement. The main purpose of TEIAS is the improvement of teacher education programs and institutes, in order to bring up highly professionalized primary and secondary teachers. It will be fully implemented by 2009 (MOE and HRD, 2007c). There are two serious problems within the teacher education system. The first is the wide gap between the supply and the demand of initial teachers, and the other is the inadequacy of quality management for teacher education programs and institutes. Even though there are recently very low employment rates, there are still so many teacher education programs and institutes, as well as so many people who have teacher qualifications. For example, there are total 392 secondary teacher education programs and institutes, but less than the 20% nationwide employment rates of 2005 (Baek, 2007). Additionally, there is almost no quality management system for teacher education in South Korea. In order to solve these problems, the accreditation system of teacher education institutes should be implemented as soon as possible.

Educational Evaluation in South Korea

43

VII. CURRENT ISSUES AND FUTURE PROSPECTS Since 1998, the Korean society has been characterized with democratization and concerns for the public welfare, ICT (information communication technology)-oriented industrializations, openness and globalization. There have been many educational reforms to reduce private tutoring expenses, enhance the quality of human resources, build infrastructures for lifelong learning, and expand investment for the disadvantage. Educational reforms in South Korea will be an ongoing effort, striving to improve every sector of education, including educational evaluation. The current issues of educational evaluation in South Korea are almost the same as already described above, such as SAR, NAEA, CSAT, TCDE, and TEIAS. There are many ongoing conflicts among stakeholders with those educational evaluation policies. For example, teachers unions are strongly against the implementation of TCDE and many teacher education programs and institutes are against the implementation of TEIAS. Many teachers and professors still dislike any change or any reform within the educational evaluation system. Some people also dislike the implementation of a competition system, in order to improve the competence of teachers, as well as the quality of education. However, it is generally impossible to go against the current main stream of the 21st century, where one finds limitless competition age mixed with globalization. Educational evaluation has been used as inquiry and judgment methods, in order to optimize the evaluation object, in relation to its intended purposes or to help stakeholders determine whether the evaluation object is worthy of adoption, continuation, or expansion (Fitzpatrick, Sanders and Worthen, 2004). Therefore, educational evaluation will be used continually as main tools for the development and improvement of primary, secondary, and higher education in South Korea. South Koreans hope that every individual will have an equal and easy access to high quality education at any time and any place. There is an old proverb in South Korea: A father who wants to make plans for the next 10 years, plants a tree for his son, while a father who makes plans for the next 100 years, invests in the education of his son. We believe prospects for our education in the 21st century are very bright, because we not only acknowledge the importance of education, but are also making substantial investments to build a powerful state in education.

REFERENCES Baek, S. G. (2007). Political suggestions for the successful implementation of the accreditation system on teacher education institutes. Journal of Educational Evaluation, 20(2), 25-49. Fitzpatrick, J. L., Sanders, J. R., and Worthen, B. R. (2004). Program evaluation: Alternative approaches and practical guidelines (3rd ed.). NY: Pearson Education, Inc. Korea Institute of Curriculum and Evaluation. (2007). The implementing plan for the college scholastic ability test in 2007. KICE homepage: http://www.kice.re.kr/. Korean Educational Development Institute. (2007). 2006 educational statistics in South Korea. KEDI homepage: http://www.kedi.re.kr/.

44

Sun-Geun Baek

Ministry of Education and Human Resources Development. (2007a). New teacher’s manual for school activities records. MOE and HRD homepage: http://www.moe.go.kr/. Ministry of Education and Human Resources Development. (2007b). New implementation plan for the teacher competence development evaluation. MOE and HRD homepage: http://www.moe.go.kr/. Ministry of Education and Human Resources Development. (2007c). New implementation plan for the teacher education institute accreditation system. MOE and HRD homepage: http://www.moe.go.kr/. Seoul National University. (2007). The selecting plan for new coming students. SNU homepage: http://www.snu.ac.kr/.

In: Educational Evaluation in East Asia Editor: Samuel S. Peng and John Ch-Kin Lee

ISBN 978-1-60692-887-5 © 2009 Nova Science Publishers, Inc.

Chapter 4

DEVELOPMENT OF EDUCATIONAL EVALUATION AND CHALLENGES OF PRACTICING EDUCATIONAL EVALUATION IN THAILAND Sirichai Kanjanawasee Center on Educational Testing and Evaluation for Educational and Professional Development, Faculty of Education, Chulalongkorn University, Thailand

ABSTRACT This paper presents a brief overview of current educational evaluation issues and challenges in Thailand. Thai basic education is divided into 4 levels: 1) Grades 1-3 2) Grades 4-6 3) Grades 7-9 ,and 4) Grades 10-12. At the final year of each level, students must pass 4 types of assessments correspondence to each institution’s standards: 1) 8 subject areas assessment, 2) reading, thinking and writing skills’ assessment, 3) desired learner attributes assessment, and 4) assessment of the participation in a developmental activities program. Therefore, we need to develop phototypes of effective integrated measurement tools and a procedure manual for a systematic assessment. The new university admission system was designed and implemented in 2006, using integrated multiple indicators to evaluate student achievement and performance. These included: 1) The overall cumulative grade point average (GPAX) for Grades 10-12, 2) the grade point average (GPA) in Grades 10-12 in 3 to 5 of the 8 subject groups of the core curriculum; and 3) results of the Ordinary National Educational Test (O-NET) and/or the Advanced National Educational Test (A-NET). These designs are facing new challenges regarding the practice of evaluation techniques that are meaningful and fair for all students.

Keywords: educational assessments, quality assurance, basic education, admission system.

46

Sirichai Kanjanawasee

INTRODUCTION This paper provides general information on Thailand’s education system relating to the basic education, professional standards for teachers and higher education admission system. It presents an overview on related aspects of academic achievement assessments in basic education and the evaluation of student achievement and performance for entering a new higher education admission system. It also addresses the challenges of the evaluation role to improve educational quality system and provides some policy recommendations. At present, the framework of education reform in Thailand is based on the 1999 National Education Act which provides principles and challenging guidelines for the provision and development of Thai education in order to prepare all Thai people to live in a learning society. In this regard, Thai people shall attain full development in terms of physical and spiritual health, intellect, morality and integrity, as well as a desirable way of life that focuses on living in harmony with other people. The educational system is classified into 3 types: formal education, non-formal education; and informal education. The formal education services are provided to these inside the school system, through both public and private bodies. It is divided into 2 levels: basic education and higher education. The basic education covers pre-primary, 6 years of primary education, 3 years of lower secondary education, and 3 years of upper secondary education. Higher education is provided at universities, institutions, colleges or other types of institutions. It is divided into two levels: associate degree and degree levels. The non-formal education services are provided by both public and private bodies to those outside the school system. Informal education enables learners to learn by themselves according to their interests, potential, readiness and the opportunities available that are organized by individuals, families, communities, authorities or other learning networks.

EDUCATIONAL STANDARDS AND QUALITY ASSURANCE As specified in the 1999 National Education Act, educational reform in Thailand must establish education standards which are the specification of desired educational quality for benchmarking, promoting, monitoring and quality assurance. There are currently three types of standards: national education standards, and standards for internal quality assurance and for external quality assessment.

National Education Standards The Office of the Education Council (OEC) is responsible for proposing national education standards. Consequently, sets of standards were formulated by the Office in cooperation with the offices responsible for basic, vocational, and higher education as well as the Office for National Education Standards and Quality Assessment. With approval from the Council of Ministers, on October 26, 2004, agencies providing education at all levels are expected to abide by the national education standards, which are comprised of three categories:

Development of Educational Evaluation and Challenges…

47

(I) Desirable characteristics of the Thai people, as both citizens of the country and members of the world community, consist of 5 indicators: 1) 2) 3) 4) 5)

sound physical and mental health; required knowledge and skills sufficient for leading a meaningful life and social development; skills in learning and self-adjustment; social skills; righteousness, public-mindedness, and consciousness of their citizenship of Thailand and the world.

(II) Guidelines for educational provision consist of 3 indicators: 1) 2) 3)

development of a diversified curricula and ambiance enabling learners to develop themselves in line with their natural inclinations and to the best of their potential; and systematic and effective development of administrators, teachers, faculty staff and education personnel; practice of school-based management.

(III) Guidelines for creating a learning society/knowledge society consist of 3 indicators: 1)

2) 3)

provision of academic services and establishment of cooperation between educational institutions and community, so as to transform educational institutions into a learning society/knowledge society; research and study, promotion of and support for learning sources and mechanisms; generation and management of knowledge for the benefit of all levels and components of the society.

The national education standards serve as the basis for setting assessment standards of internal and external quality assurance mechanisms.

Educational Standards for Internal Quality Assurance Currently, all agencies concerned have made progress in the development of relevant educational standards for internal quality assurance as a guideline in the provision of education by educational institutions under their supervision. For example, learning standards for basic education have been formulated to respond to the 2001 curriculum. In addition, the Committee for Development of a System to Evaluate Higher Education Quality has appointed a sub-Committee to develop and set national standards for higher education.

48

Sirichai Kanjanawasee

Educational Standards for External Quality Assessment The Office for National Education Standards and Quality Assessment (ONESQA) conducts external quality assessment at the levels of basic education and higher education (lower-than-degree and degree levels) in line with relevant educational standards that focus on assessment of educational institutions in the following respects: (1) educational achievement (output/outcome); (2) input/process; and (3) efficiency in administration and leadership.

1) Educational Standards for External Quality Assessment at Basic Education Level Educational standards for external quality assessment of basic education institutions were approved by the Council of Ministers, in January 2000. These standards, which are composed of 14 standards and 53 indicators, can be classified into 3 groups as follows: 1.1) Standards of Learners, consisting of 7 standards with 22 indicators, aim at physical, spiritual, intellectual and social development; 1.2) Standards of process, consisting of 3 standards with 21 indicators, focus on administrative and teaching-learning processes; and 1.3) Standards of Imput specify the characteristics or readiness of administrators, teachers and the curriculum. They are composed of 4 standards, with 10 indicators.

2) Educational Standards for External Quality Assessment at Higher Education Level Educational standards for external quality assessment of higher education institutions have been stipulated. These include: (1) Graduates Quality; (2) Teaching and Learning; (3) Academic Supports; (4) Research and Innovation; (5) Academic Services; (6) Preservation of Arts and Cultures; (7) Management and Administration; and (8) Internal Quality Assurance System.

DEVELOPMENT OF A QUALITY ASSURANCE SYSTEM A quality assurance system is comprised of quality control, quality audit and quality assurance. Quality assurance can be divided into 2 main factors: internal quality assurance; and external quality assessment. To enable linkage and consistency between internal quality assurance and external quality external quality assessment, the system criteria and methods for internal quality assurance as well as for external quality assessment have been formulated.

Internal Quality Assurance Internal Quality Assurance, primarily serves as a basis for external assessment. All educational institutions are required to implement an internal quality assurance system comprised of self-control, self-audit, and self-assessment toward their educational standards. The institutions must disseminate their Self-Study Report (SSR) each year.

Development of Educational Evaluation and Challenges…

49

The continuous process of internal quality assurance of all educational institutions consists of planning, assessment and improvement of their performance. Each educational institution is required to prepare its own educational development plan in line with the objectives and principles of the National Education Act, the National Education Standards as well as the aims, philosophy, and charter of the institution; clearly determine the time frame of implementation; continuously monitor and assess its own performance; and finally use the assessment results to improve and develop a high quality of education. Each educational institution is required to complete its internal quality assurance report before the beginning of the following academic year. The assessment results of educational quality as well as guidelines or programs for improvement and development of educational quality must be presented during the following year. As for the report, it must be submitted to the parent organization, the agencies concerned, the ONESQA and be made available to the public.

External Quality Assessment The Office for National Education Standards and Quality Assessment (ONESQA), established as a public organization in November 2000 to oversee external quality assessment. The ONESQA, utilizing qualified external assessors under contract to assess both basic and higher education institutions following standards relating to educational achievement (output/outcome); input/processes; and efficiency in administration and leadership. The external quality assessment of all institutions is conducted at least once every five years. The assessment outcomes will be submitted to the agencies concerned and made available to the general public. Besides the education standards, the National Education Act also stipulated that teachers’ Council of Thailand shall be regarded as the Teachers and Educational Personnel Council. It is incorporated with the principal objective to determine professional standards, issue and revoke licenses, supervise and monitor the compliance with the professional standards and code of ethics, including professional development, so that educational professional practitioners, namely, those who are licensed to practice the teaching profession, educational institution administrators, educational administrators and other educational personnel (educational supervisors), shall have knowledge and understanding in the practice of educational profession which is the licensed profession. The Secretariat Office of the Teachers’ Council of Thailand therefore established these Educational Professional Standards. The Educational Professional Standards, which consist of standards of professional knowledge and experience, standards of performance and standards of conduct, have been announced as the Teachers Council of Thailand Regulation on Professional Standards and Ethics in 2005. In the educational professional practice, those who enter such profession shall meet the specified standards of professional knowledge and experience, so as to be eligible to obtain the license to practice the profession.

50

Sirichai Kanjanawasee

PROFESSIONAL STANDARDS FOR TEACHERS Standards of Professional Knowledge and Experience •

Standards of Teachers’ Knowledge

Have minimum qualifications with Bachelor’s degree in education or the equivalent or other degrees as accredited by the Teachers’ Council of Thailand, with knowledge in the following areas: 1) 2) 3) 4) 5) 6) 7) 8) 9)

Language and technology for teachers. Curriculum development. Learning management. Psychology for teachers. Educational measurement and evaluation. Classroom management. Educational research. Educational innovation and information technology. Teachership.

•

Standards of Experience for Teachers

Have completed their teaching operations at educational institutions under the educational degree curriculum for a minimum of one year and passed the criteria for evaluation of the teaching operations in accordance with the rules, procedures and conditions as set out by the Teachers’ Council of Thailand Board as follows: 1) Training in professional practice during study. 2) Teaching operations in educational institutions on specific subjects.

Standards of Performance 1) Regularly practice academic activities relating to the development of the teaching profession. 2) Make decisions to practice various activities taking into account consequences for the learners. 3) Be committed to developing learners to reach their full potential. 4) Develop teaching plans for effective implementation. 5) Regularly develop instructional media to be effective. 6) Organize instructional activities focusing on permanent results for learners. 7) Systematically report on results of learners’ quality development. 8) Conduct themselves as good role models for learners. 9) Constructively cooperate with others at the educational institution. 10) Constructively cooperate with others in the community.

Development of Educational Evaluation and Challenges…

51

11) Seek and use information for development. 12) Create opportunities for learners to learn under all circumstances.

Standards of Conduct •

Personal ethics 1)

•

Professional ethics 2)

•

4)

5) 6)

7)

Education professional practitioners shall care for and be merciful to, pay attention to, assist and encourage their students and clients on an equal treatment basis, in accordance with their roles and duties. Education professional practitioners shall encourage their students and clients to achieve learning, skills, good and proper conduct, in accordance with their roles and duties, to their full capability and generosity. Education professional practitioners shall conduct themselves as a good role model in terms of physical conduct, speech and mental conduct. Education professional practitioners shall not act in any manner, which is against the physical, intellectual, mental, emotional and social development of their students and clients. Educational professional practitioners shall provide service honestly and equally without asking for, accepting or acquiring any interests, which would be considered abuses of their authority.

Collegial ethics 8)

•

Education professional practitioners shall have love, faith, integrity and responsibility for the profession and act as good members of professional organizations.

Client centered ethics 3)

•

Education professional practitioners shall have self-discipline and improve their professional practice, personality and vision to keep up with the academic, economic social and political development.

Education professional practitioners shall provide assistance and support to one another constructively by adhering to the morality and for crative synergy among themselves.

Societal ethics 9)

Education professional practitioners shall conduct themselves as leaders in economic, social, religious, art and cultural, intellectual and environmental

52

Sirichai Kanjanawasee conservation and development as well as preserve the public interests and adhere to the democratic regime of government with the King as Head of the State.

According to the teachers and Educational Personnel Act of B.E.2547 (2004), there is a six-scale teacher classification framework based on academic status for the new salary structure. The classification ranges from assistant teachers, teachers, experienced teachers, higher experienced teachers, expert teachers, to specialized teachers. The Teacher Civil Service Commission sets up the criteria for performance-based teacher evaluation system including three areas: (1) The discipline, morality and ethical conduct, (2) Performance quality/core and specific competencies, and (3) The performance results. The evaluation system tries to promote teachers to higher academic status. Teachers who get promoted to a higher level will get extra monthly allowance. In addition, all public school teachers will be evaluated their performance every six months for a salary increase.

BASIC EDUCATION In accordance with the focus on educational reform, many steps - from national policy to institutional level – are being taken to develop the basic education curriculum and the teaching-learning process, to improve the assessment and evaluation of achievement, and to expand access to formal education.

Present Status of Basic Education In 2002, in accordance with the National Education Act, 12 years of free basic education was made available to students throughout the country, for the first time. The government later increased that number to 14 years to include two years of pre-primary education. Basic education covers pre-primary instruction, 6 years of primary, 3 years of lower secondary, and 3 years of upper secondary education, with the 9 years of primary and lower secondary levels considered compulsory.

Basic Education Curriculum A 2001 update of the Basic Education Curriculum called for “core requirements” as prescribed by the Basic Education Commission, and “specific content”, is directly related to local needs and contexts, to be developed by the educational institutions themselves.

(1) Basic Education Curriculum at the National Level The 2001 Curriculum for Basic Education covers 12 years of basic education (Grades 112), and is divided into 4 three-year stages, consisting of 1,000 – 2,000 hours per year. In this curriculum, the knowledge and skills specified in Section 23 of the National Education Act have been grouped into 8 subject areas: Thai Language, Mathematics, Science, Social Studies, Religion and Culture, Health and Physical Education, Art, Career- and

Development of Educational Evaluation and Challenges…

53

Technology- Related Education, and Foreign Language. Additionally, activities that focus on responding to the learner’s specific interests are also included. In 2005, following nationwide training programmes for administrators, supervisors, teachers, and personnel in related departments, the 2001 Curriculum for Basic Education became fully implemented for all grades.

(2) Basic Education Curriculum at the Institutional Level As stipulated in the National Education Act, educational institutions are required to develop curriculum content relating to the needs of the community and society, and include local wisdom and desirable attributes for members of the family, community, society and the nation. The proportion of core curriculum to local content developed by the institution should be approximately 70:30 flexibly applied in compliance with the nature of each subject.

ACADEMIC ACHIEVEMENT ASSESSMENTS IN BASIC EDUCATION Basic Education in Thailand is divided into 4 levels: Level 1. Primary 1 – Primary 3. (Grade 1 – Grade 3) Level 2. Primary 4 – Primary 6. (Grade 4 – Grade 6) Level 3. Secondary 1 – Secondary 3. (Grade 7 – Grade 9) Level 4. Secondary 4 – Secondary 6. (Grade 10 – Grade 12)

Educational Assessments at the Institutional Level At each level, Grades 3, 6, 9, and 12, students must pass four types of assessments correspondence to each institution’s standards, they are: 1) Eight subjects areas’ assessment, 2) Reading, critical thinking and writing skills’ assessment, 3) Desired learner attributes’ assessment, and 4) Assessment of participation in developmental activities program.

Educational Tests at the National Level Since 2001, students studying in Grades 3, 6, and 9, are obliged to take national educational tests administered by the Office of the Basic Education Commission (OBEC) in 2 to 5 subjects. Beginning in 2005, Grade 12 students wishing to pursue higher education must take the tests administered by the National Institute of Educational Testing Service (NIETS) as shown in Table 1. The national tests for Grade 12 students are of 2 types, the Ordinary National Educational Tests (O-NET) and the Advanced National Educational Test (A-NET). Both O-NET and ANET require tests in 5 subjects; however, A-NET focuses more on critical thinking and analytical skills. Results from O-NET and A-NET are used as one of the factors in the revised higher education admission system.

54

Sirichai Kanjanawasee Table 1. Subjects Required in National Educational Tests Levels of Education

Mathematics

Grade 3 Grade 6 Grade 9 Grade 12

x x x x

Subjects Required Thai Science Language x x x x x x x

English x x x

Social Sciences

x x

Organizers OBEC OBEC OBEC NIETS

Source: Bureau of Educational Testing, OBEC and The National Institute of Educational Testing Service.

Current Issues and Policy Recommendations Educational assessments of basic education are facing multi-level problems as follows.

At the National Level 1. Quality Assurance System Schools are carrying out their internal quality assurance not for self-development to improve student learning but to avoid being compared with the external standards and criteria set by ONESQA. The National education policy should be to apply quality assurance as quality management for linking external quality assurance with internal quality assurance to ensure that schools are fulfilling their own purposes and meeting their own standards, as well as meeting predetermined minimum criteria or national quality standards. 2. Teacher Evaluation Based on performance-based teacher evaluation systems, applying teachers are evaluated by authorities from schools and local educational service areas. Many teachers are passed and get promoted to a higher academic status, whereas students’ achievement remains the same, and in some schools students’ achievement becomes worse. The Teacher Civil Service Commission must make clear the goals and roles of the teacher evaluation. The goals must be to elevate the quality of teaching and learning, while the roles must involve both formative and summative purposes. Additionally, the evaluation criteria needs to be revised to focus on classroom practices, student development, and learning outcomes. At the Institutional level Three problems with assessments of students in schools are obvious. 1. Teachers assess students’ learning in eight subject areas, by focusing on memorizing knowledge. 2. There are no well-constructed tools for assessing students’ ability in reading, thinking and writing.

Development of Educational Evaluation and Challenges…

55

3. The assessments of desired learners’ attributes are based on the subjective judgment of teachers. The policy recommendations are as follows: 1. The school principal should impose effective teacher training regarding the assessment of the eight subject areas that focus on higher ability measurement. 2. One key role for school principal is to ask academic staff and assessment experts to develop prototypes for integrated reading, critical thinking and writing assessment tools for teachers. 3. The school principal needs to ask academic staff and assessment experts to create a procedure manual for systematic assessment of desired learners’ attributes.

ACADEMIC ACHIEVEMENT ASSESSMENT FOR HIGHER EDUCATION Higher education is essential to human resource development, especially in this era of globalization, where a country’s international competitiveness depends a great deal on the capability of its citizens to thrive in a knowledge-based economy and society. In response to government policy, the Office of the Higher Education Commission proposed a roadmap to serve as guiding principles for promoting and developing the quality of Thai higher education. Covering the years between 2005 and 2008, the roadmap consists of measures to enhance the quality of graduates, lecturers, research projects and the management of higher education institutions. To increase access to and improvement in the quality of higher education, several endeavors have been undertaken: the provision of the Income Contingent Loan; establishment of new universities and the transformation of existing public institutions into autonomous universities; reform of the central university admission system; and the promotion of research and innovation of Thai higher education institutions.

Access to Higher Education The previous national admission system, in effect since 1967, it was well-accepted as one of the most fair educational procedures in Thailand. However it depended solely upon an entrance examination that evaluated a student’s academic performance only through that particular test. As a result, students tended to place less importance on learning for knowledge and skills in their regular classrooms and concentrated their efforts on tutorials for particular subjects, in order just to be successful on the entrance examination. Consequently, responsible agencies, such as the Office of the Higher Education Commission, the Council of University Presidents of Thailand, and the educational institutions themselves called for a change from the previous approach, with a new Central University Admission System (CUAS).

56

Sirichai Kanjanawasee

Reform of the Higher Education Admission System Beginning in 2006, students completing Grade 12 are obliged to take national educational tests of 2 types: the Ordinary National Educational Test (O-NET) and the Advanced National Educational Test (A-NET). Both examinations are administered by the National Institute of Educational Testing Service to assess student proficiency in Mathematics, Thai language, Science, English language and Social Studies, Religion and Culture. The A-NET tests focus more on critical thinking and analytical skills, as well as aptitude in relation to professional disciplines. The newly-modified university admission system can be summed up as follows:

(1) The Direct University Admission System Under this system, each higher education institution determines and administers its own admission criteria and procedures within certain fields of study consistent with its mission and philosophy. For admission to some fields of study, the Direct Admission System may also include O-NET and/or A-Net results. Aptitude tests not included in the national tests, but required for certain fields of study, will be administered by the particular institution. (2) The Central University Admission System (CUAS) The admission system was redesigned and implemented in 2006, using integrated multiple indicators to evaluate student achievement and performance. These include: (1) the overall cumulative grade point average (GPAX) for Grades 10-12; (2) the grade point average (GPA) for Grades 10-12 in 3 to 5 of the 8 subject groups of the core curriculum; and (3) results of the O-NET and/or A-NET Tests. The agencies involved are striving to further improve the CUAS so that students will be able to study in the university programs they select, while at the same time focusing on attaining a well-rounded education at the secondary level. (3) Special Programmes and the Quota System Apart from the normal admission system, a number of students are also admitted through special programs and a quota arrangement set by the institution. For example, in cooperation with the Ministry of Public Health, the faculties of medicine at Chulalongkorn, Chiang Mai, Khon Kaen, Mahidol and Prince of Songkla universities admit about 15 percent of their students through the ‘Promotion of Medical Science Education for Rural Areas’ Project. Other programs admitting students through special university quotas include the ‘Development and Promotion of Science and Technology Talents’ Project, the Sports Promotion Programme and the Arts Promotion Programme.

New Designs, New Challenges and Policy Recommendations (1)New Designs Higher education admission system is always a high stake and crucial issue in Thailand. It means the future life for students and their families, and there are many stakeholders involved.

Development of Educational Evaluation and Challenges…

57

Table 2. Components of the New Central University Admission Systems Components 1. GPAX 2. GPA (3-5 subjects areas from 8subjects areas) 3. O-NET* 4. A-NET*

Academic Year 2006 10%

2007 10%

2008 10%

20%

30%

40%

35-70% 0-35%

60%

50%

*

Remark Both O-NET and A-NET examinations are administered by the National Institute of Educational Testing Service (NIETS).

The Office of Higher Education Commission and the Council of University Presidents of Thailand has concluded the integrated multiple indicators to evaluate student achievement and performance for entering higher education institutions as shown in Table 2.

(2) New Challenges The O-NET and A-NET, national standard tests are constructed by the newly established public organization, NIETS. How did the NIETS develop the O-NET tests, which represent subject area content for different school curricula? How will the NIETS develop the A-NET tests, which correspond to subject areas for different disciplines? How will the NIETS manage the large scale testing effectively? The students’ GPAX and GPA are grading from each school which is different in typical curriculum, different tools, and probably different quality standards. How will we figure out school’s GPA inflation rates and how can we make adjustments for GPAX and GPA? (3) Policy Recommendations The O-NET and A-NET National Standard Tests: •

•

•

The Basic Education Commission must call for national content standards of core curriculum in the eight subject areas. The NIETS should develop the O-NET Tests which cover the national content standards for each subject area. The NIETS should develop the A-NET, Tests which focus more on critical thinking and analytical skills as well as aptitude measures in relation to well-defined groups of professional disciplines, such as medicine, engineering, education, architecture, foreign languages, fine arts and music. The NIETS should establish testing standards for test construction, test administration, test scoring and reporting. The NIETS should prepare for computerized adaptive testing.

The Students’ GPAX and GPA:

58

Sirichai Kanjanawasee •

•

•

The Basic Education Commission should promote investigative research regarding basic education achievement, GPA inflation, development of GPA inflation rates, and linking of GPA and GPAX. The Higher Education Commission should encourage university scholars to develop and compare GPA equating and linking methods among the linear method, equipercentile method, regression method, and item response theory method to find out the most appropriate model. The Council of University Presidents of Thailand should set up a taskforce to conduct research on weighting the integrated multiple indicators and develop composite index calibration methods that are meaningful and fair for all students.

CONCLUSION As specified, in the 1999 National Education Act, educational reform in Thailand must establish educational standards, revised curriculum, and quality assurance. Currently, there are three types of standards: national standards, and standards for internal quality assurance and for external quality assessment. In accordance with the focus on educational reform, the Basic Education Commission called for curriculum change at the national level (core – curriculum) and at the institution level (local – curriculum). The proportion of core curriculum to local content developed by the institution should be approximately 70:30 flexibly applied in compliance with the nature of each subject. The Basic Education is divided into 4 levels: 1) Grades 1-3, 2) Grades 4-6, 3) Grades 7-9, and 4) Grades 10-12. At each level, upon leaving grades 3, 6, 9, and 12 students must pass 4 types of assessments correspondence to each institution standards: 8 subject areas’ assessment; reading, thinking and writing skills’ assessment; desired learner attributes assessment; and assessment of the participation in developmental activities program. The Thai Basic Education System is facing many assessment barriers. The national quality assurance policy should link external quality assurance with internal quality assurance to serve local wisdom and to meet minimum national standards. The new teacher evaluation approach should focus on classroom practices, student development and learning outcomes. The system needs effective in-service teacher training on the assessment of learners’ higher ability. It also needs to develop phototypes of integrated reading, thinking and writing assessment tools and procedure manual for systematic assessment of desired learners’ attributes. The previous national admission systems for Higher Education depended solely upon an entrance examination that evaluated a student’s academic performance only through that particular test. As a result, students tended to pay less attention to learning in classrooms and concentrated more on private tutorials, in particular on subjects that counted for the entrance examination. Consequently, responsible agencies called for a change in the previous approach to a new Central University Admission System (CUAS). The system was designed and implemented in 2006, using integrated multiple indicators to evaluate student achievement and performance. These included: 1) The overall cumulative grade point average (GPAX) for Grades 10-12, 2) the grade point average (GPA) in Grades 10-12 in 3 to 5 of the 8 subject groups of the core curriculum; and 3) results of the Ordinary National Educational Tests (O-

Development of Educational Evaluation and Challenges…

59

NET) and/or the Advanced National Educational Tests (A-NET). The admission system is facing the challenges of developing the O-NET tests that represent content standards for different school curricula; the A-NET tests that corresponds to well-defined groups of professional disciplines. The system also needs to develop effective GPA – equating methods and composite index calibration methods, probably based on IRT, that will be meaningful and fair for all students.

REFERENCES Kanjanawasee, S. et al. (2007). The research and development of monitoring and evaluation system on Thailand education. Bangkok: Office of the Education Council. Kanjanawasee, S. et al. (2006). The achievement evaluation of Ministry of Education’s performance project. Bangkok: Faculty of Education, Chulalongkorn University. Kanjanawasee, S. (2000). Learning assessment : Executive recommendations. Journal of Research Methodology, 13(1), 75 – 91. Kanjanawasee, S. (2007). Modern test theories. (3rd ed.). Bangkok: Chulalongkorn University Book Center. Kolen, M. J. and Brennan, R. L. (1995). Test equating: Methods and practices. New York: Springer. Office of the National Education Commission. (1999). The National Education Act of B.E. 2542(1999). Bangkok: Seven Printing Group. Office of the Education Council. (2006). Education in Thailand. Bangkok: I.C.C. International Public Co., Ltd. Office of the Education Council. (2005). National education standards. Bangkok: VTC communication Ltd. Partnership. Secretariat office of the teachers’ Council of Thailand. (2005). Education professional standards. Bangkok: Professional Standards Bureau.

In: Educational Evaluation in East Asia Editor: Samuel S. Peng and John Ch-Kin Lee

ISBN 978-1-60692-887-5 © 2009 Nova Science Publishers, Inc.

Chapter 5

EDUCATIONAL EVALUATION IN HONG KONG: STATUS AND CHALLENGES John Chi-Kin Lee Department of Curriculum and Instruction, The Chinese University of Hong Kong, China

ABSTRACT This paper provides a brief overview of current educational evaluation measures and related issues in Hong Kong. The foci of measures for discussion are quality assurance mechanisms especially school self-evaluation (SSE) and external school review (ESR), and Territory-wide System Assessment (TSA). Three questions are posed for discussion: (a) How can we promote coherence among educational reform (highlighting students’ whole-person development) and educational supervision and evaluation policies (stressing academic performances) in both rhetoric and practice? (b) How can we promote a systematic linkage between centralized systems (ESR, SSE, BCA/TSA) and localized systems (teacher evaluation) so as to drive school improvement/effectiveness and teacher development/effectiveness? (c) How can we help relieve teachers’ workload and anxieties linked with educational evaluation and foster a shared community of educational evaluation among government agencies, SSB/SMC/IMC and teachers?

Keywords: Educational evaluation; quality assurance; Hong Kong.

I. INTRODUCTION The term “educational evaluation” has diverse meanings and its scope is wide encompassing many sub-fields and topics ranging from student evaluation (e.g., external (public) examinations), personnel evaluation (e.g., teacher and principal evaluation) to programme/project evaluation, school evaluation and system evaluation (e.g., international studies of educational achievement, state and school district evaluation) (Kellaghan and

62

John Chi-Kin Lee

Stufflebeam, 2003a, 2003b). In this Forum for Educational Evaluation in East Asia – Emerging Issues and Challenges, topics of presentations and sessions reveal the diversity ranging from school evaluation systems, school-based assessments, teacher evaluation, language proficiency assessment, international assessment (such as PISA), information technology and psychological applications, to mobile devices and learning assessment, and institutional evaluation (capacity building of evaluation personnel). While I shall give a very brief overview of some of these topics, the paper will concentrate on the issues related to school evaluation and teacher evaluation. According to a seminar organized by the Asian Network of Training and Research Institutions in Educational Planning (ANTRIEP) (De Grauwe and Naidoo, 2004, p.22), school evaluation is “part of the decision-making process in education; it involves judgments about the performance of schools through systematically collecting and analyzing information and relating this to explicit objectives, criteria and values. Ideally, school evaluation involves an (internal and external) assessment that covers all aspects of a school and their impact upon student learning…From this perspective, school evaluation is not an end in itself, but the first step in the process of school improvement and quality enhancement.” In that seminar, three main evaluation tools, (external review or inspection, examinations and school self-evaluation or internal evaluation) were discussed. With regard to teacher evaluation, this covers a broad range of issues, such as licensure testing for beginning teachers, innovations in teacher assessment and teacher tests and connections between teacher evaluation, accountability and pay (Pearlman and Tannenbaum, 2003).

II. CONTEXT AND OVERVIEW OF EDUCATIONAL EVALUATION IN HONG KONG Since 1997, Hong Kong has experienced not only political changes but also the gradual introduction of a comprehensive range of educational reforms, aimed at enhancing quality in education (Lee, 2005, 2006). Of the various reform measures, one is school-based management (SBM). Schools are expected to transform themselves into dynamic and accountable professional learning communities, to develop a proper performance appraisal system for the evaluation and development of staff and to engage in internal self-evaluation with external assessment (Advisory Committee on School-based Management, 2000). Assessment for learning and school-based assessments in the area of student evaluation have become increasingly important in the educational agenda in Hong Kong. This is partly because of the advocacy of curriculum reform and partly because of the intended introduction of school-based assessment (SBA) as a component of many future senior secondary (grades 10-12) subjects’ public examinations. Concomitant with educational reforms, schools have encountered the challenges of closure resulting from falling school enrollment, a consequence of a sharp decline in the birth rate. Against this background, the Government and the school sector have given more attention to educational evaluation issues, which are to some extent linked with the notions of accountability, competition and the quest for excellence. Before going into details about system evaluation, programme evaluation as well as external school review and school self-

Educational Evaluation in Hong Kong: Status and Challenges

63

evaluation, it is useful to understand briefly the main bodies for coordinating and arranging different evaluation policies and measures in Hong Kong. As shown in Figure 1, most public examinations and assessment measures (e.g., Basic Competency Assessment and Territorywide System Assessement) are under the purview of the Hong Kong Examinations and Assessment Authority (HKEAA) while the Education Bureau (EDB) (formerly known as the Education and Manpower Bureau (EMB)) has a Curriculum and Quality Assurance Branch to implement quality assurance measures such as external school reviews and quality assurance inspections. On the other hand, the EDB may commission external bodies (e.g., universities) to conduct evaluation studies or large-scale assessments such as the PISA (Programme for International Assessment). With regard to system evaluation, Hong Kong has taken part in a number of large-scale international studies of educational achievement such as PISA, TIMSS (Trends in International Mathematics and Science Study) (e.g., Leung, 2005) and PIRLS (Progress in International Reading Literacy Study) (e.g., Tse and Loh, 2007). Taking PISA, organized by the OECD (Organization for Economic Co-operation and Development) as an example, the analysis of findings from PISA revealed that “Hong Kong’s education system has achieved both excellence and equality of opportunity when compared with other participating countries…Hong Kong has the highest percentage of immigrant students among the participating countries/regions, and the achievement gap between the non-native students and local students is substantial” (Ho, 2005a, p.2). The findings from PISA inform us about other issues, such as the influence of students’ self-regulated learning on their achievements (Ho, 2004) as well as student performance in various subjects in different medium-of-instruction schools (e.g., Ho and Man, 2007).

Figure 1. Main educational bodies and selected educational evaluation measures affecting Hong Kong schools.

64

John Chi-Kin Lee

On the other hand, there are public examinations and territory-wide assessments organized by the Hong Kong Examinations and Assessment Authority (HKEAA). The former are exemplified by the Hong Kong Certification of Education Examination (HKCEE) at the end of secondary five (grade 11) and the Hong Kong Advanced Level Examination (HKALE) at the end of the sixth form (secondary seven, grade 13). It is worth noting that “standards of performance on the HKCEE and HKALE have for many years been benchmarked against standards in comparable subjects on British GCE and A/AS-Level examinations”. (http://eant01.hkeaa.edu.hk/hkea/switch.asp?p_left=ir_left.aspandp_clickurl=http://www.hkea a.edu.hk/zh/ir_intro.htm access: 1-11-2007). The latter refers to the Territory-wide System Assessment (TSA), which will be discussed in the following section. Moreover, the Education Bureau implemented a policy of language proficiency assessment for teachers teaching English and Putonghua in which the HKEAA is responsible for arranging the Language Proficiency Assessment for Teachers (LPAT). As regards programme evaluation, schools have started to become acquainted with monitoring or review of projects, especially those submitted to the Quality Education Fund (QEF), established in 1998 to promote school-based, educational innovations. The QEF stipulates that leaders of QEF projects need to submit progress reports, involving selfevaluation of project effectiveness (with regard to attainment of the stated objectives, success of the activities in the light of the impact on the participants and resources (e.g., people, time and equipment, etc.) used for the activities). For the final report, the project impact in terms of a project’s effects on learning effectiveness, professional development and/or school development should be addressed and evaluated using an evidence-based approach. In addition, evaluation of a project needs to include the quality and dissemination value of the project deliverables (http://qef.org.hk/eng/main.htm?plan/plan01.htm, access: 26-7-2007). On the other hand, teachers in some schools may have the chance to engage in small-scale action research (e.g., Lee and Leung, 2006; Lee, 2007; Lee, Yin and Zhou, 2008) or learning studies (Lo, 2006) through participation in university-school partnership projects (Lee and Williams, 2006).

III. QUALITY ASSURANCE, SELF-EVALUATION AND SCHOOL IMPROVEMENT FOR HONG KONG SCHOOLS In 1997, the Quality Assurance Inspectorate (QAI) was established and since 2003 there has been “School Development and Accountability” (SDA) framework, set up to promote school improvement through a combination of School Self-evaluation (SSE) and External School Review (ESR). Such a framework reflects a gradual shift from an emphasis on external inspection to a school’s own review of its performance as a starting point, to be validated by ESR and the school as the center for change and improvement (Tang, 2008). In the handbook Self-evaluation: Background, principles and key learning (MacBeath, 2005), published by the National College for School Leadership, there is a section on “Selfevaluation and school review in Hong Kong” (pp.39-40). This praised Hong Kong’s model as “exemplary of the sequential model, one that lies closest to the New Relationship in Schools in England and one that offers some pointers to both the strengths and potential pitfalls of

Educational Evaluation in Hong Kong: Status and Challenges

65

shorter, sharper inspection centred on the school’s own internal evaluation” (p.39). It also highlights the following issues (p.39): • •

There may be disagreements over the final grade of different aspects of school quality or a lack of democratic process in arriving at the final grade; and Downgrading of grades by the external school review (ESR) team may sometimes lead to disappointment or resentment but in most cases the school accepts the professional judgment and opinions of the ESR team

In Hong Kong, some school improvement projects were launched that helped schools to familiarize themselves with, and implement, self-evaluation. Pang (2004, p.5), for example, initiated projects adopting a bottom-up approach that aimed to help schools: develop their own models of school-based management in the spirit of the recommendations of the Education Commission Report No.7; institutionalize a self-evaluation framework in daily practices for continuous improvement; and develop their own sets of school-based performance indicators for use in school self-evaluation. On the other hand, the Education and Manpower Bureau of the Hong Kong SAR Government launched a partnership project known as the School Development through School Self-evaluation Project with twenty-one schools (McGlynn, 2004; QAC, EMB, 2003a, 2003b). The project proposed three focuses or circles of evaluation: learning at the center or first level; culture, which refers to the enabling conditions and ethos conducive to student learning, as the second level; and leadership as the third level (Kennedy and Lee, in press). The SSE activities chosen by the project schools ranged from reading and/or writing, peer observation/collaborative teaching, students’ evaluation of teachers’ performance, using a staff appraisal process to promote school self-evaluation, teachers’ view of the ideal and actual school, appraising the school’s performance through surveys of stakeholders to students’ perceptions of the quality of school life and comparison with teachers’ views to evaluation of the programme on personal and social education, evaluation of the staff development programme in a new school, evaluation of mathematics teaching and students’ views on their first week in primary school (QAC, EMB, 2003a, p.12). In addition to the efforts by various universities and the Government, some professional organizations played a part in promoting school self-evaluation. Under the auspices of the District Teachers’ Network, the Hong Kong Primary Educational Research Association and the Education and Manpower Bureau co-organised with the School of Educational Management of Beijing Normal University a “Sharing and Action Research on School Selfevaluation in Primary School” project involving nine schools (Hong Kong Primary Educational Research Association, 2003). The project aimed to promote a culture of school self-evaluation through workshops on using schools’ performance indicators, teachers’ and students’ self-evaluation, peer observation and action learning on teaching practices (Kennedy and Lee, in press; Wu and Lee, 2005). MacBeath and Clark, commissioned by the Quality Assurance Division (QAD), Education and Manpower Bureau (EMB) (now called the Education Bureau), led and conducted two phases (Phase I and II) of “The Impact Study on the Effectiveness of External School Review in Enhancing School Improvement through School Self-evaluation in Hong

66

John Chi-Kin Lee

Kong” (QAC, EMB, 2005, 2006). It is noteworthy that “the role and task of the SIT/SSE team holds the key to embedding self-evaluation in the school (QAC, EMB, 2005, p.11) and “school leadership is ultimately the single most important ingredient in making SSE work.” (QAC, EMB, 2006, p.14). According to a Legislative Council paper submitted by the Education and Manpower Bureau in June 2007 (LC Paper No. CB(2)2071/06-07(01)), the major achievements and observations of the SDA framework are (EMB, 2007, pp.4-5): (a) (b) (c) (d)

giving impetus to nurturing the culture of school self-evaluation (SSE); promoting the use of data and evidence as a basis for SSE; creating a greater sense of openness and transparency within schools; and inducing a positive impact on learning and teaching.

In the second cycle, scheduled to start in 2008/09, the following guidelines will be used (based on p.7 and p.8): (a) the process will continue to be improvement oriented; (b) the process will become more focused and school-specific, making reference to the findings of the first ESR report for individual schools; (c) the process will embed SSE in ongoing school practice, strengthening SSE and ESR as integral parts of the school improvement process; and (d) the process will streamline the SDA process (such as refining performance indicator (PI) areas and Key Performance Measures (KPMs)), further reducing teachers’ workload.

IV. BASIC COMPETENCY ASSESSMENT AS “STATE” EVALUATION: A FORM OF SYSTEM EVALUATION The Basic Competency Assessment (BCA) has two components: student assessment and system assessment (now called the Territory-wide System Assessment (TSA)). The former refers to a resource bank on assessment for learning provided through the internet and the latter is administered at the territory level at the three levels of Primary three, Primary six and Secondary three. The TSA is developed as a low-stake assessment tool intended to provide feedback to schools about their standards in the three subjects of Chinese language, English language and mathematics so that schools and teachers could make plans to improve effectiveness in learning and teaching (http://cd1.edb.hkedcity.net/cd/eap_web/ bca/index3.htm access 25-72007). Despite the good intention of having TSA as a low-stake assessment, some schools worried that the data, if unfavourable to the school, would affect the parents’ and EDB image of the school and be used as a reason for school closure by the government. Some schools were found cheating by encouraging weak students to abstain from the examinations while some others arranged intensive drilling for students. There were also concerns that the TSA scores of new immigrants and special needs students would be calculated on a separate basis as this was perceived to be unfair to schools.

Educational Evaluation in Hong Kong: Status and Challenges

67

V. TEACHER (STAFF) APPRAISAL AS A FORM OF SCHOOL-BASED TEACHER EVALUATION In Hong Kong, it is basically up to the individual School Sponsoring Body (SSB) and/or School Management Committee (SMC)/Incorporated Management Committee (IMC) to decide upon their own teacher appraisal system (Lee, Lam and Li, 2003). These observations are pertinent: •

• •

•

While staff appraisal systems and the peer observation in schools are more popular, they are still in various stages of development depending on individual SSB, SMC or IMC. As staff appraisal systems are not usually and directly linked with incentives, the assessment does not tend to be very rigorous (meeting the threshold and having a central tendency in the case of having different raters). Most teachers do not feel threatened by in-house evaluation. Teacher evaluation places more emphasis on overall professional duties than on actual classroom teaching practices and/or student learning effectiveness/quality. There is also an increasing tendency to involve more than one evaluator in the process of school-based teacher evaluation and having principal evaluation conducted by teachers. There is little linkage or loose coupling between teacher evaluation and school improvement.

VI. INITIAL OBSERVATIONS ON PRINCIPALS’ AND TEACHERS’ PERCEPTIONS OF QUALITY ASSURANCE MEASURES (E.G., ESR AND SSE, TSA AND TEACHER APPRAISAL) According to a survey conducted by the Hong Kong Primary Educational Research Association and Education Convergence (2006), primary school teachers perceived that the following policy measures had pressured them (in rank order): Territory-wide Assessments (TSA), Language Proficiency Assessment (Benchmarking) for language teachers; implementation of inclusive education; self-evaluation mechanism under school management; and external school review. For secondary school counterparts, the policy measures perceived to have pressured them (in rank order): external school review, senior secondary curriculum reform, change to a 3-34 system (six years’ secondary school system and four years’ university system); Language Proficiency Assessment (Benchmarking) for language teachers; and self-evaluation mechanism under school management. These evaluation and assessment measures that have generated teachers’ anxiety should not be taken lightly by policymakers. The writer has recently conducted studies of principals’ and teachers’ perceptions of quality assurance measures (e.g., ESR and SSE, TSA and teacher appraisal) in five primary schools in Hong Kong and the following are initial observations:

68

John Chi-Kin Lee •

•

•

•

While different schools revealed varying stages of development in school selfevaluation, almost all principals saw the positive benefits brought about by school self-evaluation with external school review. Some teachers felt that SSE and ESR brought an increased workload in preparing documents, together with threats of uncertainty, while some others perceived that ESR could provide a recognition of a school’s accomplishments. Some teacher interviewees were still worried that less than outstanding and good TSA scores would be used as evidence of unsatisfactory performance in external school review. This could be used as an implicit reason for school closure and follow-up monitoring by the EDB. While most schools used TSA as feedback for improving learning and teaching, they placed different emphasis on and had varying modes (e.g., extra lessons, TSA format assessment, soliciting parental support in speech training) of preparing students for TSA.

VII. THE WAY FORWARD Educational evaluation is a powerful approach to appraise the effectiveness and quality of inputs, processes and outputs in an educational system. It is also a useful approach to provide opportunities for self-evaluation, reflection and continuous improvement as well as to provide timely and important information to various stakeholders. Nonetheless, in an education system like Hong Kong, accountability and performance are highlighted, which may induce stress to teachers and school resistance to the adoption of evaluation measures. Also, the cultural heritage emphasizing academic excellence in Hong Kong may exacerbate competitiveness among students, teachers and schools, defeating the good intentions of evaluation (such as in the case of TSA and school self-evaluation) in providing feedback for improvement. It is therefore imperative for policymakers, university professors and frontline practitioners to explore solutions for resolving challenges that result from evaluation policies and these solutions entail: (a) promoting coherence among educational reform (highlighting students’ wholeperson development) and educational supervision and evaluation policies (stressing academic performances) in both rhetoric and practice; (b) promoting a systematic linkage between centralized systems (ESR, SSE, BCA/TSA) and localized systems (teacher evaluation) so as to drive school improvement/effectiveness and teacher development/effectiveness; and (c) helping reduce teachers’ workloads, relieving any anxieties linked with educational evaluation and fostering a shared community of educational evaluation among government agencies, SSB/SMC/IMC and teachers.

Educational Evaluation in Hong Kong: Status and Challenges

69

ACKNOWLEDGMENT The author would like to thank EDB’s colleagues, Mr Tony Tang, PAS(QA) for his considerable support in providing useful information and Ms Po-ling Wu, PEO(QA) for her help in interviews conducted for this study. Sincere thanks are extended to Professor Michael Williams for his advice.

REFERENCES Advisory Committee on School-based Management (2000). Transforming schools into dynamic and accountable professional learning communities: School-based management consultation document. Hong Kong: Printing Department. De Grauwe, A., and Naidoo, J. P. (2004). Seminar report: School evaluation for quality improvement: issues and challenges. In A. De Grauwe and J. P. Naidoo. (Eds.), School evaluation for quality improvement: An ANTRIEP report (pp.15-40). UNESCO: International Institute for Educational Planning. Education and Manpower Bureau (EMB) (2007). Report on the implementation of the school development and accountability and the planning of the second cycle. LC Paper No. CB(2)2071/06-07(01). Kennedy, K., and Lee, J. C. K. (in press). The changing role of schools in Asian societies: Schools for the knowledge society. London: Routledge. Ho. E. S. C. (2004). The nature and impact of self-regulated learning on student’s achievement: What we have learned from the first cycle of PISA. Hong Kong: Faculty of Education and Hong Kong Institute of Educational Research, The Chinese University of Hong Kong (in Chinese). Ho, E. S. C. (2005). Can basic education system in Hong Kong be equal and excellent: Results from PISA2000+. Hong Kong: Faculty of Education and Hong Kong Institute of Educational Research. Ho, E. S. C. and Man, E. Y. F. (2007). Student performance in Chinese medium-ofinstruction (CMI) and English medium-of-instruction (EMI) schools: What we learned from the PISA study. Hong Kong: Faculty of Education and Hong Kong Institute of Educational Research. Hong Kong Primary Educational Research Association (Ed.). (2003). Action research and sharing of primary schools Self-evaluation. Hong Kong: Hong Kong Primary Educational Research Association. (in Chinese). Hong Kong Primary Educational Research Association (HKPERA) and Education Convergence (EC) (2006). A research study on Hong Kong teachers’ stress: Preliminary analysis. Educational Research Report Series No. 2. Hong Kong: HKERA and EC. Kellaghan, T. and Stufflebeam, D. L. (Eds.). (2003a). International handbook of educational evaluation (Part one: Perspectives). Dordrecht: Kluwer Academic Publishers. Kellaghan, T. and Stufflebeam, D. L. (Eds.). (2003b). International handbook of educational evaluation (Part two: Practice). Dordrecht: Kluwer Academic Publishers. Lee, J. C. K. (2005). School effectiveness and improvement in Hong Kong: Review and prospect. Educational Journal, 33(1-2), 1-23. (in Chinese).

70

John Chi-Kin Lee

Lee, J. C. K. (2006). Commentary on papers presented at the MORE symposium at the ICSEI Annual Conference, Barcelona, Spain, 2005. Educational Research and Evaluation, 12(6), 595-600. Lee, J. C. K. (Li Zijian) (2007). Foreword: University and school partnership research: From 4P to 4R. Shanghai Research on Education, 8, 1. Lee, J. C. K., Lam, W.P. and Li, Y. Y. (2003). Teacher evaluation and effectiveness in Hong Kong: Issues and challenges. Journal of Personnel Evaluation in Education, 17(1), pp.41-65. Lee, J. C. K. and Leung, Y. M. J. (2006). Exploring the core elements of improving classroom learning and teaching. Hong Kong: Hong Kong Institute of Educational Research, The Chinese University of Hong Kong. Lee, J. C. K., and Williams, M. (Eds.) (2006). School improvement: International perspectives. New York: Nova Science Publishers, Inc. Lee, J.C.K., Yin, H. and Zhou, Z. (2008). Facilitating teacher professional development through ‘4-P’ model: Experiences from Hong Kong’s Partnership for Improvement of Learning and Teaching (PILT) project. Journal of Educational Research and Development, 4(2), 17-47. (in Chinese) Leung, F. K. S. (2005). Some characteristics of East Asian mathematics classrooms based on data from the TIMSS 1999 video study. Educational Studies in Mathematics, 60(2), 199215. Lo, M. L. (2006). Learning study – the Hong Kong version of lesson study: Development, impact and challenges. In M. Matoba, Crawford, K. A., Mohammad, R. S. Arani (Eds.), Lesson study: International perspective on policy and practice (pp. 133-157). Beijing: Educational Science Publishing House. McGlynn, A. (2004). Self-evaluation or self-delusion: The choice is yours. in N.S.K. Pang, J. MacBeath and A. McGlynn (Eds), Self-evaluation and school development (pp.17-22). Hong Kong: Faculty of Education and Hong Kong Institute of Educational Research, The Chinese University of Hong Kong, Hong Kong. MacBeath, J. (2005). Self-Evaluation: Background, Principles and Key Learning. Nottingham: National College of School Leadership. Pang, N. S. K. (2004). Developing schools through self-evaluation, in N. S. K. Pang, J MacBeath and A. McGlynn (Eds), Self-evaluation and school development (pp.1-9). Hong Kong: Faculty of Education and Hong Kong Institute of Educational Research The Chinese University of Hong Kong. Pearlman, M., and Tannenbaum, R. (2003). Teacher evaluation practices in the accountability era. In T. Kellaghan and D. L. Stufflebeam (Eds.), International handbook of educational evaluation (Part two: Practice) (pp.609-642). Dordrecht: Kluwer Academic Publishers. Quality Assurance Division, Education and Manpower Bureau (QAC, EMB) (2003a). Is our school a Hong Kong school of today and tomorrow: A self-evaluating school is the way to success. Retrieved July 26, 2007, from http://www.edb.gov.hk/FileManager/ Tc/Content_705/SSE_Pub1.pdf. Quality Assurance Division, Education and Manpower Bureau (QAC, EMB) (2003b). School self-evaluation: Making it happen in twenty one schools. Retrieved July 26, 2007, from http://www.edb.gov.hk/FileManager/Tc/Content_705/SSE_Pub2.doc. Quality Assurance Division, Education and Manpower Bureau (QAC, EMB) (2005). The impact study on the effectiveness of external school review in enhancing school

Educational Evaluation in Hong Kong: Status and Challenges

71

improvement through school self-evaluation in Hong Kong. Phase I report executive summary. Retrieved July 24, 2007, from http://www.edb.gov.hk/filemanager/en/ content_176/phase_i_impact_study_on_esr_executive_summary_eng_0505.pdf. Quality Assurance Division, Education and Manpower Bureau (QAC, EMB) (2006). The impact study on the effectiveness of external school review in enhancing school improvement through school self-evaluation in Hong Kong. Phase II report executive summary. Retrieved July 24, 2007, from http://www.edb.gov.hk/filemanager/en/ content_176/phase_ii_impact_study_on_esr_executive_summary_eng_0806.pdf. Tang, T.F.Y. (2008). School development and accountability in Hong Kong: A quality assurance framework. Journal of Quality School Education, 5, 39-55. Tse, S. K., and Loh, E. K. Y. (2007). The impact of PIRLS in the Hong Kong, SAR. In K. Schwippert (Ed.), Progress in reading literacy: The impact of PIRLS 2001 in 13 countries (pp. 109-126). Münster: Waxmann Verlag GmbH. Wu, S. W. and Lee, K. K. (2005). Improve school self-evaluation by practice. Hong Kong Teachers' Centre Journal, 4, 109-14. (in Chinese).

PART II. VARIATIONS AND EVOLUATION

In: Educational Evaluation in East Asia Editor: Samuel S. Peng and John Ch-Kin Lee

ISBN 978-1-60692-887-5 © 2009 Nova Science Publishers, Inc.

Chapter 6

EDUCATIONAL EVALUATION AND QUALITY ASSURANCE: THE CASE OF HONG KONG Nicholas Sun-keung Pang Department of Educational Administration and Policy, The Chinese University of Hong Kong, China

ABSTRACT In recent years, the Education and Manpower Bureau (now Education Bureau) has implemented a two-pronged approach to assuring the quality of education in Hong Kong: an external mechanism via inspection and an internal framework via self-evaluation. However, both of these are mainly top-down and less effective than expected. This essay argues that quality assurance in collaboration with an external independent agency through a bottom-up approach is a more effective way to sustain development in schools. Schools can only improve continuously when they have institutionalized a bottom-up approach to school self-evaluation. After introducing the details of a series of schooluniversity partnership projects concerned with school self-evaluation in Hong Kong, this paper investigates the potential effectiveness of implementing self-evaluation in Hong Kong schools and the factors that have both hindered and facilitated such a policy.

Keywords: quality assurance, school self-evaluation, school inspection, sustainable development, school self-renewal, bottom-up approach.

THE QUALITY ASSURANCE MOVEMENT IN HONG KONG There have been rapid changes in both the educational system and the schools themselves in Hong Kong, due to the recommendations of the Education Commission Report No. 7 (ECR7) issued in 1997. The ECR7 recommended a two-pronged approach to ensure the quality of education in Hong Kong: an external assurance mechanism and an internal quality assurance framework. While the external quality assurance mechanism was achieved through

76

Nicholas Sun-keung Pang

the establishment, in 1997, of a Quality Assurance Inspectorate (QAI) to which schools were accountable, the internal quality assurance framework relied on schools’ own ability to evaluate themselves as part of the process of school improvement. The external quality assurance mechanism worked by adopting a whole-school approach to inspection by the QAI, which assessed schools' effectiveness, identified their strengths and weaknesses, made suggestions on ways of improvement and development in the schools, and released inspection reports for public reference. In order to continuously improve the quality of school education, all schools were also expected to engage in cyclical processes of planning, implementation and evaluation (Pang, MacBeath, and McGlynn, 2004). Every school was expected to work towards meeting the educational needs of its students as effectively as it could, and self-evaluation would provide information on which to base plans for improvement. As for this self-evaluation, all schools were to produce documents which outlined the long-term goals, prioritized development areas, set out specific targets for implementation, evaluated progress made in work during the previous school year, and set improvement or development targets for the coming year (Ministry of Education, 1984; Scottish Office, 2002). However, in the initial stage, the two strategies in this two-pronged approach to assuring educational quality in Hong Kong might not have been as effective as expected. For one thing, as for the external assurance mechanism, whole-school inspections could only take place for two- or three-week periods (each time) and the QAI would not re-visit the same school for at least five to six years, given the QAI teams’ manpower. Thus while wholeschool inspections were implemented in Hong Kong in the 1997/98 school year, between that year and 2001/02 the number of schools inspected was only about 11.4% of the total. Secondly, in conducting whole-school inspections in over 200 schools from 1998 to 2002, the QAI found that most of these schools had not established a self-evaluation framework, and in fact at that time there were still no appropriate school-based indicators for school selfevaluation. Schools can only improve continuously when they have institutionalized a self-evaluation framework in daily practice and when there is a set of valid, reliable and school-based performance indicators available for use in self-evaluation. Practicing self-evaluation enables schools (i) to develop formal procedures for setting school goals; (ii) to have the participation of teachers, parents and alumni in school management, development, planning, evaluation and decision-making; (iii) to assess their progress towards goals as well as their own performance over time; and (iv) to take appropriate steps toward self-improvement. When school-based indicators are derived from the schools’ own stated aims, they are useful tools for measuring and monitoring school performance in areas of interest. Self-evaluation with appropriate school-based indicators provides information to schools, teachers, parents, students and the community, using for reference the general profiles of schools and the comparison among schools of similar background or quality. School self-evaluation and school-based performance indicators are the crucial elements needed for continuous improvement in and of schools (Cuttance, 1994; Scottish Office, 2002). A summary of the pros and cons of this two-pronged approach to quality assurance in education is given in Table 1.

Educational Evaluation and Quality Assurance

77

Table 1. The pros and cons of the two-pronged quality assurance strategy Quality Assurance 1. Approach 2. Changing agent 3. Changes 4. Drives 5. Pace of Change

External Mechanism An innovative approach Quality Assurance Unit A “crash-through” approach with radical “surgery” Top-down and external Fast, on schedule and intermittent

Internal Mechanism An incremental approach School Self-Evaluation An evolutionary development Bottom-up and internal Slow, continuous, and long lasting

QUALITY ASSURANCE THROUGH SELF-EVALUATION Evidence-based organizational change is a very recent trend in the school reform and improvement movement. It is important that schools’ organizational change should be based on objective and reliable evidence of school performance. Once they have implemented selfevaluation, schools should have a self-renewal mechanism for managing change, one which can be built upon (1) clear and appropriate diagnosis of the school as an organization, and (2) the role of the administration. Experience in both research and practice has shown that if school reforms are to succeed, organizational changes need the active support of the principal. The principal needs to be an advocate of self-evaluation and be prepared to articulate a vision of self-renewal for the school. School development cannot be copied or otherwise imposed from outside. The principle needs to understand the current situation, including strengths and weaknesses, opportunities and threats (SWOT) to the organization; he or she needs to determine the goals to be attained within the next 1-3 years, and develop the strategies to be pursued in order to achieve them. Institutionalization of self-evaluation in the organizational framework and daily managerial practices allow the principal to lead the school towards effective educational change (MacBeath, Jakobsen, Meuret and Schratz, 2000; Gamage and Pang, 2003). To successfully institutionalize a self-renewal framework in daily managerial practices as well as to lead and manage change effectively, the principle or leader needs to: (1) acquire appropriate knowledge and understanding of the theoretical framework of school selfevaluation; (2) develop and acquire the necessary skills and attitudes for self-evaluation and manipulation of performance indicators; (3) think through his/her leadership role as a guide to action; and (4) clarify for him/herself the strategic elements essential to an effective implementation of the school development plan. Then the principal should examine the types of knowledge, skills and attitudes which need to be developed for successful implementation of organizational change.

What Is School Self-Evaluation? School self-evaluation (SSE) is a mechanism through which a school can help itself review the quality of the education it provides, improve continuously and develop itself into a more effective school. The three major questions usually asked in the process of school selfevaluation are: (1) How can we know about our school’s present performance? (2) What is

78

Nicholas Sun-keung Pang

our school's present performance? (3) What will we do after knowing this? These seem to be simple questions, but if we want to have a full picture or thorough understanding of our own school, we need to go through a systematic and objective self-evaluation.

The Basic Steps in School Self-Evaluation School self-evaluation contains several identifiable stages. These are: problem recognition, identifying priorities, defining important questions, data collection, data analysis, reporting and communicating, school development planning, school culture building, and feedback and evaluation. All these stages need to be fully implemented in a sequential order. Thus, the following schema will be useful to a leader initiating change in the organization. It needs to be re-emphasized that the leader should have a good understanding of the concept and process of school self-evaluation and consequent self-renewal. Here is a fuller description of these stages: ™ Problem identification: An organization becomes aware of the existence of a problem that needs to be fixed. The leader either recognizes and confronts it, or overlooks and ignores it. ™ Identifying priorities: There may be problems in a school’s various domains, for example, organization and management, teaching and learning, ethos and support, academic and affective performance. However, a school cannot solve all these problems within a single year. Administrators should prioritize the problems based on the teachers' will and students' needs. ™ Defining important questions: For any identified problem, the school should know the key questions involved. These will be answered through a systematic process of data collection and analysis. ™ Data collection: Data can be collected through questionnaires, observations and/or interviews, in order to ascertain whether the problem has been fixed or still exists. Consideration should be given to the sources of data which may make significant contributions to assessments of the school's performance. ™ Data analysis: On the basis of data collected through questionnaires, observations or interviews, attempts should be made to identify, clarify, and re-define the problems. ™ Reporting and communicating: Staff should be briefed on diagnostic data and involved in developing strategies to solve the problems, providing opportunities for staff training on group dynamics, communication techniques, and goal setting. ™ School development planning: An attempt should be made to bridge the gap between the current situation and what should have happened. A consultant or expert can help in determining: What steps should be taken? By whom? When? And how? Implementation of steps should be monitored to resolve any difficulties which may arise. ™ School culture building: Efforts should be made to build trust and confidence, improve communication, build teams, develop skills in problem solving, and strengthen cooperation among different subsystems of the school. ™ Feedback and evaluation: Feedback should be provided to staff as the final step in the school self-evaluation cycle. A new cycle then needs to be initiated in order to

Educational Evaluation and Quality Assurance

79

institutionalize school self-renewal (self-development) as a continuous process of innovation and change (Rudd and Davies, 2000). Schools should gather information based on the perceptions of administrators, teachers, parents, students and the community, for comparison with schools of similar background or within the same “quality circle” in order to promote continuous improvement and development.

A Bottom-Up Approach to School Self-Evaluation in Hong Kong After conducting the first few cycles of whole-school inspection starting in 1998, Hong Kong’s QAI (in 2003 renamed the Quality Assurance Division, QAD) found that a selfevaluation framework had not been generally established in Hong Kong schools and that no appropriate school-based indicators had been developed for use in school self-evaluation. In response to these weaknesses the author, with the support of the Hong Kong Centre for the Development of Educational Leadership (HKCDEL) and the Quality Education Fund (QEF), launched a series of school improvement projects to help Hong Kong schools institutionalize a self-evaluation framework in daily practice, acquire the skills and techniques needed for school self-evaluation, and develop a set of valid, reliable and school-based indicators for use in self-evaluation. These projects aimed at helping schools to (i) develop their own models of school-based management in the spirit of the recommendations of Education Commission Report No. 7; (ii) acquire the skills and techniques of school self-evaluation (SSE) and institutionalize a self-evaluation framework in daily practice for continuous improvement; (iii) develop their own sets of school-based performance indicators for use in school self-evaluation; (iv) enhance the capability of coping with challenges arising from educational reforms that would have a great impact on schools in the near future; and (v) keep developing continuously and fostering a culture of organizational learning among all participating schools. In total, 107 kindergartens, 154 primary schools and 197 secondary schools in Hong Kong have taken part in these projects in the period 2000-2010 (see Table 2). Table 2. The school improvement projects organized by the School Development and Evaluation Team (SDET) in Hong Kong Period

Project Project Title KinderDuration gartens 2000-2002 2-year School Self-evaluation and Schoolbased Performance Indicators (Quality Education Fund) 2001-2002 1-year School Self-evaluation and Renewal 2002-2003 1-year

SSE Training Programs for Heads and 22 Teachers of the Catholic Church

2002-2004 2-year

Self-Evaluation and School Development

Primary Schools 10

Secondary Total Schools 10 20

2

2

4 22

10

10

20

80

Nicholas Sun-keung Pang Table 2. (Continued)

Period

Project Project Title KinderDuration gartens 2003-2004 1-year Initiating Organizational Change via School Self-evaluation 2004-2005 1-year School Self-evaluation: Reinforcing Organizational Change 2004-2005 1-year Self-Evaluation, External Review and School Development 2004-2006 2-year Transforming Schools into Learning Organizations (Quality Education Fund) 2005-2006 1-year School Self-evaluation, External Review and Sustainability 2005-2006 1-year Self-evaluation, External Review and Extended School Development (Teaching and Learning) 2005-2006 1-year School-based Self-evaluation and Development 2006-2007 1-year School Self-evaluation: Surviving Educational Reform 2006-2007 1-year Self-evaluation of Teaching and Learning 2006-2008 2-year Learning to Learn: The Application of Metacognition in Learning and Teaching 2006-2008 2-year Metacognition in Learning and Teaching: Supporting Students’ Learning Needs (Quality Education Fund) 2007-2008 1-year School Self-evaluation: Surviving the Educational Reform 2007-2009 2-year Metacognition in Learning and Teaching 2007-2009 2-year School Self-evaluation: Optimization 42 of School Management 2008-2010 2-year Use of Metacognitive Strategies in Promoting Self-regulated Learning 2008-2010 2-year

2008-2010 2-year

Enhancing Effectiveness of Early Childhood Education through the Development of a Self-evaluation Culture From Assessment for Learning to Promoting Self-regulated Learning in Early-childhood Education (Kindergarten and Lower Primary Levels) (Quality Education Fund) Total No. of Schools

Primary Schools 23

Secondary Total Schools 27 50

10

14

24

25

36

61

10

10

20

20

36

56

2

12

14

2

3

5

4

5

9

2

6

8

3

0

3

9

9

18

1

2

3

4

4

8 42

7

11

33

18 33

10

10

107

154

20

197

458

Educational Evaluation and Quality Assurance

81

SETTING-UP A SELF-EVALUATION FRAMEWORK In order to establish a climate of quality assurance and to cultivate a quality culture in the participating schools, a normative re-education strategy was used to initiate changes in these schools: this was a user-centered, bottom-up approach to school reform. In the improvement projects all teachers in a school were allowed to participate in shaping the reform by identifying their own needs, assessing the school’s present conditions and performance, and formulating school development plans. Hence the activities organized in each school included (i) setting-up of a School Development and Evaluation Committee (SDEC) in the management structure; (ii) providing staff development programs for members of the SDEC; and (iii) conducting workshops for all teachers in the school, in order to facilitate the practice of school self-evaluation and the formulation of a school development plan. Each of the participating schools established a School Development and Evaluation Committee (SDEC) to facilitate efficient and effective school self-evaluation in daily practice. In each school, the principal and a group of three senior teachers were SDEC members. The major duties of the SDEC members were: (i) identifying priorities in the areas related to school development; (ii) defining important questions to be investigated and answered in self-evaluation; (iii) collecting appropriate data for consideration and reference; (iv) analyzing data for interpretation and sharing; (iv) reporting and communicating results of self-evaluation to concerned stakeholders; and (v) formulating a school development plan and an action plan. A staff development program was provided to all SDEC members in the participating schools. This staff development program, which consisted of ten modules covering a wide range of topics, helped the schools to practice school self-evaluation, to develop school-based performance indicators for use in self-evaluation, and to grow, develop, and improve continuously and effectively (Pang, 2007). The modules of the program were as follows: Module 1: School-based Management and School Improvement Module 2: School Development Planning Module 3: School Self-evaluation and School-based Performance Indicators Module 4: Theory and Practice in Teacher Supervision: A New Perspective Module 5: Home-School-Community Co-operation Module 6: Staff Growth and Development Module 7: Curriculum Design, Implementation and Evaluation Module 8: Strategic Planning and Financial Management in Schools Module 9: Problem-based Management and Conflict Resolution Module 10: Management Data Analysis and Basic Statistics Two concurrent programs were run for SDEC members, one for the primary schools and the other for the secondary schools. Modules 1-5 were conducted in the first year, and Modules 6-10 in the second year. The programs aimed at enabling the principals and teachers to grow and develop and to initiate changes in their own schools. Activities in the form of workshops were designed to facilitate principals’ and teachers’ reflection on the concepts introduced in the courses. There were training packages for each module; that is, in total ten different packages for the program. The training packages were designed and produced for

82

Nicholas Sun-keung Pang

self-learning purposes, with contents that were adaptable with minimal preparation or background training. A participatory approach was adopted for conducting the school self-evaluation exercises. It should be stressed that practicing self-evaluation is the responsibility not only of the SDEC members but also of the stakeholders in the schools, including school managers, administrators, teachers, parents and students. Through the processes of self-evaluation a selfrenewal strategy has been institutionalized in the schools’ management structure, and through participatory school self-evaluation a quality culture has been fostered in these schools. The projects lasted for either one or two years, depending on each school’s own preference. Four workshops within two years were offered for all teaching staff from each of the participating schools. The first whole-school workshop in Year 1 aimed to (i) facilitate setting up a self-evaluation framework in each of the participating schools; (ii) help teachers incorporate self-evaluation exercises in their daily work; and (iii) enable teachers to acquire the skills of self-evaluation and knowledge of how to use performance indicators. The second whole-school workshop in Year 1 aimed to facilitate a participatory approach, in order to (i) interpret the results of self-evaluation; and (ii) formulate a school development plan for the next year. These schools were also provided with consultancy and in-service training in interpreting self-evaluation reports and formulating school development plans in the workshops. The first whole-school workshop in Year 2 aimed to (i) reinforce and consolidate the school’s self-evaluation culture; (ii) facilitate the school’s development of its own ‘quality culture’; and (iii) foster a culture of organizational learning in the school. The second wholeschool workshop in Year 2 aimed to (i) present the results of school self-evaluation for the past two years to all staff members of the school; (ii) facilitate a participatory approach to interpreting the longitudinal self-evaluation data for the purpose of comparison; and (iii) facilitate a participatory approach to the formulating of a school development plan for the next few years.

Relationships among Performance Indicators, School Improvement and Educational Research School self-evaluation (SSE) and performance indicators (PIs) are both essential to schools’ continuous improvement. Schools should have information about the perceptions of administrators, teachers, parents, students and the community for reference and also for comparison among schools of similar background. It is difficult to “see” or demonstrate continuous school improvement, if appropriate performance indicators have not been developed for use in school self-evaluation. A theoretical framework for the relationships among performance indicators, school improvement and educational research is proposed in Figure 1. When we evaluate the performance of a number of schools against a standardized set of performance indicators, we are obviously doing one kind of educational research. The group of schools we are looking at in our research will form a quality circle for comparison. Findings from such research will inform us about the best schooling practices and tell us what we can learn from the experience of failure or poor performance.

Educational Evaluation and Quality Assurance

83

PI 10 PI 9

School Improvement

PI 8 PI 7 PI 6 PI 5 PI 4

Educational Research

PI 3 PI 2 PI 1 Sch 1 Sch 2 Sch 3 Sch 4 Sch 5 Sch 6 Sch 7 Sch 8 Sch 9 Sch 10 Figure 1. The Relations among Performance Indicators, School Improvement, and Educational Research.

On the other hand, when a school’s performance is evaluated against different sets of indicators, the results of this evaluation will show the school’s performance in different areas of concern, and thus it will become clear which areas are weakest and most in need of improvement for a particular school.

Theoretical Framework for Educational Evaluation In light of the above considerations, it will be clear that the development of sets of performance indicators (PIs) for use in self-evaluation is also an important task within the larger project of school self-improvement. Two kinds of PIs were employed in these school improvement projects: (1) a standardized set of PIs for comparing the performance of the participating schools, and (2) sets of school-based PIs (SBPIs) developed by teachers for their own use in selfevaluation in different schools. A theoretical framework for educational evaluation is proposed in Figure 2. This framework allows the School Development and Evaluation Committee (SDEC) members to consider the kinds of indicators that will suit their needs, the areas of evaluation that they are interested in, and the sources of data that they will need in order to design a set of school-based performance indicators for their own use in self-evaluation. The probable areas to be evaluated within the schools include administration and management, teaching and learning, quality of school life for teachers and students, staff development and organizational learning, school ethos and culture, home-school-community cooperation, and so forth.

84

Nicholas Sun-keung Pang

Figure 2. Theeoretical Framework for Educational Evaluation.

A Standardized Set of Performance Indicators for Comparison For the sake of comparison and forming a quality circle, all schools participating in the projects have used these sets of PIs for evaluating school performance. These PI sets were mainly developed by the School Development and Evaluation Team (SDET) of the Chinese University of Hong Kong. These sets are theoretically grounded and have good validity and reliability. Furthermore, the areas for evaluation are different in the two-year school improvement projects. The areas to be evaluated in the participating schools for Year 1 are summarized in Table 3, for Year 2 in Table 4. Table 3. Areas for evaluation in Year 1 Quality of Students’ School Life Students’ Perception of School Climate Students’ Socio-Economic Status Students’ Perception of Parental Styles

Teachers’ Feelings about School Life School Organizational Values School Organizational Learning Parents’ Perception of the School

Table 4. Areas for evaluation in Year 2 Students’ Meta-cognition Students’ Emotional Intelligence Teachers’ Meta-cognition Teachers’ Emotional Intelligence Teachers’ Job Satisfaction

Instructional Leadership Instructional Climate Teaching Effectiveness Effectiveness of the Appraisal System Parents’ Perception of the Child

Educational Evaluation and Quality Assurance

85

The combinations of different evaluation areas allow principals and teachers to understand a given school’s strengths and weaknesses from different perspectives and thus allow the triangulation of findings.

Sets of School-Based Performance Indicators The sets of school-based performance indicators (SBPIs) were independently developed by the members of School Development and Evaluation Committees (SDECs). The SDEC members had to identify their own problems in the schools and define their priorities with regard to school improvement. They had to develop a set of SBPIs for use in self-evaluation to suit their individual needs. The SDEC members acquired the skills and techniques needed for developing SBPIs when they attended the training program offered in these projects. It must be stressed that the PI sets developed should be school-based, i.e., grounded in the context of a particular school and suited to the needs of that school. Thus in the choice, development and testing of these PI sets the SDEC members should already have consulted most teachers concerning their school’s own preferences and interests. In this way, consensus and compromise could be reached prior to the full use of SBPIs in school self-evaluation.

CONCRETE BENEFITS TO SCHOOLS Since change and transformation are slow, progressive, incremental processes, it would be unrealistic to expect or seek sudden, disruptive changes in schools. Nevertheless, concrete benefits of the project were as follows: (i) schools started to foster a culture of self-evaluation, that is, a culture of organizational learning for continuous improvement (Pang, 2006a); (ii) a few training packages on school self-evaluation and the use of school-based performance indicators were developed, which were useful for other schools engaged in a similar endeavor; and (iii) publications resulted, and these disseminated good practices in school selfevaluation. The participating Hong Kong schools have been benefited by these projects in four main ways. First, a self-renewal strategy has been institutionalized in the schools through establishing a self-evaluation framework and using school-based indicators in self-evaluation; this self-renewal strategy allows schools to enhance their capacity for change and develop continuously. Secondly, school administrators’ and teachers’ skills and techniques in selfevaluation have been enhanced through a series of training courses designed specifically for them; their professional competence, confidence and performance is promoted by their participation in these programs and courses. Thirdly, the learning of students will be improved when schools’ effectiveness has been enhanced and when a quality culture has been cultivated in these schools. Fourthly, parents and the wider community will be benefited because the school self-evaluation process has led to annual reports containing fair, reliable and objective information about the schools. These reports ensure that schools are accountable to parents and to the society.

86

Nicholas Sun-keung Pang

THE EFFECTIVENESS OF THE IMPLEMENTATION OF SCHOOL SELF-EVALUATION IN HONG KONG After conducting the school-university partnership projects over the course of a few years, the author conducted follow-up qualitative research to solicit the principals' and teachers’ views on the effectiveness and usefulness of these projects and explore the factors that seemed to help and hinder the implementation of school self-evaluation (SSE) in Hong Kong schools. A sample of 20 schools was randomly selected for this qualitative research project, and 18 of their principals were interviewed. Teachers' views on the same subject matter were also solicited during whole-school workshops held in the schools. In total, about 900 teachers from the 20 schools participated in group discussions through the activities organized in the workshops. The principals’ and teachers' views and opinions, thus collected, were summarized and transcribed, and subsequently analyzed and categorized into themes. These are summarized below.

Factors That Hindered the Implementation of SSE Generally, most principals and teachers opined that school self-evaluation had not been the normal practice in schools and it was, to them, a new and innovative concept. They thought that since the implementation of SSE involved a paradigm shift in school management and change in normal school life for all teachers, external support including financial resources, staff development programs and in-house and consultancy services should be provided. In addition, most principals and teachers would like SSE to be implemented phase by phase, thus giving them more time and “space” to acquire the relevant new knowledge and skills. They reported that in the present turbulent school environment, where there were already many school reforms and innovations, the further introduction of new concepts such as school self-evaluation would inevitably lead to resistance. There were also many specific factors that hindered the implementation of SSE in Hong Kong schools, and these are summarized and classified below at the system level and the school organizational level.

Hindrances at the System Level 1. A loosely-coupled system. The Hong Kong education system is still a loosely-coupled system, with subsidized schools forming the major sector. About 80% of all schools in Hong Kong are subsidized schools, 5% are government schools, 5%, are direct subsidized schools (DSS), and 10% are private schools. While subsidized schools receive financial support from the government, they have their own school sponsoring bodies and management committees. Compared to government schools, subsidized schools have greater autonomy and more freedom of choice in responding to requests for change and the implementation of new educational policies by the Education Bureau. That is, resistance to change in the Hong Kong educational system is much greater than that in the educational systems of other countries, where the state or government schools form the largest sector.

Educational Evaluation and Quality Assurance

87

2. A too-ambitious plan. There are approximately 1,000 schools, including primary and secondary, in the Hong Kong education system. Conducting whole-school territorywide inspections or external school reviews within a few short years was an unrealistic plan. Expecting most schools to be self-reliant in conducting selfevaluations, and to become more open to change within a year or so, was again impractical and unattainable. 3. Too many existing reforms. There have already been many new reform proposals for the educational system in Hong Kong in the 21st century, in addition to those left over from the last decade. Furthermore, the implementation of these reforms and policies has mainly been attempted without good planning and co-ordination. Schools have been suffering from the great confusion and other burdens arising from these reforms. Any introduction of further new reforms and programs in schools would cause, at best, indifference and perhaps resistance because of the tremendous pressure and workloads already existing in schools. 4. School self-evaluation is a complex process. The implementation of school selfevaluation involves a change of school culture and a change of general practices in school life. Such changes cannot be achieved only through directives issued by the education authorities, but need a well-planned, bottom-up strategy of initiation and introduction which means extra resources and support from external sources. 5. Lack of resources. Effective implementation of new reforms or initiatives needs extra resources and support. At a time of economic recession in the early years of the new millennium in Hong Kong, the shortage of financial and human resources has created more difficulties for the implementation of school self-evaluation throughout the territory.

Hindrances at the School Organizational Level Implementation of SSE at the school level is not an easy task, given the present turbulent environment and conservative culture found in most Hong Kong schools. Based on the above-mentioned research, the author observed several major factors that hindered the effective implementation of school self-evaluation. Since these factors are commonly found in most Hong Kong schools, they are worthy of the special attention of school leaders and administrators. 1. The plurality of categories of stakeholders and the diversity of views and opinions in schools may have led to many excellent reforms being opposed. 2. Past failures in the implementation of educational policy caused schools to take a passive and conservative approach to educational reforms. 3. Schools are inevitably political arenas and power struggles are common. These created resistance to educational policy change in the schools. 4. School leaders and teachers in some schools might have been embroiled in conflicts which caused tension, fear, and low morale among teachers, not conducive to selfreflection and self-evaluation. 5. The communication breakdown between teachers and administrators found in some schools meant a very weak basis for professional collaboration and commitment. 6. Most teachers and principals had little knowledge of or skill in school selfevaluation, and misconceptions about SSE were common.

88

Nicholas Sun-keung Pang 7. There were no guidelines and other criteria for measuring success that were commonly agreed upon for use in evaluation. 8. There were no formal, systematic, in-depth, well-designed professional training programs to train people in the implementation of SSE. 9. Most schools were either passive or actively opposed to change, and their organizational learning culture was not well-developed. 10. Most schools lacked a long-term vision of, or ability to plan for, school development and improvement.

Factors That Facilitated the Implementation of SSE Though there were a predominance of factors that seem to have hindered the effective implementation of self-evaluation in Hong Kong schools, there were also a few factors, at both the system level and the school level, that facilitated self-evaluation in the school context.

Facilitators at the System Level 1. A leaner, flatter governance structure of the central education authority. Based on the recommendations of the report on the Education Department (Education and Manpower Bureau, 1998), the Hong Kong Government successfully merged the Education and Manpower Bureau (EMB) with the Education Department (ED) in 2002. Therefore the governance and ruling structure of the school educational system has changed from a three-tier structure (EMB-ED-Schools) to a two-tier one (EMBSchools). The interdependence between policy making and policy implementation has consequently been strengthened, and the school educational system has become more tightly organized. The central educational authority is now more interactive, and more responsive in addressing the problems and difficulties which arise in the implementation of quality assurance mechanisms. 2. An evolving model of quality assurance. Though the two-pronged strategy (an external assurance mechanism and internal quality assurance framework) for guaranteeing the quality of education in Hong Kong has remained unchanged since 1997, the internal framework for quality assurance has been evolving in order to meet the new needs of schools in the ever-changing external environment. A new framework, which enhances school development and accountability through school self-evaluation (SSE) and external school review (ESR), was introduced in 2003 (Education and Manpower Bureau, 2003). This framework stresses the internal mechanism more than the external one, and more resources have been put into the promotion of school self-evaluation. 3. Availability of tools for self-evaluation. Following a few years of development and continuous testing, the former Education and Manpower Bureau was successful in developing some sets of tools for use in schools’ self-evaluation. These tools included performance indicators and key performance measures in the domains of management and organization, teaching and learning, school support and ethos, and academic and non-academic outcomes (Quality Assurance Division, 2002). They provided balanced coverage and a common platform for assessment of different

Educational Evaluation and Quality Assurance

89

aspects of schoolwork and student performance for SSE and ESR, and also established territory-wide norms against which school performance could be measured and assessed. Stakeholders’ questionnaires for teachers, pupils and parents, and other tools in the social and affective domains for students, were developed for use in SSE, and schools were allowed to choose the relevant indicators for their own needs and uses.

Facilitators at the School Organizational Level A few schools participating in the projects successfully created a culture of selfevaluation and organizational change. Such a culture already existed, at least in potential form, in some of these schools before they took part in the projects; others had to create this sort of culture after joining the projects. The school improvement projects provided guidance and consultancy on school development, which facilitated the implementation of selfevaluation in these schools and enhanced their transformation into learning organizations (Pang, 2006a). Factors that facilitated the implementation of school self-evaluation in these schools are summarized below. 1. Enhanced leadership. The management of organizational change calls for “strong” leadership (Gamage and Pang, 2003), and there was a tendency at this time toward stronger leadership in the schools that was better able to succeed in implementing school self-evaluation and initiating organizational change. While some organizational components have a limiting influence on others because of the presence of multiple and often conflicting goals, the success in achieving beneficial organizational change in these schools was due to a strong leadership that eliminated these tensions by deciding upon unified goals and clarifying technology. 2. Shared values. There were, to a considerable extent, shared values among the staff members in the schools which promoted school self-evaluation in the management and organizational structure. Sharing values is the one fundamental activity that holds staff together and unified when they are faced with changes in long-term and short-term goals. If organizations are determinate means-ends structures for attaining preferred outcomes, then agreement about preferences is the only source of order that is left. 3. Focused attention. There was special attention given to human relations in the management system in the schools that had successfully implemented school selfevaluation. Small-step strategies within a confused, turbulent and ever-changing environment may produce more effective, efficient, interesting, varied, and thoughtful organizational changes. Leaders in these schools have compensated for multiple and conflicting goals by carefully selecting targets, controlling resources, and acting forcefully. 4. Good team spirit, high staff morale and a strong sense of professionalism. The schools that were very successful in the implementation of SSE possessed a very strong teaching force that had good team spirit and high staff morale, as well as a strong sense of professionalism. The formation of the strong, professional teaching force was not an accident but the result of deliberate and careful selection during the recruitment of personnel. High teacher morale and a strong team spirit were also the result of the enhanced leadership and effective management systems in the schools.

90

Nicholas Sun-keung Pang

The above are then the findings of the author’s qualitative research into the effectiveness of implementing self-evaluation in schools and the factors that both hindered and facilitated organizational change. These findings shed light on how school administrators can lead and manage the kind of organizational change that will really promote school development and improvement.

CONCLUSION Evidence-based organizational change has become a very recent trend in the school reform and improvement movement, a movement in which school self-evaluation plays an important role. School self-evaluation provides a framework which allows school leaders to institutionalize a self-renewal strategy in daily managerial practices as well as to lead and manage change (Pang, 2006c). Due to various hindrances at both the educational system level and the school organizational level, many Hong Kong schools have not been effectively establishing a self-evaluation framework, and many still have a relatively weak culture of self-renewal. Nevertheless, a normative re-education strategy may be effective in helping schools to surmount the resistance and the hindrances found at both system and organizational levels. The school improvement projects launched in this study are a case in point, showing that organizational change can be initiated through the putting into practice of school self-evaluation. The practical experience gained from even a few successful schools in the project can shed light on the problem of how to help other schools transform into learning organizations through this implementation of SSE. In order to facilitate change in schools, administrators need to demonstrate strong leadership that can unify their school’s goals and make clear what kind of technology is needed for achieving them. Such leadership is also necessary in order to promote the sharing of values among all members and their agreement on preferences. It is necessary for the focusing of leaders’ attention by carefully selecting targets, for controlling resources, and of course for acting forcefully (Gamage and Pang, 2006). A good team spirit, high staff morale and strong sense of professionalism do not only form the crucial basis for change, they also help reduce resistance to change (Pang, 2006b). Effective leaders are those who can adopt these approaches in order to be flexible in coping with the challenges created by an everchanging external environment, and in continuing to lead their organizations towards excellence.

REFERENCES Cuttance, P. F. (1994). Monitoring educational quality through performance indicators for school practice. School Effectiveness and School Improvement, 5(2), 101-126. Education and Manpower Bureau. (1998). Review of the Education Department. Hong Kong: The Government Printer. Education and Manpower Bureau. (2003). Enhancing school development and accountability through school self-evaluation and external school review. EMB Circular No. 23/2003. June 12, 2003.

Educational Evaluation and Quality Assurance

91

Gamage, D. T. and Pang, N. S. K. (2003). Leadership and management in education: developing essential skills and competencies. Hong Kong: The Chinese University Press. Gamage, D. T. and Pang, N. S. K. (2006). Facing the challenges of the 21st century: Preparation of school leaders in Australia and Hong Kong. Educational Research Journal, 21(1), 21-46. MacBeath, J., Jakobsen, L., Meuret, D., Schratz, M. (2000). Self-evaluation in European Schools: a Story of Change. London: Routledge/Falmer. Ministry of Education. (1984). School self-evaluation. Victoria: Ministry of Education. Pang, N. S. K., MacBeath, J., and McGlynn, A. (2004). Self-evaluation and School Development. School Education Reform Series No.19. Hong Kong: The Faculty of Education of the Chinese University of Hong Kong and Hong Kong Institute of Educational Research. Pang, N. S. K. (2006a). Schools as Learning Organizations. In J. C. K. Lee, and M. Williams, (Eds.), School Improvement: International Perspectives (pp. 65-86). New York: Nova Science Publishers Inc. Pang, N. S. K. (Ed.). (2006b). Globalization: Educational research, change and reforms. Hong Kong: The Chinese University Press, the Hong Kong Educational Research Association and the Hong Kong Institute of Educational Research. Pang, N. S. K. (2006c). Managing school change through self-evaluation in the era of globalization. In N. S. K. Pang (Ed.), Globalization: Educational research, change and reforms (pp. 293-313). Hong Kong: The Chinese University Press, the Hong Kong Educational Research Association and the Hong Kong Institute of Educational Research. Pang, N. S. K. (2007). The continuing professional development of principals in Hong Kong. Frontiers of Education in China, 2(4), 605-619. Quality Assurance Division. (2002). Performance Indicators for Hong Kong Schools. Hong Kong: Education Department. Rudd, P. and Davies, D. (2000). Evaluating school self-evaluation. Paper presented at the British Educational Research Association Conference, Cardiff University, September 710, 2000. Scottish Office. (2002). How good is our school? Self-evaluation using performance indicators. Edinburgh: the Scottish Office of Education and Industry Department.

In: Educational Evaluation in East Asia Editor: Samuel S. Peng and John Ch-Kin Lee

ISBN 978-1-60692-887-5 © 2009 Nova Science Publishers, Inc.

Chapter 7

SCHOOL-BASED ASSESSMENTS (SBA): IMPLEMENTATION AND CHALLENGES IN THE MALAYSIAN CONTEXT Suan Yoong Educational Testing Center, Universiti Pendidikan Sultan Idris, Malaysia

ABSTRACT National examinations are important annual events in Malaysia. This paper briefly provides a critical review of the assessment system currently used in Malaysia, outlines the effects of over-obsession with examinations, provides an overview of the Ministry of Education’s efforts to dilute the obsession with examinations, reviews the proposed implementation of school-based assessment, and explains the dominant forces shaping educational assessments in Malaysia, above all taking into account the cultural contexts.

Keywords: School-based assessment, high-stakes assessment.

1. INTRODUCTION National examinations are important annual events in Asia, especially when the stakes are high in determining entry into premier universities. Come examination time, it has become a common sight in many East Asian cities to see large crowds of parents or relatives waiting in the vicinity of the examination halls, giving moral and material support to their loved ones who are taking the examination. In fact, around examination time it is not uncommon to see parents readying their children for the last dash through the “dreaded and stressful” hurdles of examinations. They may do this by preparing “dietary tonics and fortifying medicines” (e.g., jinseng) and other food essences to supplement the examinee’s diet, or by extending this “fortifying culture” in the form of supplementary tutoring by ferrying students to and from private tuition centers. Indeed, the glaring presence and

94

Suan Yoong

vigorous growth of these cram schools, designed purely for examination preparation in Japan, South Korea, Mainland China, Taiwan, Hong Kong, Macao, Singapore and Malaysia, is further testimony to the presence of East Asia’s examination-oriented culture (Bray, 2003; Cheng, 1990; 1995; Chew and Leong, 1995; Foondun, 1992; 1998; Harnisch, 1994; Hussein, 1987; Kim, 2000; Kwok, 2004; Rohlen, 1980; Sorensen, 1994; Zeng, 1999). The internationally renowned “boulevard" of tuition centers on Nanjing Street, near the Taipei Train Station in Taiwan, is perhaps the most vivid example of this culture (Chen, 1996). Come D-day and the announcement of examination results, the local media rush to flash news reports that may highlight the most successful high schools, glorify outstanding individual performances, and present ranking tables that compare school performances. Of course, learning cultures in other parts of the world are also to some degree examination-oriented. However, in East Asia this phenomenon and its side-effects are more pronounced, and thus here especially they are felt to warrant a closer examination. The extraordinary emphasis placed by East Asian students and parents on examinations is also one of the pre-eminent characteristics of the educational system in Malaysia. While the examination orientation has been a force driving many students to excel in their studies, this obsession has also created both educational and social problems. Many Malaysian educators, practitioners, curriculum developers and policy makers have long recognized the negative impacts and detrimental effects of over-obsession with high-stake assessments, and have criticized such practices on the grounds that they distort the basic objectives of education. Like a record playing over and over again, these concerns have been publicly voiced year in and year out, especially after public examination results are splashed in the print media. This paper briefly provides a critical review of the assessment system currently used in Malaysia, outlines the effects of the national assessment system, provides an overview of the advantages and disadvantages of an over-obsession with examinations, reviews the implementation of the school-based assessment (SBA) system in Malaysia, and explains the dominant forces shaping educational assessments in Malaysia, with a particular emphasis on the cultural contexts.

2. THE MALAYSIAN EDUCATIONAL AND EXAMINATION SYSTEM Malaysia is a multi-ethnic country, being 66% Bumiputra (mainly Malays plus a small proportion of other indigenous groups such as Ibans, Kadazans and Bidayuh), 25% Chinese, 8% Indian (Hindu), and 1% other ethnicities (Malaysia, 2006). This Southeast Asian country has inherited a pluralistic system of education from its colonial master, England, one that mirrors to a large extent the legacy of the system in Great Britain. The difference is that, as in most Asian countries, the educational system of Malaysia is highly centralized under the full control of the Ministry of Education (MOE). The educational system in Malaysia is essentially three-tiered—primary, secondary and tertiary education—with the tiers linked to each other through a series of formal, nationallyconducted public exams (see Figure 1). At the age of 6 plus, Malaysian children begin a formal primary education that will last for 6 years. Malay, Chinese and Tamil government primary schools exist side by side, each using their respective mother-tongues as the main medium of instruction. Government secondary schools, which use Malay as the main medium

School-Based Assessments (SBA)

95

of instruction, offer 3 years of lower secondary and 2 years of upper secondary education. Students are streamed into academic or technical and vocational tracks at upper secondary levels. Pre-university education (sixth form or matriculation classes) is offered in selected secondary schools for 2 more years for students wishing to enter government-sponsored public universities.

Figure 1. Malaysian Educational System and Examinations.

2.1. Assessments and Public Obsessions Students completing primary education must sit for the UPSR (Primary School Assessment) Examination. After completing their lower secondary education, they must take the PMR (Lower Secondary Assessment) Examination. Upon completing their upper secondary education, students in the academic and technical tracks will sit for the SPM (Malaysian Certificate of Education) Examination (equivalent to the British O-level examination, now referred to as the GCSE), while students in the vocational track will sit for the SPMV (Malaysian Certificate of Education -Vocational) Examination. Students at preuniversity levels must sit for either the STPM (Higher School Certificate) Examination (equivalent to the British A-level examination) or for matriculation examinations accredited by the local universities. All public examinations at the end of the various levels, namely primary, lower secondary, upper secondary, and pre-university/matriculation, are controlled and administered by the Malaysian Examination Syndicate (MES) set up by the Ministry of Education. Although these public examinations were initially designed for selection purposes, student

96

Suan Yoong

promotions through primary to upper secondary grades were made automatic when Malaysia instituted 11-year free universal education. However, SPM results will determine whether or not students can further their education in government pre-university or matriculation classes, while the STPM matriculation exams remain the major selection examinations for entry into local government universities. Although promotion of students from grade to grade is automatic until the secondary level, the obsession with public examinations remains very great in Malaysia. Education as a social activity carries with it, after all, the social values specific to particular societies and cultures. Yet while in many societies having a prestigious career requires attending an elite university, in Malaysia (and countries like it) attending an elite university means passing examinations at the end of every tier, and ultimately passing the university entrance examination with a high enough grade. Furthermore, in order to pass the latter with a high enough grade to enter one of the best schools, students must begin preparing at an early age (Feiler, 1991). The final prize of an elite higher education being so attractive, the sociocultural value of assessment results over-ride their academic value. For example, though the UPSR and PMR have since been designed as diagnostic examinations, their results are still used primarily as a basis for channeling students into premier secondary schools or streaming them into or onto academic or technical-vocational tracks at the upper secondary level (Ministry of Education, 2004). Thus the SPM and STPM are high-stakes exams that determine entry into national universities and/or local private universities and colleges with overseas university collaboration programs. Through a policy of rewards and other forms of recognition dating back to the colonial era, education has, in other words, become the most significant instrument in determining one's future job (career) and consequent social status. Students therefore need to overcome a series of examination hurdles in their pursuit of the final prize, and this “scramble” to gain entry into premier elitist schools starts from an early age. It is only natural that examinations gained real legitimacy in Malaysia as a reliable means of facilitating and regulating upward social mobility.

2.2. Chinese High Schools Apart from the public secondary schools, there also exist 60 privately-managed Chinese independent high schools where most subjects are taught in Chinese. About 20% of the students who completed primary education in public Chinese primary schools continue their education in Chinese independent high schools. Chinese independent high schools offer 3 years of Junior Middle and 3 years of Senior Middle education. Upon completing each tier of high school education, students must sit for the UEC (Unified Examination Certificate for Chinese Schools) Examination conducted by a private Examination Board that is itself set up by a united Chinese school community organization, the Dong Jiao Zong. The Senior Middle UEC is recognized by over 300 universities and colleges around the world. With the UEC results, independent high school students can further their studies in overseas universities, especially in Taiwan, China, Japan, the United States, Canada, the United Kingdom, Australia and New Zealand, or enter local private colleges which have twinning programs with overseas universities. Some Chinese independent high schools offer dual programs, teaching in both Malay and Chinese, so that the students can sit for the PMR, SPM, STPM and/or UEC.

School-Based Assessments (SBA)

97

The UEC is an equally high-stakes exam as far as the students and parents of Chinese independent high schools are concerned, since it offers the opportunity for students to gain entry into overseas universities in the United States, Canada, the United Kingdom, Australia and New Zealand. Moreover, the Chinese independent high schools are also more stringent in their automatic promotion policy, retaining students longer at each grade level if they fail to perform adequately at that level. A significant number of premier Chinese independent high schools also control their student intake by conducting entrance exams and only admitting the top students from Chinese public primary schools. Thus an extraordinary emphasis is placed on education in Malaysia and especially by Chinese students and parents, necessitating a high-stakes examination system.

3. THE NEGATIVE INFLUENCE OF THE EXAMINATION ORIENTATION Public exams in Malaysia are essentially assessments of achievement in a number of core and elective subjects, focusing mainly on cognitive aspects of learning based on the national curriculum. Because the educational system is intensely competitive and hierarchical, only a small percentage of students are selected for the top stream and an even smaller percentage making it into a local university. For the Malaysian Chinese there is an added element: the Confucian heritage influences their perception of public examinations. For over a thousand years, formally conducted public examinations in China have given millions of families hope for a better future for their children and grandchildren, even though extremely few have fulfilled this hope (Ho, 1986; Lee, 1996). Upward social mobility has long been achieved in China primarily through education. The reliance on examinations, closely tied to the Confucian heritage, has thus become deeply embedded in Chinese culture. This traditional examination-orientation has also been extended to other East Asian countries, notably Japan and Korea, whose cultures were themselves long ago influenced by Chinese Confucian culture: hence university entrance exams are important annual events in Japan, Korea, Taiwan, Singapore, Malaysia and almost all overseas Chinese communities in the region (Rohlen, 1980; Sorensen, 1994). It is perhaps no exaggeration to say that an exam-orientation is the soul of Chinese educational culture, whether inside or outside China (Cheng, 1995). Furthermore, this ethos has been extended from the Chinese community to the other ethnic groups in Malaysia.

3.1. Pressure to Excel The consequences of this traditional examination-orientation has had a great impact on schools in various countries. The pressure to excel has resulted in schools’ attempting to control their student intake by selecting the most able students ahead of those with learning difficulties in order to raise or maintain test performance levels. The exam-orientation causes schools to adopt policies that are only useful for passing high-stakes exams rather than focusing more generally on skills useful in helping prepare students for later life. Indeed, “teaching to the test” by focusing intensively on test-preparation activities, using test-

98

Suan Yoong

preparatory texts and past test papers is a distinct instructional style that can be seen at all levels in most Malaysian schools. The obsession with teaching-to-the-test has also led to the phenomenon of curriculum "narrowing" on the part of teachers (William, 2001; quoted in Gregory and Clarke, 2003), that is, de-emphasizing or neglecting subject areas not tested or emphasized on exams, and precluding any supplementary curricula that encourage students’ all-round development. Classroom instructional and assessment strategies that mirror the content and format of the examination tend to be emphasized, thus leaving significant portions of the “non-examinable” content untaught or quickly skimmed through. Concerned educators worry that subject areas not tested (for instance, fine arts, physical education, music, and other “low-stakes” subjects) are accorded less class time or set aside altogether. Malaysian schools in particular are also devoting non-critical subject times to examination subjects, even in the early grades.

3.2. Exam-Orientation in Instructional Strategy and Private Tuition However, public examinations are obviously overly simplistic, overly objective, reductionist and thus very limited; these paper-and-pencil assessments tend to measure only a “myopically defined, narrow, test-defined set of skills” (Madaus and Clarke, 2001; quoted in Horn, 2003). They often fail to measure higher cognitive skills, a vital part of a person’s complex mind and personality. Public exams have been criticized for promoting rote memorizing of facts on the part of students, rather than problem-solving skills or (of course) creativity. Critics argue that this “spoon-fed” education legitimized by the examination culture produces students who can only regurgitate uncritically what they have acquired through rote learning. Such practices not only hamper real intellectual growth but mean the assessment is less able to select students with active, independent, creative minds, since those who succeed on exams may have passed by means of memorizing standard answers to stereotyped examination questions. This exam-orientation in instructional strategy has also led teachers to teach in ways that contradict the notions of sound educational practice, resulting in the “de-professionalization” of the teachers themselves. It is a common observation that practicing teachers who have been trained to teach problem-solving strategies are compelled by examination pressure to teach the subject matter in science and mathematics as knowledge to be memorized rather than as concepts to be applied to real problem-solving or inquiry. Similarly, science laboratory experiment sessions have also been compromised by the drive to prepare students for public examinations. In effect, high-stakes assessment brings with it a "default" philosophy of education that “holds in high regard a narrow bundle of knowledge and skills” (Gunzenhauser, 2003) to be memorized. Because of the power of this default philosophy, teachers in the exam-orientation climate may find themselves doing things that fall short of their visions of themselves as true educators, such as spending extra time drilling students on practice tests, de-emphasizing or eliminating untested subject matter, or teaching to the test (Jones, Jones, Hardin, Chapman, Yarborough, and Davis, 1999). Critics have also charged that over-obsession with exams is draining the fun out of schooling, and placing undue pressure on young children by increasing their levels of anxiety, stress, and fatigue. The anxiety has driven the majority of them to attend test preparation activities such as personal tutoring sessions, coaching, and extended test-oriented classes,

School-Based Assessments (SBA)

99

which in turn increase the children’s stress and fatigue. The demand for tutoring is spawned by the exam-oriented learning culture, that is, by the increasingly high levels of competition for social mobility (Bray, 2003), but also by the multi-faceted relationships between and among supply and demand patterns in the economy and other factors leading to the desirability of tutoring jobs (Kwok, 2004). Small wonder then that private tutoring that is parasitic on daytime schooling has become popular and is developing into a thriving enterprise in many countries, including more recently Malaysia. Moreover, students who view the passing of exams as an insurmountable barrier may ultimately drop out of school early. High-visibility examples of security breaches, rampant cheating in examinations and test-tampering have also been reported; leakage of examination questions has been reported as occurring sometimes even at the primary school level in Malaysia (Kong, 2007).

4. DILUTING THE INFLUENCE OF EXAM-OBSESSION: THE MINISTRY’S EFFORTS There have been large-scale school and educational reforms initiated at primary and secondary levels, with the goal of enlarging educational opportunities by reducing the hurdles at each level of education, minimizing the backwash of public examinations, and fostering continuous assessment or SBA. Since 1970, the Ministry of Education has initiated curriculum reforms, notably in 1976, 1983, 1995, and 1999, to enhance the quality of education by enriching and diversifying the curriculum, focusing on student-centered pedagogical changes, and stressing outcomes-based learning. However, none of these has involved any serious attempt to reform the public examination system in Malaysia until recently. Malaysia began to democratize its educational system in the early 1990s by introducing a policy of 11 years of free universal education. This new policy included a change in the way of grading public examinations at the end of primary (UPSR) and lower secondary (PMR) school: now the goal of grading would be diagnostic in nature rather than, as before, strictly serving a selective function—that is, promotion to the educational tier or level. However, these exams still retained their high-stakes character since their results were still being used to select students into premier secondary schools and stream them into science, arts, technical or vocational tracks at the upper secondary levels.

4.1. School-Based Projects and Formative Assessments Then in 1996 the Ministry of Education introduced the Secondary School Progressive Assessment or PKSM. This required students who sat for the PMR (lower secondary) and SPM (upper secondary) examinations to submit coursework output in the form of portfolios and projects for History, Geography, and Integrated Living Skills and Design. These were school-based assignments or projects graded by the school teachers following standardized marking schemes supplied by the Malaysian Examination Syndicate (MES), and the grades were submitted to the MES to be included in the final stage of PMR or SPM grading. Similar practices had also been instituted by the Chinese Independent Schools Examination Board for

100

Suan Yoong

the UEC examination. For the PMR examination, oral assessments are also planned for Malay and English languages, while practical assessment is planned for science. Unfortunately, grade inflation and cheating in the form of non-authentic or plagiarized work are so widespread that the school-based assignments or oral practice sessions have been criticized heavily by the public. Teachers are unhappy because these have also increased their workload considerably. Moreover, the effect of both practices on reducing examination pressure was minimal. However, both of them—school-based assignments and oral practice sessions—are still in use today. Most of the Ministry’s subsequent efforts seemed to be aimed at circumventing the system by enhancing the teaching and learning process as a means to reduce the pressure of final public examinations on students, and to reinforce the importance of consistent learning throughout the school years. This initiative was promoted through PKBS (School-Based Progressive Assessments). PKBS was a program of formal assessments in the form of weekly, monthly or term tests, projects and regular class assignments that were set by the respective subject teachers and conducted throughout the year, in order to evaluate pupils’ progress at all levels in the cognitive, affective and psychomotor domains. These assessments were done through classroom observations, orals, and written works. The information from these non- standardized assessments was used to reinforce the instructional process. However, teachers were unhappy with this program because their workload was increased considerably. Moreover, the results of these assessments were neither included in the public examination grades nor needed for the selection and promotion of students to the next grade. As a result, the teachers kept reverting to their old ways of teaching to the test.

4.2. The Planned New Role of ICT in Assessment and Smart Schools Then in 1996, Malaysia began to plan its own Multimedia Super Corridor (MSC) as a dynamic hub for the global ICT industry, in order to leap forward into the information and knowledge age. The Multimedia Super Corridor is a 15km by 40km area, encompassing the Kuala Lumpur City Centre and Kuala Lumpur International Airport, which will be developed as a high growth centre. MSC Malaysia now hosts more than 1000 multinationals, including foreign-owned and home-grown Malaysian companies, focused on multimedia and communications products, solutions, services and research and development (MSC Malaysia, 2008). The Malaysian government has equipped core sectors of MSC Malaysia with highcapacity global telecommunications and logistics networks. Six innovative flagship applications have been developed in MSC Malaysia to accelerate its growth, among them the Smart School flagship application (Malaysia, 1997). The Malaysian Smart School Flagship was premised on the strong belief that information and communications technology is a key enabler, that it can stimulate a desire to learn in everyone. Deemed one of the most forwardlooking ICT-mediated learning initiatives in the world, the Smart School Flagship was established in order to reinvent the core teaching-learning processes by integrating ICT with the curricula, pedagogy, and assessment practices of Malaysia’s schools. The goal would be to help students learn more effectively and efficiently while simultaneously gaining a competitive position in an increasingly globalized, ICT-driven market. Computerized management software was developed to help the “smart schools” manage more efficiently and effectively the resources and processes required to support teaching-learning functions,

School-Based Assessments (SBA)

101

thus enabling students to practice self-accessed and self-directed learning, and making the teaching-learning process easier, more fun and more effective. The Smart School system was piloted in 87 schools beginning in 1999, and will have been expanded to include 10,000 schools by 2010. A key element of the Smart School system is a smart assessment system that involves a significant departure from the “current assessment system which is characterized by a single delivery (one session only) at a fixed time of the year to students of a given age group” (Malaysia, 1997). Within this system, and integrated with ICT, criterion-referenced on-going assessment would be emphasized to provide a more holistic and accurate picture of a student’s performance. Teachers, students and parents will be able to access on-line assessment items as well as get a more accurate picture of a student’s achievement, readiness, progress, aptitude, learning styles, and abilities. Smart School assessment will be flexible and learner-friendly, while the quality of the assessment information will be assured by using multiple instruments and approaches. This will lead to a flexible assessment and certification system that involves using a combination of classroom-based, school-based, and centralized assessments, using authentic, alternative and performance-based assessment instruments and approaches. The Smart School assessment system not only attests to a student’s cumulative accomplishments but will also promote continued improvement on a lifetime basis. According to the plan, assessment for achievement tests will be implemented first, and assessment for readiness, progress and aptitude tests will be developed as teachers grow more capable in the use of the skills and tools that will be made available to them. The Smart School management system would have the capability to store assessment records that document students’ accomplishments during their entire educational career, to be identified by a single ID (Malaysia, 1997). However, evaluation of the Smart Schools program revealed that although ICT holds great promise, the reality in the Malaysia Smart schools can hardly match or reflect the stateof-the-art in technological advances, and the application of ICT to instruction, assessment, and testing in the Smart Schools is still in its infancy (MDC, 2004; Frost and Sullivan, 2004). Therefore, despite these proposed innovative efforts, the smart assessment system has yet to surface in the Malaysian school system, and even in the pilot Smart schools there seems to have been no noticeable change in the Malaysian examination culture.

4.3. Blueprint for Educational Development In 2001 the Ministry of Education unveiled a 10-year educational development plan (Ministry of Education, 2001), reiterating the need to strengthen SBA as a way to reduce emphasis on centralized examinations while integrating the elements of criterion-referenced assessments into existing assessments and strengthening formative evaluation. The practical assessment of science in the UPSR examination was proposed; SPM would be made an open certificate examination, allowing students to takes at least 6 core subjects and open up more electives, with no limit on the number of subjects to be taken (Ministry of Education, 2001: 99). While this system allows the students to choose examination subjects based on their own interests and potential, the effort was counter-productive as many students chose to take many more examination subjects than were required - some as many as 17 per sitting.

102

Suan Yoong

More recently, the Ministry of Education unveiled the Blueprint for Educational Development 2006-2010 in an effort to reform the educational system and "improve the longterm international competitive edge of Malaysia” (Ministry of Education, 2006). The Blueprint is specifically for the Ninth Malaysia Plan: 2006-2010, which stresses that the implementation of the SBA system will be accelerated to enable continuous evaluation of, and support for, creativity as well as analytical problem-solving skills (Malaysia, 2006). This Blueprint specifically notes that “presently emphasis has been placed on summative evaluation that focuses on student achievements and not on identifying strengths and weaknesses in teaching-learning” (Ministry of Education, 2006). According to the Blueprint, the assessment system will be modified so as to make schooling less oriented toward examinations; this will be done by reducing the number of public examination subjects, while other subjects will be assessed at the school-based level. The Blueprint also charts the way toward achieving a new examination reform, a new examination policy that will strengthen SBA because it will be holistic, task-based and criterion-referenced, and will also promote character-building. Specific recommendations include changing the emphasis of public examinations: rather than content-based they will now be skill/ability-based; they will assess ability rather than achievement. Also, there will be more room for alternative assessments and for the quality assurance of assessments, and national benchmark standards will be established (Ministry of Education, 2006).

5. SCHOOL-BASED ASSESSMENT: THE ROAD MAP “We will empower teachers to conduct SBA and give them the tools and training to carry it out effectively…School assessment will be given due recognition and acknowledgement.” These are the words of Dr. Adi, the Malaysia Examinations Syndicate Director of Examinations, who delivered the keynote address at a forum on educational assessment in Kuala Lumpur, Malaysia (Adi, 2007; Simrit Kaur, 2007). According to Dr. Adi, the ultimate aim is to reduce over-reliance on data about students obtained through centralized public examinations. “There has been too much attention given to public examinations. School is now looked upon as a process of preparing students for public examinations. School assessment has lost its significance and the outcome is not taken into account in students’ final grade,” said Dr. Adi. “The focus should not only be on academic aspects. We should also assess students’ personal and character development and involvement in extra-curricular activities.” (Bernama, 2007). Accordingly, a new assessment system for schools will be implemented by 2010, in order to make way for a major change in the Malaysian educational system: centralized examinations will give way to a combination of centralized examinations and SBA. For the first time, Dr. Adi has gone a step further to announce that, following the proposed introduction of SBA, the Malaysian Examinations Syndicate proposes the abolishment of two major public examinations – the UPSR (at the end of the primary level) by 2012 and the PMR (at the end of lower secondary level) by 2015. Five forms of assessment for students in the national education system have been proposed:

School-Based Assessments (SBA) • • • • •

103

SCHOOL assessment – planned, developed, conducted, examined and reported by teachers in schools. CENTRAL assessment – involving standardized instruments, data analyses and guidelines provided by the MES. CENTRAL examinations – fully conducted by the MES. PSYCHOMETRIC tests – to measure students’ innate abilities. PHYSICAL activity assessment –to evaluate and measure students’ performance and involvement in extra-curricular activities.

Dr. Adi nonetheless noted that this will change the whole culture of knowledge acquisition in Malaysia, for students may begin to feel that learning can actually be fun. However, he also said that it would take time, because the transition would require several stages—from getting input from academicians, teachers, parents and students to choosing the right methods of assessment in the Malaysian context (Koh, 2007a).

5.1. Public Reactions Public reactions to the announcement ranged from expressions of full support to those of having reservations. The following news report is typical: A parent, V. Vasantha, said she had reservations about teachers being objective when it came to school-based assessment. Psychometric testing, for example, is something new and will require training for teachers, or they could come out with "10 conclusions about one student." And could teachers cope with assessing so many students? There may be adjustment problems but "We do not want any bias in the grading or assessments, so the standards must be comparable for schools to want to compete with each other" (Koh, 2007a). [I’m assuming these news reports were translated by author from Chinese or Malay?] No! it was actually from the Online news report - Yoong

Many had urged the Education Ministry to be cautious in its implementation of the system, saying it would really need to depend on teachers’ performance and the guidelines for all schools when it came to assessment (Chok, 2007; Koh, 2007b, Kong, 2007). "It is good that it will be more school-based but there should be a guarantee that there is a standard for all. Some others find hope in the Education Ministry’s plan to reform the assessment system for students, which will shift the emphasis from examinations to continuous forms of SBA such as aptitude, personality and diagnostic tests” (Koh, 2007a). The following excerpt from a local news report records a typical reaction to the recent proposal to scrap the UPSR and PMR examinations: “I remember a time when students had to do well enough in Primary school, passing all the exams before they were allowed to go to Secondary school. And then you had to pass your SRP before you could move to Form 4. And then our Government decided to make things easier for students, dumbing down the content of the education, and now we are even thinking of removing Public Examinations! Do you really trust individual schools to set their own test and examinations? I don’t. Not that I don’t trust teachers, but it’s more that I don’t trust the schools. There will be leaks in the papers, and with no proper guidelines for marking and

104

Suan Yoong grades, what will happen is that on paper the Government will suddenly show how “Smart” our students are, when in reality our educational system is going down the drain. I think we should be making our exams harder, getting some quality students out there, not people who can get an A in English yet have really terrible grammar and not be able to hold a conversation. Gone are the days when our education meant something, and when students took exams seriously. Back in the day, you studied for the SRP1 because if you failed, you would be held back a year!” (Simrit Kaur, 2007)

5.2. A Culture with an Entrenched Examination-Orientation In Asia, public examinations have always meant an extremely high status for education. An understanding of the cultural dimensions of East Asian societies, including Malaysia, and of the role in them of education will prove helpful in understanding the deeply-entrenched examination-oriented culture in Malaysia, especially when it comes to educational evaluation or assessment—for this is seen by the community as being very important, as having high stakes. In Malaysia, different sectors of the community—whether teachers, parents, students, administrators, businessmen, political leaders, or even professionals—all seem to share a very similar cultural attitude toward what education should be and how children should be educated. The fear of losing a competitive edge in the international arena also foster concerns about the quality of education, the latter being most often cast in terms of education attainments, competitiveness, world class standards, and academic excellence. Thus, the Ministry of Education has always focused on enhancing the knowledge, skills and expertise of future generations: these will be needed to support a knowledge-based economy and realize Malaysia’s vision of becoming a developed nation by the year 2020. The centrality of education in East Asian societies with identifiable common goals has fostered a virtual coalition between parents and teachers in the quest for excellence in education. As a result, students are required to adapt themselves to the system, and indeed adaptability is an essential crucial quality in the Asian education process. The dialectics of work and play, happiness and pain, give and take, tolerance and gain, ease and difficulty, are deep-rooted in these societies and could explain the rationale for adaptability behind many educational practices. Cheng (1995) noted that Chinese parents and students prefer to adapt to the system instead of intervening in the educational process, and thus that they do (like their children) expect learning to continue beyond school hours. In such a socio-culturaleducational context, the students have little room for any activities apart from formal academic learning, and the latter can take place in or out of schools, can extend into private homes (including those of private tutors) and after-school hours. A clear percentage of students adapted well and achieved an excellent academic performance. Unfortunately, a large proportion of those who could not make it were “legitimately” ignored for failing to do so. Given the fact of this Malaysian culture with its deeply-entrenched examinationorientation, it is not surprising that the public generally supports public exams as the best way to maintain objectivity, higher standards, and stricter accountability in the educational system. The Malaysian public feels strongly that examinations, while not perfect, are probably the 1

Note: SRP examination was replaced by PMR in the 80s.

School-Based Assessments (SBA)

105

best way to hold all students accountable for learning what they are expected to learn during their school careers. Given this entrenched attitude, it is clear that the Ministry of Education will have a hard time reducing (let alone eradicating) the public’s obsession with examinations as the fairest means of student evaluation.

CONCLUDING REMARKS One must be clear about the policy goals of examination reform: know the strengths and weaknesses of all testing instruments; recognize the political, social, and educational tradeoffs involved in using one form of assessment over another; and, most importantly, make no demand of any testing instrument for which it was not designed. One-shot assessment, no matter what the format, cannot address multiple needs (Goertz and Duffy, 2003). Public examinations are designed to meet certain purposes, notably to provide reliable and comparable scores for individuals as well as groups. They are cost-efficient in terms of development, scoring, and administration, and sample a broad set of curriculum objectives within a limited time frame. In general, the competencies tapped are those that lend themselves to being assessed in these ways, while aspects of learning that cannot be assessed under such constrained conditions are not addressed. To design new kinds of assessment methods as alternative approaches to large-scale assessment for capturing the complexity of cognition and learning will require breaking out of the current paradigm. It is now a common practice to characterize student achievement in terms of multiple aspects of proficiency rather than relying on a single score. It is also possible to chart students' progress over time instead of simply measuring performance at a particular point in time. Schools should use multiple paths in evaluating and look for alternative forms of valued performance. Many of these methods are not yet widely used because they are not easily understood or packaged in accessible ways. Furthermore, having a broad array of models does not mean the measurement problem has been solved. There is a need to focus, in psychometric model-building, on the critical features of achievement that we want to assess. Information and communication technologies are helpful in removing some of the constraints that have limited the practice of evaluation in the past, and new ICTs are expanding the types of constructs that can be tapped through assessment. By enriching assessment situations through the use of multimedia, interactivity, and control over the stimulus display, it is possible to evaluate a much wider array of constructs than was previously possible. The Smart School assessment system offers opportunities in this direction. New capabilities afforded by technology include directly assessing problem-solving skills, making visible the sequences of actions taken by learners in solving problems, and modeling complex reasoning tasks (National Research Council, 2002). Technology also makes possible the simplification of teachers’ tasks. (Chudowsky and Pellegrino, 2003) Introducing SBA into the public examination system improves the validity of assessments. Some important outcomes which cannot be evaluated within the context of a written examination can readily be assessed through SBA (e.g., practical work, theatres and workshops, research projects, portfolios of creative work, active artistic or design projects). SBA must be designed so that evaluations become an integral part of teaching and learning within school subjects and not an ‘add-on’. There should be no significant increase in the

106

Suan Yoong

workload of teachers. In order to carry out SBA in a rigorous and consistent manner, teachers will need to receive appropriate training and support. They need to develop a wider range of assessment skills which will enable them to better facilitate the implementation of SBA. In the implementation of any educational reform, the capacity of the system to support change in practice must be addressed. Research on SBA programs shows that clear goals and incentives are necessary but not sufficient to motivate teachers to reach their school's student achievement goals. Teacher motivation is also influenced by the presence of various capacitybuilding conditions, most obviously the opportunity for meaningful professional development. In addition, teacher knowledge and skills related to improved instruction are important (Sloane and Kelly, 2003). Moreover, teacher education must be funded along with the various complementary testing programs. In other words, schools need knowledge, human and financial resources to successfully implement any reforms. The Ministry of Education’s latest effort in examination-system reform will involve a huge effort to overcome a deeply-entrenched exam-taking culture. Cultural reform is very difficult to achieve; deeply-entrenched beliefs and practices are hard to eradicate, let alone transform. The public needs to be more broadly educated about the possible range of testing and test-interpretation formats, if people are ever going to shake off their traditional concept of assessment and embrace a new assessment culture that is forward-looking, psychometrically sound, and humanistic. Such a reform must be accompanied, then, by a structural change in the society and culture as well as in the educational system. The highly centralized Malaysian system of education must be further decentralized and democratized, and schools must be appropriately empowered. This depends, once again, on a clearer understanding of the deeply-entrenched examination-orientation of Chinese and Malaysian culture, the tendency to see exam-taking as a kind of one-shot, high-stakes “gamble.” An example showing that progress in this direction is possible is the PTS (Level One Assessment) aptitude test, introduced to select grade 3 students so that they might skip grade 4 and proceed directly to grade 5. (This was in line with the new education policy which stipulated that primary education can be completed in five to seven years.) Pupils were tested on their abilities and potential in verbal, quantitative and thinking skills. Of course, this PTS inevitably became a high-stakes test in the eyes of teachers and parents as well as students; fortunately, due to the pressures it generated, it was discontinued two years later. On the other hand, allowing almost everyone to enter university does not necessarily reduce examination pressure, as students will still be competing to get into the best universities. Thus, irrespective of what changes are made in the assessment mode, the newly-introduced measure always becomes high-stakes unless there is a strong will to dilute and diversify the incentives and reward systems that bind upward social mobility so tightly with education. We must be able to deal with the fact that if and when School-Based Assessments are given a more significant weight in university entrance selection, they, too, will become relatively high-stakes—but hopefully relatively less high-stakes than winner-take-all exams. We are still a long way, in Malaysia, from the goal of overcoming a deeply-entrenched examination-orientation culture. There are no simple solutions. The key point to keep in mind is that the final goal of standards and assessment systems is not the race for ever-higher scores, but the race for students’ solid preparation for the workplace or postsecondary education. Furthermore, even if the SBA program is increasingly implemented, accountability remains an important problem to be reckoned with, most notably in the context of the

School-Based Assessments (SBA)

107

documented gaps in achievement between students of different economic, ethnic, and racial backgrounds.

REFERENCES Adi B. T. (2007). A proposal towards a more holistic education assessment system in Malaysia. Keynote address at International Conference on Assessment (KLICA) of the National Educational Assessment System (NEAS): Looking Ahead, organized by the Examinations Syndicate, Ministry of Education, Malaysia. 7-9 May 2007 at Sunway Resort Hotel and Spa, Pelaling Jaya, Malaysia. Bernama. (2007). New Assessment System for Schools To Be Implemented In 2010. Bernama News Agency, Malaysia. May 17, 2006. Bray, M. (1999). The shadow education system: Private tutoring and its implications for planners. Paris: UNESCO / IIEP. Bray, M. (2003). Adverse effects of private supplementary tutoring. Paris: UNESCO / IIEP. Chen, J. (1996). Cram Culture - A Fresh Look at the Buxiban. Retrieved July 15, 2007, from http://home.sina.com/sinorama/1196/article2/index.html. Cheng, K. M. (1995). Excellence in education: is it culture-free? Keynote address presented at the 9th annual meeting of the Educational Research Association, November, 22–24, Singapore. Cheng, K. M. (1990). The culture of schooling in East Asia. In N. Entwistle (Ed.), Handbook of education ideas and practice (pp. 163-173). London: Routledge. Chew, S. B., and Leong, Y. C. (Eds.). (1995). Private tuition in Malaysia and Sri Lanka: A comparative study. Kuala Lumpur: University of Malaya, Department of Social Foundations in Education. Chok S. L. (2007). It takes courage to cut the exam knot. New Straits Times, June 3, 2007. Chudowsky, N. and Pellegrino, J. W. (2003) Large-scale assessments that support learning: what will it take? Theory Into Practice, Winter, 2003. Feiler, B. S. (1991) Learning to bow - Inside the heart of Japan. New York: Ticknor and Fields. Foondun, A. R. (1992). Private tuition in Mauritius: The mad race for a place in a ‘five-star’ secondary school. IIEP Research and Studies Programme, Monograph Vol. 8. Paris: IIEP. Foondun, A. R. (1998). Private tuition: A comparison of tutoring practices at the primary level in Mauritius and some South East Asian countries. Bangkok: UNICEF East Asia and Pacific Regional Office. Frost and Sullivan. (2004). Benchmarking of the Smart School Integrated Solution. Retrieved July 15, 2007, from http://www.msc.com.my/smartschool/downloads/benchmarking.pdf. Goertz, M. and Duffy, M. (2003). Mapping the landscape of high-stakes testing and accountability programs. Theory into Practice, Winter, 2003. Gregory, K. and Clarke, M. (2003) High-stakes assessment in England and Singapore. Theory into Practice, Winter, 2003. Gunzenhauser, M. G. (2003). High-stakes testing and the default philosophy of education Theory into Practice, Winter, 2003. Harnisch, D. L. (1994). Supplemental education in Japan: Juku schooling and its implications. Journal of Curriculum Studies, 26(3), 323-334.

108

Suan Yoong

Ho, D. Y. F. (1986). Chinese patterns of socialization: A critical review. In M. H. Bond (Ed.), The Psychology of the Chinese People. Hong Kong: Oxford University Press. Horn, C. (2003). High-stakes testing and students: stopping or perpetuating a cycle of failure? Theory into Practice, Winter, 2003. Hussein, M. G. A. (1987). Private tutoring: A hidden educational problem. Educational Studies in Mathematics, 18(1), 91-96. Jones, G. M., Jones, B. D., Hardin, B., Chapman, L., Yarborough, T., and Davis, M. (1999). The impact of high-stakes testing on teachers and students in North Carolina. Phi Delta Kappan, 81(3), 199-203. Kim, I. (2000). Private tutoring in Korea: A cultural analysis. Paper presented at the Annual Conference of Comparative and International Education Society (CIES), March 2000, San Antonia, Texas, USA. Koh L. C. (2007a) School-based assessment: Thumbs up mostly, but some foresee teething problems, NST Online. Retrieved March 6, 2007, from http://www.nst.com.my/ NST/index_html Koh, L. C. (2007b). Taking formal education beyond exams. Retrieved March 22, 2007, from http://www.nst.com.my/Current_News/NST/Sunday/SundayInterview/20070304082920/ Article/index_html. Kong S. H. (2007, September 14). Parents want probe into exam ‘leaks’. The Sun. p.8. Kwok. P. (2004). Examination-oriented knowledge and value transformation in East Asian cram schools. Asia Pacific Education Review 2004, 5(1), 64-75. Lee, W. O. (1996). The cultural context for Chinese learners: Conceptions of learning in the Confucian tradition. In D.A. Watkins and J.B. Biggs, J.B. (Eds), The Chinese Learner: Cultural, psychological and contextual influences, (pp. 25-41). Hong Kong: Comparative Education Research Centre and Victoria, Australia: The Australian Council for the Educational Research. Madaus, G., and Clarke, M. (2001). The impact of high-stakes testing on minority students. In M. Kornhaber and G. Orfield (Eds.), Raising Standards or Raising Barriers: Inequality and High Stakes Testing in Public Education (pp. 85-106). New York: Century Foundation. Malaysia, Government of (1997) The Malaysian smart school: An MSC flagship application, a conceptual blueprint. Kuala Lumpur: Government of Malaysia, Smart School Project Team. Malaysia, Government of (2006). The Ninth Malaysia Plan (2006-2010). The Economic Planning Unit, Prime Minister's Department, Putrajaya. 2006. p. 255, Ch.11 on Enhancing Human Capital. Kuala Lumpur, Malaysia: Government Printing Press. Ministry of Education. (2001). Pembangunan Pendidikan 2001-2010: Perancangan Bersepadu Penjana Kecemerlangan Pendidikan. (Educational Development 2001-2010: Integrated Planning towards Excellence in Education). Kuala Lumpur, Malaysia: Ministry of Education. Ministry of Education. (2004). The Development of Education: National Report of Malaysia. Kuala Lumpur, Malaysia: Ministry of Education. Ministry of Education. (2006). Pelan Induk Pembangunan Pendidikan 2006-2010 (Blueprint for Education Development 2006-2010). Kuala Lumpur, Malaysia: Ministry of Education.

School-Based Assessments (SBA)

109

MDC, Multimedia Development Corporation. (2004). Multimedia Super Corridor: Impact Survey 2004. Multimedia Development Corporation Bhd, MSC Headquarters, Malaysia: www.msc.com.my. MSC Malaysia. (2008). MSC Malaysia Status / MSC Malaysia Status Check. Retrieved July 15, 2007, from http://www.mscmalaysia.my/topic/MSC+Malaysia+Status+Companies. National Research Council. (2002). Technology and assessment: Thinking aheadProceedings of a workshop. Washington, DC: National Academy Press. Rohlen, T. P. (1980). The juku phenomenon: an exploratory essay. Journal of Japanese Studies, 6(2), 207-242. Simrit Kaur. (2007, May 9). Assessment to replace UPSR and PMR. The Star. Sloane, Finbarr C. and Kelly, Anthony E. (2003). Issues in high-stakes testing programs. Theory into Practice, Winter, 2003. Sorensen, C. W. (1994). Success and education in South Korea. Comparative Education Review, 38(1), 10-35. Zeng, K. M. (1999). Dragon gate: Competitive examinations and their consequences. London: Cassell.

In: Educational Evaluation in East Asia Editor: Samuel S. Peng and John Ch-Kin Lee

ISBN 978-1-60692-887-5 © 2009 Nova Science Publishers, Inc.

Chapter 8

CURRENT ISSUES REGARDING THE SCHOOL EVALUATION SYSTEM IN JAPAN Yasuhiko Washiyama Tokyo Gakugei University, Japan

1. HISTORICAL DEVELOPMENT OF THE SCHOOL EVALUATION SYSTEM IN JAPAN Evaluation of schools in Japan was conducted according to the “Local Inspector System” until the end of World War II, and under the supervision of local “Boards of Education” thereafter. Rather than actual evaluation, however, these systems were more in the nature of compliance-oriented supervision and guidance provided to each school on an individual basis, and with no effects outside the school in question. School “evaluation” in the true sense of the term has only taken off in Japan in the last 10 years or so. The pursuit of deregulation and decentralization has occasioned a review of the regulations on school administration, which prescribe the authority held over schools by Boards of Education. This review has resulted in an expansion of schools’ discretionary powers, making the concept of “autonomous school administration” a reality. The introduction of the school councilor system in 2000 brought school administration and development beyond the limited control of the principal and teaching staff and enabled active participation by students’ guardians—here defined as their parents or primary caretakers—as well as the local community. In the context of these trends, the need emerged for a system to evaluate each school’s autonomous activities and to effect improvements in educational quality. As a first step towards promoting evaluation within schools, the “Regulative Standard for School Establishment” applied by the government to new kindergartens, elementary, junior and senior high schools was revised in 2002 to include new provisions on “self-inspection and self-assessment.” Next, a clause to the effect that all schools operating within the established framework of compulsory education, hereafter called “compulsory education schools,” should be subjected to external evaluation was included in the 2005 “Basic Policies for Economic and Fiscal Management and Structural Reform.” In that same year, the Central

112

Yasuhiko Washiyama

Council for Education released its report on “Redesigning Compulsory Education for a New Era,” which proposed the formulation of guidelines in order to enhance evaluation systems for schools. The “Evaluation Guidelines for Compulsory Education Schools” were released in 2006, marking a new beginning for school evaluation in Japan.

2. EVALUATION GUIDELINES FOR COMPULSORY EDUCATION SCHOOLS I would like to introduce three concepts which provide the framework for the Evaluation Guidelines for Compulsory Education Schools: Goals, Methods and Indicators. The following four points are listed under “Goals”: (1) To establish a Plan-Do-Check-Act management cycle, realizing systematic and continuous improvement in each school through setting goals, monitoring progress and reviewing initiatives; (2) To make the results of both self-evaluation and external evaluation public, creating an open school atmosphere founded on the understanding and involvement of guardians and the wider community; (3) To clarify responsibilities for both education and management, enabling schools and their teachers to discharge their duties to the full and the school’s founding body to provide support and establish the conditions for operation of schools in accordance with evaluation results, thereby assuring and enhancing educational quality; (4) To apply evaluation of lessons and of teaching staff to the development of scholastic achievement guarantees, clear academic goals and guidelines, and school administration systems that are directly connected to overall school evaluation processes. “Methods” includes the following three points: (1) Self-evaluation (goal-setting, collecting and consolidating data, implementing evaluations, producing evaluation reports, identifying points for improvement); (2) External evaluation (establishing an external evaluation committee, implementing evaluations, producing evaluation reports); (3) General information for guardians and community members, and general measures for improving the school, provided by the school’s founding body. Ten points are listed as “Indicators”: (1) Curriculum and academic guidelines; (2) student behavioral guidance; (3) student career and further education guidance; (4) safety management; (5) health management; (6) special needs education; (7) organizational operations; (8) teacher training; (9) collaboration with guardians and members of the local community; (10) facilities.

Current Issues Regarding the School Evaluation System in Japan

113

3. SCHOOL EVALUATION AS A MEANS OF STRUCTURAL REFORM OF THE EXISTING COMPULSORY EDUCATION SYSTEM Until recently, evaluation of compulsory education providers was practiced from a “compliance” perspective. Inspections were carried out to ensure that the curriculum followed the Education-Ministry-approved course of study, that the required number of class hours was being offered, and that the school was being operated correctly in accordance with legal stipulations. Educational quality was maintained and enhanced through the application of uniform national standards to school “inputs and outputs.” The recent round of reforms has been characterized by the shift to a quality assurance approach, focusing on evaluation of educational outcomes. The initial “input stage” involves laying the foundations for school operations: addressing legal requirements, setting goals, ensuring fiscal stability, and so on. In the “process stage” each school must develop autonomous initiatives, assuming greater authority over school activities, setting specific objectives and finding ways to achieve them in collaboration with the community. The “outcomes stage” entails continuous and progressive inspection and evaluation by reference to educational outcomes, addressing matters such as learning effectiveness and the development of teachers’ professional capabilities. This process requires the formulation of a new evaluation system within each individual school. This is to be a three-tiered system comprising the school’s “self-evaluation” of its own activities and operations by reference to its own educational objectives, “external evaluation” by students’ guardians, local residents and other stakeholders, and “third party evaluation” conducted objectively and scientifically by outside groups unfettered by any association with the school. The system, then, is founded first and foremost on the “self-evaluation” process. The combination of quantitative evaluation, based on numerical targets and performance indicators, with qualitative evaluation that focuses on activity indicators reflects the importance of properly evaluating educational processes, each child’s degree of development, stakeholder satisfaction, and the nature of linkages with the wider community. To date, self-evaluation has been implemented in over 95% of all schools in Japan’s compulsory education system, while around 80% have undergone external evaluation. It is believed that an organization will be established to undertake third-party evaluation following analysis of the outcomes of trial programs conducted in the 2006 and 2007 school years. The challenge is how to establish school evaluation as part of the Plan-Do-Check-Act management cycle, ensuring that the system of setting goals, monitoring progress and reviewing initiatives will generate systematic and ongoing improvement. It is vital that the items for evaluation, the evaluators themselves, and the results of evaluation all be structured appropriately. This structuring would clarify what needs to be achieved, by whom, and when, facilitating strategic improvement in and of schools through across-the-board collaboration.

4. CONCERNS ABOUT THE SCHOOL EVALUATION SYSTEM While aiming for improvement of educational activity across the board, the central concern of school evaluation is obviously that of guaranteeing sound scholastic ability on the

114

Yasuhiko Washiyama

part of students. Scholastic ability is thus employed as a principal indicator in the evaluation process, which highlights the issue of degrees of scholastic achievement. In April of 2007, a national test of scholastic ability was conducted in Japan for the first time in 43 years. Students in the sixth year of elementary school and the third year of junior high school were tested in Japanese and arithmetic/mathematics, with almost all national and other public school students participating along with 60% of those in private schools. This test is an important tool for comprehending the true current state of scholastic achievement in Japan and applying this knowledge in guidance and instruction activities. However, there is also the potential for this test to be used in ways which are highly problematic. One such case has already emerged in Adachi Ward, part of the Metropolis of Tokyo, which has been conducting its own scholastic ability test since before the national test was instituted. It was revealed that teachers proctoring the test at schools in Adachi Ward had been communicating answers to students taking the test by means of signals or cues. The worst-performing area in Tokyo, Adachi Ward authorities had been under pressure to improve test results. They sought to achieve this goal by making all the schools in their jurisdiction compete in the test, publishing a ranking of schools based on students’ test results, reflecting these results in budget allocations to schools, and encouraging families to refer to the results when choosing schools to attend. The improper conduct during the test on the part of both teachers and students was a direct outcome of this competition-based approach. As this example shows, scholastic ability tests run the risk of being used to exacerbate competition, generate hierarchies and create disparities between and among schools – outcomes not envisaged in the original concept of school evaluation. This is an important problem and one which calls for careful consideration; school administrators must be able to approach the publication and use of test results in a manner which accords with the spirit of honest and objective school evaluation. If asked to identify three more general problems which may arise from the new school evaluation system, many would name the following: (1) greater disparities between and among different schools and teachers and more cases of loss of motivation and mental illness in the teaching profession; (2) encroachment by the evaluation workload on the time teachers usually spend with their students and on studying and preparing teaching materials; (3) the treatment of evaluation as a routine practice, carried out as a matter of form, with content being buried in the demands of paperwork. Measures must be instituted to forestall these problems. Evaluation is often thought of as the responsibility of principals, head teachers and other school administrators, and not the concern of either mid-career or recently-qualified teachers. However, it is important that evaluation be pursued as a school-wide initiative. Teachers must look beyond the confines of their own classroom, adopting a broad and multi-layered perspective which incorporates the overall organizational activities of the school and shares ideals regarding the school’s direction. Teachers must work together with guardians and members of the wider community in activities aimed at enhancing not just students’ scholastic ability but also their study habits and lifestyles. The crucial point here is to use the evaluation process as a means to develop mutual trust and confidence among teachers.

Current Issues Regarding the School Evaluation System in Japan

115

5. MEASURES TO ADDRESS THE PROBLEMS Change is the essential purpose of school evaluation. Teachers change, pupils change, schools change – all to attain a greater capacity for self-directed learning and research. This capacity is further strengthened through the development of an open and accessible school environment, open and accessible to those outside as well as within the school. Understanding and support are the most powerful forces behind all educational activity. School reform and changes in the consciousness of students, teachers, guardians, administrators and all school community members must develop as mutually-defining processes. Evaluation is characterized by the proximity between evaluators and those being evaluated. I would like to suggest the following three points as means to maximize this proximity and make school evaluation a truly meaningful system: (1) Implementation of “shadowing”: one-day shadowing of each class and each individual teacher. (2) A conceptual shift from school evaluation to “school diagnosis.” (3) Fully developing a collegial spirit among teaching staff through formative evaluation. Shadowing refers to a practice whereby evaluators stay close to, or shadow, those being evaluated, facilitating close observation. Shadowing of a class group, for instance, enables assessment not only of in-class lessons but also of pupils’ attitudes, movements and everyday interactions during breaks and meal times. Over a two-day period, the evaluator can thereby gain a complete picture of the teacher’s and students’ educational conduct, which will contribute to the envisaging of current conditions and challenges for the school as a whole. The workings of a school, after all, are the sum of its individual teachers. Viewing each teacher’s activities in their entirety helps to bring concrete issues into focus, thus enabling formative evaluation based on dialogue, generating relationships of trust, and making it possible to conduct evaluation in accordance with each teacher’s individual characteristics and distinctive qualities. Surely there is no greater force for improvement than this.

CONCLUSION School evaluation must not be limited to mere questionnaire-based data collection. It is crucial that external interests and third parties be included in practice-based school diagnosis activities, the results of which are used in feedback to teachers; here opportunities must be provided for expression of opinions and criticism of the results, to further enable their verification. Senior high schools in Tokyo now publish evaluation details produced through this kind of consensual process, an experiment which appears to have been received favorably. The great Chinese thinker Laozi said that “the highest good is like water.” Ideally, the Plan-Do-Check-Act evaluation cycle should also be like water: fluid and supple, able to move unhampered through any location, following the current to permeate the lowest spaces. The flow of this highest good to schoolchildren, to teachers, to guardians and to community

116

Yasuhiko Washiyama

members is the quintessential ideal of all educational evaluation. Ultimately, in the river’s lower reaches, it is the task of school principals to bring everything together – both problems and achievements – and use them to put a new cycle into motion.

REFERENCES Ministry of Education, Culture, Sports, Science and Technology-Japan. (2007). Regulative Standard for School Establishment, Retrieved September 28, 2007, from http://www.mext.go.jp/b_menu/houdou/18/03/06032817/003.htm.

In: Educational Evaluation in East Asia Editor: Samuel S. Peng and John Ch-Kin Lee

ISBN 978-1-60692-887-5 © 2009 Nova Science Publishers, Inc.

Chapter 9

TEACHER EVALUATION IN THAILAND: CURRENT ISSUES AND FUTURE PROSPECTS Pruet Siribanpitak Faculty of Education, Chulalongkorn University, Thailand

ABSTRACT The purpose of this paper is to examine the current number and qualifications of public school teachers, and to explore the current issues in, and future prospects of, public school teacher evaluation in Thailand. Most secondary school teachers in Thailand are in fact public school teachers who have completed their undergraduate study; ninetytwo percent of them have completed a bachelor’s degree. Teacher evaluations have become more important since 1999. There are three types of teacher evaluation for public school teachers: (1) Induction Program Evaluation, (2) Annual Performance Evaluation, and (3) Academic Status Evaluation. The importance of teacher evaluation is reflected in the controversies regarding divergent criteria for performance-based evaluation and professional judgment in performance assessment. The Teacher Civil Service Commission-Ministry of Education has set up a taskforce to review the teacher evaluation criteria. It is likely to accept three new dimensions: (1) teachers’ discipline, morality and ethical conduct; (2) quality of teaching and student development; and (3) results of teaching and student development.

Keywords: Thailand, public schools, teacher evaluation, basic education.

1. INTRODUCTION Teacher Evaluation in Thailand has gained increasing importance since 1999, a fact which is partly attributed to the public’s demand for accountability in education. The 1999 National Education Act acknowledged the importance of educational standards and quality assurance, and it is clear that the improvement of teacher evaluations will be beneficial to educational quality and students as well as to the society at large.

118

Pruet Siribanpitak

2. OBJECTIVES The purposes of this paper are to examine the quantity and qualifications of teachers and the current issues in teacher evaluation* and to explore the future prospects of teacher evaluation in Thailand, emphasizing the development of public school teachers.

3. QUANTITY AND QUALIFICATIONS OF TEACHERS 3.1. Number of School Teachers in Basic Education in 2001-2005 The total number of school teachers in basic education** in Thailand gradually increased during the academic years of 2001 to 2005 from 633,818 teachers in 2001 to 680,272 teachers in 2005. (See Table 1). Table 1. Number of Teachers in Basic Education, (Academic Year) 2001-2005 Responsible Agencies Ministry of Education Ministry of Tourism and Sports Ministry of Culture Ministry of Interior - Department of Local Administration - BMA Ministry of Social Development and Human Security Royal Thai Police Total

2001 588,899

Academic Years 2003 2004 532,814 597,680 116 1,131 886 980 57,408 72,142 43,286 57,720

42,923 29,407

2002 568,283 1,128 12,538 44,054 29,792

2005 614,481 983 1,009 61,863 47,372

13,516

14,262

14,135 755

14,422 779

14,491 106

633,818

1,805 650,548

1,669 593,648

1,778 674,490

1,830 680,272

Source: Office of the Education Council. (2006). Education in Thailand: 2005–2006. Bangkok: Office of the Education Council. * School teachers in basic education. ** Basic education includes: Early Childhood Education, Elementary Education, Secondary Education, and others (except higher education).

3.2.Number of Faculty in Higher Education, (Academic Year) 2001-2005 The total number of teachers in higher education increased steadily during the academic years 2001-2005, from 42,080 teachers in 2001 to 61,395 teachers in 2005. (See Table 2).

3.3. Number and Qualifications of School Teachers, (Academic Year) 2004 Most of the school teachers (in basic education) are public school teachers. In the academic year 2004, there were 418,880 public school teachers as opposed to only 51,598 private school teachers. (See Table 3).

Teacher Evaluation in Thailand

119

Table 2. Number of Faculty in Higher Education, (Academic Year) 2001-2005 Responsible Agencies Ministry of Education Ministry of Tourism and Sports Ministry of Culture Total

Academic Years 2001 2002 42,080 42,541

42,080

2003 46,992

42,541

46,992

2004 57,159

2005 60,262

874

988

176 58,209

145 61,395

Source: Office of the Education Council. (2006). Education in Thailand: 2005-2006 Bangkok: Office of the Education Council.

Table 3. Percentage and Number of Elementary and Secondary School Teachers in Thailand by Degrees Earned, and Types of School, (Academic Year) 2004

Types of School Total Public Schools Poverty Enrollment Low High Community Type Urban Suburban and Rural Other Grade Level Elementary* Secondary Private Schools Grade Level Elementary Secondary

Degrees Earned Less than Bachelor’s Degree 5 (23,827) 4 (16,843)

Bachelor’s Degree 91 (427,037) 92 (348,358)

Master’s Degree or Higher 4 (19,632) 4 (17,679)

Total

-

-

-

-

4 (6,260) 4 (9,246) 4 (1,337)

92 (147,112) 92 (204,754) 92 (32,492)

4 (6,942) 4 (9,455) 4 (1,292)

100 (160,314) 100 (223,445) 100 (35,121)

8 (24,890) 2 (2,579) 13 (6,984)

88 (285,992) 65 (80,248) 83 (42,679)

4 (12,606) 33 (40,802) 4 (1,935)

100 (323,488) 100 (123,629) 100 (51,598)

16 (6,253) 6 (731)

81 (31,985) 87 (10,694)

3 (1,935) 7 (884)

100 (39,289) 100 (12,309)

100 (470,496) 100 (418,880)

Definitions: a. Elementary grade level refers to Kindergarten to Grade 6. b. Secondary grade level refers to Grades 7-12. c. Private schools are those schools that are not primarily funded nor administered by the local or central government. d. Community type refers to the whole community in the case of small provinces with only one school district. * Includes those teachers who teach lower secondary school classes in elementary schools. Source: Siribanpitak, P. and Boonyananta, S. (2006). The Qualifications of the Teaching Force in Thailand. In: A Comparative Study of Teacher Preparation and Qualifications in Six Nations. R. M. Ingersoll (ed.), p. 89. Philadelphia: Consortium for Policy Research in Education.

120

Pruet Siribanpitak

Concerning the qualifications of school teachers, most Thai elementary and secondary school teachers completed their undergraduate study. Ninety-one percent completed a bachelor’s degree, 5% held less than a bachelor’s degree and 4% completed a master’s degree or higher. There are pronounced differences between public schools and private schools. The data also shows that students in private schools had less access to qualified teachers. For example, 92% of all public school teachers completed a bachelor’s degree as opposed to 83% of private school teachers, and 5% held less than a bachelor’s degree as opposed to 13% of private school teachers. Moreover, about 33% of all public secondary school teachers held a master’s degree or higher, as compared with 7% of private secondary-school teachers. (See Table 3). The data in Table 3 also reveals that community type (urban or suburban) does not have a significant impact on teacher qualification levels. In other words, there is virtually no difference between suburban and urban schools regarding the percentage of teachers with degrees. For example, about 92% of all teachers in urban, suburban and rural areas held a bachelor’s degree and about 4% held a master’s degree or higher. This means that the qualifications of teachers in the 175 educational service areas (school districts) in Thailand are roughly equivalent. On the other hand, teachers at different grade levels in public schools do tend, predictably, to have different qualifications. For example, about 33% of all secondary school teachers have completed a master’s degree or higher, as opposed to only 4% in the case of elementary school teachers.

4. TEACHER CLASSIFICATION FRAMEWORK AND SALARY STRUCTURE IN PUBLIC SCHOOLS When individuals join the teaching profession, they are placed on a salary scale that corresponds to their qualifications and prior experience and the civil service salary structure. The base level monthly salary for teachers with a four-year bachelor’s degree is 7,360 Baht (190 US$), and 9,320 Baht (233 US$) for teachers with a master’s degree. By comparison, other professionals such as doctors and engineers typically earn about 30,000 – 50,000 Baht per month (750-1,250 US$). The Teachers and Educational Personnel Act of B.E. 2547 (2004) proposed a new salary structure, using a six-scale teacher classification framework based on academic status. The classifications, moving upward, are as follows: assistant teachers, teachers, experienced teachers, highly experienced teachers, expert teachers, specialized teachers. The entry requirement for assistant teachers is either a five-year or a 4+1 year of post-secondary preservice teacher training. An assistant teacher’s salary is 8,360 Baht per month (209 US$), and a teacher’s salary is 11,470 Baht per month (287 US$). Teachers who get promoted to a higher level (higher academic status) based on their performance receive an extra monthly allowance. The scale for the additional monthly allowances is 3,500 Baht (88 US$), 5,600 Baht (140 US$), 9,900 Baht (248 US$), and 13,000 Baht (325 US$) for experienced teachers, highly experienced teachers, expert teachers, and specialized teachers respectively (See Table 4).

Teacher Evaluation in Thailand

121

Table 4. New Teacher Classification Framework, 2004 Position

Academic Status

1. Assistant Teacher

-

2. Teacher

2.1 Experienced Teacher 2.2 Highly Experienced Teacher 2.3 Expert Teacher 2.4 Specialized Teacher

Monthly Salaries Beginning 8,360 (THB) (209 US$) 11,470 (THB) (287 US$) 14,810 (THB) (370 US$)

Highest -

Extra Monthly Allowance -

-

-

32,250 (THB) (806 US$)

3,500 (THB) (88 US$)

18,180 (THB) (455 US$)

45,620 (THB) (1,141 US$)

5,600 (THB) (140 US$)

22,230 (THB) (556 US$) 27,450 (THB) (686 US$)

48,600 (THB) (1,215 US$) 61,860 (THB) (1,547 US$)

9,900 (THB) (248 US$) 13,000 (THB) (325 US$)

Source: The Ministry of Education, 2006; OBEC, 2004. Note: The 2004 rate of exchange was about $1.00 US = 40 Thai Baht

With this new framework the teacher will earn a higher monthly salary than with the previous framework (See Table 5). The aim is not only to attract talented high school students into pre-service teacher training programs but also to attract talented graduates into the teaching profession. Table 5. Previous Teacher Classification Framework Position (Baht) Teacher (Level 1) Teacher (Level 2) Instructor (Level 1) Instructor (Level 2) Instructor (Level 3)

Salary Scale 1-3 2-4 3-5 5-6 (7)* 6-8 (9)*

Monthly Salaries 4,230-13,550 5,050-16,650 6,210-20,340 9,320-25,180 (30,710)* 11,450-43,440 (46,280)*

Source: The Ministry of Education, 2006; OBEC, 2004. * The salary scale (and thus monthly salary) will be upgraded to a higher level according to the teacher’s academic performance.

5. TYPES OF TEACHER EVALUATION There are three types of teacher evaluation for public school teachers: (1) Induction Program Evaluation, (2) Annual Performance Evaluation, and (3) Academic Status Evaluation.

Induction Program Evaluation According to the Teachers and Educational Personnel Act of B.E.2547 (2004), all new recruited public school teachers shall be appointed as “assistant teachers” for two years. During this two-year period, the new teachers must participate in the induction program and

122

Pruet Siribanpitak

must be evaluated by the school committee in order to get promoted to the position of “teacher.”

Annual Performance Evaluation According to the Administrative Procedures for Civil Servants Act of B.E. 2538 (1995), all public school teachers’ performances shall be evaluated every six months to determine whether a salary increase is justified. There are two categories of salary increase: 1) Regular Increase, which is a half-step salary increase, and 2) Special Increase, which is a one-step salary increase.

Academic Status Evaluation According to the Teachers and Educational Personnel Act of B.E. 2547 (2004), all public school teachers shall be promoted to a higher academic status (rank). There are four titles to mark academic status: (1) Experienced Teacher, (2) Highly Experienced Teacher, (3) Expert Teacher, and (4) Specialized Teacher. Teachers who get promoted to a higher academic rank will get a higher level of monthly extra allowance. (See Table 4)

6. CURRENT ISSUES IN TEACHER EVALUATION The importance of teacher evaluation is reflected in the controversies regarding evaluation criteria and procedures.

6.1. Criteria for Performance-Based Teacher Evaluation According to the National Education Act of B.E. 2542 (1999), the Teacher Civil Service Commission, Ministry of Education (TCSC- MOE) is the central organization responsible for administering teachers. The personnel administration will be based on the principle of decentralization to educational service areas and schools. A central concern in creating criteria for performance-based teacher assessment, however, is one that has plagued teacher assessment for decades and has often been described as intractable: How to define “good teaching” in a way that is appropriate for assessment purposes and yet remains faithful to the art and science of teaching as it is experienced by knowledgeable practitioners? The Teachers’ Civil Service Commission set up the following criteria for performancebased teacher evaluation. This will include three areas:

Teacher Evaluation in Thailand

123

1. Discipline, Morality and Ethical Conduct 1.1. 1.2. 1.3. 1.4. 1.5.

Good discipline Good behavior Appropriate way of life Faith in the teaching profession Sense of responsibility to the profession

2. Performance, Quality and Competency 2.1. Core competencies (1) (2) (3) (4)

Results and achievement oriented Good record of service Self-development Teamwork

2.2. Specific competencies (1) (2) (3) (4)

Learning (curriculum) design Learner (student) development Classroom management Analytical and synthetical competence (2)

3. Performance Results and Impact 3.1. 3.2. 3.3. 3.4. 3.5. 3.6. 3.7. 3.8.

Learning (classroom) management Academic (curriculum) development Impact on learners (students’ achievement) Academic papers (at least one paper) (1) Impact on (results for) teacher’s learning (2) Academic papers (at least 2 papers including 1 research paper) (2) Impact on (results for) the Community (3) Academic papers (at least 2 papers, including 1 research and 1 development paper (3)

The Teacher Civil Service Commission also set the minimum scores for promotion to the next level as follows:

124

Pruet Siribanpitak

(1)

Additional criteria for highly experienced teachers. Additional criteria for expert teachers. (3) Additional criteria for specialized teachers. (2)

Currently, there is some controversy regarding the assessment criteria for the academic paper – which is required for highly experienced teachers, expert teachers, and specialized teachers. The arguments are important because they reflect upon both the quality of teaching and students’ achievement.

6.2. Professional Judgment in Teacher Performance Assessment According to Shinkfield and Stufflebeam (1995), specification of the assessment criteria is an important part of developing any performance assessment, but the success of the effort as a whole can only be evaluated in light of the ability of assessors to use the criteria to reach technically and professionally defensible conclusions. Unlike traditional multiple-choice testing, where most of the professional judgment comes into play during the preadministration phase of test development, professional judgment in performance assessment is required in both the development and implementing phases of the assessment. In Thailand, the quality of this professional judgment affects many aspects of the assessment’s validity, including fairness, cognitive complexity, and construct representation.

Teacher Evaluation in Thailand

125

7. FUTURE PROSPECTS OF TEACHER EVALUATION The Teacher Civil Service Commission has set up a taskforce to review the teacher evaluation criteria. This taskforce is likely to utilize three new dimensions: 1. Teacher Discipline, Morality and Ethical Conduct (Remains the same) 2. Quality of Teaching and Student Development 2.1. 2.2. 2.3. 2.4. 2.5. 2.6. 2.7. 2.8. 2.9.

Teacher’s report on teaching and student development Lesson plan Test/ test analysis Student worksheet Teacher’s report on learners Student learning report School supervisor’s memo Assessor’s report Other test results/ reports

3. The Results of Teaching and Student Development 3.1. 3.2. 3.3. 3.4. 3.5. 3.6. 3.7. 3.8.

Results of the national education test Results for internal and external quality assurance Results for academic development Results for student assistance (1) Results for peer assistance (2) Results for analysis, synthesis, and instructional development research or Instructional innovation or National or international awards for teaching and learning (2)

(1) Additional criteria for Highly Experienced Teachers (2) Additional criteria for Expert Teachers and Specialized Teachers The proposed minimum scores for promotion using the new criteria are as follows:

Criteria 1. Discipline, morality, and ethical conduct 2. Quality of teaching and student development 3. Results of teaching and student development

Total Score -

Minimum Scores Highly Experienced Experienced Teacher Teacher pass/ pass/ fail fail

Expert Teacher

Specialized Teacher

pass/ fail

pass/ fail

100

60%

70%

80%

90%

100

60%

70%

80%

90%

126

Pruet Siribanpitak

8. IMPLICATIONS AND CONCLUSIONS The implementation of teacher evaluation to promote both the career growth of teachers and quality education assumes and reinforces the authentic performance evaluation of teachers. The most workable strategies for teaching evaluation involve establishing more appropriate criteria, more effective procedures and better professional judgment on the part of performance assessors. Qualified assessors are after all necessary in order for teacher evaluation to be successful. Under-qualified assessors pose an important problem. Another significant problem is that of establishing criteria which are not too subjective. Both of these factors could result in too many teachers either meeting or not meeting the criteria, achieving or not achieving the minimum scores for promotion. There is growing agreement today that there may be a set of particular teacher behaviors which taken together tend to promote student learning. However, according to Shinkfield and Stufflebeam (1995), the question of the validity of basing teacher effectiveness on student learning has led to more disagreement than any other issue in the history of teacher evaluation. Thus by the early 1970s the battle lines were drawn between those arguing against and those supporting student learning as a basis for teacher evaluation. The main challenges facing those concerned with teacher evaluation are the purpose of the evaluation itself and the desired outcomes. The problem that has dogged both theoreticians and practitioners in how the evaluation can achieve both organizational ends and increase the skills and self-esteem of teachers. If the goals of teacher evaluation are decided by external authorities and the behavioral objectives and anticipated outcomes set by them, then the evaluation will be summative orientation. But, if summative evaluations are used, the organizational context may be ignored. In contrast, the type of evaluation procedure which emphasizes professional development will involve the teachers themselves in all aspects of its planning. Feedback, reformation of goals and positive encouragement will inevitably lead to teacher improvement. Thus, it may be more productive to design evaluation systems on the basis, not of their formative or summative orientations but of the informed decisions that need to be made at the level of the teacher, the principal, the school district, and the Ministry of Education. We will need to keep all of these participants in mind if we want to design a truly valid and effective system of teacher evaluation.

REFERENCES Shinkfield, A. J. and Stufflebeam, D. (1995). Teacher evaluation: Guide to effective practice. London: Kluwer Academic Publishers. Siribanpitak, P. and Boonyananta, S. (2006). The qualifications of the teaching force in Thailand. In Ingersoll, R. M. (Ed.) A comparative study of teacher preparation and qualifications in six nations, Philadelphia: Consortium for Policy Research in Education (CPRE). Office of Basic Education Commission. (2004). Statistical data of basic education in Thailand. Bangkok: Donmuang Printing.

Teacher Evaluation in Thailand

127

Office of Education Council. (2006). Education in Thailand 2005-2006. Bangkok: Office of the Education Council. Office of National Education Commission. (1999). The National Education Act of B.E. 2542. Bangkok: Seven Printing Group.

In: Educational Evaluation in East Asia Editor: Samuel S. Peng and John Ch-Kin Lee

ISBN 978-1-60692-887-5 © 2009 Nova Science Publishers, Inc.

Chapter 10

PISA AS A CATALYST FOR CAPACITY BUILDING FOR EDUCATIONAL EVALUATION IN MACAO Kwok-cheung Cheung Faculty of Education, University of Macau, China

ABSTRACT Macao is a small city situated on the western side of Pearl River estuary. It has a population of less than half a million and a history of more than four hundred years. Macao’s educational system is characterized by the fact that the majority of its schools are privately-operated. Many of these private schools still receive a large annual subsidy from the government. This is because the policy in Macao is 15 years of free obligatory education and this is applicable not only to the few public schools but also to private schools joining the school-nets. In Macao, all private schools are given full administrative autonomy and complete freedom of instruction, and there is as yet no territory-wide public examination from kindergarten through high school. While this might seem like a good thing, it also means that the public has no way to objectively evaluate the educational achievement of schools, even though the schools may be rather different and aim to serve different communities. Thus a system of standardized evaluation with clearly spelled out benchmarks is necessary so that the government can justify to the public the amount of money it is spending on education. Therefore, it is heartening to learn that PISA (the Program for International Student Assessment), a flagship research program conducted by the Educational Testing and Assessment Research Centre (ETARC) of the University of Macau (UM), now serves as a catalyst for capacity building of essential research personnel for conducting educational evaluation in Macao. In particular, the assessment of mathematical, scientific, and reading literacy in PISA 2003, 2006 and 2009 studies covers pertinent school subject areas. This allows the public to see for the first time, from both intra-system and intersystem educational perspectives, how 15-year-old students perform after completing their obligatory basic education in the Macao school system (see Lo and Cheung, 2005; Cheung and Sit, 2007).

Keywords: Macao, PISA, literacy, educational evaluation, international assessment.

130

Kwok-cheung Cheung

1. INTRODUCTION There is no public examination from kindergarten up to the end of the senior year in high school (i.e. grades 1-12) in Macao. Basically student assessment is in the hands of the individual schools which conduct school-based evaluations regularly for their students. Because of this tradition, for the past four hundred years Macao citizens have not had objective assessment information with which to judge the quality of their schools. Since the majority of Macao schools are private, they have been legally entitled to enjoy complete freedom in educational administration and autonomy in classroom instruction. It is a known weakness of the Macao educational system that the government has no mechanism to arrange the students sit for publicly administered examinations. This is in spite of the fact that many of the private schools which join the school-nets are heavily financed by the government. Given this awkward situation, the introduction of PISA (the Program for International Student Assessment) was considered by the government to be a way to begin capacity building of essential research personnel for conducting territory-wide educational evaluation in Macao. Early in 2002, the DSEJ (i.e. Macao Education and Youth Affairs Bureau) asked the Hong Kong-PISA Centre to help it launch the PISA 2003 Mathematical Literacy Study. Because this study was mainly executed by Hong Kong’s educational researchers, many of the test instruments were simply borrowed from and much of the research infrastructure modeled after Hong Kong. As a result of this special arrangement, capacity building of essential research personnel was only developed on a very limited scale at this early stage of study. The PISA 2006 Scientific Literacy Study provided Macao its first chance to learn how to conduct a full-scale international assessment project in accordance with established technical standards set up by PISA. This chapter seeks to document seven benefits Macao gains, with regard to capacity building of essential research personnel for conducting territory-wide educational evaluation, by participating in the various cycles or stages of PISA studies. The seven benefits are: 1. Forge links and promote dialogues within and across the various operational levels and administrative infrastructure of PISA projects; 2. Acquire a first-hand understanding of perspectives that will enable a literacy assessment framework for test units design; 3. Gain a first-hand understanding of conceptual frameworks that will make possible a roadmap for questionnaire design; 4. Acquire cutting-edge methodologies for handling weighting and sampling errors in complex sampling design; 5. Acquire cutting-edge methodologies for handling scaling and measurement errors in complex test booklet design; 6. Learn well-planned strategies for reporting PISA results to Macao schools and the public; 7. Be involved in pioneering research and development in the assessment of electronic reading literacy.

PISA as a Catalyst for Capacity Building for Educational Evaluation in Macao

131

2. FORGING LINKS AND PROMOTING DIALOGUES WITHIN AND ACROSS THE VARIOUS OPERATIONAL LEVELS AND ADMINISTRATIVE INFRASTRUCTURE OF PISA PROJECTS In the late 1990s, the Organization for Economic Co-operation and Development (OECD) commissioned the Program for International Student Assessment (PISA) as an ongoing international project undertaking periodic comparative study of the mathematical, scientific and reading literacy of 15-year-old students amongst participating countries/economies (OECD, 1999). Most countries/economies took advantage of this opportunity to use PISA as a catalyst for capacity building of essential research personnel in the area of educational evaluation, and Macao is no exception (OECD, 2001). The infrastructure for PISA projects can be envisaged as operating at a number of levels, as described concisely below (See Figure 1, a diagram on the MyPISA website which is being used for the internal management of PISA projects). OECD Secretariat

PISA Governing Board

International Contractor(s) z z z z z z z z z z

Quality standards and assurance Communications Sampling Framework development Instrument development Translation/verification Assessment operations Data capture and processing Statistical analyses Reporting

z z z z z z

Mathematics Expert Group Reading Expert Group Science Expert Group Questionnaire Expert Group Technical Expert Group Others as needed

National Centre z z

National Project Manager National Project Team

Participating Countries z

z

Open Forum

OECD member countries Partner countries

(Reading, Science, Mathematics)

Experts

Figure 1. Operative levels and administrative infrastructure for PISA projects

First, at the highest level, all PISA projects are overseen by the secretariat of the OECD in Paris. Second, senior educational administrators, delegates and observers from participating countries/economies form the PISA Governing Board (PGB) – a committee

132

Kwok-cheung Cheung

within the OECD which sets policy priorities through collaborative processes in order to steer projects. Third, for each PISA project a National Project Manager (NPM) is appointed to coordinate all activities of the National Center (NC) of participating countries/economies, which is responsible for executing the PISA projects. At each NC, the NPM needs to work closely with the PGB, and forms working teams that take care of project development, implementation, and the reporting of results to participating schools and the public. Fourth, there are Consortiums, i.e. groups of international research and academic organizations, which develop test instruments and questionnaires and are responsible for implementing PISA projects in accordance with pre-determined time-lines. Fifth, Technical Advisory Groups and Expert Groups are formed to supervise and to solve problems raised by Consortiums during the planning and implementation stage of PISA projects. Evidently, researchers working at the Macau-PISA Center, the NC within ETARC of UM responsible for undertaking all PISA projects in Macao, are benefited immensely by participating in such a large-scale international student assessment program. This is because they are able to forge links and engage in interactive dialogues with superb, internationally-oriented administrative infrastructures.

3. FIRST-HAND UNDERSTANDING OF PERSPECTIVES LEADING TO A LITERACY ASSESSMENT FRAMEWORK FOR TEST UNITS DESIGN The PISA assessments take a literacy perspective. They focus on the extent to which students can use the knowledge and skills they have gained and practiced at school when confronted with situations and challenges for which that knowledge may be relevant (OECD, 2005a). That is, PISA assesses the extent to which students can use their scientific knowledge and skills to understand, interpret and resolve various kinds of scientific situations and challenges. Additionally, it judges the extent to which students can use their mathematical knowledge and skills to solve various kinds of mathematical challenges and problems, and the extent to which they can use their reading skills to understand and interpret various kinds of written material that they are likely to meet in their daily lives (OECD, 1999 and 2006). When PISA initiates an international assessment of the three literacy domains, an assessment framework with sample items will be drafted and released for consultation by the participating countries/economies. This framework describes the breadth of each of the three literacy domains, and the sub-areas and skills associated with each (see Figure 2 for the PISA 2006 science assessment framework, see also OECD, 2007d, p.35). In the assessment framework shown in Figure 2, context refers to engagement with science in a variety of life situations which are not limited to life in school (i.e. personal, social and global situations within application areas such as health, natural resources, environment, hazards, and frontiers of science and technology). These applications require students to demonstrate three main kinds of scientific competencies (i.e. identify scientific issues, explain phenomena scientifically, and use scientific evidence). How students do so is influenced by their knowledge (i.e. knowledge of science and knowledge about science) and attitudes (i.e. interest in science, support for scientific enquiry, and responsibility toward resources and environments). Knowledge of science is further classified into four categories:

PISA as a Catalyst for Capacity Building for Educational Evaluation in Macao

133

(1) physical systems, (2) living systems, (3) earth and space systems, and (4) technology systems, whereas knowledge about science into two categories: (1) scientific enquiry and (2) scientific explanations. In the test booklets, there is a balance of items assessing the various components of the scientific assessment framework. Context Life situations that involve science and technology

Require people to

Competencies ƒ Identify scientific issues ƒ Explain phenomena scientifically ƒ Use scientific evidence

How they do so is influenced by

Knowledge a) What they know: ƒ about the natural world and technology (knowledge of science); ƒ about science itself (knowledge about science).

Attitudes b) How they respond to science issues (interest, support for scientific enquiry, responsibility).

Figure 2. Framework for PISA 2006 science assessment.

4. FIRST-HAND UNDERSTANDING OF CONCEPTUAL FRAMEWORKS THAT WILL MAKE POSSIBLE A ROADMAP FOR QUESTIONNAIRE DESIGN The PISA survey collects information from students on various aspects of their home, family and school background, as well as information from schools about various aspects of their organization and educational programs. This information is collected to facilitate a detailed study of intra- and international factors associated with varying levels of reading, mathematical and scientific literacy among the 15-year-old students of participating countries/economies. Topics of thematic reports need to be determined before the Questionnaire Expert Group (QEG) knows what questions will need to be asked in the various questionnaires (i.e. student questionnaire, school questionnaire, parent questionnaire,

134

Kwok-cheung Cheung

educational career questionnaire, information and communication technology (ICT) questionnaire). For those countries/economies that adopt a grade-based sample the teacher questionnaire may be deployed to link teacher data directly to student or class data. Table 1. Rubrics of questionnaires designed for the PISA 2009 Study Questionnaire Rubrics 1. Student Questionnaire (11 rubrics) • Basic student characteristics and school career • Parental education and occupation, immigration status, home resources and family wealth • Reading engagement outside school as measured in PISA 2000 • Learning strategies and learning preferences • Generally supportive teaching and learning environments • Classroom and school climate • Monitoring and feedback • Opportunity to learn reading literacy at school • Extended measures of reading engagement • Extended measures of school conditions that are expected to support reading engagement • Extended measures of meta-cognition of learning strategies 2. School Questionnaire (9 rubrics) • Basic school characteristics • School resources • School climate • Supportive teaching and learning environment • Admission, selection, tracking, setting and ability grouping • Decentralization • Monitoring and evaluation • Equity-oriented policies and practices • Educational leadership 3. Parent Questionnaire (5 rubrics) • Parents’ report on reading engagement • Home reading resources • Socio-economic status • Parents’ perception of and involvement in their child’s school • Parents’ reports and opinions concerning policies and practices of their child’s school Questionnaire Rubrics 4. Educational Career Questionnaire (5 rubrics) • Student’s past, current and anticipated educational career • Change of study program • Setting and ability grouping arrangement • Participation in additional instruction and courses relevant to reading literacy • School choice 5. Information and Communication Technology Questionnaire (4 rubrics) • Availability and use of computers • Availability and use of newer ICT devices, services and applications • Self-rated computer and internet skills • Attitude toward computers and internet

PISA as a Catalyst for Capacity Building for Educational Evaluation in Macao

135

For example, in the PISA 2009 reading literacy study, a roadmap entitled “From Conceptual Framework to Questionnaire Content” was drafted by Professor Jaap Scheerens in liaison with the Reading Expert Group (REG). In this roadmap, four themes are proposed which cover: (1) system level indicators, cost effectiveness, and equity; (2) school effectiveness, educational leadership, equity and cost effectiveness; (3) establishing an effective learning environment in reading literacy at the instructional setting level; and (4) establishing an effective learning environment in reading literacy at the individual student level (OECD, 2007b). The Questionnaire Framework for PISA 2009 Reading Literacy Study sketches data sources, and includes a rationale of the themes chosen, as well as the antecedents, amenable processes and outcomes entailed. Table 1 summarizes the rubrics of the five questionnaires used in the PISA 2009 Reading Literacy Study. Questions are drafted and included in the respective questionnaires, and thus intended for inclusion in the pilot study and then in the main study.

5. CUTTING-EDGE METHODOLOGIES FOR HANDLING WEIGHTING AND SAMPLING ERRORS IN COMPLEX SAMPLING DESIGN PISA is an age-based survey; it assesses 15-year-old students in grade 7 or higher. These students are approaching the end of their compulsory schooling in most participating countries/economies, and school enrolment at this level is close to universal in almost all OECD countries/economies. Most countries/economies adopt a two-stage sampling procedure. During the first stage schools are randomly selected within designated strata, with probabilities proportional to their number of eligible 15-year-old students (i.e. PPS sample). During the second stage students in larger schools will have a smaller probability of being selected than students in smaller schools, so that the sum of the final student weights is equal to the total number of eligible 15-year-old students in the population (see OECD 2005b for details). This sampling procedure guarantees that all sampled students will have the same probability of being selected. However, PISA data stills needs to be weighted, and capacitybuilding of essential research personnel is necessary to the processing and analysis of such kind of sampled data. There are three reasons that cause the variation in final student weights: (1) Some countries/economies may over-sample or under-sample certain strata of the population so as to achieve better sample representation; (2) PPS is calculated using estimated size measures instead of actual size measures for schools listed on the sampling frame; (3) weight adjustments need to be applied at both school and student levels in cases where some students do not respond the tests and/or questionnaires. Thus the calculation of sampling errors with common complex analytical statistics (e.g. coefficients in multiple regressions) for complex clustered sampling design proves extremely difficult for the young researchers. In PISA, Balanced Repeated Replication (BRR) and its variant, Fay’s modification, is used so that the statistical results are computed initially for the whole sample, and then again for each of the 80 jackknife replicate samples, in order to calculate the sampling error for the required statistics. There are special statistical programs for handling weighting and sampling errors for complex sample designs. Capacity building of essential research personnel is therefore very important because these are cutting-edge methodologies.

136

Kwok-cheung Cheung

6. CUTTING-EDGE METHODOLOGIES FOR HANDLING SCALING AND MEASUREMENT ERRORS IN COMPLEX TEST BOOKLET DESIGN There is a need to construct quantitative measurement scales with qualitative graded descriptions in order to gauge students’ proficiency levels in the three literacy domains, and to chart performance trends across a 3-year assessment cycle (OECD, 2001, 2002, 2004a, 2004b and 2004c). For instance, in the PISA 2003 Study, test units are packaged together in the form of thirteen 30-minute clusters (i.e. C1 to C13), and every student needs to take one of the thirteen 2-hour test booklets (i.e. Booklet 1 to 13). Each booklet consists of 4 clusters of test units arranged in designated time blocks. Table 2 presents the complex test booklet design used in the PISA 2003 Study (OECD, 2005b). With such a design each cluster appears 4 times, once in each testing time block, and each pair of clusters appears once and only once. Not all clusters are answered by every sampled student, and the use of Rasch scaling ensures that the sample-free item calibration procedures will not be influenced by the respective location of the test items in the different booklets. On the contrary, the clusters enable itemfree measurement of sampled students’ proficiency when literacy scales are shown to conform to the requirements of the Rasch scaling procedures. As an international assessment project it is imperative for PISA to reduce measurement errors in making inferences about the target population being sampled. Errors may arise due to poor conceptualization of the literacy constructs measured, unstable mental and physical dispositions of the sampled students on the day of assessment, and adverse testing conditions experienced by the students. Furthermore, these measurement errors may be smaller for average students, and larger for the low and high achievers. In international assessment, population estimates are always more important than individual estimates, and therefore measurement errors should be minimized as far as possible. In PISA, each student’s literacy proficiency is gauged by a set of five plausible values (PVs), which represent the range of abilities that a student might reasonably have. Table 2. Test booklet design used in the PISA 2003 Study

Booklet 1 Booklet 2 Booklet 3 Booklet 4 Booklet 5 Booklet 6 Booklet 7 Booklet 8 Booklet 9 Booklet 10 Booklet 11 Booklet 12 Booklet 13

Block 1 C1 C2 C3 C4 C5 C6 C7 C8 C9 C10 C11 C12 C13

Block 2 C2 C3 C4 C5 C6 C7 C8 C9 C10 C11 C12 C13 C1

Block 3 C4 C5 C6 C7 C8 C9 C10 C11 C12 C13 C1 C2 C3

Block 4 C10 C11 C12 C13 C1 C2 C3 C4 C5 C6 C7 C8 C9

PISA as a Catalyst for Capacity Building for Educational Evaluation in Macao

137

Capacity building of essential research personnel is necessary because researchers have to be trained to use PVs in the reporting of PISA results at both the national and international levels (see Cheung and Sit, 2007 for such attempts).

7. WELL-PLANNED STRATEGIES FOR DISSEMINATING PISA RESULTS TO PARTICIPATING SCHOOLS AND THE PUBLIC Sit and Cheung (2008) was the first attempt to report PISA 2006 Study results to participating schools and the public. Reporting PISA findings and assessment results is a job needed meticulous attention. This is because schools will decline to join PISA assessment if schools find that the findings and assessment results released are not useful to them for purposes of school improvement. Often, below-average positions in the international assessment league table appear as headlines in local newspapers. Officials in charge of education are then very nervous if they cannot provide satisfactory explanations for the adverse results reported, and this is particularly sensitive when there is a decline in literacy performance from the level reported in the last international assessment administered three years ago. For those interested, the official report of the Macao-PISA 2006 Study comprises six bilingual chapters: (1) Conduct of enquiry; (2) A profile of literacy performance for 15-yearolds in Macao; (3) Quality science education indicators; (4) Literacy-ESCS relationships for Macao schools; (5) International comparison of literacy performance; and (6) Thematic reports and follow-up studies (Cheung and Sit, 2007). During the school briefing session, the following figure was found to be of considerable interest to the principals because it showed the slightly non-linear relationship between between-school scientific literacy performance and the economic, social and cultural status of the home of the students enrolled (see Figure 3; see also Cheung and Sit, 2007, p.57-58). Based on the plot shown in Figure 3, Macao school administrators on the one hand are able to judge whether their schools are highperforming, average-performing or low-performing, and on the other hand to determine the index of the economic, social and cultural status of students’ homes, so as to have a better understanding of the achievement and social class of the students they serve. In a nutshell, if we classify the scientific literacy performance of the 43 schools participating in the PISA 2006 Study into high-performing (13 schools), average-performing (17 schools), and low-performing (13 schools), the following three observations may be made: 1. The ESCS of the four schools classified as “International/Portuguese” in the PISA 2006 Study are the highest amongst the 43 sampled schools (mean ESCS > 0.50). However, as far as scientific literacy performance is concerned, two are highperforming (top one-third of the 43 schools) and two are low-performing (bottom one-third of the 43 schools). Therefore, there is no apparent relationship between the four schools’ scientific literacy performance and their ESCS. 2. The eleven highest-performing schools in scientific literacy (excluding the two highperforming International/Portuguese schools mentioned above) are generally associated with homes with a higher ESCS (i.e. -1.1 < ESCS < 0.5).

138

Kwok-cheung Cheung 3. The eleven lowest-performing schools in scientific literacy (excluding the two lowperforming International/Portuguese schools mentioned before) are generally associated with homes with a lower ESCS (i.e. -1.80 < ESCS < -0.90). However, two low-performing schools (schools number 15 and 43) have students coming from homes with a higher ESCS (i.e. ESCS = -0.44 and -0.01). ＋Average-performing school

▲ High-performing school 600

■ Low-performing school 8

3

27

30

20 24

550

500

34

22

37

33 38

9

7

5

39

4

35 13 18

1

31

23

21

12

16 32

40

450

14

42

17 26

Sc i en ti f i c L i ter ac y

6

19

29 43

2

10

25 15

41

400

28

36

350 11

300 -2

-1.5

-1

-0.5

0

0.5

1

Index of Economic, Social and Cultural Status (ESCS) Figure 3. Plot of between-school scientific literacy performance with ESCS.

8. PIONEERING RESEARCH AND DEVELOPMENT IN THE ASSESSMENT OF ELECTRONIC READING LITERACY At the turn of the century, electronic reading literacy emerged as a new form of literacy which needed to be inculcated during the basic education stage of schooling (OECD, 2007a). Nowadays, quite a lot of students (especially in the more developed countries) are reading electronically. Therefore, electronic reading literacy has been emerging as an increasingly

PISA as a Catalyst for Capacity Building for Educational Evaluation in Macao

139

important skill in the educational agenda. However, assessment of this type of competence demands the appropriate delineation of a sound assessment framework, one which deals with both print-based and electronic reading texts simultaneously. In response to the digital age, the PISA 2009 Reading Literacy Study seeks to achieve this goal (OECD, 2007c).

Figure 4. Relationship between fixed reading texts and competences of assessment tasks presented in the print medium.

Figure 5. Relationship between dynamic reading texts and competences of assessment tasks delivered in the electronic médium.

140

Kwok-cheung Cheung

Compared with print-based reading literacy, there are three aspects of electronic reading literacy needed to be addressed if a sound assessment framework is to be developed (see Figure 4 and 5 for the comparison). The three aspects are: (1) students who read online access and retrieve information via search engines and embedded hyperlinks; (2) students construct and integrate texts in accordance with access structures they opt to follow, i.e. by clicking hyperlinks and searching for usable information in multiple texts until they judge that the information collected has been adequately and meaningfully synthesized; (3) students need to reflect on and evaluate critically authorship, as well as the accuracy and credibility of information retrieved. Again, capacity building of essential research personnel is urgently needed in this innovative area of reading literacy assessment.

CONCLUSION Supported by renowned scholars and institutions, PISA studies are able to furnish reliable and valid data in order that the government may hold schools accountable for the subsidies rendered to them as part of the Macao’s obligatory universal basic education program. Because there is no tradition of public examination in Macao, capacity building of essential research personnel for the conduct of territory-wide educational evaluation is waiting to be developed. There are five benchmarks at which the outcomes of PISA studies in Macao can be targeted, and of which capacity building of essential research personnel can be conceived as successful: (1) Monitoring of quality of Macao’s educational system in the inculcation of reading, mathematical and scientific literacy for 15-year-olds across successive cycles of PISA studies; (2) Monitoring of equity in the distribution of learning opportunities as measured by the impact of the economic, social and cultural status of the home on school’s literacy performance; (3) Maintenance of consistency of literacy standards amongst different types of schools, especially between the public and private schools; (4) Revelation and explanation of differential gender difference patterns in the three domains of literacy, i.e. reading, mathematical and scientific literacy; (5) Revelation and explanation of teaching and learning environments facilitative of the life-long learning for 15-year-old students. In sum, the seven benefits delineated and gained by Macau-PISA Centre via active participation in PISA 2003, 2006 and 2009 studies are pertinent to answer whether PISA can serve as a catalyst for capacity building of essential research personnel for educational evaluation in Macao.

REFERENCES Cheung, K. C., and Sit, P. S. (2007). Macao PISA 2006 Study Report Number One: Assessment of Scientific, Mathematical and Reading Literacy Performance of 15-yearold Students from an International Comparison Perspective. Macao: Educational Testing and Assessment Research Centre, University of Macau. Sit, P. S., and Cheung, K. C. (2008). Key Findings of the Macao-China PISA 2006 Scientific Literacy Study. Invited paper presented at the International Conference on “PISA 2006:

PISA as a Catalyst for Capacity Building for Educational Evaluation in Macao

141

The Performance of Education Systems in Countries and Regions” jointly convened by the Autonomous Province of Trento and the OECD on 3-4 April 2008 at Trento, Italy. Lo, L. F., and Cheung, K. C. (2005). 2003 Macao-PISA Final Report: Assessment of 15Year-Old Macao Students. Macao: The Macau-PISA Center. OECD. (1999). Measuring Student Knowledge and Skills: A New Framework for Assessment. Paris: OECD. OECD. (2001). Knowledge and Skills for Life: First Results from the OECD Programme for International Student Assessment (PISA) 2000. Paris: OECD. OECD. (2002). Reading for Change: Performance and Engagement across Countries Paris: OECD. OECD (2004a). Learning for Tomorrow’s World: First Results from PISA 2003. Paris: OECD. OECD (2004b). What Makes School Systems Perform? Seeing School Systems through the Prism of PISA. Paris: OECD. OECD (2004c). Problem-solving for Tomorrow’s World: First Measures of Cross-curricular Competencies from PISA 2003. Paris: OECD. OECD (2005a). The Definition and Selection of Key Competencies: Executive Summary. Paris: OECD. OECD (2005b). PISA 2003 Data Analysis Manual: SPSS Users. Paris: OECD. OECD (2006). Assessing Scientific, Reading and Mathematical Literacy: A Framework for PISA 2006. Paris: OECD. OECD (2007a). Reading Literacy: A Framework for PISA 2009 (Document: NPM0709/7b). Paris: OECD. OECD (2007b). Questionnaire Framework for PISA 2009 (Document: NPM0709/5a). Paris: OECD. OECD (2007c). Electronic Reading Assessment: Updates and Field Trial Arrangements (Document: NPM0709/2). Paris: OECD. OECD (2007d). PISA 2006 – Science Competencies for Tomorrow’s World. Volume 1: Analysis. Paris: OECD.

CONTRIBUTORS Sun-Geun Baek, Ph.D. is currently a Professor of Department of Education, College of Education in Seoul National University (SNU), Korea. He earned his bachelor’s degree in Education from SNU. He also obtained his master’s degree in Educational Measurement and Evaluation from SNU. Then he obtained his doctoral degree in Quantitative Methods in Education from University of California at Berkeley, USA. His full and rich experiences include Associate Dean at College of Education in SNU, Director of Department of International Cooperation at Education Research Institute in SNU, Expert Adviser at Presidential Committee on Education Innovation, Director of Department of Curriculum Development in National Academy for Educational Administrators, Senior Research Fellow in Korea Institute of Curriculum and Evaluation, Senior Researcher in Korean Educational Development Institute, Research and Post-doctoral Researcher in University of California at Berkeley. He was awarded for the best research professor in SNU, the best research report in the Korea Institute of Curriculum and Evaluation, the best work in the National Training Center for Statistics, and the best research report in the Korean Educational Development Institute. He also got Korean Government Overseas Scholarship and graduated as “top scholar” in the Department of Education at SNU. Kwok-cheung Cheung, National Project Manager of Macao-China PISA 2009 Study, is currently serving as Professor of Curriculum and Instruction and Director of Educational Testing and Assessment Research Centre at the Faculty of Education of the University of Macau. He specializes in testing, measurement and evaluation. Sirichai Kanjanawasee is professor of education in the Department of Educational Research and Psychology, Faculty of Education, Chulalongkorn University, where he chairs the Educational Measurement and Evaluation Doctoral Program Committee. He is also the Director of the Testing and Evaluation Center for Educational and Professional Development. He obtained his Bachelor’s Degrees in Science and Mathematics Teaching from Chulalongkorn University and Sanitarial Science from Mahidol University, and Master’s Degree in Educational Research from Chulalongkorn University. He earned his doctorate in Measurement and Evaluation from the University of California at Los Angeles. Among several awards and honors he received in recent years are Alumni of Distinction, Faculty Distinguished A ward, Chulalongkorn University Excellent Research Award, and Ngarmjitburachat Excellent Research Award. He authored and co-authored numerous textbooks on Advanced Measurement and Evaluation Theories. John Chi-Kin Lee is Professor at the Department of Curriculum and Instruction, Dean of Education and Director of The Centre for University and School Partnership, The

144

Contributors

Chinese University of Hong Kong. He has led the development of large-scale school improvement project including the Accelerated Schools for Quality Education Project and the Partnership for Improvement of Learning and Teaching Project. He is the Chief Editor of Journal of Basic Education and Regional Editor (Asia) of Educational Research and Evaluation (Routledge). He is author and co-author of four academic books, including Curriculum, Teaching and School Reforms: Educational Development in a New Century (The Chinese University Press, in Chinese, 326pp.) and (with Kerry Kennedy) The Changing Role of Schools in Asian Societies: Schools for the Knowledge Society (Routledge, 228pp.). He is also the co-editor (with Leslie Lo and Allan Walker) of Partnership and Change for School Development (The Chinese University Press), (coedited with Roger Cheng and Leslie Lo) Values Education for Citizens in the New Century, (co-edited with Michael Williams), School Improvement: International perspectives (Nova Science Publisher, Inc.) and (co-edited with Michael Williams) Environmental and Geographical Education for Sustainability: Cultural contexts (Nova Science Publisher, Inc.). Mu-Lin Lu is the Political Deputy Minister, for the Ministry of Education, Taiwan. As an experienced administrative officer in charge of Taiwan’s higher education, international cultural and educational relations, since 2000, Dr. Lu has implemented important policies for the facilitation of innovated educational policies in Taiwan. Prior to his current Ministry position, Dr. Lu was the Administrative Deputy Minister, 2000-2006; the Secretary General for the Ministry of Interior, 1997-1998; and the Director of the Department of Cultural and Educational Affairs for the Mainland Affairs Council, 19921997. Dr. Lu received his doctorate degree from the University of Texas at Austin, in the United States. His research interests include: supervision of instruction, teacher development and education policy. Nicholas Sun-Keung Pang is a Professor of Department of Educational Administration and Policy in the Chinese University of Hong Kong. His research interests include educational administration, management and leadership, as well as school effectiveness and improvement. He also serves as Executive for the Hong Kong Centre for the Development of Educational Leadership, Leader of the School Development and Evaluation Team (SDET) and Programme Director of the Preparation for Principalship Course and School Managers Course commissioned by the Hong Kong Education Bureau. He was elected the Chairman of Hong Kong Educational Research Association for the year 1999-2001 and served as member of the Advisory Committee on Quality Assurance Inspectorate and the Working Group on the Study of Effectiveness of Publicsector Secondary Schools of the then Education and Manpower Bureau. He is an appointed Sector/Subject Specialist of the Hong Kong Council for Accreditation of Academic and Vocational Qualification for 2008-2011. He has been publishing widely in local and international journals. He is Co-author of an academic book entitled Leadership and Management in Education: Developing Essential Skills and Competencies (322 pages) published in 2003 and the Editor of an academic book entitled Globalization: Educational Research, Change and Reforms, published in 2006. He was invited to speak at major international academic meetings in Hong Kong, mainland China, Macau, Taiwan, Vietnam, India, Australia, Slovenia, and the United States. Samuel S. Peng is a Chair Professor of the National Taiwan Normal University (NTNU). Before joining NTNU, he was a Chair Professor of the National Tsing Hua University

Contributors

145

(2003-2006). From 1981 to 2003, he was a program director at the National Center for Education Statistics, U.S. Department of Education, responsible for several national data systems. He was a reviewer for professional journals such as American Educational Research Journal, Educational Researcher, and Educational Evaluation and Policy Analysis. In addition, he was the Editor in Chief of the New Waves -- Educational Research and Development, the quarterly journal of the Chinese American Educational Research and Development Association from 1996 to 2006. Currently he is the Editor in Chief of the Journal of Higher Education in Taiwan. Pruet Siribanpitak is an Associate Professor and Dean of Faculty of Education at Chulalongkorn University Bangkok, Thailand. He got his bachelor’s degree in Elementary Education, master’s degree in Educational Administration from Chulalongkorn University, and doctoral degree in International and Development Education concentration in Economics of Education from University of Pittsburgh, Pennsylvania. His experiences including a faculty member of Department of Education Research, Department of Educational Administration and Department of Foundations of Education, head of Department of Foundations of Education, Associate Dean for Planning and Development, Dean of Faculty of Education at Chulalongkorn University, president of Thailand Deans of Education Council, expert member of the Committee on Education of the House of Representatives and the Research Project on Teacher Education, the Office of Education Council, Ministry of Education. He also has a variety of international experiences including resource person and consultant in Vietnam, Nepal, Lao, and India. He used to serve as a member of advisory group for the UNESCO on “Reorienting Teacher Education for a Sustainable Future” at Thessaloniki, Greece. Ramanathan Subramaniam holds the concurrent positions of Associate Dean (Graduate Academic Programs) in the Graduate Programmes and Research Office, and Associate Professor in the Natural Sciences and Science Education Academic Group at the National Institute of Education in Nanyang Technological University. He has diverse research interests: physical chemistry, physics education, chemistry education, primary science education, ICT in education, informal science education, teacher education, science communication, and creativity. He has over 50 research papers to his credit in international (refereed) journals published in the USA and Europe. He has also over 25 refereed chapters to his credit in books of international publishers in the USA and Europe. Four of his books were recently published by major international publishers. His biography appears in the Dictionary of International Biography (Cambridge, UK) and the Marquis Who’s Who in Asia (New Jersey, USA). Suan Yoong,B.Sc.,M.Ed. ( Malaya ); M.S.,Ph.D.(Indiana), is a Fellow/Professor of Education at the Sultan Idris University of Education, Malaysia and Deputy Chair of the International Organisation for Science Education, IOSTE. His bachelor’s degree was in Chemistry and Mathematics, but he specialized in science and technology education and psychometric at the masters and doctoral levels. He was the Chairman of Higher Degree Program and a Professor of Education at the School of Educational Studies, University of Science Malaysia. He was also the Project Coordinator of the Malaysian General Science Project at the Malaysian Curriculum Development Center. While at Indiana University, he won the Outstanding Graduate Student award and was inducted into Phi Delta Kappa, the National Honor Society in Education. He was a recipient of the UNESCO Fellowship Award for the 1992-93 Participation Program for research in Psychometric and Testing -

146

Contributors

as Research Fellow at the California Testing Bureau-McGrawHill, Monterey, California, and visiting scholar to the Educational Testing Service at Princeton, New Jersey, the Lindquist Center at University of Iowa and the Psychological Corporation at San Antonio, Texas. He was also a Research Fellow at the University of California at Berkeley, and the Center for Evaluation and Testing, University of Wisconsin at Madison, USA (1994-1995) Yasuhiko Washiyama is the President of Tokyo Gakugei University and a Professor in the Department of Education. His experiences in Tokyo Gakugei University are rich. He was a Lecturer, an Assistant professor, a Professor and the Chair of the Department of Education. He was also the director of the library. For educational background, he has a graduate degree in Humanities.

INDEX A academic, 1, 4, 8, 13, 14, 15, 17, 19, 20, 23, 25, 28, 30, 33, 35, 36, 39, 46, 47, 49, 50, 51, 52, 54, 55, 58, 61, 68, 78, 88, 95, 96, 102, 104, 112, 118, 120, 121, 122, 124, 125, 132, 144 academic performance, 55, 58, 61, 68, 104, 121 accountability, 2, 6, 10, 24, 31, 33, 62, 68, 69, 70, 71, 88, 90, 104, 106, 107, 117 accreditation, 15, 16, 17, 21, 42, 43, 44 accuracy, 4, 140 achievement, 8, 10, 23, 30, 34, 35, 37, 46, 48, 49, 52, 54, 58, 59, 61, 63, 69, 97, 101, 102, 105, 106, 123, 124, 129, 137 achievement test, 10, 101 action research, 33, 64 adaptability, 104 adjustment, 14, 47, 103 administration, 4, 7, 13, 14, 19, 20, 32, 48, 49, 57, 77, 83, 105, 111, 112, 122, 124, 130, 144 administrative, 3, 8, 13, 15, 19, 20, 35, 48, 129, 130, 131, 132, 144 administrative efficiency, 20 administrators, 2, 6, 10, 47, 48, 49, 53, 79, 82, 85, 87, 90, 104, 114, 115, 132, 137 advocacy, 62 after-school, 104 age, 9, 13, 25, 43, 94, 96, 100, 101, 135, 139 agent, 24, 77 alternative, 9, 101, 102, 105 antecedents, 135 anxiety, 67, 98 application, 18, 100, 101, 108, 113, 132 aptitude, 56, 57, 101, 103, 106 arithmetic, 114 artistic, 105 Asia, 1, 2, 3, 6, 9, 17, 24, 62, 93, 94, 104, 107, 108, 144, 145

Asian, 2, 3, 9, 10, 31, 35, 62, 69, 70, 93, 94, 97, 104, 107, 108, 144 Asian countries, 94 assessment, 6, 7, 13, 24, 26, 29, 30, 33, 34, 38, 45, 46, 47, 48, 49, 52, 53, 55, 58, 59, 62, 63, 64, 66, 67, 68, 88, 93, 94, 96, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 115, 117, 122, 124, 129, 130, 132, 133, 136, 137, 139, 140 assessment procedures, 24 assessment tools, 6, 38, 55, 58 atmosphere, 7, 112 attitudes, 77, 115, 133 auditing, 18 Australia, 14, 15, 27, 91, 96, 97, 108, 144 authority, 23, 25, 26, 29, 32, 51, 88, 111, 113 autonomy, 4, 8, 86, 129, 130 awareness, 2

B barrier, 99 barriers, 58 BCA, 61, 66, 68 beginning teachers, 62 behavior, 123 Beijing, 65, 70 beliefs, 106 benchmark, 24, 26, 102 benchmarking, 24, 29, 46, 107 benchmarks, 8, 34, 129, 140 benefits, 68, 85, 130, 140 Best Practice, 28 bias, 103 bilingual, 137 birth, 62 birth rate, 62 blocks, 136 BMA, 118 Boston, 34

148

Index

bottom-up, 65, 75, 81, 87 breaches, 99 breakdown, 87 Britain, 14, 15, 18 broadband, 33 buildings, 32 business management, 17

C caliber, 13 calibration, 58, 59, 136 Canada, 96, 97 candidates, 1, 4, 35, 39 capacity building, 8, 62, 129, 130, 131, 140 cast, 104 catalyst, 8, 129, 131, 140 Catholic, 17, 79 Catholic Church, 79 certificate, 25, 37, 101 certification, 29, 101 changing environment, 89 cheating, 66, 99, 100 childhood, 80 children, 93, 94, 97, 98, 104 China, 3, 9, 91, 96, 97, 144 citizens, 21, 47, 55, 130 citizenship, 47 civil service, 120 classes, 30, 37, 95, 96, 98, 119 classification, 52, 120 classroom, 38, 54, 58, 67, 70, 100, 101, 114, 123, 130 classroom practice, 54, 58 classrooms, 9, 55, 58, 70 clients, 51 closure, 62, 66, 68 clusters, 136 cognition, 84, 105, 134 coherence, 6, 61, 68 collaboration, 5, 8, 75, 87, 96, 112, 113 colleges, 4, 9, 10, 13, 15, 17, 19, 20, 23, 25, 27, 28, 39, 40, 41, 46, 96 communication, 10, 24, 43, 59, 78, 87, 105, 134, 145 communication technologies, 105 communities, 46, 62, 69, 97, 129 community, 1, 6, 8, 10, 47, 50, 53, 61, 68, 76, 79, 82, 83, 85, 96, 97, 104, 111, 112, 113, 114, 115, 116, 119, 120 community cooperation, 83 competence, 2, 42, 43, 44, 85, 123, 139 competition, 5, 43, 62, 99, 114 competitiveness, 2, 13, 24, 27, 30, 55, 68, 104 complexity, 105, 124

compliance, 49, 53, 58, 111, 113 components, 47, 66, 89, 133 comprehension, 38, 39 compulsory education, 7, 111, 113 concentration, 145 conceptualization, 136 concrete, 85, 115 confidence, 78, 85, 114 confusion, 87 consciousness, 47, 115 consensus, 4, 6, 20, 85 conservation, 52 constraints, 28, 31, 105 construction, 57 consumers, 9 control, 7, 24, 33, 35, 37, 94, 97, 105, 111 cost effectiveness, 135 costs, 19 Council of Ministers, 46, 48 coupling, 67 course design, 16 covering, 81 creativity, 31, 34, 98, 102, 145 credibility, 140 credit, 145 critical thinking, 53, 55, 56, 57 criticism, 115 cues, 114 cultural heritage, 68 culture, 2, 5, 7, 31, 34, 65, 66, 78, 79, 81, 82, 83, 85, 87, 89, 90, 93, 97, 98, 101, 103, 104, 106, 107 curriculum, 1, 6, 7, 38, 45, 47, 48, 50, 52, 53, 56, 57, 58, 62, 67, 94, 97, 98, 99, 105, 113, 123 curriculum change, 58 cycles, 79, 130, 140 cyclical process, 76

D data analysis, 78 data collection, 78, 115 database, 14, 18 dating, 96 decentralization, 6, 7, 111, 122 decision-making process, 62 decisions, 8, 28, 50, 126 definition, 9 delivery, 101 delusion, 70 democratization, 43 Department of Education, 35, 75, 143, 144, 145, 146 deregulation, 7, 111 developed countries, 34, 138 deviation, 40

Index diet, 93 dietary, 93 diffusion, 24 directives, 87 disabled, 41 disappointment, 65 discipline, 8, 14, 25, 51, 52, 117, 123 discretionary, 111 disseminate, 48 distribution, 8, 140 diversity, 24, 62, 87 doctors, 120 downsizing, 24 duration, 25 duties, 9, 51, 67, 81, 112

E ears, 100, 118 earth, 133 East Asia, 1, 2, 3, 6, 9, 10, 31, 62, 70, 93, 94, 97, 104, 107, 108 economic development, 1, 24 economic status, 8, 134 education, 4, 5, 6, 7, 13, 14, 15, 17, 18, 19, 23, 24, 25, 28, 34, 35, 40, 42, 43, 45, 46, 47, 48, 49, 51, 52, 53, 54, 55, 56, 57, 58, 59, 63, 64, 65, 66, 67, 69, 70, 71, 75, 79, 80, 86, 88, 90, 91, 95, 96, 101, 102, 103, 107, 108, 109, 111, 112, 113, 117, 118, 119, 122, 126, 127, 129, 130, 141, 143, 144, 145, 146 education reform, 46 educational assessment, 7, 45, 93, 94, 102 educational background, 146 educational career, 101, 134 educational institutions, 4, 30, 47, 48, 49, 50, 52, 53, 55 educational objective, 113 educational policies, 1, 86, 144 educational policy, 1, 87 educational practices, 104 educational process, 104, 113 educational programs, 133 educational quality, 1, 46, 49, 76, 90, 111, 112, 117 educational research, 38, 82, 130 educational system, 2, 5, 8, 10, 35, 37, 46, 68, 75, 86, 87, 88, 90, 94, 97, 99, 102, 104, 106, 129, 130, 140 educators, 2, 5, 9, 94, 98 e-learning, 33 electives, 101 elementary school, 114, 119, 120 emotional, 51 Emotional Intelligence, 84

149

employment, 24, 42 empowered, 106 encouragement, 33, 38, 126 energy, 18, 32 engagement, 132, 134 England, 19, 64, 94, 107 English Language, 28 enrollment, 4, 36 enrollment rates, 36 enterprise, 31, 99 environment, 86, 87, 89, 115, 132, 135 epistemological, 24 equality, 63 equating, 58, 59 equity, 1, 2, 135, 140 ERA, 69 ESR, 6, 61, 64, 65, 66, 67, 68, 88, 89 ethics, 9, 49, 51 ethnic groups, 97 Europe, 145 exaggeration, 97 examinations, 2, 5, 7, 10, 23, 25, 26, 27, 28, 29, 30, 31, 32, 33, 56, 57, 61, 62, 63, 64, 66, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 109, 130 expert teacher, 52, 120, 124 expertise, 19, 104 external environment, 7, 88, 90 eyes, 106

F failure, 7, 82, 108 fairness, 4, 14, 20, 21, 124 faith, 51 family, 31, 53, 133, 134 fatigue, 98 fear, 87, 104 feedback, 26, 66, 68, 78, 115, 134 fees, 14 financial resources, 86, 106 financial support, 4, 86 fishing, 41 fitness, 28 flight, 24 flow, 115 fluid, 115 focusing, 7, 50, 54, 56, 90, 97, 99, 113 food, 93 foreign language, 57 formal education, 46, 52, 108 France, 14 freedom, 8, 86, 129, 130 full capacity, 18

150

Index

funding, 4, 13, 14, 15, 19

G gauge, 32, 136 gender, 8, 36, 140 gender differences, 8 General Certificate of Education (GCE), 25, 28, 64 generation, 33, 47 global competition, 1 global economy, 31 globalization, 2, 4, 22, 24, 43, 55, 91, 144 goal setting, 78 goals, 6, 9, 10, 15, 16, 21, 38, 54, 76, 77, 89, 90, 104, 105, 106, 112, 113, 126 goal-setting, 112 going to school, 7 governance, 2, 88 government, 1, 2, 4, 6, 10, 13, 14, 16, 18, 22, 23, 38, 42, 52, 55, 61, 66, 68, 86, 94, 96, 100, 111, 119, 129, 130, 140 government policy, 55 GPA, 45, 56, 57, 58 grades, 10, 25, 53, 58, 62, 65, 96, 98, 99, 100, 104, 130 grading, 57, 99, 103 grants, 19 Great Britain, 94 Greece, 145 group activities, 37 grouping, 134 groups, 13, 45, 48, 56, 57, 58, 94, 97, 105, 113, 132 growth, 94, 98, 100, 126 guessing, 5, 32 guidance, 89, 111, 112, 114 guidelines, 2, 4, 7, 43, 46, 49, 66, 88, 103, 112 guiding principles, 55

H handling, 13, 130, 135 hands, 130 happiness, 104 harm, 46 harmony, 46 hazards, 132 health, 46, 47, 112, 132 heart, 107 high school, 2, 5, 9, 13, 35, 36, 37, 38, 39, 40, 42, 94, 96, 97, 111, 115, 121, 129, 130 high school grades, 9 high scores, 10

higher education, 4, 5, 13, 14, 15, 16, 17, 18, 19, 22, 27, 35, 37, 39, 40, 43, 46, 47, 48, 49, 53, 55, 56, 57, 96, 118, 144 high-level, 20 high-stakes testing programs, 109 hiring, 1 holistic, 28, 101, 102, 107 Hong Kong, 1, 3, 6, 61, 62, 63, 64, 65, 67, 68, 69, 70, 71, 75, 76, 79, 84, 85, 86, 87, 88, 90, 91, 94, 107, 108, 130, 144 House, 70, 145 hub, 100 human, 5, 23, 43, 55, 87, 89, 106 human resource development, 55 human resources, 5, 23, 43, 87

I identification, 36, 78 IEA, 2, 24, 26, 30, 34 immigrants, 66 immigration, 134 implementation, 2, 3, 4, 5, 7, 8, 10, 17, 18, 19, 25, 38, 43, 44, 49, 50, 67, 69, 76, 77, 86, 87, 88, 89, 90, 93, 94, 102, 103, 106, 126, 132 incentives, 67, 106 inclusion, 23, 135 increased workload, 68 independence, 15 India, 144, 145 Indian, 94 Indiana, 145 indication, 32 indicators, 4, 16, 20, 45, 47, 48, 56, 57, 58, 76, 79, 82, 83, 85, 89, 113, 135, 137 indices, 30 indigenous, 5, 94 individual characteristics, 115 individual differences, 1 Indonesia, 29 induction, 8, 121 industry, 19, 100 infancy, 101 inferences, 136 inflation, 57, 58, 100 information and communication technology (ICT), 24, 33, 43, 100, 101, 134, 145 information technology, 50, 62 infrastructure, 130, 131 inherited, 94 initiation, 87 innovation, 2, 5, 10, 31, 34, 48, 50, 55, 79, 125, 143 inspection, 6, 7, 62, 64, 65, 75, 76, 79, 111, 113 inspections, 63, 76, 87

Index institutionalization, 18 institutions, 1, 4, 13, 14, 15, 19, 21, 30, 46, 47, 48, 49, 50, 52, 53, 55, 57, 140 institutions of higher education, 13 instruction, 8, 10, 25, 52, 63, 69, 94, 101, 106, 114, 129, 130, 134, 144 instructional activities, 50 instruments, 24, 33, 101, 103, 105, 130, 132 integrity, 46, 51 intellect, 46 intentions, 68 interactions, 115 interactivity, 105 interdependence, 88 international standards, 23, 33 internationalization, 17, 22 internet, 33, 66, 134 interview, 40, 41 interviews, 28, 69, 78 investigative, 58 investment, 43 island, 23 Italy, 141 item response theory, 58

J jackknife, 135 Japan, 3, 7, 9, 24, 34, 94, 96, 97, 107, 111, 112, 113, 114, 116 Japanese, 109, 114 jobs, 99 judge, 130, 137, 140 judges, 132 judgment, 43, 55, 65, 117, 124, 126 junior high, 114 junior high school, 114 jurisdiction, 114

K K-12, 4 kindergarten, 8, 129, 130 King, 52 knowledge acquisition, 103 knowledge economy, 13 knowledge-based economy, 55, 104 Korea, 5, 35, 36, 38, 41, 43, 97, 108, 143 Korean, 38, 39, 40, 43, 143

L L1, 28 labor, 24 lack of opportunities, 9

151

land, 24 language, 2, 27, 56, 62, 64, 66, 67 language proficiency, 62, 64 large-scale, 63, 99, 105, 132, 144 later life, 97 leadership, 6, 10, 48, 49, 65, 66, 77, 89, 90, 134, 135, 144 leakage, 99 leaks, 103, 108 learners, 46, 47, 50, 51, 55, 58, 105, 108, 123, 125 learning, 2, 4, 5, 7, 8, 9, 10, 16, 21, 23, 24, 26, 27, 28, 29, 30, 31, 32, 33, 34, 38, 46, 47, 51, 54, 55, 58, 62, 63, 64, 65, 66, 67, 68, 69, 70, 78, 79, 82, 83, 85, 88, 89, 90, 94, 97, 98, 99, 100, 101, 103, 104, 105, 107, 108, 113, 115, 123, 125, 126, 134, 135, 140 learning culture, 88, 94, 99 learning difficulties, 97 learning environment, 134, 135, 140 learning outcomes, 54, 58 learning process, 100, 101 learning society, 46, 47 learning styles, 101 licenses, 49 lifelong learning, 43 lifestyles, 114 lifetime, 101 linear, 58, 137 linkage, 6, 48, 61, 67, 68 links, 32, 33, 130, 132 listening, 38 literacy, 8, 71, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 140 local community, 111, 112 local government, 96 location, 115, 136 logistics, 30, 100 London, 10, 69, 91, 107, 109, 126 Los Angeles, 143

M Macao, 3, 8, 9, 94, 129, 130, 131, 132, 137, 140, 141, 143 Macau, 8, 129, 132, 140, 141, 143, 144 machinery, 25 Madison, 146 Mainland China, 94 maintenance, 140 Malaysia, 3, 7, 93, 94, 96, 97, 99, 100, 101, 102, 103, 104, 106, 107, 108, 109, 145 management, 1, 3, 4, 6, 7, 17, 18, 35, 42, 47, 50, 54, 55, 62, 65, 67, 69, 76, 78, 79, 81, 82, 83, 86, 88, 89, 91, 100, 101, 112, 113, 123, 131, 144

152

Index

management committee, 86 mandates, 7 manipulation, 77 manpower, 76 market, 100 mathematical knowledge, 132 mathematics, 2, 4, 5, 23, 24, 25, 26, 27, 29, 30, 31, 33, 34, 65, 66, 70, 98, 114 Mauritius, 107 meanings, 61 measurement, 2, 45, 50, 55, 105, 130, 136, 143 measures, 6, 10, 21, 55, 57, 61, 62, 63, 67, 68, 88, 112, 134, 135 media, 24, 26, 30, 33, 50, 94 medicine, 17, 20, 56, 57 membership, 14 memorizing, 54, 98 mental health, 47 mental illness, 114 MES, 95, 99, 103 mid-career, 114 middle schools, 42 Ministry of Education, 7, 8, 13, 14, 25, 27, 28, 29, 30, 32, 33, 39, 44, 59, 76, 91, 93, 94, 95, 96, 99, 101, 102, 104, 105, 106, 107, 108, 116, 117, 118, 119, 121, 122, 126, 144, 145 minority students, 108 mirror, 1, 98 misconceptions, 34, 87 missions, 9 mobile device, 62 mobility, 96, 97, 99, 106 modeling, 105 models, 2, 3, 9, 20, 50, 65, 79, 105 modules, 81 MOE, 4, 8, 13, 14, 15, 17, 19, 20, 21, 36, 39, 42, 44, 94, 122 momentum, 28 money, 19, 24, 129 moral behavior, 10 morale, 87, 89, 90 morality, 8, 46, 51, 52, 117, 125 mother tongue, 25 motion, 116 motivation, 106, 114 movement, 6, 75, 77, 90 MSC, 100, 108, 109 multi-ethnic, 94 multimedia, 100, 105 multiple regression, 135 music, 57, 98

N

Nanyang Technological University, 23, 145 nation, 13, 24, 53, 104 nation building, 24 National Center for Education Statistics, 145 national policy, 52 National Research Council, 105, 109 natural, 5, 23, 28, 31, 47, 96, 132 natural resources, 5, 23, 28, 31, 132 neoliberal, 7 Nepal, 145 Netherlands, 14 New Jersey, 145, 146 New York, 10, 59, 70, 91, 107, 108 New Zealand, 27, 96, 97 newspapers, 137 nongovernmental, 19 nongovernmental organization, 19 non-native, 63 normal, 9, 31, 56, 86 normal development, 9 norms, 24, 34, 89 North Carolina, 108

O objectivity, 4, 14, 20, 104 observations, 26, 66, 67, 78, 100, 137 OECD, 2, 63, 131, 132, 135, 136, 138, 141 online, 140 on-line, 101 openness, 43, 66 oral, 100 organizational culture, 20 orientation, 94, 97, 98, 104, 106, 126 outsourcing, 24 overweight, 28 ownership, 33

P Pacific, 107, 108 pain, 104 paradigm shift, 86 parental support, 68 parents, 2, 8, 10, 42, 66, 76, 79, 82, 85, 89, 93, 94, 97, 101, 103, 104, 106, 111 Paris, 10, 107, 132, 141 partnership, 6, 23, 64, 65, 70, 75, 86 passive, 18, 20, 87, 88 pedagogical, 1, 9, 99 pedagogy, 100 peer, 65, 67, 125 Pennsylvania, 145 percentile, 40

Index perception, 97, 134 perceptions, 65, 67, 79, 82 performance appraisal, 62 performance indicator, 65, 66, 76, 77, 79, 81, 82, 83, 85, 88, 90, 91, 113 periodic, 131 personality, 51, 98, 103 Philadelphia, 119, 126 Philippines, 29 philosophy, 49, 56, 98, 107 physical chemistry, 145 physical education, 98 physical fitness, 28 physics, 145 pilot study, 135 PISA, 8, 9, 62, 63, 69, 129, 130, 131, 132, 133, 134, 135, 136, 137, 139, 140, 141, 143 pitch, 32 planning, 15, 18, 49, 69, 76, 78, 87, 126, 132 plants, 43 play, 104, 124 pluralistic, 94 plurality, 87 policy making, 88 policymakers, 67, 68, 94 political leaders, 104 pools, 15 poor, 82, 136 poor performance, 82 population, 35, 38, 129, 135, 136 portfolios, 99, 105 positive attitudes, 2 postsecondary education, 1, 106 power, 6, 42, 87, 98 powers, 111 PPS, 135 preference, 82 president, 145 pressure, 7, 10, 31, 87, 97, 98, 100, 106, 114 primary care, 111 primary school, 5, 25, 31, 33, 34, 37, 42, 65, 67, 69, 79, 81, 94, 96, 97, 99 private, 9, 14, 19, 43, 46, 58, 86, 93, 96, 99, 104, 107, 114, 118, 120, 129, 130, 140 private-sector, 14, 19 probability, 32, 135 probe, 108 problem solving, 78 problem-solving skills, 26, 98, 102, 105 problem-solving strategies, 98 productivity, 2 professional development, 49, 64, 70, 91, 106, 126 professional duties, 9, 67

153

professional teacher, 9 professionalism, 9, 89, 90 professionalization, 7, 98 program, 1, 6, 8, 9, 10, 35, 42, 45, 53, 58, 81, 85, 100, 101, 106, 121, 129, 132, 134, 140, 145 progress reports, 64 public, 1, 2, 4, 8, 10, 15, 19, 20, 21, 22, 26, 27, 29, 30, 38, 42, 43, 46, 47, 49, 52, 55, 57, 61, 62, 63, 64, 76, 94, 95, 96, 97, 98, 99, 100, 102, 104, 105, 106, 112, 114, 117, 118, 120, 121, 122, 129, 130, 132, 137, 140 public domain, 26 public education, 42 public interest, 52 public schools, 8, 117, 120, 129 public welfare, 43 publishers, 145 pupils, 89, 100, 115

Q qualifications, 8, 27, 42, 50, 117, 118, 120, 126 qualitative research, 86, 90 quality assurance (QA), 2, 6, 22, 29, 42, 45, 46, 47, 48, 49, 54, 58, 61, 63, 67, 69, 71, 75, 76, 77, 81, 88, 102, 113, 117, 125 quality control, 7, 23, 24, 25, 48 quality improvement, 10, 16, 42, 69 questionnaires, 38, 78, 89, 132, 134, 135 quorum, 41 quotas, 17, 19, 56

R race, 106, 107 random, 5, 20 range, 3, 26, 62, 81, 106, 136 ratings, 13, 21 reading, 4, 6, 8, 45, 54, 55, 58, 65, 71, 129, 130, 131, 132, 133, 134, 135, 138, 139, 140 reading skills, 132 reality, 101, 104, 111 reasoning, 105 recession, 87 recognition, 17, 31, 68, 78, 96, 102 refining, 66 reflection, 68, 81 reforms, 2, 5, 43, 62, 77, 79, 86, 87, 91, 99, 106, 113 regional, 34 regression method, 58 regressions, 135 regular, 14, 23, 38, 41, 55, 100 regulations, 14, 20, 111 relationship, 9, 38, 39, 137

154

Index

relationships, 82, 99, 115, 137 relatives, 93 reliability, 28, 29, 84 reputation, 15 Research and Development, 59, 70, 100, 130, 138, 145 research design, 38 resentment, 65 resistance, 68, 86, 87, 90 resource allocation, 1 resources, 1, 5, 7, 13, 16, 19, 23, 28, 31, 43, 64, 86, 87, 88, 89, 90, 100, 106, 132, 134 responsibilities, 9, 112 revenue, 19 rewards, 96 rhetoric, 6, 61, 68 righteousness, 47 risk, 114 roadmap, 55, 130, 135 rote learning, 98 rubrics, 31, 134, 135 rural, 41, 120

S safeguard, 4, 15 safety, 112 salaries, 31 salary, 52, 120, 121, 122 sample, 86, 105, 132, 134, 135, 136 sample design, 135 sampling, 130, 135 sampling error, 130, 135 SAR, 3, 5, 6, 35, 36, 37, 40, 41, 43, 65, 71 satisfaction, 23, 42, 113 SBA, v, 7, 62, 93, 94, 99, 101, 102, 103, 105, 106 scaling, 130, 136 schema, 78 scholastic achievement, 112, 114 school activities, 37, 44, 113 school climate, 134 school community, 96, 115 school culture, 78, 87 school enrollment, 62 school management, 1, 67, 76, 86 school performance, 2, 76, 77, 84, 89, 94 school support, 88 school work, 82, 86 schooling, 2, 31, 36, 82, 98, 102, 107, 135, 138 science education, 137, 145 scientific knowledge, 132 scores, 10, 28, 37, 40, 66, 68, 105, 106, 123, 125, 126 search, 2, 140

search engine, 140 second language, 27 secondary education, 26, 29, 46, 52, 95 secondary schools, 23, 28, 42, 79, 81, 94, 96, 99 secondary teachers, 42 secretariat, 132 Secretary General, 144 security, 99 selecting, 7, 20, 39, 41, 44, 89, 90, 97 self, 6, 15, 20, 48, 64, 65, 69, 70, 76, 77, 78, 79, 80, 81, 86, 91, 112, 123, 134 self-assessment, 7, 48, 111 self-control, 48 self-discipline, 51 self-esteem, 126 self-evaluations, 14, 20, 87 self-improvement, 13, 76, 83 self-reflection, 42, 87 self-renewal, 75, 77, 78, 79, 82, 85, 90 self-report, 28 series, 6, 75, 79, 85, 94, 96 services, 14, 20, 29, 46, 47, 86, 100, 134 Shanghai, 70 shape, 19 shaping, 7, 81, 93, 94 shares, 114 sharing, 6, 69, 81, 90 shortage, 4, 87 short-term, 89 signals, 114 Singapore, 3, 5, 23, 24, 25, 26, 27, 28, 29, 30, 31, 33, 34, 94, 97, 107 skills, 7, 9, 26, 28, 35, 37, 42, 45, 47, 51, 52, 53, 55, 56, 57, 58, 77, 78, 79, 82, 85, 86, 91, 97, 98, 101, 102, 104, 105, 106, 126, 132, 134 Slovenia, 144 social behavior, 35, 36 social class, 137 social context, 2, 3 social development, 47, 48, 51 social problems, 94 social services, 20 social skills, 47 social status, 96 socialization, 107 software, 100 South Korea, 3, 5, 35, 36, 38, 40, 41, 42, 43, 94, 109 Southeast Asia, 94 Spain, 70 specialization, 9 speech, 51, 68 speed, 17, 32 spiritual, 46, 48

Index sports, 31 SPSS, 141 Sri Lanka, 107 SSB, 61, 67, 68 stability, 113 staff development, 65, 81, 83, 86 stages, 19, 29, 52, 67, 68, 78, 103, 130 stakeholder, 33, 113 stakeholders, 2, 5, 10, 23, 24, 29, 33, 43, 56, 65, 68, 81, 82, 87, 113 standard deviation, 40 standard of living, 24 standards, 2, 5, 6, 7, 8, 17, 23, 24, 26, 27, 28, 29, 30, 32, 33, 42, 45, 46, 47, 48, 49, 50, 51, 53, 54, 57, 58, 59, 64, 66, 102, 103, 104, 106, 108, 113, 117, 130, 140 statistics, 43, 135 stigma, 30 stimulus, 105 strategies, 3, 76, 77, 78, 89, 98, 126, 130, 134 streams, 25, 28 strength, 25 stress, 68, 69, 98 structuring, 113 student achievement, 2, 9, 45, 46, 56, 57, 58, 102, 105, 106 student behavior, 112 student characteristics, 134 student development, 8, 54, 58, 117, 125 student proficiency, 56 subjective, 55, 126 subsidies, 14, 19, 140 subsidy, 129 suburban, 120 suffering, 87 Sun, 4, 5, 6, 17, 35, 75, 108, 143, 144 Sunday, 108 superiority, 32 supervision, 6, 47, 61, 68, 111, 144 supervisor, 125 supervisors, 49, 53 supply, 42, 99 surgery, 77 survival, 31 sustainable development, 15, 20, 75 synthesis, 125 systems, 2, 3, 4, 6, 8, 9, 10, 28, 31, 54, 58, 61, 62, 67, 68, 86, 89, 106, 111, 112, 126, 133, 145

T Taiwan, 1, 3, 4, 13, 14, 15, 17, 18, 19, 20, 21, 22, 94, 96, 97, 144 talent, 13, 15

155

target population, 136 targets, 7, 76, 89, 90, 113 taxonomy, 26 teacher assessment, 62, 122 teacher effectiveness, 126 teacher preparation, 126 teacher training, 6, 55, 58, 112, 120, 121 teachers, 2, 4, 5, 6, 7, 8, 9, 10, 16, 26, 31, 32, 33, 35, 36, 38, 39, 42, 43, 46, 47, 48, 49, 50, 52, 53, 54, 55, 59, 61, 64, 65, 66, 67, 68, 69, 76, 78, 79, 81, 82, 83, 85, 86, 87, 89, 98, 99, 100, 101, 102, 103, 104, 105, 106, 108, 112, 113, 114, 115, 117, 118, 119, 120, 121, 122, 124, 126 teaching, 1, 2, 4, 7, 8, 9, 14, 16, 18, 19, 20, 21, 28, 30, 33, 35, 37, 38, 42, 48, 49, 50, 52, 54, 64, 65, 66, 67, 68, 70, 78, 82, 83, 88, 89, 96, 97, 98, 100, 102, 105, 111, 112, 114, 115, 117, 120, 121, 122, 123, 124, 125, 126, 134, 140 teaching evaluation, 14, 18, 126 teaching quality, 16, 21 technology, 6, 24, 30, 50, 89, 90, 100, 105, 132, 134, 145 telecommunications, 100 tension, 87 territory, 64, 66, 87, 89, 129, 130, 140 tertiary education, 94 test items, 38, 39, 136 test scores, 7 testimony, 94 Texas, 108, 144, 146 textbooks, 143 Thai, 45, 46, 47, 52, 54, 55, 56, 58, 118, 120, 121 Thai Language, 52, 54 Thailand, vi, 3, 6, 8, 9, 29, 45, 46, 47, 49, 50, 53, 55, 56, 57, 58, 59, 117, 118, 119, 120, 124, 126, 127, 145 Thessaloniki, 145 thinking, 6, 13, 18, 20, 26, 45, 54, 58, 103, 106 third party, 29, 32, 113 threatened, 67 threats, 68, 77 threshold, 67 time, 5, 10, 16, 32, 33, 37, 40, 43, 49, 52, 56, 64, 76, 86, 87, 89, 93, 98, 101, 102, 103, 105, 114, 129, 132, 136 time frame, 49, 105 Tokyo, 111, 114, 115, 146 tolerance, 104 Tongue, 25, 28 top-down, 75 tracking, 134 trade-off, 105 tradition, 7, 108, 130, 140

156

Index

training, 6, 15, 20, 33, 53, 55, 58, 68, 78, 81, 82, 85, 88, 102, 103, 106, 112, 120, 121 training programs, 88 transfer, 36 transformation, 55, 85, 89, 108 transition, 7, 103 transparency, 66 transparent, 20, 29 trees, 32 trial, 42, 113 triangulation, 85 trust, 42, 78, 103, 114, 115 TSA, 6, 61, 64, 66, 67, 68 tuition, 93, 107 turbulent, 86, 87, 89 tutoring, 43, 93, 98, 107, 108 twinning, 96

U uncertainty, 68 undergraduate, 117, 120 UNESCO, 10, 69, 107, 145 UNICEF, 107 uniform, 113 unions, 5, 43 United Kingdom (UK), 27, 96, 97, 145 United States, 9, 14, 17, 96, 97, 144 universities, 4, 9, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 27, 37, 39, 40, 46, 55, 56, 63, 65, 93, 95, 96, 97, 106 university collaboration, 96 university education, 13, 25, 26, 95

validity, 27, 28, 29, 84, 105, 124, 126 values, 3, 6, 28, 62, 89, 90, 96, 136 variability, 26, 27 variables, 38, 39 variation, 135 Victoria, 91, 108 Vietnam, 144, 145 visible, 105 vision, 22, 51, 77, 88, 104 vocational, 13, 25, 46, 95, 96, 99 vocational tracks, 95, 96, 99 voice, 32 volunteer work, 37

W weakness, 7, 130 wealth, 134 well-being, 28 Western countries, 24 wind, 32 winning, 17 Wisconsin, 146 wisdom, 10, 53, 58 workers, 24 workforce, 13, 24 working hours, 31 workload, 7, 33, 61, 66, 100, 105, 114 workplace, 106 World War, 111 World War I, 111 World War II, 111 writing, 6, 40, 41, 45, 53, 54, 55, 58, 65

Y

V yield, 30 validation, 32