EDUCATION IN A COMPETITIVE AND GLOBALIZING WORLD SERIES
SPECIALIZED RASCH MEASURES APPLIED AT THE FOREFRONT OF EDUCATI...
49 downloads
711 Views
4MB Size
Report
This content was uploaded by our users and we assume good faith they have the permission to share this book. If you own the copyright to this book and it is wrongfully on our website, we offer a simple DMCA procedure to remove your content from our site. Start by pressing the button below!
Report copyright / DMCA form
EDUCATION IN A COMPETITIVE AND GLOBALIZING WORLD SERIES
SPECIALIZED RASCH MEASURES APPLIED AT THE FOREFRONT OF EDUCATION No part of this digital document may be reproduced, stored in a retrieval system or transmitted in any form or by any means. The publisher has taken reasonable care in the preparation of this digital document, but makes no expressed or implied warranty of any kind and assumes no responsibility for any errors or omissions. No liability is assumed for incidental or consequential damages in connection with or arising out of information contained herein. This digital document is sold with the clear understanding that the publisher is not engaged in rendering legal, medical or any other professional services.
EDUCATION IN A COMPETITIVE AND GLOBALIZING WORLD SERIES Success in Mathematics Education Caroline B. Baumann (Editor) 2009. ISBN: 978-1-60692-299-6 Mentoring: Program Development, Relationships and Outcomes Michael I. Keel (Editor) 2009. ISBN: 978-1-60692-287-3 Mentoring: Program Development, Relationships and Outcomes Michael I. Keel (Editor) 2009. ISBN: 978-1-60876-727-4 (E-book) Motivation in Education Desmond H. Elsworth (Editor) 2009. ISBN: 978-1-60692-234-7 Evaluating Online Learning: Challenges and Strategies for Success Arthur T. Weston (Editor) 2009. ISBN: 978-1-60741-107-9 Enhancing Prospects of Longer-Term Sustainability of Cross-Cultural INSET Initiatives in China Chunmei Yan 2009. ISBN: 978-1-60741-615-9 Reading at Risk: A Survey of Literary Reading in America Rainer D. Ivanov 2009. ISBN: 978-1-60692-582-9 Reading: Assessment, Comprehension and Teaching Nancy H. Salas and Donna D. Peyton (Editors) 2009. ISBN: 978-1-60692-615-4
Reading: Assessment, Comprehension and Teaching Nancy H. Salas and Donna D. Peyton (Editors) 2009. ISBN: 978-1-60876-543-0 (E-book) Multimedia in Education and Special Education Onan Demir and Cari Celik (Editors) 2009. ISBN: 978-1-60741-073-7 Rural Education in the 21st Century Christine M.E. Frisiras (Editor) 2009. ISBN: 978-1-60692-966-7 Nutrition Education and Change Beatra F. Realine (Editor) 2009. ISBN: 978-1-60692-983-4
What Content-Area Teachers Should Know About Adolescent Literacy National Institute for Literacy 2009. ISBN: 978-1-60741-137-6 The Reading Literacy of U.S. Fourth-Grade Students in an International Context Justin Baer, Stéphane Baldi, Kaylin Ayotte, Patricia J. Green and Daniel McGrath 2009. ISBN: 978-1-60741-138-3 Teacher Qualifications and Kindergartners’ Achievements Cassandra M. Guarino, Laura S. Hamilton, J.R. Lockwood, Amy H. Rathbun and Elvira Germino Hausken 2009. ISBN: 978-1-60741-180-2 PCK and Teaching Innovations Syh-Jong Jang 2009. ISBN: 978-1-60741-147-5
IT- Based Project Change Management System Faisal Manzoor Arain and Low Sui Pheng 2009. ISBN: 978-1-60741-148-2 Learning in the Network Society and the Digitized School Rune Krumsvik (Editor) 2009. ISBN: 978-1-60741-172-7 Effects of Family Literacy Interventions on Children'sAcquisition of Reading Ana Carolina Pena (Editor) 2009. ISBN: 978-1-60741-236-6 Developments in Higher Education Mary Lee Albertson (Editor) 2009. ISBN: 978-1-60876-113-5 Career Development Hjalmar Ohlsson and Hanne Borg (Editors) 2009. ISBN: 978-1-60741-464-3 Special Education in the 21st Century MaryAnn T. Burton (Editor) 2010. ISBN: 978-1-60741-556-5 Academic Self-Beliefs in the Spelling Domain Günter Faber 2010. ISBN: 978-1-60741-603-6 Approaches to Early Childhood and Elementary Education Francis Wardle 2009. ISBN: 978-1-60741-643-2
Challenges of Quality Education in SubSaharan African Countries Daniel Namusonge Sifuna and Nobuhide Sawamura 2009. ISBN: 978-1-60741-509-1
Academic Administration: A Quest for Better Management and Leadership in Higher Education Sheying Chen (Editor) 2009. ISBN: 978-1-60741-732-3 Academic Administration: A Quest for Better Management and Leadership in Higher Education Sheying Chen (Editor) 2009. ISBN: 978-1-61668-571-3 (E-book) Recent Trends in Education Borislav Kuzmanović and Adelina Cuevas (Editors) 2009. ISBN: 978-1-60741-795-8 Expanding Teaching and Learning Horizons in Economic Education Franklin G. Mixon, Jr. and Richard J. Cebula 2009. ISBN: 978-1-60741-971-6 New Research in Education: Adult, Medical and Vocational Edmondo Balistrieri and Giustino DeNino (Editors) 2009. ISBN: 978-1-60741-873-3
Collaborative Learning: Methodology, Types of Interactions and Techniques Edda Luzzatto and Giordano DiMarco (Editors) 2010. ISBN: 978-1-60876-076-3 Handbook of Lifelong Learning Developments Margaret P. Caltone (Editor) 2010. ISBN: 978-1-60876-177-7 Sexuality Education Kelly N. Stanton (Editor) 2010. ISBN: 978-1-60692-153-1
Disadvantaged Students and Crisis in Faith-Based Urban Schools Thomas G. Wilson 2010. ISBN: 978-1-60741-535-0 Educational Change Aden D. Henshall and Bruce C. Fontanez (Editors) 2010. ISBN: 978-1-60876-389-4 Delving into Diversity: An International Exploration of Issues of Diversity in Education Vanessa Green and Sue Cherrington (Editors) 2010. ISBN: 978-1-60876-361-0 Health Education: Challenges, Issues and Impact André Fortier and Sophie Turcotte (Editors) 2010. ISBN: 978-1-60876-568-3 Educational Games: Design, Learning and Applications Frej Edvardsen and Halsten Kulle (Editors) 2010. ISBN: 978-1-60876-692-5 Attracting International Students for Higher Education Russell C. Carswell (Editor) 2010. ISBN: 978-1-60741-666-1 Attracting International Students for Higher Education Russell C. Carswell (Editor) 2010. ISBN: 978-1-61668-388-7 (E-book) The Process of Change in Education: Moving from Descriptive to Prescriptive Research Baruch Offir 2010. ISBN: 978-1-60741-451-3
Adopting Blended Learning for Collaborative Work in Higher Education Alan Hogarth 2010. ISBN: 978-1-60876-260-6 America's Historically Black Colleges and Universities Giovani Lucisano (Editor) 2010. ISBN: 978-1-60741-510-7 Medical Education: The State of the Art Rossana Salerno-Kennedy and Siún O’Flynn (Editors) 2010. ISBN: 978-1-60876-194-4 Reading in 2010: A Comprehensive Review of a Changing Field Michael F. Shaughnessy (Editor) 2010. ISBN: 978-1-60876-659-8 Music Education João Hermida and Mariana Ferreo (Editors) 2010. ISBN: 978-1-60876-655-0 Computer-Assisted Teaching: New Developments Brayden A. Morris and George M. Ferguson (Editors) 2010. ISBN: 978-1-60876-855-4 Handbook of Curriculum Development Limon E. Kattington (Editor) 2010. ISBN: 978-1-60876-527-0 Handbook of Curriculum Development Limon E. Kattington (Editor) 2010. ISBN: 978-1-61668-350-4 (E-book) Sexuality Education and Attitudes Jovan Stanovic and Milo Lalic (Editors) 2010. ISBN: 978-1-60741-662-3
National Financial Literacy Strategy Toma P. Hendriks (Editor) 2010. ISBN: 978-1-60741-827-6
Reading First Impact Study - Part II Martha O. Parkis (Editor) 2010. ISBN: 978-1-60876-891-2
Virtual Worlds: Controversies at the Frontier of Education Kieron Sheehy, Rebecca Ferguson and Gill Clough (Editors) 2010. ISBN: 978-1-60876-261-3
Collaborative and Individual Learning in Teaching Julien Mercie, Caroline Girard, Monique Brodeur and Line Laplante 2010. ISBN: 978-1-60876-889-9
Education in Asia Hiroto S. Nakamura (Editor) 2010. ISBN: 978-1-61668-617-8
Postsecondary Education and Student Aid Jennifer G. Hartley (Editor) 2010. ISBN: 978-1-60876-935-3
Becoming an Innovative Teacher Educator: Designing and Developing a Successful Hybrid Course Qiuyun Lin 2010. ISBN: 978-1-60876-465-5 Higher Education: Teaching, Internationalization and Student Issues Magnus E. Poulsen (Editor) 2010. ISBN: 978-1-61668-548-5 Educational Assessment: Measuring Progress in Schools Oliver T. Ricoha (Editor) 2010. ISBN: 978-1-60876-818-9 Decentralization and Marketization: The Changing Landscape of China's Adult and Continuing Education Ning Rong Liu 2010. ISBN: 978-1-60876-835-6 Issues in English Language Learning Gerald. E. Albertson (Editor) 2010. ISBN: 978-1-60876-848-6 Multiculturalism in the Age of the Mosaic Michael Ọladẹjọ Afọláyan (Editor) 2010. ISBN: 978-1-60876-995-7
Postsecondary Education and Student Aid Jennifer G. Hartley (Editor) 2010. ISBN: 978-1-61668-428-0 (E-book) Education to Meet New Challenges in a Networked Society Leo Jansen, Paul Weaver and Rietje van Dam-Mieras 2010. ISBN: 978-1-61668-245-3 Time Out: Examining Seclusion and Restraint in Schools Laura E. Kentley (Editor) 2010. ISBN: 978-1-60876-932-2 Education Reforms in Ghana: Curriculum in Junior High Schools George M. Osei 2010. ISBN: 978-1-60876-948-3 Effective Teaching for Intended Learning Outcomes in Science and Technology (Metilost) Bernardino Lopes, Paulo Cravino and A.A. Silva 2010. ISBN: 978-1-60876-958-2
Specialized Rasch Measures Applied at the Forefront of Education Russell F. Waugh (Editor) 2010. ISBN: 978-1-61668-032-9 Applications of Rasch Measurement in Education Russell Waugh (Editor) 2010. ISBN: 978-1-61668-026-8 Striving for the Perfect Classroom: Instructional and Assessment Strategies to Meet the Needs of Today’s Diverse Learners Kelli R. Paquette and Sue A. Rieg 2010. ISBN: 978-1-61668-039-8
Lifelong Learning: Theoretical and Practical Perspectives on Adult Numeracy and Vocational Mathematics Gail E. FitzSimons 2010. ISBN: 978-1-61668-291-0 Lifelong Learning: Theoretical and Practical Perspectives on Adult Numeracy and Vocational Mathematics Gail E. FitzSimons 2010. ISBN: 978-1-61668-705-2 (E-book) Spelling Skills: Acquisition, Abilities, and Reading Connection Blake C. Fabini (Editor) 2010. ISBN: 978-1-61668-472-3
Medical Education in the New Millennium Amal A. El-Moamly 2010. ISBN: 978-1-61668-206-4
Spelling Skills: Acquisition, Abilities, and Reading Connection Blake C. Fabini (Editor) 2010. ISBN: 978-1-61668-531-7 (E-book)
Medical Education in the New Millennium Amal A. El-Moamly 2010. ISBN: 978-1-61668-456-3 (E-book)
Kindergartens: Programs, Functions and Outcomes Spencer B. Thompson (Editor) 2010. ISBN: 978-1-61668-530-0
Sen's Capability Approach and Institutions Shankaran Nambiar 2010. ISBN: 978-1-61668-508-9
Kindergartens: Programs, Functions and Outcomes Spencer B. Thompson (Editor) 2010. ISBN: 978-1-61668-711-3 (E-book)
EDUCATION IN A COMPETITIVE AND GLOBALIZING WORLD SERIES
SPECIALIZED RASCH MEASURES APPLIED AT THE FOREFRONT OF EDUCATION
RUSSELL F. WAUGH EDITOR
Nova Science Publishers, Inc. New York
Copyright © 2010 by Nova Science Publishers, Inc. All rights reserved. No part of this book may be reproduced, stored in a retrieval system or transmitted in any form or by any means: electronic, electrostatic, magnetic, tape, mechanical photocopying, recording or otherwise without the written permission of the Publisher. For permission to use material from this book please contact us: Telephone 631-231-7269; Fax 631-231-8175 Web Site: http://www.novapublishers.com NOTICE TO THE READER The Publisher has taken reasonable care in the preparation of this book, but makes no expressed or implied warranty of any kind and assumes no responsibility for any errors or omissions. No liability is assumed for incidental or consequential damages in connection with or arising out of information contained in this book. The Publisher shall not be liable for any special, consequential, or exemplary damages resulting, in whole or in part, from the readers‟ use of, or reliance upon, this material. Independent verification should be sought for any data, advice or recommendations contained in this book. In addition, no responsibility is assumed by the publisher for any injury and/or damage to persons or property arising from any methods, products, instructions, ideas or otherwise contained in this publication. This publication is designed to provide accurate and authoritative information with regard to the subject matter covered herein. It is sold with the clear understanding that the Publisher is not engaged in rendering legal or any other professional services. If legal or any other expert assistance is required, the services of a competent person should be sought. FROM A DECLARATION OF PARTICIPANTS JOINTLY ADOPTED BY A COMMITTEE OF THE AMERICAN BAR ASSOCIATION AND A COMMITTEE OF PUBLISHERS. LIBRARY OF CONGRESS CATALOGING-IN-PUBLICATION DATA Specialized Rasch measures applied at the forefront of education / editor, Russell F. Waugh. p. cm. Includes index. ISBN 978-1-61209-908-8 (eBook) 1. Educational tests and measurements--Cross-cultural studies. 2. Rasch models. I. Waugh, Russell. LB3051.S69 2009 371.26--dc22 2010001176
Published by Nova Science Publishers, Inc. † New York
CONTENTS Preface
xi
Author Biographies Chapter 1
Chapter 2
Chapter 3
Chapter 4
Chapter 5
Chapter 6
xiii
A Rasch Measure of Student Receptivity to Project Work at a Junior College in Singapore Choe Kee Cheng and Russell Waugh
1
Rasch Measures for Sports, Drama and Music Student Self-Views Based on Gardner Intelligences Ahdielah Edries and Russell F. Waugh
23
Teacher Guttman Scales and Teacher Views at an Islamic College Ahdielah Edries and Russell F. Waugh
49
Rasch Measures of Form Constancy of Letters and Numbers, and Letters in Words for Young Children Janet Richmond, Russell F. Waugh and Deslea Konza
65
Rasch Measures of Number Discrimination and Reversal, and Numbers in Calculations for Young Children Janet Richmond, Russell F. Waugh and Deslea Konza
83
Rasch Measures of Self-Discipline and Moderation in Mathematics Education Liu Shiueh Ling and Russell F. Waugh
Chapter 7
Rasch Measures of Dependability and Responsibility Liu Shiueh Ling and Russell F. Waugh
Chapter 8
A Rasch Measure of the Student Entrepreneurial Mindset in Singapore Wong Heng Aik Jason and Russell Waugh
101 131
153
x Chapter 9
Chapter 10
Index
Contents A Rasch Measure Linking Self-Reported Student Attitude and Behavior to Mathematics Radha Devi Unnithan and Russell Waugh
181
A Rasch Measure of University Students‟ Receptivity to Peers with Disabilities Across Two cultures Minoti Biswas and Russell Waugh
205 225
PREFACE This book contains Rasch measurement research papers that were based on part of some of my recent doctoral student theses that I supervised within the Faculty of Education and Arts at Edith Cowan University and the Graduate School of Education at the University of Western Australia (2006 to 2009). My doctoral students worked very hard on their theses investigating the relevant literature, mastering Rasch measurement and the Rasch Unidimensional Measurement Models (RUMM) computer program with its relevant statistics. Most of the doctoral students had families and jobs at the same time as they performed their research and, while all would say that it was a wonderful, rewarding experience during which time they learned a lot, they would also say that it was very hard work writing and re-writing their research output. In this book, they share some of their research output with you. To the best of our knowledge, all the Rasch measures reported here have not been performed by any other researcher in the world, up to the time of the thesis. In making a measure of a variable intended for Rasch analysis, items are designed to be conceptually ordered by difficulty along an increasing continuum from easy to harder for that variable. For the purpose of explanation here, I shall use three items ordered from easy to medium to hard. In designing the items, one keeps in mind that the respondent measures of the variable are conceptualised as being ordered along the continuum from low to high according to certain conditions. The conditions in the three-item example are that respondents with low measures will have a high probability of answering the easy items positively, and a low probability of answering the medium and hard items positively. Respondents with medium measures will have a high probability of answering the easy and medium items positively, and a low probability of answering the hard items positively. Respondents with high measures will have a high probability of answering the easy, medium and hard items positively. These conditions are tested through a Rasch analysis which provides a test of the structure of the variable. Data are collected from respondents on the items and scored dichotomously (0/1), as in, for example, but not limited to, wrong/right, no/yes, none/a lot, disagree/agree, some/often, bad/good, slow/fast, or the items can be scored with three or more responses as, for example, with none (0), some (1), most (2) and always (3). It is better to have an ordered response set and the RUMM computer program will test whether the response categories are being answered consistently and logically for each item. My students and I hope that the Rasch measurement papers in this book will help you in your desire to do some good educational research measurement that improves our knowledge in education for the benefit of young people.
Russell F. Waugh
xii Best wishes Russell Waugh January 2010
In: Specialized Rasch Measures… Editor: Russell F. Waugh, pp. xiii-xiv
ISBN: 978-1-61668-032-9 © 2010 Nova Science Publishers, Inc.
AUTHOR BIOGRAPHIES Minoti BISWAS is a specialist teacher, currently at Narrogin Senior High School in Western Australia. She holds the degrees of BA (Hons.), BEd, MEd, and PhD. Her PhD thesis (2007) was titled University Students‟ Receptivity to Peers with Disabilities. Minoti has extensive experience teaching in India and in Western Australian secondary education. She has a special interest in students with disabilities and her cross-cultural measure of disabilities (India/Western Australia) is important for policy development. CHOE Kee Cheng is a specialist senior teacher in Singapore. He holds the degrees of BA (Hons.), Dip. Ed., MA (English Studies), and EdD. His Doctor of Education thesis (2006) was titled Student Engagement with Project Work in a Junior College in Singapore. Project Work was an initiative of the Singapore Government where all secondary students were required to do Project Work in groups in order to improve their cooperative and creative abilities for the future benefit of their country. Ahdielah EDRIES is Principal of one of three campuses of an Australian Islamic College in Perth, Western Australia and a current Fulbright Scholar. She holds the degrees of BSc, Grad. Dip. of Ed., MEd, and EdD. Many of her students come from war torn countries like Somalia, Ethiopia and Lebanon, and some have not been to school before coming to Australia. Ahdielah works very hard to build up the Australian Islamic College so that its students have the same standard and the same opportunities as other Australian students. Her Doctor of Education thesis (2009) was titled Student and Teacher Identified Attitudes and Needs at an Australian Islamic College. Her research involved investigating the attitudes, interests and needs of the students (but not all of this is reported here). Only some Rasch measures about student self-views based on some Gardner Intelligences are reported in this book. LUI Shiueh Ling is a specialist educator in Singapore. She holds the degrees of Diploma of Teaching, BEd (Hons.), MEd and EdD. Her Doctor of Education thesis (2009) was titled A Caring Thinking Module in Mathematics: Its Impact on Social Attitudes and behavior in Students. Gambling addiction is a problem in many countries, including Singapore, and there is a need to include an understanding of statistics and responsibilities associated with gambling, which was done in the thesis, but not all reported here, in teaching some mathematics. Shiueh Ling has held various senior teaching and administrative positions in
xiv
Russell F. Waugh
Singapore and she recently administered the International Baccalaureate, amongst other things, at a premier high school there. Deslea KONZA is Associate Professor in the Faculty of Education and Arts at Edith Cowan University and a specialist in language education. She holds the degrees of BA, Dip.Ed., Dip. Special Ed., MEd, and PhD. Deslea has had wide experience teaching students of all ages with a range of special needs, including those associated with blindness, profound hearing impairment, intellectual disabilities, physical disabilities and multiple disabilities. She also has wide experience in language development and she has published widely in books and journal articles throughout her university career. Janet RICHMOND has extensive experience in occupational therapy, language and reading education in South Africa, Victoria and Western Australia. She holds the degrees of B. Occ. Therapy, M. Occ. Therapy (Hons), and PhD. Her PhD thesis (2009) was titled: Visual Discrimination of Alphabet Letters and Numbers with Young Children. She continues to help young children improve their reading and language skills mainly through her university work. Rahda Devi UNNITHAN is a specialist Mathematics teacher in Singapore where she has had extensive experience in Mathematics teaching. She holds the degrees of BSc., Post. Grad. Dip. of Ed., MEd and EdD. Her Doctor of Education thesis (2007) was titled Singapore Secondary School Students‟ Conceptions and Misconceptions of Algebraic Equation Solving. Russell F. WAUGH works at two universities in Perth, Western Australia. He is a Professor in the Faculty of Education and Arts at Edith Cowan University and a Senior Research Fellow in the Graduate School of Education at the University of Western Australia, and he supervises doctoral students at both universities. He holds the degrees of BSc, MSc, BEd, MEd, and PhD (UWA). Russell is a former Fulbright Scholar and specializes in Rasch measurement using the Rasch Unidimensional Measurement Models (RUMM) computer program developed by Professors David Andrich, Barry Sheridan and Guanzhong Luo, mainly applied to psychological and educational variables in the human sciences. Russell has published widely through journals and books, nearly all with Rasch measures. Russell can be contacted at r.waugh@ecu.edu.au WONG Heng Aik Jason is a specialist teacher and educator in Singapore. He has taken a special interest in the entrepreneurial mindset of young Singapore students so that they will excel in business later in life for the benefit of themselves and for the benefit of Singapore society. An emphasis on creativity and entrepreneurship in schools was a recent initiative of the Government of Singapore. Wong holds the degrees of BA., Dip. Ed., MEd and EdD. His Doctor of Education thesis (2009) was titled The Entrepreneurial Mindset of Students in Singapore.
In: Specialized Rasch Measures… Editor: Russell F. Waugh, pp. 1-21
ISBN: 978-1-61668-032-9 © 2010 Nova Science Publishers, Inc.
Chapter 1
A RASCH MEASURE OF STUDENT RECEPTIVITY TO PROJECT WORK AT A JUNIOR COLLEGE IN SINGAPORE Choe Kee Cheng1 and Russell Waugh2 1
Singapore Graduate School of Education University of Western Australia 2
ABSTRACT Group Project Work was introduced as a compulsory subject in Singapore to encourage team work, critical thinking and communication. Student Receptivity to Project Work was conceptualised as related to six key aspects (goal management, selfmanagement, learning styles, collaboration, knowledge application, and communication) and a questionnaire composed of 54 stem-items was based on these. Items were conceptualised from easy to hard (nine stem-items for each of the six aspects) and answered in two perspectives (an attitude self-view and a behaviour self-view) making 108 items. Data from 738 students in a junior college were analysed with a Rasch Unidimensional Measurement Model computer program (RUMM2020). Results showed that there was limited support for the model of the variable as 43 items had to be deleted. There was a good fit to the measurement model for the remaining group of 65 items (item-trait chi-square = 594, df=585, p=0.39) and reliability was good (Person Separation Index = 0.95). Items from all six aspects fitted the measurement model. The ideal perspective (this is what I aim for) was easier than the actual behaviour perspective (this is what I actually do), as was conceptualised for the structure of the variable but, for some items, only the ideal perspective, or only the behaviour perspective, fitted the measurement model. Some re-wording of the items, with further Rasch analysis, is needed for further understanding of the variable.
2
Choe Kee Cheng and Russell Waugh
INTRODUCTION It has become increasingly clear that in today‟s society where human capital is perceived to command premium value in driving the engines of the new knowledge-based economy (Goh, 1997), what is valued in individuals by states are the abilities to be able to think critically, and creatively, work independently, and be adaptable enough to apply one‟s knowledge and learning, innovatively and flexibly (Shanmugaratnam, 2003, 2005; Teo, 2000). In many countries, as such, changes were introduced in the education system to offer students more variety and choice of subjects to develop these qualities (Ball, 1998). Singapore, too, was not unaffected by such changes. The prevailing school of thought since the advent of the 21st century has been that the reorientation of educational emphasis on innovative handling of information technology, team-working, networking, and life-long learning, might lead to a lesser dependence on pen-and-paper type examinations, and to a greater reliance on project work assessment and/or other process skills assessments, which would, ideally, create the type of innovative, adaptable workers for which the Singapore economy was perceived to be in great need. Thus, methods of teaching, and assessment had to emphasise creative problem-solving, and critical pursuits in thinking. Such other generic skills that were deemed to be able to enhance the economic value of the individual included being able to communicate ideas and information, use mathematical ideas and techniques, work and collaborate with others in teams, utilise technology, plan and organise activities; collect, systematise and conduct data analysis, and think critically and creatively to solve problems (Kennedy, 1999). It was in this light that, as conceptualised by the Singapore Ministry of Education, Project Work was introduced as a compulsory academic subject that would infuse these much-desired skills and capabilities in students in Singapore.
WHAT IS PROJECT WORK? Project Work is a curricular programme designed to provide students with opportunities to explore the interconnectedness of subject specific knowledge (Ministry of Education, Singapore, 2004). Above and beyond the fact that the grade obtained for the subject counts for ten per cent of local university admission requirements from 2003 onwards (Nirmala, 1999), it aims to enable students to firstly, apply creative and critical thinking skills; secondly, improve their communication skills (both oral and written); thirdly, foster collaborative learning; and fourthly, develop self-directed inquiry, and life-long learning skills (Ministry of Education, n.d.). Based on the Singapore Ministry of Education‟s guidelines, there are five key features of Project Work: interdisciplinary; involves collaborative learning; requires an oral presentation; focuses on both the process and product; and builds in Just-In-Time Skills instruction (Ministry of Education, 2002). As part of the implementation guidelines, the Singapore Ministry of Education also stipulates that Project Work differs from subject-based projects in that students gather and process information from various sources, and apply, and integrate knowledge, and skills, from different subjects to create new knowledge. Not only that, it has to be carried out during curriculum time so that teachers can work with students more closely. A typical project task, as such, requires students to identify an aspect of society and to evaluate how this particular
A Rasch Measure of Student Receptivity to Project Work at a Junior College…
3
feature can change for the betterment or detriment of the said society. Students have then to come up with measures to enhance or counteract the development of the characteristic of society (Ministry of Education, 2004, 2005). Typically, two project tasks are given to students from which they must select one to do a project on. Each of the tasks revolve around different themes and each task, in turn, is further broken down into a number of bullet points which explicitly instruct students on the type of action they should engage in for different tasks. Changes introduced by the Education Ministry to streamline the structure of Project Work in 2005, and which informs the way Project Work is currently implemented and assessed, have retained the form such project tasks assume.
LITERATURE REVIEW Recent research on the effectiveness of doing project work from students‟ perspectives have reported mixed findings. Several studies have found that students, on the whole, perceive Project Work positively (Chang & Chang, 2004; Chua, 2004; Hays & Vincent, 2004; Lee, 2001; Mueller & Fleming, 2001; Payne, Monk-Turner, Smith & Sumter, 2006; T.L.S. Tan, 2002; Wong, 2001). Such findings can result from the fact that students saw doing project work as a way of acquiring authentic, marketable life skills, such as working with others in a group, data analysis, researching, action planning, and organising, which would be beneficial to their working life. To students, therefore, doing projects was identical to gaining (future) work experience (Bourner, Hughes & Bourner, 2001). Other merits of doing project work included students perceiving a project work environment to be motivating, particularly for groups that could cooperate (Willis et al., 2002), and which provided the opportunity for them to collaborate in teams and think and learn independently (Kucharski, Rust & Ring, 2005; Payne & Monk-Turner, 2006; G.C. I. Tan, 2004). Similarly, students in Singapore, like the students from other parts of the world, perceive Project Work positively for the perceived benefits to be gained from doing the subject: an autonomous learning environment which allows students to explore their own interests (Chin & Kayalvizhi, 2005), have fun (G.C.I. Tan, 2004), collaborative learning opportunities leading to increased peer interaction and friendship (Chin & Kayalvizhi, 2005; Quek & Wong, 2001; G.C.I. Tan, 2004), valuable learning experience (Chin & Kayalvizhi, 2005; O.S. Tan, 2004), and a deeper understanding of the topics under investigation (Chin & Kayalvizhi, 2005; G.C.I. Tan, 2004). However, the body of literature that has reported unfavourable findings have also begun to increase (Cantwell & Andrews, 2002; Phipps, Phipps, Kask, & Higgins, 2001; Zanolli, Boshuizen & De Grave, 2002). These studies challenge the view that students accept, or agree, that the Project Work approach to learning is one that they favoured, and highlight instead the weaknesses of the Project Work instructional and learning approach, especially with regard to issues pertaining to the internal group dynamics of working in a studentoriented learning environment. For example, McPhee (2002) reported that some students in her study felt the demerits of a group project or problem-based learning method keenly, which included group conformity, dysfunctional groups, too much time spent on fruitless discussion, social-loafing, the lack of understanding of content as a result of students not receiving notes and intimidating experience of presenting to peers. Senior students‟
4
Choe Kee Cheng and Russell Waugh
perceptions of the weaknesses of such a learning method revolve around main issues of group discussions being dominated by individuals, unproductive discussion sessions where no real learning occurs, and an absence of teacher feedback. Nijhuis, Segers and Gijselaers (2005) have also reported that such similar views were echoed by the students in their study who acquired a negative view of time efficacy when they were engaged in a project or problembased learning model. More strikingly, in another study involving 109 students, which used a traditional pre-test and post-test design and occurred over a period of eight weeks, Elen and Clarebout (2001) argued that students did not perceive that more effort put into problembased learning would gain benefits for them. Not only that, their beliefs in the power of technology and problem-based collaboration as aspects of a positive learning environment diminished after the study. The conclusions of these findings seem obvious: students do not have a positive perception of Project Work. In Singapore, while the literature on students‟ perspectives to Project Work has, for the most part, reported positive evaluations, there also appears to be a clear and present theme of student dissatisfaction that has emerged in a number of studies. I.G.C. Tan‟s (2004) study (N=241), for instance, reported that notwithstanding the 652 positive statements that students made of Project Work, another 303 statements related to student preference for the conventional transmission type of teaching, the extent to which doing Project Work was timeconsuming, admission of an inability to learn much vis-à-vis Project Work, and group conflicts. Other researchers‟ studies, such as O.C. Tan (2004), and Chin and Kayalvizhi (2005), made similar findings. While the majority of the 100 undergraduate-participants in O.C. Tan‟s (2004) study had positive learning experiences with the problem-based learning model of Project Work, some participants also acknowledged that: firstly, problem-based learning conflicted with their approach to learning, and learning styles; secondly, there was a lack of overview and direction; thirdly, they lacked mental and emotional readiness for problem-based learning, and felt overwhelmed by numerous issues and self-directed learning; and finally, they were frustrated by a lack of time. Likewise, the 39 Primary Six students in Chin and Kayalvizhi‟s (2005) study reported that the difficulty of coming up with good research designs, and group conflicts, comprised their negative experiences of problem-based learning in Science. These findings offer substantial evidence to suggest that two issues that may commonly plague Project Work is the lack of time and group conflicts. Yet other researchers in Singapore have reported additional workload, and increased stress as weaknesses of Project Work. According to Chang & Chang (2004), students felt that their workload was massive as they had to manage five main academic subjects along with co-curricular activities, community involvement programmes, and enrichment programmes, and now, Project Work – the nature of which required students to work on their projects beyond the time spent on discussions of their ideas for their projects within the classroom during curriculum time. In effect, Project Work vied for the limited amount of time students have to apportion for all their other commitments that, for them, were equally important as Project Work, if not more. Chin and Chia (2004) have argued that one source of stress stemmed from students‟ inability to ask the right questions when working in their project work groups and to have them addressed. These researchers warned that if “students feel they are not making the desired progress, they may become frustrated and unmotivated in pursuing their learning tasks” (Chin & Chia, 2004, p. 724). In summary, although several researchers have asserted that a group-based project or problem-based learning model is one to which students would be receptive, there has also
A Rasch Measure of Student Receptivity to Project Work at a Junior College…
5
been dissenting voices that argue for the contrary. This study was thus an attempt which sought to contribute to the literature on doing project work in relation, specifically, to students in a junior college in Singapore, using a Rasch measurement model as the research base.
AIMS OF STUDY There were two main objectives to this study. Firstly, the study intended to create a questionnaire on Student Receptivity to Project Work based on the following six aspects of Project Work: Goal Management, Self-Management, Learning Styles, Collaboration, Knowledge Application, and Communication. Secondly, the study aim to construct a linear scale of Student Receptivity to Project Work using the RUMM2020 Rasch computer program and thus understand the structure of the variable and its meaning.
METHOD The questionnaire was trialled and revised before being administered to the first year cohort of 913 Science students who took Project Work as a subject in a major junior college in Singapore in 2004. The students were studying for their GCE „A‟ levels before going to university including Physics, Chemistry and Mathematics.
Sample These students were all first year junior college Science students who encountered Project Work for the first time as a subject that was to be compulsorily included in their future application to university entry in Singapore. It was decided that there should be a spread of participants across all the Science classes so that representative student views and perspectives on Project Work from the cohort of first year Science students could be garnered through the writing of diaries, as well as through individual interviews (not reported here).
Administrative and Ethics Approval Prior to the administering of the questionnaire, permission had been obtained from the Principal, as well as the Head of Department for Project Work, for the questionnaire to be administered to the students. Initial approval to conduct the survey was obtained from The University of Western Australia's Ethics Committee. Following this, permission was obtained from the Cluster Superintendent who oversaw the school cluster to which this college belonged, the Principal of the college, as well as the Head of Department for Project Work, to conduct research in the college. Written informed consent was then sought from the participants themselves, and their parents, for their voluntary participation.
6
Choe Kee Cheng and Russell Waugh
Administering of Questionnaire During the administering of the questionnaire, students read the cover letter which informed them of their right to withdraw from participating in the questionnaire, and signed their consent to participate in this research study, based on the condition that their identities would remain anonymous. The cover and informed consent sheet from the questionnaire were collected before the students began doing the questionnaire proper. Completed questionnaires were subsequently placed into an envelope by the students. Altogether, out of a cohort of first year 913 students, 849 attended on the day, and 738 useable questionnaires, in which all questions were answered, were retained for the Rasch analysis
DESIGN OF THE QUESTIONNAIRE Within each aspect, items were conceptually designed from easy to hard. For example, under Goal Management (Expectations), stem-item 1 was set expectations that I want to achieve in Project Work and this was expected to be relatively easy. Stem-item 2 was do my best to attain the expectations in Project Work that I set for myself and this should be harder because it involves setting attainments compared to expectations. Stem-item 3 was evaluate my performance against the expectations in Project Work that I set for myself and this should be harder still because it involves evaluating which itself is harder and involves more effort than stem-item 2. In a similar way, the items for the other aspects were designed conceptually from easy to hard but they are not described here to avoid repetition. A reader can easily work out the conceptual design by looking at Table 1. All the items were written in a positive sense and answered in two perspectives so it would be evident to students what was being measured, and what they were expected to answer. The two perspectives were what I aim for (to measure what students ideally would like to do and was expected to be easy), and What I actually do (to measure what students do in practice which was expected to be harder). The behaviour was expected to be harder than the attitude because to actually do „something‟ requires effort and includes the attitude to actually do that „something‟. Students were required to answer the items in an ordered response format: none of the time (score 1), some of the time (score 2), most of the time (score 3) and all of the time (score 4), in line with good measurement practice. The conceptual ordering of the item difficulties and the conceptual ordering of the response categories can now be tested by collecting appropriate data and analysing it with a Rasch measurement computer program.
DATA ANALYSIS WITH THE RUMM 2020 PROGRAM The RUMM 2020 program produces tables and graphical output that enables one to check the fit of the data to the measurement model using the Partial Credit Model of Rasch. Firstly, item thresholds and Response Category Curves were inspected so that only those items with ordered thresholds, which indicated that the response categories for the items were answered consistently, and logically, were included in the final linear scale. Then, the itemtrait test-of-fit chi-square was examined to check on the consistency of agreement of the item
A Rasch Measure of Student Receptivity to Project Work at a Junior College…
7
parameters (item difficulties) for all student measures along the scale. After this, individual item fits to the measurement model were examined and those items that did not fit were deleted. Forty-three non-performing items out of 108, as established through the steps above, were subsequently removed from the scale, leaving 65 items whose data fitted the Rasch measurement model and could be used to create a linear scale in which the measures and item difficulties were calibrated on the same scale, including the attitude and behaviour item difficulties.
RESULTS The item-trait interaction chi-square with 65 items was 594.2, df=585, p=0.39. This means that there was very good agreement amongst all 738 students, right along the scale, about the difficulties of the 65 items and, hence there was a good overall fit to the measurement model. The Student Separation Index was 0.95 and high. This means that the measures were well-separated in comparison to the measurement errors, as required for good measurement. Based on the Separation Index, the power of the tests-of-fit for this scale was rated as „excellent‟. Of the 65 items that fitted the measurement model, 25 evaluated an ideal aspect (What I aim for), and 40 evaluated an actual aspect (What I actually do). This means that the actual aspect made a stronger contribution to the variable (Student Receptivity to Project Work) than the ideal aspect. Eighteen of the 65 items had ideal and actual aspects that matched each other, and which fitted the measurement model. That is, for nine stem-items, there were nine ideal and nine corresponding actual aspects for those items. Seven ideal and 22 actual aspects of the items made up the remaining 65 items. For Goal Management 15 out 18 items fitted: For Self-Management 8 out of 18 items fitted: For Learning Styles nine out of 18 items fitted: For Collaboration nine out of 18 items fitted; For Knowledge Application 14 out of 18 fitted: For Communication ten out of 18 items fitted. This means that there was only limited support for the conceptualised structure. The item difficulties are given in Table 1. Table 1. Student Receptivity to Project Work (N=738) Item No.
Item wording
GOAL MANAGEMENT Expectations 1-2 Set expectations that I want to achieve in PW. 3-4 Do my best to attain the expectations in PW that I set for myself. 5-6 Evaluate my performance against the expectations in PW that I set for myself Interest 7-8 Read widely to learn more about and find out what is relevant for my topic in PW out of curiosity. 9-10 Display enthusiasm for and commitment to PW. 11-12 Solve problems with which others have difficulty because I am keenly interested. Goal Setting 13-14 Set myself realistic goals in PW.
What I aim for
What I actually do
-0.12 -0.64
0.98 0.40
0.51
No fit
No fit
No fit
0.11 0.38
0.95 1.11
-0.50
0.27
Choe Kee Cheng and Russell Waugh
8
Table 1. (Continued). Item No.
Push myself to ensure that I achieve my goals in PW. 17-18 Challenge myself to overcome difficulties to reach my goals in PW. SELF-MANAGEMENT Personal motivation 19-20 Seek to obtain as high a band as possible for PW. 21-22 Like the interaction with peers and social relationships in solving problems for PW. 23-24 Enjoy the intellectual challenge and social learning involved with PW. Time management 25-26 Adhere as closely as possible to my group‟s timeline of work. 27-28 Organise my time to make the optimum use of it for PW. 29-30 Work steadily and consistently for PW, rather than leave everything until the last minute to be done. Tasks 31-32 Decide what tasks should be done at what time for my project.
-0.25
What I actually do 0.59
-0.11
0.83
-0.87 No fit
-0.13 No fit
No fit
No fit
-0.77
No fit
No fit
1.14
-0.26
No fit
-0.43
0.67
33-34
No fit
0.46
No fit
No fit
No fit
No fit
-0.08
No fit
No fit
0.60
No fit
-0.74
No fit
0.18
No fit
No fit
-1.04
-0.04
-1.00
-0.19
No fit
0.18
Item wording
15-16
35-36
Make an effort to achieve these tasks by a set deadline
Challenge myself to use different strategies to accomplish these tasks even when I have difficulties LEARNING STYLES Independent learning 37-38 Rely upon others to guide me on what I need to do for PW. 39-40 Work out for myself exactly what is needed to be done, not just accept what I am told to do for PW. 41-42 Make every effort to seek out necessary information, resources and ways that best enable me to achieve for PW. Learning from others 43-44 Listen to others during group discussions to learn about different points of view in relation to PW. 45-46 Learn actively and deliberately from others who have more knowledge and experience than I have. 47-48 Participate in group discussions to share views and generate new ideas to improve the quality of my project. Learning from the Supervising Tutor (ST) 49-50 Seek and listen to the ST's suggestions on my project. 51-52 Reflect and act upon the ST's suggestions where appropriate for my project. 53-54 Modify or integrate the ST's suggestions with other ideas as best as I can to create new approaches for my project.
What I aim for
A Rasch Measure of Student Receptivity to Project Work at a Junior College… Item No.
Item wording
COLLABORATION Discussion behaviours Spend adequate time discussing how 55-56 improvements to the project can be made. Discuss and implement different strategies to 57-58 overcome difficulties encountered in the project Evaluate the effectiveness of strategies taken to 59-60 improve group progress. Working in teams Pay attention and listen to others during PW 61-62 discussions. Respect others‟ views and show tact in my 63-64 responses as much as I can during PW discussions. Involve others actively, appropriately and 65-66 consistently to the best of my ability during PW discussions Helping each other Volunteer willingly for tasks and responsibilities 67-68 for PW to help others fulfil tasks and meet deadlines. Encourage, motivate and support others to stay 69-70 focused and on-task for PW. Involve others actively to solve difficulties in PW 71-72 effectively as a cohesive group. KNOWLEDGE APPLICATION Initiating research 73-74 Decide on the proper focus of my project. 75-76 Brainstorm and plan how my project can best be completed for PW. 77-78 Seek, summarise and categorise information from a wide range of sources for my project. Managing research 79-80 Examine and analyse information for its usefulness to my project. 81-82 Evaluate the accuracy and credibility of information from different sources. 83-84 Synthesise and relate different information pertinently, cogently and meaningfully to my project discussions, written reports and presentations Applying research 85-86 Recognise the value of the knowledge and skills that I have gained for having done PW. 87-88 Relate the knowledge and skills I gained from PW to other subjects, topics or courses, whenever possible 89-90 Use the knowledge and skills consciously and actively in dealing with other people and situations in life COMMUNICATION Content of presentation 91-92 Link my presentation to my project explicitly. 93-94 Organise my presentation in a coherent fashion. 95-96 Make my presentation captivating and "listenerfriendly".
What I aim for
What I actually do
No fit
0.44
No fit
0.60
0.11
No fit
No fit
-0.88
No fit
No fit
No fit
0.00
-0.47
0.05
No fit
0.34
No fit
0.51
-0.77 No fit
0.15 0.31
-0.72
0.33
No fit
-0.24
-0.95
-0.12
No fit
0.06
0.15
1.06
0.49
No fit
0.46
1.33
No fit No fit No fit
-0.38 -0.70 -0.24
9
Choe Kee Cheng and Russell Waugh
10
Table 1. (Continued). Item No.
What I aim for
What I actually do
Presentation matters 97-98 Project my voice and make some eye contact with
No fit
-0.69
99-100
Articulate confidently and fluently on the project.
No fit
-0.45
Express myself with poise, and engage the audience with my personality, appropriate eye contact, tone and relevance of content
No fit
-0.29
Addressing questions 103-104 Provide a relevant answer and elaborate on it for PW.
No fit
0.33
105-106
Offer an insightful response supported with pertinent details for PW.
-1.04
0.59
107-108
Engage the audience with wit, appropriate pacing, and clarity of my response.
-1.20
No fit
Item wording
the audience on PW.
101-102
Table 2. Global Fit Statistics for the Student Receptivity to Project Work Scale Items
Students
Number
65
738
Location mean
0.00
0.86
Standard deviation
0.62
0.81
Fit Statistical mean
0.10
-0.50
Standard deviation
0.99
2.45
Notes on Table 2 1. The item means are constrained to zero by the measurement model. 2. When the data fit the measurement model, the fit statistic approximates a distribution with a mean near zero, and a SD near one (a good fit for the items of this scale, and a not-so-good, but acceptable fit for the student measures).
TARGETING OF THE SCALE Figure 1 shows the item thresholds ordered from easy to hard (at the bottom) and the student measures ordered from low to high (on the top). The thresholds „cover‟ the range of student measures and are thus well targeted at the measures, but some harder items are needed to be added to the scale to „cover‟ the very high student measures.
A Rasch Measure of Student Receptivity to Project Work at a Junior College…
11
Notes 1. The scale is in logits, that is, the log odds of answering the response categories. 2. Student measures (low to high) are placed on the upper side of the scale, and thresholds (easy to hard) are placed on the lower side of the scale. Figure 1. Student Receptivity Measures and Item Thresholds on the same scale.
Figure 2. Student Receptivity by Gender. Note: The RUMM program has the colours wrong: Females are maroon (not red), males are green (not blue).
Choe Kee Cheng and Russell Waugh
12
USING THE RESPONSE CATEGORIES LOGICALLY Testing whether the response categories are used logically and consistently is tested by two methods with the RUMM2020 program: (1) threshold ordering and (2) response category curves. Thresholds are points between adjacent categories where the odds are 1:1 of answering in either category. These thresholds should be ordered in line with the ordering of the response categories, if the students use them consistently and logically. Table 3 (below) illustrates that the thresholds for the Goal Management items are ordered in line with the conceptual ordering of the response categories. Threshold data from the other aspects confirmed that the response categories were answered consistently and logically, but are not reported her to avoid repetition. Figure 3 shows a response category curve for item 1 where the probability of answering each of the response categories is given by student measure. For a low measure, there should be a high probability of answering the lowest response category and, as the student measures increase, the probability of answering this low category should decrease and the probability of answering the next category should increase. The process should continue until the highest measures where the probability of answering the highest category should be high and the probability of answering the other categories low. Table 3. Item Thresholds Values for the Goal Management Aspect Item Number
Mean location (Difficulty)
THRESHOLDS 1
2
3
Item 1
-0.12
-1.55
-0.63
1.82
Item 2
0.98
-1.55
1.04
3.46
Item 3
-0.64
-1.86
-1.04
0.97
Item 4
0.40
-2.04
0.35
2.88
Item 5
0.51
-0.65
0.10
2.06
Item 9
0.11
-0.94
-0.32
1.58
Item 10
0.95
-1.07
0.91
3.00
Item 11
0.38
-1.19
0.25
2.06
Item 12
1.11
-1.04
1.33
3.03
Item 13
-0.50
-1.89
-0.81
1.21
Item 14
0.27
-1.57
0.17
2.21
Item 15
-0.25
-1.54
-0.67
1.46
Item 16
0.59
-1.78
0.87
2.68
Item 17
-0.11
-1.75
-0.30
1.72
Item 18
0.83
-1.50
0.91
3.10
Notes on Table 3 1. Mean location (item difficulty) is the mean threshold location. 2. At a threshold, the odds are 1:1 of answering adjacent response categories. In this case, all the thresholds are logically ordered from easy to hard, in line with the ordering of the response categories.
A Rasch Measure of Student Receptivity to Project Work at a Junior College…
13
Figure 3. Item Category Curve for Item 100 (good-fitting item). Note: Threshold 1 is about -2.2 logits; Threshold 2 is about -0.8 logits; and Threshold 3 is about +1.8 logits.
ITEM CHARACTERISTIC CURVE BY GENDER The RUMM program provides characteristic curves for each item and Figure 4 shows such a curve for item 1 by gender. The expected values follow the ogive curve and so the students have answered this item as expected for a good fit to the measurement model and there is no significantly differential item response by gender (chi-square= 2.22, df=1,9, p= 0.14).
Figure 4. Item Characteristic Curve By Gender (Item 1).
14
Choe Kee Cheng and Russell Waugh
DISCUSSION Meaning of the Scale The 65 items that fitted the measurement model define the variable Student Receptivity to Project Work. The 65 items involve ideal attitudes and behaviour items from six aspects, namely: Goal Management, Self-Management, Learning Styles, Collaboration, Knowledge Application, and Communication. Some items did not have both the attitude and behaviour perspectives fitting the measurement model (only 18 items did so) and from this it might be inferred that some attitudes did not directly influence a corresponding behaviour, contrary to a generally accepted theory (such as proposed by Ajzen, 1989). Where both attitude and behaviour items fitted the measurement model, attitude was easier than behaviour and, in these cases, it can be inferred that attitude did directly affect the corresponding behaviour.
Goal Management: Expectations, Interest, Goal Setting Fifteen out of 18 items for this aspect fitted the measurement model. The five easiest attitude items were (in descending order of easiness): 1. Do my best to attain the expectations in Project Work that I set for myself (ideal expectations); 2. Set myself realistic goals in Project Work (ideal goal-setting); 3. Push myself to ensure that I achieve my goals in Project Work (ideal goal-setting); 4. Set expectations that I want to achieve in Project Work (ideal expectations); 5. Challenge myself to overcome difficulties to reach my goals in Project Work (ideal goal-setting). The five hardest behaviour items for students were (in descending order of difficulty): 1. Solve problems with which others have difficulty because I am keenly interested (actual interest); 2. Set expectations that I want to achieve in Project Work (actual expectation); 3. Display enthusiasm for and commitment to Project Work (actual interest); 4. Challenge myself to overcome difficulties to reach my goals in Project Work (actual goal-setting); and 5. Push myself to ensure that I achieve my goals in Project Work (actual goal-setting).
Self-Management: Personal Motivation, Time Management, Tasks Eight out of 18 items for this aspect fitted the measurement model. The five easiest items are (in descending order of easiness): 1.Seek to obtain as high a band as possible for Project Work (ideal personal motivation); 2.Adhere as closely as possible to my group‟s timeline of work (ideal time management); 3. Decide what tasks should be done at what time for my project (ideal tasks); 4. Work steadily and consistently for Project Work, rather than leave everything until the last minute to be done (ideal time management); and 5.Seek to obtain as high a band as possible for Project Work (actual personal motivation). There are three hard behaviour items (in descending order of difficulty): 1.Organise my time to make the optimum use of it for Project Work (actual tasks); 2. Decide what tasks should be done at what time for my project (actual tasks); and 3. Make an effort to achieve these tasks by a set deadline (actual time management).
A Rasch Measure of Student Receptivity to Project Work at a Junior College…
15
Learning Styles: Independent Learning, Learning from Others, and Learning from the Supervising Tutor Nine out of 18 items for this aspect fitted the measurement model. The five easiest items are (in descending order of easiness): 1. Seek and listen to the supervising tutor‟s suggestions on my project (ideal, learning from the supervising tutor); 2. Reflect and act upon the supervising tutor‟s suggestions where appropriate for my project (ideal, learning from the supervising tutor‟s); 3. Listen to others during group discussions to learn about different points of view in relation to Project Work (actual, learning from others); 4. Reflect and act upon the supervising tutor‟s suggestions where appropriate for my project (actual, learning from the supervising tutor); and 5. Work out for myself exactly what is needed to be done, not just accept what I am told to do for Project Work (ideal, independent learning). There are only three hard items (in descending order of difficulty): 1. Make every effort to seek out necessary information, resources and ways that best enable me to achieve for Project Work (actual, independent learning); 2. Modify or integrate the supervising tutor‟s suggestions with other ideas as best as I can to create new approaches for my project (actual, learning from the supervising tutor‟s); and 3. Learn actively and deliberately from others who have more knowledge and experience than I have (actual, learning from others).
Collaboration: Discussion Behaviours, Working in Teams, and Helping Each Other Nine out of 18 items for this aspect fitted the measurement model. There were two easy items (in descending order of easiness): 1. Pay attention and listen to others during Project Work discussions (actual, working in teams); and 2. Volunteer willingly for tasks and responsibilities for Project Work to help others fulfil tasks and meet deadlines (ideal, helping each other). The five hardest items (in descending order of difficulty) were: 1. Discuss and implement different strategies to overcome difficulties encountered in the project (actual, discussion behaviours); 2. Involve others actively to solve difficulties in Project Work effectively as a cohesive group (actual, helping each other); 3. Spend adequate time discussing how improvements to the project can be made (actual, discussion behaviours); 4. Encourage, motivate and support others to stay focused and on-task for Project Work (actual, helping each other); and 5. Evaluate the effectiveness of strategies taken to improve group progress (ideal, discussion behaviours).
Knowledge Application: Initiating Research, Managing Research, and Applying Research Fourteen out of 18 items for this aspect fitted the measurement model. The five easiest items are (in descending order of easiness): 1. Evaluate the accuracy and credibility of information from different sources (ideal, managing research); 2. Decide on the proper focus of my project (ideal, initiating research); 3. Seek, summarise and categorise information from
16
Choe Kee Cheng and Russell Waugh
a wide range of sources for my project (ideal, initiating research); 4. Examine and analyse information for its usefulness to my project (actual, managing research); and 5. Evaluate the accuracy and credibility of information from different sources (actual, managing research). The five hardest items (in descending order of difficulty) are: 1. Use the knowledge and skills consciously and actively in dealing with other people and situations in life (actual, applying research); 2. Recognise the value of the knowledge and skills that I have gained for having done Project Work (actual, applying research); 3. Relate the knowledge and skills I gained from Project Work to other subjects, topics or courses, whenever possible (ideal, applying research); 4. Use the knowledge and skills consciously and actively in dealing with other people and situations in life (ideal, applying research); and 5. Seek, summarise and categorise information from a wide range of sources for my project (actual, initiating research).
Communication: Content of Presentation, Presentation Matters, and Addressing Questions Ten out of 18 items for this aspect fitted the measurement model. The five easiest items are (in descending order of easiness): 1. Engage the audience with wit, appropriate pacing, and clarity of my response (ideal, addressing questions); 2. Offer an insightful response supported with pertinent details for PW (ideal, addressing questions); 3. Organise my presentation in a coherent fashion (actual, content of presentation); 4. Project my voice and make some eye contact with the audience on Project Work (actual, presentation matters); and 5. Articulate confidently and fluently on the project (actual, presentation matters). There are two very hard items (in descending order of difficulty): 1. Offer an insightful response supported with pertinent details for Project Work (actual, addressing questions); 2. Provide a relevant answer and elaborate on it for Project Work (actual, addressing questions).
WHY DON’T 43 ITEMS FIT THE MEASUREMENT MODEL? The RUMM program does not explain why 43 items did not fit the measurement model: it just provides statistics to show that 43 items did not fit and these items were discarded. Three items (65, 99 and 101) were deleted because students did not use the response categories consistently and it is difficult to understand why. Maybe the items were too complicated and need to be simplified but they were made complex because some hard items were needed for good targeting. Most items did not fit the measurement model because students could not agree on their difficulty. What seemed to be happening was that a particular group such as those with medium measures might be split with say 50% agreeing that it was a medium difficulty item and the other 50% saying that it was a hard item (or sometimes an easy item). This seemed to occur because of difficulties in particular Project Work groups as not all of them ran smoothly. Sometimes, students with different measures (either low or high) could not agree on the difficulty of particular items. For example, stem-item 3 in the actual perspective did not fit because, apparently, some students with high measures did not evaluate their
A Rasch Measure of Student Receptivity to Project Work at a Junior College…
17
performance in their Project Work group (and thus found this item hard) and some with high measures did evaluate their performance and found that it was only of medium difficulty. A different example is stem-item 11 where there was such variability in the cohesion within the Project Work groups (some students considered that they did most of the work and some others did very little, some were friendly and others were not) that friction occurred making problems with relationships. Where this occurred, there were clearly problems for students agreeing with the difficulties of the items. When one looks through the items that did not fit the model and conducts interviews with the students afterwards, it is possible in many cases to see why the disagreements about difficulties occurred.
CONCLUSION This paper has described the creation of a reliable, unidimensional scale of Student Receptivity to Project Work based on six aspects and 65 items, using the Partial Credit Model of Rasch and the RUMM 2020 computer program. There was good individual-item fit, good global-item fit, good Separation Index, and good targeting of the items of this scale. Of the 65 items that fitted, and comprised the linear scale, there are 25 self-reported ideal (attitude), and 40 self-reported actual (behaviour), items. The results show the structure of the variable and that it involves both attitude and behaviour for six aspects: (1) goal management, (2) selfmanagement, (3) learning styles, (4) collaboration, (5) knowledge application, (6) and communication. Of the six aspects, the two easiest aspects are Communication, and Learning Styles, while the two hardest are Goal Management, and Knowledge Application. Females have significantly higher measures than males on the scale Receptivity to Project Work (t=6.90, df=736, p=0.0000). The findings above clearly point to a need for more understanding of the various ways in which Project Work has impacted on students, and on their perspectives of doing Project Work.
REFERENCES Ball, S. (1998). Big policies/small world: An introduction to international perspectives in education policy. Comparative Education, 34 (2), 119-130. Bourner, J., Hughes, M., & Bourner, T. (2001). First-year undergraduate experiences of group project work. Assessment and Evaluation in Higher Education, 26 (1), 19-39. Chang, T.T., & Chang, A. (2004). Assessing Project Work: Teachers and students‟ perspectives. In A. Khoo, M.A.Heng, L. Lim, & R.P. Ang (Eds.), Innovation and Diversity in Education (pp. 64-79). Singapore: McGraw-Hill. Chin, C., & Chia, L.G. (2004). Problem-based learning: using students‟ questions to drive knowledge construction. Science Education, 88, 707-727. Chin, C., & Kayalvizhi, G. (2005). What do pupils think of open science investigations? A study of Singaporean primary 6 pupils. Educational Research, 47 (1), 107-126. Chua, J.J. (2004). Evaluating the effects of Project Work in learning in a primary school. Unpublished master‟s thesis, National Institute of Education, Nanyang Technological University, Singapore.
18
Choe Kee Cheng and Russell Waugh
Goh C.T. (1997). Shaping our future: Thinking schools, learning nation. Speech by Prime Minister Goh Chok Tong at the Opening of the 7th International Conference on Thinking at the Suntec City Convention Centre Ballroom. Retrieved Mar 1, 2006, from http://www.moe.gov.sg/speeches/1997/020697_print.htm Hays, J.R., & Vincent, J.P. (2004). Students‟ evaluation of problem-based learning in graduate Psychology course. Teaching of Psychology, 31 (2), 124-126. Lee, L.C. (2001). Evaluating critical thinking pedagogy to support primary school Project Work using an action research approach. Unpublished master‟s thesis, National Institute of Education, Nanyang Technological University, Singapore. Kennedy, K.J. (1999). Constructing the School Curriculum for the Global Society. Innovating schools, rapporteur. Donald Hirsch. Paris: Organisation for Economic Co-operation and Development. Ministry of Education, Singapore (n.d.) Retrieved Jan 1, 2006, from http://www.moe.gov.sg/projectwork/ Ministry of Education, Singapore. (2002). Project Work Student Handbook. Singapore: Ministry of Education. Mueller, A., & Fleming, T. (2001). Cooperative learning: Listening to how children work at school. The Journal of Educational Research, 94 (5), 259-265. Nirmala, M. (1999, July 14). New criteria for university entry in 2003. The Straits Times, p.1. Payne, B.K., Monk-Turner, E., Smith, D., & Sumter, M. (2006). Improving group work: Voices of students. Education, 126 (3), 441-448. Shanmugaratnam, T. (2003). The next phase in education: Innovation and enterprise. Speech by Mr Tharman Shanmugaratnam, Acting Minister for Education, at the Ministry of Education Work Plan Seminar 2003 at Ngee Ann Polytechnic. Retrieved Mar 11, 2006, from http://www.moe.gov.sg/speeches/2003/sp20031002_print.htm Shanmugaratnam, T. (2005). Achieving quality: Bottom up initiative, top down support. Speech by Mr Tharman Shanmugaratnam, Minister for Education, at the Ministry of Education Work Plan Seminar 2005 at Ngee Ann Polytechnic Convention Centre. Retrieved Mar 11, 2006, from http://www.moe.gov.sg/speeches/2005/ sp20050922_ print.htm Teo C.H. (2000). Ability-driven education – Putting the system in place. Speech by RAdm Teo Chee Hean, Minister for Education and Second Minister for Defence at the Work Plan Seminar 2000 at Nanyang Polytechnic Auditorium. Retrieved Mar 11, 2006, from http://www.moe.gov.sg/speeches/2000/sp23092000_print.htm Tan, T.L.S. (2002). Using Project Work as a motivating factor in lower secondary Mathematics. Unpublished master‟s thesis, National Institute of Education, Nanyang Technological University, Singapore. Wong, H.M. (2001). Project work in primary schools. Unpublished master‟s thesis, National Institute of Education, Nanyang Technological University, Singapore.
A Rasch Measure of Student Receptivity to Project Work at a Junior College…
19
APPENDIX A Goal Management Item Location Goal Management Item 1 -0.12 Item 2 0.98 Item 3 -0.64 Item 4 0.40 Item 5 0.51 Item 9 0.11 Item 10 0.95 Item 11 0.38 Item 12 1.11 Item 13 -0.50 Item 14 0.27 Item 15 -0.25 Item 16 0.59 Item 17 -0.11 Item 18 0.83
SE
Residual
df
ChiSq
Prob
0.06 0.06 0.06 0.06 0.05 0.05 0.05 0.05 0.06 0.06 0.05 0.06 0.06 0.05 0.06
-0.48 -0.77 -1.06 -1.81 1.13 0.68 -1.10 0.04 0.08 0.16 0.50 -0.47 -1.19 -0.61 -1.10
723.66 723.66 723.66 723.66 723.66 723.66 723.66 723.66 723.66 723.66 723.66 723.66 723.66 723.66 723.66
3.81 9.69 6.77 21.97 9.05 5.60 11.83 3.95 14.03 2.92 5.01 12.98 15.38 9.53 16.95
0.92 0.38 0.66 0.01 0.43 0.78 0.22 0.91 0.12 0.97 0.83 0.16 0.08 0.39 0.04
Notes: 1. Location refers to the difficulty of the item on the linear scale. 2. SE refers to standard error, that is, the degree of the uncertainty in a value; in this case, the standard error for each of the items is low (0.05 to 0.06 logits). 3. Residual represents the difference between the expected value on an item, calculated according to the Rasch measurement model, and its actual value. 4. df (degrees of freedom) refers to the number of scores in a distribution that are free to change without changing the mean of the distribution. 5. ChiSq means chi-square. 6. Prob means probability, and refers to the levels of certainty to which an item fits the measurement model, based on its chi-square. 7. All the numbers are given to two decimal places because the errors are to two decimal places.
Goal Management has three stem-items ordered conceptually under Expectations, three stem-items conceptually ordered under Interest and three conceptually ordered under Goal Setting. The first result to note is that three of the 18 items did not fit the measurement model.
Choe Kee Cheng and Russell Waugh
20
APPENDIX B Fit of Items based on the Self Management aspect of Student Receptivity to Project Work Item Location Self Management Item 19 -0.87 Item 20 -0.13 Item 25 -0.77 Item 28 1.14 Item 29 -0.26 Item 31 -0.43 Item 32 0.67 Item 34 0.46
SE
Residual
df
ChiSq
Prob
0.06 0.05 0.06 0.05 0.05 0.06 0.05 0.05
0.05 -1.43 -0.91 0.91 2.43 -0.19 -0.83 0.02
723.66 723.66 723.66 723.66 723.66 723.66 723.66 723.66
10.22 10.12 11.64 15.10 12.84 9.76 6.68 12.16
0.33 0.34 0.23 0.09 0.17 0.37 0.67 0.20
APPENDIX C Fit of Items Based on the Learning Styles Aspect of Student Receptivity to Project Work Item Location Learning Styles Item 39 -0.08 Item 42 0.60 Item 44 -0.74 Item 46 -0.18 Item 49 -1.04 Item 50 -0.04 Item 51 -0.99 Item 52 -0.19 Item 54 0.18
SE
Residual
df
ChiSq
Prob
0.05 0.06 0.06 0.05 0.06 0.05 0.06 0.05 0.05
1.37 0.29 1.47 0.60 -0.31 2.40 -0.61 2.12 -0.02
723.66 723.66 723.66 723.66 723.66 723.66 723.66 723.66 723.66
10.25 4.62 14.25 5.54 8.21 11.71 6.26 10.84 8.74
0.33 0.87 0.11 0.78 0.51 0.23 0.71 0.29 0.46
APPENDIX D Fit of Items based on the Collaboration aspect of Student Receptivity to Project Work Item Collaboration Item 56 Item 58 Item 59 Item 62 Item 66 Item 67 Item 68 Item 70 Item 72
Location
SE
Residual
df
ChiSq
Prob
0.44 0.60 0.11 -0.88 0.00 -0.47 0.05 0.34 0.51
0.06 0.06 0.05 0.06 0.06 0.06 0.06 0.05 0.05
-0.05 -0.29 0.68 0.40 0.68 -0.47 -0.77 -1.06 -0.55
723.66 723.66 723.66 723.66 723.66 723.66 723.66 723.66 723.66
9.66 6.34 9.53 7.12 11.21 2.92 8.28 11.00 4.90
0.38 0.71 0.39 0.62 0.26 0.97 0.51 0.28 0.84
A Rasch Measure of Student Receptivity to Project Work at a Junior College…
21
APPENDIX E Fit of Items based on the Knowledge Application aspect of Student Receptivity to Project Work Item Location Knowledge Application Item 73 -0.77 Item 74 0.15 Item 76 0.31 Item 77 -0.72 Item 78 0.33 Item 80 -0.24 Item 81 -0.95 Item 82 -0.12 Item 84 0.06 Item 85 0.15 Item 86 1.06 Item 87 0.49 Item 89 0.46 Item 90 1.33
SE
Residual
df
ChiSq
Prob
0.06 0.06 0.06 0.06 0.06 0.06 0.06 0.06 0.06 0.05 0.05 0.05 0.05 0.05
-1.22 -0.48 0.23 -1.06 0.62 0.64 -1.27 0.03 -0.18 0.88 0.27 2.64 2.18 1.24
723.66 723.66 723.66 723.66 723.66 723.66 723.66 723.66 723.66 723.66 723.66 723.66 723.66 723.66
8.44 8.87 4.14 10.82 14.35 4.40 12.60 6.70 3.50 10.72 7.95 15.93 10.71 9.74
0.49 0.45 0.90 0.29 0.11 0.88 0.18 0.67 0.94 0.30 0.54 0.07 0.30 0.37
APPENDIX F Fit of Items based on the Communication aspect of Student Receptivity to Project Work Item Communication Item 92 Item 94 Item 96 Item 98 Item 100 Item 102 Item 104 Item 105 Item 106 Item 107
Location
SE
Residual
df
ChiSq
Prob
-0.38 -0.70 -0.24 -0.69 -0.45 -0.29 0.33 -1.04 0.59 -1.20
0.06 0.06 0.06 0.06 0.06 0.06 0.05 0.06 0.05 0.07
-0.02 -1.12 0.35 0.48 0.38 0.56 0.94 0.03 0.79 -0.25
723.66 723.66 723.66 723.66 723.66 723.66 723.66 723.66 723.66 723.66
6.85 14.40 8.91 4.00 2.81 4.91 4.96 12.71 5.60 10.80
0.65 0.11 0.45 0.91 0.97 0.84 0.84 0.18 0.78 0.29
In: Specialized Rasch Measures… Editor: Russell F. Waugh, pp. 23-47
ISBN: 978-1-61668-032-9 © 2010 Nova Science Publishers, Inc.
Chapter 2
RASCH MEASURES FOR SPORTS, DRAMA AND MUSIC STUDENT SELF-VIEWS BASED ON GARDNER INTELLIGENCES Ahdielah Edries1 and Russell F. Waugh2 1
2
Australian Islamic College Faculty of Education and Arts; Edith Cowan University Mount Lawley; Western Australia
ABSTRACT A co-educational Independent Australian Islamic College has three campuses which cater for migrant students from war-torn countries and others with culturally and linguistically, diverse backgrounds. This paper is part of a larger study to identify the strengths and interests of Islamic students, across eight of Gardner‟s intelligence domains, as perceived by the students, so that the College could better meet the needs of these students. This study is important for the Islamic College because it is hoped that the study will lead to the provision of opportunities for students to increase their confidence, self-esteem and motivation, and to achieve better in academic and non-academic areas. Student self-views were based on three aspects: (1) Things I really like; (2) Things I enjoy; and (3) Things I prefer, with items answered in two perspectives What I would like to do and What I actually do. This paper reports a Rasch analysis of student self-views based on three Gardner Intelligences: Sports, Drama and Music (N=321). All 12 items fitted the measurement model for Sports Self-Views, 9 out of 12 items for Drama Self-Views and all 12 items for Music. For all items, students found it easier to say what they would like to do than to actually do it. The item-trait interaction chi-squares are respectively: x2 =69.56, df=48, p=0.02; x2 =43.39, df=36, p=0.41and x2 = 52.85, df = 48, p= 0.29 showing no significant interaction between student measures and item difficulties along the scale, thus supporting uni-dimensional scales. The Person Separation Indices are respectively 0.88, 0.89 and 0.88 with standard errors of about 0.10 logits showing acceptable separation of measures compared to errors, and improvements could be made by adding more items to all measures.
24
Ahdielah Edries and Russell F. Waugh
BACKGROUND FOR CHAPTERS TWO AND THREE Three campuses of an Australian Islamic College cooperate and work together by following the same policies to provide a holistic education for all the students by integrating Islamic values and information technology in all the subject matter. The first campus opened its doors in February, 1986. This campus was established with two teachers and 50 students and it delivered an academic education based on a framework of Islamic ethos and values. Its students have now been incorporated into three main campuses. An addition was constructed in 1990 with its Technological Centre being developed in 1994. Its total enrolment in 2009 is 550 students. The second campus was established in 1996 and a new double-storey building on the school ground was opened in March 2003 to accommodate the increasing number of enrolments. This college caters for students between Kindergarten and Year 10 (5 to 15 years old), total enrolment in 2009 is 708 students. The third campus is the most recent addition and this was purchased in 2000 to cater for the increasing number of Islamic high school students. This College offers Kindergarten through to Year 12 and has been successful in producing graduates who have entered tertiary education in recent years (N=120). This year (2009), the three campuses of the Australian Islamic College have approximately 2300 students, over 200 teachers and supporting staff (Magar, 2008 p. 13). Small class sizes in the Colleges enable students and teachers to interact more efficiently and productively then they could with larger classes. Class sizes range from around 25-30 students in Lower Primary with a Teacher and Teacher Aide per class (Kindy – Year 1) to 20-25 students in Middle Primary and in High School (Edries, 2008). The philosophy, Islamic Values and Academic Excellence for Your Children’s Success in this Life and the Hereafter, sums up how the Islamic Colleges govern themselves and educate their students. All classroom curricula are based on the Western Australian Curriculum Framework (Curriculum Framework, 2008), outcome-based teaching and learning where teachers plan, conduct lessons and assess their students through outcomes. Portfolios, fortnightly assessments, formal testing (English literacy and numeracy), National English Testing (National Assessment Program Literacy and Numeracy), Interschool Competitions (University of New South Wales) and progress maps are used to record students‟ academic performance throughout their education at the Colleges. Islamic values are integrated in all the different subject areas, thus allowing students to learn classroom concepts and relate them to life and their faith. The students are encouraged to excel in secular and non-secular subjects, integrate with other communities and strive towards a goal in life. The Islamic Colleges encourage academic excellence in all areas, as well as good behaviour and conduct through a reward system amongst the students where positive points may be accumulated to be exchanged for a prize at the end of a week (Edries, 2008). On the other hand, a negative-point system also exists to discourage anti-social behaviour and to monitor the students‟ conduct through parent-teacher conferences and shared concerns. The school conducts several inter-school visits with neighbouring government and private schools, where students meet to exchange ideas about religion and form friendships outside their Islamic communities. Other types of visits include celebratory occasions such as Harmony Day, and Inter-Faith Sports for Peace Carnival (Edries, 2008). In the primary Islamic colleges, students attend English, Mathematics, Science, Society &
Rasch Measures for Sports, Drama and Music Student Self-Views…
25
Environment lessons. In addition to these core subjects, the students have to attend the LOTE (Arabic), Islamic Studies, Sports, Health, Art and Computing classes (Deria, 2006a). Each college day includes six, 45-50 minute periods, a recess and lunch break, prayers after lunch and morning and afternoon assemblies. Most of the teachers are trained in Australia and they are encouraged to utilise the best and latest pedagogy in their classrooms, share resources and promote the best teaching methods at the weekly meetings after school (Deria, 2006a). In the secondary Islamic colleges, qualified teachers are employed to give the students the best education possible, especially in Senior School (Deria, 2006b). The boys and girls are separated into single-sex classes and are taught by male and females teachers respectively. The aim is to discourage any distraction in class and allow more comfortable interactions between staff and students than might be possible with mixed boys and girls classes. Secondary college subjects taught at Middle school level are English, Mathematics, Science, Society & Social Environment, Computing, Art, Health, Sports and Islamic Studies (Deria, 2006b). In the senior years (Years 11 & 12), the subjects vary a little depending on student choices and numbers; examples of Tertiary Entrance Ranked subjects include LOTE-Arabic, English, Senior English, English as a Second Language (ESL), English Literature, Geography, History, Political and Legal Studies, Calculus, Geometry & Trigonometry, Physics, Chemistry, Biology, and Human Biology (Deria, 2006c). Each College runs its own education support system where the educational and emotional needs of the students are catered for by a school psychologist through consultations and testing. Education support classes are provided for the weaker students from each year group and, while gifted and talented classes are offered for the advance students of each year group, resources are limited and the Colleges would like to do more in this regard (Edries, 2008). Behaviour management consultations (such as the National Safe School‟s Friendly Policy, 2008) are provided where needed. Although all the students share the same faith of Islam, they originate from various cultural groups. A small percentage (20%) of students‟ are Australian-born (first generation Australians) and the next largest group (30%) of students was born in Iraq and Somalia. The remaining students (50%) come from all the continents of the world such as Africa, America, Asia, Europe and the Pacific Islands. Due to this diversity of cultures, the students speak a variety of languages and dialects, as well as conforming to different customs in areas of community life. The wide variety of cultural groups in the Australian Islamic College has positive aspects and limitations. Such a diverse group of students in one College makes the teaching of diversity, multiculturalism and tolerance easier because the students can learn from each other and they bring their own „voice‟ to the classroom. Celebrations of multiculturalism brings a deeper meaning when the students come dressed in their cultural clothing, bring home-cooked food of their nations and share experiences from their country of origin. However, such a diverse cultural mix also has its problems because of difficulties in communication, low English literacy acquisition and cultural conflicts. College Parent-Teacher conferences require translators and newsletters sent home have to be translated in other languages such as Bosnian, Somali, Arabic and Afghan (Edries, 2009). The Islamic Colleges also spend a lot of funds purchasing and maintaining literacy and numeracy programs that are IT-based for every classroom, such as the Accelerated Reader and Accelerated Mathematics Programs (Edries, 2008). These programs encourage users to take some charge of their learning and become independent learners to a greater extent. The Accelerated Reader Program is linked to the
26
Ahdielah Edries and Russell F. Waugh
Colleges‟ library catalogues and students borrow a book every week to read and then sit for a computer quiz that assesses their reading level and directs them to another reading level book. The Accelerated Mathematics Program is a collection of Mathematics topics (categorized by year groups) that the computer produces, such as exercises, tests and revision questions for students to complete and that are later marked through the computer program (Edries, 2008). Every student has a different Mathematics level and can progress at his or her own pace, thus making their learning less stressful and more positive. These programs are conducted as extension exercises to the students‟ usual study load. The Australian Islamic Colleges have New Arrivals classes that are an essential starting point for many of its migrant students. In these classes, specialist teachers help to integrate the students into the formal education system and provide the basic English literacy skills to cope with their age-level classes in six months to a year‟s time. However, this time duration isn‟t sufficient for most of these students to achieve at the mainstream standard. They often find it very difficult to cope at the standard of the mainstream, not because they are academically incapable, but because they simply haven‟t mastered the English language yet (Edries, 2009; personal observation as Principal). Many of the classroom teachers have to adapt their pedagogy to suit the multiple student needs in their classrooms. In the Islamic College mainstream classrooms, the assumption that all the students who have come from overseas, or government schools in Perth, have basic English literacy and numeracy skills is challenged when these students still fall into the lower percentile of academic performance compared with their same year level peers across Western Australia. In addition to the usual pressure to produce excellent results nationally (Tertiary Entrance Ranking table), the College also has to contend with national testing such as the National Assessment Program in Literacy and Numeracy (previously known as Western Australian Literacy and Numeracy Assessment) each year (Edries, 2008). The Islamic Colleges have taken this challenge to improve English Literacy and Numeracy standards by adapting their hiring policy to encourage very motivated teachers, and by retaining excellent staff through meritocracy-based salary reviews (Magar, 2008a). Handling such a dependent and disadvantaged group of students (and their families) is also draining on the resources of the Australian Islamic College as there are insufficient funds and staffing to implement all that is considered necessary. Computers are essential tools for the IT-based programs and the growing number of students will require more access to up-todate computers and computer programs. The computer programs are expensive and have to be maintained by support staff, thus adding further staffing costs. To prepare the students for national testing and general school work, the Islamic Colleges also run after-school, weekend and holiday classes that see committed teachers volunteer their time and effort to help the students achieve a better understanding of their school work. All these mean that the Colleges run almost all day, every day, and this accrues high energy costs (Edries, 2009; personal observation as Principal). Cultural conflicts and misunderstandings are unavoidable in the Islamic Colleges, especially when new students and families arrive in Australia for the first time. These people bring their own conceptions about education and social interactions, and handling confrontations is an acquired skill at the Colleges. The administration, Heads of schools and teachers have to deal with these interactions sensitively to avoid escalation in conflict (Edries, 2009; personal observation as Principal). Education Support services are essential in this area but often there is a lack or unavailability of resources, including interpreters and mediators.
Rasch Measures for Sports, Drama and Music Student Self-Views…
27
The Islamic Colleges often have to rely on external services offered by independent organizations, or small government funded associations, to assist with helping these new families adapt to new life in Australia and therefore ease their children into the Australian school system. Families who come from war-torn countries, or who have lived in refugee camps for several years, tend to arrive with children who have emotional and social difficulties at school thus affecting their already limited schooling and literacy acquisition (Edries, 2009; Haig & Oliver, 2007). Generally, the Intensive English classes are small in number (N=15) and the students are given quality time, thus affording them the opportunity to make some progress. However, some students still experience great difficulty in their learning, and continue to struggle throughout the mainstream classes (Edries, 2009). This is why interventions involving programs based on multiple intelligence theory could be important to the Islamic College students. If student abilities across Gardner‟s intelligences can be ascertained, then programs can be tailored to give each student some success in at least one subject, involving at least one of the intelligences.
RATIONALE FOR THE STUDY There are few significant studies (Haig & Oliver, 2007) which have dealt with the attitudes and needs of students from cultural and linguistically diverse (CALD) backgrounds in Australia (especially migrants and refugees from war-torn countries); and no recent studies in Australia could be found that have investigated how Australian schools could effectively improve and enhance the learning of these migrant students. Since the majority of the students within the Islamic Colleges are born overseas (particularly from war-torn countries), or live in families with parents born overseas, the students‟ English literacy and numeracy limitations have to be addressed by any Islamic College, as little help comes from the homes. In particular, research has never been undertaken in the Western Australian Islamic Colleges to investigate the intelligences, strengths and interests of the students, and the teacher perceived needs of their students. Cook-Sather (2002, p.3) stated that there are certain pedagogical merits in understanding and utilising student perspectives in schools. Choe (2006) in his recent study in Singapore asserts that hearing and listening to student voices is important in education because of the various ways it can improve educational practices, reinform existing conversations about educational reform, and point to the discussions and reform efforts yet to be undertaken. Students‟ who feel engaged with what they are learning, feel empowered because their views are valued, or teachers who, in seeing the world from the students‟ point of view, adopt more effective instructional approaches to teach their students, but these statements arose from research with advantaged students, not migrant students with poor English and numeracy skills. Secondly, there is a need to address the issues that students from Culturally and Linguistically Diverse Backgrounds have to contend with in a traditional classroom. On a daily basis, the administration and staff at the Islamic Colleges grapple with issues such as helping students deal with their past traumatic experiences and their new way of life in Australia; the consequences of severely disrupted prior schooling; a lack of social understanding and learning strategies to process content; and poor literacy in their first
28
Ahdielah Edries and Russell F. Waugh
language (which is needed to support the acquisition of a second language); meeting classroom demands for literacy and communication; obtaining appropriate learning resources; and a lack of parental support due to lack of education. According to Haig and Oliver (2007) these deficiencies are compounded by the complexity and specificity of cognitive academic knowledge used in schools. Cognitive development which has taken place over many years in the classroom is clearly one of the key elements which students from Non-English speaking backgrounds are missing when they have had interrupted (or no prior) schooling (Miller, Mitchell, & Brown, 2005). This makes it difficult for staff at the Australian Islamic College to target learning at the right level, as students in their classes have variable skills and gaps in their learning. It is hoped that the current study will provide some information to develop and implement effective school and educational practices to cater for its students at the Islamic colleges in Perth. Student learning, motivation and engagement in a variety of academic and non-academic areas are important for success at school (Bragg, 2005). Whilst there are numerous factors that impact on the wellbeing, learning and teaching of students, it is not in the scope of the present study to investigate all of these. The present study will only focus on the need to determine student interests in relation to some Gardner intelligence areas and their perceived interests and strengths. Gardner views "intelligence" as a biological and psychological potential that is capable of being realized to a greater or lesser extent in everyone, depending on one's experience, education, social environment, and other factors (Gardner, 1993a). The majority of students attending the Islamic Colleges in Perth overcome obstacles such as adjusting to their host country, facing confronting rules and issues in school (something to which a large proportion of these students have not been exposed to in their own countries); and, coupled with compliance to learning from a traditional curriculum (which often does not take into account their prior background and past experiences) makes school difficult for them. When students‟ views, strengths and interests are known, action can be taken to address difficulties that students might be facing and further develop their strengths. Consequently, this study attempts to use students‟ and teachers‟ perspectives on what can be done to enhance student learning, and how their needs can be effectively met within the school environment. Unidimensional, linear scales for self concepts relating to Interpersonal, Intrapersonal, English, Mathematics, Art, Sport, Music, and Drama domains were created using a Rasch Measurement Model computer program, RUMM 2020 (Andrich, Sheridan & Luo, 2005). This paper reports on the RUMM 2020 output, that is, the statistics showing a good fit to the measurement model for the data relating to the Interpersonal and Intrapersonal domains. These domains were chosen to present first because they are important to students who come from war-torn countries like Lebanon, Somalia, Iraq and Ethiopia. While the academic subject self concepts and sports self-concepts are important too, it was considered, more important to be able to make valid inferences about student Interpersonal and Intrapersonal self-concepts first. Rasch measures for student self-views based on other Gardner Intelligences were made as part of a larger study, but not reported here.
Rasch Measures for Sports, Drama and Music Student Self-Views…
29
GARDNER INTELLIGENCES Definitions of Gardner’s Intelligence Domains The Intelligence Domains are based on Gardner‟s Multiple Intelligence Theory (1993, 1999, 2004) and the definitions below are taken from his work. Linguistic Intelligence refers to the capacity to use words effectively, whether orally or in writing. This includes the ability to manipulate the structure and syntax of language, the sounds of language, the meanings of language, and the practical uses of language, such as for explaining, remembering, and persuading. This intelligence is evident in children who demonstrate strength in the language arts: speaking, writing, reading, listening. They learn best by saying, hearing and seeing words and they are good at memorizing names, places, dates and trivia. These students have always been successful in traditional classrooms because their intelligence lends itself to traditional teaching. Logical/Mathematical Intelligence refers to the capacity to use numbers effectively and to reason well. This includes awareness of logical patterns and relationships, functions, and cause and effect. This intelligence is evident in children who display an aptitude for numbers, reasoning and problem solving. They like to participate in experiments, figure things out, ask questions, explore and discover patterns and relationships. They learn best by categorizing, classifying, and working with abstract patterns. These children typically do well in traditional classrooms where teaching is logically sequenced and students are asked to conform. Visual/spatial Intelligence refers to the ability to perceive the visual and spatial world accurately, including sensitivity to colour, line, shape, form, space, and the relationships between them. Includes the capacity to visualize, make graphic representations, and orient oneself in spatial surroundings. This intelligence is evident in children who learn best visually and organise things spatially. They like to see what you are talking about in order to understand. They enjoy charts, graphs, maps, tables, illustrations, art, imagining things, puzzles, costumes (basically anything that is eye catching). Students learn best by visualizing, dreaming, working with colours and pictures. Bodily/Kinesthetic Intelligence refers to the ability to use one's whole body to express ideas and feelings, and the ability to fashion or transform with one's hands. This includes skills such as coordination, balance, dexterity, strength, flexibility, speed, and other physical skills. This intelligence is evident in children who experience learning best through activity: games, movement, hands-on tasks, building. They learn best by touching, moving, interacting with space and processing knowledge through bodily sensations. These children are often labelled „overly active‟ in traditional classrooms where they are told to sit and be still. Musical Intelligence refers to the ability to perceive, distinguish between, and express oneself in musical forms. It includes sensitivities to rhythm, pitch or melody, timbre, and tone colour. It can apply to either an intuitive grasp of music, or an analytic or technical understanding of it, or both. This intelligence is evident in children who learn well through songs. They like to sing, hum tunes, listen to music, patterns, rhythm, play an instrument and respond to musical expression. These children are good at picking up sounds, remembering melodies, noticing pitches and rhythms and keeping time. It is easy to overlook children with this intelligence in traditional education.
30
Ahdielah Edries and Russell F. Waugh
Interpersonal Intelligence refers to the capacity to perceive and distinguish differences in the moods, intentions, motivations, and feelings of others. It includes sensitivity to facial expressions, gestures, and body language. This intelligence also includes the ability to respond to these cues effectively, to work well with others, and to lead. This intelligence is evident in children who are noticeably people orientated and outgoing, and do their learning cooperatively. They are good at understanding people, leading others, organizing, communicating, manipulating and mediating conflicts. They learn best sharing, helping others and asking for help. These children may have typically been identified as „talkative‟ or „too concerned about being social‟ in a traditional setting. Intrapersonal Intelligence refers to the capacity for self-knowledge and understanding, and the ability to act on the basis of that knowledge. It includes having an accurate picture of one's own strengths and limitations, inner moods, intentions, feelings, motivations, needs, and desires, and a capacity for self-discipline and self-esteem. This intelligence is evident in children who are especially in touch with their own feelings, values, and ideas. They may tend to be more reserved, like to work alone and reflect on problems, but they are actually quite intuitive about what they learn and how it relates to themselves. They are good at being independent, and learn best by being given time to think. Naturalist Intelligence refers to the capacity of children to love the outdoors, to be with animals, and to go on field trips. They are good at categorizing, organizing a living area, planning a trip, preservation and conservation. Learns best by studying natural phenomenon in a natural setting learning about how things work. These children love to pick up on subtle differences in meanings. The traditional classroom has not been accommodating these children. Existentialist Intelligence refers to the capacity of children to learn in the context of where humankind stands in the „big picture‟ of existence. They ask, “Why are we here?” and “What is our role in the world?” This intelligence is seen in the discipline of philosophy. Gardner (1999) suggested that the nine intelligences very rarely operate independently. Rather, the intelligences are used concurrently and typically complement each other as individuals develop skills or solve problems. Viewed in this way human intelligence is not restricted to only the more narrow linguistic and mathematical abilities measured by the common standardized tests in which high scores traditionally described students in school as being „smart‟. There are research studies that confirm that by addressing students‟ culture, language, and social status with appreciation, inclusion, and sensitivity increases students‟ academic successes (Grant & Tate, 1995; Jimenez, 1997). This chapter reports on the RUMM 2020 output and data analysis for the data relating to Sports, Drama and Music. This analysis features important inferences that can validly be made from the reliable Rasch-created linear measures, describes the RUMM 2020 output for item difficulties (easiest and hardest), and the weakest 25 student measures for Sports, Drama and Music Self-Concepts. Data from the targeting graphs by gender enable valid inferences to be made by identifying the easy and hard items and the weak students who may require intervention. A summary of main findings is listed at the end of this chapter. Suggested improvements and implications for helping and improving students are not discussed here in order that the actual results can be separated from the discussion and implications which are presented in the last thesis chapter.
Rasch Measures for Sports, Drama and Music Student Self-Views…
31
INITIAL RASCH ANALYSIS The RUMM 2020 computer program (Andrich, Sheridan & Luo, 2005) was used to analyse the 12 items for each of the Self-Concepts for Sports, Drama and Music, separately, based on the Partial Credit Model of Rasch (Masters, 1997). While the 12 items for the Sports Self-Concept and Music Self-Concept fitted the measurement model, only nine items fitted the measurement model for Drama Self-Concept. For Drama Self-Concept, items 5, 6 and 10 were deleted and the RUMM analysis repeated. For the Music Self-Concept, the response categories for items 9 and 11 were collapsed from three to two because the two higher response categories were not answered consistently and logically, and the RUMM analysis repeated.
Final Rasch Analysis The results of the final Rasch analyses are now presented for the Self-Concepts of Sports (12 items), Drama (9 items) and Music (12 items).
Dimensionality The item-trait chi-square values for the Self-Concepts of Sports, Drama and Music are, respectively , x2 = 69.56, df=48, p=0.02; x2 = 43.39, df=36, p=0.41; and x2 = 52.85, df=48, p=0.29. These mean that there is good agreement (less so for Sports Self-Concept) amongst the students about the difficulties of the items along scales and that there is a good fit of data to the measurement model involving a unidimensional trait (less so for Sports Self-Concept) Reliability The standard errors of measurement were about 0.1 logits for Sports, Drama and Music Self-Concepts, and the Person Separation Indices are 0.88, 0.89 and 0.88 respectively, indicating excellent separation of measures in comparison to the errors. These indicate that the measures are well separated in comparison to the errors. Table 1. Reliability Indices for Sports, Drama and Music Self-Concept Sports Drama Music
Standard Error of Measurement 0.10 0.10 0.10
Person Separation Index 0.88 0.89 0.88
Power of test-offit Excellent Excellent Excellent
Item Fit to the Measurement Model The RUMM 2020 program calculates individual item fits to the measurement model and these are given in Tables 2, 3 and 4 respectively for the Sport, Drama and Music Self-Concepts. The tables are accompanied by some explanatory notes relating to individual item fits to the measurement model. From the analysis it was established that for Sport self-concept, 12 out of
Ahdielah Edries and Russell F. Waugh
32
12 items fitted the measurement model with a probability greater than p=0.01 and, the power of test-of-fit is excellent, based on a Person Separation Index of 0.88. For the Drama SelfConcept, all nine items fitted the measurement model with a probability of greater than p=0.05 and, the power of the test-of-fit were excellent, based on a Person Separation Index of 0.89. For Music Self-Concept, all 12 items fitted the measurement model with a probability greater than p=0.04 and, again the power of the test-of-fit were excellent, based on a Person Separation Index of 0.88. Table 2. Fit of items to Rasch Measurement Model (Sport Self-Concept) Item 1 2 3 4 5 6 7 8 9 10 11 12
Location -0.59 -1.21 -0.11 -0.71 1.31 0.99 0.02 -0.41 0.78 0.16 0.19 -0.32
SE 0.11 0.13 0.10 0.11 0.09 0.09 0.10 0.10 0.09 0.09 0.09 0.10
Residual -1.36 -1.02 -1.81 -0.86 1.54 2.35 -1.15 -0.82 -0.17 -0.21 0.38 0.90
df 241.00 241.00 241.00 241.00 241.00 241.00 241.00 241.00 241.00 241.00 241.00 241.00
Chi-Square 14.51 1.24 8.92 27.2 3.50 13.00 5.21 10.05 3.91 4.04 0.92 1.55
Probability 0.01 0.87 0.06 0.61 0.48 0.01 0.27 0.04 0.42 0.40 0.92 0.82
Table 3. Fit of items to Rasch Measurement Model (Drama Self-Concept) Item 1 2 3 4 5 6 7 8 9 10 11 12
Location +0.80 -0.48 +0.39 -0.75 N/F N/F +0.30 -0.69 +0.49 N/F -0.57 -0.38
SE 0.10 0.09 0.10 0.09 N/F N/F 0.09 0.09 0.09 N/F 0.09 0.09
Residual -1.46 -0.44 0.07 -0.85 N/F N/F -0.67 0.35 0.64 N/F 0.11 -0.17
df 252.85 252.85 251.06 251.06 N/F N/F 252.85 252.85 251.95 N/F 252.85 252.85
Chi-Square 5.70 2.51 4.58 4.14 N/F N/F 3.33 3.42 5.78 N/F 1.42 1.97
Probability 0.22 0.64 0.33 0.39 N/F N/F 0.49 0.22 0.10 N/F 0.84 0.74
Explanatory Notes on Table 2, 3 & 4 1. Location refers to the difficulty of the item on the linear scale. 2. SE refers to standard error, that is, the degree of the uncertainty in a value: in this case, the standard error for each of the items is reasonable, ranging from 0.09 to 0.14 logits (for table 1) and 0.08 to 0.1 logits (for table 2) respectively. 3. Residual represents the difference between the expected value on an item, calculated according to the Rasch measurement model, and its actual value. 4. df (degrees of freedom) refers to the number of scores in a distribution that are free to change without changing the mean of the distribution. 5. Chi-square is the statistic used to determine fit to the measurement model.
Rasch Measures for Sports, Drama and Music Student Self-Views…
33
6. Probability, refers to the levels of certainty to which an item fits the measurement model, based on its chi-square. 7. N/F means no fit (hence, the item was deleted) 8. All values are given to two decimal places because the errors are to two decimal places. Table 4. Fit of items to Rasch Measurement Model (Music Self-Concept) Item
Location
SE
Residual
df
Chi-Square
Probability
1
-0.67
0.09
0.10
268.63
1.73
0.79
2
-1.05
0.10
-1.72
268.63
9.86
0.04
3
0.60
0.09
0.70
267.72
4.59
0.33
4
-0.54
0.09
1.37
267.72
3.56
0.47
5
-0.22
0.09
0.25
268.63
3.13
0.54
6
-0.59
0.09
-0.54
268.63
5.91
0.21
7
1.05
0.10
-0.37
267.72
1.15
0.89
8
-0.08
0.10
-1.52
268.63
5.64
0.23
9
0.91
0.14
-0.21
268.63
3.28
0.51
10
-0.11
0.09
0.90
267.72
9.53
0.05
11
0.58
0.13
0.39
268.63
1.89
0.76
12
0.12
0.09
0.10
267.72
2.58
0.63
Item-Person Fit Interactions The fit residual data for both items and students (Self-Views in Sports) has a mean near zero and a standard deviation near one showing that the data fit the measurement model satisfactorily and this means that there is a good consistency of student-item response patterns (see Table 5). The fit residual data for Drama Self-Views (see Table 6) and for Music SelfViews (see Table 7) similarly show good consistency of student-item response patterns. Table 5 Global Item and Person Fit to the Measurement Model for Sports Self-Views Item Locations
Item Residuals
Student Locations
Student Residuals
Mean
0.00
-0.19
1.27
-0.22
SD
0.73
1.26
1.29
1.01
Table 6. Global Item and Person Fit to the Measurement Model for Drama Self-Views Item Locations
Item Residuals
Student Locations
Student Residuals
Mean
0.00
-0.23
0.24
-0.22
SD
0.83
0.68
1.53
1.03
Ahdielah Edries and Russell F. Waugh
34
Table 7. Global Item and Person Fit to the Measurement Model for Music Self-Views Item Locations
Item Residuals
Student Locations
Student Residuals
Mean
0.00
-0.05
-0.19
-0.17
SD
0.67
0.91
1.36
1.04
Explanatory Notes on Tables 5, 6, & 7 1. Item location is item difficulty in logits 2. Person location is person measure in logits 3. SD is standard deviation 4. The mean item difficulty is constrained to zero by the RUMM 2020 program 5. Fit residuals are the difference between the actual values and the expected values calculated according to the measurement model (standardised the data fit the measurement model (a good fit for these data). They have a mean near zero and an SD near 1 when the data fit the measurement model (as is the case for these data). 6. All values are given to two decimal places because the errors are to two decimal places.
TARGETING
Notes on Figures 1, 2 and 3 1. The scale is in logits, that is, the log odds of answering the response categories. 2. Person measures (low to high) are given on the upper side in logits. 3. Item thresholds (easy to hard) are given on the lower side in logits. Figure 1. Person measure/Item Threshold Graph for Sport Self-Views.
It can be seen from the distribution of items in Figure 1 that the item difficulties mostly cover the middle range (-1.6 to +1.4 logits) of Sports Self-View measures. This indicates that some easier and harder items need to be added in any revision of the scale to cover students
Rasch Measures for Sports, Drama and Music Student Self-Views…
35
with the lowest measures (-2.73 logits) and students with the highest (+3.41 logits) measures. Eight students (of which seven were female) had a very low Sports Self-View, their raw scores ranging from 0 (extremely low) to a score of 7 for the lower end of the scale. Similarly, fifty-six students (nine females compared with their (47) male counterparts) had an extremely high self-view of Sport with a raw score of 24. Hence, targeting could be improved by adding some easy and hard items appropriate for students with the lowest and the highest measures and this would improve the measure.
Note: The Drama Self-View measures are on the upper side from low to high and the item thresholds are on the lower side of the graph from easy to hard. Figure 2. Person measure/Item Threshold Graph for Drama Self-Views.
Figure 2 shows that two-hundred and twenty-six students (102 males and 124 females) out of a total of three-hundred and twenty-one students were able to answer the items positively relating to Drama Self-Views (that is, the item difficulties mostly covered the middle range (-1.8 logits to +1.4 logits) of Drama Self-View measures. Thirty-two students (twenty-six males and six females) were in the lower range (-1.8 logits to -3.22 logits) of the scale with scores ranging from 0 (extremely low) to a score of 2. In contrast, sixty-three students (twenty males compared with their (43) female counterparts) were in the higher range (+1.4 logits to +3.24 logits) of the scale with scores ranging from 12 to 19 (extreme high), indicating that some easier and harder items need to be added in any revision of the scale to cover the students with these measures.
36
Ahdielah Edries and Russell F. Waugh
Figure 3. Person measure/Item Threshold Graph for Music Self-Views.
Figure 3 shows the item thresholds for Music Self-Views range from easy (about -1.4 logits) to reasonably hard (about +1.4 logits); and the student measures calibrated on the same scale from low (about -3.25 logits) to high (about 3.21 logits). Forty-seven students (thirtythree males and fourteen females) had a very low Self-Views of Music, their raw scores ranging from 0 (extremely low) to a score of 3 for the lower end of the scale. Similarly, thirtyfour students (seven males compared with their (27) female counterparts) had an extremely high Self-View of Music with a raw score of 22. These measures indicate that some harder items need to be added to better target the attitudes and behaviours of students with high measures (that is, students who obtained scores from 14 to 22).
INFERENCES FROM THE LINEAR SCALES Since there was a good or reasonable fit to the measurement model for all three Self-Views, valid inferences can now be made about item difficulties and student measures on Sports, Drama and Music Self-Views. As stated previously, there is a prediction that the attitude items should be easier than the behaviour items and, where both fit the model, attitude items are easier than their corresponding behaviour items. Where attitudes are easier than their corresponding behaviors, it can be inferred that attitudes influence corresponding behavior. Tables 8, 9, and 10 below show the Item Difficulties for Student Sport Self-View, Drama Self-View, and Music Self-View respectively.
Rasch Measures for Sports, Drama and Music Student Self-Views…
37
Table 8. Item Difficulties for Student Sport Self-View Item
1 2 3 4 5 6
SPORT SELF-CONCEPT Things that I really like Playing sports Playing competitive games and sports Things I enjoy Watching sporting programs on TV like Fox Sport Being involved with sports and games Things I prefer Playing in sports teams where others watch me Playing ball games where I have to react fast
What I actually do
What I‟d like to do
-0.59 -0.11
-1.21 -0.71
+1.31 +0.02
+0.90 -0.41
+0.78 +0.19
+0.15 -0.32
Table 9. Item Difficulties for Student Drama Self-Views Item
DRAMA SELF-CONCEPT Things that I really like Being involved in drama classes Acting out different scenes and characters Things I enjoy Watching movie stars act out scenes and characters in movies Being involved with acting and plays Things I prefer Acting in plays where others watch my presentation Acting in musicals, comedies and plays
1 2 3 4 5 6
What I actually do
What I‟d like to do
+0.81 +0.39
-0.47 -0.75
N/F
N/F
+0.30
-0.69
+0.49 +0.50
N/F -0.57
Table 10. Item Difficulties for Student Music Self-Views Item
1 2 3 4 5 6
MUSIC SELF-CONCEPT Things that I really Singing and listening to music Playing a musical instrument Things I enjoy Watching musical shows on TV such as Idol and Video Hits. Being involved with musical productions Things I prefer Playing a musical instrument in a band or group Writing and reading music
What I actually do
What I‟d like to do
-0.67 +0.60
-1.05 -0.53
-0.22
-0.59
+1.05
-0.08
+0.91 +0.58
-0.12 +0.12
Explanatory Notes for table 8, 9, and 10 N/F means no fit (hence, the item was deleted). All values are given to two decimal places because the errors are to two decimal places.
ITEM DIFFICULTIES For the Sport Self-View attitudes (What they would like to do), students found it very easy to say that they would like to play sports (difficulty -1.21 logits), and they found it moderately easy to say that they would like to play competitive games and sports (difficulty -0.71 logits).
38
Ahdielah Edries and Russell F. Waugh
Students found it moderately hard to have an atittude that they would prefer to play in sports teams where other people watch them (difficulty +0.15 logits), and very hard to have an attitude that they like to enjoy watching sporting programs like „Fox Sport‟ on TV (difficulty +0.90 logits). In contrast, for actual behavior (What they actually do), students found it moderately easy to actually enjoy playing competitive games and sports (difficulty -0.11 logits). They found it moderately easier to actually play sports without competition (difficulty -0.59 logits). Students found it hard to actually play in sports teams where they are being watched by others (difficulty +0.78 logits), and very hard to say that they actually enjoyed watching sporting programs like „Fox Sport‟ (difficulty +1.31 logits). For the Drama Self-View attitudes (What they would like to do), students found it very easy to say that they would like to hold an attitude to like acting out different scenes and characters (0.75 logits). They found it moderately easy to have an attitude that they would like to enjoy being involved with acting and plays (difficulty -0.48 logits), and moderately easy to have an attitude that they prefer to act in musicals, comedies and in plays (difficulty -0.57 logits). In contrast, for actual behavior (What they actually do), students found it moderately hard to actually enjoy being involved in acting and plays (difficulty +0.30 logits), and moderately hard to actually enjoy acting out different scenes and characters (difficulty +0.39 logits). Students found it moderately hard to actually enjoy acting in musicals, comedies and plays (difficulty +0.50 logits), and very hard to be involved in Drama classes (difficulty +0.80 logits). For Music Self-View attitudes (What they would like to do), students found it very easy to say that they would really like to sing and listen to music (difficulty -1.05 logits), moderately easy to say that they would like to enjoy watching musical shows on TV like, „Idol‟ and „Video Hits‟ (difficulty -0.59 logits), and moderately easy to say that they would like to enjoy being involved with musical productions (difficulty -0.08 logits). They found it moderately hard to say that they would like to prefer to read and write music (difficulty +0.12 logits). In contrast, for actual behavior (What they actually do), they found it easy to actually like singing and listening to music (difficulty -0.67 logits) , and moderately easy to actually watch musical shows on TV like, „Idol‟ and „Video Hits‟ (difficulty -0.22 logits). Students found it hard to actually write and read music (difficulty +0.58 logits), hard to actually play a musical instrument (difficulty +0.60 logits), and very hard to actually play a musical instrument in a band or group (difficutly +0.91 logits) and very hard to actually be involved with musical productions (difficulty +1.05 logits).
LOWEST STUDENT MEASURES OF SELF-VIEWS The lowest 25 student measures were identified for Sports, Drama, and Music Self-Views (see Table 11, 12 and 13 respectively). These are the students who could be given extra support with these self-concepts to encourage them to take part in school activities better, to gain selfconfidence and to improve their knowledge and abilities. The 25 number cut-off for the lowest measures is somewhat arbitrary, but the highest measure in each of these tables represents answers of „some of the time‟, which represents a low self-view in Sports, Drama and Music respectively. A discussion of what could be done to help these students is given in the last chapter but, in the present chapter, the weakest students will be identified.
Rasch Measures for Sports, Drama and Music Student Self-Views…
39
Comments on the 25 students who have been identified as having a low Sport Self-Views (Table 11), a low Drama Self-View (Table 12), and a low Music Self-View (Table 13) are now given. The majority of students in the bottom twenty-five were females (20) compared to their male (5) counterparts. The raw scores indicate that these females have a very low Sport Self-Concept and might require urgent intervention. The above results are indicative that males have different perceptions of Sport than females, and this needs to be reviewed. Four students (2 males and 2 females) have the highest measure of 10, which represents answers of „some of the time‟ to only 10 of the 24 items, representing a low self-view in Sport. Table 11. Students with Lowest 25 Measures for Sport Self-View ID 237 45 145 153 14 243 228 233 241 91 165 108 2 49 46 40 231 62 3 242 52 55 251 299 11
Raw Score 1 3 3 4 5 6 6 7 8 8 8 8 8 8 9 9 9 9 9 9 9 10 10 10 10
Student Measure -2.73 -1.81 -1.81 -1.51 -1.26 -1.04 -1.04 -0.84 -0.66 -0.66 -0.66 -0.66 -0.66 -0.66 -0.48 -0.48 -0.48 -0.48 -0.48 -0.48 -0.48 -0.31 -0.31 -0.31 -0.31
SE 0.85 0.59 0.59 0.53 0.49 0.47 0.47 0.45 0.43 0.43 0.43 0.43 0.43 0.43 0.42 0.42 0.42 0.42 0.42 0.42 0.42 0.42 0.42 0.42 0.42
Residual -0.29 -0.02 0.76 1.18 -1.57 0.84 -0.35 -0.35 1.16 0.78 -0.34 -0.11 -1.62 0.56 0.27 -0.96 -0.82 -0.04 1.09 0.34 0.72 1.87 0.87 0.05 -1.63
Table 12. Students with Lowest 25 Measures for Drama Self-View ID 37 230 188 215 1 181 285 196 226 123 302 180
Raw Score 0 0 0 0 0 0 0 0 0 1 1 1
Student Measure -3.22 -3.22 -3.22 -3.22 -3.22 -3.22 -3.22 -3.22 -3.22 -2.38 -2.38 -2.38
SE 1.23 1.23 1.23 1.23 1.23 1.23 1.23 1.23 1.23 0.88 0.88 0.88
Residual -0.90 -0.90 0.41
Ahdielah Edries and Russell F. Waugh
40
Table 12. (Continued). ID 257 267 179 197 175 172 250 318 279 272 299 222 221
Raw Score 1 1 1 1 1 1 1 1 1 1 2 2 2
Student Measure -2.38 -2.38 -2.38 -2.38 -2.38 -2.38 -2.38 -2.38 -2.38 -2.38 -1.81 -1.81 -1.81
SE 0.88 0.88 0.88 0.88 0.88 0.88 0.88 0.88 0.88 0.88 0.69 0.69 0.69
Residual -0.90 -0.90 -0.90 -0.34 -0.58 -0.90 -0.90 -0.90 0.25 -0.46 0.24 0.20 -0.50
Table 13. Students with Lowest 25 Measures for Music Self-View ID 287 174 175 171 267 268 271 180 112 250 285 17 189 305 155 191 193 54 49 46 226 32 197 179 279
Raw Score 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1
Student Measure -3.25 -3.25 -3.25 -3.25 -3.25 -3.25 -3.25 -3.25 -3.25 -3.25 -3.25 -3.25 -3.25 -3.25 -3.25 -3.25 -3.25 -3.25 -2.51 -2.51 -2.51 -2.51 -2.51 -2.51 -2.51
SE 1.15 1.15 1.15 1.15 1.15 1.15 1.15 1.15 1.15 1.15 1.15 1.15 1.15 1.15 1.15 1.15 1.15 1.15 0.82 0.82 0.82 0.82 0.82 0.82 0.82
Residual -0.69 0.64 -0.29 -1.15 -1.15 -0.72 -0.29
Nine students (7 males and 2 females) had an extreme low raw score of 0 (-3.22 logits), and will require urgent extra support in Drama. Thirteen students (11 males and 2 females) had a score of 1, and will also require extra support. The remaining two students (lowest 25 measures) had a raw score of 2. These students answered, „some of the time‟ to the items, and will also benefit from extra support and attention in Drama. More males had a low Music Self-View than females (that is, 18 males compared with 7 females), indicating that intervention may be required as well as extra support in Music for
Rasch Measures for Sports, Drama and Music Student Self-Views…
41
these males. Raw scores for these students were quite low ranging from 0 to 1 (-2.51 to -3.25 logits).
Gender Differences The graphical data for gender is displayed in Figure 4 for Sports Self-Concept, Figure 5 for Drama Self-Concept and Figure 8.6 for Music Self-Concept respectively. Males had a higher mean Sports Self-Concept than females and this is statistically significantly higher (t=7.8, df=319, p-0.000). For Music Self-Concept, females had a mean higher Self-Concept than males and this is statistically significantly higher (t=6.7, df=319, p=0.000). For Drama Self-Concept, females had a higher mean Self-Concept than males and this is statistically significantly higher (t=4.25, df=319, p=0.000).
Figure 4. Target Graph by Gender for Sport Self-Views. Note: There is a colour error in the RUMM program. Purple corresponds to red (female) and green corresponds to blue (male). Self-View measures are on the upper side and item difficulties are on the lower side in standard logit units.
42
Ahdielah Edries and Russell F. Waugh
Figure 5. Target Graph by Gender for Drama Self-Views. Note: There is a colour error in the RUMM program. Purple corresponds to red (female) and green corresponds to blue (male). Self-View measures are on the upper side and item difficulties are on the lower side in standard logit units.
Note: There is a colour error in the RUMM program. Purple corresponds to red (female) and green corresponds to blue (male). Self-View measures are on the upper side and item difficulties are on the lower side in standard logit units. Figure 6. Target Graph by Gender for Music Self-View.
Further conclusions can be made from the graph for Sports Self-Views (Figure 4). From about -1.00 to -2.73 logits, there are 2.1% of students (1 male and 6 females), who have extremely low levels of Sport Self-Views. From -1 to 0 logits, 11.5% of students (9 males and
Rasch Measures for Sports, Drama and Music Student Self-Views…
43
28 females) have low levels of Sport Self-Views. From 0 to +1.0 logits, there are 30.8 % of students (34 males and 65 females) who have a medium level of Sport Self-Views. From+1.0 to +2.0 logits, there are 28% of students (37 males and 53 females) who have moderately high levels of Sport Self-Views. From +2.0 to +3.41 logits, there are 27.7% of students (68 males and 21 females) who have very high levels of Sport Self-Views. From Figure 5, further conclusions can be made about Drama Self-Views. There were 25.2 % of students (60 males compared with 21 females) who had low levels of self-view (1.00 to -3.25 logits). The majority of students (50.8%), comprising of 72 males and 91 females had a self-view in the range –1.00 to +1.00 logits. The remaining 25.5 % (25 males and 57 females) had a high self-view in the +1.0 to + 3.2 logits range, with 10 males and 19 females having a very high score of 3.2 logits. From Figure 6, further conclusions can be made for Music Self-Views. There were 21.2 % of students (60 males compared with 21 females) who had low levels of self-view (-1.00 to -3.25 logits). The majority of students (59.8%), comprising of 88 males and 104 females had a self-concept in the range –1.00 to +1.00 logits. The remaining 19.3 % (12 males and 50 females) had a high self-view in the +1.0 to + 3.3 logits range.
SUMMARY OF FINDINGS The reliability of the scale data relating to Sport Self-Views (12 items), Drama SelfViews (10 items), and Music Self-Views (12 items) was shown by: 1. Good global and person item fit to the measurement model, and good individual fit to the measurement model. 2. The standard errors of measurement are about 0.1 logits for Sports, Drama and Music Self-Views, and the Person Separation Indices are 0.88, 0.89 and 0.88 respectively, indicating excellent separation of measures in comparison to the errors. 3. The item-trait interaction for the 12 items of Sports Self-Views, the 9 items of Drama Self-Views, and the 12 items of Music Self-Views were satisfactory, indicating a satisfactory overall fit to the measurement model for each variable and the measurement of a uni-dimensional trait. 4. Reasonable targeting of the items against the person measures, although some easier and harder items need to be added for any future use of the three scales. Since the scale data were shown to be reliable, the following valid inferences were drawn from the scales. 1. All attitude relationships (What I would like to do) were easier than their corresponding actual behaviors (What I actually do). 2. The easiest attitude item (What I would like to do) for Sports Self-View was playing sports (and very easy, at -1.21 logits). The hardest attitude item (What I would like to do) for Sports Self-Views was watching sporting programs like „Fox Sport‟ on TV (and hard at +0.90 logits). The easiest behaviour item (What I actually do) for Sports Self-Views was playing sports (moderately easy at -0.59 logits). The hardest behavior
Ahdielah Edries and Russell F. Waugh
44
3.
4.
5. 6. 7.
item (What I actually do) for Sport Self-Views was watching sporting programs like „Fox Sport‟ on TV (and very hard at +1.31 logits). The easiest attitude item (What I would like to do) for Drama Self-Views was acting out different scenes and characters (and very easy, at -0.75 logits). The hardest attitude item (What I would like to do) for Drama Self-Views was being involved in Drama classes (and easy at -0.48 logits). In contrast, the easiest behavior item (What I actually do) for Drama Self-Views was being involved in acting and plays (moderately hard at +0.30 logits). The hardest behavior item (What I actually do) for Drama Self-Views was being involved in Drama classes (and very hard at +1.39 logits). The easiest attitude item (What I would like to do) for Music Self-Views was singing and listening to music (and very easy, at -1.05 logits). The hardest attitude item (What I would like to do) for Music Self-View was reading and writing music (and moderately hard at +0.12 logits). In contrast, the easiest behavior item (What I actually do) for Music Self-View was singing and listening to music (and easy at 0.67 logits). The hardest behavior item (What I actually do) for Music Self-View was being involved with musical productions (and very hard at +1.05 logits). Males have a statistically significantly higher Sports Self-Views than females. The weakest male and female students have been identified for possible intervention for improving their enjoyment and achievement in sports. Females have a statistically significantly higher Drama Self-Views than males. The weakest male and female students have been identified for possible intervention for improving their enjoyment and achievement in Drama. Females have a statistically significantly higher Music Self-Views than males. The weakest male and female students have been identified for possible intervention for improving their enjoyment and achievement in Music.
REFERENCES Allerup, P. (1997). Rasch meassurement theory. In J. P. Keeves (Ed.), Educational Research, Methodology, and Measurement: An International Handbook (2nd ed., pp. 863-874). Cambridge University Press, UK: Elsevier Science Ltd Publishers. Andrich. (1988a). Rasch models for measurement. Paper presented at the Sage university on quantitative applications in the social sciences, series number 07/068, Newbury Park, CA: Sage Publications Andrich, D. (1988b). A general form of Rasch's Extended Logistic Model for partial credit scoring. Applied Measurement in Education, 1(4), 363-378. Andrich, Sheridan, B., & Luo, G. (2005). RUMM: A windows-based item analysis program employing Rasch unidimensional measurement models. Perth:WA: RUMM Laboratory. Armstrong, T. (1994). Multiple Intelligences in the classroom. Alexandria, VA: Association for Supervision and Curriculum Development. Armstrong, T. (2000). Multiple intelligences in the classroom. (2nd ed.). Alexandria, VA: Association for Supervision and Curriculum Development.
Rasch Measures for Sports, Drama and Music Student Self-Views…
45
Bragg, J. (2005). The Effects of Problem-Based Learning on Student Engagement and Motivation. In L. P. McCoy (Ed.), Studies in Teaching 2005 Research Digest. WinstonSalem, NC: Wake Forrest University. Campbell, B., Campbell, L., & Dickinson, D. (1992). Teaching and Learning Through Multiple Intelligences. Australia: Hawker Brownlow Education. Choe, K. C. (2006). Student Engagement with Project Work in a Junior College in Singapore. Unpublished Doctor of Education thesis, Graduate School of Education, University of Western Australia. Christodoulou, J. A. (2009). Applying multiple intelligence: how it matters for schools today, 25 years after its introduction by Howard Gardner. School Administrator, 66(2), 22-25. Cook-Sather, A. (2002). Authorising Students' Perspectives: Toward trust, dialogue, and change in education. Educational Researcher, 31(4), 3-14. Curriculum Framework. (2008). Retrieved 25 March, 2008, from http://www.curriculum. wa.edu.au/pages/framework00.htm Deria, A. (2006a). Australian Islamic College Primary Student Handbook. Perth, WA: Australian Islamic College Deria, A. (2006b). Australian Islamic College Middle School Student Handbook. Perth, WA: Australian Islamic College Deria, A. (2006c). Australian Islamic College Senior School Student Handbook. Perth, WA: Australian Islamic College Edries, A. (2008). Australian Islamic College (Dianella) Annual Report. Perth, WA: The Australian Islamic College. Edries, A. (2009). Personal observation as Principal. Unpublished memo, Australian Islamic College, Perth, WA Furnham, A., Clark, K., & Bailey, K. (1999). Sex differences in estimates of multiple intelligences. European Journal of Personality, 13, 247-259. Furnham, A., & Ward, C. (2001). Sex differences, test experience and self-estimation of multiple intelligence. New Zealand Journal of Psychology, 30, 52-60. Furnham, A., Wytykowska, A., & Petrides, K. V. (2005). Estimates of multiple intelligences: A study of Poland. European Pyschologist, 10, 51-59. Gardner, H. (1983). Frames of Mind: The Theory of Multiple Intelligences (Second ed.). London: Fontana Press. Gardner, H. (1993a). Frames of Mind: The Theory of Multiple Intelligences (2 ed.). London: Fontana Press. Gardner, H. (1999). Multiple Intelligence Theory. Australian Journal Of Education, 43(1), 289. Gardner, H. (2004). Audiences for the Theory of Multiple Intelligences. Teachers College Record, 106(1), 212-220. Gardner, H., Goleman, D., & Csikszentmihalyi, M. (1998). Optimising Intelligences: Thinking, Emotion & Creativity. On Video [Video]: National Professional Resources, Inc. Goodnough, K. (2001). Multiple intelligence theory: a framework for personalising science curricula. School Science and Mathematics, 101(4), 180-193. Grant, C. A., & Tate, W. E. (1995). Multicultural education through the lens of the multicultural education research literature. In J. A. Banks & C. A. M. Banks (Eds.),
46
Ahdielah Edries and Russell F. Waugh
Handbook of Research on Multicultural Education. New York: MacMillan, Harvard University Library. Haig, Y., & Oliver, R. (2007). Waiting in Line: African Refugee Students in Western Australian Schools. Bunbury: WA. Hickey, M. G. (2004). "Can I pick more than one Project?" Case Studies of Five Teachers Who Used Multiple Intelligence-Based Instructional Planning. Teachers College Record, 106(1), 77-86. Jimenez, R. T. (1997). The strategic reading abilities and potential of five low-literacy Latino readers in middle school. Reading Research Quarterly, 32(3), 363-383. Johnson, M. (2007). An Extended Lietarure Review: The Effect of Multiple Intelligences on Elementary Student Performance. Unpublished Master of Science in Education, Dominican University of California, San Rafael, CA. Kornhaber, M. L., Fierros, E., & Veenema, S. (2004a). Multiple Inteligences: Best ideas from theory and practice. Needham Heights: MA: Allyn & Bacon. Leitao, N. C. (2008). Teacher-Student Relationships in Primary Schools in Perth. Unpublished Doctor of Education thesis, Edith Cowan University, Perth. Loori, A. A. (2005). Multiple intelligences: A comparison study between the preferences of males and females. Social Behaviour and Personality, 33(1), 77-88. Luo, G. (2007). The relationship between the Rating Scale and the Partial Credit Models, and the implication for disordered thresholds of Rasch models for polytomous items. In E. V. Smith & R. M. Smith (Eds.), Rasch measurement: Advanced and specialized applications (pp. 181-201). Maple Grove, MN: JAM Press. Magar, A. (2008a). Australian Islamic College Annual School Report. Perth, WA: Australian Islamic College Magar, A. (2008b). Teacher Induction Booklet. Perth, WA: Australian Islamic College Masters, G. N. (1982). A Rasch model for partial credit scoring. Psychometrica, 47, 149-174. Masters, G. N. (1988). The analysis of partial credit scoring. Applied Measurement in Education, 1(4), 279-297. Masters, G. N. (1997). Partial Credit Model. In J. P. Keeves (Ed.), Educational Research, Methodology and Measurement: An International handbook (2nd ed., pp. 857-863). Cambridge, UK: Cambridge University Press. Michell, J. (1990). An introduction to the logic of psychological measurement. Hillsdale, NJ: Lawrence Erlbaum Associates. Michell, J. (1997). Quantitative science and the definition of psychology. British Journal of Psychology, 88, 355-383. Michell, J. (1999). Measurement in psychology: A critical history of a methodoological concept. Cambridge, UK: Cambrige University Press. Miller, J., Mitchell, J., & Brown, J. (2005). African Refugees with interrupted schooling in the highschool mainstream:Dilemmas for teachers. Prospect, 20(2), 19-33. Moran, S., Kornhaber, M., & Gardner, H. (2007). Multiple Intelligences: building active learners. Teacher, 177, 26-30. Nolen, J. L. (2003). Multiple Intelligences in the Classroom. Education (Chula Vista, Calif.), 124(1), 115-119. Park, J., & Niyozor, S. (2008). Madrasa education in South Asia and Southeast Asia: Current issues and debates. Asia Pacific Journal of Education, 28(4), 323-351.
Rasch Measures for Sports, Drama and Music Student Self-Views…
47
Rasch, G. (1960/1980). Probabilistic Models for Intelligence and Attainment Tests. Chicago: IL: MESA Press. Shah, S. (2008). Leading multi-ethnic schools: Adjustments in concepts and practices for engaging with diversity. British Journal of Sociology of Education, 29(5), 523-536. Waterhouse, L. (2006). Multiple Intelligences, the Mozart Effect, and Emotional Intelligence: A critical review. Educational Psychologist, 41(4), 207-225. Waugh, R. F. (2003a). Measuring Attitudes and Behaviours to Studying and Learning for University Students: A Rasch Measurement Model Analysis. Journal of Applied Measurement, 4(2), 164-180. Waugh, R. F. (2003b). On the Forefront of Educational Psychology. New York: Nova Science Publishers. Waugh, R. F. (2005b). Frontiers in Educational Psychology. New York: Nova Science Publishers. Waugh, R. F. (2006). Rasch Measurement. In N. J. Salkind (Ed.), Encyclopedia of Measurement and Statistics (Vol. 3, pp. 820-825). Thousand Oaks, CA: Sage Publications. Whitaker, D. (2002). Multiple intelligences and after-school environments. Nashville: TN: School-Age NOTES. White, J. (1998). Do Howard Gardner's multiple intelligences add up? London: Institute of Education, University of London. Wikipedia. (2009). Theory of multiple intelligences. Retrieved 23 April, 2009, from http://www.en.wikipedia.org/wiki/Theory_of_multiple_intelligences Wright, B. D. (1999). Fundamental Measurement for Psychology. In S. E. Embretson & S. L. Hershberger (Eds.), The New Rules of Measurement: What every psychologist and educator should know (pp. 65-104). Mahwah, NJ: Lawrence Erlbaum Associates.
In: Specialized Rasch Measures… Editor: Russell F. Waugh, pp. 49-64
ISBN: 978-1-61668-032-9 © 2010 Nova Science Publishers, Inc.
Chapter 3
TEACHER GUTTMAN SCALES AND TEACHER VIEWS AT AN ISLAMIC COLLEGE Ahdielah Edries1 and Russell F. Waugh2 1
2
Australian Islamic College Faculty of Education and Arts; Edith Cowan University Mount Lawley; Western Australia.
ABSTRACT A co-educational Independent Australian Islamic College has three campuses which cater for migrant students from war-torn countries and others with culturally and linguistically, diverse backgrounds. This paper is part of a larger study to identify teacher views about the needs of students and the College. This study is important for the Islamic Colleges because it is hoped that the study will lead to improvements to the College and opportunities for Islamic students to have opportunities to achieve better in academic and non-academic areas. Three Guttman scales measured teacher perceptions (N=32) of: (1) Priority Activities Providing Links to the Western Culture; (2) General Types of Resources Needed; and (3) School Needs for Professional Areas. Teachers views were requested and they were analysed to produce eight propositions.
INTRODUCTION The background to this chapter is given in Chapter Two. This chapter explains the analysis of the main needs of students and the College, as perceived by their teachers. Data were collected from three Guttman Scales: (1) three items relating to Priority Activities Providing Links to the Western Culture (see Table 9.1); (2) three items relating to Measuring General Types of Resources Needed to meet the students needs (see Table 9.2); and (3) three items relating to School Needs for Professional Areas (see Table 9.3). Three reliable, but non-linear Guttman Scales (1944, 1950) were created, where the items were arranged in order of difficulty from easy to hard, and the total raw score on these three items arranged from low to high respectively. The open-ended questions answered
50
Ahdielah Edries and Russell F. Waugh
by the teachers that could lead to improvements to be made within the College were analysed by creating some propositions that could be supported by many of the teachers. These were made by abstracting similar comments and then combining them into suitable propositions. A summary of findings is listed at the end of this chapter. While a Rasch measurement model can be used to create a linear unidimensional scale, as was done for the students‟ self-concepts in the previous chapter, large samples (N=250+) are needed to do so. When samples are low, as in the present teacher sample (N=32), the next best measurement model to use is a Guttman Scale which is undimensional but non-linear. This means that, while the scale measures one trait, equal differences between total raw scores on the scale do not represent equal amounts of the trait. Guttman scales are difficult to construct because they require a special structure of ordered items. In the three Guttman scales created for the present study, the items are conceptually ordered from easy (item 1) to hard (item 3). Teachers who answer the hardest item 3 positively are logically expected to answer items two and one positively. Teachers who answer the second item positively (but not the hardest item) are expected to answer item 1 positively. It is often difficult to create this ordered structure of items and, in practice, there are often deviations from the ideal pattern. Guttman (1944, 1950) suggested that up to a 10% error rate could be tolerated so that valid inferences might still be made from the scale.
FIRST GUTTMAN SCALE The first Guttman Scale (see Table 1) measured Priority Activities Providing Links to the Western Culture (N=32). This includes adding Singing, Music, Drama, Cultural Activities, Sports and Games to provide a holistic curriculum and a College environment that will enable the students to integrate into the Australian community more easily. While there was a reasonable fit to the Guttman pattern, this pattern was not ideal. The responses of teachers 1001, 1003 and 1027 did not fit the ideal pattern, but 87 out 96 (90.1%) of the responses did fit the Guttman pattern and this is within the ten percent error limits set by Guttman. Thus, it can be claimed that the scale is unidimensional and reliable so that valid inferences can be drawn from it. It was conceptualised that adding Singing, Music or Drama would be the easiest item to include in the curriculum to improve links to the western culture, but this was still expected to be judged as hard by the teachers. This is because of some restrictions pertaining to music under Islamic teachings and also due to some possible parental discontent given to teachers in some cases by some parents. From Table 1, this item was indeed the easiest (but it is still hard) as only 18 out of 32 teachers supported it in their priority statements, but 12 out of the 32 did not state it as a priority (with some strongly against it), indicating that it is a hard item and its implementation would probably be divisive in the College. It was expected that item 2, adding Cultural Activities, would be harder than item 1 because not all our students are fully integrated into the community (due to their past experiences in other Islamic countries and their limited experience with Australian culture which is quite different from their own) and because they and their families want to „hold onto their Islamic identity and culture‟ to a large extent.
Teacher Guttman Scales and Teacher Views at an Islamic College
51
It was conceptualised that item 3, adding Sports and Games, would be the hardest item because girls have different perceptions about sport than boys in the Islamic culture. This is at least partly due to girls being required to adhere to the Islamic female dress code (that is, wearing the veil, long sleeve shirt and long trousers) and perceived parental disapproval with regard to certain games and sports that would be deemed inappropriate, especially for girls. Table 1 shows that item 3 is harder than item 2 which, in turn, is harder than item 1. Item 3 is very hard as only seven out of 32 teachers (21.8%) stated it as a priority for the College. It should be noted that 12 teachers scored zero on this Guttman Scale. That is, these 12 teachers (37.5% of teaching staff) did not state that any of these three additions listed in this scale were a priority for the College and student improvement. Some of the additions they suggested include extra support in each of the core subjects to improve language comprehension skills to enable students to understand the concrete meaning behind the words, open-ended investigations to be included into programs, providing hands on activities to develop students critical perception and abstract thinking, identify and understand students emotional intelligence (that is, encourage students to be aware of how they feel when they find a piece of work difficult), closer monitoring of individual education plans for students who are gifted and who require educational support, and to develop student intrapersonal skills in the classroom. Table 1. Priority Activities Providing Links to the Western Culture (non-linear scale) Teacher
1031 1032 1014 1022 1007 1004 1013 1027• 1015 1029 1000 1024 1003• 1011 1025 1026 1028 1030 1009 1001• 1006 1023 1021 1020 1019 1018 1017 1016
Add Singing, Music, or drama (item 1, easiest) Y Y Y Y Y Y Y Y Y Y Y Y N Y Y Y Y Y Y N N N N N N N N N
Add Cultural Activities (Item 2, harder) Y Y Y Y Y Y Y N Y Y Y Y Y N N N N N N Y N N N N N N N N
Add Sports & Games (Item 3, hardest) Y Y Y Y Y Y Y Y N N N N N N N N N N N N N N N N N N N N
Total Score (non-linear) 3 3 3 3 3 3 3 2 2 2 2 2 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0
Ahdielah Edries and Russell F. Waugh
52
Table 1. (Continued). Teacher
1012 1010 1002 1008
Add Singing, Music, or drama (item 1, easiest) N N N N
Add Cultural Activities (Item 2, harder) N N N N
Add Sports & Games (Item 3, hardest) N N N N
Total Score (non-linear) 0 0 0 0
Explanatory Note for Table 1 1. All responses were converted into a raw score (that is, Yes = 1 and No = 0) 2. A dot next to the teacher number (1001, 1003 and 1027) indicates that the teacher‟s pattern of responses do not conform to the Guttman pattern. 3. Knowing only a teacher‟s raw total score on the Guttman scale predicts the exact pattern of responses, if there is a good fit to the measurement model.
SECOND GUTTMAN SCALE The second Guttman Scale (see Table 2) measured General Types of Resources Needed (N=29). The responses of four teachers (numbers 1005, 1011, 1029 and 1032 did not fit the ideal Guttman pattern but 83 out of 87 responses (95%) did fit the Guttman pattern, well within the 10% error limit set by Guttman (1950). Hence, it can be claimed that the scale is sufficiently unidimensional and reliable to enable valid inferences to be drawn from it. It was conceptualised that adding IT Learning Centres, Library resources and a Naturalistic Centre would be an easy item for teachers to list on the needs of the College. This is because the College already implements some of these variables to some extent, except for the naturalistic centre which needs some further investigation. A Naturalistic Centre might include developing an outdoor classroom, collecting objects from nature, initiating projects from the food chain and water cycle, researching environmental issues, researching local and global environmental concerns, categorising species of animals and plants, outdoor activities such as camping, hiking, or climbing, setting up sensory skill activities, and showing DVD‟s about nature, Science or animals. From table 9.2, this item was the easiest as 21 out of 29 teachers (72.4%) supported it in their needs statements. It was noted that 8 out of 29 teachers (27.6%) did not make this a priority need. It was conceptualised that item 2, adding Resources for Sports and Physical Activities (Ovals) would be hard. This was because of limited support for this in the Islamic culture (referred to previously) and because of the lack of physical space in the current school grounds. From Table 2, item 2 is harder than item 1 as only 12 out of 29 teachers (41.4%) supported it in their needs statements and 17 out of 29 teachers (58.6%) did not include it their needs statements. It was conceptualised that item 3, adding Extra-Curricular Activities, would be the hardest for a number of reasons. About 75% of the students are taken home by bus at 3.50 pm daily and most parents would not be expected to allow their children to participate in extracurricular activities after normal school hours despite the desirability of these activities being
Teacher Guttman Scales and Teacher Views at an Islamic College
53
supervised at the College. From Table 2, item 3 was harder than item 2 and very hard. Only 7 out of 29 teachers (24%) stated that extra-curricular activities were needed. It should be noted that 4 out of 29 teachers (13.79%) did not list any of these three resources in this scale as a priority for the College and the students. Some of the additions they suggested include developing student interpersonal and intrapersonal skills in the classroom and encouraging students to do more outside activities, exposing students to opportunities to demonstrate their knowledge, teacher punctuality, teacher motivating students, hard but fair discipline, presenting interesting lessons, introducing varied teaching and learning techniques to pique student interests to enhance their learning. These teachers also felt that there should be more parental support and involvement with their children‟s learning, as well as the need for the College to address the educational needs of students who were not performing well academically. Table 2. Measuring General Types of Resources Needed (non-linear scale) Teacher
Add IT Learning centres, Resources, Naturalist, (Item 1, easiest)
1021 1000 1004 1007 1013 1031 1024 1019 1020 1002 1022 1005• 1009 1010 1011• 1012 1014 1015 1016 1017 1023 1026 1028 1029• 1032• 1006 1025 1018 1030
Y Y Y Y Y Y Y Y Y Y Y N Y Y N Y Y Y Y Y Y Y Y N N N N N N
Add Resources – Ovals Physical Activities (Item 2, harder) Y Y Y Y Y Y Y Y Y N N Y N N Y N N N N N N N N Y N N N N N
Add Extra Curricular Activities (Item 3, hardest) Y Y Y Y Y Y N N N N N N N N N N N N N N N N N N Y N N N N
Total Score (non-linear)
3 3 3 3 3 3 2 2 2 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0
Explanatory Note for Table 2 1. All responses were converted into a raw score (that is, Yes = 1 and No = 0)
54
Ahdielah Edries and Russell F. Waugh 2. A dot next to the teacher number (1005, 1011, 1029 and 1032) indicates that the teacher‟s pattern of responses do not conform to the Guttman pattern. 3. Knowing only a teacher‟s raw total score on the Guttman scale predicts the exact pattern of responses, if there is a good fit to the measurement model.
THIRD GUTTMAN SCALE The third Guttman Scale measured School Needs in Professional Areas (N=32). This includes incorporation of Gardner‟s intelligence domains into the curriculum, adding pastoral care and educational support, adding a differential curriculum to cater for individual student needs. The responses of four teachers (numbers 1018, 1002, and 1023), did not fit the ideal Guttman pattern but 90 out of 96 (93.7%) of responses did fit the pattern, well within the 10% error limits set by Guttman (1950). Hence, it can be claimed that the scale is unidimensional and reliable so that valid inferences can be made from it. It was conceptualised that item 1, incorporating Gardner‟s Intelligence Domains into the curriculum would be an easy item for teachers to list on what could be done to meet the needs of their students. This was expected due to the dedication and commitment of present staff to enhancing the learning of their students, to helping students achieve their potential across many subject areas, and by seeking „other‟ ways, consistent with Islamic culture, of helping their students learn and achieve. From Table 3, this item was the easiest as 16 out of 32 teachers (50%) supported this in their statements, but it was still hard. It was noted that 12 out of 32 teachers (37.5%) did not mention this in their statements. So there is only moderate support for implementing item 1 into the school curriculum to improve the academic and social attitudes and achievements of the students. It was conceptualised that item 2, adding Pastoral Care and Educational Support, would be harder than item 1. This means that whilst staff realise the importance of extra educational support for disadvantaged students and the need for greater pastoral care within the College, they perceive a greater need for academic and professional support for all their students. Although the College has policies in place in relation to this aspect, it was expected that most teachers probably believe that more has to done in the academic areas of learning before resources are placed into pastoral care and educational support. It was expected too that staff would want more professional support staff such as a school nurse, a school psychologist and a school dentist. For these reasons, it was expected that adding Pastoral Care and Support, while needed would be of great value to the students at the College, it would have a lower position of need than academic support and professional support, as judged by the teachers. From Table 3, item 2 is harder than item 1 as only 12 out of 32 teachers (37.5%) supported it in their needs statements and 20 out of 32 teachers (62.5%) did not support it in their needs statements. So there is not strong support to implement item 2 aspects at the College. It was conceptualised that item 3, Providing Opportunities for Students and having a Differential Curriculum (that is, a holistic curriculum that comprises of academic and nonacademic subjects) would be the hardest because the College has many practices in place to meet the needs of all its students. However, due to limited finances, limited resources, lack of physical space, lack of support from parents, students prior experiences (and trauma), students academic ability (or lack thereof), it is not always possible to accomplish these goals.
Teacher Guttman Scales and Teacher Views at an Islamic College
55
From Table 3, item 3 is harder than item 2 and very hard. Only 6 out of 32 teachers (18.8%) stated it as a priority for the College. Hence, there is not strong support for implementing item 3 at the College. It should be noted that 12 out of 32 teachers (37.5%) did not list any of these aspects from Scale 3 as a Professional Need for the students and the College. They suggested things such as providing more resources, professional development for ESL in the mainstream, encourage parents to learn along with their own children, improving remedial teaching for literacy and numeracy, improving thematic approaches to learning, providing opportunities for groups with similar intelligences to meet on a regular basis and focus on their strengths to boost their learning, helping staff to develop whole school policies to cover all eights areas of learning, adopting a more practical approach to teaching - not just pencil and paper, but restructuring lessons to incorporate learning through investigation rather than content based. Table 3. School Needs in Professional Areas (non-linear scale) Teacher
1016 1017 1020 1032 1014 1018• 1019 1022 1028 1005 1000 1003 1002• 1004 1006 1007 1008 1009 1010 1023• 1021 1024 1025 1026 1029 1031 1027 1001 1015 1030 1011 1012
Incorporating Gardner‟s Intelligence Domains in Academic Work (item 1, easiest) Y Y Y Y N N Y Y Y Y Y Y N Y Y Y Y Y Y N N N N N N N N N N N N N
Adding Pastoral Care and Support
(Item 2, harder) Y Y Y Y Y Y Y Y Y Y N N Y N N N N N N Y N N N N N N N N N N N N
Adding a Differential Curriculum to Cater for Individual Needs (Item 3, hardest) Y Y Y Y Y Y N N N N N N N N N N N N N N N N N N N N N N N N N N
Total Score (non-linear)
3 3 3 3 2 2 2 2 2 2 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0
56
Ahdielah Edries and Russell F. Waugh
Explanatory Note for Table 3 1. All responses were converted into a raw score (that is, Yes = 1 and No = 0) 2. A dot next to the teacher number (1002, 1018, and 1023) indicates that 3. the teacher‟s pattern of responses do not conform to the Guttman pattern. 4. Knowing only a teacher‟s raw total score on the Guttman scale predicts the exact pattern of responses, if there is a good fit to the measurement model.
PROPOSITIONS FROM THE TEACHER OPEN-ENDED QUESTIONS Teachers‟ responses to the open-ended questions were read and similar comments were placed in similar groups under appropriate headings. Thereafter, similar comments were reread to check that they had been placed in the correct group together under the correct headings. The comments were checked again and the following eight propositions were created from the teachers‟ comments now under common, appropriate headings. Some partquotations supporting the eight propositions are given below with a number referring to the particular teacher (not identified by name for ethical reasons). The comments and propositions were meant to add further information in relation to issues measured in the Guttman scales.
PROPOSITION 1 While it may seem reasonable to introduce singing, music or drama into the College as part of a way to forge some links between the Islamic students and the other Australian students, it is a potentially divisive issue. If singing, music or drama were to be introduced at the College, it would have to be done in a sensitive way, probably in very small stages after extensive consultation with staff and parents. Adding singing, music or drama to the curriculum did not have universal support with the teachers as they are aware that music is not well supported in the Islamic culture, hence, they do not see this addition as a priority for the College. Remarks from staff who support adding of singing, music or drama are evident in the following comments: The two main domains that I believe are not being met within my classroom are the musical domain and the naturalist domain. While they have been met partially, they can certainly be improved on. (1028) Music and associated learning areas e.g. drama and dance. Improve Interpersonal relations particularly with other cultural groups and experiences. Be more flexible in areas of music and literature provided to students and adopt the multicultural Australian way of life. (1031) Put music and dance in the curriculum. Singing is great for health and mood. Help students to become independent thinkers that are able to solve their own problems. (1022) We do not use and practice music as part of our curriculum. Offer music in our curriculum and or make music an option for students and parents to select. (1011)
Teacher Guttman Scales and Teacher Views at an Islamic College
57
PROPOSITION 2 The majority of staff agreed that the College already had some practices in place to cater for the non-academic needs of their students. The practices they referred to include sport, art, gardening, excursions to recreational places, the supportive pastoral care within the College, community service participation, and inter-faith school visits. However, some teachers seem to be suggesting that more could be done to meet the non-academic needs of the students in the area of activities, games and sports to provide better links with the western culture. Others, for example, suggest helping students to become independent learners and thinkers, teaching students about life skills and issues, and introducing extra-curricular activities to enrich the students‟ lives. If extra activities, games and sports were to be introduced at the College, this would have to done in a sensitive way, probably in small stages after extensive consultation with staff and parents. Adding games and sports did not have universal support amongst the teachers and many did not see it as a priority despite games and sports being a strong priority in Australian culture. The non-academic needs that the majority of staff deem necessary to meet the needs of their students are outlined in the following comments: Incorporate more practical hands on learning areas e.g. vet subjects and not focus on pure academic areas (sic). Be more flexible in areas of music and literature provided to students and adopt the multicultural Australian way of life. Be more emphasis on structured sport in school (sic). Incorporate and integrate slowly into the learning areas. Engage students in more puzzles and strategy games where student use logic, problem solving and critical thinking. (1031) I believe that we as a school do our best to meet the needs of the students. However areas which we could improve on are the following; more computers that are faster, bigger, library with a variety of books for students to read and use for projects, cleaner canteen with a variety of healthy foods, more sports equipment and a oval so children can play freely and get some exercise. (1020) Closer monitoring of individual education plans for students who are gifted and remedial. - More technology used inside and outside of the classroom. - Provide a more rounded education. Emphasize importance of the 4 secular areas but provide more opportunities for learning in the areas of health, sport, art, incursions and excursions. More community involvement with those from a non-Muslim background. (1012) More group work - allows those with high interpersonal skills to shine, continued use of pictorial representation and drama to explain concepts and vocabulary. Greater use of games in learning followed up by written work to allow for differing skill/intelligence. Provide a more open-learning environment with a smaller emphasis on strict use of text books and a cross curricular approach to planning. (1008)
PROPOSITION 3 Whilst some teachers acknowledged the importance of pastoral care and educational support for students within the College, they perceived a greater need to enhance the academic and professional support for their students. The needs to which the teachers are referring include improving academic areas of learning for all students (that is, enhancing the learning of their students, to helping students achieve their potential across many subject areas, and by seeking „other‟ ways, consistent with Islamic culture, of helping their students
58
Ahdielah Edries and Russell F. Waugh
learn and achieve) and to provide more professional support staff such as a school nurse, a school psychologist and a school dentist. These teachers were suggesting things such as professional development for teachers to enable them to develop whole school policies to cover all eight areas of learning, more focus on remedial teaching in literacy and numeracy, provision of opportunities for groups with similar intelligences to meet on a regular basis and focus on their strengths to boost their learning, adopting a more practical approach to teaching and learning. It was acknowledged that implementation in the College of the suggestions would require extra resources, specialist teachers, ovals, adding learning centres, more co-operative learning in classes, enhancing IT within the College, reviewing the teaching and learning methodologies of staff and students respectively. However, whilst all these are interesting suggestions, the College would not be able to accommodate some of these suggestions due to financial difficulties as a result of most of the parents paying very little or no fees. Teachers provided the following comments: The existing learning programme has to be modified. We tend to teach from the syllabus only which excludes real life situations. We should get more professionals to visit students with special needs at school to assess their educational needs and assist teachers in developing programmes. We should promote differential learning, offer more subjects. (1010) Provide more opportunities for logical/mathematical students to compete in interschool events-designing, creating solutions for scientific/mathematical problems through some fun activities interactive, free from tests). Provide after-school specialists for various student interests, such as, drama media school's newsletter created by talented students, art, other languages, playing instruments, sports and photography. (1017) We need to cater more for the spatial/body/kinesthetic intelligence domain student. Our linguistic/logical centered approach caters for a select few. We need to extend our topics across all learning areas. (1013) Closer monitoring of individual education plans for students who are gifted and remedial. More technology used inside and outside of the classroom. Provide a more rounded education. Emphasize importance of the four secular areas but provide more opportunities for learning in the areas of health, sport, art, incursions and excursions. More community involvement with those from a non-Muslim background. (1012) Fundraise to better facilitate and resource catering for a variety of intelligences….. (1000) I feel the school can provide the necessary needs by buying equipment for the above discussed. Funds should be raised to cater for all this. (1016) Provide better resources for the areas of creative and practical arts…. (1014) Provide more resources, PD in ESL in the mainstream (1021) …. Pastoral care for students who have school and home life issues e.g. school chaplain in public schools…. (1012)
Teacher Guttman Scales and Teacher Views at an Islamic College
59
PROPOSITION 4 The staff was divided with regard to whether extra-curricular activities should be added to the current academic curriculum. While there was some support for this, there was some comment against introducing any extra-curricula activities, due to extenuating factors such as, the length of the school day, student transport, parental disapproval and students‟ unwillingness to participate in certain activities, limited access to resources and finances to run these activities. In contrast with this, some teachers suggested adding academic-type activities such as introducing practical hands-on activities, puzzles and games to encourage logical thinking, activities that tap into students individual strengths and interests, more sophisticated IT, better resources to encourage creativity, activities to promote the inclusiveness of the eight intelligences to enhance the current academic curriculum. The following quotes by staff highlight their relevant perceptions. Provide better resources for the areas of creative and practical arts. Modify the timetable to ensure more time is provides for physical activity. Vastly enhance the computer network throughout the college and provide programs that improve the learning environment of the students. (1014) I feel the need for specialist teachers in the areas of art, drama and sport rather than P.E (sic). More funds needs to be placed in the areas mentioned previously e.g. Bodily/kinesthetic (sic), musical and naturalist intelligence. (1015) Provide more opportunities for logical/mathematical students to compete in interschool events-designing, creating solutions for scientific/mathematical problems through some fun activities interactive, free from tests). Provide after-school specialists for various interests drama media school's newsletter created by talented students, art, other languages, playing instruments, sports and photography. (1017) In subjects such as math/science in order to improve understanding and develop skills I can apply in my lesson strategies such as; Body/kinesthetic - Science; students to enact kinetic theory of matter, structure of the action, planets and space - model. More (sic) skills re-inforcement in the science laboratory. Math Algebra - enact shapes, estimate distance. (1006) I believe that we as a school do our best to meet the needs of the students. However areas which we could improve on are the following; more computers that are faster, bigger, library with a variety of books for students to read and use for project, cleaner canteen with a variety of healthy foods, more sports equipment and a oval so children can play freely and get some exercise. (1020)
PROPOSITION 5 Adding IT Learning Centres or library resources to enhance the learning and cater for the needs of students was very strongly supported by 72% of the staff. It was stated by some teachers that the College was already effectively catering for some students‟ needs related to these areas, however, they felt that the College could do more with better resources, such as, for example, better IT facilities and Learning Centres, more „hands-on‟ activities to draw on
60
Ahdielah Edries and Russell F. Waugh
student prior learning experiences and to make their learning more practical. It was also suggested that students should be exposed to opportunities to demonstrate their knowledge, and to pique their interests. These suggestions are described by the following quotes. Provide better resources for the areas of creative and practical arts. Modify the timetable to ensure more time is provided for physical activity. Vastly enhance the computer network throughout the College and provide programs that improve the learning environment of the students. (1014) Extra support in English; - More focus on problem solving;- Open ended investigations to be included into programs;- improve facilities and IT support. (1006) I believe that we need to encompass much more hands on learning in order to cater for those children who struggle with written work. To do this we also need space to set up learning centers (sic). Maybe if we do get the new mosque/building, the old mosque can be put to good use. By making use of thematic learning we can look at creating a whole school curriculum that builds on previous learning. (1008) Students are lacking in critical perception, abstract thinking and inability to operate as an independent learner mainly because they do not have hands on experiences. Most of the time (sic) its teacher- student traditional teaching and learning. (1010) I believe that we as a school do our best to meet the needs of the students. However areas which we could improve on are the following; more computers that are faster, bigger, library with a variety of books for students to read and use for project, cleaner canteen with a variety of healthy foods, more sports equipment and a oval so children can play freely and get some exercise. (1023) Provide more resources, PD in ESL in the mainstream, encourage parents to learn along with their own children. (1024)
PROPOSITION 6 Some teachers suggested the introduction of a Naturalist Centre (which could be very meaningful for the students especially taking their backgrounds and prior learning experiences into consideration). A Naturalist Centre could be good for the students because it could tap into their strengths (without the need to show their lack of academic ability). It can also enhance their undiscovered strengths, and provide students the opportunity to develop naturally by being involved with practical „hands-on‟ learning in a non-intrusive environment. A Naturalist Centre could involve developing an outdoor classroom, collecting animals, plants and rocks from nature, growing plants and vegetables, caring for animals, researching local and global environmental issues and concerns, and categorising species of animals and plants. Linked to this could be outdoor activities such as camping and hiking, setting up sensory skill activities, and showing DVD‟s about nature, Science and animals. Naturalist: We could have a little garden patch with chicken grow vegetables (sic) to cater for children that display naturalist intelligence. (1017)
Teacher Guttman Scales and Teacher Views at an Islamic College
61
I feel that the students‟ spatial awareness is lacking and bodily/ kinesthetic experiences. Naturalist activities such as cooking and recycling needs must be integrated in school life. (1024) ….. Naturalist domain - more emphasis in this area particularly forming or joining youth groups, scouts, brownies and girl guides. (1031) Do more naturalistic hands on activities such as caring for animals, worms and composting. (1005) Plan more tactile activities, more building, creating and designing of physical materials and projects (sic). Hands on activities (sic). Space is an issue but the year 7 garden is excellent it would be great to get the kids involved in planting and tending plants. I would love to do cooking with my kids.... (1025) …. Naturalist domain - more emphasis in this area particularly forming or joining youth groups, scouts, brownies and girl guides. (1034)
PROPOSITION 7 Incorporating Gardner‟s Intelligence Domains into the curriculum was strongly supported by teachers on what could be done to meet the needs of their students. This support was primarily due to staff wanting to explore other avenues to enhance the learning of their students, to helping students achieve their potential across many subject areas, consistent with Islamic culture, of helping their students learn and achieve. Staff suggested improving the academic and social attitudes and achievements of their students by introducing activities linked to the eight intelligences, adopting thematic approaches to make students‟ learning more meaningful, incorporating varied learning techniques to enhance students interest to attain their full potential, providing opportunities for deep critical thinking, investigative open-ended tasks, and to plan more inclusive projects for students to stimulate their learning. These ideas are described below: I can use the ideas of the seven intelligences to improve the abilities of my students through using more open-ended tasks and through allowing students to compete a task using a variety of methods which is best suited to their intelligence. (1028) I believe that in my class I already use most of the 7 intelligences with my students. However there are a few of the intelligences that I would like to put into practice if I was given the opportunity in order to improve our students learning. I would like the opportunity to teach my students a musical instrument. It would be useful to take students on camp to learn more about the environment and learn how to live /survive in the bush. I would also like to provide my students with the opportunity to act in plays to boost their confidence especially students who are very shy. I would like to give my students the opportunity to take part in building models, more hands on activities to improve themselves and boost their confidence. (1020) Plan more tactile activities, more building, creating and designing of physical materials and projects (sic). More hands on activities. Space is an issue but the year 7 garden is excellent it would be great to get the kids involved in planting and tending to
62
Ahdielah Edries and Russell F. Waugh plants. I would love to do cooking with my kids. Drama and public speaking could be greater valued and respected, this helps confidence and improves the presentation. (1022) Program in a holistic way (that is, using a thematic approach). Divide children according to their strengths (within a year level) across say 3 intelligences then provide opportunities for the groups to come together on a regular basis and focus on their strengths to boost learning. (1021) In subjects such as math/science in order to improve understanding and develop skills I can apply in my lesson strategies such as; Body/kinesthetic - Science; students to enact kinetic theory of matter, structure of the action, planets and space - model. Skills reinforcement in using science lab. Math Algebra; enact shapes, estimate distance. Interpersonal skills and linguistic. (1006)
PROPOSITION 8 While it may seem natural that the College should provide reasonable physical space for the students there was little support by the teachers to add an oval as a necessary resource. This was primarily due to the lack of physical space within the College (already having some inadequate sport and play areas), limited finances to upgrade, need for better resources, and the perception by staff, parents and the community that there were „other student needs‟ that should be addressed with more priority, such as developing student interpersonal and intrapersonal skills in the classroom, exposing students to opportunities to demonstrate their knowledge, introducing varied teaching and learning techniques to pique student interests and enhance their learning, greater parental support and involvement with their children‟s learning, improving literacy and numeracy, and addressing the educational needs of students who were not performing well academically, that could be of more benefit to the students. Remedial teaching for literacy and numeracy with a strict timetable. More resources to help slow learners. More(sic) extra curricular activities to increase focusing and to remain on task. (1030) Intrapersonal - for the Islamic student, deep thinking is encouraged and could be a great advantage to successful learning and also to reflect conscientiously in achieving their learning goals across the curriculum framework. Students need to develop their faculty of 'deep thinking'. Different ways of learning gives a better chance of understanding. Students are given the choice as to which of the seven intelligences they wish to undertake. Students think about what they can improve. (1019) Playground - An oval to run and play on - Specialist teachers; Art to teach subject as it should be taught. (1007) Teach students about the reality of working and their future. Real issues about what they will face in the future - taxes, salaries and job options so that they have a better idea of what they want to do and work towards a goal. Motivation and confidence to study. I have spoken to many students and they have all expressed their interest in this area. Also, the power of reflection, letting children keep a diary to record reflections to better themselves and learn from their mistakes. (1023)
Teacher Guttman Scales and Teacher Views at an Islamic College
63
I believe that there is a significant number of students suffering from lack of parenting. We need to work with the parents to maximize student learning. … Students‟ emotional issues (particularly in upper primary/high school) need to be addressed. (1018)
SUMMARY OF FINDINGS Data were collected from thirty-two teachers to ascertain their views about the College meeting the needs of its students. The data involved two parts; (1) three Guttman Scales and (2) answers to open-ended questions from which eight propositions were created. In part 1, three Guttman Scales were created for three items measuring „Priority Activities Providing Links to the Western Culture‟, three items measuring General Types of Resources needed to meet the students‟ needs, and three items measuring School Needs in Professional Areas. The items were conceptually arranged in order of difficulty from easy to hard, and the total raw score on its three items arranged from low to high measured the three variables in a unidimensional, but non-linear scale. All three scales had acceptable teacher item-response patterns. In part 2, the eight propositions were created by collating similar comments under appropriate headings and then creating propositions that reflected the teacher comments. Following this, the original teacher data were re-read to check that the data supported the eight propositions. As a summary, nine main inferences can be drawn from the data. 1. Adding singing, music or drama to the curriculum was supported by 59% of teachers in their priority statements. However, 41% teachers did not see it as a priority, as there could be some adverse implications stemming from this that would be divisive in the College. 2. Adding Cultural Activities to the curriculum was considered to be hard, as some of the teachers expressed concern that there may be some cultural barriers that would need to be addressed first before implementing any activities. 3. Only 24% of teachers stated that adding sports and games to the curriculum was a priority for the College. This was due to some gender issues that would need to addressed with sensitivity and consultation, particularly with regard to the female students. 4. Adding IT Learning Centres, Library Resources and a Naturalist Centre was well supported as the College already implements (at least partially) most of these variables, but the Naturalist domain needs to be further investigated. 5. Due to lack of physical space within the College grounds, and a „limited‟ sports program currently being offered within the College, only 41.4% supported Adding Resources for Sports and Physical Activities (Ovals) as resources that are needed within the College to meet the needs of the students. 6. There was a unanimous agreement amongst staff that adding Extra-Curricular Activities, whilst necessary, would not be able to be incorporated into the school curriculum as most parents would not agree to allow their children to participate due to the College‟s long hours and most students needing the school transport after school.
64
Ahdielah Edries and Russell F. Waugh 7. There was strong support to incorporate Gardner‟s Intelligence Domains into the curriculum due to the dedication and commitment of the staff to enhance and optimise the learning of their students. 8. Although the College has Policies in place, as well as excellent staff members who strive to meet the needs and well-being of all their students, most staff believe that more has to be done to meet the needs of the students. 9. Most staff acknowledged that they strive to meet the needs of all their students, however, due to limited finances, limited resources, lack of physical space, lack of support from parents, students prior experiences (and trauma), students academic ability (or lack thereof), it is not always possible to accomplish goals that they would like to for their students.
In: Specialized Rasch Measures… Editor: Russell F. Waugh, pp. 65-81
ISBN: 978-1-61668-032-9 © 2010 Nova Science Publishers, Inc.
Chapter 4
RASCH MEASURES OF FORM CONSTANCY OF LETTERS AND NUMBERS, AND LETTERS IN WORDS FOR YOUNG CHILDREN Janet Richmond, Russell F. Waugh and Deslea Konza Faculty of Education and Arts, Edith Cowan University, Perth, Western Australia
ABSTRACT English and number literacy are very important topics and the Australian Government runs numeracy and literacy tests, administered through the State Education Departments, for all Year 3 (8 years old), Year 5 (10 years old) and Year 7 (12 years old) students. Results of these tests are reported to schools and parents with a view to ensuring that all children meet certain literacy standards and that children who are „falling behind‟ are detected early so that remedial work can be given. Rasch measures were then created with the RUMM2020 computer program for visual discrimination regarding Form Constancy of Letters and Numbers (FCL&N) and Letters in Words (LinW). The student sample was N=324 pre-primary and primary students in Perth, Western Australia, aged 49 years old. Data on 24 items for FCL&N and 75 items for LinW, where each item was scored in one of two categories (wrong scored zero and correct scored one), were Rasch analysed to create two linear scales. Six of the initial 24 items for FCL&N were deleted due to item misfit statistics, leaving 18 items. Forty-one of the original 75 items for LinW were deleted due to item misfit, leaving 34 items. The final data for FCL&N and LinW were used to create two highly reliable, linear, uni-dimensional scales (Student Separation Indices of 0.94 and 0.97 and Cronbach Alphas of 0.94 and 0.97, respectively) where the items are ordered from easy to hard and the student measures from low to high on the same scale. The two scales showed no statistically significant interaction of student measures on item difficulties along the scale, meaning that there was good agreement about the item difficulties along each scale, and each scale was unidimensional. The item-trait chi-squares are respectively, χ² = 69.69, df=54, p=0.07, and χ² = 117.59, df=102, p=0.14. The fit residual statistics for each of the two scales was reasonable and the targeting was reasonable.
66
Janet Richmond, Russell F. Waugh and Deslea Konza
INTRODUCTION This report presents a Rasch analysis with the RUMM 2020 computer program (Andrich, Sheridan & Luo, 2005) in which two linear, unidimensional scales were created: (1) Form Constancy of Letters and Numbers, and (2) Letters in Words. These two scales relate to visual perceptual concepts of „form constancy‟ and „figure ground‟. This report describes the measurement results in terms of Rasch measurement fit statistics including global item and person fit to the measurement model, dimensionality, person separation indices, distribution of item-person interactions, and discrimination. Some discussion is included of the non-fitting items, as well as good fitting items, and the person-item threshold distribution (targeting). This is followed by mean Rasch measures by group and final items for the Form Constancy and Figure Ground Scales discussion. Finally, inferences drawn from the linear Rasch measurement data analysis and the summary of the results are presented.
INITIAL RASCH ANALYSIS An initial Rasch analysis was performed on the original items for Form Constancy of Letters and Numbers (24 items) and Letters in Words (41 items) where each item was scored in one of two categories (incorrect answer scored zero and correct answer scored one). Six of the initial 24 items of Form Constancy of Letters and Numbers were deleted due to item misfit statistics. The remaining 18 items were found to have an excellent fit to the measurement model for the 324 persons included in this study. For Letters in Words, seven of the initial 41 items were deleted due to item misfit statistics. The remaining 34 items displayed an excellent fit to the measurement model. The Rasch analysis with the RUMM program does not indicate how to alter an item in order to make it fit the measurement model. In order to include, in a future measure, the deleted items which were initially considered conceptually valid, would need to be changed and re-tested.
FINAL RASCH ANALYSIS RESULTS The following material shows the results for the final Rasch analysis for the three scales: (1) Form Constancy of Letters and Numbers (18 items), and (2) the Figure Ground Scales of Letters in Words (34 items).
Summary of Fit Statistics The RUMM2020 program estimates an item-person interaction which establishes the overall fit statistics that determine whether the item estimations contribute meaningfully to the measurement of one construct. This calculation thus examines the consistency with which students responses agree with the calculated difficulty of each item on the scale. The standardised fit residual statistics (see Table 1) have a distribution with a mean near zero and
Rasch Measures of Form Constancy of Letters and Numbers…
67
a standard deviation near one when the data fit the measurement model, as is the case with these three measures. This means too that there is a good pattern of person and item responses consistent with a Rasch measurement model.
Dimensionality For Form Constancy of Letters and Numbers, there was an item-trait interaction chisquare of 69.69 with df=0.94 and a probability of 0.07. This means that the scale is constructed with acceptable, but not ideal, agreement amongst the students about the linear progressive difficulty of the items. The item-trait interaction chi-square for Letters in Words was 117.59 with df=0.97 and a probability of 0.14, showing a similar acceptable agreement amongst the students about the linear progressive difficulty of the items along the scale. This means that the students agree as to which items are easy, which are of medium difficulty and which are hardest. Table 1. Global Item and Student Fit Residual Statistics (N=324) ITEMS Location Fit Residual Form Constancy of Letters and Numbers (I=18) Mean 0.00 -0.45 Standard Dev. 0.65 0.89 Letters in Words (I=34) Mean 0.00 -0.68 Standard Dev. 0.82 1.11 Numbers in Calculations (I=15) Mean 0.00 -0.35 Standard Dev. 0.59 0.04
PERSONS Location
Fit Residual
+1.97 2.06
-0.20 0.75
+2.00 2.56
-0.43 1.25
+1.29 2.11
-0.08 0.84
Comment on Table 1: Fit residuals have a mean near zero and a standard deviation near one when the data fit the measurement model (as is the case here). This reflects good consistency of item and student scoring patterns.
Person Separation Index The Person Separation Index is an estimate of the true score variance among the students and the estimated observed score variance using the estimates of their ability measures and the standard error of these measures (Andrich & van Schoubroeck, 1989). For Form Constancy of Letters and Numbers and Letters in Words, the Person Separation Indices are 0.94, and 0.97 respectfully. For a good measure, it is desirable that this index should be 0.9 or greater, as it is an indicator that the student measures are separated by more than their standard errors. Based on this index, the Form Constancy of Letters and Numbers and Letters in Words scales demonstrate very good separation of measures in comparison to the errors of measurement.
Janet Richmond, Russell F. Waugh and Deslea Konza
68
Individual Item Fit Items are ordered by calibrated values to evaluate their fit to the measurement model. The location of each item on the scale is the item difficulty in standard units, called logits (log odds of answering successfully). All the items in Form Constancy of Letters and Numbers fit the measurement model with probabilities greater than p=0.03 (see Table 2). The residuals shown in Table 2 represent the difference between the observed responses and the expected responses calculated from the Rasch measurement parameters. Standardised residuals should fall within the range of -2 and +2. Table 2 shows that all items for Form Constancy of Letters and Numbers have acceptable residuals except for item 14. For Figure Ground Letters in Words, all the items fit the measurement model with probabilities greater than p=0.06 (see Table 3), but a few of the residuals are a little outside what might be considered good limits. Table 2. Individual Item Fit Statistics for Form Constancy of Letters and Numbers Item
Location
SE
Residual
DegFree
ChiSq
DegFree
Prob
18
-0.93
0.26
0.49
143.56
9.11
3
0.03
1
-0.72
0.25
-0.61
143.56
1.35
3
0.72
23
-0.70
0.25
+0.08
143.56
4.50
3
0.21
21
-0.63
0.24
-0.36
143.56
1.31
3
0.73
19
-0.50
0.24
+0.86
143.56
4.53
3
0.21
20
-0.40
0.23
+0.74
143.56
6.23
3
0.10
5
-0.33
0.23
-0.08
143.56
0.77
3
0.86
2
-0.33
0.23
-0.67
143.56
1.97
3
0.58
3
-0.26
0.23
-0.49
143.56
5.09
3
0.17
8
-0.12
0.23
-1.01
143.56
8.32
3
0.04
14
+0.10
0.22
-2.41
143.56
6.24
3
0.10
13
+0.28
0.21
-0.71
143.56
7.18
3
0.07
16
+0.29
0.21
-1.65
143.56
3.81
3
0.28
11
+0.51
0.21
+0.22
143.56
1.42
3
0.70
17
+0.70
0.20
-1.46
143.56
2.13
3
0.55
9
+0.71
0.20
+0.26
143.56
0.98
3
0.81
7
+0.93
0.20
-1.58
143.56
3.43
3
0.33
-0.03
143.56
1.32
3
0.70
4
+1.39
0.19
Notes on Table 2 and 3: 1. Location refers to the difficulty of the item on the linear scale. 2. SE means Standard Error, and refers to the degree of uncertainty in a value. 3. Residual represents the difference between the expected value of an item, calculated according to the Rash measurement model and the actual value. 4. DegFree stands for degrees of freedom, and refers to the number of scores in a distribution that are free to change without changing the mean distribution. 5. ChSq stands for Chi-square 6. Prob relates to the probability based on the Chi-square and refers to the levels of certainty to which an item fits the measurement model.
69
Rasch Measures of Form Constancy of Letters and Numbers… Table 3. Individual Item Fit Statistics for Figure Ground Letters in Words Item
Location
SE
Residual
DegFree
ChiSq
DegFree
Prob
11 13 26 9 3 15 23 6 27 12 19 5 20 7 8 25 10 16 18 29 17 14 30 22 24 31 41 33 28 36 40 35 38 34
-1.16 -1.13 -1.13 -1.07 -0.98 -0.83 -0.81 -0.78 -0.71 -0.68 -0.65 -0.61 -0.55 -0.53 -0.41 -0.38 -0.22 -0.02 +0.17 +0.30 +0.36 +0.43 +0.68 +0.69 +0.70 +0.84 +0.86 +0.88 +0.96 +0.99 +1.08 +1.17 +1.19 +1.36
0.27 0.27 0.27 0.27 0.26 0.25 0.25 0.25 0.24 0.24 0.24 0.24 0.23 0.23 0.23 0.23 0.22 0.21 0.20 0.20 0.20 0.20 0.19 0.19 0.19 0.19 0.19 0.19 0.18 0.18 0.18 0.18 0.18 0.18
-0.68 -0.57 -0.59 -1.18 -0.23 -1.11 -1.96 +0.69 +0.31 -1.52 -2.12 -1.67 -1.71 -0.58 -1.42 -0.88 -0.69 -0.49 -1.56 -0.57 0.55 -0.59 -0.22 +1.98 -0.33 +0.75 -1.03 -2.31 +0.65 -2.40 +0.09 -0.53 -2.13 +1.97
176.65 176.65 176.65 176.65 176.65 176.65 176.65 176.65 176.65 176.65 176.65 176.65 176.65 176.65 176.65 176.65 176.65 176.65 176.65 176.65 176.65 176.65 176.65 176.65 176.65 176.65 176.65 176.65 176.65 176.65 176.65 176.65 176.65 176.65
3.20 4.06 5.23 2.09 1.17 3.78 3.00 2.43 2.85 3.29 4.92 2.24 4.24 3.18 3.13 2.65 6.37 6.68 5.05 0.82 3.00 0.57 4.64 7.48 2.33 4.87 2.19 3.25 0.72 6.82 0.86 3.65 3.34 3.52
3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3
0.36 0.25 0.16 0.55 0.76 0.29 0.39 0.49 0.41 0.35 0.18 0.52 0.24 0.37 0.37 0.45 0.09 0.08 0.17 0.84 0.39 0.90 0.20 0.06 0.51 0.18 0.53 0.36 0.87 0.08 0.84 0.30 0.34 0.32
Targeting The RUMM2020 program produces a student-measure item-difficulty or targeting graph on which the student measures are placed on the same scale as the item difficulties in standard units called logits. For Form Constancy of Letters and Numbers (see Figure 1), this targeting graph shows that the student measures cover a range of about -3.5 to +3.5 logits and the item difficulties cover a range of about -1.0 to +1.4 logits. From the graph it can be seen that many students (about 245) were able to answer the items correctly, while about 30 students were unable to answer any of these items correctly. This indicates that the targeting of the items needs to be improved in any future use of the scale by adding in some easier and more difficult items to „cover‟ the students with the lowest and highest measures.
70
Janet Richmond, Russell F. Waugh and Deslea Konza
Note: Student measures are on the upper side in logits. Item difficulties are on the lower side of the same scale in logits. Many students (about 245) answered the items correctly. Figure 1. Targeting Graph for Form Constancy of Letters and Numbers.
Note: Student measures are on the upper side in logits. Item difficulties are on the lower side of the same scale in logits. Many students (about 175) answered the items correctly. Figure 2. Targeting Graph for Figure Ground Letters in Words.
For Figure Ground Letters in Words (see Figure 2), the targeting graph shows that the student measures cover a range of about -4.4 to +4.3 logits and the item difficulties cover a range of about -1.2 to +1.4 logits. From the graph it can be seen that many students (about 205) were able to answer the items correctly, while about 45 students were unable to answer these items. This indicates that the targeting of the items needs to be improved in any future use of the scale by adding in some easier and more difficult items to „cover‟ the students with the lower and higher measures.
Rasch Measures of Form Constancy of Letters and Numbers…
71
Discrimination Item Characteristic Curves examine the relationship between the expected response and the mean group student measures. These curves display how well the item discriminates between groups of persons. An example of one item characteristic curve for each of the three constructs will be presented. Figure 3 shows the Item Characteristic Curve for Item 1 Form Constancy of Letters and Numbers. This curve shows that the item discriminates well for students with different measures. The Item Characteristic Curves for all the other items were checked and found to be satisfactory (but are not reported here to avoid unnecessary repetition).
Figure 3. Item Characteristic Curve: Item 1 – Form Constancy of Letters and Numbers.
Figure 4. Item Characteristic Curve: Item 16 –Figure Ground of Letters in Words.
72
Janet Richmond, Russell F. Waugh and Deslea Konza
Figures 4 shows the Item Characteristic Curves for Item 16 of Figure Ground of Letters in Words. This item discriminates well for students with different measures. The Item Characteristic Curves for all the other items in both measures were checked and found to be satisfactory (but are not reported here to avoid unnecessary repetition).
Consistency of Use of Scoring Categories The RUMM2020 program produces graphs of the scoring categories for each item. The Scoring Category Curves show the relationship between the probability of scoring in each category (zero for incorrect answer and one for correct answer) on each item. Figure 5 is the Scoring Category Curve for item 1 of Form Constancy of Letters and Numbers. This figure shows that the scoring was done logically and consistently. When students have low measures on item 1, then they have a high probability of obtaining a zero score (answer incorrect) and, when they have a high measure, they have a high probability of scoring 1 (answer correct). The Scoring Category Curves for all the other items were checked and they were satisfactory too. The Scoring Category Curves for all the items of the other variable, Figure Ground Letters in Words, were checked and they were also found to be satisfactory, but they are not presented here to avoid repetition.
Figure 5. Scoring Category Curve: Item1 – Form Constancy of Letters and Numbers.
CHARACTERISTICS OF THE SAMPLE (FCLN, FGLIW) The measures for Form Constancy of Letters and Numbers (FCLN) were displayed in a graphical format separated by gender (Figure 6), type of school (Figure 7), age (Figure 8), grade (Figure 9) and whether intervention had been received (Figure 10). Females have a higher mean measure than males for Form Constancy of Letters and Numbers but this is not statistically, significantly different (t=0.76, df=321, p=0.25). Public school students have a higher mean measure than private school students for Form Constancy of Letters and
Rasch Measures of Form Constancy of Letters and Numbers…
73
Numbers but this is not statistically, significantly different (t=0.93, df=321, p=0.18). As would be expected, the mean measures generally increased by age from four years of age (lowest) to nine years of age (highest) and this was statistically, significantly different (t=7.9, df=65, p=0.000). Again, as expected, the mean measures generally increased by grade from Pre-primary (lowest) to Year 3 (highest) and this was statistically, significantly different (t=12.0, df=126, p=0.000). The mean measures for intervention/ no intervention was not statistically, significantly different (t=0.88, df=321, p=0.20).
Note: There is a colour error in the RUMM program. Purple represents the females (not red) and green represents the males (not blue). Figure 6. Target Graph by Gender for Form Constancy of Letters and Numbers.
Note: There is a colour error in the RUMM program. Purple represents other schools (not red) and green represents the public schools (not blue). Figure 7. Target Graph by Type of School for Form Constancy of Letters and Numbers.
74
Janet Richmond, Russell F. Waugh and Deslea Konza
Figure 8. Target Graph by Age for Form Constancy of Letters and Numbers.
Note: There is a colour error in the RUMM program. Pre-primary is represented by green (not blue), Year 1 is represented by purple (not red), Year 2 is represented by pink (not green), and Year 3 is represented by maroon (not purple). Figure 9. Target Graph by School Year for Form Constancy of Letters and Numbers.
Rasch Measures of Form Constancy of Letters and Numbers…
75
Figure 10. Target Graph by Intervention for Visual Form Constancy of Letters and Numbers Note: There is a colour error in the RUMM program. Green represents no intervention and purple intervention.
The graphical data for Figure Ground Letters in Words were checked in the RUMM computer program but is not produced here to avoid repetition but the graphs are similar to those produced for Form Constancy of Letters and Numbers. Females had a higher mean measure than males for Figure Ground Letters in Words but this is not statistically, significantly different (t=1.90, df=321, p=0.025). Public school students had a higher mean measure than private school students for Figure Ground Letters in Words and this is statistically, significantly different (t=3.6, df=321, p=0.000) in favour of the public schools. As would be expected, the mean measures generally increased by age from four years old (lowest) to ten years old or older (highest) and this was statistically, significantly different (t=8.10, df=66, p=0.000). Again, as expected, the mean measures generally increased by grade from Pre-primary (lowest) to Year 3 (highest) and this was statistically, significantly different (t=21.2, df=127, p=0.000). While the mean measure for no intervention was higher than for intervention, this was not statistically, significantly different (t=0.71, df=321, p=0.25).
FINAL ITEMS FOR THE FORM CONSTANCY AND FIGURE GROUND SCALES The final 18 items and their difficulties are presented, in order from easiest to hardest, in Table 4 for Form Constancy of Letters and Numbers. The students found it easy to identify the reversed item for the letter „a‟ and for numbers. They found it moderately easy to identify the reversed letters that are not often reversed in the font used in this scale (e.g. e, b, c), moderately difficult to identify letters that could be reversed or letters that had a body and a tail (e.g. s, q, y) and most difficult to identify the reversed letters that are commonly written in a reversed orientation by young students (e.g. j, g, d).
Janet Richmond, Russell F. Waugh and Deslea Konza
76
In the Figure Ground Letters in Words (see Table 5 for the 34 item difficulties ordered from easy to hard), students found it easy to identify words as correct when they did not contain a reversed letter, such as: the, ran, that, know, and moderately easy to identify words as correct or incorrect when they had a mixture of long and short letters, for example ,
,
. Longer words containing a reversed letter were moderately difficult for students
to identify as correct or incorrect, for example , , ; while the most difficult words to identify as correct or incorrect were those with reversed orientation of g and u (e.g.
,
,
).
Table 4. Difficulties for 18 Final Items in Form Constancy of Letters and Numbers
Note: Items are ordered from easiest (item 18, -0.93 logits) to hardest (item 4, +1.39 logits).
Rasch Measures of Form Constancy of Letters and Numbers…
77
Table 5. Difficulties for 34 Final Items in Figure Ground Letters in Words Scale
Note: Items are ordered from easiest (item 11, -1.16 logits) to hardest (item 34, +1.36 logits).
COMMENTS ON THE NON-FITTING ITEMS DELETED FROM THE THREE SCALES Six items were deleted from the Form Constancy of Letters and Numbers Scale due to poor fit to the Rasch measurement model. Usually the main reason for non-fit is poor agreement in regard to the item difficulty. For example, half of the medium ability students may say an item is easy and half say that it is hard, thus it does not fit the measurement model and is deleted. The six items deleted in Form Constancy of Letters and Numbers Scale were: f, k, p, t, 6, and 9. The students may have disagreed on these letters due to the font used in this assessment, however it was noted that many students chose the upper case letter or the same letter as the reversed letter in a number of these situations as well as the same number or the number that had been made smaller. It is also of particular interest that most of the letters and
78
Janet Richmond, Russell F. Waugh and Deslea Konza
numbers deleted due to disagreement were the letters and numbers that students often tend to reverse. In Figure Ground Letters in Words, seven of the original 41 words were deleted due to non-fit to the Rasch measurement model. The deleted letters were the words one (with reversed e), come, ate (with reversed t), think, fast (with reversed t), together (with reversed h) and never (with reversed n). It is noticeable that five of the words with poor fit had reversed letters; however there is no noticeable pattern of the similarity of letter or position of the reversed letter in the words. The font used in the assessment may have been a contributing factor to the students‟ interpretation of these words; however this does not present as an obvious influencing factor.
INFERENCES FROM THE MEASURES OF THE THREE LINEAR RASCH SCALES Linear scales were created showing good fit to the measurement model for the Form Constancy of Letters and Numbers, Figure Ground Letters in Words and Figure Ground Numbers in Calculations. Valid inferences can now be made about the student measures for form constancy and figure ground perception from these three linear scales. The bottom 49 student measures for Form Constancy of Letters and Numbers have been taken because these students all scored 6/18 or less, meaning that they were the students who were unable to identify the letters (other than a) and were only able to achieve some of the items that contained numbers. Twenty-two students had a score of zero with a location of -3.45, a standard error of 1.24. These student measures are presented in Table 7. The students who scored zero in Form Constancy of Letters and Numbers were unable to answer any of the items correctly, suggesting that they either misunderstood the instruction or are unable to identify when numbers or letters are reversed when the letters and numbers are presented in a variety of fonts. Students who scored 6 had difficulty identifying the reversed letters, but were more capable when identifying reversed numbers in different fonts. Students scoring poorly in Form Constancy of Letters and Numbers have difficulty identifying when letters and numbers of differing fonts are reversed and may need extra assistance to improve this skill. The bottom 53 student measures for Figure Ground Letters in Words have been taken because these students scored less than 17 out of 34, meaning that they were unable to identify more than half of the items as having or not having a reversed letter within the word. These student measures are presented in Table 8. Students, who scored 7, were only able to correctly identify items where no reversed letters occurred in the word. The students scoring 17 correct answers were able to identify words containing no reversals and the easiest four words containing a reversed letter. The four easiest items containing a reversed letter consisted of three words where a letter with a body as well as a head (long letter) and one short letter with only a body. These student measures identify students who may require assistance to improve their skill in identifying when a letter is reversed within a word. They may also be the students who reverse their letters in reading, spelling and or writing.
79
Rasch Measures of Form Constancy of Letters and Numbers… Table 7. Lowest 49 Student Measures for Form Constancy of Letters and Numbers ID 151 199 167 166 165 164 324 162 203 153 163 150 27 21 19 4 3 108 37 156 323 161 200 64 110
Raw score 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 2
Location
SE
Residual
ID
-3.45 -3.45 -3.45 -3.45 -3.45 -3.45 -3.45 -3.45 -3.45 -3.45 -3.45 -3.45 -3.45 -3.45 -3.45 -3.45 -3.45 -3.45 -3.45 -3.45 -3.45 -3.45 -2.62 -2.62 -2.04
1.24 1.24 1.24 1.24 1.24 1.24 1.24 1.24 1.24 1.24 1.24 1.24 1.24 1.24 1.24 1.24 1.24 1.24 1.24 1.24 1.24 1.24 0.89 0.89 0.71
-0.93 0.16 -0.58
80 5 119 18 84 111 223 76 78 23 268 49 66 46 16 224 319 234 65 205 83 22 297 317
Raw score 3 3 3 4 4 4 4 5 5 5 5 5 5 5 5 5 6 6 6 6 6 6 6 6
Location
SE
Residual
-1.62 -1.62 -1.62 -1.28 -1.28 -1.28 -1.28 -0.99 -0.99 -0.99 -0.99 -0.99 -0.99 -0.99 -0.99 -0.99 -0.73 -0.73 -0.73 -0.73 0.73 -0.73 -0.73 -0.73
0.62 0.62 0.62 0.57 0.57 0.57 0.57 0.54 0.54 0.54 0.54 0.54 0.54 0.54 0.54 0.54 0.52 0.52 0.52 0.52 0.52 0.52 0.52 0.52
-0.59 -0.43 0.12 0.30 1.06 -0.95 -0.11 -1.09 -1.09 -0.62 -1.09 -1.09 0.56 -1.09 -0.99 -0.85 -1.28 -0.60 1.18 1.33 0.35 -0.95 0.51 0.85
Table 8. Lowest 53 Student Measures Figure Ground Letters in Words ID 324 65 66 80 82 83 150 64 156 79 164 166 167 276 199 203 205 151 23 18 81 12
Raw score 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
Location
SE
Residual
ID
-4.20 -4.20 -4.20 -4.20 -4.20 -4.20 -4.20 -4.20 -4.20 -4.20 -4.20 -4.20 -4.20 -4.20 -4.20 -4.20 -4.20 -4.20 -4.20 -4.20 -4.20 -4.20
1.22 1.22 1.22 1.22 1.22 1.22 1.22 1.22 1.22 1.22 1.22 1.22 1.22 1.22 1.22 1.22 1.22 1.22 1.22 1.22 1.22 1.22
-
24 25 26 27 2 84 67 202 237 162 3 57 20 4 8 209 110 78 74 62 206 208
Raw score 0 0 0 0 0 6 7 8 8 9 9 9 12 13 13 13 14 14 14 14 15 15
Location
SE
Residual
-4.20 -4.20 -4.20 -4.20 -4.20 -1.69 -1.49 -1.31 -1.31 -1.15 -1.15 -1.15 -0.69 -0.55 -0.55 -0.55 -0.41 -0.41 -0.41 -0.41 -0.27 -0.27
1.22 1.22 1.22 1.22 .122 0.46 0.44 0.42 0.42 0.41 0.41 0.41 0.38 0.38 0.38 0.38 0.37 0.37 0.37 0.37 0.37 0.37
0.38 0.04 -1.60 1.29 -1.10 -1.01 1.99 -0.89 -1.45 -1.45 0.13 1.66 -2.72* 0.24 -2.16 0.19 -2.54*
Janet Richmond, Russell F. Waugh and Deslea Konza
80
Table 8. (Continued). ID
Raw score
Location
SE
Residual
ID
Raw score
Location
SE
Residual
323
0
-4.20
1.22
-
114
16
-0.14
0.37
-3.30*
5 16
0 0
-4.20 -4.20
1.22 1.22
-
111 317
16 17
-0.14 -0.01
0.37 0.37
-1.20 -1.05
22 37
0 0
-4.20 -4.20
1.22 1.22
-
200
17
-0.01
0.37
4.15*
Notes on Table 8:*: Fit residual value exceeds limit set for test of fit.
SUMMARY OF FINDINGS Linear scales were created for Form Constancy of Letters and Numbers and Figure Ground Letters in Words using the RUMM2020 Program (Andrich, Sheridan, & Luo, 2005). The reliability of the two scales was shown by: 1. Global item fit as well as person item fit to the measurement model; 2. Good Person Separation Indices indicating that the person measures were reasonably well, or acceptably well, separated in relation to the errors; 3. Good item-trait interaction chi-squares indicating the measurement of a unidimensional trait; 4. Targeting of items against the person measures was reasonable, but indicates the need for easy and more difficult items in the scales for future use. Valid inferences may be drawn from the scales as the scale data were shown to be reliable. Inferences are that it is easiest for students to identify reversed numbers in a variety of fonts rather than the reversed letters and that the most difficult letters for students to identify as reversed when presented among a variety of fonts were long letters as in z, j, g, and d. For Form Constancy of Letters and Numbers, girls scored more highly than boys, but this was not statistically significant. There was no statistical significant difference between private and public schools, although public schools scored a higher mean average. Furthermore, there was as expected, a statistically significant difference in the performance of students as their age and grade increased, with younger students in lower grades scoring significantly lower than the older students in the higher grades. Students with the lowest scores were those who had most difficulty identifying reversed letters and numbers among a selection of letters and numbers presented in a variety of fonts. For Figure Ground Letters in Words the girls scored a higher mean average than boys, but this was not statistically significant. Public schools scored a statistically significant higher mean value than private schools. The younger students in the lower grades scored a lower mean value than the older students in the higher grades and this was statistically significant as would be expected. Students with the lowest scores had difficulty identifying words that contained a reversed letter as opposed to words that did not have a reversed letter embedded in the word.
Rasch Measures of Form Constancy of Letters and Numbers…
81
REFERENCES Andrich, D., Sheridan, B.E., & Luo , G. (2005). Rasch Unidimensional Measurement Models (RUMM2020): A windows-based item analysis program employing Rasch models. Perth, WA: RUMM Laboratory Andrich, D. & van Schoubroeck, L. (1989). The General Health Questionnaire: A psychometric analysis using latent trait theory. Psychological Medicine, 19, 469-485 Rasch, G. (1960/1980). Probabilistic models for some intelligence and attainment tests (expanded edition). Chicago, IL: MESA Press (original work published in 1960).
In: Specialized Rasch Measures… Editor: Russell F. Waugh, pp. 83-99
ISBN: 978-1-61668-032-9 © 2010 Nova Science Publishers, Inc.
Chapter 5
RASCH MEASURES OF NUMBER DISCRIMINATION AND REVERSAL, AND NUMBERS IN CALCULATIONS FOR YOUNG CHILDREN Janet Richmond, Russell F. Waugh and Deslea Konza Faculty of Education and Arts, Edith Cowan University, Perth, Western Australia.
ABSTRACT Number literacy is a very important topic and the Australian Government runs numeracy and literacy tests, administered through the State Education Departments, for all Year 3 (8 years old), Year 5 (10 years old) and Year 7 (12 years old) students. Results of these tests are reported to schools and parents with a view to ensuring that all children meet certain numeracy standards and that children who are „falling behind‟ are detected early so that remedial work can be given. Rasch measures were created with the RUMM2020 computer program for Visual Discrimination of Numbers (VDN) and Figure Ground Numbers in Calculations (FGNC). The student sample was N=324 pre-primary and primary students in Perth, Western Australia, aged 4-9 years old. Data on 20 items for VDN and 28 items for FGNC, where each item was scored in one of two categories (wrong scored zero and correct scored one), were Rasch analysed to create two linear scales. Six of the initial 20 items for VDN were deleted due to item misfit statistics, leaving 14 items. Thirteen of the initial 28 items for FGNC were deleted due to item misfit statistics, leaving 15 items. The final data for VDN and FGNC were used to create two highly reliable, linear, uni-dimensional scales (Student Separation Indices of 0.75 and 0.95 respectively) where the items are ordered from easy to hard and the student measures from low to high on the same scale. The two scales showed no statistically significant interaction of student measures on item difficulties along the scale, meaning that there was good agreement about the item difficulties along each scale, and each scale was unidimensional. The item-trait chi-squares are respectively, χ² = 68.34, df=0.92, p=0.12, and χ² = 58.83, df=60, p=0.52. The fit residual statistics for each of the two scales was reasonable and the targeting was reasonable.
84
Janet Richmond, Russell F. Waugh and Deslea Konza
INTRODUCTION: LITERATURE REVIEW A National Literacy and Numeracy Plan was instituted in Australia to improve literacy and numeracy standards in the Australia. To identify students at risk, the National Assessment Program Literacy and Numeracy (NAPLAN) (Department of Education and Training, 2008) has been instituted to assess children in Year Three, Five and Seven, however this does not fit with early identification of students as stated by the Australian Council for Educational Research. In addition, the final report of literacy and numeracy review in Western Australia found that there was a need for pre-primary diagnostic assessment of numeracy skills to identify the students at risk (Department of Education and Training, 2007), while the Western Australian Government has developed a plan to improve the numeracy outcomes of students in Western Australia (Government of Western Australia, 2007). These policies and plans require relevant, linear, user friendly assessments to identify students at risk so that the plans to improve numeracy skills at the earliest opportunity can be implemented. Children having difficulty with the mechanics of mathematics (dyscalculia) are slow to grasp the relative size of figures, to learn tables, to remember the sequencing of digits, and to understand the meaning of mathematical signs or master fractions (Green & Chee, 1997). To manage mathematics as an academic subject, children need to use visual imagery in order to display planning, problem solving, and organisation, as well as have a good working memory (Green & Chee, 1997; Loikith, 1997). This link between symbolic language and mathematics was also identified by Johnson and Myklebust (1978), who found that the practical function in mathematics was to express quantitative and spatial relationships and the theoretical function in mathematics was to facilitate thinking. In addition, Lucas and Lowenberg (1996) separated mathematical concepts into two major aspects: (1) recognition and manipulation of numbers, and (2) acquisition and application of the language of mathematics, which in turn makes problem solving possible. To carry out mathematical computations, children must have an understanding or grasp of basic perceptions of shape, space, symbols, copying and numeracy (Chinn, 2002; Miles, Chinn, & Peer, 2000; Schneck, 1996). Furthermore, the manipulation of numbers in mathematics also requires good visual perceptual skills such as visual discrimination, directionality, sequencing, organisation of work (spatial), correct alignment of columns for calculation (placement of number values), figure ground and memory (Chinn, 2002). For example, many rows of calculations on a worksheet could be disorganising for the child with figure-ground problems. Spatial perceptual skills are required in geometry and visual memory is required when multiple steps are required in a sum (Schneck, 1996). A number of authors agree that to solve mathematical problems, understand geometric relationships and use graphs, children require recognition skills, the ability to discriminate and the ability to compare objects, form and space (including inversions, rotations and distortions) (Chinn, 2002; Fisher, Murray, & Bundy, 1991; Hung, Fisher, & Cremak, 1987; Levine, 1991; Schneck, 1996). Siegel (1999) described dyscalculia as “a crippling ailment that prevents one from learning math” (p. 305), while others (Fisher et al., 1991; Lucas & Lowenberg, 1996) found that difficulties with language may affect mathematical skills in the area of problem solving where problems are written in words rather than numbers. It has also been found that some learners had specific learning difficulties in mathematics where they could manipulate
Rasch Measures of Number Discrimination and Reversal…
85
numbers orally and mentally, but were unable to record the responses as mathematical manipulations were primarily conducted in the right cerebral hemisphere of the brain while writing was primarily conducted in the left cerebral hemisphere (Fisher et al., 1991; Lucas & Lowenberg, 1996). The right hemisphere has an important role in understanding and applying mathematical concepts. Fisher et al. (1991) suggested that this deduction was based on associations between visual-spatial abilities and the understanding of mathematical concepts. The visual-spatial abilities can be determined in picture completion and copying tasks which are important predictors of arithmetic (mathematical) achievement (Belka & Williams, 1979; Sorter & Kulp, 2003). Thus, it appears that mathematical ability is affected by visual perceptual skills. These visual perceptual skills include, but are not limited to, visual memory, visual sequential memory, visual perception and specifically visual spatial ability (Belka & Williams, 1979; Chinn, 2002; Fisher et al., 1991; Green & Chee, 1997; Hung et al., 1987; Levine, 1991; Miles et al., 2000; Schneck, 1996; Simpson, 1987).
RASCH REPORT This report presents a Rasch analysis with the RUMM 2020 computer program (Andrich, Sheridan & Luo, 2005) in which two linear, unidimensional scales were created: (1) Visual Discrimination of Numbers and (2) Figure Ground Numbers in Calculations. These two scales relate to visual perceptual concepts of „visual discrimination‟ and „figure ground‟. This report describes the measurement results in terms of Rasch measurement fit statistics including global item and person fit to the measurement model, dimensionality, person separation indices, distribution of item-person interactions, and discrimination. Some discussion is included of the non-fitting items, as well as good fitting items, and the personitem threshold distribution (targeting). This is followed by mean Rasch measures by group and final items for the Visual Discrimination and Figure Ground Scales discussion. Finally, inferences drawn from the linear Rasch measurement data analysis and the summary of the results are presented.
INITIAL RASCH ANALYSIS An initial Rasch analysis was performed on the original items for Visual Discrimination of Numbers (20 items) and Figure Ground Numbers in Calculations (28 items) where each item was scored in one of two categories (incorrect answer scored zero and correct answer scored one). Six of the initial 20 items of Visual Discrimination of Numbers were deleted due to item misfit statistics. The remaining 14 items were found to have a good fit to the measurement model for the 324 persons included in this study. For Figure Ground Numbers in Calculations, 13 of the initial 28 items were removed because of item misfit statistics with the remaining 15 items were found to have an excellent fit to the measurement model. The Rasch analysis with the RUMM program does not indicate how to alter an item in order to make it fit the measurement model. In order to include, in a future measure, the deleted items
Janet Richmond, Russell F. Waugh and Deslea Konza
86
which were initially considered conceptually valid, these would need to be changed and retested.
FINAL RASCH ANALYSIS RESULTS The following material shows the results for the final Rasch analysis for the two scales: (1) Visual Discrimination of Numbers (14 items), and (2) the Figure Ground Scale of Numbers in Calculations (15 items).
Summary of Fit Statistics The RUMM2020 program estimates an item-person interaction which establishes the overall fit statistics that determine whether the item estimations contribute meaningfully to the measurement of one construct. This calculation thus examines the consistency with which students responses agree with the calculated difficulty of each item on the scale. The standardised fit residual statistics (see Table 1) have a distribution with a mean near zero and a standard deviation near one when the data fit the measurement model, as is the case with these three measures. This means too that there is a good pattern of person and item responses consistent with a Rasch measurement model.
Dimensionality For Visual Discrimination of Numbers, there was an item-trait interaction chi-square of 68.34 with df = 0.92 and a probability of 0.12. This means that the scale is constructed with acceptable agreement amongst the students about the linear progressive difficulty of the items. For Numbers in Calculations, the item-trait interaction chi-square was 58.83 with df=0.93 and a probability of 0.52 respectively, showing very good agreement amongst the students about the item difficulties along the scale. This means that the students agree as to which items are easy, which are of medium difficulty and which are hardest. Table 1. Global Item and Student Fit Residual Statistics (N=324) ITEMS Location Visual Discrimination of Numbers (I=14) -0.42 -0.42 0.92 0.92 Numbers in Calculations (I=15) Mean 0.00 Standard Dev. 0.59
Fit Residual
PERSONS Location
Fit Residual
-0.45 0.89
+1.97 2.06
-0.20 0.75
-0.35 0.04
+1.29 2.11
-0.08 0.84
Comment on Table 1: Fit residuals have a mean near zero and a standard deviation near one when the data fit the measurement model (as is the case here). This reflects good consistency of item and student scoring patterns.
87
Rasch Measures of Number Discrimination and Reversal…
Person Separation Index The Person Separation Index is an estimate of the true score variance among the students and the estimated observed score variance using the estimates of their ability measures and the standard error of these measures (Andrich & van Schoubroeck, 1989). For Visual Discrimination of Numbers and Numbers in Calculations, the Person Separation Indices are 0.75 and 0.95 respectfully. For a good measure, it is desirable that this index should be 0.9 or greater, as it is an indicator that the student measures are separated by more than their standard errors. Based on this index, the Visual Discrimination of Numbers demonstrates an acceptable separation, while Figure Ground Numbers in Calculations scale demonstrates very good separation of measures in comparison to the errors of measurement.
Individual Item Fit Items are ordered by calibrated values to evaluate their fit to the measurement model. The location of each item on the scale is the item difficulty in standard units, called logits (log odds of answering successfully). All the items in Visual Discrimination of Numbers fit the measurement model with probabilities greater than p=0.05 (see Table 2). The residuals shown in Table 2 represent the difference between the observed responses and the expected responses calculated from the Rasch measurement parameters. Standardised residuals should fall within the range of -2 and +2. Table 2 shows that all items for Visual Discrimination of Numbers have acceptable residuals. Table 2. Individual Item Fit Statistics for Visual Discrimination Numbers Item 6 17 2 4 7 3 19 10 9 18 5 13 11 20
Location -1.79 -1.34 -1.08 -0.81 -0.79 -0.78 -0.38 -0.33 0.69 0.93 0.96 1.43 1.53 1.78
Notes on Table 2:
7. 8. 9.
SE 0.33 0.28 0.25 0.23 0.23 0.23 0.21 0.20 0.16 0.16 0.16 0.15 0.15 0.15
Residual -0.98 -0.43 0.68 -1.65 -1.46 -1.25 -0.19 -0.89 -0.07 0.33 -0.75 -0.38 -1.30 1.80
DegFree 191.29 191.29 191.29 191.29 191.29 191.29 191.29 191.29 191.29 191.29 191.29 191.29 191.29 191.29
ChiSq 1.92 3.26 8.63 3.98 6.20 5.83 5.60 2.89 1.98 1.22 6.18 2.75 9.58 8.32
DegFree 4 4 4 4 4 4 4 4 4 4 4 4 4 4
Prob 0.75 0.52 0.07 0.41 0.18 0.21 0.23 0.58 0.74 0.87 0.19 0.60 0.05 0.08
Location refers to the difficulty of the item on the linear scale. SE means Standard Error, and refers to the degree of uncertainty in a value. Residual represents the difference between the expected value of an item, calculated according to the Rash measurement model and the actual value. 10. DegFree stands for degrees of freedom, and refers to the number of scores in a distribution that are free to change without changing the mean distribution. 11. ChSq stands for Chi-square
Janet Richmond, Russell F. Waugh and Deslea Konza
88
12. Prob relates to the probability based on the Chi-square and refers to the levels of certainty to which an item fits the measurement model.
For Figure Ground Numbers in Calculations, all the items fit the measurement model with probabilities greater than p=0.08 (see Table 3) and residuals are very satisfactory. Table 3. Individual Item Fit Statistics for Figure Ground Numbers in Calculations Item
Location
SE
Residual
DegFree
ChiSq
DegFree
Prob
13
-0.94
0.20
-0.46
159.60
2.45
4
0.65
12
-0.88
0.20
-0.44
159.60
3.26
4
0.52
7
-0.64
0.20
+0.61
159.60
2.00
4
0.74
14
-0.57
0.19
-1.19
159.60
4.76
4
0.31
8
-0.32
0.19
+0.73
159.60
2.19
4
0.70
11
-0.17
0.19
-0.27
159.60
1.93
4
0.75
10
-0.05
0.19
+1.22
159.60
2.22
4
0.70
9
+0.03
0.18
+1.12
159.60
4.67
4
0.32
20
+0.22
0.18
-1.33
159.60
3.99
4
0.41
21
+0.22
0.18
-1.52
159.60
5.38
4
0.25
16
+0.38
0.18
-0.29
159.60
4.55
4
0.34
15
+0.40
0.18
+0.79
159.60
3.36
4
0.50
25
+0.43
0.18
-1.52
159.60
8.21
4
0.08
27
+0.81
0.18
-0.79
159.60
2.90
4
0.58
-1.97
159.60
7.00
4
0.14
24
+1.07
0.18
Notes on Table 3:
1. 2. 3. 4. 5. 6.
Location refers to the difficulty of the item on the linear scale. SE means Standard Error, and refers to the degree of uncertainty in a value. Residual represents the difference between the expected value of an item, calculated according to the Rash measurement model and the actual value. DegFree stands for degrees of freedom, and refers to the number of scores in a distribution that are free to change without changing the mean distribution. ChSq stands for Chi-square Prob relates to the probability based on the Chi-square and refers to the levels of certainty to which an item fits the measurement model.
Targeting The RUMM2020 program produces a student-measure item-difficulty or targeting graph on which the student measures are placed on the same scale as the item difficulties in standard units called logits. For Visual Discrimination of Numbers (see Figure 1), the targeting graph shows that the student measures cover a range of about -1.2 to +3.8 logits and the item difficulties cover a range of about -1.8 to +1.8 logits. From the graph it can be seen that many students (about 215) were able to answer the items correctly and the targeting of the items needs to be improved in any future use of the scale by adding in some harder items to „cover‟ the students with the higher measures.
Rasch Measures of Number Discrimination and Reversal…
89
Note: Student measures are on the upper side in logits. Item difficulties are on the lower side of the same scale in logits. Many students (about 215) answered the items correctly. Figure 1. Targeting for Visual Discrimination Numbers.
For Figure Ground Numbers in Calculations (see Figure 2), the targeting graph shows that the student measures cover a range of about -3.4 to +3.3 logits and the item difficulties cover a range of about -1.0 to +1.2 logits. From the graph it can be seen that many students (about 195) were able to answer the items correctly, while about 42 were unable to answer any items correctly, thus the targeting of the items needs to be improved in any future use of the scale by adding in some easier and more difficult items to „cover‟ the students with the lower and higher measures.
Note: Student measures are on the upper side in logits. Item difficulties are on the lower side of the same scale in logits. Many students (about 215) answered the items correctly. Figure 2. Targeting for Figure Ground Numbers in Calculations.
90
Janet Richmond, Russell F. Waugh and Deslea Konza
Discrimination Item Characteristic Curves examine the relationship between the expected response and the mean group student measures. These curves display how well the item discriminates between groups of persons. An example of one item characteristic curve for each of the two constructs will be presented. Figure 3 shows the Item Characteristic Curve for Item 13 Visual Discrimination of Numbers. This curve shows that the item discriminates well for students with different measures. The Item Characteristic Curves for all the other items were checked and found to be satisfactory (but are not reported here to avoid unnecessary repetition).
Figure 3. Item Characteristic Curve: Item 13 – Visual Discrimination Numbers.
Figure 4. Item Characteristic Curve: Item 4 – Figure Ground Numbers in Calculations.
Figure 4 shows the Item Characteristic Curves for Item 4 of Figure Ground Numbers in Calculations. These items discriminate well for students with different measures. The Item
Rasch Measures of Number Discrimination and Reversal…
91
Characteristic Curves for all the other items in the measure were checked and found to be satisfactory (but are not reported here to avoid unnecessary repetition).
Consistency of Use of Scoring Categories The RUMM2020 program produces graphs of the scoring categories for each item. The Scoring Category Curves show the relationship between the probability of scoring in each category (zero for incorrect answer and one for correct answer) on each item. Figure 5 is the Scoring Category Curve for item 2 of Visual Discrimination of Numbers. This figure shows that the scoring was done logically and consistently. When students have low measures on item 2, then they have a high probability of obtaining a zero score (answer incorrect) and, when they have a high measure, they have a high probability of scoring 1 (answer correct). The Scoring Category Curves for all the other items were checked and they were satisfactory too. The Scoring Category Curves for all the items of the other variable, Figure Ground Numbers in Calculations, were checked and they were also found to be satisfactory, but they are not presented here to avoid repetition.
Figure 5. Scoring Category Curve: Item2 – Visual Discrimination of Numbers.
CHARACTERISTICS OF THE SAMPLE (VDN AND FGNC) The measures for Visual Discrimination of Numbers (VDN) were displayed in a graphical format separated by gender (Figure 6), type of school (Figure 7), age (Figure 8), grade (Figure 9) and whether intervention had been received (Figure 10). Females have a higher mean measure than males for Visual Discrimination of Numbers but this is not statistically, significantly different (t=1.78, df=320, p=0.04). Public school students have a higher mean measure than private school students for Visual Discrimination of Numbers and this is not statistically, significantly different (t=1.39, df=320, p=0.03). As would be expected, the mean measures generally increased by age from four years old (lowest) to ten years old or
92
Janet Richmond, Russell F. Waugh and Deslea Konza
older (highest) and this was statistically, significantly different (t=8.79, df=65, p<0.000). Again, as expected, the mean measures generally increased by grade from Pre-primary (lowest) to Year 3 (highest) and this was statistically, significantly different (t=13.01, df=125, p<0.000). While the mean measure for no intervention was higher than for intervention, this was not statistically significantly different (t=1.21, df=320, p=0.10).
Note: There is a colour error in the RUMM program. Purple represents the females (not red) and green represents the males (not blue). Figure 6. Target Graph by Gender for Visual Discrimination of Numbers.
Note: There is a colour error in the RUMM program. Purple represents other schools (not red) and green represents the public schools (not blue). Figure 7. Target Graph by Type of School for Visual Discrimination of Numbers.
Rasch Measures of Number Discrimination and Reversal…
93
Note: There is a colour error in the RUMM program. Four and five year olds are represented by green (not blue), six year olds are represented by Purple (not red), seven year olds are represented by pink (not green), eight year olds are represented by maroon (not purple), nine year olds are represented by black (not brown-green) and ten years and above are represented by brown-green (not black). Figure 8. Target Graph by Age for Visual Discrimination of Numbers.
Note: There is a colour error in the RUMM program. Pre-primary is represented by green (not blue), Year 1 is represented by purple (not red), Year 2 is represented by pink (not green), and Year 3 is represented by maroon (not purple). Figure 9. Target Graph by School Year for Visual Discrimination of Numbers.
94
Janet Richmond, Russell F. Waugh and Deslea Konza
Note: There is a colour error in the RUMM program. Green represents no intervention and purple intervention. Figure 10. Target Graph by Intervention for Visual Discrimination of Numbers.
The graphical data for Figure Ground Numbers in Calculations was also checked in the RUMM computer program but is not produced here to avoid too much repetition; however the graphs are similar to those produced for Form Constancy of Letters and Numbers. Females have a higher mean measure than males for Figure Ground Numbers in Calculations and this is statistically, significantly different (t=2.98, df=322, p=0.000). Public school students had a higher mean measure than private school students for Figure Ground Numbers in Calculations and this is statistically, significantly different (t=2.44, df=322, p=0.002) in favour of the public schools. As would be expected, the mean measures generally increased by age from four years old (lowest) to ten years old or older (highest) and this was statistically, significantly different (t=10.2, df=66, p=0.000). Again, as expected, the mean measures generally increased by grade from Pre-primary (lowest) to Year 3 (highest) and this was statistically, significantly different (t=22.5, df=127, p=0.000). While the mean measure for no intervention was higher than for intervention, this was not statistically significantly different (t=1.64, df=322, p=0.05).
FINAL ITEMS FOR THE FORM CONSTANCY AND FIGURE GROUND SCALES The final 14 items and their difficulties are presented, in order from easiest to hardest, in Table 4 for Visual Discrimination of Numbers. The students found it very easy to discriminate reversed and non-reversed numbers when the number could not be reversed such as the 1 and 8, and found it moderately easy to discriminate numbers that could be reversed but were presented in the correct orientation such as 2, 4, and 5. Moderate difficulty was experienced in discriminating reversed numbers for example , , with the reversed 3 ( ) being the most difficult number for students to discriminate. In the Figure Ground Numbers in Calculations (see Table 5 for the 15 item difficulties ordered from easy to hard), the students found it very easy to identify the reversed numbers in
Rasch Measures of Number Discrimination and Reversal… a simple plus or subtract calculation where the numbers were under 20 (e.g.
95 ,
) and found it moderately easy to identify the reversed number in addition and subtraction calculations where the numbers were in the teens or above 20 such as . Moderate difficulty was experienced in identifying reversed numbers in larger numbers or when the division sign was used for example: , and the students found it most difficult to identify the reversed number in vertically arranged calculations such as
.
Table 4. Difficulties for 14 Final Items in Visual Discrimination for Numbers Scale
Note: Items are ordered from easiest (item 6, -1.79 logits) to hardest (item 20, +1.78 logits).
Table 5. Difficulties for 15 Final Items in Figure Ground Numbers in Calculations Scale
Note: Items are ordered from easiest (item 13, -0.94 logits) to hardest (item 24, +1.07 logits).
96
Janet Richmond, Russell F. Waugh and Deslea Konza
COMMENTS ON THE NON-FITTING ITEMS DELETED FROM THE TWO SCALES Six items were deleted from the Visual Discrimination of Numbers Scale due to poor fit to the Rasch measurement model. Usually the main reason for non-fit is poor agreement in regard to the item difficulty. For example, half of the medium ability students may say an item is easy and half say that it is hard, thus it does not fit the measurement model and is deleted. The numbers excluded from the analysis were 7, 3, 8 and the reversed numbers 5, 5, and 2. The first 8 in the assessment received good agreement among the students and was one of the easiest items, however when the 8 appeared the second time in the scale, there was disagreement among the student as to the difficulty of this item. This second number 8 was situated between two other numbers (3 and reversed 2) where there was poor agreement of difficulty on the second last line of the scale, thus the positioning of the items may have had an influencing factor on the students‟ response. Both of the reversed number 5‟s caused poor agreement on the difficulty of the item for students. Thirteen of the original calculations were deleted in Figure Ground Numbers in Calculations due to non-fit to the Rasch measurement model. The calculations excluded from the analysis were three items where the student identified the number of pictures, and three calculations with numbers all under five (six easiest calculations), three horizontal divide or multiplication calculations as well as four horizontally positioned calculations, which included reversed numbers in any position of the calculation. The alignment of the calculations and the operation sign may have had an influencing factor on the students‟ responses.
INFERENCES FROM THE MEASURES OF THE TWO LINEAR RASCH SCALES Linear scales were created showing good fit to the measurement model for the Visual Discrimination of Numbers Figure Ground Numbers in Calculations scales. Valid inferences can now be made about the student measures for visual discrimination and figure ground perception of numbers from these two linear scales. The bottom 15 student measures for Visual Discrimination of Numbers were chosen because these students scored less than eight out of fourteen, meaning that they were unable to identify or discriminate any of the reversed numbers in the scale. These student measures are presented in Table 6. Students who scored four out of 14 were only able to discriminate the symmetrical numbers and the number 6. Students scoring seven were unable to discriminate any of the reversed numbers and also had difficulty with the number 9. The font may have affected the discriminatory ability of some of the numbers such as the „9‟; however the font makes most of the numbers distinguishable in a standard hand written form. The bottom 45 student measures for Figure Ground Numbers in Calculations have been chosen because these students scored three or less out of fifteen correct, meaning that they were only able to identify reversed numbers in calculations containing numbers smaller than 12 and the reversed number was standing alone in the equation and was not part of a number greater than nine. These student measures are presented in Table 7. Students who scored zero
97
Rasch Measures of Number Discrimination and Reversal…
out of 15 were unable to identify any reversed numbers. Students scoring three were only able to identify reversed numbers in simple calculations where the reversed number stood alone. Table 6. Lowest Student Measures Visual Discrimination Numbers ID
Raw score
Location
SE
Residual
151
4
-1.14
0.65
0.33
80
5
-0.75
0.63
-0.86
113
5
-0.75
0.63
0.03
208
6
-0.39
0.62
-0.51
58
6
-0.39
0.62
0.37
27
6
-0.39
0.62
2.40
57
6
-0.39
0.62
-0.28
51
6
-0.39
0.62
0.22
150
6
-0.39
0.62
-1019
81
7
-0.02
0.61
0.59
234
7
-0.02
0.61
0.91
78
7
-0.02
0.61
0.86
301
7
-0.02
0.61
0.74
200
7
-0.02
0.61
1.08
139
7
-0.02
0.61
-0.69
Table 7. Lowest 45 Student Measures Figure Ground Numbers in Calculations ID 2 201 167 166 324 162 156 153 151 139 203 16 163 83 82 81 80 79 66 65 64 42 37
Raw score 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
Location
SE
Residual
ID
-3.25 -3.25 -3.25 -3.25 -3.25 -3.25 -3.25 -3.25 -3.25 -3.25 -3.25 -3.25 -3.25 -3.25 -3.25 -3.25 -3.25 -3.25 -3.25 -3.25 -3.25 -3.25 -3.25
1.27 1.27 1.27 1.27 1.27 1.27 1.27 1.27 1.27 1.27 1.27 1.27 1.27 1.27 1.27 1.27 1.27 1.27 1.27 1.27 1.27 1.27 1.27
-
18 323 161 27 24 20 208 223 111 5 87 67 200 206 205 165 23 57 150 202 26 307
Raw score 0 0 0 1 1 1 1 2 2 2 2 2 2 2 2 3 3 3 3 3 3 3
Location
SE
Residual
-3.25 -3.25 -3.25 -2.41 -2.41 -2.41 -2.41 -1.81 -1.81 -1.81 -1.81 -1.81 -1.81 -1.81 -1.81 -1.37 -1.37 -1.37 -1.37 -1.37 -1.37 -1.37
1.27 1.27 1.27 0.90 0.90 0.90 0.90 0.73 0.73 0.73 0.73 0.73 0.73 0.73 0.73 0.64 0.64 0.64 0.64 0.64 0.64 0.64
-0.95 -0.90 -0.90 -0.41 -0.68 +0.10 -0.23 -0.07 -0.67 -0.19 -0.23 -0.31 -0.50 -0.38 -0.94 -1.16 +0.09 -0.82 +1.10
98
Janet Richmond, Russell F. Waugh and Deslea Konza
SUMMARY OF FINDINGS Linear scales were created for Visual Discrimination of Numbers and Figure Ground Numbers in Calculations using the RUMM2020 Program (Andrich, Sheridan, & Luo, 2005). The reliability of the three scales was shown by: 1. Good global item fit as well as person item fit to the measurement model; 2. Good Person Separation Indices indicating that the person measures were reasonably well, or acceptably well, separated in relation to the errors; 3. Good item-trait interaction chi-squares indicating the measurement of a unidimensional trait; 4. Targeting of items against the person measures was reasonable, but indicates the need for easy and more difficult items in the scales for future use. Valid inferences may be drawn from the scales as the scale data were shown to be reliable. Inferences are that in Visual Discrimination of Numbers, the girls scored a statistically higher mean average than the boys. Although public schools had a higher mean average than private schools, this was not statistically significant. Mean values increased with age from the youngest students (four years old) to the oldest students (10 plus years old) with a statistically significant difference. The mean values increased by grade from Pre-primary to Grade 3 with a statistically significant difference. Students with the lowest measures had difficulty discriminating reversed numbers as well as number „9‟. In Figure Ground Numbers in Calculations, the girls scored a statistically significantly higher mean average than the boys. Students in public schools had a statistically significantly higher mean average than private schools for this scale. Mean values increased with age from the youngest students (four years old) to the oldest students (10 plus years old) with a statistically significant difference. The mean values increased by grade from Pre-primary to Grade 3 with a statistically significant difference. Students with the lowest measures had difficulty identifying reversed numbers within the context of a calculation.
REFERENCES Andrich, D., Sheridan, B., & Luo, G. (2005). RUMM2020. A windows-based item analysis program employing unidimentional measurement models. Perth, WA: RUMM Laboratory. Andrich, D., & van Schoubroeck, L. (1989). The General Health Questionnaire: A psychometric analysis using latent trait theory. Psychological Medicine, 19, 469-485. Belka, D. E., & Williams, H. G. (1979). Prediction of later cognitive behaviour from early school perceptual-motor, perceptual and cognitive performances. Perceptual and Motor Skills, 49, 131-141. Chinn, S. (2002, September 2002). Dyslexia and mathematics. Paper presented at the 28th Annual conference of South African Association of learners with educational difficulties, University of the Western Cape, Bellville, Cape Town.
Rasch Measures of Number Discrimination and Reversal…
99
Department of Education and Training. (2007). Literacy and numeracy review taskforce: The final report. Retrieved 28 April, 2009, from www.literacyand number acy review.det.wa.edu.au Department of Education and Training. (2008, March 4, 2008 ). National Assessment Program Literacy and Numeracy Retrieved 28 April, 2009, from http://www.det.wa.edu.au/education/walna/ Fisher, A. G., Murray, E. A., & Bundy, A. C. (1991). Sensory integration theory and practice. Philadelphia: F.A.Davis Company. Government of Western Australia. (2007). Western Australia's plan to improve literacy and numeracy outcomes: National Forum Agenda. Retrieved 28 April, 2009, from http://www.socialpolicy.dpc.wa.gov.au/documents/literacyNumeracy_20070409.pdf Green, C., & Chee, K. (1997). Understanding ADHD. A parent's guide to attention deficit hyperactivity disorder in children. London: Vermilion. Hung, S. S., Fisher, A. G., & Cremak, S. A. (1987). The performance of learning disabled and normal young men on the Test of Visual Perceptual Skills. American Journal of Occupational Therapy, 41(12), 790-797. Johnson, D. J., & Myklebust, H. R. (1978). Learning disabilities - educational principles and practices. New York: Grune and Stratton Inc. Levine, K. J. (1991). Fine motor dysfunction. Therapeutic strategies in the classroom. Arizona: Therapy Skill Builders (A Division of Psychological Corporation). Loikith, C. C. (1997). Visual perception: Development, assessment and intervention. In M. Gentile (Ed.), Functional visual behaviour: A therapist's guide to evaluation and treatment options. (pp. 197-247). Rockville, MD: American Occupational Therapy Association Inc. Lucas, E., & Lowenberg, E. (1996). The right way - A guide for parents and teachers to encourage visual learners. Glenashley, Durban: TEL Publishers. Miles, T., Chinn, S., & Peer, L. (2000). Dyslexia and mathematics. A guide for parents and teachers.: British Dyslexia Association. Schneck, C. M. (1996). Visual perception. St Louis, MI: Mosby Year Book Inc. Siegel, L. S. (1999). Issues in the definition and diagnosis of learning disabilities: A perspective on Guckenberger v. Boston University. Journal of Learning Disabilities, 33(4), 304-319. Simpson, M. (1987). Presenting characteristics and deficits. Paper presented at the Talk prepared by the occupational therapy department as an inservice, Brown's School, Durban. Sorter, J. M., & Kulp, M. T. (2003). Are the results of the Beery-Buktenica Developmental Test of Visual-Motor Integration and its subtests related to achievement test scores? Optometry and Vision Science, 80(11), 758-763.
In: Specialized Rasch Measures… Editor: Russell F. Waugh, pp. 101-130
ISBN: 978-1-61668-032-9 © 2010 Nova Science Publishers, Inc.
Chapter 6
RASCH MEASURES OF SELF-DISCIPLINE AND MODERATION IN MATHEMATICS EDUCATION Liu Shiueh Ling and Russell F. Waugh Graduate School of Education University of Western Australia
ABSTRACT This paper is part of a larger study of „caring-thinking‟ with Mathematics students in Singapore. „Caring Thinking‟ was used as a means of developing values in students by creating the context in teaching Mathematics to allow students to engage in valuational thinking, values realisation and values clarification. The sample was N=226 Year 10 students in secondary education. A Rasch analysis was used to create linear, unidimensional scales for Self-Discipline (16 items) and Moderation (12 items). Items were answered in two perspectives: (1) I aim for this (attitude) and (2) I actually do this (behavior), with attitude conceptualized as easier then behavior. The item-trait chisquares were χ2 = 63.18, df = 48, p = 0.07 for Self-Discipline and χ2 = 49.84, df = 36, p = 0.06 for Moderation, showing a moderate fit to the measurement model. The Student Separation Indices were 0.77 and 0.70, and the Cronbach Alphas were 0.81 and 0.68, respectively, showing good scale reliability. There were good item fits for each scale and the items were ordered from easy to hard providing good information relating to each variable.
INTRODUCTION In 1997, Singapore‟s Prime Minister unveiled the vision of Thinking Schools, Learning Nation to prepare the youth of Singapore for the future (Ministry of Education, Singapore, 1997a). The first step in the journey to realise the vision was to define the goals (Teo, 1998). These were spelt out in 1998, when the Ministry of Education published a document on the Desired Outcomes of Education (Ministry of Education, Singapore, 1998). In 2005, at the Ministry of Education Work Plan Seminar on Achieving Quality: Bottom Up Initiative, Top Down Support, the Minister for Education called for a greater emphasis and ownership in the character development of students (Shanmugaratnam, 2005). He expounded on how schools
102
Liu Shiueh Ling and Russell F. Waugh
will be given more ownership and encouraged to place greater emphasis on character development (Shanmugaratnam, 2005). The following material provides the background and context for a study on the impact of „Caring Thinking‟ on students‟ social attitudes and behavior as a result of being taught values in a mathematics context, that is, as part of a mathematics module at one of the premier high schools in Singapore.
MATHEMATICS EDUCATION A Global Scan In the late 1950s and early 1960s, the American mathematics curriculum began a series of dramatic shifts. The first major shift came about in response to the Soviet Union being the first in space in 1957, with the artificial satellite Sputnik, and the first to achieve manned orbital flight, in 1961. Thorough investigation of scholastic trends, and further data analysis, showed that the percentages of American high school students taking advanced mathematics, science and foreign language courses had been in steady decline for years and that the reason for the decline was that high schools had consistently lowered their course requirements for nearly every major field of study, including foreign language, mathematics and science (Jeynes, 2007). American mathematicians updated and upgraded mathematics instruction, as it was perceived to lack quality (Schoenfeld, 1987). A revised mathematics curriculum, with more significant mathematical content, was developed (Schoenfeld, 1987). A decade later, in the early 1970s, the revised mathematics curriculum was generally considered to be a failure (Schoenfeld, 1987). The public perception was that school children not only failed to understand the revised mathematics curriculum, but were no longer able to add, subtract, multiply, or divide. While the revised mathematics curriculum was mathematically rich, it was not supported by a comparably rich theory of pedagogy (Schoenfeld, 1987). This began the “back to basics” movement in 1983, an educational reform movement initiated by the presidential administration of Ronald Reagan (1981 to 1989) and started largely as a result of the decline in achievement test scores (Jeynes, 2007). Bill Bennett, Secretary of Education in the Reagan administration, encouraged schools to focus on teaching basic material such as reading, mathematics and science (Jeynes, 2007). For the next 10 years, teachers used drill and practice to ensure that their students would acquire basic foundation skills in mathematics (Schoenfeld, 1987). Problem solving was born (or reborn) in the late 1970s in reaction to the failures of the “back to basics” movement: students were memorising rote procedures without understanding them, they were not any better at them than the previous generation of students, and they could not use them in problems that called for even the simplest application (Carpenter et al., 1983). Problem solving, thus, became the theme of the 1980s for American school mathematics and was to stay for two decades. Prystay (2004), in his article, `As math skills slip, U.S. schools seek answers from Asia‟ (The Wall Street Journal, 13 December 2004), highlighted that American schools, concerned with the deterioration of students‟ mathematics skills were importing the mathematics curriculum used in Singapore. In the Trends in International Mathematics and Science Study (TIMMS) 2003, Singapore was the top performing country at both the fourth and eighth
Rasch Measures of Self-Discipline and Moderation in Mathematics Education
103
grades in mathematics and science (International Association for the Evaluation of Educational Achievement, 2003). The hope was that American students, taught mathematics in the way that Singaporean students are taught, would improve their mathematics scores. The approach has been adopted in about 200 schools nationwide, from rural Oklahoma to the inner cities of New Jersey. Early indications suggested that many United States students taught with textbooks imported from Singapore do perform better in mathematics. Some children who once found the subject frustrating say they now like it (Prystay, 2004). Critics in the Prystay (2004) article asserted that mathematics teaching has been „dumbed down‟ in the USA over the past two decades. They said that too much emphasis is placed on making the subject accessible and fun and not enough on vital, if repetitive, drills such as multiplication tables. Another criticism was that the USA mathematics curricula tended to cover plenty of subject areas but not in sufficient depth (Prystay, 2004). Based on the historical scan of American school mathematics, it is observed that the debate in the USA between proponents of drill-and-practice and of problem solving which started in the late 1950s, continues to wage on into the new millennium. In Malaysia, the teaching and learning of mathematics in many schools has been reported to be too teacher-centred and that students are not given enough opportunities to develop their own thinking skills (Ministry of Education, Malaysia, 2001). In his paper `Directions and Policy: Mathematics Education and National Development‟, the Malaysian Director-General of Education encouraged mathematics teachers to steer the perceptions and interests of students towards the concept of “Mathematics is Fun” (Abd. Rafie Mahat, 2002). It was hoped that by building good attitudes towards learning mathematics, mathematics performance can also improve.
Mathematics Education in Singapore According to Prystay (2004), Singapore and other South East Asian countries take a different tack in mathematics education from that in the USA. Singapore's mathematics curriculum was developed over the past few decades by mathematics experts hired by the Ministry of Education, who continually interviewed mathematics teachers to find out what „works‟ and where students need help. The Singapore elementary textbooks cover only onethird of the topics typically found in USA textbooks, but the material is taught far more thoroughly in Singapore. While rote learning plays a part, students in Singapore also learn to use visual tools to understand abstract concepts. Singapore mathematics texts, for example, ask students to draw bars and other diagrams to visualise problems – a technique called „bar modeling‟. When this strategy is applied consistently over a number of years, children appear to be better able to break down complex problems and do rapid calculations in their head (Prystay, 2004). The primary aim of Singapore‟s mathematics education is to develop student‟s ability in mathematical problem solving. This is done through five inter-related components: (1) development of conceptual understanding; (2) mastery of essential mathematical skills; (3) acquisition of mathematical and thinking processes; (4) inculcation of positive attitudes towards mathematics; and, (5) development of meta-cognitive insight (Tang & Ong, 2001). The development of mathematics concepts take place spirally with topics sequenced appropriately across levels in increasing depth. In this way, students maintain continuity in
104
Liu Shiueh Ling and Russell F. Waugh
learning. Within the spiral approach, students progress from the concrete level to the pictorial level, and then, to the abstract level. This is evident in the coverage of topics from Primary One to Secondary Four. Among the teaching strategies adopted by mathematics teachers in the classroom are the use of information technology, group work or pair work and individual practice. The teaching strategies adopted in the entire process of teaching and developing the topic are in line with the components in the mathematics framework (Tang & Ong, 2001). In Singapore (and around the world), schools were trying to ensure concept and skills attainment in mathematics and there was some movement to making the subject accessible and fun for students (Abd. Rafie Mahat, 2002; Tang & Ong, 2001). There was little emphasis on values education within the mathematics curriculum and, if values were taught at all, it was more often incidental than intentional (Bishop, 2001) and this, it was claimed, was also a problem in the mathematics education of Singapore students (Teo, 1998). A question was being asked – could mathematics be used as a „platform‟ for the discussion of values through the use of a „Caring Thinking‟ approach to character development in Singapore schools, while not neglecting concept, problem-solving and skills development? In October 1996, the Ministry of Education invited a senior academic to lead an external review of the Ministry‟s curriculum and assessment system with a view to including values and „Caring Thinking‟ into the curriculum (Teo, 1998). The external review team submitted its report, Learning, Creating, Communicating: A Curriculum Review, to the Ministry of Education in August 1997. In the report, the team stated that the interplay of three factors: character development, motivation and ability, determined what can be made of any student in our education system (Ministry of Education, Singapore, 1997b). The team also stated that character development is the most important of the three factors because the students must decide how they want to use their knowledge and abilities (Ministry of Education, Singapore, 1997b). As part of the holistic education of students, schools were urged to place greater emphasis on character development within the national education curriculum (Shanmugaratnam, 2005). The Civics and Moral Education curriculum was reviewed and revised to include, not only the „right‟ values, but also social and emotional skills, such as for example, self-management and relationship management skills (Shanmugaratnam, 2005; Ministry of Education, Singapore, n.d.). To develop students‟ personality, knowledge and ability „holistically‟ and “to achieve the Desired Outcomes of Education, schools must place more emphasis on this whole-school approach to character development” (Shanmugaratnam, 2005, p. 7).
CONTEXTUAL BACKGROUND Cognition and Affect in Mathematics Education Predominantly, research in mathematics education has focused on the cognitive aspects of teaching and learning mathematics (Jeynes, 2007; McLeod & Adams, 1989), although there is also much research on attitudes to mathematics (see for example Bishop, 2001; Waugh & Chapman, 2005). The term affect is used here to refer to a wide range of feelings and moods that are generally regarded as something different from pure cognition (McLeod & Adams,
Rasch Measures of Self-Discipline and Moderation in Mathematics Education
105
1989). Beliefs, attitudes and emotions are important factors in research on the affective domain in mathematics education (Mandler, 1984). These terms vary in three aspects: (1) level of intensity of the feelings they represent – the terms beliefs, attitudes and emotions are listed in order of increasing intensity; (2) stability – beliefs and attitudes are generally thought to be relatively stable and resistant to change compared with emotional responses; and, (3) cognitive component – beliefs are mainly cognitive in nature, while emotional responses have a much stronger affective component (McLeod & Adams, 1989). The term attitude is reserved for affective responses that involve positive or negative feelings of moderate intensity and reasonable stability. Attitudes toward mathematics are multidimensional – there are many different kinds of mathematics, and a variety of feelings about each type of mathematics (Leder, 1987). Attitudes toward mathematics appear to develop in two different ways: firstly, attitudes may result from the automatising of a repeated emotional reaction to mathematics. With time, the emotional reaction becomes more automatic and stable and can be measured through use of a questionnaire (McLeod & Adams, 1989). Secondly, attitudes may result from the assignment of an already existing attitude to a new but related task. That is, the attitude from one schema is attached to a second, related schema. Research on attitude towards mathematics has generally proceeded in isolation from more contemporary, cognitive approaches to research on mathematics learning (Leder, 1987). While the dominant thrust in research on mathematics education may have been from the perspective of cognitive science (Jeynes, 2007; McLeod & Adams, 1989), if cognitive science is to have an impact on mathematics education, it is inadequate to avoid the role of affect in mathematics education, complicating though it might be (Bishop, 2001; McLeod & Adams, 1989). Teaching and learning on affect can be incorporated into teaching and learning on cognition. Mandler‟s (1984) theoretical framework for affect analyses the more intense emotional reactions to non-routine mathematics problems and the role of culture, especially the importance of values, in mathematics learning has also been emphasised by Bishop (2001) and Mandler (1984).
Values Education Eyre and Eyre (1993) see a true and universally acceptable „value‟ as one that produces behaviour that is beneficial both to the practitioner and to those on whom it is practised. It is a principle that accomplishes well-being or prevents harm (or does both). It is something that helps or something that prevents hurt. A value is a quality distinguished by: (a) its ability to multiply and increase in our possession even as it is given away; and (b) the fact (even the law) that the more it is given to others, the more it will be returned by others and received by ourselves (Eyre & Eyre, 1993). Raths, Harmin and Simon (1987) describe seven criteria for calling something a value. Their criteria are: (1) choosing freely; (2) choosing from alternatives; (3) choosing after thoughtful consideration of the consequences of each alternative; (4) prizing and cherishing; (5) affirming; (6) acting upon choices; and (7) repeating. They add „those processes collectively define valuing. Results of this valuing process are called values‟ (Raths et al. 1987, p.201). Whether implicitly or explicitly, mathematics teachers teach values. What values are taught implicitly in mathematics? And what opportunities are there for the explicit teaching of values through the teaching of mathematics? Bishop (2001) suggests that understanding more
106
Liu Shiueh Ling and Russell F. Waugh
about values is the key to generating more possibilities for mathematics teaching. However, there is currently, little known about the values that mathematics teachers think they are imparting, or how successful they are in imparting them. There is research on values education per se, but much less is known about what currently happens with values teaching in mathematics classrooms. Many mathematics teachers would not even consider that they are teaching any values when they teach mathematics, compared with Humanities teachers, for example. Changing the perception of mathematics teachers may prove to be one of the biggest hurdles to overcome. Lickona (1991) considers it to be wasting a great opportunity if we fail to use the academic curriculum as a vehicle for developing values and ethical awareness. Teaching values in academic curriculum mainstreams values education, giving ethical concerns the status they deserve in the scheme of educating students and developing their whole personality. One of the objectives of introducing the Pastoral Care and Career Guidance (PCCG) programme in Singapore schools in 1988 was to develop attitudes of caring in students, both toward themselves and others, especially toward those less fortunate than themselves (Thomas et al., 1992). In 2001, the Psychological and Guidance Services Branch launched the PCCG Framework with three key features: (1) whole school approach in the delivery of care; (2) structured programmes to promote the well-being and development of pupils; and, (3) special assistance to specific groups of pupils who need extra help (Ministry of Education, Singapore, n.d.). In 2005, with the review and revision of the Civics and Moral Education curriculum to teach, not only the „right‟ values, but also social and emotional skills, schools were urged to take greater ownership of Civics and Moral Education from 2006, to adapt the curriculum to their students‟ needs (Shanmugaratnam, 2005).
‘Caring Thinking’ in Singapore „Caring Thinking‟ consists of four distinct but interdependent aspects – valuational thinking, affective thinking, active thinking and normative thinking (Lipman, 1994). Requiring emotional sensitivity and depth, valuational thinking taps into two dimensions: Firstly, appreciating concrete things for their sensuous and aesthetic appeal, rather than material worth; and secondly, understanding abstract human qualities, including attitudes, behaviours and values (Lipman, 1994). These form part of affective thinking. Valuational thinking (or active thinking) results in „Values Realisation‟ – awareness of what one‟s values are and the ethical principles that guide one‟s actions. Following „Values Realisation‟ is the „Values Clarification‟ process (or normative thinking), where time is needed to reflect on one‟s relationship with people, objects, ideas and situations. This „Values Clarification‟ process involves choosing from alternatives, believing in the importance of your chosen values or beliefs and asserting your beliefs in communication with others (Lipman, 1994). Values clarification involves a process of choosing from alternatives, believing and rejoicing in the importance of the chosen value and affirming this value to others in one‟s action, which are related to „good character‟ (Brunt, 1996). „Good Character‟ involves three components: (1) moral knowing – intellectual side of character which involves personal choice; (2) moral feeling – emotional side of character which shows how much we care about the values; and, (3) moral action – doing what one knows and feels to be right (Lickona,
Rasch Measures of Self-Discipline and Moderation in Mathematics Education
107
1991). These aspects are also part of the discussion in the „Caring Thinking‟ mathematics curriculum module. The „Values Clarification‟ process in the mathematics module in Singapore involves choosing one‟s principles from alternatives and abiding by them in one‟s words and actions, themes supported by Brunt (1996), Lipman (1994) and Lickona (1991). The present study investigates „Caring Thinking‟ when used in conjunction with a mathematics module as a means of developing values in students through the „Caring Thinking‟ way by creating the context in mathematics teaching to allow students to engage in valuational thinking, values realisation and values clarification. For example, in the First Component, students are taught probability concepts, knowledge and problem-solving, with a focus on the mathematics. During the Second Component, Discussion on Values, students are shown an introductory video on a probability game. After the distribution of the lesson notes, students read the article on their own “To me, he is as good as dead”, about a person becoming addicted to gambling. In pairs or in groups, students discuss and identify the personal value traits needed to prevent one from falling into a similar fate. A few groups are invited to present the personal values that they believe are important and explain why those values are chosen. Based on the discussion, the values of Self-Discipline, Moderation, Dependability and Responsibility are highlighted and emphasised by the teacher and the students. At the conclusion of the lesson, the class decides on the motion for debate during the subsequent lessons: (1) an increase in legalised gambling results in a growing (or lesser) number of Singaporeans addicted to gambling; or (2) building casinos results in more (or less) Singaporeans addicted to gambling. Students are required to gather information and evidence to support their arguments before the debate.
THE PRESENT STUDY Aims The study aims to develop interval level, unidimensional scales of Self-Discipline and Moderation (reported in this Chapter Six), and of Dependability and Responsibility (reported in the next Chapter Seven), with attitude items linked to behavior items, based on conceptual models of the four values, using a Rasch measurement model.
SIGNIFICANCE AND RELEVANCE Since the introduction of „Caring Thinking‟ into the school‟s Thinking Model in 2003, there has been little research done at the school level on the impact of „Caring Thinking‟. Despite the generally positive implications of „Caring Thinking‟ on students‟ moral development, virtually nothing is known about how „Caring Thinking‟ affects the social attitudes and behaviors of students in our school. This study aims to investigate whether the „Caring Thinking‟ way presents an effective way to teach values implicitly in conjunction with a mathematical probability module. The school aims to develop students who think critically, creatively and with caring values. It aims to place „Caring Thinking‟ as an important part of the school curriculum and
108
Liu Shiueh Ling and Russell F. Waugh
programmes because it is thought, from anecdotal evidence, that while most of the students are critical thinkers and some are creative thinkers, very few of them think with caring values. The results of this study will have implications on: (1) the way caring values are taught in school – not didactically in moral education lessons, but through subject curriculum and school programmes; (2) the role of teachers in the classroom – not just as deliverers of subject content, but also as teachers of caring values; and, (3) the way students learn caring values – the values will come alive for them because they are contextualised in the real world through subject matter. The present study aims to develop four interval-level, unidimensional scales of SelfDiscipline, Moderation, Dependability and Responsibility, with attitude items linked to behavior items, based on conceptual models of the four variables. The two perspectives (attitude and behavior) in each of the four scales will be linked by students‟ answering each item from both perspectives and the data analyzed together to create a scale in which the conceptualized relationships will be tested. The attitude perspective of Self-Discipline, for example, includes whether the students values Self-Discipline and whether they aim to abide by it in actions. The behavior perspective of Self-Discipline includes whether students actually practice Self-Discipline in their thoughts, words and deeds. The successful development of these four interval level, unidimensional scales of Self-Discipline, Moderation, Dependability and Responsibility have not been produced anywhere in the world before this study although attitudes and behavior have been measured on the same Rasch scale for other variables. The development of the current scales could pave the way for developing scales for measuring other values. The strength of these scales lies in being able to attach quantitative performance indicators to an attitude towards a value and to behaviors directly connected to that value.
LIMITATIONS The main limitations of the study include: (1) the data obtained only apply to the school in this study and not to Singapore as a whole or to any other country; (2) the data obtained only apply to secondary four (year 10) students and not to any other grade level; and, (3) the data obtained only apply to male students and not to female students because the school in this study is a school for boys from secondary one (year 7) to secondary four (year 10).
RASCH MEASUREMENT FOR SELF-DISCIPLINE AND MODERATION The initial Rasch analysis for Self-Discipline showed that two out of 16 items did not fit the Rasch measurement model and these items had their response categories collapsed from four to two. The initial Rasch analysis for Moderation showed that four out of 16 items did not fit the Rasch measurement model and these items were deleted. A brief explanation of these analyses is given first, using various statistical indicators. The final analyses in which 16 items for Self-Discipline and 12 items for Moderation fitted the measurement model satisfactorily with acceptable reliability, is described in more detail here, through discussions and Tables 1 to 8. This is followed by a discussion of the item characteristic curves and the
Rasch Measures of Self-Discipline and Moderation in Mathematics Education
109
category response curves of some items (see Figures 1 to 6), as well as the person-item threshold distribution graphs (targeting) (see Figures 7 to 12) for the participants. Some commentary on the non-fitting items is given and then some commentary on the good-fitting items is given, followed by the presentation of the scale of Moderation (see Table 9) and the scale of Self-Discipline (see Table 10). This chapter ends with the summary of the main findings of the RUMM programme analysis of the pretest data for Self-Discipline and Moderation.
INITIAL PRETEST RASCH ANALYSIS: SELF-DISCIPLINE (N=225, I=16) AND MODERATION (N=193, I=12) Although 277 students (139 from the experimental group and 138 from the control group) was the total number of students from the eight participating classes, only 256 students participated in the survey. Twenty-one students did not participate because they were either on sick leave or they were out representing the school in competitions on the days the survey was carried out. Of the 256 participants, 30 (12 from the experimental group and 18 from the control group) submitted incomplete questionnaires. Thus, subsequently only 226 responses to the questionnaire were entered into an excel file. Participants in the control group were assigned serial numbers from „1001‟ to „1105‟ and participants in the experimental group were assigned serial numbers from „1106‟ to „1226‟. For each participant, the responses to the questionnaire were entered in terms of response categories, „1‟, „2‟, „3‟ and „4‟. „1‟ was for „never‟, „2‟ was for „rarely‟, „3‟ was for „sometimes‟, and „4‟ was for „often‟. For the Self-Discipline scale, at the beginning, the computer programme discarded one student because of extreme data; that is, the student entered all „4s‟ in response to the items in the survey (In Rasch measurement, measures cannot be estimated from extreme data). Thus, 225 participants remained out of 226 for subsequent analysis. There were eight items in the questionnaire, each answered in two perspectives, „I aim for this‟ and „I actually do this‟, thus making a Rasch item analysis of 16 items, item numbers 1 to 16. A linear scale was created with the data from the 16 items but two items (11 and 13) had to have their response categories collapsed from four to two („1‟=„0‟, „2‟=„0‟, „3‟=„1‟, „4‟=„1‟) because of disordered thresholds, meaning that the response categories of 1,2,3 and 4, were not answered consistently in line with their conceptual ordering (the reason for this is unknown). The overall fit to the measurement model was satisfactory (item-trait chi-square = 63.18, df = 48, p = 0.07). This means that there was reasonable agreement among participants about the difficulties of the items along the scale (although the agreement was not perfect). The final analysis for Self-Discipline involved 16 items and this analysis is reported in some detail in a following section. For the Moderation scale, at the beginning, the computer programme discarded 33 students because of extreme data, thus leaving 193 participants out of 226 for subsequent analysis. There were also eight items in the questionnaire each answered in two perspectives, „I aim for this‟ and „I actually do this‟, thus making a Rasch item analysis of 16 items, item numbers 17 to 32. Four items (21, 25, 26 and 32) showed disordered thresholds meaning that the students did not use the response categories consistently and logically in line with their
110
Liu Shiueh Ling and Russell F. Waugh
conceptual ordering. The four non-fitting items were deleted and the response categories of the 12 remaining items were collapsed from four to two („1‟=„0‟, „2‟=„0‟, „3‟=„1‟, „4‟=„1‟). A linear scale for Moderation was created with the data from the 12 items and the overall fit to the measurement model was satisfactory (item-trait chi-square = 49.84, df = 36, p = 0.06). This means that there was satisfactory, although not perfect, agreement about the difficulties of the items along the scale. The final analysis involved 12 items and this analysis is reported in some detail in a following section.
FINAL ANALYSIS SELF-DISCIPLINE (N=225, I=16) AND MODERATION (N=193, I=12) Summary of Fit Statistics Table 1 is a summary of the fit statistics for Self-Discipline. The statistics show a global fit residual mean of -0.43 logits and a standard deviation 0.82 for the items and a global fit residual mean of -0.39 logits and a standard deviation of 1.17 for the students. Table 2 is a summary of the fit statistics for Moderation. It shows a global fit residual mean of -0.18 logits and standard deviation 1.23 for the items and a global fit residual mean of -0.31 logits and a standard deviation of 0.91 for the students. These are close to ideal fit residuals of mean near zero and standard deviation near one which shows an overall satisfactory fit to the measurement model. That is to say there is a good consistency of item parameters (person measures and item difficulties) for the measure. Table 1. Summary of Fit Statistics for Self-Discipline from the Rasch Analysis (N = 226, I = 16) ITEM-PERSON INTERACTION ITEMS Location Mean 0.00 SD 0.87 ITEM-TRAIT INTERACTION Total Item Chi-Square Total Degree of Freedom Total Chi-Square Probability
Fit Residual -0.43 0.82 63.19 48.00 0.07
PERSONS Location Fit Residual 1.69 -0.39 0.93 1.17 RELIABILITY INDICES Separation Index 0.77 Cronbach Alpha 0.81 POWER Power is GOOD [Based on Separation Index of 0.77]
Notes on Table 1 1.The Index of Student Separation is the proportion of observed variance that is considered true and is reasonably high (77%). It means that the measures are separated satisfactorily in comparison to the errors. 2.The item and student global fit statistics have an expected mean of near zero and a standard deviation of near one, when the data fit the measurement model. The fit statistics in this case are satisfactory. 3. The item-trait interaction chi-square test indicates that students of differing self-discipline level responded to the item difficulties according to what is expected of them by the measurement model and that a unidimensional trait has been measured.
Rasch Measures of Self-Discipline and Moderation in Mathematics Education
111
4. All numbers are given to two decimal places because the errors are two decimal places (see Table 6.3). Power is the ability to test any non-compliance with the measurement model and, in this case, is good.
Table 1 also shows the Cronbach Alpha (0.81) and the Separation Index (0.77) for the 16 items for Self-Discipline. This shows that the Self-Discipline scale has good traditional internal scale reliability and that the measures are separated satisfactorily in comparison to the errors. However, the Cronbach Alpha of 0.68 and 0.70 for Separation Index for the Moderation (Table 6.2) shows that, while the reliability can be accepted for this study, it would need improving in a future use. The RUMM programme rates the overall power of test-of-fit for the two scales as good (see Tables 6.1 and 6.2) which shows that there is adequate power to detect any major non-compliance with the measurement model. Table 2. Summary of Fit Statistics for Moderation from the Rasch Analysis (N = 226, I = 12) ITEM-PERSON INTERACTION ITEMS Location Mean 0.00 SD 1.09 ITEM-TRAIT INTERACTION Total Item Chi-Square
Fit Residual -0.18 1.23 49.84
PERSONS Location Fit Residual 1.31 -0.31 1.25 0.91 RELIABILITY INDICES Separation Index 0.70
Total Degree of Freedom
36.00
Cronbach Alpha
Total Chi-Square Probability
0.06
0.68
POWER Power is GOOD [Based on Separation Index of 0.70]
Notes on Table 2: 1. The Index of Student Separation is the proportion of observed variance that is considered true and is reasonably high (70%). This means that the measures are separated satisfactorily in comparison to the errors of measurement. 2. The item and student global fit statistics have an expected mean of near zero and a standard deviation of near one, when the data fit the measurement model. The fit statistics in this case are satisfactory. 3. The item-trait interaction chi-square test indicates that students of differing measuring levels responded to the item difficulties according to what is expected of them by the measurement model and that a unidimensional trait has been measured. 4. All numbers are given to two decimal places because the errors are two decimal places (see Table 6.4). 5. Power is the ability to test any non-compliance with the measurement model and, in this case, is good.
Individual Item-Fit The Locations are the item difficulties in logits. For the Self-Discipline scale, as shown in Table 3, of the 16 items, the easiest is item 9 (difficulty -1.58 logits) and the most difficult is item 4 (difficulty +1.26 logits). For the Moderation scale, of the 12 items, the easiest is item 18 (difficulty -1.73 logits) and the most difficult is item 28 (difficulty +1.81 logits) as shown
Liu Shiueh Ling and Russell F. Waugh
112
in Table 4. The table also shows the Standard Errors in conjunction with the estimates of the Locations (Difficulties) of the items. The Standard Errors are smaller in the region where there are more students. Table 3. Item Difficulties (Locations), Standard Errors (SE), Residuals and Fit to the Measurement Model for the Self-Discipline scale Item 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
Difficulty -1.03 +0.69 -0.03 +1.26 -0.90 +0.50 -0.19 +0.28 -1.58 +0.55 -1.40 +0.13 -0.59 +0.62 +0.57 +1.14
SE 0.15 0.10 0.11 0.10 0.13 0.11 0.11 0.11 0.12 0.11 0.29 0.10 0.21 0.09 0.09 0.09
Residual -1.96 +0.84 -0.57 -0.26 -0.95 -0.36 +0.00 +0.45 -0.85 -0.28 -1.57 +0.17 -1.67 +0.63 -0.57 +0.11
DegFree 208.25 208.25 208.25 208.25 208.25 208.25 208.25 208.25 208.25 208.25 208.25 208.25 208.25 208.25 208.25 208.25
DataPts 225 225 225 225 225 225 225 225 225 225 225 225 225 225 225 225
ChiSquare 7.42 5.21 7.53 3.27 3.11 1.30 2.06 1.21 3.13 2.07 6.13 0.87 10.50 6.69 2.13 0.55
Prob 0.06 0.16 0.06 0.35 0.38 0.73 0.56 0.75 0.37 0.56 0.11 0.83 0.01 0.08 0.54 0.91
Notes on Table 3: 1.The Difficulty of each item is in logits (the log odds of giving a positive response to an item). 2.SE is standard error in logits. They are smaller in the region where there are more students. 3.Residual is the difference between the observed and expected response. 4.Prob is the probability based on the chi-square fit to the measurement model and is dependent on sample size.
Table 4. Item Difficulties (Locations), Standard Errors (SE), Residuals and Fit to the Measurement Model for the Moderation scale Item 17 18 19 20 22 23 24 27 28 29 30 31
Difficulty -1.49 -1.73 -0.78 -0.03 +0.57 +0.05 +1.40 +1.20 +1.81 -0.21 -0.48 -0.31
SE 0.24 0.26 0.20 0.17 0.16 0.17 0.16 0.16 0.17 0.18 0.18 0.18
Residual -1.24 -1.30 -1.65 +0.48 +0.60 -1.36 -0.01 +0.75 +0.64 -1.22 -0.36 +2.47
DegFree 176.00 176.00 176.00 176.00 176.00 176.00 176.00 176.00 176.00 176.00 176.00 176.00
DataPts 193 193 193 193 193 193 193 193 193 193 193 193
ChiSquare 1.62 2.31 3.41 3.84 2.86 6.54 2.03 2.91 3.15 4.17 3.17 13.84
Prob 0.66 0.51 0.33 0.28 0.41 0.09 0.57 0.41 0.37 0.24 0.37 0.00
Notes on Table 3 are the same as for Table 4.
Tables 3 and 4 have a column that shows the residuals. These are the differences between the actual responses and the responses estimated from the Rasch measurement parameters. Standardised residuals are generally expected to be within the range of -2 and +2. For the
Rasch Measures of Self-Discipline and Moderation in Mathematics Education
113
Self-Discipline scale, Table 6.3 shows that all the 16 items have acceptable residuals. For the Moderation scale, Table 4 shows that except for one item, item number 31, the rest of the items have acceptable residuals. Item 31 was not deleted because its deletion made for a worse overall fit of all the items to the measurement model. Tables 3 and 4 also have columns showing the chi-square and its associated probability. This is a statistic that is calculated from the discrepancies between the observed mean in the class intervals and the expected values according to the measurement model. If the probability has a value of less than 0.01, then it implies that the discrepancy between the observed mean and the expected value is large relative to chance and that item should be examined. For the Self-Discipline scale, Table 3 shows that all 16 items have acceptable probabilities of values more than 0.01 and 15 out of 16 items have acceptable probabilities of values more than 0.05. For the Moderation scale, Table 4 shows that except for one item, item number 31, the rest of the items have acceptable probabilities. However, for a large sample, this probability is often low, thus it is important to consider the graphical display before discarding the item (see Figures 1 and 2).
Item Threshold Distribution Table 5 shows that the 16 items for the Self-Discipline scale that fit the Rasch model are polytomous, meaning that there are four ordered response categories, „0‟ for „never‟, „1‟ for „rarely‟, „2‟ for „sometimes‟, and „3‟ for „often‟, for each item except for item numbers 11 and 13. These items have two ordered response categories, instead of four ordered response categories. Of the 16 Rasch items, item number 1 corresponds to the perspective „I aim for this‟ and item number 2 corresponds to the perspective „I actually do this‟ and so on. That is, the odd numbered items correspond to the perspective „I aim for this‟ and the even numbered items correspond to the perspective „I actually do this‟. The 16 items that fit the Rasch model are a mixture of these two perspectives. Table 5. Item Specification of the Self-Discipline Scale Rasch Item No. (Original No) 1
Test items
Original categories 4
Thresholds
Polytomous
Response categories 4
2
Polytomous
4
4
3
3
Polytomous
4
4
3
4
Polytomous
4
4
3
5
Polytomous
4
4
3
6
Polytomous
4
4
3
7
Polytomous
4
4
3
8
Polytomous
4
4
3
9
Polytomous
4
4
3
10
Polytomous
4
4
3
11
Dichotomous
2
4
1
3
Liu Shiueh Ling and Russell F. Waugh
114 Rasch Item No. (Original No) 12
Test items
Original categories 4
Thresholds
Polytomous
Response categories 4
13
Dichotomous
2
4
1
14
Polytomous
4
4
3
15
Polytomous
4
4
3
16
Polytomous
4
4
3
3
Notes for Tables 5, 6 and 7 1. Polytomous means that each item has more then two ordered response categories and dichotomous means that there are two response categories. 2. Key is the number of ordered response categories in the final scale. 3. Category refers to the number of original responses to each item. 4. Thresholds are points between adjacent categories where there are odds of 1:1 of answering in adjacent categories. Where there are four response categories, there are three thresholds and, where there are two response categories, there is one threshold.
Table 6 shows three thresholds calculated by the RUMM computer programme, as there are four response categories to each item, except for item numbers 11 and 13. A threshold is a point between two response categories where there is an equal probability of answering either category. The first threshold shows the point between response categories „0‟ and „1‟, numbered according to the Rasch programme, where there is equal probability of responding either a „0‟ or „1‟. The second threshold shows the point between categories „1‟ and „2‟, numbered according to the Rasch program, where there is equal probability of responding either a „1‟ or „2‟. The third threshold shows the point between categories „2‟ and „3‟, numbered according to the Rasch program, where there is equal probability of responding either a „2‟ or „3‟. The thresholds are ordered in line with the ordering of the response categories showing that the students have answered the response categories consistently and logically. Table 7 shows that 12 items for the Moderation scale fit the Rasch model. The four items that do not fit, item numbers 21, 25, 26 and 32, were deleted. The 12 remaining items have two ordered response categories, „0‟ and „1‟, instead of, four ordered response categories. The 12 items that fit the Rasch model are a mixture of two perspectives – the odd numbered items correspond to the perspective „I aim for this‟ and the even numbered items correspond to the perspective „I actually do this‟. Table 6. Item Thresholds – Uncentralised (Self-Discipline Scale) Item
Difficulty
Mean
Item 1 Item 2 Item 3 Item 4 Item 5 Item 6 Item 7 Item 8 Item 9
+1.03 +0.69 -0.03 +1.26 -0.90 +0.50 -0.19 +0.28 +1.58
+1.03 +0.69 -0.03 +1.26 -0.90 +0.50 -0.19 +0.28 +1.58
Thresholds 1 -1.95 -0.44 -1.01 -0.75 -2.63 -0.80 -0.95 -0.85 -5.12
2 -1.15 +0.16 -0.28 +1.09 -0.59 -0.28 -0.24 -0.26 -0.58
3 +0.01 +2.36 +1.21 +3.45 +0.52 +2.58 +0.60 +1.93 +0.97
Rasch Measures of Self-Discipline and Moderation in Mathematics Education Item
Difficulty
Mean
Item 10 Item 11 Item 12 Item 13 Item 14 Item 15 Item 16
+0.54 -1.40 +0.13 -0.59 +0.62 +0.57 +1.14
+0.54 -1.40 +0.13 -0.59 +0.62 +0.57 +1.14
Thresholds 1 -1.21 -1.40 -1.42 -0.59 -0.26 +0.24 -0.10
2 +0.02 n/a +0.10 n/a +0.53 +0.45 +1.26
115
3 +2.83 +1.70 +1.58 +1.02 +2.26
Table 7. Item Specification of the Moderation Scale Rasch Item No. (Original No.) 17 18 19 20 22 23 24 27 28 29 30 31
Test responses Dichotomous Dichotomous Dichotomous Dichotomous Dichotomous Dichotomous Dichotomous Dichotomous Dichotomous Dichotomous Dichotomous Dichotomous
Response categories 2 2 2 2 2 2 2 2 2 2 2 2
Original categories 4 4 4 4 4 4 4 4 4 4 4 4
Thresholds 1 1 1 1 1 1 1 1 1 1 1 1
Table 8. Item Thresholds – Uncentralised (Moderation Scale) Item Item 17 Item 18 Item 19 Item 20 Item 22 Item 23 Item 24 Item 27 Item 28 Item 29 Item 30 Item 31
Item Difficulty -1.49 -1.73 -0.78 -0.03 +0.57 +0.05 +1.40 +1.20 +1.81 -0.21 -0.48 -0.31
Thresholds 1 -1.49 -1.73 -0.78 -0.03 +0.57 +0.05 +1.40 +1.20 +1.81 -0.21 -0.48 -0.31
Notes on Table 8: 1. There is one threshold since each item has two response categories. Thresholds are points between adjacent categories where there are odds of 1:1 of answering in adjacent categories. In this case there are only two response categories and hence only one threshold which is the item difficulty. 2. The item difficulties and thresholds are given in standard units called logits, the log odds of successfully answering the item.
Table 8 shows the thresholds (item difficulties) calculated by the RUMM computer programme where there are two response categories to each item. The threshold shows the point between response categories „0‟ and „1‟, numbered according to the Rasch programme,
116
Liu Shiueh Ling and Russell F. Waugh
where there is equal probability of responding either a „0‟ or „1‟. The item difficulties range from -1.73 logits to +1.81 logits. The ordering of the item difficulties is in line with their conceptual order, where attitude and behaviour for the corresponding item content fit the measurement model, for all item pairs except items 17 and 18. For example, item 19 refers to attitude and item 20 refers to behaviour for the same item content and attitude (-0.73 logits) is easier than behaviour (-0.03 logits). The reverse ordering for item 17 attitude (-1.49 logits) and item 18 behaviour (-1.73 logits) is understandable when the wording is examined. The wording refers to avoid speaking vulgarities.
Item Characteristic Curves
Note: This item discriminates rather well, as required for compliance with the Rasch measurement model. Figure 1. Item Characteristic Curve for item 17 (Moderation scale).
Figure 1 shows the item characteristic curve for item 17 of the Moderation scale. This is an easy item (the location difficulty is -1.49 logits). The observed means, shown as dots, in the four class intervals are rather close to the curve. This shows that the item fits rather well to the theoretical curve of the Rasch model according to this criterion (the chi-square probability of fit is 0.66). It means that the item discriminates between the students rather well as specified by the model; that is, item 17, „I avoid using vulgarities when I speak with my elders (I aim for this)‟, has expected values close to those necessary for a good fit to the measurement model. The item characteristic curves for 11 out of 12 items were satisfactory and are not all reported here. One of the 12 items in the Moderation scale, item 31, did not fit the Rasch model satisfactorily, but its removal and subsequent re-analysis did not improve the fit of the other items and, because it conceptually fits in the scale, it was left in.
Rasch Measures of Self-Discipline and Moderation in Mathematics Education
117
Note on Figure 2: This item does not fit well, as specified by the Rasch measurement model. Figure 2. Item Characteristic Curve for item 31 (Moderation scale).
This is an easy Moderation item (the location difficulty is -0.31 logits) but it is not discriminating like it should. The observed means, shown as dots, in the four class intervals are not close to the characteristic curve (ogive) as required by the measurement model for good discrimination. This shows why the item does not fit well to the theoretical measurement model of Rasch (the chi-square probability of fit is 0.00, to two decimal places). It means that, for this item, many of them are obtaining the same expected value when their overall measure is quite different. For the Self-Discipline scale, all 16 items fit the Rasch model with chi-square probabilities (p) greater than 0.01 and 15 out of 16 items fit the Rasch model with p greater than 0.05. The item characteristic curves for all 16 items in the Self-Discipline scale were satisfactory and are not reported here. As part of the study of the fit of the data to the Rasch measurement model, it was also necessary to check whether the items work in the same way for students in one group as they do for students in another group. This can be done by considering the Item Characteristic Curves differentiated by group. The requirement is, as best as can be estimated from the data, for the same location of a person, the expected value on an item should be the same, irrespective of what group the person belongs. That is, the test is that there is no differential item functioning (DIF) relative to whether students are in one group or another. Figure 3 shows item 2 for the Self-Discipline scale where the two groups (students with a low level of participation in community service and students with a high level of participation in community service) effectively have the same item functioning. Although the curves are different, the difference between students with similar measures of Self-Discipline but different levels of participation in community service is not statistically significant (F=0.10, df=225, 1, p = 0.75).
118
Liu Shiueh Ling and Russell F. Waugh
Note: The item does not significantly discriminate between students with a low level of participation in community service and students with a high level of participation in community service F=0.10, df=225,1, p = 0.75). Figure 3. Item Characteristic Curve for item 2 (Self-Discipline scale) by Level of Participation in Community Service.
Figure 4. Item Characteristic Curve for item 13 (Self-Discipline scale) by Level of Participation in Community Service.
In another example, Figure 4 shows the characteristic curve for item 13 for the SelfDiscipline scale. Although the curves are separated by low and high levels of participation in community service, this difference is not statistically significant (F=5.48, df=225,1, p=0.02) at the 0.01 level. There was no statistically significant discrimination between the low and high levels of participation in community service groups on each of the remaining 14 goodfitting items for the Self-Discipline scale, as displayed by their item characteristic curves.
Rasch Measures of Self-Discipline and Moderation in Mathematics Education
119
Response Category Curves Figure 5 shows the response category curves for item 4 of the Self-Discipline scale. The vertical axis represents the probability of responding in a particular response category and the horizontal axis represents the students‟ person location (measure) in logits. The Rasch programme converts the response categories 1, 2, 3, and 4 to 0, 1, 2, and 3 respectively. In Figure 5, the category 0 response curve indicates that a student with a Self-Discipline measure of -5.0 logits (Person Location) has about a 0.98 probability of responding in this category („Never‟), whereas a student with a Self-Discipline measure of 3.0 logits has a near zero probability of responding in the same category for item 4. The category 1 curve of Figure 5 shows that a student with a Self-Discipline measure of about 0.1 logits has a probability of about 0.56 of responding in the category („Rarely‟) for item 4, whereas a student with a Self-Discipline measure of -5.0 logits has a probability of about 0.01 of responding in the same category. For category curve 2, a student with a Self-Discipline measure of about 2.3 logits has a probability of about 0.62 of responding in the category („Sometimes‟) for item 4, whereas a student with a Self-Discipline measure of -2.0 logits has a near zero probability of responding in the same category. Finally, for category curve 3, a student with a Self-Discipline measure of about 0.0 logits has a probability near zero of responding in the category („Often‟) for item 4, whereas a student with a Self-Discipline measure of 7.0 logits has a probability of about 0.97 of responding in the same category. This shows that the students used the four response categories for item 4 logically and consistently.
Figure 5. Response Category Curves for Item 4 (Self-Discipline scale).
When the categories are ordered, it is expected that the boundaries between the categories should also be ordered. Figure 5 shows such a case for the Rasch item 4 of the Self-Discipline scale with four ordered categories. The thresholds (Υ1 ,
Υ2 and Υ3) which define the
categories are estimated in the model and are ordered. They show the points where the probability of responding either 0 or 1, 1 or 2, and 2 or 3 respectively, are equally likely. These thresholds were ordered in line with the order of the response categories for item 4 and for all the other Self-Discipline items. The category response curves for all 16 items were
Liu Shiueh Ling and Russell F. Waugh
120
checked and they were found to be satisfactory and operating as they should when the data fit the measurement model. Figure 6 shows the Response Category Curves for item 17 of the Moderation scale. The Rasch programme collapsed the response categories 1 and 2 to 0, and, 3 and 4 to 1. In Figure 6.6, the category 0 response curve indicates that a student with a Moderation measure of -6.0 logits (Person Location) has about a 0.99 probability of responding in this category, whereas a student with a Moderation measure of 3.0 logits has about a 0.01 probability of responding in the same category for item 17. For category curve 1, a student with a Moderation measure of about -6.0 logits has about a 0.01 probability of responding in the category for item 17, whereas a student with a Moderation measure of 3.0 logits has a probability of about 0.99 of responding in the same category. This shows that the students used the two response categories for item 17 logically and consistently. The threshold
Υ1 shows the point where the probability of responding either 0 or 1 is
equally likely. Item 17, „I avoid using vulgarities when I speak with my elders‟, in the attitude perspective („I aim for this‟), fits the Rasch model well (the chi-square probability is 0.66). The category response curves for all 12 items were checked and they were found to be satisfactory and operating as they should.
Figure 6. Response Category Curves for Item 17 (Moderation scale).
Person-Item Threshold Distribution (Targeting) Figures 7, 8 and 9 show the distribution of student measures and item thresholds for the 226 secondary four students on the same linear Self-discipline scale. The targeting of the items against the student measures of Self-Discipline is good (they cover about the same range) but, in any future use of the scale for these students, some very hard items need to be added to cater for those students who have very high measures of Self-Discipline.
Rasch Measures of Self-Discipline and Moderation in Mathematics Education
121
Figure 7. Person-Item Threshold Graph showing the distribution of 226 student measures and the 16 item thresholds (Self-Discipline scale).
RASCH MEASURES BY INDEPENDENT VARIABLES There were three independent variables, considered as indicator variables: Type of Home (2/3 room public housing, 4/5 room public housing, executive public apartment, private dwelling); Level of Compliance with parents and teachers (never, rarely, sometimes, often, always); and Participation in Community Service (low, high). Type of Home is an indicator variable which measures „something‟ happening in the home that is expected to influence Self-Discipline and Moderation (possibly the educational values exerted by the parents on the students with higher levels associated with larger houses and higher levels of money and education that, in turn, produce higher Self-Discipline and Moderation). Level of Compliance is an indicator variable of docility with more docility associated with higher compliance that, in turn, produces higher Self-Discipline and Moderation. Participation in Community Service is an indicator variable associated with helping others and this, in turn, is associated with higher levels of Self-Discipline and Moderation. The measures of Self-Discipline and Moderation were made against each of the three independent variables and are now reported.
Self-Discipline by Housing Type From Figure 8, the mean Self-Discipline measure for students who live in two- or threeroom public housing apartments is 2.06 logits (SD = 0.96, N = 18). The mean Self-Discipline measure for students who live in four-room or five-room public housing apartments is 1.66 logits (SD = 0.89, N = 86). This is not statistically significantly different (t=1.66, df=102, p=0.05) at the p=0.01 level.
122
Liu Shiueh Ling and Russell F. Waugh
Note: There is a colour error in the RUMM2020 computer program. Brown in the bar graph is represented by purple in the numbers on the top right hand side. Green in the bar graph is represented by blue in the numbers. Purple in the bar graph is represented by green in the numbers. Figure 8. Person-Item Threshold Graph showing the distribution of student measures and item thresholds by Housing Type (Self-Discipline scale).
The mean Self-Discipline measure for students who live in executive public housing apartments is 1.38 logits (SD = 1.04, N = 20) and the mean Self-Discipline measure for students who live in private apartments or houses is 1.74 logits (SD = 0.90, N = 94). This is not statistically significantly different (t=1.58, df=112, p=0.04). The mean Self-Discipline measure for students who live in 2/3 room public housing is 2.06 logits (SD = 0.96, N = 18) and the mean Self-Discipline measure for students who live in executive public housing is 1.38 logits (SD = 1.04, N = 20). This is not statistically significantly different (t=2.03, df=36, p=0.04).
Self-Discipline by Level of Compliance From Figure 9, the mean Self-Discipline measure for students who never comply with their parents is 0.05 logits (SD = 0.30, N = 3). The mean Self-Discipline measure for students who rarely comply with their parents is 0.84 logits (SD = 1.33, N = 3). The mean SelfDiscipline measure for students who sometimes comply with their parents is 1.53 logits (SD = 0.84, N = 56). The mean differences between never/rarely and sometimes for Level of Compliance are statistically significantly different (t=2.89, df=60, p<0.001). The mean SelfDiscipline measure for students who often comply with their parents is 1.72 logits (SD = 0.89, N = 137). The mean differences between sometimes and often for Level of Compliance is not statistically significantly different (t=1.39, df=191, p=0.09). The mean Self-Discipline measure for students who always comply with their parents is 2.16 logits (SD = 0.96, N = 27) and this is statistically significantly different from the mean for sometimes (t=12.97, df=81, p<0.001). In terms of groups, students‟ mean Self-Discipline measure increases as their Level of Compliance with their parents increases.
Rasch Measures of Self-Discipline and Moderation in Mathematics Education
123
Note: There is a colour error in the RUMM2020 computer program. Brown in the bar graph is represented by purple in the numbers on the top right hand side. Green in the bar graph is represented by blue in the numbers. Purple in the bar graph is represented by green in the numbers. Figure 9. Person-Item Threshold Graph showing the distribution of student measures and item thresholds by Level of Compliance (Self-Discipline scale).
Self-Discipline by Level of Participation in Community Service From Figure 10, the mean Self-Discipline measure for students who have a low level of participation in community service is 1.47 logits (SD = 0.92, N = 99). The mean SelfDiscipline measure for students who have a high level of participation in community service is 1.864 logits (SD = 0.90, N = 127). This is not statistically significantly different (t=1.31, df=224, p=0.09) and so it is concluded that Level of Participation in Community Service is not an indicator of some variable that is influencing Self-Discipline.
Note: There is a colour error in the RUMM2020 computer program here. Maroon in the bar graph represents red in the numbers at the top right hand side of the graph. Green in the bar graph represents blue in the numbers on the top right side of the graph. Figure 10. Person-Item Threshold Graph showing the distribution of student measures and item thresholds by Level of Community Service (Self-Discipline scale).
124
Liu Shiueh Ling and Russell F. Waugh
Moderation by Housing Type From Figure 11, the mean Moderation measure for students who live in two-room or three-room public housing apartments is 1.80 logits (SD = 1.14, N = 18). The mean Moderation measure for students who live in four-room or five-room public housing apartments is 1.17 logits (SD = 1.18, N = 86). This is not statistically significantly different at the p=0.01 level (t=2.03, df=102, p=0.03). The mean Moderation measure for students who live in private houses is 1.26 logits (SD=1.30, N=94) and those who live in executive public housing apartments is 1.55 logits (SD = 1.30, N = 20). The mean difference between students who live in 2/3 room public housing and those who live in private houses or apartments is not statistically significant (t=1.60, df=110, p=0.06). The mean Moderation measure for students who live in private apartments or houses is 1.27 logits (SD = 1.30, N = 94). This mean is not statistically significantly different from those who live in executive apartments or houses (t=0.90, df=114, p=0.19). It is concluded that size of public housing is not an indicator variable of something that is influencing Moderation in students.
Note: There is a colour error in the RUMM2020 computer program. Green in the bar graph represents blue in the numbers on the top right hand side. Brown in the bar graph is represented by purple in the numbers. Purple in the bar graph is represented by red in the numbers. Figure 11. Person-Item Threshold Graph showing the distribution of student measures and item thresholds by Housing Type (Moderation scale).
Moderation by Level of Compliance From Figure 12, the mean Moderation measure for students who never comply with their parents is -0.55 logits (SD = 0.46, N = 3). The mean Moderation measure for students who
Rasch Measures of Self-Discipline and Moderation in Mathematics Education
125
rarely comply with their parents is -1.00 logits (SD = 0.71, N = 3). The mean Moderation measure for students who sometimes comply with their parents is 1.27 logits (SD = 1.23, N = 56). The mean differences between those who never/rarely comply and those who sometimes comply is statistically significantly different (t=3.96, df=60, p<0.001). The mean Moderation measure for students who often comply with their parents is 1.36 logits (SD = 1.23, N = 137) and the mean Moderation measure for students who always comply with their parents is 1.60 logits (SD = 1.06, N = 27). The mean differences between those who sometimes comply and often comply is not statistically significant (t=0.42, df=191, p=0.3) and between those who always comply and often comply is not statistically significant (t=0.96, df=162, p=0.16). In terms of groups, students‟ Level of Compliance is only an indicator variable of Moderation for those who rarely or never comply with their parents (less compliance, less Moderation).
Note: There is a colour error in the RUMM2020 computer program. Green in the bar graph represents blue in the numbers on the top right hand side. Brown in the bar graph is represented by purple in the numbers. Purple in the bar graph is represented by red in the numbers. Figure 12. Person-Item Threshold Graph showing the distribution of student measures and item thresholds by Level of Compliance (Moderation scale).
Moderation by Level of Participation in Community Service From Figure 13, the mean Moderation measure for students who have a low level of participation in community service is 1.16 logits (SD = 1.37, N = 99). The mean Moderation measure for students who have a high level of participation in community service is 1.42 logits (SD = 1.14, N = 127). These means are not statistically significantly different (t=0.62, df=224, p=0.27) and so it is concluded that Level of Participation in Community Service is not an indicator variable of Moderation.
126
Liu Shiueh Ling and Russell F. Waugh
Note: There is a colour error in the RUMM2020 computer program here. Maroon in the bar graph represents red in the numbers at the top right hand side of the graph. Green in the bar graph represents blue in the numbers on the top right side of the graph. Figure 13. Person-Item Threshold Graph showing the distribution of student measures and item thresholds by Level of Participation in Community Service (Moderation scale).
NON-FITTING ITEMS All 16 items of the Self-Discipline scale fit the Rasch measurement model. Four of the 16 items of the Moderation scale did not fit the Rasch measurement model. These four items were deleted from the final Moderation scale. There are two possible reasons why these four items did not fit the Rasch model: (1) reverse thresholds of the items; and, (2) students are not in agreement on the difficulty of the items on the scale. A reverse threshold for an item means that students did not answer the response categories consistently and logically as intended. Regarding the second reason, as an example, some students with low Moderation level may find an item difficult while other students of the same Moderation level may find the same item easy. This is generally interpreted that this item is not measuring the same variable (trait) as the other items and is deleted for the next analysis as the aim is to measure a undimensional variable.In future use of all the items, the non-fitting items will require rewording so that they fit the Rasch model in the same consistent way as the good-fitting items. The re-crafted model will then need to be tested with a new data set.
GOOD-FITTING ITEMS Self-Discipline Table 9 shows the items representing all the aspects of the Self-Discipline scale and all the items are good-fitting in both perspectives. Items 11 and 13 had to have the response categories collapsed from four to two because of reversed thresholds. Students found item 9 (I discipline myself to achieve at academic and non-academic work), in the attitude perspective, the easiest (-1.58 logits) in the Self-Discipline scale, while they found item 4 (When I have
Rasch Measures of Self-Discipline and Moderation in Mathematics Education
127
other distractions (TV, music, friends, sport, etc.), I still discipline myself to complete my assignments on time), in the behaviour perspective, the most difficult (+1.26 logits) in the same scale. Table 9. The Items of the Self-Discipline Scale and their Difficulties (Locations) Self-Discipline (Physical) Item No. 1/2
Item
I discipline myself to complete my assignments on time. 3/4 When I have other distractions (TV, music, friends, sport, etc.), I still discipline myself to complete my assignments on time. Self-Discipline (Mental)
I aim for this (Attitude) -1.03
I actually do this (Behaviour) +0.69
-0.03
+1.26
5/6
I discipline myself to achieve academically.
-0.90
+0.50
7/8
I discipline myself to speak nicely to others.
-0.19
+0.28
I discipline myself to achieve at academic and nonacademic work. 11/12 I discipline myself to speak nicely and be polite in dealing with others. Self-Discipline (Financial)
-1.58
+0.55
-1.40
+0.13
13/14
-0.59
+0.62
+0.57
+1.14
9/10
15/16
I discipline myself not to spend more than I need to. I discipline myself to save more and earn extra money.
Notes on Tables 9 and 10:
1. 2.
Item difficulties are in logits. Attitudes are easier than their corresponding behaviours, when both fit the model.
Construct Validity for Self-Discipline From Table 9, it can be seen that the item difficulties for attitude and behaviour (for each corresponding item) are in the conceptualised order of easy to hard. That is, the attitude items are all easier than their corresponding behaviour items, as was conceptualised. It can also be seen that, within each aspect, the item difficulties go from easy to hard vertically down, as was conceptualised. That is, the conceptualised structure for Self-Discipline is supported by the Rasch measures of item difficulties calibrated on the same linear scale. This is strong support for the construct validity of the measure of Self-Discipline.
Moderation Table 10 shows the items representing all the aspects of the Moderation scale and it is noted that the fit is in a mix of both perspectives. For example, item 31 (I have 150 minutes of exercise each week) fitted the Rasch model only in the attitude perspective. Students found this a moderately easy item and its difficulty is -0.31 logits. The same item (I have 150
128
Liu Shiueh Ling and Russell F. Waugh
minutes of exercise each week) in the behaviour perspective (item 32) does not fit the measurement model. Items 23 and 24 (I have a balanced diet and I do not snack between meals) fit the Rasch model in both perspectives, while item 22 (I make healthy choices in my diet), a moderately difficult item (+0.57 logits), fits the model only in the behaviour perspective. Students found item 28 (I stop eating when I feel three-quarters full) the most difficult item (+1.81 logits). The same item in the attitude perspective (item 27) also fits the model (+1.20 logits).
Construct Validity for Moderation There were only 12 good-fitting items in the Moderation scale. Some fitted the Rasch model only in the attitude perspective, some fitted the Rasch model only in the behaviour perspective and others fitted the Rasch model in both perspectives. The 12 items are found in all the aspects of the Moderation scale, namely speaking, eating, and exercising. Where the items do fit the model, it can be seen that the item difficulties for attitude and behaviour are in accord with the conceptualised structure. That is, the attitude items are all easier than their corresponding behaviour items, as was conceptualised. It can also be seen that, within each aspect where the items fit the model, the item difficulties go from easy to hard vertically down, as was conceptualised. So there is only partial support for the conceptualised structure of Moderation. Table 10. The Items of the Moderation Scale and their Difficulties (Locations) Moderation (Speaking) Item Item No. 17/18 I avoid using vulgarities when I speak with my elders. 19/20 I avoid using vulgarities when I speak with anyone because using vulgarities gives others a poor image of me. Moderation (Eating) 21/22 I make healthy choices in my diet. 23/24 I have a balanced diet and I do not snack between meals. 25/26 I over-eat during meals. 27/28 I stop eating when I feel three-quarters full. Moderation (Exercising) 29/30 I have 20 minutes of exercise each week. 31/32 I have 150 minutes of exercise each week.
I aim for this (Attitude) -1.49
I actually do this (Behaviour) -1.73
-0.78
-0.03
Did not fit +0.05
+0.57 +1.40
Did not fit +1.20
Did not fit +1.81
-0.21 -0.31
-0.48 Did not fit
SUMMARY Through the use of the computer programme, Rasch Unidimensional Measurement Models (RUMM 2020) (Andrich, Sheridan & Luo, 2005), linear measures of Self-Discipline (16 items) and Moderation (12 items) were created. The Self-Discipline measure had good
Rasch Measures of Self-Discipline and Moderation in Mathematics Education
129
construct validity (the conceptualised structure was supported), but the conceptualised structure of Moderation was only partially supported as some item data did not fit the measurement model and were deleted. The two linear scales had: 1. Good global item-person and global person-item fit. This is shown by a fit residual of mean -0.43 and standard deviation 0.82 for the items and a fit residual of mean -0.39 and a standard deviation of 1.17 for the students in the Self-Discipline scale. In the moderation scale, this is shown by a fit residual of mean -0.18 and standard deviation of 1.23 for the items, and a fit residual of mean -0.31 and a standard deviation of 0.91 for the students. In both scales, the values are satisfactorily close to ideal fit residuals of mean near zero and standard deviation near one; 2. Good values of Cronbach Alpha and Person Separation Index. The Cronbach Alpha and Person Separation Index for the Self-discipline scale are 0.81 and 0.77 respectively. The Cronbach Alpha and Person Separation Index for the Moderation scale are 0.68 and 0.70 respectively. The maximum value for both the Cronbach Alpha and the Separation Index is 1 and these values for both scales showed that the student measures are well-separated in comparison to the errors (about 0.1 logits); 3. Acceptable item-trait interaction with reasonable agreement among students about the difficulties of the items along both scales; 4. Good individual item fit statistics for all 16 items of the Self-Discipline scale and for 12 items out of 16 items of the Moderation scale, with ordered item thresholds; 5. Good Response Category Curves for the good-fitting items showing that the students answered the items consistently and logically; and, 6. Distribution graphs showing that the targeting of the items against the student measures needs to be improved. More difficult items need to be added in both scales for future use and this will be discussed in the final chapter on Discussion and Implications. It was concluded that reliable linear scales were created for Self-Discipline and Moderation, and that they could be used in subsequent experiments from which valid inferences could be made. The following inferences can be summarised about the difficulties of the items in each variable.
Self-Discipline (Attitude Items) The two easiest attitude items are I aim to discipline myself to achieve at academic and non-academic work (item 11, difficulty=-1.58 logits and very easy) and I aim to discipline myself to speak nicely and be polite in dealing with others (item 13, difficulty=-1.40 logits and very easy). The two hardest attitude items are When I have other distractions (TV, music, friends, sport etc), I still aim to discipline myself to complete my assignments on time (item 3, difficulty=-0.03 logits and moderately hard) and I aim to discipline myself to speak nicely to others (item 7, difficulty=-0.19 and moderately easy).
130
Liu Shiueh Ling and Russell F. Waugh
Self-Discipline (Behavior Items) The two easiest behaviour items are I actually discipline myself to speak nicely and be polite in dealing with others (item 12, difficulty=+0.13 logits and moderately hard) and I actually discipline myself to speak nicely and to others (item 8, difficulty=+0.28 logits and moderately hard). The two hardest behaviour items are When I have other distractions (TV, music, friends, sport etc), I still discipline myself to complete my assignments on time (item 4, difficulty=+1.26 logits and very hard) and I actually discipline myself to save more and earn extra money (item 16, difficulty=+1.14 and very hard).
Moderation (Attitude Items) The two easiest attitude items are I aim to avoid using vulgarities when I speak with my elders (item 17, difficulty=-1.49 logits and very easy) and I aim to avoid using vulgarities when I speak with anyone because using vulgarities gives others a poor image of me (item 19, difficulty=-0.78 logits and quite easy). The two hardest attitude items are I aim to stop eating when I am three-quarters full (item 27, difficulty=+1.20 logits and very hard) and I aim to have a balanced diet and not snack between meals (item 23, difficulty=+0.05 logits and moderately hard).
Moderation (Behavior Items) The two easiest behaviour items are I actually avoid using vulgarities when I speak with my elders (item 18, difficulty=-1.73 logits and very easy) and I actually have 20 minutes of exercise each week (item 30, difficulty=-0.21 logits and moderately easy). The two hardest behaviour items are I actually stop eating when I am three-quarters full (item 28, difficulty=+1.81 logits and very hard) and I actually have a balanced diet and I do not snack between meals (item 24, difficulty=+1.40 logits and very hard). It should be noted that students with the lowest and highest measures of Self-Discipline and Moderation can be identified, because the RUMM 2020 program lists the measures in order by student number. Those students who need advice and help could then be identified and provided with specialist help.
In: Specialized Rasch Measures… Editor: Russell F. Waugh, pp. 131-152
ISBN: 978-1-61668-032-9 © 2010 Nova Science Publishers, Inc.
Chapter 7
RASCH MEASURES OF DEPENDABILITY AND RESPONSIBILITY Liu Shiueh Ling and Russell F. Waugh Graduate School of Education University of Western Australia.
ABSTRACT This paper follows through from the research in Chapter Six and is part of a larger study of „caring-thinking‟ with Mathematics students in Singapore. „Caring Thinking‟ was used as a means of developing values in students by creating the context in teaching Mathematics to allow students to engage in valuational thinking, values realisation and values clarification. The sample was N=226 Year 10 students in secondary education. A Rasch analysis was used to create linear, uni-dimensional scales for Dependability (16 items) and Responsibility (11 items). Items were answered in two perspectives: (1) I aim for this (attitude) and (2) I actually do this (behavior), with attitude conceptualized as easier then behavior. The item-trait chi-square was χ2 = 71.15, df = 48, p = 0.02 for SelfDiscipline and χ2 = 25.87, df = 33, p = 0.81 for Moderation, showing a very good overall fit to the measurement model for the latter (and not so good for the former). The Student Separation Indices were 0.89 and 0.82, and the Cronbach Alphas were 0.89 and 0.82, respectively, showing good scale reliability. There were good item fits for each scale and the items were ordered from easy to hard providing good information relating to each variable.
BACKGROUND The background to this paper is given in Chapter Six.
132
Liu Shiueh Ling and Russell F. Waugh
RASCH MEASUREMENT FOR DEPENDABILITY AND RESPONSIBILITY The initial Rasch analysis for Dependability showed that all 16 items fitted the Rasch measurement model but two items had their response categories collapsed from four to three and one item had its response categories collapsed from four to two. The initial Rasch analysis for Responsibility showed that five out of 16 items did not fit the Rasch measurement model and these items were deleted. Three of the remaining 11 items had their response categories collapsed: one item from four to three and two items from four to two. A brief explanation of these analyses is given first, using various statistical indicators. The final analyses in which 16 items for Dependability and 11 items for Responsibility fitted the measurement model satisfactorily, with acceptable reliability, for the former and excellently for the latter, is described in more detail here, through discussions and Tables 1 to 8. This is followed by a discussion of the item characteristic curves and the category response curves of some items (see Figures 1 to 4), as well as the person-item threshold distribution graphs (targeting) (see Figures 5 to 10) for the participants. Some commentary on the non-fitting items is given and then some commentary on the good-fitting items is given, followed by the presentation of the scale of Dependability (see Table 9) and the scale of Responsibility (see Table 10). This chapter ends with the summary of the main findings of the RUMM programme analysis of the data for Dependability and Responsibility.
INITIAL PRETEST RASCH ANALYSES DEPENDABILITY (N=223, I=16) AND RESPONSIBILITY (N=214, I=11) Although 277 students (139 from the experimental group and 138 from the control group) was the total number of students from the eight participating classes, only 256 students participated in the survey. Twenty-one students did not participate because they were either on sick leave or they were out representing the school in competitions on the days the survey was carried out. Of the 256 participants, 30 (12 from the experimental group and 18 from the control group) submitted incomplete questionnaires. Thus, subsequently only 226 responses to the questionnaire were entered into an excel file. Participants in the control group were assigned serial numbers from „1001‟ to „1105‟ and participants in the experimental group were assigned serial numbers from „1106‟ to „1226‟. For each participant, the responses to the questionnaire were entered in terms of response categories, „1‟, „2‟, „3‟ and „4‟. „1‟ was for „never‟, „2‟ was for „rarely‟, „3‟ was for „sometimes‟, and „4‟ was for „often‟. For the Dependability scale, at the beginning, the computer programme discarded three students because of extreme data; that is, the students entered all „4s‟ in response to the items in the survey (in Rasch measurement, measures cannot be estimated from extreme data). Thus, 223 participants remained out of 226 for subsequent analysis. There were eight items in the questionnaire, each answered in two perspectives, „I aim for this‟ and „I actually do this‟, thus making a Rasch item analysis of 16 items, item numbers 33 to 48. A linear scale was created with the data from the 16 items but two items (41 and 46) had to have their response categories collapsed from four to three („1‟=„0‟, „2‟=„0‟, „3‟=„1‟, „4‟=„2‟) and one item (45)
Rasch Measures of Dependability and Responsibility
133
from four to two („1‟=„0‟, „2‟=„0‟, „3‟=„1‟, „4‟=„1‟) because of disordered thresholds, meaning that the response categories of 1, 2, 3 and 4, were not answered consistently in line with their conceptual ordering (the reason for this is unknown). The final analysis and creation of a linear scale for Dependability is described in the next section. For the Responsibility scale, at the beginning, the computer programme discarded 12 students because of extreme data, thus leaving 214 participants out of 226 for subsequent analysis. There were also eight items in the questionnaire each answered in two perspectives, „I aim for this‟ and „I actually do this‟, thus making a Rasch item analysis of 16 items, item numbers 49 to 64. Five items (52, 53, 55, 57 and 62) showed disordered thresholds meaning that the students did not use the response categories consistently and logically in line with their conceptual ordering. The five non-fitting items were deleted and the response categories of three of the 11 remaining items were collapsed: item number 58 from four to three („1‟=„0‟, „2‟=„0‟, „3‟=„1‟, „4‟=„2‟) and item numbers 61 and 63 from four to two („1‟=„0‟, „2‟=„0‟, „3‟=„1‟, „4‟=„1‟).
FINAL RASCH ANALYSES DEPENDABILITY (N=223, I=16) AND RESPONSIBILITY (N=214, I=11) For Dependability, the overall fit to the measurement model was satisfactory using 16 items (item-trait chi-square = 71.15, df = 48, p = 0.02) and the revised response categories. This means that there was reasonable agreement among participants about the difficulties of the items along the scale (although the agreement was not perfect). The final analysis involved 16 items and this analysis is reported in some detail in this chapter. For Responsibility, a linear scale was created with the data from 11 items and the overall fit to the measurement model was excellent (item-trait chi-square = 25.87, df = 33, p = 0.81). This means that there was good agreement about the difficulties of the items along the scale. The final analysis involved 11 items and this analysis is reported in some detail in a following section.
Summary of Fit Statistics Table 1 is a summary of the fit statistics for Dependability. The statistics show a global, standardised fit residual mean of -0.45 logits and a standard deviation 1.52 for the items and a global, standardised fit residual mean of -0.44 logits and a standard deviation of 1.29 for the persons. Table 7.2 is a summary of the fit statistics for Responsibility. It shows a global, standardised fit residual mean of -0.51 logits and standard deviation 0.68 for the items and a global, standardised fit residual mean of -0.45 logits and a standard deviation of 1.05 for the persons. These are close to ideal fit residuals of mean near zero and standard deviation near one which shows an overall satisfactory fit to the measurement model. That is to say there is a good consistency of item parameters (person measures and item difficulties) for the measure.
134
Liu Shiueh Ling and Russell F. Waugh Table 1. Summary of Fit Statistics for Dependability from the Rasch Analysis (N = 226, I = 16) ITEM-PERSON INTERACTION ITEMS Location Mean 0.00 SD 0.99 ITEM-TRAIT INTERACTION Total Item Chi-Square Total Degree of Freedom Total Chi-Square Probability
Fit Residual -0.45 1.52 71.15 48.00 0.02
PERSONS Location Fit Residual 1.39 -0.44 1.31 1.29 RELIABILITY INDICES Separation Index 0.89 Cronbach Alpha 0.89 POWER Power is EXCELLENT [Based on Separation Index of 0.89]
Notes on Table 1: 1.The Index of Student Separation is the proportion of observed variance that is considered true and is reasonably high (89%). It means that the measures are separated satisfactorily in comparison to the errors. 2. The item and student global fit statistics have an expected mean of near zero and a standard deviation of near one, when the data fit the measurement model. The fit statistics in this case are satisfactory. 3. The item-trait interaction chi-square test indicates that students of differing dependability level responded to the item difficulties according to what is expected of them by the measurement model and that a unidimensional trait has been measured. 4. All numbers are given to two decimal places because the errors are two decimal places (see Table 7.3). 5. Power is the ability to test any non-compliance with the measurement model and, in this case, is excellent.
Table 1 also shows the Cronbach Alpha (0.89) and the Separation Index (0.89) for the 16 items for Dependability. This shows that the Dependability scale has good traditional internal scale reliability and that the measures are well-separated in comparison to the errors. The Cronbach Alpha of 0.82 and 0.82 for Separation Index for the Responsibility scale also shows that the scale has good traditional internal scale reliability and that the measures are wellseparated in comparison to the errors. The RUMM2020 programme rates the overall power of test-of-fit for the Dependability scale as excellent (see Table 1) and that for the Responsibility scale as good (see Table 2) which show that there is adequate power to detect any major noncompliance with the measurement model. Table 2. Summary of Fit Statistics for Responsibility from the Rasch Analysis (N = 226, I = 11) ITEM-PERSON INTERACTION ITEMS Location Mean 0.00 SD 0.99 ITEM-TRAIT INTERACTION Total Item Chi-Square
Fit Residual -0.51 0.68 25.87
PERSONS Location Fit Residual 2.05 -0.45 1.38 1.05 RELIABILITY INDICES Separation Index 0.82
Rasch Measures of Dependability and Responsibility ITEM-PERSON INTERACTION ITEMS Total Degree of Freedom
33.00
Total Chi-Square Probability
0.81
PERSONS Cronbach Alpha
135
0.82
POWER Power is GOOD [Based on Separation Index of 0.82]
Notes on Table 2: 1. The Index of Student Separation is the proportion of observed variance that is considered true and is reasonably high (82%). This means that the measures are separated satisfactorily in comparison to the errors of measurement. 2. The item and student global fit statistics have an expected mean of near zero and a standard deviation of near one, when the data fit the measurement model. The fit statistics in this case are satisfactory. 3. The item-trait interaction chi-square test indicates that students of differing measuring levels responded to the item difficulties according to what is expected of them by the measurement model and that a unidimensional trait has been measured. 4. All numbers are given to two decimal places because the errors are two decimal places (see Table 7.4). Power is the ability to test any non-compliance with the measurement model and, in this case, is good.
Individual Item-Fit The item difficulties (in Tables 3 and 4) are measured in standard units, called logits (the log odds of successfully answering). Item fit to the measurement model is given by a chisquare and its associated probability. This is a statistic that is calculated from the discrepancies between the observed mean in the class intervals and the expected values according to the measurement model. If the probability has a value of less than 0.01, then it implies that the discrepancy between the observed mean and the expected value is large relative to chance and that item should be examined. For the Dependability scale, Table 3 shows that all 16 items have acceptable probability values more than 0.01 and, since all 16 items were part of the theoretical construct, they are included in the measure. For the Responsibility scale, Table 4 shows that except all 11 items have acceptable probability values more than 0.01 and they are included in the measure. This means that, although five other items were initially included as part of the theoretical construct, they did not fit the measurement model and were discarded. Tables 3 and 4 have a column that shows the residuals. These are the differences between the actual responses and the responses estimated from the Rasch measurement parameters. Standardised residuals are generally expected to be within the range of -2 and +2. For the Dependability scale, Table 7.3 shows that all the items have acceptable residuals except for item numbers 36, 37, 39, 41 and 48. For the Responsibility scale, Table 4 shows that all the 11 items have acceptable residuals. The five items with unacceptable residuals in the Dependability scale were not deleted because their deletion made for a worse overall fit of all the items to the measurement model.
Liu Shiueh Ling and Russell F. Waugh
136
Table 3. Item Difficulties (Locations), Standard Errors (SE), Residuals and Fit to the Measurement Model for the Dependability scale Item 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48
Difficulty -1.22 -0.46 -0.42 +0.42 +0.14 +0.57 +0.71 +1.43 +0.52 +0.46 +0.63 +1.19 -2.54 -0.60 -0.77 -0.05
SE 0.12 0.12 0.11 0.10 0.10 0.10 0.09 0.10 0.11 0.10 0.09 0.10 0.37 0.14 0.12 0.11
Residual -0.36 +1.60 -0.19 +2.19 -2.15 -1.11 -2.56 -1.83 -2.13 -0.66 -1.63 +0.10 -0.32 -0.02 -0.61 +2.55
DegFree 206.38 206.38 206.38 206.38 206.38 206.38 206.38 206.38 206.38 206.38 206.38 206.38 206.38 206.38 206.38 206.38
DataPts 223 223 223 223 223 223 223 223 223 223 223 223 223 223 223 223
ChiSquare 3.60 4.26 0.60 8.30 5.83 1.36 5.73 3.16 9.59 5.42 2.54 5.53 2.43 4.15 0.71 7.95
Prob 0.31 0.23 0.90 0.04 0.12 0.71 0.13 0.37 0.02 0.14 0.47 0.14 0.49 0.25 0.87 0.05
Table 4. Item Difficulties (Locations), Standard Errors (SE), Residuals and Fit to the Measurement Model for the Responsibility scale Item 49 50 51 54 56 58 59 60 61 63 64
Difficulty -0.02 +0.64 +0.68 +0.13 +0.50 +0.84 +0.10 +0.74 -1.92 -1.92 +0.21
SE 0.12 0.11 0.10 0.12 0.12 0.13 0.11 0.11 0.36 0.36 0.12
Residual -0.48 -0.26 -0.73 +0.30 -0.34 -0.67 -1.17 -1.83 -0.59 -0.59 +0.74
DegFree 192.09 192.09 192.09 192.09 192.09 192.09 192.09 192.09 192.09 192.09 192.09
DataPts 214 214 214 214 214 214 214 214 214 214 214
ChiSquare 0.51 0.93 1.22 9.01 0.12 1.83 1.96 2.70 2.74 2.74 2.12
Prob 0.92 0.82 0.75 0.03 0.99 0.61 0.58 0.44 0.43 0.43 0.55
Notes on Tables 3 and 4: 1. The Difficulty of each item is in logits (the log odds of giving a positive response to an item). 2. SE is standard error in logits. They are smaller in the region where there are more students. 3. Residual is the difference between the observed and expected response. 4. Prob is the probability based on the chi-square fit to the measurement model and is dependent on sample size.
Item Threshold Distribution Table 5 shows that the 16 items for the Dependability scale that fit the Rasch model are polytomous, meaning that there are four ordered response categories, „0‟ for „never‟, „1‟ for „rarely‟, „2‟ for „sometimes‟, and „3‟ for „often‟, for each item except for item numbers 41, 45 and 46. Item numbers 41 and 46 have three ordered response categories, instead of four ordered response categories, while item number 45 has two ordered response categories, instead of four ordered response categories.
Rasch Measures of Dependability and Responsibility
137
Table 5. Item Specification of the Dependability Scale Rasch Item No. (Original No) 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48
Test items Polytomous Polytomous Polytomous Polytomous Polytomous Polytomous Polytomous Polytomous Polytomous Polytomous Polytomous Polytomous Dichotomous Polytomous Polytomous Polytomous
Response categories 4 4 4 4 4 4 4 4 3 4 4 4 2 3 4 4
Original categories 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
Thresholds 3 3 3 3 3 3 3 3 2 3 3 3 1 2 3 3
Notes for Tables 5, 6 and 7: 1. Polytomous means that each item has more then two ordered response categories and dichotomous means that there are two response categories. 2. Response categories is the number of ordered response categories in the final scale. 3. Original categories refers to the number of original responses to each item. 4. Thresholds are points between adjacent categories where there are odds of 1:1 of answering in adjacent categories. Where there are four response categories, there are three thresholds and, where there are two response categories, there is one threshold.
Table 6. Item Thresholds – Uncentralised (Dependability Scale) Item
Difficulty
Mean
Item 33 Item 34 Item 35 Item 36 Item 37 Item 38 Item 39 Item 40 Item 41 Item 42 Item 43 Item 44 Item 45 Item 46 Item 47 Item 48
-1.22 -0.46 -0.42 +0.42 +0.14 +0.57 +0.71 +1.43 +0.52 +0.46 +0.63 +1.19 -2.54 -0.60 -0.77 -0.05
-1.22 -0.46 -0.42 +0.42 +0.14 +0.57 +0.71 +1.43 +0.52 +0.46 +0.63 +1.19 -2.54 -0.60 -0.77 -0.05
Thresholds 1 -3.12 -2.35 -1.55 -1.72 -1.18 -0.93 -0.44 -0.34 -0.30 -1.23 -0.06 -0.47 -2.54 -2.17 -1.75 -1.62
2 -0.83 -0.94 -0.53 +0.65 +0.02 +0.16 +0.52 +1.34 +1.34 +0.15 +0.10 +1.07 n/a +0.98 -1.05 -0.47
3 +0.28 +1.92 +0.82 +2.33 +1.59 +2.50 +2.05 +3.27 n/a +2.45 +1.85 +2.96 n/a n/a +0.50 +1.94
Table 6 shows three thresholds calculated by the RUMM computer programme, as there are four response categories to each item, except for item numbers 41, 45 and 46. A threshold is a point between two response categories where there is an equal probability of answering either category. The first threshold shows the point between response categories „0‟ and „1‟,
Liu Shiueh Ling and Russell F. Waugh
138
numbered according to the Rasch programme, where there is equal probability of responding either a „0‟ or „1‟. The second threshold shows the point between categories „1‟ and „2‟, numbered according to the Rasch programme, where there is equal probability of responding either a „1‟ or „2‟. The third threshold shows the point between categories „2‟ and „3‟, numbered according to the Rasch programme, where there is equal probability of responding either a „2‟ or „3‟. The thresholds are ordered in line with the ordering of the response categories showing that the students have answered the response categories consistently and logically. Table 7 shows that 11 items for the Responsibility scale fit the Rasch model. The five items that do not fit, item numbers 52, 53, 55, 57 and 62, were deleted. The remaining items have four ordered response categories except for three items: item number 58 has three ordered response categories, „0‟, „1‟ and „2‟, instead of, four ordered response categories and item numbers 61 and 63 have two ordered response categories, „0‟ and „1‟, instead of, four ordered response categories. The thresholds are ordered in line with the ordering of the response categories showing that the students have answered the response categories consistently and logically. Table 7. Item Specification of the Responsibility Scale Rasch Item No. (Original No.) 49 50 51 54 56 58 59 60 61 63 64
Test responses Polytomous Polytomous Polytomous Polytomous Polytomous Polytomous Polytomous Polytomous Dichotomous Dichotomous Polytomous
Response categories 4 4 4 4 4 3 4 4 2 2 4
Original categories 4 4 4 4 4 4 4 4 4 4 4
Thresholds 3 3 3 3 3 2 3 3 1 1 3
Table 8. Item Thresholds – Uncentralised (Responsibility Scale) Item Item 49 Item 50 Item 51 Item 54 Item 56 Item 58 Item 59 Item 60 Item 61 Item 63 Item 64
Item Difficulty -0.02 +0.64 +0.68 +0.13 +0.50 +0.84 +0.10 +0.74 -1.92 -1.92 +0.21
Thresholds 1 -1.44 -0.83 -0.24 -1.24 -1.19 -0.67 -0.86 -1.41 -1.92 -1.92 -1.63
Thresholds 2 -0.29 -0.05 +0.20 -0.78 -0.11 +2.35 -0.37 +0.58
Thresholds 3 +1.67 +2.81 +2.09 +2.41 +2.81
-0.46
+2.72
+1.52 +3.06
Notes on Table 8: 1. There is one threshold for items 61 and 63 since each item has two response categories. Thresholds are points between adjacent categories where there are odds of 1:1 of answering in adjacent
Rasch Measures of Dependability and Responsibility
139
categories. In these cases, there are only two response categories and, hence, only one threshold which is the item difficulty. 2. The item difficulties and thresholds are given in standard units called logits, the log odds of successfully answering the item.
Table 8 shows the thresholds (item difficulties) calculated by the RUMM computer programme where there are four response categories to each item except for item number 58 where there is three response categories and item numbers 61 and 63 where there are two response categories. Each threshold shows the point between adjacent response categories, for example, between „0‟ and „1‟, numbered according to the Rasch programme, where there is equal probability of responding either a „0‟ or „1‟. The item difficulties range from -1.92 logits to +0.84 logits.
Item Characteristic Curves
Note: This item discriminates well, as required for compliance with the Rasch measurement model. Figure 1. Item Characteristic Curve for item 35 (Dependability scale).
Figure 1 shows the item characteristic curve for item 35 of the Dependability scale. This is an easy item (the location difficulty is -0.42 logits). The observed means, shown as dots, in the four class intervals are close to the curve. This shows that the item fits well to the theoretical curve of the Rasch model according to this criterion (the chi-square probability of fit is 0.90). It means that the item discriminates between the students well as specified by the model; that is, item 35, „When I am busy with my own commitments, I still help my family members (I aim for this)‟, has expected values close to those necessary for a good fit to the measurement model. The item characteristic curves for 15 out of 16 items in the Dependability scale were satisfactory and are not all reported here. One of the 16 items in the Dependability scale, item 41, did not fit the Rasch model satisfactorily, but its removal and subsequent re-analysis did not improve the fit of the other items and, because it conceptually fits in the scale, it was left in.
140
Liu Shiueh Ling and Russell F. Waugh
Note on Figure 2: This item does not fit well, as specified by the Rasch measurement model. Figure 2. Item Characteristic Curve for item 41 (Dependability scale).
This is a difficult Dependability item (the location difficulty is +0.52 logits) which does not fit as well as one would like, but it is still discriminating satisfactorily. The observed means, shown as dots, in the four class intervals are all close to the characteristic curve (ogive) as required by the measurement model for good discrimination, so it is difficult to see why it doesn‟t fit the measurement model. It seems that about half the high measuring students found this item difficult and the other half found it easy or moderately difficult. For the Responsibility scale, the item characteristic curves for all 11 items in were satisfactory and are not reported here to avoid repetition.
Response Category Curves Figure 3 shows the response category curves for item 33 of the Dependability scale. The vertical axis represents the probability of responding in a particular response category and the horizontal axis represents the students‟ person location (measure) in logits. The Rasch programme converts the response categories 1, 2, 3, and 4 to 0, 1, 2, and 3 respectively. In Figure 3, the category 0 response curve indicates that a student with a Dependability measure of -7.0 logits (Person Location) has about a 0.98 probability of responding in this category („Never‟), whereas a student with a Dependability measure of 0.0 logits has a near zero probability of responding in the same category for item 33. The category 1 curve of Figure 3 shows that a student with a Dependability measure of about -2.1 logits has a probability of about 0.60 of responding in the category („Rarely‟) for item 33, whereas a student with a Dependability measure of -7.0 logits has a probability of about 0.02 of responding in the same category. For category curve 2, a student with a Dependability measure of about -0.3 logits has a probability of about 0.45 of responding in the category („Sometimes‟) for item 33, whereas a student with a Dependability measure of -5.0 logits has a near zero probability of responding in the same category. Finally, for category curve 3, a student with a Dependability measure of about -3.0 logits has a probability near zero of
Rasch Measures of Dependability and Responsibility
141
responding in the category („Often‟) for item 33, whereas a student with a Dependability measure of 4.0 logits has a probability of about 0.97 of responding in the same category. This shows that the students used the four response categories for item 33 logically and consistently.
Υ1
Υ2 Υ3
Figure 3. Response Category Curves for Item 33 (Dependability scale).
When the response categories are used as expected, in line with their conceptual ordering, the boundaries between the categories at the 50:50 point (called thresholds) should also be ordered. Figure 7.3 shows such a case for the Rasch item 33 of the Dependability scale with four ordered categories. The thresholds (Υ1 , Υ2 and Υ3) which define the categories are estimated in the model and are ordered. They show the points where the probability of responding either 0 or 1, 1 or 2, and 2 or 3 respectively, are equally likely. These thresholds were ordered in line with the order of the response categories for item 33 and for all the other Dependability items. The category response curves for all 16 items were checked and they were found to be satisfactory and operating as they should. Figure 4 shows the Response Category Curves for item 49 of the Responsibility scale. In Figure 4, the category 0 response curve indicates that a student with a Responsibility measure of -6.0 logits (Person Location) has about a 0.99 probability of responding in this category, whereas a student with a Responsibility measure of 1.0 logits has about a 0.01 probability of responding in the same category for item 49. For category curve 3, a student with a Responsibility measure of about -1.0 logits has about a 0.01 probability of responding in the category for item 49, whereas a student with a Responsibility measure of 6.0 logits has a probability of about 0.98 of responding in the same category. Figure 4 shows that the students used the four response categories for item 49 logically and consistently. The category response curves for all 11 items were checked and they were found to be satisfactory and operating as they should.
142
Liu Shiueh Ling and Russell F. Waugh
Υ1 Υ
Υ3
2 Figure 4. Response Category Curves for Item 49 (Responsibility scale).
Person-Item Threshold Distribution (Targeting) Figures 5, 6 and 7 show the distribution of student measures and item thresholds for the 226 secondary-four students on the same linear Dependability scale. The targeting of the items against the student measures of Dependability is good (they cover about the same range) but, in any future use of the scale for these students, some very hard items need to be added to cater for those students who have very high measures of Dependability.
RASCH MEASURES BY INDEPENDENT VARIABLES There were three independent variables, considered as indicator variables: Type of Home (2/3 room public housing, 4/5 room public housing, executive public apartment, private dwelling); Level of Compliance with parents and teachers (never, rarely, sometimes, often, always); and Participation in Community Service (low, high). Type of Home is an indicator variable which measures „something‟ happening in the home that is expected to influence Dependability and Responsibility (possibly the educational values exerted by the parents on the students with higher levels associated with larger houses and higher levels of money and education that, in turn, produce higher Dependability and Responsibility). Level of Compliance is an indicator variable of docility with more docility associated with higher compliance that, in turn, produces higher Dependability and Responsibility. Participation in Community Service is an indicator variable associated with helping others and this, in turn, is associated with higher levels of Dependability and Responsibility. The measures of Dependability and Responsibility were made against each of the three independent variables and are now reported.
Rasch Measures of Dependability and Responsibility
143
Dependability by Housing Type From Figure 5, the mean Dependability measure for students who live in two- or threeroom public housing apartments is 1.22 logits (SD = 0.89, N = 18). The mean Dependability measure for students who live in four-room or five-room public housing apartments is 1.41 logits (SD = 1.12, N = 86). This is not statistically significantly different (t=1.66, df=102, p=0.05) at the p=0.01 level. The mean Dependability measure for students who live in executive public housing apartments is 1.45 logits (SD = 1.76, N = 28) and the mean Dependability measure for students who live in private apartments or houses is 1.39 logits (SD = 1.39, N = 94). This is not statistically significantly different (t=1.58, df=112, p=0.04) at the p=0.01 level. The mean Dependability measure for students who live in 2/3 room public housing is 1.22 logits (SD = 0.89, N = 18) and the mean Dependability measure for students who live in executive public housing is 1.45 logits (SD = 1.76, N = 28). This is not statistically significantly different (t=2.03, df=36, p=0.04) at the p=0.01 level.
Note: There is a colour error in the RUMM2020 computer program. Brown in the bar graph is represented by purple in the numbers on the top right hand side. Green in the bar graph is represented by blue in the numbers. Purple in the bar graph is represented by green in the numbers. Figure 5. Person-Item Threshold Graph showing the distribution of student measures and item thresholds by Housing Type (Dependability scale).
Dependability by Level of Compliance From Figure 6, the mean Dependability measure for students who never comply with their parents is 0.54 logits (SD = 0.50, N = 3). The mean Dependability measure for students who rarely comply with their parents is 0.93 logits (SD = 0.89, N = 3). The mean Dependability measure for students who sometimes comply with their parents is 1.08 logits (SD = 1.39, N = 56). The mean differences between never/rarely and sometimes for Level of Compliance are statistically significantly different (t=2.89, df=60, p<0.001) at the p=0.01
144
Liu Shiueh Ling and Russell F. Waugh
level. The mean Dependability measure for students who often comply with their parents is 1.37 logits (SD = 1.17, N = 137). The mean differences between sometimes and often for Level of Compliance is not statistically significantly different (t=1.39, df=191, p=0.09) at the p-0.01 level. The mean Dependability measure for students who always comply with their parents is 2.27 logits (SD = 1.51, N = 27) and this is statistically significantly different from the mean for sometimes (t=12.97, df=81, p<0.001) at the p=0.01 level. In terms of groups, students‟ mean Dependability measure increases as their Level of Compliance with their parents increases.
Note: There is a colour error in the RUMM2020 computer program. Brown in the bar graph is represented by purple in the numbers on the top right hand side. Green in the bar graph is represented by blue in the numbers. Purple in the bar graph is represented by green in the numbers. Figure 6. Person-Item Threshold Graph showing the distribution of student measures and item thresholds by Level of Compliance (Dependability scale).
Dependability by Level of Participation in Community Service From Figure 7, the mean Dependability measure for students who have a low level of participation in community service is 0.92 logits (SD = 1.10, N = 99). The mean Dependability measure for students who have a high level of participation in community service is 1.76 logits (SD = 1.34, N = 127). This is not statistically significantly different (t=1.31, df=224, p=0.09) at the p=0.01 level and so it is concluded that Level of Participation in Community Service is not an indicator of some variable that is influencing Dependability.
Rasch Measures of Dependability and Responsibility
145
Note: There is a colour error in the RUMM2020 computer program here. Maroon in the bar graph represents red in the numbers at the top right hand side of the graph. Green in the bar graph represents blue in the numbers on the top right side of the graph. Figure 7. Person-Item Threshold Graph showing the distribution of student measures and item thresholds by Level of Participation in Community Service (Dependability scale).
Responsibility by Housing Type From Figure 8, the mean Responsibility measure for students who live in two-room or three-room public housing apartments is 2.54 logits (SD = 1.26, N = 18). The mean Responsibility measure for students who live in four-room or five-room public housing apartments is 1.92 logits (SD = 1.28, N = 86). This is not statistically significantly different at the p=0.01 level (t=2.03, df=102, p=0.03) at the p=0.01 level. The mean Responsibility measure for students who live in private houses is 2.15 logits (SD=1.46, N=94) and those who live in executive public housing apartments is 1.78 logits (SD = 1.41, N = 28). The mean difference between students who live in 2/3 room public housing and those who live in private houses or apartments is not statistically significant (t=1.60, df=110, p=0.06) at the p=0.01 level. The mean Responsibility measure for students who live in private apartments or houses is 2.15 logits (SD = 1.46, N = 94). This mean is not statistically significantly different from those who live in executive apartments or houses (t=0.90, df=114, p=0.19) athe p=0.01 level. It is concluded that size of public housing is not an indicator variable of something that is influencing Responsibility in students.
146
Liu Shiueh Ling and Russell F. Waugh
Note: There is a colour error in the RUMM2020 computer program. Green in the bar graph represents blue in the numbers on the top right hand side. Brown in the bar graph is represented by purple in the numbers. Purple in the bar graph is represented by red in the numbers. Figure 8. Person-Item Threshold Graph showing the distribution of student measures and item thresholds by Housing Type (Responsibility scale).
Responsibility by Level of Compliance From Figure 9, the mean Responsibility measure for students who never comply with their parents is 1.17 logits (SD = 3.40, N = 3). The mean Responsibility measure for students who rarely comply with their parents is -0.40 logits (SD = 1.65, N = 3). The mean Responsibility measure for students who sometimes comply with their parents is 1.42 logits (SD = 0.98, N = 56). The mean differences between those who never/rarely comply and those who sometimes comply is statistically significantly different (t=3.96, df=60, p<0.001). The mean Responsibility measure for students who often comply with their parents is 2.17 logits (SD = 1.30, N = 137) and the mean Responsibility measure for students who always comply with their parents is 3.10 logits (SD = 1.28, N = 27). The mean differences between those who sometimes comply and often comply is not statistically significant (t=0.42, df=191, p=0.3) and between those who always comply and often comply is not statistically significant (t=0.96, df=162, p=0.16) at the p=0.01 level. In terms of groups, students‟ Level of Compliance is only an indicator variable of Responsibility for those who rarely or never comply with their parents (less compliance, less Responsibility).
Rasch Measures of Dependability and Responsibility
147
Note: There is a colour error in the RUMM2020 computer program. Green in the bar graph represents blue in the numbers on the top right hand side. Brown in the bar graph is represented by purple in the numbers. Purple in the bar graph is represented by red in the numbers. Figure 9. Person-Item Threshold Graph showing the distribution of student measures and item thresholds by Level of Compliance (Responsibility scale).
Responsibility by Level of Participation in Community Service From Figure 10, the mean Responsibility measure for students who have a low level of participation in community service is 1.73 logits (SD = 1.15, N = 99). The mean Responsibility measure for students who have a high level of participation in community service is 2.29 logits (SD = 1.49, N = 127). These means are not statistically significantly different (t=0.62, df=224, p=0.27) at the p=0.01 level and so it is concluded that Level of Participation in Community Service is not an indicator variable of Responsibility.
Note: There is a colour error in the RUMM2020 computer program here. Maroon in the bar graph represents red in the numbers at the top right hand side of the graph. Green in the bar graph represents blue in the numbers on the top right side of the graph. Figure 10. Person-Item Threshold Graph showing the distribution of student measures and item thresholds by Level of Participation in Community Service (Responsibility scale).
Liu Shiueh Ling and Russell F. Waugh
148
NON-FITTING ITEMS All 16 items of the Dependability scale fit the Rasch measurement model. Five of the 16 items of the Responsibility scale did not fit the Rasch measurement model. These five items were deleted from the final Responsibility scale. There are two possible reasons why these five items did not fit the Rasch model: (1) reverse thresholds of the items; and, (2) students are not in agreement on the difficulty of the items on the scale. A reverse threshold for an item means that students did not answer the response categories consistently and logically as intended. Regarding the second reason, as an example, some students with low Responsibility level may find an item difficult while other students of the same Responsibility level may find the same item easy. This is generally interpreted that this item is not measuring the same variable (trait) as the other items and is deleted for the next analysis as the aim is to measure a undimensional variable. In future use of all the items, the non-fitting items will require rewording so that they fit the Rasch model in the same consistent way as the good-fitting items. The re-crafted model will then need to be tested with a new data set.
GOOD-FITTING ITEMS Items Measuring Dependability Table 9 shows the items representing all the aspects of the Dependability scale and all the items are good-fitting in both perspectives. Items 41 and 46 had to have the response categories collapsed from four to three because of reversed thresholds. For the same reason, item 45 had to have its response categories collapsed from four to two. Students found item 45 (I help my friends), in the attitude perspective, the easiest (-2.54 logits) in the Dependability scale, while they found item 40 (When I am busy with my own commitments, I still volunteer to serve the school), in the behaviour perspective, the most difficult (+1.43 logits) in the same scale. Table 9. The Items of the Dependability Scale and their Difficulties (Locations) Dependability (Family) Item No. 33/34
Item
I aim for this (Attitude) -1.22
I actually do this (Behaviour) -0.46
When I am busy with my own commitments, I still help my family members. Dependability (School)
-0.42
+0.42
37/38
I volunteer to serve the school.
+0.14
+0.57
39/40
+0.71
+1.43
41/42
When I am busy with my own commitments, I still volunteer to serve the school. I make contributions to the school.
+0.52
+0.46
43/44
I look for opportunities to contribute to the school.
+0.63
+1.19
I help my family members.
35/36
Rasch Measures of Dependability and Responsibility
149
Dependability (Family) Item Item No. Dependability (Friends)
I aim for this (Attitude)
I actually do this (Behaviour)
45/46
I help my friends.
-2.54
-0.60
47/48
When I am busy with my own commitments, I still help my friends.
-0.77
-0.05
Notes on Table 9: 1. Item difficulties are in logits. 2. Attitudes are easier than their corresponding behaviours, when both fit the model.
Construct Validity for Dependability From Table 9, it can be seen that the item difficulties for attitude and behaviour (for each corresponding item) are in the conceptualised order of easy to hard. That is, the attitude items are all easier than their corresponding behaviour items, as was conceptualised. It can also be seen that, within each aspect, the item difficulties go from easy to hard vertically down, as was conceptualised. That is, the conceptualised structure for Dependability is supported by the Rasch measures of item difficulties calibrated on the same linear scale. This is strong support for the construct validity of the measure of Dependability.
Items Measuring Responsibility Table 10 shows the items representing all the aspects of the Responsibility scale and it is noted that the fit is in a mix of both perspectives. For example, item 51 (I do what my mum and dad ask me to do quickly and happily) fitted the Rasch model only in the attitude perspective. Students found this a difficult item and its difficulty is +0.68 logits. The same item (I do what my mum and dad ask me to do quickly and happily) in the behaviour perspective (item 52) does not fit the measurement model. Items 59 and 60 (I sensitise myself towards, and anticipate the needs of, others in my school) fit the Rasch model in both perspectives, while item 54 (I finish the service that my teachers need me to do), a moderately difficult item (+0.13 logits), fits the model only in the behaviour perspective. Students found item 58 (I am considerate towards others in school) the most difficult item (+0.84 logits). The same item in the attitude perspective (item 57) does not fit the model. Table 10. The Items of the Responsibility Scale and their Difficulties (Locations) Responsibility (Family) Item Item No. 49/50 I do what my mum and dad ask me to do. 51/52 I do what my mum and dad ask me to do quickly and happily. Responsibility (School) 53/54 I finish the service that my teachers need me to do.
I aim for this (Attitude) -0.02 +0.68
I actually do this (Behaviour) +0.64 Did not fit
Did not fit
+0.13
Liu Shiueh Ling and Russell F. Waugh
150
Table 10. (Continued). Responsibility (Family) Item Item No.
I aim for this (Attitude)
I actually do this (Behaviour)
55/56
Did not fit
+0.50
Did not fit +0.10
+0.84 +0.74
-1.92 -1.92
Did not fit +0.21
57/58 59/60
I finish the service that my teachers need me to do and I do it to the best of my abilities. I am considerate towards others in school. I sensitise myself towards, and anticipate the needs of, others in my school.
Responsibility (Friends) 61/62 I help my friends, if they ask me. 63/64 I help my friends if they ask me, and I do it quickly and happily.
Note on Table 10: 1. Item difficulties are in logits. 2. Attitudes are easier than their corresponding behaviours, when both fit the model.
Construct Validity for Responsibility There were only 11 good-fitting items in the Responsibility scale. Some fitted the Rasch model only in the attitude perspective, some fitted the Rasch model only in the behaviour perspective and others fitted the Rasch model in both perspectives. The 11 items are found in all the aspects of the Responsibility scale, namely family, school, and friends. Where the items do fit the model, it can be seen that the item difficulties for attitude and behaviour are in accord with the conceptualised structure. That is, the attitude items are all easier than their corresponding behaviour items, as was conceptualised. It can also be seen that, within each aspect where the items fit the model, the item difficulties go from easy to hard vertically down, as was conceptualised. So there is only partial support for the conceptualised structure of Responsibility.
SUMMARY Through the use of the computer programme, Rasch Unidimensional Measurement Models (RUMM 2020) (Andrich, Sheridan & Luo, 2005), linear measures of Dependability (16 items) and Responsibility (11 items) were created. The Dependability measure had good construct validity (the conceptualised structure was supported), but the conceptualised structure of Responsibility was only partially supported as some item data did not fit the measurement model and were deleted. The two linear scales had: 1. Good global item-person and global person-item fit. In the Dependability scale, this is shown by a standardised, fit residual mean of -0.45 and standard deviation of 1.52 for the items, and a mean of -0.44 and a standard deviation of 1.29 for the students. In the Responsibility scale, this is shown by a standardised fit residual mean of -0.51 and standard deviation of 0.68 for the items, and a mean of -0.45 and a standard
Rasch Measures of Dependability and Responsibility
151
deviation of 1.05 for the students. In both scales, the values are satisfactorily close to ideal fit residuals of mean near zero and standard deviation near one; 2. Good values of Cronbach Alpha and Person Separation Index. The Cronbach Alpha and Person Separation Index for the Dependability scale are 0.89 and 0.89 respectively. The Cronbach Alpha and Person Separation Index for the Responsibility scale are 0.82 and 0.82 respectively. The values for both scales showed that the student measures are well-separated in comparison to the errors (about 0.1 logits); 3. Acceptable item-trait interaction with reasonable agreement among students about the difficulties of the items along both scales. This shows that unidimensional scales were created for both variables; 4. Good individual item fit statistics for all 16 items of the Dependability scale and for 11 items out of 16 items of the Responsibility scale, with ordered item thresholds; 5. Good Response Category Curves for the good-fitting items showing that the students answered the items consistently and logically; and, 6. Distribution graphs showing that the targeting of the items against the student measures needs to be improved. More difficult items need to be added in both scales for future use and this will be discussed in the final chapter on Discussion and Implications. It was concluded that reliable linear scales were created for Dependability and Responsibility, and that they could be used in subsequent experiments from which valid inferences could be made. The following inferences can be summarised about the difficulties of the items in each variable.
Dependability (Attitude Items) The two easiest attitude items are I help my friends (item 45, difficulty=-2.54 logits and very easy) and I help my family members (item 33, difficulty=-1.22 logits and very easy). The two hardest attitude items are When I am busy with my own commitments, I still volunteer to serve the school (item 39, difficulty=+0.71 logits and quite hard) and I look for opportunities to contribute to the school (item 43, difficulty=+0.63 and quite hard).
Dependability (Behavior Items) The two easiest items are I help my friends (item 46, difficulty=-0.60 logits and moderately easy) and I help my family members (item 34, difficulty=-0.46 logits and moderately easy). The two hardest behaviour items are When I am busy with my own commitments, I still volunteer to serve the school (item 40, difficulty=+1.43 logits and very hard) and I look for opportunities to contribute to the school (item 44, difficulty=+1.19 and very hard).
152
Liu Shiueh Ling and Russell F. Waugh
Responsibility (Attitude Items) The two easiest attitude items are I help my friends, if they ask me (item 61, difficulty=1.92 logits and very easy) and I help my friends if they ask me, and I do it quickly and happily (item 63, difficulty=-1.92 logits and very easy). The two hardest attitude items are I do what my mum and dad ask me to do quickly and happily (item 51, difficulty=+0.68 logits and quite hard) and I sensitise myself towards, and anticipate the needs of, others in my school (item 59, difficulty=+0.10 logits and moderately hard).
Responsibility (Behavior Items) The two easiest behaviour items are I finish the service that my teachers need me to do (item 54, difficulty=+0.13 logits and moderately hard) and I help my friends if they ask me, and I do it quickly and happily (item 64, difficulty=+0.21 logits and moderately hard). The two hardest behaviour items are I am considerate towards others in school (item 58, difficulty=+0.84 logits and quite hard) and I sensitise myself towards, and anticipate the needs of, others in my school (item 60, difficulty=+0.74 logits and quite hard). It should be noted that students with the lowest and highest measures of Dependability and Responsibility can be identified, because the RUMM 2020 program lists the measures in order by student number. Those students who need advice and help could then be identified and provided with specialist help.
In: Specialized Rasch Measures… Editor: Russell F. Waugh, pp. 153-180
ISBN: 978-1-61668-032-9 © 2010 Nova Science Publishers, Inc.
Chapter 8
A RASCH MEASURE OF THE STUDENT ENTREPRENEURIAL MINDSET IN SINGAPORE Wong Heng Aik Jason1 and Russell Waugh2 1
Singapore Graduate School of Education University of Western Australia 2
ABSTRACT This study arises out of a growing interest in the entrepreneurship phenomenon in Singapore. It aims to investigate Students‟ Entrepreneurial Mindset (SEM) by creating a linear measure using a Rasch measurement model and by exploring the relationship between SEM and factors such as one‟s personal background. SEM is based on four aspects, Motivation to excel, Leadership, Managing opportunity and conflicts, and Creativity and innovation, seen from two perspectives (attitude and behavior). The sample size of 490 secondary and junior college students comes from two premier schools in Singapore. A linear scale measuring SEM was created with 22 items that fitted the measurement model. The model has a very good overall fit (item-trait interaction chisquare = 130.5, df = 154, p = 0.92), good global person and item fits, and good separation of measures (Person Separation Index = 0.80). These results will provide useful information for future education planning for secondary school students in Singapore.
INTRODUCTION The present study seeks to highlight the importance of entrepreneurship education in Singapore through investigating the Student Entrepreneurial Mindset for secondary school students in two premier schools in Singapore. This chapter begins by providing a background to the growing interest in entrepreneurship, both globally and locally. This is followed by a description of what constitutes the Student Entrepreneurship Mindset by relating it to what defines the adult entrepreneurial mindset. Next, the rationale and the aims of the study, the research questions, and the significance and limitations of the study are stated.
154
Wong Heng Aik Jason and Russell Waugh
BACKGROUND TO THE STUDY There has been a growing interest in the entrepreneurship phenomenon in Singapore in recent years. This was due in part to the Singapore Study of the Global Entrepreneurship Monitor1 (Centre for Entrepreneurship, National University of Singapore, 2002), which showed that Singapore fared very badly globally in terms of the entrepreneurial mindset in its people. The study initially ranked Singapore at 27th out of 29 nations in 2001 in total entrepreneurial activity. Another reason for this increased interest was the focus and attention given to innovation and enterprise by the government and the Ministry of Education. Globally, the interest and focus in entrepreneurship studies began to escalate quickly in the 1980s. The period was also known as “The Age of the Conference” in the field of entrepreneurship (Katz & Shepherd, 2003). The first major conference on entrepreneurship was convened in Baylor in 1980 (Kent et al., 1982). This was then followed by many other conferences on entrepreneurship, such as the “Gateways to Entrepreneurship Research Conferences” (Katz & Shepherd, 2003). Soon, more studies related to entrepreneurship began to take place. The most prominent studies among them are the Panel Study of Entrepreneurial Dynamics2 (Gartner, 2004) and the Global Entrepreneurship Monitor3 (Reynolds et al., 1999). The former study was carried out by Babson College, USA, while the latter by both Babson College and the London Business School. The series of Global Entrepreneurship Monitor studies (Reynolds et al., 1999; Reynolds et al., 2000; Reynolds et al., 2001; Reynolds et al., 2002; Reynolds et al., 2003; Zoltan et al., 2004; Minniti et al., 2005; Bosma & Harding, 2006; Bosma et al., 2007; and Bosma et al., 2008) began in 1999 with ten countries but soon grew quickly to 43 countries in 2008. This reflects a growing interest in entrepreneurship globally. Locally, Singapore began to participate in the study in 2000. The focus on entrepreneurship in the education sector is also increasingly becoming obvious in Singapore. Both the two main local universities, the National University of Singapore (NUS) and the Nanyang Technological University (NTU) have their own arms on entrepreneurship education. NUS set up the NUS Entrepreneurship Centre, while NTU set up the Nanyang Technopreneurship Centre to offer courses in entrepreneurship education for graduates. Some of these courses lead to a Masters degree. The two universities also pioneered studies in entrepreneurship development and research in innovation, offer incubation laboratories to researchers hoping to become budding entrepreneurs, and garner venture support for these risk takers. At the same time, the government has also in recent years approved and set up two other universities offering courses in the business and management areas, which will also contribute to the entrepreneurship phenomenon in this country. The Singapore Management University became the third local university and its campuses now lie in the very heart of the business and commercial district. In 2005, the Singapore Institute of Management was also given university status. Both of these business
1
The Global Entrepreneurship Monitor (GEM) was funded by the Kaufman Foundation and initiated in 1999 to benchmark the level of entrepreneurial activities across countries and to understand key national environmental factors affecting a country‟s entrepreneurship; it is coordinated by Babson College, London Business School. 2 Retrieved, July 18, 2007, from http://www.psed.isr.umich.edu/documents.php?c=i 3 Retrieved, July 18, 2007, from http://www.gemconsortium.org/
A Rasch Measure of the Student Entrepreneurial Mindset in Singapore
155
and management related universities will certainly play a vital role in enhancing the focus on entrepreneurship in Singapore. Over the last few years, rapid and significant changes have been made to the nation‟s educational policy from primary schools to junior colleges in the hope of better preparing the next generation for the future. One of these was the focus on innovation and enterprise, targeting at preparing a workforce with a strong innovative and entrepreneurial spirit. At its annual Work Plan Seminar in 2003, the Ministry of Education announced its new phase in education, which was to place special focus on innovation and enterprise in the schools (Shanmugaratnam, 2003). Its goal was to better prepare the next generation to be ready to manage the changing landscape of the economy in a more complex future. In academia, research on entrepreneurship has gone full circle. Initially, it focused on the individual level, that is, the entrepreneur. Then it moved on to multi-levels, encompassing the factors contributing to entrepreneurial startups, such as the role of venture capital, the place of creativity and innovation in entrepreneurship, technological entrepreneurship, corporate entrepreneurship, the role of culture in entrepreneurship. In recent years, it shifted back to the characteristics of the individual entrepreneur. Globally, much research has been done on adult involvement in entrepreneurial ventures. Research on adults in entrepreneurship in Singapore is still in its early stages. However, there is no research on entrepreneurship or the entrepreneurial mindset of secondary school students either globally or in Singapore. There is also no local entrepreneurship journal published. The few papers that have been published about Singapore are only found in journals published overseas in the United Kingdom and the United States. In consequence, research in the Singapore students‟ entrepreneurial mindset is absent. Hence, the present research has set out to study Student Entrepreneurial Mindset in Singapore and, in the process, to highlight the importance of entrepreneurship education in Singapore through investigating this mindset.
WHAT CONSTITUTES THE STUDENT ENTREPRENEURIAL MINDSET? In order to comprehend what constitutes the characteristics of the student entrepreneurial mindset, this research examines the characteristics of the adult entrepreneurial mindset. Timmons and Spinelli (2004) proposed a number of aspects of which four aspects were then identified by the present study as characteristics of the entrepreneurial mindset in successful adult entrepreneurs. This section describes the four aspects.
Aspects of the Successful Adult Entrepreneurial Mindset The first aspect is the intrinsic Motivation to excel in whatever they do. Successful adult entrepreneurs are intrinsically motivated to excel in their daily dealings and ventures. They tend to be internally driven by their own self-imposed, challenging goals to achieve their own goals. Simpler goals often lead to higher goals. They are usually neither power nor status hungry, but tend to share their vision and goals with their team through working collaboratively with them to achieve their commonly held goals. The second aspect is strong
156
Wong Heng Aik Jason and Russell Waugh
Leadership qualities. Entrepreneurial leaders display their leadership in three different phases of the venture cycle, namely, the pre-creation, start-up and consolidation phases. Each of the phases requires different leadership behaviours. Hence, successful adult entrepreneurs are transactional, transformational and instrumental all at once. On the transactional level, they are experts in their industry and in general management and administration. They are persistent problem solvers but are also people who are not afraid of withdrawing from a wrong decision if they have to. On the transformational level, they are patient, trusting, and are able to build faith in their team members. They have strong interpersonal skills and possess good team dynamics so that they convince rather than coerce their members to get things done. Although seen as strong communicators, they are usually more pragmatic than invincible. This pragmatism and vulnerability often wins them support from their team. On the instrumental level, they are courageous risk takers, but are also visionaries, self-initiators, and strategists who stay focused until their goals are achieved. The third aspect is their strength in Creativity and innovation. Successful adult entrepreneurs operate in uncertain situations where change is a daily constant. They are resilient, and they adapt and respond to business and economic changes very quickly so as not to be adversely affected by any downturn in the economy or disruptions in their industry. They are innovative all the time and see possibilities for innovation in every appropriate business and economic situation. Because they are not afraid of failures, successful entrepreneurs will keep on trying so that they may just succeed. They somehow believe that the iterative process of trial-and-error will make them better entrepreneurs the next time round. The fourth aspect is their Obsession with opportunity. The successful entrepreneur is an opportunist who never fails to see an opportunity, both from within the company and without. He often seeks to create new resources from existing materials and to create new values for potential customers. Because of the swift rate of change of opportunities in the economy in today‟s context, the entrepreneur is expected to be highly tolerant of risks, ambiguity and uncertainty.
Identifying the Student Entrepreneurial Mindset There are three considerations in identifying what should constitute the Student Entrepreneurial Mindset. The first identifies the aspects of the Student Entrepreneurial Mindset. The second identifies the two perspectives to be considered, the attitudinal and the behavioural perspective. The third identifies what are some factors present in the students‟ lives such as his personal attributes and his personal experiences which may contribute to developing his entrepreneurial mindset. Firstly, following the four aspects of the adult entrepreneurial mindset, the four aspects identified for the Student Entrepreneurial Mindset model are Motivation to excel, Leadership, Creativity and innovation, and Managing opportunities and conflicts. The first three aspects are similar to three of the aspects identified as characteristics of the successful adult entrepreneurial mindset. Besides managing opportunities, the fourth aspect included managing conflicts as well. To assist students in interpreting these aspects in their own level of understanding and experiences, it was decided that students should think about their own experiences in school as they think about each of the four aspects. Thus students were encouraged to relate their own experiences in the classroom, in their school project work
A Rasch Measure of the Student Entrepreneurial Mindset in Singapore
157
activities, in their leadership roles in school as well as in their participation in additional activities in school such as their individual, compulsory Co-curricular Activity4. Secondly, since students did not have any actual entrepreneurial experiences, it was necessary to identify what students thought about the mindset. Thus there were two perspectives that were considered in this study. The first perspective focuses on students‟ attitudes towards their own entrepreneurial experiences. This is classified as the Ideal perspective. This perspective requires students to take a contemplative reflection of how they would have liked to react, or think they would usually react, to a particular situation. Since behaviour is often seen as the outcome of one‟s attitude towards different situations (Jaccard & Blanton, 2005), this perspective puts the students in a reflective mode before they reflect on their actual behaviours. The second perspective focuses on students‟ actual behaviours in the four aspects. This is classified as the Real perspective. This perspective requires students to recall how they actually reacted in real situations. Because our behaviours sometimes differ from our attitudes, since it is often harder to carry out one‟s intentions due to various other factors surrounding the situation, these two perspectives help to provide a better picture of the Student Entrepreneurial Mindset. It is generally expected that respondents will find it easier to respond to the Ideal perspective than the Real perspective. Thirdly, it was necessary to identify the factors that are present in the students‟ lives such as their personal attributes and personal experiences which may contribute to developing their own entrepreneurial mindsets. The seven factors considered in this study are the participants‟ Gender, Age, Year of study (in school), Academic ability, Leadership experiences, Types of Co-Curricular Activities, and Parents’ occupations. It was believed that younger male participants and those who are more academically able are probably more entrepreneurial in their mindsets. Since the Ministry of Education has realigned its emphasis to focus on innovation and enterprise in the last few years, it was believed that the younger participants will have had more opportunities to gain from this focus and learned to be innovative and enterprising at a younger age. Also as one gets older and gains more experience, it is possible that one become less risk conscious and take the safer options. As for gender, cultural and societal norms about women‟s roles in society affect how each of them may picture themselves and their future careers. While more women have taken up careers of their own in this country, the number of female entrepreneurs is still relatively smaller compared to males. As for academic ability, those who do well in school are often seen as those who will be more successful in their endeavours. Those who have more exposure to leadership roles in school were also believed to have better entrepreneurial mindsets. The influence of parents, their careers and their entrepreneurial experiences, are also believed to contribute to the child‟s ambitions and career options. Parents who are businessmen or entrepreneurs are more likely to influence their children‟s attitudes to entrepreneurship and risk taking.
4
Co-Curricular Activity (CCA) in school refers to an activity in school that a student takes up outside of his classroom experience. This is a compulsory activity for all students. It could refer to a sport, a uniformed group like the Scouts Troop, a musical society or an academic related club.
158
Wong Heng Aik Jason and Russell Waugh
RATIONALE There are four main reasons this study was undertaken. Firstly, as mentioned at the start, there is a growing interest in the area of entrepreneurship, both globally and locally. This interest has led to many studies on the entrepreneurial frameworks of nations, and studies on individual entrepreneurs. In Singapore, this interest has prompted many new initiatives from both the public and private sectors to improve the mindset of the citizenry, such as the focus on Innovation and Enterprise by the Ministry of Education. Secondly, findings from the Singapore Study of the Global Entrepreneurship Monitor have shown that Singaporeans fared badly in relation to many countries globally. Singaporeans do not seem to be as entrepreneurial as others. However, the changing landscape of education in Singapore in recent years seems to disagree with this finding. The increasing emphasis given to entrepreneurship education at the post secondary and tertiary level must certainly mean that more Singaporeans are becoming more entrepreneurial than the earlier generations. With the focus on Innovation and Enterprise in the education sector, it is expected that schools will place more emphasis in this area, and hence encourage the entrepreneurial mindset. Thirdly, schools provide plenty of experiences to their students in developing the four aspects of the entrepreneurial mindset, although this is not specifically and systematically done. Many of the activities in school provide students with experiences in leadership, creativity and innovation, motivation to excel and opportunity management. Yet, there has been no attempt to see how these activities contribute to the development of the student entrepreneurial mindset. Fourthly, many private organizations have begun to offer programmes in Entrepreneurship to schools. These programmes often aim to teach students how to start a business or write business plans. At the High School where the survey for this study is conducted, an Entrepreneurship Programme was also initiated in the last few years by this researcher and a few other teachers in the school. The programme, which also aims to teach students to be more entrepreneurial, takes a pragmatic approach to learning through case studies, and through interactions with local entrepreneurial companies as well as local entrepreneurs and leaders in the political and economic arena who play an important role in shaping the nation‟s entrepreneurial environment. The present study has taken a mix of quantitative and qualitative approaches. There are certain advantages in using a quantitative survey in this study. Firstly, since there is no existing instrument to measure the entrepreneurial mindset of young school-going students, a quantitative approach will aid in designing a linear scale that can be used in future. Secondly, a survey will reach a larger and wider sample. A stratified sample will ensure that the sample is representative and helps to see how different groups of students, such as those with different personal experiences (leadership opportunities and attendance at an entrepreneurial course) or from different home background, age, achievement or gender rate themselves on the scale. To create a linear measure, Rasch analysis (Waugh, 2006; Wright, 1999; Andrich, 1988) is considered instead of traditional methods of analysis because Rasch analysis not just provides a linear measure, it also provides a test of fit to the measurement model and it checks for unidimensionality. It is also currently the world‟s best measurement practice in the human sciences and the RUMM computer programme (Andrich et al., 2005) is the best programme to make the linear measures because of its extensive graphical and numerical outputs. To supplement the findings of a quantitative study, a qualitative portion helps to gather students‟
A Rasch Measure of the Student Entrepreneurial Mindset in Singapore
159
opinions on how the study can be further improved in future, as well as how the entrepreneurial mindset in students can be further developed in schools in future.
AIMS OF THE STUDY This research aims to study students‟ entrepreneurial mindset by creating a linear measure based on four aspects: Motivation to excel, Leadership, Creativity and innovation, and Managing opportunities and conflicts. It will also explore some of the independent factors that affect this mindset. These factors include Gender, Age, Year of study (in school), Academic ability, Leadership experiences, Types of Co-Curricular Activities, and Parents’ occupations. Briefly, this research aims to create a linear measure of the dependent variable, Student Entrepreneurial Mindset. This created measure involves both an attitudinal and a behavioral perspective, to be termed the Ideal perspective and the Real perspective respectively. Attitudes are expected to be easier than behaviors (Jaccard & Blanton, 2005) because behaviors are usually seen as outcomes of attitudes, while an attitude refers to a state of mind, and one often finds it easier to think of what one wants to do, then to actually do it.
RESEARCH QUESTIONS What is the level of entrepreneurial mindset in Singaporean secondary school students?” The specific questions are: 1. Is it possible to create a linear measure of Student Entrepreneurial Mindset? 2. Are attitudes easier than behaviours for all Student Entrepreneurial Mindset items? 3. Which aspects in entrepreneurship are easiest and which aspects are hardest for students? 4. How do students rate themselves on a Student Entrepreneurial Mindset scale? 5. What are some things schools can do to raise the level of entrepreneurial mindset of students? Second research question “What is the relationship between the dependent variable, that is the Student Entrepreneurial Mindset, and the seven independent variables, that is Gender, Age, Year of study (in school), Academic ability, Leadership experiences, Types of Co-Curricular Activities, and Parents’ occupations?”
SIGNIFICANCE OF THE STUDY This study is significant for three reasons. Firstly, this study helps to highlight to the leaders in education the need to raise the level of entrepreneurial consciousness among students to a higher level. Entrepreneurship has been shown to contribute to the growth of an
160
Wong Heng Aik Jason and Russell Waugh
economy (see the Global Entrepreneurship Monitor Reports, 2000-2008). When the citizenry is more entrepreneurial, more economic activities take place, thereby increasing the vibrancy of the national economy. In view of this, many economies have embarked on promoting entrepreneurship among their citizens. Singapore, too, have, in recent times, pushed for a more innovative society. New initiatives and new moves have been made to encourage a more entrepreneurial culture in the country, such as the government‟s shift of focus from productivity to enterprise in 2001 when they positioned the Singapore Productivity and Standards Board to Standards, Productivity and Innovation Board, Singapore (or SPRING Singapore as it is commonly labeled), by including the word, Innovation, to enable and promote enterprise development. Additionally, there is the call for a focus on Innovation and Enterprise in schools. However, research on entrepreneurship has, up till now, been carried out mostly with adult participants. Usually these are entrepreneurs or entrepreneurial organizations. Very few studies focused on the next generation, which are the youths of the nations or students in secondary schools. This study has highlighted the need to raise the level of entrepreneurial consciousness of students in Singapore, and the need to systematically teach entrepreneurship in schools. Secondly, since studies have focused on the adult population all this while, there exists no scale to measure the entrepreneurial mindset of students. This research has developed and created a new linear scale to measure the Student Entrepreneurial Mindset in Singapore using the world‟s best measurement model, the Rasch measurement model. This model defines the aspects and perspectives to measure Student Entrepreneurial Mindset. It shows which are the easier and which are the harder aspects and perspectives of the mindset. This scale will benefit students and schools. By using this scale, the idea of an entrepreneurial mindset is brought to a conscious level in students. Students will be able to gauge their own entrepreneurial strengths. This may provide a prompt for them to decide which areas to work and focus on developing. Also, students who are lower in the entrepreneurial mindset can work on the areas they are lacking in entrepreneurial strengths. Schools can also begin to redesign their curriculum and activities to actively promote the development of the entrepreneurial mindset in students. The third follows from the above two. This research complements the growing interest in the entrepreneurship phenomenon among Singaporean schools in recent years. Many private organisations have started to offer their entrepreneurship programmes to schools but none of these are grounded on „proper‟ research. Some schools, such as the High School where this research was carried out, have also crafted their own programmes for their students, but these are also not the result of proper research. This study calls for more research in this area, and has given some directions for further research in entrepreneurship education in secondary schools. In this way, it lends support to the proper crafting of an Entrepreneurship curriculum at the secondary school level in Singapore.
A Rasch Measure of the Student Entrepreneurial Mindset in Singapore
161
LIMITATIONS OF THE STUDY Restricted by time and space, there are three main limitations to this study. Firstly, the study is based mainly on the perspectives of students. The participants involved in the three phases of the pilot study as well as in the actual study are students. No teachers or other adults, such as entrepreneurs, were consulted in the process. Hence this study only focused on the opinions of students alone. However there is no need for too much concern here since the results of the Rasch analysis shows a high level of overall fit to the measurement model (item-trait interaction chi-square = 130.5, df = 154, p = 0.92), good global person fit, good global item fit, good individual item fit, good targeting of the student measures against the item difficulties, good discrimination, and good separation of measures compared to errors of measurement (the Person Separation Index was 0.80). Secondly, the sample for this study comes from only two top secondary schools in Singapore. The students are among the top three percent of their respective cohorts in terms of their academic abilities. It may thus seem that the results, and the scale that was created, may not be relevant for the rest of the student population, since the students in the sample comes from the upper percentile of the population in terms of their academic ability. However, this may not necessarily be a bad thing for a few reasons. First, it would certainly be relevant for rest of the students in the upper percentiles of the population. Second, the feasibility of creating a linear scale of the Student Entrepreneurial Mindset for this mindset implies that this scale can be further improved to include a wider, if not, the whole spectrum of the student population. Third, the findings related to some of the aims have important implications for the whole student population in Singapore, and perhaps the rest of the world, if educational leaders here and elsewhere begin to see that there are indeed many areas that can be improved in the school curriculum so that students can become more entrepreneurial in their mindsets. Thirdly, the participants in the sample involved in the focus group interviews are all Secondary Three and Four students from the High School. These students have either been involved in working on a long term entrepreneurship project in school, or are undertaking a course on entrepreneurship in the school, in an Entrepreneurship Programme put together by a few of the teachers in the school. This may have limited the findings on the role of Entrepreneurship Programme in school to only the opinions of those who are already in the programme. It does not take into consideration the opinions of those who are not in the programme. But this may not necessarily be a negative thing. Those who are undergoing the programme or are involved in a long term entrepreneurship project will be in a better position to make comparisons and share their actual experiences, and hence make informed decisions and opinions of the usefulness of such a programme. This chapter describes the process of data analysis to create the Student Entrepreneurial Mindset Scale, using the Rasch Unidimensional Measurement Model (RUMM 2020) program (Andrich, Sheridan & Luo, 2005). First, it describes how, through a process of item re-scoring and deletion, a linear scale of Student Entrepreneurial Mindset was created with 22 items. Second, it summarises the statistics of the scale and provides information about the items that fit the scale. Third, it compares the expected order of item difficulty with the actual order of item difficulty for all the items in terms of the four aspects of the Student Entrepreneurial Mindset Scale. Fourth, it provides a summary.
Wong Heng Aik Jason and Russell Waugh
162
INITIAL ANALYSIS USING THE RUMM 2020 PROGRAM The Rasch Unidimensional Measurement Model (RUMM 2020) program (Andrich, Sheridan & Luo, 2005) was used to analyse which of the 44 items in the questionnaire fit a linear scale for the Student Entrepreneurial Mindset. To do this, 814 responses to the 44 items, and all personal information about the students were first collated and inputted into a text file that was read by the RUMM 2020 program. The personal data included such aspects as Gender, Age, Year of study (in school), Academic ability, Leadership experiences, Types of Co-curricular Activities, and Parents’ occupations. Table 1 shows the range of personal data that was read into the RUMM programme. Table 1. Range of personal data input into the RUMM programme Type of data
Response Categories
1
Student responses to the 44 items
2
Age
3
Year of study
4
Gender
5
Academic Ability
6
Leadership experiences
7
Types of Cocurricular Activities
If this has NEVER happened in your school life If this SELDOM happens in your school life If this happens SOMETIMES in your school life If this OFTEN happens in your school life Missing information 12 years old 13 years old 14 years old 15 years old 16 years old 17 years old 18 years old 19 years old 20 years old Missing information Secondary One Secondary Two Secondary Three Secondary Four Junior College 1 Junior College 2 Missing information Male Female Missing Information Lower ability Mid ability High ability Missing Information None Lower level Mid level High level Missing Information Clubs and societies Performing arts and musical groups Sports Uniformed groups Response Categories
Type of data
Input into RUMM 2002 programme 1 2 3 4 9 12 13 14 15 16 17 18 19 20 99 1 2 3 4 5 6 9 1 2 9 1 2 3 9 0 1 2 3 9 1 2 3 4 Input into RUMM 2002
A Rasch Measure of the Student Entrepreneurial Mindset in Singapore 8
programme 4 3 2 1
Business Professional Others Homemaker, retired or not working
Parents‟ occupations
163
Source: Designed by author for the Student Entrepreneurial Mindset Questionnaire.
The Partial Credit Model of the Rasch was used with the RUMM 2020 program to analyse the data from the 44 items. First, item thresholds were checked to see if all items had ordered thresholds. Items that had ordered thresholds were answered consistently and logically. Five items (Items 1, 3, 11, 13 and 21) had disordered thresholds (see Table 2) and it can be seen that the inconsistency occurs in Thresholds 1 and 2. This suggests that students could not discriminate consistently and logically between the lower response categories „Never‟ and „Seldom‟ (Threshold 1) and between „Seldom‟ and „Sometimes‟ (Threshold 2) for these five items. Hence, the two lowest response categories for the five items were combined into one response category for Items 1, 3, 11, 13 and 21. Table 2. Items with disordered thresholds Item 1 3 11 13 21
Threshold 1 - 0.80 - 0.66 - 0.12 - 0.18 - 0.34
Threshold 2 - 0.99 - 1.07 - 0.84 - 0.56 - 1.10
Threshold 3 1.80 1.73 0.95 0.75 1.43
Note to Table .2 1. Thresholds are points between adjacent response categories where the odds are 1:1 of answering in either category. When category responses are used consistently and logically, thresholds are ordered in line with their conceptual order.
Second, the chi-square probabilities for the individual item–fits were examined. Through a series of item deletions based on the poor chi-square probability for the individual item–fit, a total of 22 items were deleted. These were Items 5, 6, 7, 8, 9, 10, 17, 19, 20, 21, 23, 27, 29, 30, 31, 32, 35, 36, 39, 40, 42, and 43. Of the 22 items that were deleted, 13 were from the Ideal perspective and nine from the Real perspective. In addition, 324 student responses were deleted because of severe misfit and because of maximum and minimum raw scores with 22 items, leaving N=490 in the final Rasch scale. Third, the data for the 22 items that fitted the measurement model were then re-analysed with the RUMM 2020 program to create a unidimensional, linear, Student Entrepreneurial Mindset Scale.
FINAL ANALYSIS WITH THE RUMM2020 PROGRAM The final analysis with the RUMM2020 computer program involved 22 items, N=490 students, and some re-organising of the response categories, as previously explained. This resulted in a very good fit to the measurement model with ordered item thresholds, good global item and person fits, good individual item fits, and good targeting. Evidence for these
Wong Heng Aik Jason and Russell Waugh
164
comments is now explained, and the meaning of the unidimensional scale of Student Entrepreneurial Mindset expounded.
Item Thresholds Items thresholds are positions on the scale between adjacent response categories where the odds are 1:1 of participants responding in either category, with respect to an individual‟s ability to respond to that item. For good measurement, all the thresholds should be ordered in line with the conceptual ordering of the response categories, as it now is for these data with 22 items. Table 3 sets out the thresholds by items. In this set of data, some of the items have only two thresholds although there were originally four response categories. This is because, if the items did not produce ordered thresholds, response categories 1 and 2 were combined into one category. After combining the respective categories, the thresholds for all the 22 items are ordered. Of these 22 items, four of them had combined thresholds. These are Items 1, 3, 11 and 13. Table 3. Thresholds Values for Items Measuring the Student Entrepreneurial Mindset Item 1 2 3 4 11 12 13 14 15 16 18 22 24 25 26 28 33 34 37 38 41 44
Item mean location +0.19 +0.92 −0.74 −0.07 +0.01 +0.20 −0.62 −0.51 −0.73 +0.46 +0.09 +0.03 +0.76 −0.13 +0.73 +0.70 −1.71 0.00 −0.65 +0.35 −0.08 +0.79
Threshold 1 −1.18 −1.64 −2.14 −2.93 −0.75 −1.04 −1.19 −2.21 −2.24 −1.70 −2.11 −0.81 −1.84 −1.58 −1.73 −1.60 −4.42 −2.36 −1.87 −1.73 −1.56 −1.53
Threshold 2 +1.55 +0.90 +0.67 +0.00 +0.78 −0.25 −0.06 −0.54 −1.19 +0.46 −0.13 −0.65 +0.81 −0.85 +0.95 +0.64 −1.30 0.00 −1.50 0.15 −0.50 +0.77
Threshold 3 +3.51 +2.72 +1.90 +1.22 +1.25 +2.63 +2.50 +1.54 +3.29 +2.04 +2.97 +3.07 +0.60 +2.37 +1.42 +2.63 +1.82 +3.13
Notes to Table 3 1. The item mean location, which indicates the level of item difficulty, shows the mean threshold location for the respective item. 2. Items thresholds are positions on the scale between adjacent response categories where there are odds are 1:1 of answering either of the adjacent response categories. For good measurement, all the thresholds should be ordered in line with their conceptual order.
A Rasch Measure of the Student Entrepreneurial Mindset in Singapore
165
3. Some items reflect only two thresholds as their response categories were combined in the process to ensure that the thresholds are ordered.
Global of Tests-of-Fit Statistics The RUMM 2020 program generates various statistics to assess the overall level of fit of the items and students (see Table 4). First, the mean of the fit statistic for the items and students are +0.04 and –0.30 respectively, which are near to zero, and the fit statistic standard deviation for the items and students are +0.70 and +1.38 respectively, which are near to one. When the fit means are near zero and their standard deviations are near one, there is usually a good fit to the measurement model. Second, the RUMM 2020 program generates a chi– square probability to ascertain the item-trait test–of–fit to the measurement model. In this 22item data set, the item-trait chi–square probability is 0.92 (chi–square = 130.47, total degrees of freedom = 154, probability = 0.92). This shows that there is no significant item-trait interaction along the scale and indicates that there is a very good agreement among the students about the difficulties of the 22 items right along the scale. This also means that a unidimensional scale has been created, and that each student‟s responses can be predicted using a single student parameter (measure) for each student and a single parameter (item difficulty) for each item. Third, the Student Separation Index is 0.80, and this means that the measures are well-separated in comparison to the errors (a requirement for reliable measurement) and this, in turn, means that the power of the tests-of-fit is good. Table 7.4. Summary statistics for the 22 items that fit the scale (N=490) Items Number 22 Location mean +0.00 Standard deviation 0.64 Fit Statistical mean +0.04 Standard deviation 0.70 Total item-trait interaction chi square = 130.47 Total degrees of freedom = 154.00 Chi-square probability of item-trait = 0.92 Student Separation Index = 0.80 Power of test-of-fit: Good (based on the separation index)
Students 490 +1.36 0.81 – 0.30 1.39
Notes to Table 4 1. The item means are constrained to zero by the measurement model. 2. When the data fit the measurement model, the fit statistics approximate a distribution with a mean near zero, and a standard deviation near one (an acceptable fit in this case). 3. The item-trait interaction indicates strong agreement displayed across all students for all item difficulties along on the scale. 4. The Student Separation Index is the proportion of observed variance considered true (in this scale, 0.80, which is good). 5. The numbers are given to two decimal places because the errors are to two decimal places.
166
Wong Heng Aik Jason and Russell Waugh
TARGETING OF ITEMS AGAINST MEASURES Figures 1 and 2 show the Item locations and the student measures on the same scale. Student measures range from – 1.06 logits to +4.05 logits. 68% of them have measures higher than the location of the highest item mean (+0.92 logits). So the targeting by item difficulty appears to be not so good: the set of items appear to be too easy for this group of students. But, observations from Figure 3 show that when the thresholds are taken into account, the targeting is very good. The five hardest items are from the Real perspective (Items 2, 44, 24, 26 and 28). Item 2 is the hardest item. The five easiest items are from the Ideal perspective (Items 3, 3, 15, 37, 13). Item 33 is the easiest item.
Notes to Figure 1 I0028 refers to Item 28, I0026 refers to Item 26 ... Items are grouped by their item locations. For example, the locations of I0028, I0026, I0004 and I0002 are from 0.50 logits to 1.00 logits. The bars on the left refer to student measures, which are placed on the same linear scale as the item difficulties. Figure 1. Student Measures by Item Difficulty Graph.
A Rasch Measure of the Student Entrepreneurial Mindset in Singapore
167
Notes on Figure 2 1. The scale is in logits 2. Student measures (from lowest to highest) are placed on the upper side of the scale, and item locations (from easiest to hardest) are placed on the lower side of the scale. Figure 2. Student Measure (Location) Item Difficulty Graph.
Figure 3. Student Measure (Location) Item Threshold Location Graph.
ITEM CHARACTERISTICS Individual Item Fit Table 5 shows the statistics for all the items that fit the measurement model. All items fit with p>0.14 and show a very good fit. Table 6 shows the individual probability of fit by item difficulty (location).
Wong Heng Aik Jason and Russell Waugh
168
Table 5. Item-fit statistics for Student Entrepreneurial Mindset scale Item location -1.71 (easiest)
Standard error
Residual
Degrees of freedom
Chi-square
Chi-square probability
0.09
+0.30
462.02
3.15
0.87
3
-0.74
0.09
-0.08
462.97
8.65
0.59
15
-0.73
0.08
-0.02
463.92
4.24
0.60
37
-0.65
0.08
+0.62
464.87
3.06
0.88
13
-0.62
0.10
-0.11
462.02
2.81
0.16
14
-0.51
0.07
+0.16
462.02
3.29
0.86
25
-0.13
0.08
-0.06
462.97
6.93
0.47
41
-0.08
0.07
+0.96
463.92
5.86
0.56
4
-0.07
0.08
+0.02
464.87
3.03
0.28
34
+0.00
0.07
+0.89
462.02
5.76
0.57
11
+0.01
0.08
+1.33
464.87
8.04
0.14
22
+0.03
0.07
-1.37
463.92
6.62
0.44
18
+0.09
0.08
-1.06
463.92
4.01
0.75
1
+0.19
0.08
+1.09
462.02
10.61
0.88
12
+0.20
0.07
+0.54
462.97
5.55
0.90
38
+0.35
0.07
-0.33
462.97
7.32
0.40
16
+0.46
0.07
-1.01
464.87
5.53
0.78
28
+0.70
0.07
+0.05
463.92
7.97
0.34
26
+0.73
0.07
-0.82
462.97
7.29
0.44
24
+0.76
0.07
+0.00
462.97
6.89
0.40
44
+0.79
0.07
-0.14
462.97
2.93
0.89
2
+0.92 (hardest)
0.07
-0.06
462.97
10.96
0.33
Item 33
Notes on Table 5
1. 2. 3. 4. 5. 6. 7.
The item location parameter is interpreted as item difficulty. Standard error refers to the degree of uncertainty and is low. Residual refers to the difference between the expected value on an item, calculated according to the Rasch measurement model, and its actual value. Degrees of freedom refers to the number of scores in a distribution that are free to change without changing the mean of the distribution. Chi-square probability refers to the levels of certainty to which an item fits the model. All the numbers are given to two decimal places because the errors are to two decimal places. This table is sorted in ascending order of item difficulty.
A Rasch Measure of the Student Entrepreneurial Mindset in Singapore
169
Table 6. Chi-Square Probability of Fit statistics Item
Chi-square probability
Item location
12
0.90
+0.20
44
0.89
+0.79
1
0.88
+0.19
37
0.88
-0.65
33
0.87
-1.71
14
0.86
-0.51
16
0.78
+0.46
18
0.75
+0.09
15
0.60
-0.73
3
0.59
-0.74
34
0.57
+0.00
41
0.56
-0.08
25
0.47
-0.13
26
0.44
+0.73
22
0.44
+0.03
24
0.40
+0.76
38
0.40
+0.35
28
0.34
+0.70
2
0.33
+0.92
4
0.28
-0.07
13
0.16
-0.62
11
0.14
+0.01
Notes on Table 6
1. This table is sorted in descending order of chi-square probability.
Category Response Curves The RUMM2020 program produces Category Response Curves for each item so that a check can be made on how student responses change with changes in the measure. An example is given in Figure 7.4 for Item 2. This figure shows that the students have used the response categories consistently and logically. When measures are very low, students have a high probability of responding in the lowest category (score 0) and a low probability of responding in the next category (score 1) or higher categories (score 2, 3). As the measure increases, the probability of scoring 1 increases and the probability of scoring 0 decreases, as it should. This trend continues until, at the highest measures, the probability of scoring 3 is high and the probability of scoring 2, 1 or 0 is very low. All other items had good category response curves.
170
Wong Heng Aik Jason and Russell Waugh
Notes on Figure 4 1. The RUMM 2020 program scores the responses of the questionnaire as 0, 1,2 and 3. 2. The category 0 curve indicates that if the student‟s measure is located at –4.00 logits, there is a probability of 0.9 of answering Never Happened to this item. 3. From category curves 0 and 1, a student with a measure of – 1.64 logits (Threshold 1) has a 0.5 chance of responding to either Never Happened or Seldom happens 4. From category curves 1 and 2, a student with a measure of +0.90 logits (Threshold 2) has a 0.5 chance of responding to either Seldom happens or Happens sometimes. 5. From category curves 2 and 3, a student with a measure of +3.51 logits (Threshold 3) has a 0.5 chance of responding to either Happens sometimes or Often happens. 6. From category 3 curve, we can conclude that only students with measures larger than 0 logits are able to answer positively to Item 2. Those at +6.00 logits have about 0.92 probability of answering with the response, Often happens. Figure 4. Category response curve - Item 2.
Item Characteristic Curves The RUMM2020 program produces characteristic curves for each item which show how the item discriminates near its difficulty. Such a curve is shown for Item 2 in Figure 5. Item 2, “In reality, I look for new challenges that test my abilities”, is hard, has a mean location of +0.92 logits (hardest item), and a chi–square probability of fit of 0.33 (reasonable fit). It can be seen from Figure 5 that most of the observed values are close to the expected values. The curve shows that students have responded to this item as expected, and that it discriminates very well. All the other items also showed good or acceptable characteristic curves.
A Rasch Measure of the Student Entrepreneurial Mindset in Singapore
171
Notes on Figure 5
1.
2. 3.
4.
The item characteristic curve describes the difficulty of the item, and shows how well an item can differentiate between lower scores below the item location and higher scores above the item location. The steeper the curve in its middle section, the better the item can discriminate. The flatter the curve, the less the item is able to discriminate. The curve shows the expected value for the items for different persons. The dots are class intervals of responses, and shows if the actual scores are higher or lower than the expected values. For example, the first dot at the lower end is slightly below the curve, indicating that the actual values for this class interval are slightly below the expected values. Since the actual scores are quite close to the expected values for this item, the dots appear very close to the curve; this shows that the item fits very well in the scale.
Figure 5. Item characteristic curve – Item 2.
General Description of the Student Entrepreneurial Mindset Scale The 22 items that fit the measurement model and define the scale of Student Entrepreneurial Mindset are shown in Table 7. In terms of the four aspects, eight of the 22 items are from the Motivation to Excel aspect, seven of the items are from the Leadership aspect, three of the items are from the Creativity and Innovation aspect, and four of the items are from the Managing Opportunities and Conflicts aspect. In terms of the two perspectives, nine of the items are from the Ideal perspective (Ideally, I should aim to do this), and 13 of the items are from the Real perspective (In reality, I actually do this). Of the 22 items, eight of the items in the Ideal perspective and their corresponding Real perspective fit the measurement model.
172
Wong Heng Aik Jason and Russell Waugh Table 7. Items that fit the Student Entrepreneurial Mindset scale Items in the Ideal perspective that fit the measurement model: Motivation to Excel: 1. I look for new challenges that test my abilities 3. I strive towards excellence in challenging activities 11. I like to work with team members to achieve 13. I like my team members to cooperate to achieve, rather than compete against each other
Leadership: 15. Once I have made a decision to solve a problem, I will sacrifice personal time to ensure that an answer to the problem is obtained 25. I spend time thinking about different ways my team can achieve a goal and solve problems Creativity and Innovation: 33. When I work on a new project, I will avoid mistakes I encountered in similar tasks before (Ideal) Managing Opportunities and Conflicts 37. I regularly review and make changes to my original plans when interesting opportunities arise to achieve 41. I let my team members know the kinds of conflicts involved in achieving a goal or solving a problem Items in the Real perspective that fit the measurement model: MOTIVATION TO EXCEL: 2. I look for new challenges that test my abilities 4. I strive towards excellence in challenging activities 12. I like to work with team members to achieve 14. I like my team members to cooperate to achieve, rather than compete against each other Leadership: 16. Once I have made a decision to solve a problem, I will sacrifice personal time to ensure that an answer to the problem is obtained 18. When I have made a wrong decision on a problem solving task, I try some other method to obtain an answer 22. I attribute our successes to my team members when we have achieved a goal 24. I have a strong sense of working out the best methods to use to get a team to achieve 26. I spend time thinking about different ways my team can achieve a goal and solve problems Creativity & innovation: 28. I look for alternative and innovative solutions and methods of doing things, solving problems or achieving goals 34. When I work on a new project, I will avoid mistakes I encountered in similar tasks before Managing Opportunities and Conflicts: 38. I regularly review and make changes to my original plans when interesting opportunities arise to achieve 44. There are often different methods of achieving a goal and I manage conflicts about methods within my team
A Rasch Measure of the Student Entrepreneurial Mindset in Singapore
173
Of those items that did not fit the measurement model, six of the items are from the Motivation to Excel aspect (Items 5, 6, 7, 8, 9, 10), five are from the Leadership aspect (Items 17, 19, 20, 21, 23), five are from the Creativity and Innovation aspect (Items 27, 29, 30, 31, 32), and six are from the Managing Opportunities and Conflicts aspect (Items 35, 36, 39, 40, 42, 43). The RUMM2020 computer program does not tell a researcher why these items do not fit the measurement model, only that they do not fit. An examination of these 22 non-fitting items shows that there are 16 items where both the ideal and Real perspective items do not fit. These are Items 5 and 6, 7 and 8, 9 and10, 19 and 20, 29 and 30, 31 and 32, 35 and 36, and 39 and 40. An explanation can be found for Items 7 and 8, “I like to be in-charge of the overall situation as I trust my own judgments”, and Items 9 and 10, “I like to influence my team (or group of students) to do things “my way”. These two sets of items are reversed scored items, which could have explained why the students were inconsistent in their responses to these items. Further examination of the wording of the other non-fitting items does not provide any obvious reasons for the non-fit, and it would appear that students are simply not thinking of these non-fitting items as part of the concept of an entrepreneurial mindset.
Five Easiest Items The five easiest items are shown in Table 8. While all five items are from the Ideal Perspective, they are not from a single aspect alone but from all the four aspects. There are two items from the aspect, Motivation to Excel, while the rest of the three aspects have one item each among these five easiest items. Item 33 (−1.71 logits, chi square probability = 0.87), „When I work on a new project, I will avoid mistakes I encountered in similar tasks before’, from the aspect, Creativity and Innovation, is the easiest item in the scale. This item was also expected to be an easy item. Table 8. The five easiest items, based on their item mean location Item mean location
Aspect
Item
Perspective
- 1.71
Creativity & innovation
- 0.74
Motivation to excel
33. When I work on a new project, I will avoid mistakes I encountered in similar tasks before 3. I strive towards excellence in challenging activities
- 0.73
Leadership
15. Once I have made a decision to solve a problem, I will sacrifice personal time to ensure that an answer to the problem is obtained
Ideal
- 0.65
Managing Opportunities and conflicts
37. I regularly review and make changes to my original plans when interesting opportunities arise to achieve
Ideal
- 0.62
Motivation to excel
13. I like my team members to cooperate to achieve, rather than compete against each other
Ideal Ideal
Ideal
Wong Heng Aik Jason and Russell Waugh
174
Five Hardest Items Table 9. The five hardest items, based on their item mean location Item threshold +0.92 +0.79
Aspect
Item
Perspective
Motivation to excel Managing opportunities & conflicts
2: I look for new challenges that test my abilities 44. There are often different methods of achieving a goal and I manage conflicts about methods within my team 24. I have a strong sense of working out the best methods to use to get a team to achieve 26. I spend time thinking about different ways my team can achieve a goal and solve problems 28. I look for alternative and innovative solutions and methods of doing things, solving problems or achieving goals
Real
+0.76
Leadership
+0.73
Leadership
+0.70
Creativity & innovation
Real Real Real Real
In contrast, but as expected, the five hardest items are from the Real perspective. The five items also span all the four aspects. There are two items from the aspect, Leadership, while the rest of the aspects have one item each among these five easiest items. The hardest item is Item 2 (+0.92 logits, chi square probability = 0.33), „I look for new challenges that test my abilities‟. However, this item was expected to be the hardest item; it was only expected to be slightly hard.
SUPPORT FOR THE EXPECTED MODEL OF THE VARIABLE This section compares the actual difficulty level of the items in the Student Entrepreneurial Mindset scale with the expected difficulty level. To assist in describing the difficulty level of the items, a rubric for the level of the item difficulties as used in this report is set up (See Table 10). The difficulty levels are also set out from Extremely Easy (where the item mean location, p < –0.75) through Easy/Hard (p = 0.00) and Extremely Hard (p > 0.75). Table 10. Descriptors for level of difficulty of items Item mean location (p)
Level of difficulty
p > 0.75
Extremely hard
0.50 < p < 0.75
Very hard
0.00 < p < 0.50
Hard
–0.50 < p < 0.00
Easy
–0.75 < p < –0.50
Very easy
p < –0.75
Extremely easy
A Rasch Measure of the Student Entrepreneurial Mindset in Singapore
175
Table 11. Pairs of items, where both the Ideal and Real perspective fit the model Item Motivation to Excel 1 & 2 – I look for new challenges that test my abilities 3 & 4 – I strive towards excellence in challenging activities 11 & 12 – I like to work with team members to achieve 13 & 14 – I like my team members to cooperate to achieve, rather than compete against each other Leadership 15 & 16 – Once I have made a decision to solve a problem, I will sacrifice personal time to ensure that an answer to the problem is obtained 25 & 26 – I spend time thinking about different ways my team can achieve a goal and solve problems Creativity & innovation 33 & 34 – When I work on a new project, I will avoid mistakes I encountered in similar tasks before (Ideal) Managing Opportunities and Conflicts 37 & 38 – I regularly review and make changes to my original plans when interesting opportunities arise to achieve
Ideal perspective
Real perspective
+0.19
Hard
+0.92
Extremely Hard
-0.74
Very Easy
-0.07
Easy
+0.01
Hard
+0.2
Hard
-0.62
Very Easy
-0.51
Very Easy
-0.73
Very Easy
+0.46
Hard
-0.13
Easy
+0.73
Very Hard
-1.71
Extremely easy
+0.00
Hard
-0.65
Very Easy
+0.35
Hard
The first part of this section investigates the first hypothesis of the first research question, “Students will find the Ideal perspective easier to respond to than the Real perspective. The level of difficulty for the Ideal perspective will be generally easier than the Real perspective.” There are eight pairs of items, i.e. 16 items, where both the Ideal and Real perspectives fitted the model (Table 11). In all these eight pairs of items, the students find it easier to respond to the Ideal perspective than to the Real perspective, as expected. For example, while both perspectives in the item, “I look for new challenges that test my abilities” are hard, Item 1 in the Ideal perspective measures +0.19 logits, while Item 2 in the Real perspective is harder, and measures +0.92 logits. This agrees with Hypothesis 1 of the first research question. The second part of this section investigates the actual level of difficulty of the 22 fitting items, and the expected level of difficulty for the items. Tables 12 to 15 show a comparison of the expected scores and the actual difficulty level of the item scores for all the 22 fitting items.
Motivation to Excel There are eight items that fit the measurement model for the aspect, Motivation to Excel. Table 12 shows the comparison of the actual difficulty level of the items with the expected difficulty level in the aspect. Of the eight items, four are from the sub–aspect, Achievement as a source of motivation (Items 1, 2, 3, and 4), and four are from sub–aspect, Sense of affiliation (Items 11, 12, 13 and 14). None of the items in the sub–aspect, Power as a source
176
Wong Heng Aik Jason and Russell Waugh
of motivation, fitted the model. As explained earlier, this could be because the items in this sub-aspect were the only four reversed scored items in the questionnaire. Table 12. Comparison of the actual difficulty level of the items with the expected difficulty level for the aspect, Motivation to Excel Ideal Perspective
Real Perspective
Expected difficulty
Actual difficulty
Expected difficulty
Items 1 & 2 – I look for new challenges that test my abilities
Easy
0.19 Hard
Hard
Items 3 & 4 – I strive towards excellence in challenging activities
Hard
-0.74 Very easy
Items 11 & 12 – I like to work with team members to achieve
Easy
0.01 Hard
Hard
0.20 Hard
Items 13 & 14 – I like my team members to cooperate to achieve, rather than compete against each other
Hard
-0.62 Very easy
Very hard
-0.51 Very easy
Very hard
Actual difficulty 0.92 Extremely Hard -0.07 Easy
In the sub–aspect, Achievement as a source of motivation, there is no agreement between the expected order of item difficulties and the actual order of item difficulties. Items 1 and 2, “I look for new challenges that test my abilities”, did not agree with the expected order of difficulty. Item 1 was expected to be easy but it turned out to be hard, while Item 2 was only expected to be hard but it turned out to be the hardest item in the measure. The level of difficulty was not expected to be too hard because students were given many opportunities to undertake new challenges in the schools. However, students‟ responses show that they find it hard to be looking out for new challenges. Items 3 and 4, “I strive towards excellence in challenging activities”, did not also agree with the expected order. Item 3 was expected to be hard, while Item 4 was expected to be very hard, but they turned out to be very easy and easy items. Students do not seem to find it too hard to strive towards excellence in challenging activities. In the sub–aspect, Sense of affiliation as a source of motivation, there was agreement for one of the items between the expected order and actual order of item difficulties. Item 12, “I like to work with team members to achieve” (Ideal perspective), was expected to be hard, and its measure is +0.20 logits (Hard). Item 11, “I like to work with team members to achieve” (Ideal Perspective), was expected to be an easy item too. Its measure is +0.01 logits, which is only very slightly hard; hence it can be considered to have agreed with the expected level of difficulty. Items 13 and 14, “I like my team members to cooperate to achieve, rather than compete against each other”, however, were expected to be hard and very hard but both items turned out to be very easy for the students. It seems that students are more ready to cooperate to achieve than expected.
A Rasch Measure of the Student Entrepreneurial Mindset in Singapore
177
Leadership There are seven items that fit the measurement model for the aspect, Leadership. Table 13 shows the comparison of the actual difficulty level of the items with the expected difficulty level in the aspect. Of the seven items, three are from the sub–aspect Transactional leadership (Items 15, 16 and 18), one is from the sub–aspect Transformational leadership (Item 22), and three are from the sub–aspect Instrumental leadership (Items 24, 25 and 26). There was agreement in two of the items between the expected and actual levels of difficulty. Item 15 was expected to be very easy and it turned out to be so (-0.73, Very easy) while Item 16 was expected to be hard and it turned out to be so too (+0.46, Hard). Another four items somewhat agree with the expected level of difficulties. Item 18, was expected to be Very hard, but it turned out to be only slightly hard (+0.09 logits). This was true also of Item 22 (+0.03, Hard), which was expected to be Very hard, Item 24 (+0.76, Extremely hard), which was expected to be Very hard, and Item 26 (+0.73, Very hard) which was expected to be Extremely hard. Only one of these seven items, Item 25, “I spend time thinking about different ways my team can achieve a goal and solve problems” (Ideal Perspective), did not agree with the expected level of difficulty. It was expected to be hard but its actual measure at −0.13 logits, is easier than expected. Students seem to think it would not be hard to get a team to work towards a goal or solve a problem. Table 13. Comparison of the actual difficulty level of the items with the expected difficulty level for the aspect, Leadership Ideal Perspective
Items 15 & 16 – Once I have made a decision to solve a problem, I will sacrifice personal time to ensure that an answer to the problem is obtained Item 18 – When I have made a wrong decision on a problem solving task, I try some other method to obtain an answer Item 22 – I attribute our successes to my team members when we have achieved a goal Item 24 – I have a strong sense of working out the best methods to use to get a team to achieve Items 25 & 26 – I spend time thinking about different ways my team can achieve a goal and solve problems
Real Perspective
Expected difficulty
Actual difficulty
Expected difficulty
Actual difficult y
Very easy
-0.73 Very easy
Hard
+0.46 Hard
Extremely hard
+0.09 Hard
Very hard
+0.03 Hard
Very hard
Hard
-0.13 Easy
Extremely hard
+0.76 Extreme ly Hard +0.73 Very Hard
Creativity and Innovation There are three items that fit the measurement model in the aspect, Creativity and Innovation. Table 14 shows the comparison of the actual difficulty level of the items with the expected difficulty level in the aspect. Of the three items, one is from the sub–aspect,
178
Wong Heng Aik Jason and Russell Waugh
Innovative and adaptable spirit, Item 28, and two are from the sub–aspect Managing failures, Items 33 and 34. None of the items agree with the expected levels of difficulty. Item 28, “I look for alternative and innovative solutions and methods of doing things, solving problems or achieving goals” (Real Perspective), was expected to be easy but it turned out to be Very hard. Students were apparently not as innovative as they were expected to be. Item 33, “When I work on a new project, I will avoid mistakes I encountered in similar tasks before” (Ideal Perspective), was expected to be only easy; it turned out to be the easiest item in the measure (-1.71, Extremely easy). Students must have found it easiest to believe they could avoid mistakes that they have done before. On the one hand, there seems to be no agreement between the expected and the actual measure. On the other hand, this aspect Creativity and Innovation was expected to be the easiest. Although, three of the items, Items 27 (Extremely easy), Items 29 (Very easy) and Items 31 (very easy), were expected to make it the easiest, they were non-fitting. However, this aspect can still be considered as the easiest of the four aspects since it included the easiest item in the measure (Item 33, −1.71 logits, Extremely easy), and the actual measures of the other two good-fitting items (both in the Real Perspective) were not extremely hard. These were Item 34 (0.00 logits, neither easy nor hard) and Item 28 (+0.70, Very hard). Table 14. Comparison of the actual difficulty level of the items with the expected difficulty level for the aspect, Creativity and innovation Ideal Perspective Expected Actual difficulty difficulty Item 28 – I look for alternative and innovative solutions and methods of doing things, solving problems or achieving goals Items 33 & 34 – When I work on a new project, I will avoid mistakes I encountered in similar tasks before
Easy
-1.71 Extremely easy
Real Perspective Expected Actual difficulty difficulty Easy
+0.70 Very hard
Very hard
0.00 Easy / Hard
Managing Opportunities and Conflicts There are four items that fit the measurement model for Managing opportunities and conflicts. Table 15 shows the comparison of the actual difficulty level of the items with the expected difficulty level in the aspect. Of the four items, Items 37 and 38 are from the sub– aspect, Pursuit of opportunities, and Items 41 and 44 are from the sub–aspect, Managing conflicts,. All four items agree with the expected order of difficulty.
A Rasch Measure of the Student Entrepreneurial Mindset in Singapore
179
Table 15. Comparison of the actual difficulty level of the items with the expected difficulty level for the aspect, Managing Opportunities and Conflicts Ideal Perspective Expected Actual difficulty difficulty Items 37 & 38 – I regularly review and make changes to my original plans when interesting opportunities arise to achieve Item 41 – I let my team members know the kinds of conflicts involved in achieving a goal or solving a problem Item 44 – There are often different methods of achieving a goal and I manage conflicts about methods within my team
Very easy
-0.65 Very easy
Easy
-0.08 Easy
Real Perspective Expected Actual difficulty difficulty Hard
+0.35 Hard
Extremely hard
+0.79 Extremely hard
SUMMARY A linear scale measuring Student Entrepreneurial Mindset was created with 22 items (from an original set of 44 items) using the Rasch Unidimensional Measurement Model (RUMM 2020) program (Andrich, Sheridan & Luo, 2005). This programme proved useful in identifying items that did not fit the measurement model and thus could be deleted because they produced too much „noise‟ in the measure. The 22 items that fitted the measurement model define the variable Student Entrepreneurial Mindset which involves Motivation to Excel, Leadership, Creativity and Innovation, and Managing Opportunities and Conflicts. In the creation of the 22 item Student Entrepreneurial Mindset Scale, there was: 1. A very good overall fit to the measurement model (item-trait interaction chi-square = 130.5, df = 154, p = 0.92); 2. Good global person fit to the measurement model; 3. Good global item fit to the measurement; 4. Good individual item fit (all 22 items fitted with p > 0.13); 5. Good targeting of the student measures against the item difficulties, although some improvement could be made by adding some harder items; 6. Good discrimination as shown by the Item Characteristics Curves; 7. Consistent and logical use of the response categories for each item; and 8. Good separation of measures compared to errors of measurement (the Person Separation Index was 0.80). This means that the data from the 22 good-fitting items formed a reliable linear scale from which valid inferences can be drawn. The main valid inferences that can be drawn are listed: 1. The ideal items are easier than the real items (where both fit the measurement model), as expected.
180
Wong Heng Aik Jason and Russell Waugh 2. The easiest item for Creativity and Innovation is Item 33 „When I work on a new project, I will avoid mistakes I encountered in similar tasks before’ (difficulty = −1.71 logits, extremely easy). 3. The easiest item for Motivation to Excel is Item 3 „I strive towards excellence in challenging activities’ (difficulty = −0.74 logits, very easy). 4. The easiest item for Leadership is Item 15 „Once I have made a decision to solve a problem, I will sacrifice personal time to ensure that an answer to the problem is obtained’ (difficulty = −0.73 logits, very easy). 5. The easiest item for Managing Opportunities and Conflicts is Item 37 „I regularly review and make changes to my original plans when interesting opportunities arise to achieve’ (difficulty = −0.65 logits, very easy). 6. The hardest item for Motivation to Excel is Item 2 „I look for new challenges that test my abilities’ (difficulty = +0.92 logits, very hard). 7. The hardest item for Managing Opportunities and Conflicts is Item 44 „There are often different methods of achieving a goal and I manage conflicts about methods within my group’ (difficulty = +0.79 logits, very hard). 8. The hardest item for Leadership is Item 24 „I have a strong sense of working out the best methods to use to get a team to achieve’ (difficulty = +0.76 logits, very hard). 9. The hardest item for Creativity and Innovation is Item 28 „I look for and innovative solutions and methods of doing things, solving problems or achieving goals. (difficulty = +0.70 logits, very hard)
In: Specialized Rasch Measures… Editor: Russell F. Waugh, pp. 181-203
ISBN: 978-1-61668-032-9 © 2010 Nova Science Publishers, Inc.
Chapter 9
A RASCH MEASURE LINKING SELF-REPORTED STUDENT ATTITUDE AND BEHAVIOR TO MATHEMATICS Radha Devi Unnithan and Russell Waugh Graduate School of Education University of Western Australia.
ABSTRACT Mathematics education is important for students in Singapore and it is important to understand their attitude and behaviour towards Mathematics. A questionnaire was designed using the latest techniques applicable to Rasch measurement with a sample of N=452 students in secondary grades 2, 3 and 4. Eight stem-items on goal setting, tasks and effort were answered in two perspectives (their attitude, I Aim to Do This; and their behavior, My Actual Behaviour Is), making an effective item sample of 16. The item-trait interaction chi-square was χ2 = 111, df=96, p=0.14 showing that a uni-dimensional trait was measured and 15 out of 16 items fitted the measurement model with a probability of p>0.08. The Student Separation Index and the Cronbach Alpha was 0.91 meaning that the scale was reliable. Targeting was good and valid inferences could be drawn from the scale in terms of identifying those with poor attitudes and behaviour, and in terms of which items are hard and which are easy.
STRUCTURE OF THE EDUCATION SYSTEM IN SINGAPORE In Singapore, each child is entitled to be enrolled into a primary school at the age of seven (Ministry of Education, 2004). There are two stages in the primary school education. In the first stage students go through a four year foundation stage from primary one (seven years of age) to four (ten years of age) and a two-year orientation stage from primary five (eleven years of age) to six (twelve years of age). All students at the foundation stage, study English Language, Mother Tongue and Mathematics. This stage is meant to build a firm foundation in
182
Radha Devi Unnithan and Russell Waugh
these subjects. Mother Tongue refers to subjects such as Chinese, Malay or Tamil. Students are able to learn their Mother Tongue based on their own religion or preference. Other subjects such as Art and Craft, Civics and Moral Education, Music, Social Studies and Physical Education are also included in the curriculum. Students begin to study Science from Primary Three onwards and Project Work is conducted during curriculum time but is not assessed formally as an examination subject (Ministry of Education, 2004). In order to maximize their potential, students are streamed according to their learning ability at the end of Primary four (10 years of age). All students then move on to the next stage which is known as the orientation stage. In this stage, students are channeled to one of the three language streams, namely EM1, EM2 and EM3, according to their abilities. The EM1 stream is meant for students who perform very well in English Language, Mother Tongue and Mathematics at the end of Primary four. These students learn English, Mathematics, Science and higher mother tongue in Primary five (11 years of age) and Primary six (12 years of age). The majority of the students will qualify for the EM2 stream. In this stream, students learn English Language, Mother Tongue, Mathematics and Science. The EM3 stream is meant for students who are unable to cope with the languages and Mathematics. In this stream, students will learn Foundation English, Basic Mother Tongue and foundation Mathematics. They will not be examined in Science but will be examined in Foundation English, Basic Mother Tongue and foundation Mathematics. However, students who had obtained good grades in their Mother Tongue in Primary four will be allowed to offer Mother Tongue at the standard level. From the end of 2004, the two streams EM1 and EM2 have been merged. Schools have the authority to decide to offer their students a higher standard of mother tongue. At the end of Primary six, all students sit for the Primary School Leaving Examination (PSLE) which assesses their abilities. These students are then placed in a secondary school course that suits their learning pace and aptitude. Students are placed in one of the following three streams in a secondary school. These streams are as follows: Special, Express and Normal. Subject content in these streams are designed to suit the learning abilities and interests of the students. The secondary education consists of four to five years with different curricular emphases. Most of the students qualify for the Special or Express stream while the rest enter the Normal course (Ministry of Education, 2004). Special and Express streams involve a four year programme in which the students sit for the Singapore-Cambridge General Certificate of Examination „Ordinary‟ (GCE „O‟) level examinations at the end of the fourth year. A common curriculum is designed for all students in Secondary One and Two. The subjects taught include English Language, Mother Tongue, Mathematics, Science, History, Geography, Literature, Design & Technology, Home Economics, Visual Arts, Civics and Moral Education, Music and Physical Education. Students are then streamed according to their interest and ability in secondary three. These students study seven to eight subjects for the GCE „O‟ Level examinations. Those who possess outstanding academic ability may study a ninth subject. Some of the subjects which are taught in Secondary Three and Secondary Four are as follows: English Language, Mother Tongue, Elementary Mathematics, Additional Mathematics, Physics, Chemistry, Biology, History, Geography, Literature, Art and Design, Design and Technology, Food and Nutrition, Music and Principles of Accounts (Ministry of Education, 2004). In the Normal course, students study either the Normal (Academic) course or the Normal (Technical) course. Students take four years to finish either of these courses. At the end of the
A Rasch Measure Linking Self-Reported Student Attitude…
183
fourth year, they sit for the Singapore-Cambridge General Certificate of Examination „Normal‟ (GCE „N‟) level examinations. Students in the Normal (Academic) course, study six to eight subjects while those in the Normal (Technical) course, study five to seven subjects. Those who are in the Normal (Academic) course study subjects which challenge the students academically while those in the Normal (Technical) will follow a curriculum with a technicalvocational bent (Ministry of Education, 2001). Examples of subjects which are studied in the Normal (Academic) course are: English language, Mother tongue, Mathematics, Science, History, Geography, Literature, Design and Technology and Home Economics. Examples of subjects which are studied in the Normal (Technical) course are English language, Mother tongue, Mathematics, Computer Application and Elements of Office Administration. Those who perform well in the GCE „N‟ level examination will be allowed to move on to sit for the GCE „O‟ level in the following year (Ministry of Education, 2004). Upon completion of their GCE „O‟ level examination (usually at 16 or 17 years old), students may then study in a junior college or centralized institute. The junior college offers a two year course while the centralized institute offers a three year course. At the end of their course, students then sit for the Singapore-Cambridge General Certificate of Examination „Advanced‟ (GCE „A‟) level examinations. Students have to study two compulsory subjects: General paper and Mother Tongue. They are allowed to study a maximum of four GCE „A‟ level subjects. Examples of some of the „A‟ level subjects which are offered are English Language, Mathematics, Further Mathematics, Physics, Chemistry, Biology and Economics, History, Geography and Literature (Ministry of Education, 2001). Some of the students may proceed to the polytechnics after completing their O levels. These students may then proceed to a University to obtain a degree (Ministry of Education, 2004). Mathematics is required for the development and improvement of a student‟s intellectual competence in logical reasoning, spatial visualization, analysis and abstract thought. Students develop numeracy, reasoning, thinking skills and problem solving skills through the learning and application of Mathematics. These are valued not only in science and technology, but also in everyday living and in the workplace. A Society which is highly skilled scientifically and technologically requires a workforce which has a strong foundation in Mathematics. „An emphasis on mathematics education will ensure that we have an increasingly competitive workforce to meet the challenges of the 21st century.‟ (Secondary Mathematics Syllabus, p.1, 2006)
RESEARCH QUESTIONS 1. What are students attitudes and behaviour to Mathematics? 2. Can a linear, uni-dimensional scale of student Attitude and Behavior to Mathematics be created? 3. What attitude and behaviour aspects are considered easy and what are considered hard?
184
Radha Devi Unnithan and Russell Waugh
RASCH ANALYSIS This chapter presents the questionnaire data analysis for Attitude and Behaviour to Mathematics using the Rasch Unidimensional Measurement Model (RUMM 2020) program (Andrich, Sheridan and Luo., 2005) with data from N = 452 secondary school students at one „neighbourhood‟ school in Singapore. The results are presented through the use of tables, figures and text. The questionnaire consisted of eight items, each answered in two perspectives (I aim to do this item and My actual behaviour on this item is, giving 8 x 2 (16) items. Data were analysed with the RUMM 2020 computer program (Andrich et al., 2005). First, the data were checked to see whether the response categories were answered consistently and logically. The RUMM 2020 program assesses this with two outputs, namely, response category curves and thresholds. Response category curves show the probability of answering each response category by the Attitude and Behaviour to Mathematics measure. Thresholds are points between adjacent response categories where the odds are 1:1 of answering in either category. For good measurement, thresholds should be ordered in line with the ordering of the response categories. The RUMM 2020 program produces outputs to assess fit to the measurement model, reliability and dimensionality. These are now explained.
Global Item and Person Fit Table 1 shows the global item and global person fit. The fit residuals for both the item difficulties and the person measures are the differences between the actual values and the expected values, calculated according to the measurement model. When they are standardised, they have an approximately normal distribution (mean = 0, SD =1), if the data fit the measurement model. There is no „law‟ on this, but there are 452 times 16 (7232) fit residuals, for example, and statistically one might expect the mean to be zero and the SD to be 1, if there are no problems with fit but it does not have to be so. The fit residual data are approximately normal.
Individual Item Fit Table 2 shows the individual item fit to the measurement model. Deleting the not-sogood fitting item (item 6) and re-analysing the data did not produce a better fit to the measurement model and so the 16 items were retained. Fifteen items out of sixteen have an acceptable fit to the measurement model with a probability greater than 0.05, indicating that there is an excellent individual item fit to the measurement model.
A Rasch Measure Linking Self-Reported Student Attitude…
185
Table 1. Global Item and Person Fit to the Measurement Model ITEM-PERSON INTERACTION ITEMS PERSONS Location Fit Residual Location Fit Residual Mean 0.00 SD 0.64
0.30 1.83
0.54 1.26
-0.48 1.75
Notes on Table 1. 1. Item location is item difficulty in logits. 2. Person location is person measure in logits. 3. SD is standard deviation. 4. The mean item difficulty is constrained to zero by the RUMM 2020 program. 5. Fit residuals are the difference between the actual values and the expected values calculatedaccording to the measurement model (standardised). They have a mean near zero and an SD near 1 when the data fit the measurement model (an acceptable fit for these data). 6. All values are given to two decimal places because the errors are to two decimal places.
Consistency of Answering Category Responses Table 2. Item Fit to the Measurement Model (Attitude and Behaviour to Mathematics) Item No Item 01 Item 02 Item 03 Item 04 Item 05 Item 06 Item 07 Item 08 Item 09 Item 10 Item 11 Item 12 Item 13 Item 14 Item 15 Item 16
Location 0.49 0.27 0.25 +0.25 +0.48 1.44 0.46 0.69 +0.42 +0.37 +0.57 +0.77 +1.06 0.54 +0.25 0.01
SE 0.07 0.07 0.07 0.07 0.06 0.08 0.07 0.07 0.07 0.07 0.07 0.07 0.07 0.07 0.07 0.07
Residual 1.16 +1.11 +0.73 +2.61 +2.95 2.71 0.50 1.95 0.06 0.38 0.33 +2.67 +3.22 0.30 +0.72 1.78
df 417.06 417.06 417.06 417.06 417.06 417.06 417.06 417.06 417.06 417.06 417.06 417.06 417.06 417.06 417.06 417.06
Chi-square 7.76 4.47 6.61 7.56 3.54 20.08 3.98 10.90 4.27 9.01 5.63 7.31 3.76 3.90 5.93 6.31
Probability 0.26 0.61 0.36 0.27 0.74 0.00 0.68 0.09 0.64 0.17 0.47 0.29 0.71 0.69 0.43 0.39
Notes on Table 2. 1. Location is item difficulty in logits. 2. SE is Standard Error. 3. Residual is the difference between actual value and expected value, calculated according to the measurement model. 4. df is degrees of freedom. 5. 15 out of 16 items fit the measurement model with a probability greater than 0.05. 6. All values are given to two decimal places because the errors are to two decimal places.
Radha Devi Unnithan and Russell Waugh
186
The thresholds between category responses are given in Table 3. Thresholds represent the change in probability of answering two adjacent response categories. At a threshold, the odds of answering adjacent response categories are 1:1. For good measurement, the thresholds ought to be ordered in line with the conceptual ordering of the response categories. The thresholds in the present study are ordered in line with the conceptual ordering from low to high (none of the time or almost none of the time, some of the time, most of the time and all the time or nearly all the time). This indicates that the students answered the four response categories consistently and logically. Table 3. Item Thresholds for Student Measure Mean
THRESHOLDS 1
2
3
Item 01
0.49
2.38
0.62
+1.52
Item 02
0.27
1.98
0.44
+1.62
Item 03
0.25
2.12
0.11
+1.48
Item 04
+0.25
1.44
+0.37
+1.80
Item 05
+0.48
0.82
+0.50
+1.75
Item 06
+0.44
3.84
1.25
+0.77
Item 07
0.46
2.24
0.46
+1.33
Item 08
0.69
2.20
0.58
+0.70
Item 09
+0.42
2.05
+0.84
+2.47
Item 10
+0.37
1.98
+0.63
+2.44
Item 11
+0.57
1.348
+0.48
+2.57
Item 12
+0.77
1.12
+1.06
+2.35
Item 13
+0.06
0.61
+1.32
+2.46
Item 14
0.54
2.70
0.23
+1.31
Item 15
+0.25
1.78
+0.60
+1.93
Item 16
0.01
1.87
+0.22
+1.63
Notes on Table 3. 1. Thresholds are points between adjacent response categories where the odds are 1:1 of answering the adjacent categories. With four response categories, there are three thresholds. 2. Mean thresholds are the item difficulties in logits. 3. All values are given to two decimal places because the errors are to two decimal places. 4. The thresholds for each item are ordered in line with the ordering of the response categories.
Category response curves for each item show the relationship between the probability of answering each category in relation to the Attitude and Behaviour to Mathematics measure. An example is given in Figure 8.1. This figure shows that when the measure is low, then the probability is high that the chosen student category response is low (not at all, score zero). As the measure increases, the probability of answering in the lowest category decreases and the probability of answering in the next category (some of the time) increases. As the measure increases further still, the probability of answering category two (some of the time, score 1) decreases and the probability of answering category three (most of the time, score 2)
A Rasch Measure Linking Self-Reported Student Attitude…
187
increases. As the measure increases further still, the probability of answering category three (most of the time, score 2) decreases, and the probability of answering category four (all the time or nearly all the time, score 3) increases. This means that the students have answered the four response categories logically and consistently. The response category curves for all 16 items were good.
Note on Figure 1 1. This response category curves is satisfactory and shows that the students used the response categories consistently and logically. Figure 1. Response Category Curve for Item 1.
Item Characteristic Curves Item characteristic curve show the relationship between the expected response score and the Attitude and Behaviour to Mathematics measure. An example is given in Figure 2 for item 1. It shows how the item discriminates for groups of persons near the item difficulty. In this case, the item is functioning as intended. The item characteristic curves for all 16 items showed that the items were functioning as intended.
Dimensionality An item-trait interaction chi-square determines whether a unidimensional trait has been measured. This examines the consistency with which students with measures all along the scale agree with the calculated difficulties of the items along the scale. That is, it provides a check that all the students agree that particular items are easy, of medium difficulty or hard. The total item chi-square was not significant (χ2 = 110.996, df = 96, p = 0.14). This indicates that there was no significant interaction of person measures with item difficulties along the scale, indicating good agreement by the students about which items were easy and which were hard. It can be concluded that a unidimensional trait was measured which could be called Attitude and Behavior to Mathematics.
188
Radha Devi Unnithan and Russell Waugh
Figure 2. Characteristic Curve for Item 1.
Person Separation Index The Person Separation Index is 0.91 indicating that the measures are well separated along the scale in comparison to their errors of measurement. This also implies that the power of the tests-of-fit are strong.
Targeting A Person Measure/Item Difficulty graph (see Figure 3) which shows the item difficulties from easy on the left (-1.7 logits) to hard on the right (+1.2 logits). The range of item difficulties do not cover the range of student measures and some easy and hard items need to be added in any future use of the scale.
Notes on Figure 3 1. Person measures are given on the upper side in logits. 2. Item difficulties are given on the lower side in logits. Figure 3. Student Measures / Item Difficulties on a Linear Scale.
A Rasch Measure Linking Self-Reported Student Attitude…
189
A Person Measure/Item Threshold graph (see Figure 4) which shows the item thresholds instead of item difficulties. The thresholds range from easy (about -4.1 logits) to hard (about +2.5 logits) and thus better cover the range of student attitude and behaviour measures. Nevertheless, in any future use of the scale, some harder items need to be added to better measure those students with high attitude and behaviour to mathematics.
Notes on Figure 4 1. Person measures are given on the upper side in logits. 2. Item threshold are given on the lower side in logits. Figure 4. Person Measure/Item Threshold Graph.
INFERENCES FROM THE LINEAR SCALE The Rasch analysis has calibrated the student measures on the same scale as the item difficulties and produced a linear, unidimensional scale (see Figure 3), for which the data have a good fit to the measurement model. Since it has now been shown that the scale data are reliable (there is good individual fit and acceptable global fit to the measurement model, the separation of measures is good in comparison to the errors and the students have answered the response categories consistently and logically), valid inferences can be made from the scale.
Relationships between Attitude and Behavior Attitudes are easier than their corresponding behaviors, as conceptualized, for all items in the scale.
Radha Devi Unnithan and Russell Waugh
190
Table 4. Item Wording and their Difficulties for Attitude and Behaviour to Mathematics Item No.
1
2
3
4 5
6
7 8
Item Goal Setting I set realistic, but challenging goals for myself in mathematics. When I have difficulties in reaching my goals in mathematics, I renew my efforts to achieve those goals. When I have difficulties in reaching my goals in mathematics, I renew my efforts by trying different strategies to reach those goals. Tasks I solve some mathematics problems which my peers think are difficult. I solve some mathematics problems which my teacher thinks are difficult for me. Effort When I am given a mathematics assignment (problem), I make a strong effort to obtain the correct answer. When I am given a mathematics assignment (problem), I make a strong effort to obtain the correct answer as quickly as possible. I make strong demands on myself to achieve at a high level in mathematics.
I aim to do this item
My actual behavior on this item is
0.49
+0.42
0.27
+0.37
0.25
+0.57
+0.25
+0.77
+0.48
+1.06
1.44
0.54
0.46
+0.25
0.69
0.01
Notes on Table 4. 1. Item difficulties are in logits. 2. The item difficulties for attitude (what I aim to do) are easier than they are for their corresponding behaviours for all 16 items.
Inferences about Attitude The four easiest attitude items (what students aim for) are: 1. When I am given a mathematics assignment (problem), I make a strong effort to obtain the correct answer (item 6, difficulty 1.44 logits) (very easy); 2. I make strong demands on myself to achieve at a high level in mathematics (item 8, difficulty 0.69 logits); 3. I set realistic, but challenging goals for myself in mathematics (item 1, difficulty 0.49 logits); 4. When I am given a mathematics assignment (problem), I make a strong effort to obtain the correct answer as quickly as possible (item 7, difficulty 0.46) (moderately easy).
A Rasch Measure Linking Self-Reported Student Attitude…
191
The four hardest attitude items (what students aim for) are: 1. I solve some mathematics problems which my teacher thinks are difficult for me (item 5, difficulty +0.48 logits) (hard); 2. I solve some mathematics problems which my peers think are difficult (item 4, difficulty +0.25 logits); 3. When I have difficulties in reaching my goals in mathematics, I renew my efforts by trying different strategies to reach those goals (item 3, difficulty 0.25 logits); 4. When I have difficulties in reaching my goals in mathematics, I renew my efforts to achieve those goals (item 2, difficulty 0.27 logits) (moderately hard).
Inferences about Behavior are:
The four easiest behavior items (what students reported that their actual behaviour was) 1. When I am given a mathematics assignment (problem), I make a strong effort to obtain the correct answer (item 6, difficulty 0.54 logits) (very easy) ; 2. I make strong demands on myself to achieve at a high level in mathematics (item 8, difficulty 0.01 logits); 3. When I am given a mathematics assignment (problem), I make a strong effort to obtain the correct answer as quickly as possible (item 7, difficulty +0.25 logits); 4. When I have difficulties in reaching my goals in mathematics, I renew my efforts to achieve those goals (item 2, difficulty +0.37 logits) (moderately easy).
The four hardest behavior items (what students reported that their actual behaviour was) are, and these are hard: 1. I solve some mathematics problems which my teacher thinks are difficult for me (item 5, difficulty +1.06 logits) (hard); 2. I solve some mathematics problems which my peers think are difficult (item 4, difficulty +0.77 logits); 3. When I have difficulties in reaching my goals in mathematics, I renew my efforts by trying different strategies to reach those goals (item 3, difficulty +0.57 logits); 4. I set realistic, but challenging goals for myself in mathematics (item 1, difficulty +0.42 logits) (moderately hard);
Inferences about Weakest Students The lowest measures indicate that the students have a very negative attitude towards mathematics. Of the total 452 students, 20 had very low attitude and behavior towards mathematics indicating that these students need to be guided and counselled. They could,
Radha Devi Unnithan and Russell Waugh
192
perhaps, be motivated so that they become interested in mathematics and this might, in turn, create a positive attitude and behaviour in these students. Table 5. Students with Lowest Attitude and Behavior Measure (N=20) ID
Gender
Grade
Attitude and behavior
SE
Residual
283
M
3
5.40
1.25
164
M
3
3.15
0.56
+1.21
279
F
3
2.87
0.52
0.54
188
M
3
2.63
0.49
+0.58
228
M
3
2.63
0.49
+4.14
162
M
3
2.63
0.49
+5.08
249
M
3
2.41
0.46
+0.38
234
F
3
2.21
0.44
0.55
144
F
3
2.03
0.43
+2.46
418
M
4
1.85
0.42
0.81
419
M
4
1.69
0.40
0.65
416
F
4
1.69
0.40
1.36
440
M
4
1.53
0.40
+1.69
139
F
3
1.53
0.40
+1.03
065
F
2
1.53
0.40
0.59
427
M
4
1.53
0.40
0.70
268
M
3
1.53
0.40
1.35
422
M
4
1.53
0.40
+0.47
373
M
4
1.53
0.40
3.31
173
F
3
1.53
0.40
2.48
Notes on Table 5 1. ID is student identification number. 2. Grade is the level at which the student is studying. 3. Attitude and behavior measure is in logits (minimum linear measure is 5.40 logits, maximum is +5.06). 4. SE is standard error in logits. 5. Residual is the standardised difference between the actual score and the score estimated according to the measurement model. 6. All values are given to two decimal places because the errors are to two decimal places.
A Rasch Measure Linking Self-Reported Student Attitude…
193
Inferences about Best Students The highest measures indicate that the students have a very positive attitude towards mathematics. Of the total 452 students, 20 had very high attitude and behaviour towards mathematics indicating that these students are self motivated and hence they do well in the subject. Table 6. Students with Highest Attitude and Behavior Measure (N=20) ID
Gender
Grade
Attitude and behavior
SE
Residual
117
M
2
2.54
0.46
+1.16
217
M
3
2.76
0.49
+0.58
168
F
3
2.76
0.49
0.49
142
F
3
2.76
0.49
0.94
349
F
4
2.76
0.49
+0.26
302
F
4
3.01
0.53
0.69
242
M
3
3.01
0.53
0.26
365
M
4
3.31
0.59
+0.69
309
M
4
3.31
0.59
1.20
202
F
3
3.31
0.59
+0.54
315
M
4
3.31
0.59
+0.21
410
M
4
3.31
0.59
0.21
130
M
3
3.69
0.68
0.89
267
M
3
4.25
0.86
0.88
299
F
4
4.25
0.86
0.82
001
F
2
4.25
0.86
0.88
295
F
4
4.25
0.86
0.82
077
M
2
5.06
1.23
080
M
2
5.06
1.23
010
F
2
5.06
1.23
Notes on Table 6 1. ID is student identification number. 2. Grade is the level at which the student is studying. 3. Attitude and behaviour measure is in logits (minimum linear measure is 5.40 logits, maximum is +5.06). 4. SE is standard error in logits. 5. Residual is the standardised difference between the actual score and the score estimated according to the measurement model. 6. All values are given to two decimal places because the errors are to two decimal places. 7. The Rasch measures for the top three students‟ questionnaires are estimated.
194
Radha Devi Unnithan and Russell Waugh
Inferences in Relation to Gender
Notes on Figure 5 1. Student measures are on the top of the scale from low (LHS) to high (RHS) 2. Item difficulties are on the bottom of the scale from easy (LHS) to hard (RHS) Figure 5. Student Measures by Gender.
Figure 5 shows the scale of item difficulties from easy to hard and the student measures by gender calibrated on the same scale from low (about −5.40 logits) to high (about +5.06 logits). Out of the 452 students who participated in the study, 248 were male and 204 were female. The mean measure for the female students is +0.55 logits and is slightly higher than the mean measure for the males which is +0.53 logits. However, this difference is not significant at 0.01 ( t = 0.24, df = 450, p = 0.41 ). That is, the difference in mean measures of attitude and behaviour could be explained by chance. A few harder items could have been added to cater for those students with a higher range of student measures.
Inferences in Relation to Grades Figure 6 shows the scale of item difficulties from easy to hard and the student measures by grades calibrated on the same scale from low (about −5.40 logits) to high (about +5.06 logits). 119 students were from grade two, 167 from grade three and 166 from grade four. The mean measure for grade two was +0.63 logits, grade three was +0.24 logits and grade four was +0.78 logits. The mean measure for grade 4 is significantly higher than for grade 3 (t = 3.98, df = 331, p = 0.000) but the mean measurement for grade 4 is not significantly higher than for grade 2 (t = 1.01, df = 283, p = 0.14) The average aggregate scores obtained by the students from grades two, three and four for their Primary School Leaving Examination are 210, 200 and 212 respectively. This shows that students from grade four were the best, followed by those in grade two and grade three and this is line with the mean Rasch measures for attitude and behaviour. The aggregate score
A Rasch Measure Linking Self-Reported Student Attitude…
195
determines if the students qualify to study in a particular school and in which stream. The higher the aggregate score the better it is. However, since this school is a „neighbourhood‟ school, most of the students would have obtained a low aggregate as compared to the better schools. The grade three students had the lowest mean measure. The researcher feels that this group of students are very weak in mathematics and they are not motivated to do well in their examinations. This explains why the mean measures for all the grades were significantly different and also why grade three was the worst.
Notes on Figure 6 1. Student measures are on the top of the scale from low (LHS) to high (RHS) 2. Item difficulties are on the bottom of the scale from easy (LHS) to hard (RHS) Figure 6. Student Measures by Grade.
SUMMARY A Rasch measurement analysis was conducted with eight stem-items, conceptually ordered from easy to hard, and answered in two perspectives („I aim to do this item‟ and „My actual behaviour on this item is‟) giving an effective scale of 16 items. The RUMM 2020 computer program (Andrich et al., 2005) was particularly helpful in conducting this analysis. It was found that one item did not fit the measurement model. However, deleting the not-sogood fitting item and re-analysing the data did not produce a better fit to the measurement model and so the 16 items were retained. Data from the 16 items were analysed and it was concluded that a reliable linear, uni-dimensional scale of Attitude and Behavior to Mathematics was created in which the measures were calibrated on the same scale as the item difficulties. The reliability of the scale data was shown by: 1. Acceptable global and person item fit to the measurement model; 2. Good individual fit to the measurement model; 3. The four category responses being answered in a consistent and logical way;
196
Radha Devi Unnithan and Russell Waugh 4. A good Person Separation Index indicating that the person measures were well separated in comparison to the errors; 5. A good item-trait interaction indicating the measurement of a uni-dimensional trait called Attitude and Behavir to Mathematics; 6. Reasonable targeting of the items against the person measures, although some harder items need to be added for any future use of the scale.
Since the scale data were shown to be reliable, the following valid inferences were drawn from the scale. 1. For all 16 items, it was easier to aim high than it was to achieve highly. 2. The students found it very easy to aim to make a strong effort to obtain the correct answer when a mathematics problem was given. 3. The students found it moderately easy to aim to make a strong effort to obtain the correct answer as quickly as possible when they are given a mathematics assignment (problem). 4. The students found it the hardest to aim to solve some mathematics problems which their teacher thinks are difficult for them. 5. The students found it moderately hard to aim to renew their efforts to achieve their goals in mathematics. 6. The students found it very easy to actually make a strong effort to obtain the correct answer when they are given a mathematics problem. 7. The students found it moderately easy to actually renew their efforts to achieve their goals in mathematics when they face difficulties. 8. The students found it hard to actually solve some problems which their teacher thinks are difficult for them. 9. The students found it moderately hard to actually set realistic but challenging goals for themselves in mathematics. Students who have very low attitude and behavior towards mathematics need to be guided and counselled. They have to be motivated so that they become interested in mathematics and this would in turn create a positive attitude and behaviour in these students. On the other hand, students who have a very high attitude and behaviour to mathematics are self motivated and hence they do well in the subject. There was no significant difference in the attitude and behavior of male and female students towards mathematics. Grade four students were found to have a significant better Attitude and Behavior towards Mathematics compared to students in grades 3 and 2
REFERENCES Andrich, D. (1985). A latent trait model for items with response dependencies: Implications for test construction and analysis. In S. E. Embreston (Ed.), Test design: Developments in psychology and psychometrics (pp. 245-275). Orlando: Academic Press.
A Rasch Measure Linking Self-Reported Student Attitude…
197
Andrich, D. (1988a). A general form of Rasch‟s Extended Logistic Model for Partial Credit Scoring. Applied Measurement in Education, 1(4), 363-378. Andrich, D. (1988b). Rasch models for measurement. Sage university paper on quantitative applications in the social sciences, series number 07/068. Newbury Park, California: Sage Publications. Andrich, D., & van Schoubroeck, L. (1989). The general health questionnaire: A psychometric analysis using latent trait theory. Psychological Medicine, 19, 469-485. Andrich, D., Sheridan, B., and Luo, G. (2003). Rasch unidimensional measurement models: A windows based item analysis program employing Rasch models (RUMM 2020). Perth, Australia: RUMM Laboratory. Andrich, D., Sheridan, B., & Luo, G. (2005). RUMM : A windows-based item analysis program employing Rasch unidimensional measurement models. Perth: RUMM Laboratory. Anenz-Ludlow, Adalira & Walgamuth, C. (1998). Third graders‟ interpretations of equality and the equal symbol. Educational Studies in Mathematics. 35, 153-187. Ashcraft, M.H (1992). Cognitive arithmetic: A review of data and theory. Cognition, 44, 75106. Ashcraft, M.H., Donley, R.D., Halas, M.A., & Vakali, M. (1992). Working memory, automaticity and problem difficulty. In J.I.D. Campbell (Ed.). The Nature and Origins of Mathematical Skills (pp. 301-329), Amsterdam: Elsevier. Ashlock, R.B. (1986). Error patterns in computation: A semi-programmed approach, (4th edition) Columbus, Ohio: Charles E. Merrill Publishing Company. Ayres, P. (2007). Systematic mathematical errors and cognitive load. Contemporary Education Psychology. Ayres, P. & Sweller, J. (1990). Locus of difficulty in multi-stage mathematics problems. American Journal of Psychology, 103(2), 167 – 193. Ayres, P. (2000). Proceedings of the conference of the International Group for the Psychology of Mathematics Education, Vol. 2, Hiroshima, Japan, (pp. 25 - 32). Baroody, A.J. & Ginsburg, H.P. (1983). The effects of instruction on children‟s understanding of the equals sign. Elementary School Journal, 84, 199-212. Bell, J. (1987). Doing your research project. Philadelphia: Open University Press Booth, L.R. (1984). Algebra: Children's strategies and errors: a report of the strategies and errors in secondary mathematics project. Windsor, Berkshire: NFER-NELSON. Booth, L.R. (1986). Difficulties in algebra. The Australian Mathematics Teacher, October 1986, 2-4. Booth, L.R. (1988). Children‟s difficulties in beginning algebra. In A.F. Coxford & A.P. Shulte (Eds). The ideas of algebra, K-12. (1988 Yearbook: 8-19). Reston, VA: National Council of Teachers of Mathematics. Booth, L.R. (1989). A question of structure. In S. Wagner and C. Kieran (Eds.), Research issues in the learning and teaching of algebra. NCTM. Reston, Virginia. Boulton-Lewis, G., Cooper, T.J., Atweh, B., Pillay, H., & Wills, L. (1998). Arithmetic, prealgebra and algebra: A model of transition. In C. Kanes, M. Goos, & E.Warren (Eds.), Teaching mathematics in new times. Proceedings of the 21st annual conference of the Mathematics Education Research Group of Australasia, Vol. 1, pp. 114-120. Gold Coast: MERGA.
198
Radha Devi Unnithan and Russell Waugh
Boulton-Lewis, G.M., Cooper, T.J., Pillay, A.H., & Wilss, L. (2000). Readiness for algebra. Proceedings of the conference of the international group for the psychology of mathematics education, Vol. 2, Hiroshima, Japan, (pp. 89 – 105). Bright, G.W.(1981). Student procedures in solving equations. Teaching Statistics and Probability. Paper presented at the annual Meeting of the National Council of Teachers of Mathematics, 59th, St Louis, MO, Apr 22-25, 1981. Bouma, G. (2000). The research process (4th ed.). Melbourne: Oxford University Press. Carpenter, T.P. & Franke, M. (2001). Developing algebraic reasoning in the elementary school: Generalisation and proof. In H. Chick. Stacey, J. Vincent, & J. Vincent (Eds.), The future of the teaching and learning of algebra. Proceedings of the 12th ICMI study conference (Vol. 1, pp. 155-162). Dordrecht, The Nethelands: Kluwer Academic. Carpenter, T.P, Franke, M. & Levi, L. (2003). Thinking mathematically: Integrating arithmetic and algebra in elementary school. Portsmouth: Heinemann. Carpenter, T.P., Levi, L., & Farnsworth, V. (2000). Building a foundation for learning algebra in the elementary grades. In Brief, 1(2). ED 449015. Campbell, J.I.D. & Charness, N. (1990). Age-related declines in working-memory skills: Evidence from a complex calculation task. Developmental Psychology, 26, 879 – 888. Campbell, J.I.D. & Clark, J.M. (1992). Cognitive number processing: An encoding-complex perspective. In J.I.D. Campbell (Ed.). The Nature and Origins of Mathematical Skills, 457 – 491, Amsterdam: Elsevier. Chai, C.M. & Ang, G.H. (1987). Identifing the reasons underlying pupils' particular errors in simple algebraic expressions and equations. Mathematical Education in the 1990‟s: Proceedings of Fourth Southeast Asian Conference on Mathematical Education, Singapore, 189-198. Creswell, J. (1994). Research Design: Qualitative and quantitative approaches. Thousand Oaks, California: Sage Publications. Demana, F., & Leitzel, J. (1988). Establishing fundamental concepts through numerical problem solving. The ideas of algebra, K-12. (1988 Yearbook: 61-69). Reston, VA: National Council of Teachers of Mathematics. De Vaus, D. A. (1991). Surveys in social research (3rd ed.). London: Allen & Unwin. Dubois, B & Burns, J.A. (1975). An analysis of the meaning of the question mark response category in attitude scales. Educational and Psychological Measurement, 35, 869-884. Ekenstam, A. & Nilsson, M. (1979). A new approach to the assessment of children's mathematical competence. Educational Studies in Mathematics, 10, 41-66. Eagly, A.H., & Chaiken, S. (1993). The psychology of attitudes. Fort Worth, TX: Harcourt Brace Jovanovitch. Feynman, R. P. (1992). Surely, you're joking Mr Feynman. London: Vintage Books Filloy, E., & Rojano, T. (1989). Solving equations: The transition from arithmetic to algebra. For the Learning of Mathematics, 9(2), 19-25. Fontana, A., & Frey, J. (1994). Interviewing: the art of science. In N. K. Denzin & Y. S. Lincoln(Eds.), (1994). Handbook of qualitative research, (pp.361-376). Thousand Oaks: Sage Publications. Freedman, J.L., Sears, D.O., & Carlsmith, J.M. (1981). Social psychology. Eaglewood Cliffs, NJ: Prentice-Hall. Gay, L. R. (1987). Educational Research: competencies for analysis and application (3rd ed.). Columbus: Merrill Publishing Company.
A Rasch Measure Linking Self-Reported Student Attitude…
199
Glastonbury, B. & MacKean, J. (1991). Survey methods. In G. Allan & C. Skimer (Eds), Handbook for Research Students in the Social Sciences (pp. 225 – 247). London, UK: Falmer Press. Herscovics, N., & Kieran, C. (1980). Constructing meaning for the concept of equation. The Mathematics Teacher, 73(8), 572 – 580. Herscovics, N. (1989). Cognitive obstacles encountered in the learning of algebra. In S. Wagner and C. Kieran (Eds), Research issues in the learning and teaching of algebra (pp. 60-86). Lawrence Erlbaum Associates, National Council of Teachers of Mathematics, Reston, Virgina. Herscovics, N., & Linchevski, L. (1994). A cognitive gap between arithmetic and algebra. Educational Studies in Mathematics, 27(1), 59-78. Howie, S.J., Marsh, T.A., Allummoottil, J., Glencross, M., Deliwe, C., & Hughes, C.A. (2000). Middle school students‟ performance in mathematics in the third international mathematics and science study: South African realities. Studies in Educational Evaluation, 26(1), 61-77. Human, P. (1988). Problem transformation: a fundamental strategy in arithmetic and algebra. Pythagoras, 17, 6-10. Irwin, R.J. & Irwin, K.C. (2005). Attitude towards mathematics of 11-year olds: Analysis of a questionnaire and its relation to mathematical achievement. In Frontiers in Educational Psychology (pp.89-106)(R.F.Waugh, Editor). New York: Nova Science Publishers. Irwin, K. C. & Britt, M. S. (2005). The algebraic nature of students' numerical manipulation in the New Zealand numeracy project. Educational Studies in Mathematics, 58(2). 169188. Jick, T.D. (1979). Mixing qualitative and quantitative methods: Triangulation in action. Administrative Science Quarterly, 24(1), 602-611. Kaput, J., & Blanton, M. (2001). Algebrafying the elementary mathematics experience. In H. Chick, K. Stacey, J. Vincent, & J. Vincent (Eds.), The future of the teaching and learning of algebra. Proceedings of the 12th ICMI study conference (Vol. 1, pp. 344-352). Dordrecht, The Netherlands: Kluwer Academic. Kaur, B., & Yap, S.F. (1999). TIMSS- The strengths and weaknesses of Singapore‟s lower secondary pupils‟ performance in mathematics. In M. Waas (Ed.), Enhancing learning: Challenge of integrating thinking and information technology in the curriculum. (pp. 436444). Singapore: Educational Research Association. Kieran, C. (1989). The early learning of algebra: A structural perspective. In S. Wagner and C. Kieran (Eds.). Research Issues in the Learning and Teaching of Algebra. NCTM. Reston, Virginia. Kuchemann, D.E. (1980). The understanding of generalised arithmetic by secondary school children. Unpublished doctoral thesis, Chelsea College, University of London. Kuchemann, D.E. (1981). Algebra. In K. Hart (Ed.), Children's understanding of mathematics: 11- 16(pp. 102-119). London: Murray. Kuchemann, D.E. (1983). Quantitative and formal methods for solving equations. Mathematics in school, 12(5), 17-19. Leung, F.K.S. (2001). Behind the high achievement of East Asian students, Educational Research and Evaluation: An International Journal on Theory and Practice, 8(1), 87-108. Linacre, J.M. (2005). The Partial Credit Model and the One-Item Rating Scale Model. Rasch Measurement. Transactions, 19(1), 1000-10002.
200
Radha Devi Unnithan and Russell Waugh
Linchevski, L. (1995). Algebra with numbers and arithmetic with letters: A definition of prealgebra. Journal of Mathematical Behaviour, 14(1), 113-120. Linchevski, L., & Herscovics, N. (1994). Cognitive obstacles in pre-algebra. Proceedings of the 18th Conference of the International Group for the Psychology of Mathematics Education. Lisbon, Portugal, 3, 176- 183. Linchevski, L., & Herscovics, N. (1996). Crossing the cognitive gap between arithmetic and algebra: Operating on the unknown in the context of equations. Educational Studies in Mathematics, 30(1), 39-65. Linchevski, L., & Williams, J.S. (1996). Situated intuition, concrete manipulations and the construction of mathematical concepts: The case of integers. Proceedings of the 20th conference of the International Group for the Psychology of Mathematics Education, University of Valencia, Vol. 3, (pp. 265 – 272). Logie, R.H., Gilhooly, K.J & Wynn, V. (1994). Counting on working memory in arithmetic problem solving. Memory and Cognition, 22(4), 395 – 410. Lokan, J., & Greenwood, L. (2000). Mathematics achievement at lower secondary level in Australia. Studies in Educational Evaluation, 26(1), 9-26. Macgregor, M. (1996). Curricular aspects of arithmetic and algebra. In J. Gimenez, R.C. Lins, & B. Gomez (Eds), Arithmetics and algebra education: Searching for the future. (pp. 50 – 54). Macgregor, M., & Stacey, K. (1997). Students‟ understanding of algebraic notation 11-15. Educational Studies in Mathematics, 33, 1-19. MacGregor, M., & Stacey, K. (1997). Building foundations for algebra. Mathematics teaching in the middle school, 2, 252-260 Malara, N. & Navarra, G. (2003). ArAl Project: Arithmetic pathways towards favouring prealgebraic thinking. Bologana: Pitagora Editrice. McLean, M. (1995). Educational traditions compared. Content, teaching and learning in industrialized countries. London: David Fulton. McLeod, B.D.(1992). Affect in mathematics education: A reconceptualization. In A.D. Grouws (Ed.). Handbook of research on mathematics teaching and learning: A project of the national council of Teachers of Mathematics (pp 575-596). New York: Macmillan. Masters, G.N. (1982). A Rasch model for partial credit scoring. Psychometrika, 47(2), 149174. Masters, G.N. (1997). Partial Credit Model. In John P. Keeves (Ed.). Educational Research, Methodology and Measurement (2nd Edition). (pp. 857-863). Oxford: Pergamon Press (Elsevier). Mathison, S. (1988). Why triangulate? Educational Researcher, 17(2), 13-17. Matz,M. (1982). Towards a process model for high school algebra errors. In D. Sleeman and J.S. Brown(Eds.), Intelligent tutoring systems (pp. 25-50). Academic Press, London, New York. Maurer, S.B. (1987). New Knowledge about errors and new views about learners: What they mean to educators and more educators would like to know. In A.H. Schoenfeld(Ed.). Cognitive Science and Mathematics Education. Hillsdale, N.J: Lawrence Erlbaum Associates. Michel, J. (1999). Measurement in psychology. Cambridge, U K: Cambridge University Press.
A Rasch Measure Linking Self-Reported Student Attitude…
201
Michell, J. (1990). An introduction to the logic of psychological measurement. Hilsdale, NJ: Lawernce Erlbaum Associates, Publishers. Michell, J. (1999). Measurement in psychology: Acritical history of a methodological concept. Cambridge, UK: Cambridge University press. Miles, M. B., & Huberman, A. M. (1994). Qualitative data analysis (2nd ed.). Thousand Oaks, California: Sage Publications. Minichiello, V., Aroni, R., Timewell, E., & Alexander, L. (1991). In-depth interviewing: Researching people. Melbourne: Longman Cheshire. Ministry of Education. (2006). Secondary Mathematics Syllabus. Curriculum Planning and Development Division, Singapore. Ministry of Education. (2004). Nurturing every child. Flexibility and Diversity in Singapore schools. Singapore. Movshovitz-Hadar, N., Zaslavsky, O. & Inbar, S. (1987). An empirical classfication model for errors in high school mathematics. Journal for Research in Mathematics Education, 18(1), 3-14. Mullis, I.V.S., Martin, M.O., Gonzalez, E.J., Gregory, K.D., Garden, R.A., O‟Connor, S.J., Chrostowski, S.J., and Smith, T.A. (2000). TIMSS 1999 International Mathematics Report, Centre for the Study of Testing, Evaluation and Educational Policy, Boston College, Boston. Nasser, F. & Birenbaum, M. (2005). Modeling mathematics achievement of Jewish and Arab eighth graders in Israel: The effects of learner-related variables. Educational Research and Evaluation, 11(3), 277-302. O‟Brien, D.J. (1980). Solving equations. Shell Centre for Mathematical Education, University of Nottingham. Ong, S.T. & Lim S.K. (1987). Understanding and errors in algebra. Proceedings of Fourth Southeast Asian Conference on Mathematical Education, Singapore, 199-205. Oppenheim, A. N. (1992). Questionnaire design, interviewing and attitude measurement ( New ed.). London: Pinter Publishers. Papanastasiou, C. (2000). Internal and external factors affecting achievement in mathematics: Some findings from TIMSS. Studies in Educational Research, 26(1), 1-7. Papanastasiou, C. (2000). Effects of attitude and beliefs on mathematics achievement. Studies in Educational Research, 26(1), 27-42. Papanastasiou, C. (2002). Factors that differentiate mathematics students in Cyprus, Hong Kong, and the USA. Educational Research and Evaluation, 8, 129-146. Patton, M. (1990). Qualitative evaluation and research methods (2nd ed.). Newbury Park, California: Sage Publications. Pinchback, C.L. (1991) Types of errors exhibited in a remedial mathematics course. Focus on Learning Problems in Mathematics, 13(2), 53-61. Punch, K. (1998). Introducing to social research: Quantitative and qualitative approaches. London: Sage Publications. Radatz, H. (1979). Error analysis in mathematics education. Journal for Research in Mathematics Education, 10(3), 163-72. Rasch, (1960/1980/1992). Probabilistic models for some intelligence and attainment tests (Expanded edition). Chicago, IL: MESA Press (original work published in 1960).
202
Radha Devi Unnithan and Russell Waugh
Resnick, L.B., Nesher, P., Leonard, F., Magone, M., Omanson,S. & Peled, I. (1989). Conceptual bases of arithmetic errors: The case of decimal fractions. Journal for Research in Mathematics Education , 20(1), 8-27. Rojano, T., & Sutherland, R. (1994). Towards an algebraic notion of function. The role of spreadsheets. In D. Kirshner, (Ed.), Proceedings of the 16th annual meeting of the North American Chapter of the international group of the psychology of mathematics education (pp. 278-284). Baton Rouge: Louisiana State University. Rosnow, R. L., & Rosenthal, R. (1996). Beginning behavioural research: A conceptual primer (2nd ed.). New Jersey: Prentice-Hall. Saenz-Ludlow, A. & Walgamuth, C. (1998). Third graders‟ interpretation of equality and the equal symbol. Educational Studies of Mathematics, 35, 153-187. Sleeman, D.H. (1984). Solving linear algebraic equations. Mathematics in School, 13(4), 3738. Steinberg, R.M., Sleeman, D.H. & Ktorza, D. (1990). Algebra students' knowledge of equivalence of equations. Journal for Research in Mathematics Education, 22(2), 112121. Strauss, A., & Corbin, J. (1990). Basics of qualitative research: Grounded theory procedures and techniques. California: Sage Publications. Styles, I., & Andrich, D. (1993). Linking the standard and advanced forms of the Raven‟s Progressive Matrices in both the pencil-and-paper and computer-adaptive testing formats. Educational and Psychological Measurement, 53, 905-925. Vermeulen, N., Olivier, A., & Human, P. (1996). Students‟ awareness of the distributive property. Proceedings of the 20th Conference of the International Group for the Psychology of Mathematics Education, Vol. 4, Valencia, Spain, (pp.379-386). Vlassis, J. (2002). The Balance Model: Hindrance or support for the solving of linear equations with one unknown. Educational Studies in Mathematics, 49, 341 – 359. Warren, E. (2001). The importance of learning in the early years. In H. Chick. Stacey, J. Vincent, & J. Vincent (Eds.), The future of the teaching and learning of algebra. Proceedings of the 12th ICMI study conference (Vol. 2, pp. 633-640). Dordrecht, The Nethelands: Kluwer Academic. Warren, E. (2003). Young children‟s understanding of the equals: A longitudinal study. In N. Pateman, G. Dougherty, & J.Zilliox (Eds.), Proceedings of the 27th conference of the International Group for the Psychology of Mathematics Education (Vol 4, pp. 379-387). Hawaii: PME. Warren, E. (2003). The role of arithmetic structure in the transition from arithmetic to algebra. Mathematics Education Research Journal, 15(2), 122-137. Warren, E. & Cooper, T. (2001). Theory and practice: Developing an algebra syllabus for P7. In H.Chick, K.Stacey, J. Vincent, & J. Vincent (Eds.), The future of the teaching and learning of algebra. Proceedings of the 12th ICMI study conference (Vol. 2, pp.641-648). Dordrecht, The Netherlands: Kluwer Academic. Warren, E. & Cooper, T.J. (2005). Young children‟s ability to use the balance strategy to solve for unknowns. Mathematics Educational Research Journal, 17(1), 58-72. Waugh, R.F. (2002). Creating a scale to measure motivation to achieve academically: Linking attitudes and behaviours using Rasch measurement. British Journal of Educational Psychology, 72, 65-86.
A Rasch Measure Linking Self-Reported Student Attitude…
203
Waugh, R.F. (2003). (Editor). On the forefront of educational psychology. New York: Nova Science Publishers. Waugh, R.F. (2005). (Editor). Frontiers in educational psychology. New York: Nova Science Publishers. Waugh, R.F. & Chapman, E.S. (2005a). Attitude to mathematics for primary-aged students: A comparison of True Score Theory and Rasch measurement. In Frontiers in Educational Psychology (pp.89-106) (R.F.Waugh, Editor). New York: Nova Science Publishers. Waugh, R.F. & Chapman, E.S. (2005b). An analysis of dimensionality using factor analysis (True Score Theory) and Rasch measurement: What is the difference? Which method is better? Journal of Applied Measurement, 6(1), 107-116. Waugh, R.F. (2006). Rasch Measurement. In Neil J. Salkind (Ed.), the Encyclopedia of Measurement and Statistics. Thousand Oaks, CA: Sage Publications. Witzel, B., Smith, S.W., & Brownell, M.T. (2001). How can I help students with learning disabilities in Algebra? Intervention in School and Clinic, 37(2), 101 – 104. Wright, B.D. (1999). Fundamental measurement for psychology. In: The new rules of measurement (S.E. Embretson & S.L. Hershberger, Eds., pp. 65-104). Mahwah, NJ: Lawrence Erlbaum Associates. Wright, B.D., & Masters, G.N. (1981). Rating scale analysis. Chicago: Mesa Press. Wright,B.D, & Masters, G.N (1982). Rating scale analysis: Rasch measurement. Chicago, IL: MESA Press.
In: Specialized Rasch Measures… Editor: Russell F. Waugh, pp. 205-224
ISBN: 978-1-61668-032-9 © 2010 Nova Science Publishers, Inc.
Chapter 10
A RASCH MEASURE OF UNIVERSITY STUDENTS’ RECEPTIVITY TO PEERS WITH DISABILITIES ACROSS TWO CULTURES Minoti Biswas1 and Russell Waugh2 1
2
Narrogin Senior High School Faculty of Education and Arts Edith Cowan University
ABSTRACT This study aimed to create a linear measure of university students‟ receptivity to peers with disabilities across two cultures (Western Australia and India). Data were collected at two universities in Perth, Western Australia (Edith Cowan University and the University of Notre Dame in Fremantle), and two universities in India (The University of Calcutta, and the University of Jadavpur) via a 20 stem-item questionnaire (N=996) based on six aspects supporting receptivity to peers with disabilities: Academic, Interactive, Social, Personal, Professional and Supportive, answered in two perspectives: (1) an ideal self-view (What I think I should do) and (2) their self-reported behavior (what I actually do). This makes an effective item sample of 40. Following initial analysis with a Rasch computer program (RUMM 2020), 10 items were deleted as not fitting the measurement model (5 attitude and 5 different behaviors), leaving 30 items in the final scale. The final 30 items all fitted within p=0.01; the response categories were answered consistently and logically and the Person Separation Index was 0.87, but the item-trait interaction chi-square was too high (367.6,df=270,p=0.00) showing that the scale was not uni-dimensional. Attitude items were easier than behavior items, as predicted, but many items did not fit the model, probably because the complex wording allowed different persons to respond differently to the same item for the same level of receptivity.
INTRODUCTION People with disabilities are increasingly turning to higher education to achieve their career and professional goals (Prentice, 2002), and more and more people with disabilities are
206
Minoti Biswas and Russell Waugh
taking part in university education. Students with disabilities, whether at schools, colleges or universities, are taking advantage of increasing opportunities to acquire knowledge and skills, and participate in activities similar to their regular peers. However, there are barriers which still affect those with impairments and disabilities, negatively (Treolar, 1999). During the launching of the Disability Strategy on 30 April 2001, Helen Clark, Prime Minister of New Zealand, said that many people with impairment, “are unable to reach their full potential or participate fully in our communities because of the barriers they face doing everyday things” ( New Zealand Disability Strategy, 2001, p. 5). The barriers relate to various factors, but mainly social attitudes. “Personal and societal attitudes towards people with disabilities must change… for attitude is the biggest barrier facing disabled people” observed Gary Williams (New Zealand Disability Strategy, 2001, p.4). However, attitudes towards individuals are bound to vary from culture to culture, and there is a range of differences present because attitudes are very much influenced by culture and tradition (Saravanabhaban & Saravanabhavan, 2001). Fostering and promoting the positive receptivity of students with disabilities has become an issue in the community and at universities (Prentice, 2002; Biswas, 2002). There is a need to develop a scale of receptivity to peers with disabilities that can be used across cultures with confidence.
BACKGROUND Disability Issues and Right to Higher Education The rights of people with disabilities have been an issue in dispute for many years. Some claim that those with disabilities should have the same right as normal students (the same quality of education, the same status, the same degree of respect and so on). All students, regardless of disability and circumstance, have a right to access and participate in higher education, to fairness and equity, and to services and entitlements including opportunities to be independent (source of information: Disability Policy, 1992). The United Nations Declaration made during the International Year of Disabled people in 1981 included the right to receive an education that would enable students with disabilities to develop their skills and capacities to the full. The right was also embodied in the United Nations Conventions on the Rights of the Child in 1990 (Hert, 1993, cited by Jenkinson, 1997, p.25). The Declaration also reflected a disabled student‟s right to a future-based education that was comparable to that received by the majority of students. The rights of students with disabilities included the right to appropriate assessment of educational needs (Jenkinson, 1997, p.26). The 1981 Education Act in Britain and the New South Wales Department of School Education reflected the same rights (Human Rights Australia, 1993). The right to education in the European Convention of Human Rights, coupled with the Human Rights Act (1993) may give a right to have a disability properly taken into account in the education field. Article 2 of the First Protocol to the European Convention of Human Rights reads: “No person shall be denied the right to education. In the exercise of any functions that it assumes in relation to education and to teaching, the State shall respect the right of parents to ensure such education and teaching in conformity with their own religious and philosophical convictions” (Tyrer, 2002).
A Rasch Measure of University Students‟ Receptivity to Peers with Disabilities… 207
Perspectives on Disability Although better community awareness and community education are making valuable contributions to encourage people with disabilities prove their worth (Biswas, 2002; Prentice, 2002), students with disabilities may still feel misunderstood in certain educational settings… (Treolar, 1999). As research on the welfare of people with disability, particularly students with disability, shows that western societies aim to recognize their rights of people with disability in full equality with non-disabled members of the society, youth with disabilities in India have somewhat less opportunity due to the general climate within the educational institution and the community (Disability India.com, 2004). However, educational institutions are currently bound by the terms of the disability policies and legislation, and regular university students have the freedom to accept or not to accept the policies. An explanation can be found in the results of a recent study by this researcher. In the past, students with disabilities were excluded from regular education because they lacked certain self-care skills, or the abilities of communication and ambulation, creating a fear of rejection among those with impairments or disabilities. What is important in minimizing this fear of rejection is the receptivity to and acceptance of these students by their non-disabled peers. Specialized services and appropriately integrated educational environments would prove truly beneficial for students with disabilities only when regular students hold accepting attitudes towards their peers with disabilities through interactions at universities. It is, therefore, most appropriate for regular university students to have an understanding of disability, and acceptance of the support needs of those with disabilities. “This involves treating students with disabilities as people, seeing them as able, and accepting their differences, learning the appropriate language of disability, recognizing a student who might have a disability….”(Treolar, 1999 cited by Prentice, 2002, p.2). This means that successful initiatives should be designed by the universities to aid students in accessing the educational offerings at a higher education level. There have been policy moves in Australia, the USA, Canada, New Zealand, United Kingdom and India over the past 20 years, not only for better community awareness of people with disabilities, but also for community education to accept responsibility to meet the needs of students with disabilities in the regular class environment with non-disabled peers .
Receptivity to Students with Disabilities There seems to be a lack of research, in India, from the regular students‟ perspective, whether at school, college or university level, and there were no studies involving the creation of a unidimensional scale of regular students‟ receptivity of peers with disabilities across South-East Asia and Australia (Waugh & Biswas, 2003). Thus, the present study is expected to fill a gap, to generate new knowledge, and produce a new cross-cultural measure of Receptivity of Peers with Disabilities which could provide information that would help university administrators better cater for students with disabilities. Studies exploring personal and societal attitudes and behaviors toward students with disabilities show that, in spite of the changing policies promoting equal opportunities for students with disabilities, peer attitudes and behaviors toward those with disabilities, still need to improve. Regular university students sometimes need to be motivated into accepting peers who have special needs. This is
208
Minoti Biswas and Russell Waugh
possible with student awareness and understanding of disabilities, and also appropriate behavior toward individuals with disabilities on the university campus. Just as government and university policies and legislation on disability in Western Australia seek to bring about change, provide people with equal opportunities to realize their individual capabilities and potentials through full participation in social and university activities, the Government of India has also announced a plan to make education disabledfriendly by 2020 and „to make mainstream education not just available but accessible, affordable and appropriate for students with disabilities‟…(Singh, 2001). However, it is not just enough to give admission to students with disabilities. It is also important to provide necessary facilities for them in the colleges so that they are part of the mainstream in the true sense of the word. Recent reports show that the Government of India is providing scholarships to students with disabilities to pursue studies at post school level. Findings indicate that regular university students often lack motivation and experience to establish interpersonal relationships with peers with disabilities (Biswas, 2002). That study also indicated that the general attitudes of university students toward peers with disabilities are positive, but it is hard for many to translate their attitudes into actual positive behaviors. Many students with disabilities (but perhaps not all) don‟t need special treatment so much as they need understanding, fair treatment, positive receptivity and the allocation of appropriate resources to allow them to participate. A cross-cultural measure of University Students‟ Receptivity to Peers with Disabilities would be important in this context because it would provide good information with implications for future policy directions by university administrators that would establish links between attitudes and behaviors that may lead to better receptivity and better acceptance of peers with disability at university.
Importance of Research on Disability Research in Australia indicates that the mainstreaming of secondary school with disabilities into the regular classes (now called inclusion) has a significant effect on these students and helps them to develop better self-esteem, social understanding and interpersonal relationships (Disability Policy, 2004; Noland, McLaughlin, Howard & Sweeny, 1993). Special provisions are determined for students with severe physical or sensory difficulties, and for students who have a specific learning difficulty, to assist them to access and complete the course being undertaken on an equal basis with their non-disabled peers in Western Australia. Available literature shows that there had been surveys focusing on integration, introducing steps to improve the campus climate for students, and promoting classroomawareness in high schools. Kemp (2003) and Ward, Center and Bochner (1994) investigated the possibilities of integrating children with disabilities into regular classrooms. A search of the literature found no other related studies with Western Australian data. However, literature shows that research on attitude towards students with disabilities and studies on integration in schools were conducted by Gannon and MacLean (1996). Forlin, 1997), Darcy and Daruwalla (1999), Tait and Purdie (2000), who focused on integration, introducing steps to improve the campus climate for students, and promoting classroom-awareness in high schools. The Indian scenario is somewhat different. It has been part of India‟s cultural heritage to help the poor, the aged and those with disabilities. However, it was only half a century ago that it was understood that people with disability conditions had potential and talent which
A Rasch Measure of University Students‟ Receptivity to Peers with Disabilities… 209 needed to be understood, realized and promoted. Of late, there is a realization that people with disabilities are capable of living independently and that they can be useful contributors to society (Saravanabhavan & Saravanabhavan, 2001). As understanding attitudes toward people with disabilities is an important necessity to build an effective educational system and an integrated society (Saravanabhavan & Saravanabhavan, 2001), more and more articles on the acceptance of disability have being published, and studies had been undertaken by researchers in India. Programs focused on Inclusive Education incorporated attitude-changing strategies which aimed at acceptance of peers with disabilities by the regular students (Lynch in Centre for Studies on Inclusive Education, (2002). Studies focused on educational and vocational training, social integration of the visually impaired, and rehabilitation of children with disabilities particularly the visually impaired were conducted. Singh (2001), in his book „Enabling the Differently Able‟, discussed various kinds of disabilities, and social problems faced by those with disabilities. As the research shows there is a lack of similar or related studies in Australia and India. The present study is significant because it is a first step to achieve an aim of producing a scale of university students‟ receptivity to peers with disabilities that can be used across two or more cultures. It is often difficult to create questionnaires that can be used to gather data from which linear scales can be created that provide reliable information comparing receptivity across two or more cultures. Measuring attitudes and behaviors on the same scale by having respondents answer items in both an attitude and behavior perspective is a relatively new method that is gaining favor with Rasch measurement experts (Waugh, 2003, 2005). This procedure is used in the present study to compare attitudes and behaviors. The usual procedure with True Score Theory (nonlinear measures) is to measure attitudes with one scale and behaviors with another, and then correlate them. This typically gives a correlation of the order of 0.1 to 0.4 that doesn‟t support the theory of attitudes influencing behavior as well as Rasch measures do.
AIMS There were two main aims to the present study. 1. To create a cross-cultural English questionnaire (for India and Western Australia) on University Students‟ Receptivity towards Peers with Disabilities based on six aspects- (i) Academic (Special and Alternative Programs), (ii) Interactive (Inclusive Courses and Interaction and Improvement of Self-image), (iii) Social (Promote Relationships through Recreational Programs and Recognition of Achievements), (iv) Personal (Involvement), (v) Professional (Integrated workforce) and (vi) Supportive (Special University Policies and Procedures). The questionnaire is to be applicable across university cultures in two countries (India and Western Australia). 2. To create a linear scale of the variable, University Students‟ Receptivity towards Peers with Disabilities using the RUMM2020 computer program (Andrich, Sheridan & Luo, 2005), with data collected from India and Western Australia.
210
Minoti Biswas and Russell Waugh
STRUCTURE OF THE QUESTIONNAIRE ITEMS For each of the six aspects of receptivity, the items were conceptually ordered from easy to hard. For example, under receptivity to inclusive courses, stem-item 3 was conceptually written as easier than stem-item 4. That is, taking an interest in the university trying to include peers with disabilities in all degree courses such as sports and dance was considered to be easier than encouraging friends to participate in inclusive courses that make academic goals a reality for peers with disabilities at university. This is because encouraging friends to participate involves talking to friends and persuading them to join you whereas just taking an interest doesn‟t involve much effort. Answering each item in the ideal self-view perspective was considered to be easy that answering in the behavior self-view. This is because a behavior involves actually having the attitude and then exerting some effort to actually perform the behavior and, to perform the behavior, the students would have to give up their time on some other, perhaps more desirable, pursuit. So this section of items is conceptually set out in a easy/harder pattern downwards, and an easier/harder pattern across from left to right. When the item difficulties are measured on the same scale through a Rasch-created linear scale, this conceptual structure can be tested. The items for the other aspects of receptivity were similarly designed and, although the conceptual structure is not reported here, this structure can easily be worked out from Table 5 by a reader.
METHOD Permission was obtained from the University Ethics Committee, Heads of Schools and lecturers of all the four universities concerned, to administer the questionnaires on appropriate days and time. A total of 996 regular university students comprising pre-service teaching students, and 4th and 5th Year students of Education across four universities in Western Australia and India participated in the study voluntarily and anonymously. The samples from each university were considered to be representative of their population of Bachelor Degree Teacher Education students comprising males and females, different nationalities and cultural backgrounds, and varying social and religious beliefs. Sample sizes were as follows: Edith Cowan University (N=206), University of Notre Dame (N=150), University of Calcutta (N=344) and Jadavpur University (N=296). The questionnaire was trialed with students at each of the universities to determine the relevance of the statements to the sub-headings, the appropriateness of the language, and the suitability of the questionnaire. The following familiar aspects of receptivity were included: Academic (Special and Alternative Programs), Interactive (Inclusive Courses and Interaction and Improvement of Self-image), Social (promote relationships through Recreational Programs and recognition of Achievements), Personal (Involvement), Professional (Integrated Workforce) and Supportive (Special university policies and procedures) to increase their applicability. The wording of certain items was reconstructed so that respondents were more comfortable with them, particularly those with English as a second-language. The initial Rasch analysis with 40 items (20 stem-items) showed that five attitude selfview items (stem-items 1,2,3,4,20) and five different behavior self-view items (stem-items
A Rasch Measure of University Students‟ Receptivity to Peers with Disabilities… 211 7,9,12,14,19) did not fit the measurement model and these ten items were discarded, leaving 30 items in the final analysis.
RESULTS OF RASCH ANALYSIS Data from the 996 questionnaires were analyzed together using the Rasch Unidimensional Measurement Model (RUMM) computer program (Andrich, Sheridan & Luo, 2005), in order to create a single linear scale on which measures from students at all four universities could be compared. The Partial Credit Model of Rasch was used and the equations for this are given in Masters (1997). The results are set out in figures and tables and descriptive text, and show how the data fit the measurement model through a series of item analysis checks. Table 1 shows the global fit statistics for Receptivity to Peers with Disabilities data. When the item and person data fit the measurement model, the mean fit residual should be near zero and its standard deviation should be near 1 (as is the case here). Table 1. Global Item and Person Fit (N=996, I=30) ITEM-PERSON INTERACTION Location Mean SD
0.00 1.20
ITEMS
PERSONS
Fit Residual
Location Fit
Residual
0.15 1.34
0.37 0.87
-0.31 1.42
Notes on Table 1: 1.The item means are constrained to zero by the measurement model. 2. When the data fit the measurement model, the mean fit residuals should be close to zero and the standard deviations should be close to one. In this case, there is good item and person fit to the measurement model. 3. The data are given to two decimal places because the errors are about 0.05 (one cannot have measures more accurate than the errors).
Fit of Individual Items to the Measurement Model All 30 items fitted the measurement model with a probability greater than 0.01 (see Table 2). These good individual item fits to the measurement model supported the good global item fit given in Table 1. It should be pointed out here that the student numbers are so large (N=996) that even minor deviations from the chi-square test model will show up and so the lower probability fits should not be taken too literately. There is a good individual item fit to the measurement model.
Minoti Biswas and Russell Waugh
212
Table 2. Individual Item-fit Characteristics Item
Location
SE
Residual
df
ChiSquare
Probability
2
+0.69
0.05
+0.83
954.93
7.74
0.56
4
+1.09
0.05
+1.06
954.93
14.96
0.09
6
+0.86
0.05
+2.41
954.93
21.32
0.01
8
+1.19
0.05
+2.55
954.93
14.07
0.12
9
-1.12
0.06
-0.94
954.93
11.27
0.26
10
+0.95
0.05
-0.87
954.93
17.72
0.04
11
-0.65
0.07
-1.06
954.93
10.44
0.32
12
+1.63
0.05
-0.16
954.93
7.87
0.55
13
-0.45
0.05
+0.56
953.97
7.79
0.56
15
-1.48
0.07
-0.69
953.00
6.78
0.66
16
+0.43
0.05
+3.68
953.00
11.43
0.25
17
-0.36
0.05
+0.13
952.04
17.83
0.04
19
-1.70
0.07
-0.52
952.04
11.06
0.27
20
+0.94
0.05
+1.93
952.04
6.75
0.66
21
+0.99
0.06
-0.04
952.04
7.64
0.57
22
+1.38
0.05
-0.01
952.04
9.36
0.40
23
-1.32
0.06
-1.15
951.07
11.62
0.24
25
-0.73
0.06
-0.75
951.07
12.85
0.17
26
+1.57
0.06
-0.41
951.07
15.84
0.07
27
-1.47
0.06
-0.39
951.07
9.27
0.41
29
-1.53
0.07
-1.03
951.07
18.92
0.03
30
+0.47
0.05
+0.61
951.07
11.45
0.25
31
-1.43
0.07
-1.24
951.07
12.69
0.18
32
+1.01
0.05
-0.02
951.07
12.59
0.18
33
-1.06
0.06
-1.88
951.07
18.37
0.03
34
+1.04
0.05
-0.75
951.07
9.73
0.37
35
-0.46
0.05
+0.81
951.07
11.80
0.22
36
+1.55
0.05
-1.40
951.07
13.45
0.14
37
-1.75
0.07
+0.91
951.07
10.07
0.34
40
+1.70
0.06
+2.15
942.39
14.99
0.09
Notes on Table 2. 1.Location means item difficulty measured in logits (log odds of answering positively) on the Rasch scale. 2.SE is the standard error in logits. 3.Chi-square is the test-of-fit statistic for each item using the degrees of freedom. All items fit within p=0.01. 4.df means degrees of freedom. 5.Residuals are the differences between the actual values and the expected values, calculated according to the measurement model. 6.Probability is the probability of fit to the measurement model based on the chi-square.
A Rasch Measure of University Students‟ Receptivity to Peers with Disabilities… 213
Dimensionality Table 3 provides data relating to the collective agreement amongst all students across all items. For any particular total raw score, a mean of the actual responses for item i can be calculated. This can be compared to the expected response on item i calculated from the Rasch parameter estimates and this can be done for all total scores over all items. A resulting chi-square can be calculated (see Andrich & van Schoubroeck, 1989) and, if the observed and expected values are not significantly different, then it can be inferred that there is good agreement amongst all the students regarding the difficulties of the items along the scale. This would mean that a unidimensional scale has been measured. In the present case, there is significant interaction and student agreement on the item difficulties is not ideal, so it can only be claimed that a dominant trait has been measured and overall fit to the measurement model needs to be improved in any future use of the questionnaire. This can probably be initiated by reducing the complexity of the item wordings so that there is more likelihood of getting agreement on the meaning of the item by the students from the different cultures. Table 3. Item-trait Interaction for Dimensionality ITEM-TRAIT INTERACTION FOR DIMENSIONALITY Total Item Chi-Square
367.60
Total Degrees of Freedom
270.00
Total Chi-square Probability
0.00
Table 4. Item Thresholds ( I=30 N=996 CAT=3) Thresholds Item
Mean
1
2
2 4
+0.69
-0.06
+1.44
+1.08
+0.86
+1.31
6
+0.86
+0.05
+1.67
8
+1.19
+0.45
+1.94
9
-1.12
-1.97
-0.28
10
+0.95
+0.15
+1.74
11
-0.65
-1.67
+0.37
12
+1.63
+1.10
+2.17
13
+0.45
-1.36
+0.46
15
-1.46
-2.03
-0.92
16
+0.43
-0.01
+0.87
17
+0.36
-1.25
+0.53
19
-1.70
+3.01
+0.39
20
+0.94
+0.01
+1.87
21
+0.99
+1.10
+0.02
22
+1.38
+0.82
+1.94
23
-1.32
-2.52
-0.12
25
-0.73
-1.84
+0.37
Minoti Biswas and Russell Waugh
214
Table 4. (Continued). Thresholds Item
Mean
1
2
26
+1.57
+0.82
+2.32
27
-1.47
-2.65
-0.29
29
-1.52
-2.38
-0.67
30
+0.47
-0.37
+1.31
31
-1.42
-2.59
-0.26
32
+1.00
+0.07
+1.93
33
-1.06
-1.93
-0.19
34
+1.00
+0.44
+1.64
35
-0.46
-1.13
+0.21
36
+1.55
+1.31
+1.79
37
-1.75
-2.56
-0.94
40
+1.70
+0.77
+2.63
Notes on Table 4. There are two thresholds per item: one between response categories 1 and 2, and one between response categories 2 and 3. The response categories are ordered for each item implying that the students used tem consistently and logically. The mean threshold is the item difficulty in logits.
Reliability of the Scale The Person Separation Index was 0.87 and this indicates that the student measures are well separated along the scale compared to the errors of measurement (which are about 0.05 to 0.07 logits) as required for good measurement.
Thresholds Thresholds are points between adjacent response categories where the odds are 1:1 of answering in either category. The response categories are: on hardly any occasion (score 1), on some occasions (score 2), and on many occasions (score 3). When the response categories are answered consistently and logically, the thresholds should be ordered in line with the conceptual ordering of the response categories (as is the case with the present data, see Table 4).
The Linear Scale of Receptivity to Peers with Disabilities Table 5 shows the 30 items that best fitted the measurement model and could be said to form a linear scale with one dominant dimension, here called University Students‟ Receptivity to Peers with Disabilities. The other 10 items that did not fit the measurement
A Rasch Measure of University Students‟ Receptivity to Peers with Disabilities… 215 model were deleted and not used in any further analysis. Findings from the Rasch analysis demonstrated that the ideal self-views (attitudes) are easier than the actual self-views (behaviours) for all items where items for both perspectives fit the measurement model. It can be concluded that: 1. There was a good global item fit to the measurement model; 2. There was good global person fit to the measurement; 3. There was good individual item fit to the measurement model; 4. The scale data measured a dominant trait, taken to be University Students‟ Receptivity to Peers with Disabilities, with some “noise” present; 5. The response categories were answered consistently and logically; 6. The targeting of the item thresholds against the person measures was good. The easiest attitude items (such as items 39 and 19) can be answered positively by all the students, and the hardest behavior items (such as items 40 and 12) can only be answered positively by students with a high Receptivity. Table 5. Items and their Difficulties (Final Scale, I=30) Item No.
Item Wording
What I think I should do (ideally)
What I actually do (real behaviour)
DNF
+0.69
DNF
+1.08
DNF
+0.86
DNF
+1.19
-1.12
+0.95
-0.65
+1.63
Academic Self-View at University Receptivity to implementation of alternative programs 1-2Support the idea of alternative programs being implemented for assessing peers with disabilities for access into higher education. 3-4 Support the implementation of virtual field excursions in field courses to facilitate learning for peers with mobility impairment. INTERACTIVE Receptivity to inclusive courses 5-6 Take an interest in the university trying to include peers with disabilities in all degree courses such as sports and dance. 7-8 Encourage friends to participate in inclusive courses that make academic goals a reality for peers with disabilities at university. Receptivity to inclusive interaction 9-10 Motivate regular students to assist peers with disabilities cope with assignments. 11-12 Discuss with friends, positive strategies to assist peers with disabilities to participate in on-line group activities and/or other adaptive physical activities.
SOCIAL Receptivity to promotion of social relationships through recreational programs
216
Minoti Biswas and Russell Waugh Table 5. (Continued).
Item No.
Item Wording
13-14 Discuss with the university‟s equity group their plans for interactive recreational programs to promote social relationships between non-disabled students and their peers with disabilities. Receptivity to achievements 15-16Make an effort to appreciate and recognise academic and also non-academic achievements of peers with disabilities at university.
What I think I should do (ideally) +0.45
What I actually do (real behaviour) DNF
-1.46
+0.43
+ 0.36
DNF
-1.70
+0.94
+0.99
+1.38
-1.32
DNF
PERSONAL Receptivity to personal involvement 17-18 Sign up as a mentor or „buddy‟ to support and assist mobility-limited peers to participate in outdoor activities at university. PROFESSIONAL Receptivity to integrated workforce 19-20Value the contributions of peers with disabilities in an integrated workforce at the university that provides the training. SUPPORTIVE Accommodation 21-22Encourage peers with mobility impairment to ask the university to provide affordable facilities and assistive devices, such as virtual field excursions and modified physical activity programs. Collaboration 23-24 Support the policy of collaborative teaching programs to improve transition of students with special needs into employment or work experience at university.
DNF means did not fit the measurement model (and was deleted).
AGREEMENT WITH THE MODEL BEHIND THE QUESTIONNAIRE The item difficulties demonstrated that the ideal self-views (attitudes) were easier than the actual self-views (behaviours) for all items where both perspectives fit the measurement model, in agreement with the model behind the questionnaire (but it is noted that ten items did not fit the measurement model and were deleted). It would seem that the ten non-fitting items may be too complex and the university students interpreted them in different ways which, in turn, meant that the students couldn‟t agree about the difficulty of those ten items. This would explain why the ten items did not fit the measurement model. The non-fitting items need to have their wording simplified in order that there can be a more uniform understanding of what the items mean which, in turn, would lead to a better agreement about the difficulty of the items.
A Rasch Measure of University Students‟ Receptivity to Peers with Disabilities… 217 The item difficulties of the other 30 items were ordered in agreement with the model behind the questionnaire where there were two or more items under one aspect (there was only one small disagreement between items 32 and 34, see Table 5, where the difficulties were equal). So there is reasonable support for the conceptual structure behind the questionnaire. It would seem that, if the ten deleted items could be re-worded and further data collected, the conceptual structure could be re-tested and would likely be supported. Since the structure was used across two very different cultures (India and Western Australia), this is a good advance for cross-cultural measures using Rasch measurement and it is recommended that researchers involved with cross-cultural work with disability policy take up possible improvements with the current measure. Table 6. Ideal Attitude Self-Views of Receptivity to Peers with Disabilities Item Number
Item Wording
1 (33) Encourage peers with mobility impairment to ask the university to provide affordable facilities and assistive devices, such as virtual field excursions and modified physical activity programs. 2 (13) Discuss with the university‟s equity group their plans for interactive recreational programs to promote social relationships between non-disabled students and their peers with disabilities. 3 (17) Sign up as a mentor or „buddy‟ to support and assist mobility-limited peers to participate in outdoor activities at university. 4 (35) Ensure that peers with disabilities avail themselves of opportunities provided by the university and pursue careers within the university. 5 (11) Discuss with friends, positive strategies to assist peers with disabilities to participate in on-line group activities and/or other adaptive physical activities. 6 (25) Display interest in the collaborative approaches used for program Implementation for peers with disabilities; eg the collaboration between universities and industries which train and employ graduates with a disability. 7 (33) Encourage peers with disabilities to apply for jobs within the university. 8 (9) Motivate regular students to assist peers with disabilities cope with assignments. 9 (23) Support the policy of collaborative teaching programs to improve transition of students with special needs into employment or work experience at university. 10 (31) Try to encourage friends to support equitable employment practices at university to enable peers with disabilities find jobs in a non-discriminatory environment. 11 (15) Make an effort to appreciate and recognise academic and also non-academic achievements of peers with disabilities at university. 12 (27) Support the validated instructional approach to ensure access to peers with disabilities to higher education. 13 (29) Support the policy of equal opportunity and equitable recruitment provided by the university for students with disabilities.
What I think I should do (ideal self-view) (hardest) +0.99
+0.45
-0.46
-0.65
-0.73
-1.06 -1.12 -1.32
-1.42
-1.46
-1.47 -1.52
Minoti Biswas and Russell Waugh
218
Table 6. (Continued). Item Number
Item Wording
14 (19) Value the contributions of peers with disabilities in an integrated workforce at the university that provides the training. 15 (37) Support access of peers with disabilities in all academic courses at university
What I think I should do (ideal self-view) (hardest) -1.70 -1.75 (easiest)
Note: Number in brackets is original item number.
WHAT DOES THE SCALE MEAN? The items and their order of difficulty define the scale and define what is being measured. The three easiest attitude items are: (15) Support access of peers with disabilities in all academic courses at university (easiest); (14) Value the contributions of peers with disabilities in an integrated workforce at the university that provides the training: and (13) Support the policy of equal opportunity and equitable recruitment provided by the university for students with disabilities (harder, but still very easy). The difficulties of these attitudes are as expected and nearly all the students can answer these positively. The three hardest attitude items are: (1) Encourage peers with mobility impairment to ask the university to provide affordable facilities and assistive devices, such as virtual field excursions and modified physical activity programs (hardest and reasonably hard); (2) Discuss with the university‟s equity group their plans for interactive recreational programs to promote social relationships between non-disabled students and their peers with disabilities; (3) Sign up as a mentor or „buddy‟ to support and assist mobility-limited peers to participate in outdoor activities at university (easier, but moderately hard). The difficulties of these attitudes are as expected. The three easiest behavior items are (and these are moderately hard): (16) Make an effort to appreciate and recognize academic and also non-academic achievements of peers with disabilities at university: (30) Support the policy of equal opportunity and equitable recruitment provided by the university for students with disabilities; 3 (2) Support the idea of alternative programs being implemented for assessing peers with disabilities for access into higher education. The three hardest behavior items are very hard: (40) Involve myself to promote optimal participation of peers with disabilities in quality higher education: (10) Discuss with friends, positive strategies to assist peers with disabilities to participate in on-line group activities and/or other adaptive physical activities; (26)Display an interest in the collaborative approaches used for program implementation for peers with disabilities; eg the collaboration between universities and industries which train and employ graduates with a disability. These item difficulties are as expected and only those students with a very high receptivity measure can answer these positively.
A Rasch Measure of University Students‟ Receptivity to Peers with Disabilities… 219 Table 7. Actual Self-Views of Behavior towards Peers with Disabilities Item No.
Item Wording
1 (16) Make an effort to appreciate and recognize academic and also non-academic achievements of peers with disabilities at university. 2 (30) Support the policy of equal opportunity and equitable recruitment provided by the university for students with disabilities. 3 (2) Support the idea of alternative programs being implemented for assessing peers with disabilities for access into higher education. 4 (6) Take an interest in the university trying to include peers with disabilities in all degree courses such as sports and dance. 5 (20) Value the contributions of peers with disabilities in an integrated workforce at the university that provides the training. 6 (10) Motivate regular students to assist peers with disabilities cope with assignments. 7 (32) Try to encourage friends to support equitable employment practices at university to enable peers with disabilities find jobs in a non-discriminatory environment. 8 (34) Encourage peers with disabilities to apply for jobs within the university. 9 (4) Support the implementation of virtual field excursions in field courses to facilitate learning for peers with mobility impairment. 10 (8) Encourage friends to participate in inclusive courses that make academic goals a reality for peers with disabilities at university. 11 (22) Encourage peers with mobility impairment to ask the university to provide affordable facilities and assistive devices, such as virtual field excursions and modified physical activity programs. 12 (36) Ensure that peers with disabilities avail themselves of opportunities provided by the university and pursue careers within the university. 13 (26) Display interest in the collaborative approaches used for program implementation for peers with disabilities; eg the collaboration between universities and industries which train and employ graduates with a disability. 14 (10) Discuss with friends, positive strategies to assist peers with disabilities to participate in on-line group activities and/or other adaptive physical activities. 15 (40) Involve myself to promote optimal participation of peers with disabilities in quality higher education.
What I actually do (real behavior) (Easiest but still moderately hard) +0.43
+0.47
+0.69
+0.86
+0.94
+0.95 +1.00
+1.00 +1.08
+1.19
+1.38
+1.55
+1.57
+1.63
+1.70 (hardest and very hard)
Note: Number in brackets is original item number.
220
Minoti Biswas and Russell Waugh
DISABILITY POLICY ISSUES FROM THE MEASURE As globalisation continues and universities continue to take students from many other countries (Australian universities take many students from many countries in South-East Asia), cross-cultural measures like the one in the present study become more important. The measure of University Students‟ Receptivity to Peers with Disabilities shows which items (aspects relating to disabilities) are very hard and thus indicate what the universities are not implementing very well or what policies the universities could implement better. Based on the present measure, it seems that, in both India and Western Australia, university administrations are not actively working to involve regular students in promoting or helping those students with disabilities. University administrations have not developed and implemented active policies that would encourage regular students to make time and effort in helping their peers with disabilities at university. The administrations have not developed active policies to engage regular students in academic, sporting or collaborative industry partnerships for students with disabilities, despite the growing number of students with disabilities studying at universities (see Prentice, 2002; and the official data information published by universities). The measures also show evidence that university administrations are not providing sufficient resources for students with disabilities or encouraging regular students to help students with disabilities gain those resources, which can benefit the universities as well as the students with disabilities. Even where the universities have provided resources for students with disabilities, the items relating to their use and availability are still relatively hard on the measure and are not well known by regular students, and have thus not been well promoted by the university. Behaviour items on the measure clearly point to improvements in policy that could be implemented by university administrations. The measure shows that it was easy for students to support the policy of equal opportunity and equitable recruitment provided by the universities for students with disabilities, to value the contributions of peers with disabilities in an integrated workforce, and to support access of peers with disabilities to all academic courses at the universities. So, in regard to university policy and implementation, the measure implies that the universities did not take advantage of this support to publicize and implement better policies with the support of regular students.
DIFFERENCES IN RECEPTIVITY MEASURES BY UNIVERSITY The mean measures of Students‟ Receptivity to Peers with Disabilities, and their standard deviations, are given in Table 8 and in Figure.1. Since these are all measured on the same linear scale, they can be directly compared. As expected, the means for the Indian universities (Calcutta and Jadavpur) are higher than those in Western Australia (Edith Cowan and Notre Dame), and the mean for Notre Dame University is higher than the mean for Edith Cowan University. If it is assumed that the university samples are representative of students in their respective teacher-education populations and, if it is assumed that the samples could be considered as part of the same large population of teacher-education students (Western Australia-India combined), then the separate university samples could be considered as being
A Rasch Measure of University Students‟ Receptivity to Peers with Disabilities… 221 taken from the same large Western Australia-India population of teacher-education students and a t-test between the means can be used to determine if the mean university measures are statistically significantly different. Table 8. Mean Measures by University in Logits University University of Calcutta University of Jadavpur University of Notre Dame Edith Cowan University
Number 344 294 147 208
Mean Measures 0.47 0.52 0.30 0.05
Standard Deviation 0.79 0.69 1.11 0.94
Note. A logit is the standard Rasch measurement unit.
Indian Universities Versus University of Notre Dame It was expected that the Indian university students in the teacher-education programme should have a higher Receptivity towards Peers with Disabilities than students in the teachereducation programme at the University of Notre Dame, and this was found to be the case (see Table 8). The t-test results for Students‟ Receptivity to Peers with Disabilities are: (1) between the University of Calcutta and the University of Notre Dame (t=1.92, df= 489, p=0.02) in favor of Calcutta; and (2) between the University of Jadavpur and the University of Notre Dame (t=2.55, df= 439, p=0.005) in favor of Jadavpur. The corresponding effect sizes are d=0.18 and d=0.24 which are small (Cohen, 1988).
Indian Universities Versus Edith Cowan University It was expected that the Indian university students should have a higher Receptivity towards Peers with Disabilities than students at Edith Cowan University in Perth, Western Australia and this was found to be the case. The t-test results for Students‟ Receptivity to Peers with Disabilities are: (1) between the University of Calcutta in India and Edith Cowan University in Perth, Western Australia (t=5.6, df = 550, p= 0.000) in favor of Calcutta, and between the University of Jadavpur in India and Edith Cowan University in Perth, Western Australia (t=6.45, df = 500, p= 0.000) in favor of Jadavpur. The corresponding effect sizes are d=0.48 and d=0.59 which are medium (Cohen, 1988).
University of Notre Dame Versus Edith Cowan University It was expected that the students from Notre Dame University in Perth, Western Australia, should have a higher Receptivity towards Peers with Disabilities than students at Edith Cowan University in Perth, Western Australia and this was found to be the case. The ttest results for Students‟ Receptivity to Peers with Disabilities are: between the University of Notre Dame and Edith Cowan University (t=2.28, df = 353, p = 0.003) in favor of Notre Dame. The effect size is d=0.25 which is small (Cohen, 1988).
222
Minoti Biswas and Russell Waugh
Note: 1. Person measures are on the topside of the linear scale in logits. 2. Item difficulties are on the bottom side of the linear scale in logits. 3. There is an error in the RUMM computer program on colors. Maroon is Calcutta , brown is Jadavpur (not green), green is ECU (not red), and purple is Notre Dame (not blue). Figure 1. Student Measures by University.
Four main inferences were drawn from the Rasch-created linear scale of Receptivity to Peers with Disabilities. One is that the ideal self-views (attitudes) are easier than the actual self-views (behaviors), for all items where both perspectives fit the measurement model. Two is that the students do make an effort to appreciate and recognize academic and non-academic achievements of peers with disabilities at university but find it moderately hard to do so. Three is that the students found it very hard to involve themselves in promoting optimal participation of peers with disabilities in quality higher education. Four is that Receptivity to Peers with Disabilities is significantly higher at the Universities of Calcutta and Jadavpur in India than at Edith Cowan University and the University of Notre Dame in Western Australia, and Receptivity is significantly higher at the University of Notre dame than at Edith Cowan University.
REFERENCES Andrich, D., Sheridan, B. & Luo, G. (2005). RUMM 2020: A windows-based item analysis program employing Rasch unidimensional measurement models. Perth: RUMM Laboratory Biswas, M. (2002). University students‟ acceptance of peers with disabilities. Unpublished masters thesis. Edith Cowan University, Perth, Western Australia. Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum Associates. Disability India Organization. (2004). [On-line] Available: http//www/disabilityindia.org/ din.Jour/article3.html
A Rasch Measure of University Students‟ Receptivity to Peers with Disabilities… 223 Disability Policy for Western Australia (August, 1992). An introduction to A Fair go for Everyone [Video]. Perth: LISWA. Available from Cambridge Library, Floreat Forum, Western Australia. Forlin, C. (1997). Re-designing pre-service teacher education courses: An inclusive curriculum in new times. Reports-Research. Retrieved June 6, 2001, from Eric Document: „E‟ Subscribe on-line database. Reproduction Service No .ED 425 583. Gannon, P.M. & MacLean, D. (1996). Attitudes towards disability and beliefs regarding support for a university student with quadriplegia. International Journal of Rehabilitation Research, 19 (2), 163-169 Gleason, J. (1991). Multicultural and exceptional student education: Separate but equal? Preventing School Failure, 36 (1), 47-49. Human Rights and Equal Opportunity Commission (1993). An Act Against Disability Discrimination: The Federal Disability Discrimination Act. Sydney, NSW: Human Rights Australia. Jenkinson, J. C. (1997). Mainstream or special? Educating students with disabilities. Padstow, Cornwall: T.J. Press (Padstow) Ltd. Kemp, C. (2003). Mainstream education for children with intellectual disabilities: a moral right. Macquarie University News. [On-line]. www.pr.mq.edu.au/ macnews/Show Item.asp? ItemID = 121 Lynch, J. (1994). Provision for Children with Special Educational Needs in the Asia Region. Retrieved December 13, 2002 World Wide Web: http:// inclusion.uwe.ac.uk/csie/ sensaia.htm Masters, G.N. (1997). Partial credit model. In John P. Keeves (ed.), Educational Research, Methodology and Measurement (2nd ed.) (pp.857-863). Oxford, UK: Elsevier Science New Zealand Disability Strategy, (2001). Making a World of Difference: Social Policy. Wellington, New Zealand: DPA. Noland, E.N., McLaughlin, T.F., Howard, V. F., & Sweeny, W.J. (1993). Peer attitudes toward students with disabilities: A comparison of the in-class pull-out models of service delivery. British Columbia Journal of Special Education. 17 (3), 210-217. Prentice, M. (2002). Serving students with disabilities at the community college. ERIC Clearinghouse for Community Colleges. Retrieved November 12, 2002, from WWW.gseis.ucla.edu/ERIC/digests/dig0202.htm. Rasch, G. (1960/1980/1992). Probabilistic models for some intelligence and attainment tests (Expanded edition). Chicago, IL: MESA Press (original work published in 1960). Reber, C.K. (1995). Attitudes of preservice teachers toward students with disabilities: Do practicum experiences make a difference? Retrieved June 9, 2001, from E-Subscribe, EDRS database (ED 390 825) on the World Wide Web: http://www.askeric.org /Eric Saravanabhavan, S. & Saravanabhavan, R. C. (2001). Attitudes toward disabilities across cultures. Educational Practice and Theory, Vol.23, No. 2, 2001, pp. 49-60. Singh, A. N. (2001). Enabling the differently able: Overview of policies for the disabled. Vikas Marg, Shakarpur, New Delhi: Shipra Publishers. Retrieved on April 24, 2003, from The Hindu: On-line edition of India‟s national newspaper published on Tuesday, October 09, 2001 from www.google.com Tait, K. & Purdie, N. (2000). Attitude toward Disability: teacher education for inclusive environments in an Australian University. International Journal of Disability, Development and Education, 47(1), 25-38.
224
Minoti Biswas and Russell Waugh
Treolar, L.L. (1999). Editor‟s Choice: Lessons on disability and rights of students. Community College Review, 27 (1), 30-40, (ERIC Document Reproduction No. EJ590042). Tyrer, A. (2002). The Human Rights Act in education. [On-line]. Retrieved December 2, 2002 from WWW: http://www.atyrer.demon.co.uk/stammer/dda/ed_post16.htm Ward, J., Center, Y. & Bochner, S. (1994). A question of attitudes: integrating children with disabilities into regular classrooms? British Journal of Special Education, 21(1), 34-38. Waugh, R.F. (2003). On the forefront of educational psychology. New York: Nova Science Publishers. Waugh, R.F. (2005). Frontiers in educational psychology. New York: Nova Science Publishers. Waugh, R. F. & Biswas, M. (2003). University students‟ acceptance of peers with disabilities: A Rasch measurement. In: On the Forefront of Educational Psychology (R. F. Waugh, Ed.) pp. 157-176). New York: Nova Science Publishers. Wright, B.D. (1999). Fundamental measurement for psychology. In: The new rules of measurement (S.E. Embretson & S.L. Hershberger, Eds., pp.65-104). Mahwah, NJ: Lawrence Erlbaum Associates
INDEX Ahdielah Edries, xiii,23,50 Attitude and Behavior to Mathematics Scale (Singapore Data), 190 Australian Islamic College, 24-27 Teacher Views, 50-55 Biswas, Minoti, xiii,205 Caring Thinking re Mathematics in Singapore, 101-103, 106-107 Choe Kee Cheng xiii,1 Construct Validity of Variables, Linking Attitude and Behavior, Comparing Predicted Item Difficuties to Actual Rasch measured Item Difficulties For Project Work, 6-10 For Self-Views of Sport, 37 For Self-Views of Drama, 37 For Self-Views of Music, 37 For Self-Discipline in Mathematics, 127 For Moderation in Mathematics, 128 For Dependability in Mathematics, 148-149 For Responsibility in Mathematics, 149-150 For Student Entrepreneurial Mindset, 175 For Attitude and Behavior to Mathematics, 190 For Student Receptivity to Peers with Disabilities, 215-216 Dependability in Mathematics Scale, 148-149 Dimensionality of Variables, 31, 67, 86, 110, 187-188, 213 Disability in Higher Education, 206-209 Disability Policy Issues from Rasch Measures, 220 Figure Group Letters and Words in Scale, 77 Figure Group Letters in Words Targeting, 89 Figure Ground Numbers in Calculations Scale, 95 Gardner Intelligences, 29-30 Self-Concept of Sports Measure, 37, 34-35 Self-Concept of Drama measure, 37, 35 Self-Concept of Music Measure, 37, 36 Guttman Scales, 50-55 Priority Activities Providing Links to the Western Culture, 51-52 General Types of Resources Needed, 52-53
226
Minoti Biswas and Russell Waugh
Guttman Scale, School Needs in Professional Areas, 54-55 Konza, Deslea, xiv, 65,83 Letters in Numbers Scale (Form Constancy), 76 Liu, Shiueh Ling, xiii,101,132 Mathematics Attitude and Behavior in Singapore, 190 Mathematics Education in Singapore, 103-104, 181-182 Page 226 Minoti Biswas and Russell F. Waugh Mathematics Education (Global Scan), 102-103 Moderation in Mathematics Scale, 149-150 Non-Fitting Items – Why Items Don‟t Fit the Rasch Model, 16-17,77-78,96, 126, 148, 163 Number Discrimination and Number Reversal Scale, 95 Targeting, 89 Project Work in Singapore, 2-3 Collaboration, 9 Communication, 9-10 Goal Management, 7-8 Item Discrimination by Gender, 13 Knowledge Application, 9 Learning Styles, 8 Self-Management, 8 Student Receptivity to Project Work Scale, 7-10 Receptivity of Students to Peers with Disabilities, 215-216,217-219 Importance of Research on Disability, 208-209 Cross-Cultural Scale (India and Western Australia), 215-216 Differences in Receptivity to Disabilities (India and Western Australia), 220-221 Differences by University, 220-222 Rasch Unidimensional Measurement Model (RUMM2020) computer program. The RUMM (2020) program was used to analyse the data reported in every chapter except Chapter 3. Rasch Measurement is the basis for the RUMM computer program. Richmond, Janet, xiv, 65, 83 Self-Discipline in Mathematics Scale, 127 Student Entrepreneurial Mindset (SEM), 153, 154-157, 175 Student Entrepreneurial Mindset (SEM) Aspects, 172, 173-174 Targeting, 167 Unnithan, Radha Devi, xiv,181 Visual Discrimination of Numbers Scale, 95 Wong, Heng Aik Jason, xiv, 153