Training Manual on Disability Statistics

WHO/ESCAP Training Manual on Disability Statistics

This publication was jointly prepared by Department of Measurement and Health Information Systems World Health Organization (WHO) Statistics Division United Nations Economic and Social Commission for Asia and the Pacific (ESCAP) United Nations Statistical Institute for Asia and the Pacific Subsidiary body of United Nations Economic and Social Commission for Asia and the Pacific (ESCAP)

The views expressed in the Training Manual are those of the authors and do not necessarily reflect the views and policies of the United Nations or the World Health Organization. The Training Manual has been issued without formal editing.

Copyright © United Nations 2008 Printed in Bangkok ST/ESCAP/2499 Sales No. E.08.II.F.16 ISBN: 978-92-1-120548-0 For further information on this publication, please address your enquiries to:

In Bangkok

In Geneva

In Tokyo

Director Statistics Division Economic and Social Commission for Asia and the Pacific United Nations Building Rajadamnern Nok Avenue Bangkok 10200 Thailand Fax: +66 2 288 1082 Email: [email protected] URL: http://www.unescap.org

Director Department of Measurement and Health Information Systems World Health Organization 20, Ave. Appia CH-1211 Geneva 27 Switzerland Fax: +41 22 791 1584 Email: [email protected] URL: http://www.who.int/

Director Statistical Institute for Asia and the Pacific JETRO-IDE Building 2-2 Wakaba 3-chome Mihama-ku Chiba-shi Chiba 261-8787 Japan

-i-

Fax: +81 43 299 9780 Email: [email protected] URL: http://www.unsiap.or.jp

WHO/ESCAP Training Manual on Disability Statistics

Preface “To make people count, we have to count people right". This notion is of particular relevance for the area of disability statistics, which until today is challenged by a substantial lack of accurate and comparable data. Few countries in the Asia-Pacific region collect information on disability regularly through censuses or surveys. In countries where information is collected, it is often done without consideration of international standards or focuses on only disability in terms of predefined categories of impairments (e.g. people who are blind, deaf, paraplegic or intellectually disabled). As a result, many countries in the Asia Pacific region have very limited and often non-comparable information about the extent of disability and the lived experience of person with disabilities and their needs. The demand for more standardized disability statistics in the Asia-Pacific region has increased significantly over the years. The Biwako Millennium Framework for Action towards an Inclusive, Barrier-Free and Rights-Based Society for Persons with Disabilities in Asia and the Pacific (BMF) and its supplement, the Biwako Plus Five urges governments to develop national disability data systems using international standards and methodologies and revisit their current definitions of disability. The recently adopted the Convention on the Rights of Persons with Disabilities and its Optional Protocol (CRPD) set the tone, at the global scale, of the broader approach to disability, which emphasizes the affect of environmental and attitudinal barriers on disability. CRPD calls for appropriate data collection on disability which would give effect to its implementation. In response to these needs, the World Health Organization (WHO) and the United Nations Economic and Social Commission for Asia and the Pacific (ESCAP) have taken initiatives and implemented a joint project on improving disability statistics and measurement in the region by promoting a common definition and methodology based on the International Classification of Functioning, Disability and Health (ICF). It is against this background that this publication was prepared. This training manual intends to enhance the understanding of the ICF-based approach to disability measurement. It provides an overview of the ICF framework as well as guidelines on how to operationalize the underlying concepts of functioning and disability into data collection, dissemination and analysis. This publication will benefit not only statisticians but also a wider range of national and international users of data on disability. This broader influence will result from the wide applicability of the standards, methodologies and best practices covered by the manual. It is hoped, that the manual will assist in stimulating more disability data collection in accordance with international standards and data dissemination for both national and international disability policy analysis, formulation and evaluation worldwide.

March 2008

- ii -

WHO/ESCAP Training Manual on Disability Statistics

Acknowledgement The Training Manual on Disability Statistics is a joint product of the World Health Organization (WHO) and the United Nations Economic and Social Commission for Asia and the Pacific (ESCAP). The team that provided guidance for the Manual includes Tomas Africa, Davaasuren Chultemjamts, Andrew Flatt, Haishan Fu, Pietro Gennari, Lene Mikkelsen, and Tevfik Bedirhan Ustun. The Manual was drafted by Joanne Hillerman and Nenad Kostanjsek with substantive inputs from Aiko Akiyama, Ken Black, Wei Liu and Jan Smit. It was edited by Jerome Bickenbach, Daniel Clarke, Sarah Perini, and Mercedita Tia with assistance from Chol Han, Krisana Boonpriroje, Julie Gotoh, Andres Montes, Nixie Mabanag, Varaphorn Prapatsakdi, and Prapapan Treeratpituk. WHO and ESCAP gratefully acknowledge valuable inputs received from national and international experts who participated in the series of workshops organized under the ESCAP/WHO project: Ziaullah Zia of Afghanistan; Marilyn Linggi Teo Lai of Brunei Darussalam; Seng Soeurn and Tith Vong of Cambodia; Xiao Li and Qiu Zhuoying of China; Timoci Bainimarama and Serevi Baledrokadroka of Fiji; Wai Fun Siu of Hong Kong, China; Jogeswar Dash and Inder Jeet Singh of India; Arizal Ahnaf and Tati Suryati of Indonesia; Seyedeh Susan Rafie and Samaneh Qaderi of Islamic Republic of Iran; Bounloy Sengkhamyong of Lao PDR; Dick Wah Yau and Anna Yee Ling Chan of Macao, China; Undrakhbayar Sundui and Tserenkhand Bideriya of Mongolia; Guillermo Jr. Lipio of the Philippines; Albert Bengkiat Tan of Singapore; Haputantrige Rathnasiri Gunasekera and Warahena Liyanage Don Pathman de Alwis Goonatilleke of Sri Lanka; Pattariya Jarutat, Ubonwon Wathanadilokul, Orasri Hintamai and Arpaporn Amonthum of Thailand; Nguyen Phong and Chung Nguyen Dinh of Viet Nam; Jeremiah Banda of UNSD; Jennifer Madans and Maguerite Schneider of Washington Group; Razikhan Hamdard Razikhan of UNDP; Ghazy Mujahid of UNFPA; Jean-Francois Trani of Handicap-International Afghanistan; and Betsy Van Leit of Handicap-International Cambodia. The Manual benefited from the experiences of the following countries, which participated in the pilot tests of ICF-based questionnaires in 2005 developed by WHO and the Washington Group: Fiji, Indonesia, India, Mongolia and the Philippines. The Government of the Republic of Korea generously provided funding for the ESCAP/WHO project through which the Manual was produced. The Australian Bureau of Statistics provided valuable technical support throughout the project. Photos in the Manual were kindly provided by the WHO Photo Contest “Images of Health and Disability 2002, 2003, 2004”, and the Asia-Pacific Development Center on Disability (APCD).

- iii -

WHO/ESCAP Training Manual on Disability Statistics

List of Acronyms ABS ADLs IADLs BMF

BMI CAGD CRPD CSTDA DISTAT ESCAP FIC IADLs ICD–10 ICF ICIDH NMDS NGOs Standard Rules SDAC UN WG WHO WHO DAS II WHO-FIC YLD

Australian Bureau of Statistics Activities of Daily Living Instrumental Activities of Daily Living Biwako Millennium Framework for Action towards an Inclusive, Barrier-free and Rights-based Society for Persons with Disabilities in Asia and the Pacific Body Mass Index Census Advisory Group on Disability Convention on the Rights of Persons with Disabilities and its Optional Protocol Commonwealth/State/Territory Disability Agreement Disability Statistics Economic and Social Commission for the Asia Pacific Family of International Classifications Instrumental Activities of Daily Living International Statistical Classification of Diseases and Related Health Conditions International Classification of Functioning, Disability and Health International Classification of Impairments, Disabilities and Handicaps National Minimum Data Set Non-Governmental Organizations 1993 Standard Rules on the Equalization of Opportunities for Persons with Disabilities Survey of Disability, Ageing and Carers United Nations Washington Group World Health Organization WHO Disability Assessment Schedule II WHO Family of International Classifications Years of Life lived with Disability

- iv -

WHO/ESCAP Training Manual on Disability Statistics

TABLE OF CONTENTS Preface........................................................................................................................................ ii Acknowledgement.....................................................................................................................iii List of Acronyms....................................................................................................................... iv 1. DISABILITY STATISTICS: WHY ARE THEY IMPORTANT AND WHAT DOES ICF ADD? .................................................................................................................................. 2 1.1 Introduction ............................................................................................................... 2 1.2 ICF and its purposes in disability and health statistics ...................................... 5 1.3 Purpose and outline of the manual ..................................................................... 10 2. THE ICF FRAMEWORK................................................................................................ 13 2.1 The ICF model and conceptual framework: Defining features ....................... 13 2.2 Importance of classifications and standards ..................................................... 25 2.3 ICF terminology and definitions of disability...................................................... 26 2.4 ICF structure and coding – for population data collections............................. 28 2.5 ICF training and assistance available................................................................. 32 3. DATA COLLECTION METHODS AND INSTRUMENTS........................................... 34 3.1 Measurement of disability..................................................................................... 34 3.2 Population censuses ............................................................................................. 36 3.3 Sample surveys ..................................................................................................... 37 3.4 Administrative collections ..................................................................................... 40 3.5 Joint use of census and sample survey – census screen............................... 43 3.6 Strengths and weaknesses of the different data collection methods ............ 45 3.7 Relatable data ........................................................................................................ 49 4. USING ICF IN SURVEY AND CENSUS DESIGN................................................... 52 4.1 Operationalization of ICF in disability data collections .................................... 53 4.2 Key Issues .............................................................................................................. 54 4.3 Approaches to measurement............................................................................... 61 4.4 Question formats and response scales.............................................................. 73 4.5 Additional issues in question development ....................................................... 75 4.6 Census format questions...................................................................................... 80 4.7 International disability question sets................................................................... 83 4.8 Administrative data collections, with an Australian example .......................... 89 5. SURVEY/CENSUS AND ADMINISTRATIVE-BASED DATA COLLECTION DEVELOPMENT: PLANNING AND MEASUREMENT ISSUES ...................................... 93 5.1 Overview of the survey and census cycles ....................................................... 94 5.2 Planning for data collection.................................................................................. 96 5.3 Asking the right questions .................................................................................... 96 5.4 Determining user needs ....................................................................................... 98 5.5 Privacy implications of collecting data.............................................................. 101 5.6 Other basic management issues in planning .................................................. 102

-v-

WHO/ESCAP Training Manual on Disability Statistics

5.7 Quality of survey data and total survey error .................................................. 103 5.8 Issues in collection design ................................................................................. 105 5.9 Planning for administrative-based data collection.......................................... 110 5.10 Sample design ..................................................................................................... 112 6. INSTRUMENT DEVELOPMENT............................................................................... 117 6.1 Attributes of good questions .............................................................................. 117 6.2 Testing and validation of data collection instruments .................................... 118 6.3 Data collection guide........................................................................................... 126 7. ORGANIZING THE FIELDWORK ............................................................................. 129 7.1 Preparatory activities prior to conducting the fieldwork ................................. 129 7.2 Guidelines for data collection............................................................................. 133 8. PROCESSING THE DATA........................................................................................... 141 8.1 Processing procedures ....................................................................................... 141 8.2 Processing flow.................................................................................................... 145 9. ANALYSING AND DISSEMINATING DISABILITY STATISTICS........................ 149 9.1 Planning the outputs ........................................................................................... 149 9.2 Analysis – Turning data into information.......................................................... 157 9.3 Forms of output and dissemination strategies ................................................ 161 9.4 Ensuring secure data or accomplished questionnaires storage .................. 164 9.5 Documentation ..................................................................................................... 165 10. DISABILITY STATISTICS: KEY TO SOUND POLICY FORMULATION ......... 168 10.1 Policies that can be formed out of the given disability data.......................... 168 10.2 Disability guidelines for policy formulation....................................................... 170 BIBLIOGRAPHY .................................................................................................................. 173 APPENDICES........................................................................................................................ 174 Appendix 1: Selected Countries on Disability Data Collection and Methodology . 175 Appendix 2: Questionnaire Used in the Pilot Test in Five Selected Countries...... 183 Appendix 3: Sample Data Collections.......................................................................... 195 Appendix 4: Biwako Millennium Framework for Action Targets............................... 227 Appendix 5: Biwako Plus Five ....................................................................................... 229

- vi -

WHO/ESCAP Training Manual on Disability Statistics

List of Tables

Table No.

Page

Table 2.1: ICF Levels of Functioning and Disability ...............................................................15 Table 2.2: The ICF Domain Hierarchy....................................................................................29 Table 2.3: ICF Chapters .........................................................................................................30 Table 2.4: WHO Recommended ICF Item Pool .....................................................................31 Table 3.1: Population Censuses: Advantages and Disadvantages........................................45 Table 3.2: Sample Surveys: Advantages and Disadvantages ...............................................46 Table 3.3: One-time Registration Data Collection: Advantages and Disadvantages .............47 Table 3.4: Continuous Registration Data Collection: Advantages and Disadvantages..........47 Table 3.5: Data Collection Methods for Disability Statistics: Comparative Summary ............48 Table 4.1: Value of First Qualifiers in the ICF ........................................................................74 Table 4.2: Negative Terms Relating to Disability ...................................................................77 Table 4.3: Example of WG and WHO/ESCAP question sets .................................................85 Table 5.1: Face-to-face and Telephone Interviewing: Differences in Prevalence ................107 Table 5.2: Sample Sizes of Some Disability Surveys...........................................................115 Table 6.1: Attributes of Pre-testing Techniques ...................................................................122 Table 6.2: Comparison of ‘Gold Standard’ and Survey Question Responses......................125 Table 6.3: Comparison of Disability Census Items and ‘Gold Standard’ Survey Items ........126

- vii -

WHO/ESCAP Training Manual on Disability Statistics

List of Boxes

Box No.

Page

Box 3.1: 2001 Short Form/Long Form Census in Macau.......................................... 37 Box 3.2: Dedicated Sample Survey for Disability: Australian 2003 Survey of Disability .......................................................................................................................... 38 Box 3.3: Disability Module in the Indonesian Socio Economy Survey ...................... 39 Box 3.4: An Overview of the Australian Disability Services Data Collection ............. 42 Box 3.5: Use of the Post-Census Survey Approach to Collect Disability Data in Canada.............................................................................................................. 44 Box 3.6: Disability Data Collection Methods Practiced in Selected Countries .......... 45 Box 3.7: Using Data from a Variety of Sources: The Australian Disability Services Project ............................................................................................................... 49 Box 4.1: Australian Experience with Disability Census Questions............................ 54 Box 4.2: ICF Items in Australian and Canadian Measures of Disability .................... 55 Box 4.3: Inclusion of People Living in Institutions in Disability Surveys in New Zealand ............................................................................................................. 57 Box 4.4: Disability Data Collections Approach Employed in Selected Countries ...... 66 Box 4.5: Case Study of Middle Earth: Developing Disability Measures .................... 88 Box 4.6: Australian National Minimum Dataset: Data Collection Form ..................... 91 Box 5.1: Australian Census Advisory Group on Disability ........................................ 99 Box 5.2: Consultation with Disability Groups: Examples ........................................ 101 Box 5.3: The Context in Which Disability Questions are Asked.............................. 105 Box 9.1: Summary of Disability Data in Australia, 2003.......................................... 153 Box 9.2: Age Profile of Persons with disabilities: Two Diagrams ............................ 154 Box 9.3: Disability Rates by Region, Philippines: 2000 .......................................... 156 Box 9.4: Explanatory Materials for Survey Analysis Presentation: A Guide............ 162 Box 9.5: Disability Data Statistical Releases: Some recent examples.................... 165

- viii -

WHO/ESCAP Training Manual on Disability Statistics

List of Diagrams Diagram No.

Page

Diagram 2.1: Core Impairment vs. Impairment Continuum ....................................................16 Diagram 2.2: Different Health Conditions with the Same Participation Restriction ................18 Diagram 2.3: Seeing Functions as a Continuum....................................................................20 Diagram 2.4: Overall Disability Score by country ...................................................................22 Diagram 2.5: Interactions between the Components of ICF ..................................................23 Diagram 2.6: ICF – a combination of different vectors of functioning.....................................23 Diagram 4.1: Mexican Disability Evaluation Survey, 2003 .....................................................72 Diagram 4.2: Response Scales..............................................................................................76 Diagram 5.1: The Survey Cycle .............................................................................................94 Diagram 5.2: The Census Cycle ............................................................................................95 Diagram 5.3: Sources of non-sampling error in survey/census data....................................104 Diagram 5.4: Disability rates by age and sex, Australia, 2003 .............................................109 Diagram 8.1: Flow of processing survey/census/administrative – based results .................146

List of Charts Chart No.

Page

Chart 1.1: Prevalence of Disabled Persons in Asia-Pacific Countries .....................................7

- ix -

WHO/ESCAP Training Manual on Disability Statistics

This page intentionally left blank

-x-

WHO/ESCAP Training Manual on Disability Statistics

Learning Objectives: Chapter 1 Disability Statistics: Why Are They Important and What Does the ICF Add? After reading this chapter, the reader should be able to: 1.

Understand the nature and importance of disability statistics

2.

Understand the aims and purposes of disability statistics

3.

Recognise the role of ICF in international disability statistics

“Good vision” Natalie Behring-Chisholm WHO Photo Contest “Images of Health and Disability 2003/2004”

-1-

WHO/ESCAP Training Manual on Disability Statistics

1.

1.1

DISABILITY STATISTICS: WHY ARE THEY IMPORTANT AND WHAT DOES ICF ADD?

1.1 1.1.1 1.1.2 1.1.3 1.1.4

Introduction What is disability statistics? The importance of disability statistics Purposes of disability statistics Purposes: International and regional perspectives

1.2 1.2.1 1.2.2 1.2.3 1.2.4

ICF and its purposes in disability and health statistics ICF and the ESCAP Region ICF and the purposes of disability statistics ICF: Concept of disability ICF: International language of disability

1.3

Purpose and outline of the manual

Introduction

Compared to other areas of statistics such as labour force, education, women, and older people, disability statistics has not been well developed or utilized. It is only very recently that governments have begun to recognize the urgency and importance of such statistics for better policies and program developments. Both global and regional instruments on disability stress the importance of appropriate data collection. World Programme of Action Concerning Disabled Persons (1982), Standard Rules on the Equalization of Opportunities for Persons with Disabilities ( 1993) and the landmark disability-specific human rights treaty, the Convention on the Rights of Persons with Disabilities and its Optional Protocol adopted in 2006 state a need of appropriate information collection, which would protect and promote and fulfil all human rights of persons with disabilities. Biwako Millennium Framework for Action Towards an Inclusive, Barrier-free and Rightsbased Society for Persons with Disabilities in Asia and the Pacific (BMF), the regional policy guideline for the Second Asian and Pacific Decade of Disabled Persons, and the Biwako Plus Five, the supplement to the BMF, both stress the need to develop valid, reliable, and internationally comparable disability statistics in order to put into place effective disability policies and projects. 1.1.1 What is disability statistics? Traditionally, disability statistics has been conceived as a matter of counting people who fall into specific groups – 'the blind', 'the deaf', 'wheelchair users' – in order to determine who qualifies for benefits. With very limited purpose in mind, this categorical approach gives a fragmented and distorted picture of disability since it suggests that person with disability fall neatly into a few categories with clear boundaries.

-2-

WHO/ESCAP Training Manual on Disability Statistics

However, disability statistics can provide a wealth of information on the full lived experience of persons with disabilities ranging from impairments, difficulties in undertaking and participating in activities, and barriers they face in their lives. Information can be extended from an individual to the whole population – to determine prevalence of domains of disability, for example – and further developed by adding demographic or other population features, such as age, sex, race, and socio-economic status. 1.1.2 The importance of disability statistics With a broader understanding of disability, disability statistics can play a pivotal role in all areas of policy-making, and in each every stage from development and implementation, to monitoring and assessment of effectiveness, to the analysis of cost-effectiveness. Policy without valid and reliable data is potentially costly and wasteful guesswork; it is policy without a basis of evidence and good science. Invalid or incomplete disability data, of the kind often found in the developing world, can be worse than no data at all. Below are some specific reasons why national disability statistics and valid disability databases are essential for national policy: ¾ The aspirations of the UN Disability Convention, Standard Rules and the BMF, Biwako Plus Five for the protection and promotion of the rights and the dignity of persons with disabilities are mere hopeful words without sound data to monitor and assess progress towards these goals. ¾ Information about functional status is integral to identify needs since two individuals with the same impairment may face different types of difficulties in undertaking certain activities, and so have different needs that require different kinds of interventions. ¾ Functional status data is essential for determining the broader social needs of persons with disabilities, such as provision of assistive technology for use in employment or education or broader policy and laws. ¾ Population disability data is essential for monitoring the quality and outcomes of policies for persons with disabilities. In particular, these data help to identify policy outcomes that maximize the participation of persons with disabilities in all areas of social life from transportation and communication, to participation in religious and community life. Finally, with valid and complete disability statistics, state agencies will have the tools for assessing the cost-effectiveness of policies for persons with disabilities, which in turn can provide the evidence to persuade governments of their ultimate benefit for all citizens.

-3-

WHO/ESCAP Training Manual on Disability Statistics

1.1.3 Purposes of disability statistics A number of countries in the Asia Pacific Region have conducted disability surveys or included disability questions in their censuses, but users were not satisfied with the results. The most common complaint was that the statistics were based on a handful of severe impairments and did not capture broader measures of disability. This is an example of a failure by data collection agencies to understand the needs of those who will be using the data. The lesson is obvious: design your tools to collect data that are 'fit for purpose'. Failure to match disability questions with purposes results in unusable statistics that becomes a waste of public expenditure. Of course, clearly identifying the purposes for disability statistics is not an easy task. In Chapter 5, we will look at a process for determining data user needs, one that has been successfully used in the Australian context. It is first important to note, though, that considerable work, international and regional, has been done in identifying needs that quality disability data can serve. 1.1.4 Purposes: International and regional perspectives In the abstract, the question ‘What is the purpose of disability statistics?’ seems daunting. Fortunately, at the international and regional levels there are clear statements of purpose that we can turn to. Internationally, explicit recommendations for the use of disability statistics are contained in three important UN documents: ™ The World Programme of Action Concerning Disabled Persons (1982) ™ Standard Rules for the Equalization of Opportunities of Persons with Disabilities (1993) ™ The Convention on the Rights of Persons with Disabilities and its Optional Protocol (2006) In 2001, the Washington Group on Disability Statistics was formed as a response to a suggestion made during a UN International Seminar on Measurement of Disability. Since then the Group has brought together international representatives to work on important methodological issues for improving disability data and, in particular, to facilitate the comparability of data around the globe. The Group’s primary task is to come up with general disability questions that can be used in censuses and national surveys. Along the way, the Group has offered guidance on what it took to be the overriding purposes of disability statistics: 1. To assist in the development and evaluation of programmes and policies for service provision; 2. To monitor the level of functioning in the population; and 3. To assess equalization of opportunities.

-4-

WHO/ESCAP Training Manual on Disability Statistics

Regionally, a major statement of the important purposes of disability statistics is provided by the BMF (2002). It outlines the following seven priority areas for governments in Asia and the Pacific: 1. Self-help organizations of persons with disabilities and related family and parent associations, 2. Women with disabilities, 3. Early detection, early intervention and education, 4. Training and employment, including self-employment, 5. Access to built environment and public transportation, 6. Access to information and communication, including information, communication and assistive technologies, and 7. Poverty alleviation through capacity-building, social security and sustainable livelihood programmes Each of these areas has been neglected in part because of a lack of adequate data on persons with disabilities. BMF and Biwako Plus Five, the supplement to the BMF, both have a strategy on disability data collection, emphasizing a urgent need to build capacities of governments, NGOs and other stakeholders to effectively collect data on disability. At its September 2003 meeting, ESCAP held a Workshop on Improving Disability Statistics for Policy Needs. The group reaffirmed the priority areas in BMF and identified specific areas where quality disability data is needed: service programming and development, capacity building, budgeting, and seeking international assistance, among many others. These international and regional documents report a consensus among nations of the world and within Asia and the Pacific, about what is required of a better way of gathering information on disability data. Both groups agreed that there is a need for a scientific basis and valid conceptual framework in establishing disability statistics. The World Health Organization's (WHO) International Classification of Functioning, Disability, and Health (ICF) is response to this need. The ICF has truly created a new paradigm for disability statistics. 1.2

ICF and its purposes in disability and health statistics

The ICF is both a classification system and a model of the complete experience of disability. The ICF is a member of WHO's family of health classifications, the other prominent member of which is the International Statistical Classification of Diseases and Related Health Conditions (ICD-10). As a classification system, ICF provides a common language, which guarantees the comparability of disability data between sectors within a country, and between countries. As a model of disability, the ICF offers a conceptual framework for structuring disability data.

-5-

WHO/ESCAP Training Manual on Disability Statistics

1.2.1 ICF and the ESCAP Region In Asia and the Pacific, the ICF has been implemented in Australia and New Zealand. Indonesia and Thailand have also begun to use its concepts in some of their disability data collections. The experience of integrating the ICF concepts into disability data collections has been that disability phenomenon has been described with greater clarity and precision. Yet for the majority of the countries in the ESCAP region, disability statistics are unreliable. Many of these countries report a prevalence of disability at less than 3 percent, which is implausibly low. More worrisome, the UN Disability Statistics database (DISTAT2) shows huge differences in the prevalence of disability across the countries of the region (see Chart 1.1). Disability prevalence ranges from less than 1 percent for Singapore and Lao PDR, to a high of about 19 percent for Australia and New Zealand.1 It is very hard to believe that these differences in prevalence are based in reality. It is more likely that they result from using different definitions of disability. The data for the different countries cannot be compared because they do not follow the same concept. Without reliable demographic profiles of persons with disabilities, where they live and what problems they experience, the prospects for developing and improving national plans and disability policies remain poor. Furthermore, without basic estimates of prevalence in terms that are comparable across the ESCAP region, it is difficult to see how the goals in the BMF and the Biwako Plus Five can be monitored, let alone achieved. This issue of non-comparable standards applied in the measurement and collection of disability information is precisely why implementation of the ICF is so important.

1

Responses to the ESCAP survey on the mid-point review of the BMF implementation revealed data are on the proportion of persons with disabilities from 31 Governments in the region. The proportion ranges from a low of 0.7 per cent in the Cook Islands to a high of 20 per cent in Australia, but in the majority of cases, it is below 5 per cent. Please see E/ESCAP/APDDP(2)/1 for more details.

-6-

WHO/ESCAP Training Manual on Disability Statistics

Chart 1.1: Prevalence of Persons with Disabilities in Asian and Pacific Countries

Prevalence of persons with disabilities in Asian and Pacific Countries (%) 25% 20% 15% 10% 5% 0%

1985 Admin. 1981 Census 1982 Survey 2000 Census 1995 Census 2001 Survey 1986 Survey 2001 Census 2002 Survey 2000 Survey 2005 Survey 1987 Survey 2002 Survey 2001 Survey 2003 Survey Records Singapore

Pakistan

Bangladesh

Philippines

Philippines

Thailand

Sri Lanka

India

Turkey

Hong Kong

Korea (Republic of)

China

Sources: United Nations Demographic Yearbook System (Nov.2006) and United Nations Disability Statistics Database (DISTAT)

-7-

Japan

New Zealand

Australia

WHO/ESCAP Training Manual on Disability Statistics

1.2.2 ICF and the purposes of disability statistics How can the ICF help with the task of identifying the purposes of disability statistics and determining the needs of data users? The details of the ICF will be the subject of the next chapter, but the role of the ICF in identifying purposes and determining data needs can be quickly summarised. The ICF helps because it is both a model of the concept of disability and an international language of disability: 1.2.3 ICF: Concept of disability At the core of ICF’s concept of disability are the facts that disability is multidimensional and the product of an interaction between an individual’s certain conditions and his or her physical, social, and attitudinal barriers. The bio-psychosocial model embedded in the ICF broadens the perspective of disability and allows medical, individual, social, and environmental influences on functioning and disability to be examined. Structurally, the ICF is based on three levels of functioning (body functions and structures, activities, and participation) with parallel levels of disability (impairments, activity limitations, and participation restrictions). Human functioning is understood as a continuum of health states and every human being exhibits one or another degree of functioning in each domain, at the body, person and society levels. In the ICF language, contextual factors (environmental factors and personal factors) also constitute disability. Environmental factors include availability of assistive devices, family and community support, supportive services and policies and attitudes of different people. Personal factors include health conditions (diseases, disorders and injuries). ICF conceptualizes disability, not solely as a problem that resides in the individual, but as a health experience that occurs in a context. The ICF tells us that any discussion of the purpose of disability statistics must begin with the question: Which dimension of disability do you want to collect data on? Data about impairments (problems at the level of organs and anatomical structures) is different from data about activity limitations (limitations on the capacity of a person to act or behave in a desired manner, because of a health condition), which again is different from data about participation restrictions (limitations in what a person does that result from an interaction between impairments or activity limitations and barriers created by the person’s environment). There are legitimate and vitally important needs served by all three kinds of data. However: Different purposes require different kinds of disability data ¾ If your purpose is to collect information about the prevalence of physiological or psychological functions such as mental functions, -8-

WHO/ESCAP Training Manual on Disability Statistics

perceptual functions, voice and speech function, functions of the organ systems, and so on, then you will need impairment data. ¾ If your purpose is to collect information about the capacity of a population to perform daily activities such as mobility, communication, self-care, and interpersonal relations, then you will need data about activity limitations. ¾ If your purpose is to collect information about what members of a population actually are able to do in their lives, and, in particular, what features of their physical, built, interpersonal, or social environment help or hinder them, then you need data about participation restrictions. Although it is theoretically possible to separate activity limitation information from participation restriction information, for all practical purposes, it is essential that both sets of data are collected. Otherwise, a much distorted picture of disability is presented, one that ignores the role of environmental factors in the actual lived experienced of a person with health decrements that affect capacity. For this reason, although it adds another level of information that needs to be collected, a realistic measurement of disability across a population requires the collection of information about environment. This information takes the form either of facilitators (assistive technology, accommodations to the built environment, such as curb ramps or accessible housing or transportation, or accommodations in law or social policy) or barriers (unaccommodating physical or built environments, or stereotypical and stigmatizing attitudes). For data about environmental factors, ICF is the essential instrument. Because the ICF clearly and systematically separates these dimensions of disability, it is the ideal tool for structuring and organizing any discussion of the purposes of disability statistics or the needs that a disability statistics can serve. The details of how the conception of disability that underlies the ICF makes it possible to coherently develop any questions that are ‘fit for purpose’ for any data collection activity (survey, census, or administrative collection) will be discussed in Chapter 4. 1.2.4 ICF: International language of disability The ICF is a set of classifications of the dimensions of disability phenomena and environmental factors. With their hierarchical arrangement, operational definitions of each category, and coding structure, these classifications together form an international common language of disability. Whatever purpose data users seek to achieve with a survey or other tool, that purpose is greatly enhanced by its international comparability of data.

-9-

WHO/ESCAP Training Manual on Disability Statistics

The absence of a common language of disability, including a common understanding of the multidimensional concept of disability, is the principal cause of the lack of agreement on disability data around the globe. The primary aim of the ICF and WHO’s motivation to engage in a decade-long, international collaborative venture to revise the 1980 version of the International Classification of Impairments, Disabilities and Handicaps (ICIDH) was to realize the dream of valid, reliable, and internationally comparable disability statistics. By using the common language of the ICF, every country in Asia and the Pacific will be able to benefit from the integration of disability statistics into a common worldwide data collection. 1.3

Purpose and outline of the manual

This new manual on disability statistics for the ESCAP region is needed to integrate the ICF into the process of collecting and analyzing disability statistics at the national, and eventually, international levels. ICF was endorsed by all 191 member states of the World Health Assembly in May, 2001 and WHO strongly recommends its use for all health and disability statistics worldwide. The purpose of this manual, therefore, is to provide countries and areas in the ESCAP region with a new appreciation of disability statistics in light of the framework and classification of the ICF, as well as the factors affecting the development and collection of disability data, so that national disability statistical offices can produce disability data that genuinely meets their policy needs. Another important reference material is the 2001 UN Guidelines and Principles for the Development of Disability Statistics. This provides guidelines for collecting, compiling and disseminating statistics on persons with disabilities. It includes examples from both developing and developed countries. While primarily aimed at statisticians in the collection of disability data, it is also useful to disability policy makers and program managers who develop the objectives of any data collection activity and will be the ultimate users of the resulting disability data. (Note: The UN Guidelines were published before completion of the ICF and so it does not provide much detail on applying the ICF to statistical collections.) Outline of the manual Chapter 1 introduces the basic purposes of disability statistics, and their importance for regular national statistics collections. The chapter also emphasizes the need to use the ICF as the framework for collecting disability statistics. Chapter 2 describes the basic features of the ICF in the context of disability statistics.

- 10 -

WHO/ESCAP Training Manual on Disability Statistics

Chapter 3 outlines data collection methods and instruments, including censuses, surveys and administrative collections of disability data and indicates how the ICF model can increase the usefulness of the data sets. Chapter 4 provides details of the application of ICF to data collections, an important step in operationalizing the ICF concepts of functioning and disability into statistical measurements. Chapter 5 discusses planning processes for and some measurement issues concerning quality of data from surveys, censuses and administrative-based data collection. Chapter 6 looks at instrument development techniques and testing strategies. Chapter 7 talks about the different activities that should be undertaken in preparing for data collection such as publicity campaign, coordination and others. It also guides the interviewers and supervisors on how to conduct the fieldwork and provides some guidelines on how to conduct interviews properly. Chapter 8 shows and discusses the data processing flow. This chapter is for subject matter specialists/statisticians, processors/editors, computer system analysts and programmers. Chapter 9 reviews issues on data collected, including analysing and disseminating disability data, an area sometimes overlooked in planning for statistical collections. Chapter 10 lists some examples of types of statistics particularly relevant for national policies, and describes some policies that could emanate from improved disability data. Appendix material includes a history of international mandates for disability statistics, selected case studies, a sample of the questionnaire used in pilot tests conducted in selected countries, and a guide that can be used for developing a new disability data collection.

- 11 -

WHO/ESCAP Training Manual on Disability Statistics

Learning Objectives: Chapter 2 The ICF Framework After reading this chapter, the reader should be able to:

1.

Understand ICF's model and structure

2.

Appreciate importance of standards to disability statistics

3.

Describe ICF's structure and coding for population data collection

“Counting persons with disabilities” Jerome Chatin WHO Photo Contest “Images of Health and Disability 2004/2005”

- 12 -

WHO/ESCAP Training Manual on Disability Statistics

2.

2.1

THE ICF FRAMEWORK 2.1 2.1.1 2.1.2 2.1.3 2.1.4 2.1.5 2.1.6 2.1.7 2.1.8

The ICF model and conceptual framework: Defining features Biopsychosocial Multidimensional Universal Continuous Interactive Context dependent Inclusive Neutral (language)

2.2

Importance of classifications and standards

2.3

ICF terminology and definitions of disability

2.4

ICF structure and coding – for population data collections

2.5

ICF training and available assistance

The ICF model and conceptual framework: Defining features

The WHO's ICF reflects the modern day thinking about disability and embodies a paradigm shift in the way health and disability are understood and measured. ICF is based on a bio-psychosocial model of functioning and disability, in which functioning and disability are multi-dimensional phenomena experienced at the level of the body, the person, and society. In addition, a classification of environmental factors is included that allows users of the ICF to record the positive or negative impact of the environment on a person's functioning. ICF ‘mainstreams’ the experience of disability by viewing it, not as the mark of a social minority, but as a universal human experience. By placing the capacity aspects of disability on a continuum with health, ICF makes it possible to measure health and disability with the same domains of functioning. Hence, ICF provides a common conceptual platform that links health and disability statistics. The ICF model and its underlying principles represent a significant development from its predecessor the ICIDH. In the ICIDH, disability was understood as a limitation in the person’s activity that resulted from impairment. Neither disabilities nor handicaps could be assessed in terms of degree of severity. Environmental factors were acknowledged but not classified and no linkages between disability and health status measurement were made. Due to these limitations, ICIDH was generally viewed as flawed and so was ignored by disability data users in general and advocates of the

- 13 -

WHO/ESCAP Training Manual on Disability Statistics

social model of disability (especially organizations representing persons with disabilities) in particular. In response to these and other criticisms, the ICF was developed over a seven-year period in an international collaborative process and validated by means of field trails in over 70 countries before officially being endorsed by all WHO Member States in 2001. The key, interlocking features of the ICF model are described in what follows, with particular emphasis on the value they add to disability statistics.

2.1.1 Bio-psychosocial The literature on the 'models of disability' is extensive. Two major models which have been debated over and over are the “medical model of disability” and “social model of disability.” The former regards disability only as an individual pathological problem, in need of cure, rehabilitation and adaptation to society. The latter model, though not denying medical and rehabilitative needs of persons with disabilities, draws attentions to social, physical, informational and institutional barriers, which contribute disability. For the social model, participation of persons with disabilities is regarded as an important aspect of changing a paradigm on disability. The ICF adopts a bio-psychosocial model of disability that incorporates what is true and useful in both models, and rejects what is counterproductive and distorted. Disability is a complex phenomenon that includes both a dimension at the level of a person's body, and a dimension that is a complex and primarily social phenomenon. Medical and rehabilitative interventions are perfectly relevant to the body-level aspects of disability (impairments and limitations in a person’s capacity to perform actions). By the same token, environmental and social interventions are relevant to deal with restrictions in a person's participation in educational, economic, social, cultural and political activities. For producers and users of disability statistics, the use of the bio-psychosocial model embedded in the ICF broadens the perspective of disability and allows medical, individual, social, and environmental influences on functioning and disability to be examined. Furthermore, advocates of both models can use the ICF as a platform for communication and choosing and comparing interventions. 2.1.2 Multi-dimensional Structurally, the ICF is based on three levels of functioning (body functions and structures, activities, and participation) with parallel levels of disability (impairments, activity limitations and participation restrictions). These are classified in terms of domains appropriate to each level, and can be seen in Table 2.1 below:

- 14 -

WHO/ESCAP Training Manual on Disability Statistics

Table 2.1: ICF Levels of Functioning and Disability

Dimensions of Functioning

Dimensions of Disability

Body Functions and Body Structures

Impairments

Activities

Activity limitations

Participation

Participation restrictions

Notice that the term 'disability' in the ICF refers to all three dimensions taken together. In the case of Body Functions, the domains include mental functions, voice and speech functions, and functions of the various organ systems. In the case of Activities and Participation, the domains include learning and applying knowledge, communication, movement, self care, major life areas, and so on. The ICF is a health classification, in the sense that all aspects of human functioning and disability it classified are presumed to occur in the context of health, and so (typically) in the presence of one or more health conditions – diseases, disorders, injuries, and traumas. As such, the ICF provides the language for health statistics, the data that characterize the health of individuals and populations.

Information can be collected about each dimension of disability for an individual; and it is only when all three kinds of information are collected that we have a complete portrait of the lived experience of disability for a particular person. Information about impairments alone, i.e., problems in body functions or structures, although certainly relevant to the disability experience, provides a very limited perspective on disability. From a statistical point of view, collecting data on impairments alone, although traditional in population surveys and censuses, tends to under-describe disability, and under-estimates the prevalence rate of disabilities as a whole. Holistic ICF-based disability information gives us the data required for a wide variety of data applications, including needs assessment, and intervention planning, monitoring and evaluation. Using a traditional list of impairments such as blindness, deafness, mobility problems (as characterised by paralysis or loss of use of a limb) and mental retardation can identify a sub-population with severe disabilities. The population of individuals with intellectual disabilities is very small, and for selfreport surveys, people who have seeing or hearing problems, but who are neither blind nor deaf, will not respond in the affirmative. Similarly, a person who has trouble walking, but is relatively mobile, will answer the question negatively. People with communication difficulties, or chronic health conditions, or other impairments will also answer in the negative.

- 15 -

WHO/ESCAP Training Manual on Disability Statistics

Relying on the impairment approach, in other words, identifies the persons with disabilities as a core minority of severely disabled individuals, ignoring a substantial group of individuals with moderate and mild disabilities. Diagram 2.1 indicates how disability prevalence can be seriously distorted if one adopts the traditional 'core impairment' approach.

Diagram 2.1: Core Impairment vs. Impairment Continuum Blindness Deafness Mobility Mental retardation

All other impairments -- moderate to mild

But it still might be asked: Why isn't impairment information enough? Aren't impairment data good proxies for disability? There are three interconnected reasons why impairment information, though obviously relevant to disability statistics, is inadequate:

1.

Impairment information is a partial picture of disability

Public health professionals and epidemiologists have known for decades that the effects of health conditions on people's lives can differ radically depending on the kinds of impairments, the effect of these impairments on a person's capacity to act, and most importantly, personal factors particular to the individual and the overall environmental context including social expectations about 'normality'. These contextual factors can either facilitate or hinder a person's a social and economic participation in society. Impairment information alone, in short, does not give us a complete picture of the situation of disability, either in an individual or in a population. For an individual, being told that they are blind, deaf, have intellectual or mobility problems does not tell us anything about how these impairments affect the range of capacities the person has to carry out day-to-day activities. And without information about the environment in which the person lives, nothing can be said about what the person actually does in their life -- whether they have a family, go to

- 16 -

WHO/ESCAP Training Manual on Disability Statistics

school successfully or hold a job. These paint the full picture of disability without which we would have no idea what resources or services the individual may require. 2.

Impairment data are inadequate proxies for disability

At the population level, though a few severe impairments can serve as proxies for a fairly large proportion of the overall prevalence of disability, without information about how these impairments play out in people's lives, policy makers would have no idea about the relative seriousness, or cost, of the disability associated with these impairments. Moreover, we have solid evidence from the Global Burden of Disease Study that, in terms of the top ten most 'burdensome' health conditions in terms of years of life lived with disability (YLD) amongst the 15-44 year old group, five are mental health problems such as depression and schizophrenia. These health conditions affect people's capacity to perform day-to-day activities and to live, learn, and work independently. The traditional impairments due to blindness, deafness, and certain conditions in intellectual performance and difficulty in mobility fail to identify persons who are significantly disabled from these health conditions. 3.

Impairments are not predictors of the lived experience of disability

People with the same impairments experience different kinds and degrees of incapacity and vastly different restrictions on what actually happens in their lives. Impairments are not proxies for disability; they give only one particular perspective on disability. Disability is the complete lived experience of non-fatal health outcomes, not merely body level decrements in functioning. The converse is also true: people can experience the same restrictions in what they can do in their day-to-day lives even though they have different impairments. At the level of actual performance, the contrast is even greater. Impairments as diverse as missing limbs and anxiety can both attract stigma and discrimination that may limit a person's participation in work, as illustrated in Diagram 2.2.

- 17 -

WHO/ESCAP Training Manual on Disability Statistics

Diagram 2.2: Different Health Conditions with the Same Participation Restriction

LEPROSY

Loss of sensation of extremities (b265) Facial disfigurement (b810)

Denied employment (d850) due to prejudice by employer

DEPRESSION Sadness (b152)

2.1.3 Universal Another essential aspect of the ICF model is that human functioning is understood as a continuum of health states and every human being exhibits one or another degree of functioning in each domain, at the body, person and society levels. This means that every human being has some degree of disability in at least one domain –and, more accurately, most people have many domains of functioning that are considerably less than perfect. Disability across the domains of health, in other words, is a matter of less than perfect health, from the very trivial to the very serious. (Disability is therefore a universal human situation, not some special trait of a minority group.) ICF applies to all people irrespective of their health condition, gender, or age. Instead of making disability a distinguishing mark of a discrete minority group, ICF sets out all of the domains of functioning and disability that apply to everyone. This universal approach ensures that ICF presents a comprehensive, inclusive and non-discriminatory framework applicable to all individuals around the globe. During the development stage of the ICF, enormous care was given to the applicability, both of ICF categories and the conceptual model, across cultures and languages. Specially designed culturally applicable protocols were devised for field testing of several iterations of ICF. These trials took place in 61 countries involving several thousands of participants. Translation of the ICF is regulated by protocols that guarantee the applicability of ICF terms and concepts. Because ICF follows a universal approach it is not restricted to the traditional diagnostic or impairment categories of persons with disabilities that are

- 18 -

Negative attitudes of employer (e430)

WHO/ESCAP Training Manual on Disability Statistics

commonly used in surveys and censuses. Not only the deaf, blind, paraplegic wheelchair user, or intellectually disabled individual counts as 'disabled', anyone by virtue of any impairment whose activities are restricted above a threshold set by policy makers is a person with disabilities. In disability statistics, the importance of adopting a universal perspective of disability has been recognized. Faced with the objections about the underreporting of disability and confronted with the emergence of disabling conditions like HIV/AIDS, obesity, depression, and an increasingly aging population, many countries have come to realize that the time has come to abandon the traditional categories of disability in favour of a more flexible conceptualization. Censuses, surveys, registers, and other data collection tools based on ICF's universal model will make it possible to gather more accurate and more relevant data about the full extent of disability. 2.1.4 Continuous The universal approach may seem unconventional or difficult to operationalize, because traditionally we think of disability, not as a matter of more or less, but as a matter of ‘yes or no’: you are either disabled or not. This is because, for administrative and policy purposes, it is common to use the concept of disability for eligibility requirements for benefits programmes and services, and so to think in terms of who qualifies and who does not. Disability becomes a 'yes or no' matter as soon as we identify a qualitative threshold of disability ('substantial', 'moderate', 'severe') or a more precise, quantitative threshold ('IQ less than 70', 'eyesight of 20/200', 'BMI over 30'). The threshold in each case divides everyone into two classes, those who are disabled (in this domain) and those who are not. For scientifically-based disability statistics, however, we must remember that the threshold is arbitrary, and the truth of the matter is that disability, like human functioning, is a continuous not a dichotomous phenomenon. This insight is preserved in ICF's model of functioning and disability. In the ICF model, disability is a decrement in body, personal, or social functioning. As such, the continuum of functioning partially coincides with the continuum of health. That is, impairments and person-level incapacities are health decrements. ICF domains can therefore be used in both health and disability surveys, which could contribute for a more comparable and meaningful population data. The concept of disability as a continuum is illustrated in Diagram 2.3 below, which shows varying levels of vision impairment.

- 19 -

WHO/ESCAP Training Manual on Disability Statistics

Diagram 2.3: Seeing Functions as a Continuum

Seeing Functions

10/20

Mild-Moderate vision impairment: Needs eye glasses, contact lenses…

2/20

Severe vision impairment: Needs operation

1/20

Complete vision impairment (blind): Needs assistance, e.g. pension, device, assistant, environmental modifications

Since disability, like health, lies on a continuum, there is no definitive answer to the question: 'What is the level of functioning, for a specified domain, below which a person can be said to have a disability (or be unhealthy)?'. Of course, at the extremes (total lack of functioning, or complete functioning) the thresholds are logically determined. But, in principle, any line on the continuum could be the threshold one might use to make the cut between 'disability' and 'no disability' (or health and ill-health). This makes complete sense, since every person (not merely a member of a diagnostically-specific group) either currently has, or will eventually experience some measures of functional decrement. Although, given how the term is generally used, we probably should not say that everyone is or will be a 'person with a disability'. It remains true that functional decrement is a salient fact of life for everyone and the difference between 'persons with disabilities' and people not so-called is entirely a matter of degree, not membership in a minority group. For some standard impairments (visual acuity, hearing, intelligence), the thresholds are well-established and supported by prevalence. In rehabilitation science, the same is true for many simple actions (limb movements) and even for some activities of standard living such as eating and moving around. But, for the most part, for many domains of body functions and structures and most domains of activities and participation, we have no standardized thresholds.

- 20 -

WHO/ESCAP Training Manual on Disability Statistics

The ICF does not establish, nor presume, a priori thresholds. The user can determine, for specific purposes and on whatever grounds are relevant to those purposes, where the threshold should be placed for each domain of functioning. In particular, this feature of the ICF means that collecting data about functioning and disability at the population level is independent of determining threshold levels, and so prevalence results. Thresholds can be left to the analysis stage, where the purposes of collecting the data can determine where the thresholds should be drawn. The data remains comparable though, and in fact is effectively more comparable across sources, by simply applying the same threshold to multiple population samples. There are several important advantages for disability statistics to determine where on the continuum of functioning the disability threshold should be drawn: ¾ Data collection developers are not locked into pre-existing definitions of who counts as disabled, or traditional thresholds for the domains of functioning they are interested in collecting. ¾ The threshold decision of who should be considered as disabled and who should not be can be explicitly stated by the data collector and that the method of measurement can be tailored to suit the purpose of the data collection activity, thus making the data more flexible. Diagram 2.4 below provides an example of comparisons of “disability score” continuums across five ESCAP region countries. These scores were computed from the six domains of the World Health Organization Disability Assessment Schedule II (WHO DAS II). Note that the data was collected from questionnaire pilot studies using samples with prototypical representations of the general population and thus is not data that actually represents disability in each country. However, the graph provides an example of how prevalence information can be displayed and compared on a continuum, instead of as a single percentage derived from arbitrary thresholds. If, in providing programme services, a single prevalence rate or identified persons with disability’s population is necessary, a particular composite disability score can be determined that is fit for the specific purpose.

- 21 -

WHO/ESCAP Training Manual on Disability Statistics

Diagram 2.4: Overall Disability Score by country

Overall Disability Score by country computed from question D1.1 - D6.8 with WHO DAS II polytomous scoring algorythm n=3914 120%

% o f P o p u la tio n

100%

80%

60% India Indonesia Philippines Fiji Mongolia WHO MCSS

40%

20%

0% 0

10

20

30

40

50

60

70

80

90

100

Disability score

2.1.5 Interactive The ICF model reflects the dynamic non-linear interaction between different constructs including health conditions, the components of disability as well as the personal and environmental factors. As shown in Diagram 2.5, there is also interaction among components of disability (Body Functions and Structures, Activities, and Participation). At the same time, the dimensions of disability are independent. A person may have impairments of body functions or structures without having activity limitation or participation restriction (e.g., a disfigurement in leprosy may have no effect on a person's capacity or to walk). A person may have an activity limitation without participation restriction (e.g., an individual with mobility limitations may be provided by society with a wheelchair that makes it possible for them to get around).

- 22 -

WHO/ESCAP Training Manual on Disability Statistics

Diagram 2.5: Interactions between the Components of ICF Health condition

Body Functions and Structures

Activities

Environmental Factors

Participation

Personal Factors

One of the main objections to the ICIDH-1980 was that it appeared to offer a unidirectional causal model that directly linked health condition and impairments, and from there, disabilities and handicaps. ICIDH appeared to suggest that a health condition causes impairments, impairments cause disabilities, and disabilities produce a handicap. As illustrated in Diagram 2.6, ICF rejects this approach in favour of the multidimensional interactive approach in which whatever links there are between dimensions of disability, causal or otherwise, are based on evidence, not theory. In addition, for statistical purposes, the interactive non-linear ICF model facilitates the collection of data on these constructs independently and, subsequently, the analysis of the many linkages between them. Diagram 2.6: ICF – a combination of different vectors of functioning Cognition Social Activities

Self Care

Work Mobility

Vision

Hearing

- 23 -

WHO/ESCAP Training Manual on Disability Statistics

2.1.6 Context dependent ICF conceptualises disability, not solely as a problem that resides in the individual, but as a health experience that occurs in a context. As Diagram 2.5 indicates, in the ICF, disability and functioning are, conceptually speaking, outcomes of interactions between health conditions (diseases, disorders and injuries) and contextual factors. Contextual factors consist of both environmental factors and personal factors. Environmental factors are all of the physical, social and attitudinal features that together characterize the environment in which a person lives, from climate and terrain to architectural characteristics and legal and social structures. Personal factors include gender, age, coping styles, social background, education, profession, past and current experience, overall behaviour pattern, character, and other factors that influence how disability is experienced by the individual. Personal factors are not currently classified in ICF but users may incorporate them in their applications of the classification.

Distinguishing between environmental factors and components of functioning and disability characterizes issues is of great help to producers and users of disability statistics. This is because the extent to which disability is a function of the person can be clearly distinguished from the extent of environmental impact. Disability policy depends crucially on whether improving outcomes is a matter of investing in changes to the person's capacity levels, by means of medical or rehabilitative interventions, or investing in accessibility, accommodation and other environmental changes. 2.1.7 Inclusive ICF does not make a distinction between health conditions that are 'physical' and those that are 'mental'. Both mental and physical diseases and disorders can be linked to the same kinds of activity and participation issues. So, from a disability point of view, saying that a person has a mental illness tells us very little about the associated disabilities he or she may experience. ICF subscribes to the parity of mental and physical health. Similarly, the model of the ICF ignores etiology of health conditions since there is no a priori link between a health condition and disability as it is experienced. Research into the precise determinants of disability is of course of great importance, but unwarranted assumptions about what a person with a certain disease can or cannot do in life are detrimental to good science. Thus, in the ICF we would record the loss of limb as such, regardless of whether the loss was the result of a landmine, diabetes or a traffic accident. The impairment remains the same. Similarly, we would document the fact that a person was not working regardless of whether the underlying health condition was ‘flu, depression, or back pain.

- 24 -

WHO/ESCAP Training Manual on Disability Statistics

ICF-based disability statistics therefore provides the measures for assessing the disability burden of all health conditions, and that is a great asset when clarifying and focusing disability data on the true phenomenon of disability. 2.1.8 Neutral (language) The ICF classification system uses, to the extent possible, neutral language to name its components and categories. For example, you will find in the ICF: ¾ Vision functions instead of blindness ¾ Intellectual functions, complete impairment instead of totally dull ¾ Participation restriction instead of handicap ¾ Persons with disabilities instead of disabled person The use of neutral language is a helpful challenge in the practice of using discriminatory and offensive language sometimes found in questionnaires or other data collection instruments, and which directly affects refusal rates. 2.2

Importance of classifications and standards

As a classification system, ICF offers a number of key benefits for users and producers of disability data which are listed below. The ICF taxonomy allows the aggregation and disaggregation of data: Three or four level ICF codes, mainly relevant to clinical settings, can be aggregated to two or one level codes that are generally more useful for capturing disability information at population level. Similarly, one or two level ICF items often included in census and general population surveys can be expanded to the third and fourth level for detailed follow-up studies. The ICF allows integration of disparate data sets: As a coding system, ICF makes it possible to consider multiple sources of health and disability information at various levels, and to integrate these into a common and coherent information system. This facilitates tracking the change of an individual's or a population's disability profile and service needs over time across different settings. Through analysis we can assess and compare the impact of different kind of interventions along the service and policy spectrum. The ICF provides a cross-cultural language which allows communication across countries and sectors: As a result of the extensive pilot test performed during the development stage, we know that the ICF functioning and disability framework, as well as the classification hierarchy and categories, are cross-culturally applicable. ICF provides a truly common language for describing and measuring health and disability. The need for such common language is strong in

- 25 -

WHO/ESCAP Training Manual on Disability Statistics

the area of disability statistics, which for decades has been hampered by lack of data uniformity and absence of comparability. Currently, disability survey data shows as much as 40 fold difference compared to a three-fold difference in mortality statistics and a six to eight fold difference in health survey data. The ICF is accepted as the international standard for describing and measuring health and disability: The ICF forms part of the WHO Family of International Classifications (WHO-FIC) and together with the ICD-10 it constitutes the two building blocks of health and disability statistics. The ICD-10 classifies health conditions (diseases, disorders, or injuries) and is used primarily for reporting mortality and morbidity. The ICF has been endorsed by 191 countries as the framework for disability and health data collection around the world. As such, ICF is an international standard of supreme importance for users and producers of disability data. 2.3

ICF terminology and definitions of disability

Beyond a common understanding of the ICF concepts, it is also essential that the terminology and definitions that express these concepts be understood, used and translated into other languages. ICF, as a written classification, builds on a pre-existing language (international English), and then introduces new terms and sometimes changes the meaning of commonly used terms. ICF requires, to a certain extent, users to learn a new language. The key term 'disability' is a clear example of a notion that means a variety of different things in ordinary language. That variety is the cause of confusion that is reflected in disability statistics world wide. ICF clarifies and explicitly defines the term, not by taking sides on disputed issues, but leaving behind the controversies caused by people using the same word differently and opening the door to a consensus. Definitions of some of the key terms used in ICF are given below. The terminology used in data collection vehicles such as survey questionnaires should be aligned to reflect these definitions in order to achieve consistency with ICF. Health condition is an umbrella term for disease (acute or chronic), disorder, injury, or trauma. A health condition may also include other circumstances such as pregnancy, ageing, stress, congenital anomaly, or genetic predisposition. Health conditions are coded using ICD-10. Functioning is an umbrella term for body functions, body structures, activities, and participation. It denotes the positive aspects of the interaction between an individual (with a health condition) and that of individual's contextual factors (environmental and personal factors). Disability is an umbrella term for impairments, activity limitations, and participation restrictions. It denotes the negative aspects of the

- 26 -

WHO/ESCAP Training Manual on Disability Statistics

interaction between an individual (with a health condition) and that of individual's contextual factors (environmental and personal factors). Body functions are the physiological functions of body systems, including psychological functions. “Body” refers to the human organism as a whole, and thus, includes the brain. Hence, mental (or psychological) functions are subsumed under body functions. The standard for these functions is considered to be the statistical norm for humans. Body structures are the structural or anatomical parts of the body such as organs, limbs and their components classified according to body systems. The standard for these structures is considered to be the statistical norm for humans. Impairment is a loss or abnormality in body structure or physiological function (including mental functions). Abnormality here is used strictly to refer to a significant variation from established statistical norms (i.e., as a deviation from a population mean within measured standard norms) and should be used only in this sense. Activity is the execution of a task or action by an individual. It represents the individual’s perspective of functioning. Activity limitations are difficulties an individual may have in executing activities. An activity limitation may range from a slight to a severe deviation in terms of quality or quantity in executing the activity in a manner or to the extent that is expected of people without the health condition. Participation is a person's involvement in a life situation. It represents the societal perspective of functioning. Participation restrictions are problems an individual may experience in involvement in life situations. The presence of a participation restriction is determined by comparing an individual's participation to that which is expected of an individual without disability in that culture or society. Contextual factors are the factors that together constitute the complete context of an individual’s life, and in particular, the background against which health states are classified in ICF. There are two components of contextual factors: Environmental Factors and Personal Factors. Environmental factors constitute a component of ICF, and refer to all aspects of the external or extrinsic world that form the context of an individual’s life and, as such, have an impact on that person's functioning. Environmental factors include the physical world and its features, the human-made physical world, other people in different

- 27 -

WHO/ESCAP Training Manual on Disability Statistics

relationships and roles, attitudes and values, social systems and services, and policies, rules and laws. Personal factors are contextual factors that relate to the individual such as age, gender, social status, life experiences, and so on, which are not currently classified in ICF but which users may incorporate in their applications of the classification. Facilitators are factors in a person's environment that, through their absence or presence, improve functioning and reduce disability. These include aspects such as a physical environment that is accessible, the availability of relevant assistive technology, and positive attitudes of people towards disability, as well as services, systems and policies that aim to increase the involvement of all people with a health condition in any area of life. Absence of a factor can also be facilitating, for example, the absence of stigma or negative attitudes. Facilitators can prevent an impairment or activity limitation from becoming a participation restriction. Barriers are factors in a person's environment that, through their absence or presence, limit functioning and create disability. These include aspects such as a physical environment that is inaccessible, lack of relevant assistive technology, and negative attitudes of people towards disability, as well as services, systems and policies that are either non-existent or that hinder the involvement of all people with a health condition in any area of life. Capacity is a construct that indicates, as a qualifier, the highest probable level of functioning that a person may reach in a domain in the Activities and Participation list at a given moment. Capacity is measured in a uniform or standard environment, and thus reflects the environmentally adjusted ability of the individual. The Environmental Factors component can be used to describe the features of this uniform or standard environment. Performance is a construct that describes, as a qualifier, what individuals do in their current environment, and so brings in the aspect of a person's involvement in life situations. The current environment is also described using the Environmental Factors component. 2.4

ICF structure and coding – for population data collections

ICF is organized into two parts. Part 1 classifies functioning and disability formulated in two components: (1) Body functions and structures and (2) Activities and participation Part 2 comprises the contextual factors which include the following two components:

- 28 -

WHO/ESCAP Training Manual on Disability Statistics

(1) Environmental factors and (2) Personal factors (currently not classified). These components are denoted by prefixes in each code. b for Body Functions and s for Body Structures d for Activities and Participation e for Environmental Factors

The letters b, s, d and e are followed by a numeric code that starts with the chapter number (one digit), followed by the second level (two digits), and the third and fourth level (one digit each). Table 2.2 shows the hierarchy of ICF domains and its reflection in the coding.

Table 2.2: The ICF Domain Hierarchy

Level

Example

Coding

Chapter

Chapter 2: Sensory Functions and Pain

b2

Second level

Seeing Functions

b210

Third level

Quality of Vision

b2102

Fourth level

Colour Vision

b21021

In the ICF, a person's health state and disability state may be characterized by means of an array of codes across the domains of the components of the classification, health and non-health. The maximum number of codes available for each application is 34 at the chapter level (as indicated in Table 2.3, eight Body Functions, eight Body Structures, nine Activity/Participation), and 362 at the second level. At the third and fourth level, there are up to 1424 codes available, which together constitute the full version of the classification. In real-life applications of ICF, a set of 3 to 18 codes may be adequate to describe a case with two-level (threedigit) precision. Generally, the more detailed four-level version is intended for specialist services (e.g., rehabilitation outcomes, geriatrics, or mental health), whereas the two-level classification can be used for surveys/census/administrative data collection and health outcome evaluation. - 29 -

WHO/ESCAP Training Manual on Disability Statistics

Table 2.3: ICF Chapters Body Function:

Structure:

b1 Mental Functions b2 Sensory Functions and Pain b3 Voice and Speech Functions b4 Functions of the Cardiovascular, Haematological, Immunological and Respiratory Systems b5 Functions of the Digestive, Metabolic, Endocrine Systems b6 Genitourinary and Reproductive Functions b7 Neuromusculoskeletal and MovementRelated Functions b8 Functions of the Skin and Related Structures

s1 Structure of the Nervous System s2 The Eye, Ear and Related Structures s3 Structures Involved in Voice and Speech s4 Structure of the Cardiovascular, Immunological and Respiratory Systems s5 Structures Related to the Digestive, Metabolic and Endocrine Systems s6 Structure Related to Genitourinary and s8 Reproductive Systems s7 Structure Related to Movement s8 Skin and Related Structures

Activities and Participation d1 Learning and Applying Knowledge d2 General Tasks and Demands d3 Communication d4 Mobility d5 Self Care d6 Domestic Life d7 Interpersonal Interactions and Relationships d8 Major Life Areas d9 Community, Social and Civic Life Environmental Factors e1 Products and Technology e2 Natural Environment and Human-Made Changes to Environment e3 Support and Relationships e4 Attitudes e5 Services, Systems and Policies

For reporting of health and disability for statistical purposes, however, only a few of the total 1495 codes will be needed. Table 2.4 below lists the WHO recommended ICF item pool for use in health and disability statistics. The items have been identified and used in large scale population based studies (WHO Multi-Country Survey Study, World Health Survey) and are recommended as minimum data sets for surveys and health information systems (See ICF, Annex 9).

- 30 -

WHO/ESCAP Training Manual on Disability Statistics

Table 2.4: WHO Recommended ICF Item Pool Dimension

Chapter

Code

Classification block or category

Body Functions and Structures Vision

2

b210–b220

Seeing and related functions

Hearing

2

b230–b240

Hearing and vestibular functions

Speech

3

b310–b340

Voice and speech functions

Digestion

5

b510–b535

Functions of the digestive system

Bodily excretion

6

b610–b630

Urinary functions

Fertility

6

b640–b670

Genital and reproductive functions

Sexual activity

6

b640

Genital and reproductive health

b810–b830

Skin and related structures

Skin and disfigurement 8 Breathing

4

b440–b460

Functions of the respiratory system

Pain *

2

b280

Pain

Affect *

1

b152–b180

Specific mental functions

Sleep

1

b134

Global mental functions

Energy/vitality

1

b130

Global mental functions

Cognition *

1

b140,b144, Attention, memory and higher–level b164 cognitive functions Activities and Participation

Communication

3

d310–d345

Communication receiving – producing

Mobility *

4

d450–d465

Walking and moving

Dexterity

4

d430–d445

Carrying, moving and handling objects

Self-care *

5

d510–d570

Self-care

Usual activities *

6 and 8

Interpersonal relations 7 Social functioning

9

Domestic life; Major life areas d730–d770

Particular interpersonal relationships

d910–d930

Community social and civic life

* Indicates “Candidate items for a minimal list” according to ICF, Annex 9, and by a High Level WHO Expert Committee in preparation for the WHO surveys mentioned above The ICF codes require the use of qualifiers, which record the presence and severity of the functioning problem on a five-point scale (e.g., no difficulty, mild, moderate, severe, and complete). For health and disability statistics, the use of qualifiers is an important issue when deciding on the type of response scale to be used for questions in census and surveys. To ensure compatibility with ICF, a four, or preferably, five-point scale is suggested (refer to Chapter 4 for a detailed discussion on this topic).

- 31 -

WHO/ESCAP Training Manual on Disability Statistics

2.5

ICF training and assistance available

ICF training and technical assistance is provided by the WHO in collaboration with its network of WHO Collaborating Centres. Further information can be found on the ICF website: www.who.int/classifications/icf.

- 32 -

WHO/ESCAP Training Manual on Disability Statistics

Learning Objectives: Chapter 3 Data Collection Methods and Instruments After reading this chapter, the reader should be able to:

1.

Understand process of disability data collection.

2.

Describe and evaluate population censuses, surveys, and administrative data collection.

3.

Match data collection tools to disability statistics users’ needs.

“Welded to work” Serge Corrieras WHO Photo Contest “Images of Health and Disability 2002/2003”

- 33 -

WHO/ESCAP Training Manual on Disability Statistics

3. DATA COLLECTION METHODS AND INSTRUMENTS 3.1 Measurement of disability 3.1.1 Collection instruments 3.1.2 Choice of selection and measurement unit 3.2

Population censuses

3.3 Sample surveys 3.3.1 As dedicated disability surveys 3.3.2 As a module to other sample surveys

3.1

3.4 3.4.1 3.4.2 3.4.3

Administrative collections What is an administrative data collection? What are the benefits? How can administrative data be used?

3.5

Joint use of census and sample survey – census screen

3.6

Strengths and weaknesses of the data collection methods

3.7

Relatable data

Measurement of disability

Collecting data about persons with disabilities and their lives is difficult. It poses unique problems that data collection developers need to address in the design phase of the collection process. Two issues need to be addressed at the outset: what kind of data collection instrument should be used, and what unit of measurement should be employed. 3.1.1 Collection instruments The main types of instruments for collecting data about persons with disabilities are: ¾ Population censuses ¾ Sample surveys (either general social surveys or specific health and disability surveys) ¾ Administrative collections and registries Each of these tools can be used to measure aspects of disability in a population and each has its strengths and weaknesses.

- 34 -

WHO/ESCAP Training Manual on Disability Statistics

The Population Census certainly has the advantage of providing complete population coverage. Unfortunately, it is difficult to collect accurate information about disability in a census since time constraints make it unlikely that more than 4 to 6 general disability questions can be asked. Censuses, in some instances, also undercount children with disabilities and people with mild or moderate disabilities (in cases where the response categories are limited to ‘Yes or No’ options only). Still, for a crude measure of disability, and in the absence of other collection instruments, the census is useful. Sample surveys are shorter surveys designed to be administered to a subpopulation selected by some other instrument (often a census) that focus on specific issues. They are often put into the field to answer specific questions about a population. As such, they provide the opportunity to ask more detailed questions about disability. More detailed information is useful in itself, of course, but it also helps to reduce the number of false positive and negative responses, thereby offering a more accurate prevalence measure. A sample survey may be an independent survey focusing entirely on disability, or a disability module added to an existing survey. Administrative collections and registers are composed of data that is collected as part of the normal operation of some service or programme. An example is the information found on a client intake form. These collections provide useful information on the characteristics of people accessing disability services as well as details about the services provided. They cannot give an accurate measure of disability prevalence since there is no guarantee of coverage and they tend to incorporate double counting. The quality of administrative register data is closely related to the quality of the administrative system, in particular, how well it has been maintained and how closely the concepts align with the disability concepts of interest. These three instruments for gathering disability information are discussed in detail in the rest of this chapter. 3.1.2 Choice of selection and measurement unit The second preliminary issue that needs to be addressed, whatever data collection instrument is chosen, is how to select the unit for which disability is to be measured. If the selection unit is the individual, then the individual will also be the measurement unit; if the selection unit is a collection of people – invariably a household – then, a decision has to be made as to whether the measurement unit is the household itself (that is, all individuals in the household) or some individual in the household that meets specified criteria (age, gender, unemployed, and/or others). These decisions depend in part on what kinds of data are needed. Is data required for the number of persons with disabilities and their characteristics, or for an estimate on the number of households that include individuals with disabilities? Data about individuals is important, but estimates at the household level are also useful for detailed analysis of living arrangements and access to help and assistance.

- 35 -

WHO/ESCAP Training Manual on Disability Statistics

There are also issues of practicality and efficiency. Having the household as the selection unit means that the survey or interview can be conducted in a setting in which information about other people can be efficiently collected, even if only one member of the household is given the full interview. Questions asked of a single household member may reveal another individual with a disability, thereby increasing the sample of person with disability, without adding to the number of households screened. A note of caution applies, however, if the private household is chosen as the measuring unit. In such a case, thought needs to be given to the fact that this choice excludes residential care units, retirement homes, and other institutions such as prisons. Excluding people living in institutions underestimates the levels of disability for older people and for those with particular types of disability such as psychiatric disabilities. For a complete picture, if feasible, household surveys should be supplemented with institutional surveys of disability. 3.2

Population censuses

Many countries have collected information about disability in their national population censuses (refer to Box 3.6 for the examples of countries using population censuses to collect disability statistics). The United Nations’ 2001 Guidelines and Principles for the Development of Disability Statistics suggests the inclusion of a ‘general’ disability question in censuses in order to get a rough idea of prevalence. In countries without a good household survey program, the census may be the only possible source for estimating disability prevalence and gaining an approximate estimate of types of disability in the country. The amount of information on disability that can be collected in the population census is very limited, and is often confined to a single question. With only one question, false positive and false negative responses are more common and no complete measure of the number of persons with disabilities is possible, especially among children and the elderly. Still, census data should be readily used, where available, to develop more detailed follow-up surveys (see Box 3.5 for a discussion of the Canadian experience with a post-census survey on disability). Many countries use both short and long forms of census questionnaires (see Box 3.1). The short form is for complete coverage of core topics, such as age, sex and location, and may also have a question on disability. A probability sample is then selected for the long form to be administered. The long form includes all questions in the short form plus supplementary questions for more detailed coverage of selected topics. Questions more detailed than a single general question on disability may be included in the long form.

- 36 -

WHO/ESCAP Training Manual on Disability Statistics

Box 3.1: 2001 Short Form/Long Form Census in Macau In response to an increasing concern for more information on the population of persons with disabilities, the Statistics and Census Service of Macau included questions on disability in its 2001 Census. The short form, which was administered to 80 percent of households, included one generic question: “Is anyone in your household physically or mentally disabled?”. In the long form questionnaire, completed by 20 percent of households, the same disability identification question was asked followed by a question about the respondent’s primary disability, and a second question asking whether they had ever used any disability services. The resulting estimate of the number of person with disabilities from the census was 1.3 percent of the resident population.

3.3

Sample surveys

Sample surveys are good methods of collecting data about persons with disabilities. They use sample selection procedures to identify a subpopulation that is representative of the total population, unlike censuses that are designed to enumerate every household or individual in the country. While sample surveys are considerably cheaper than censuses, the size of the sample affects the quality of the data, which is particularly subject to sampling error (refer to Chapter 5 for the detailed discussion of sampling error). This can frustrate analyses that rely on cross-tabulation. A small sample size also means that sample surveys cannot provide detailed information for very small geographical areas. Where such data is needed, census or administrative data collections or sophisticated data modelling techniques are more useful options. Determining sample sizes required to produce statistics with an acceptable level of sampling error is the job of experienced survey methodologists. Unfortunately, the technical details of sample designs are beyond the scope of this manual. The UN Guidelines and Principles for the Development of Disability Statistics has some helpful comments on sample size for disability surveys. (As a rough guide, surveys that have sufficient size to yield valid unemployment estimates for a given geographic area are usually large enough to produce disability estimates for the same area for as long as the prevalence rates are similar.) As mentioned, a sample survey for disability can either be an independent or dedicated survey, or a module to other surveys, such as a national health survey, a general social survey or labour force survey. Where these general social surveys are cyclical, it is very useful to have a disability module or a small set of disability questions added on.

- 37 -

WHO/ESCAP Training Manual on Disability Statistics

3.3.1 As dedicated disability surveys Surveys specifically dedicated to collect disability data are good sources of information about prevalence rates, the causes and types of disability, underlying health conditions, severity and duration of disability, and the use of and need for assistive devices, changes in environment, policies and public awareness on disability. Dedicated disability surveys maximise the amount of information that can be collected to meet users’ needs. Many countries have not attempted to conduct such disability surveys because they are more costly than simply adding a few questions into a census or already existing sample surveys. An example of a country that conducts a dedicated disability survey is provided in Box 3.2 Box 3.2: Dedicated Sample Survey for Disability: Australian 2003 Survey of Disability The Survey of Disability, Ageing and Carers provides data on disability prevalence, need for support, and characteristics of persons with disabilities, older people and those who provide care. Information is collected from private households and cared accommodations (hospitals, nursing homes, aged care and disability hostels and other homes such as children’s homes). The survey uses computer-assisted personal interviews. Since 1981, it has been conducted every five years. Sample size: The sample size is approximately 14,000 private dwellings and 300 non-private dwelling units. The carer sample is approximately 550 dwellings. The final sample depends on the number of people in each household or non-private dwelling, and comprises approximately 36 000 people for the household component and 5 000 people for the cared accommodation component. Data collected: ‰ ‰ ‰

Household composition Demographic information about all household members People selected for personal follow-up interview (people with long-term health conditions or with a disability, who are aged 60 years or over, or someone who regularly provides informal care to someone with a disability) ‰ Socio-economic characteristics (education, labour force participation, income, and housing) ‰ Impairments, long term health conditions associated with main disability condition ‰ Difficulties experienced and help required for activities such as self care and mobility ‰ Types of assistance received for a range of activities, met and unmet needs for assistance ‰ Use of aids and equipment

Results are available in Disability, Ageing and Carers, Australia: Summary of Findings, 2003 (Cat. No. 4430.0).

- 38 -

WHO/ESCAP Training Manual on Disability Statistics

3.3.2 As a module to other sample surveys In many countries, national household surveys – covering topics such as health, education, living conditions, family income and expenditure, labour force participation, nutrition, time use, crime, and so on – are conducted on a regular basis. Whether every five years, annually, quarterly or monthly, these surveys try to identify short and long-term social trends. A cost-efficient and effective way of collecting disability data is to add disability questions – or a ‘disability module’ – to one or several of these cyclical surveys. This approach has been used in many countries. Most commonly, disability modules have been added to labour force or health surveys, and sometimes to living conditions surveys. Box 3.3 gives an example of a disability module in a socio-economic survey. Box 3.3: Disability Module in the Indonesian Socio Economy Survey A disability module was added to Indonesia’s 2003 Socio Economy Survey. The disability population was identified using the question, “Does he/she have a disability?” with a Yes/No response. Where positive responses were received to the disability question, codes for ‘type of disability’ (7 codes for mostly severe impairments) and ‘main cause of disability’ (5 codes) were entered into the person’s matrix. This form of questions is expected to yield low disability prevalence rates because the survey relies on self-identification of disability and the only types of disability asked about are severe impairments. The general approach to including a disability module on a non-disability survey will be useful for the Indonesian government, because disability data can be combined with the other social and economic data collected to analyse the experiences of persons with disabilities (or, at least, those with selected severe impairments).

There are advantages to on-going sample surveys rather than ad hoc or oneoff sample surveys for disability. On-going surveys can make maximum use of the resources initially expended, including the time and other resources used in preparing for the data collection, as well as the trained personnel and other resources dedicated to collecting, processing, and analyzing the data on a regular basis. On-going survey programs also offer opportunities to learn from previous experiences so that the quality and usefulness of the information produced can be improved. They allow for measurement of change over time in key indicators such as frequency of types of disability, severity of disability, quality of life, opportunities and participation of persons with disabilities, and rehabilitation needs. These data can be exceedingly valuable for policy development and evaluation.

- 39 -

WHO/ESCAP Training Manual on Disability Statistics

3.4

Administrative collections

3.4.1 What is an administrative data collection? Besides surveys and censuses, administrative data collection (general purpose administrative systems or administrative registries) intended to serve persons with disabilities can be an important method of gathering information about these people and their characteristics. In this method, any information collected is organized and becomes part of normal service administration procedure such as the information gathered using standard client intake forms for health, rehabilitation, or social work services. Administrative data collections can take several forms, depending on the nature of the service, the format used, the type of information collected, and the method and frequency of collection. Some examples are population registers, vital registration systems, social security systems, registries of occupational injuries, rehabilitation programmes, assistive device services, and other services specifically designed to serve the needs of persons with disabilities. Administrative registries are databases of records of people with particular characteristics and set up as part of the administration of services to these individuals. Registers are either established during a registration survey or other point in time or they can be on-going and regularly updated. Administrative records and registers, moreover, often provide unique information about persons with disabilities. The information is usually collected for reasons concerning the administration of the programme or service, but with care, can prove to be useful. For example, data about children and adolescents in special education programmes is an indication of participation rates in education; general invalid pension registries are often organized by disease or impairment; and domestic care allowances help to track rates of children and adults with severe disabilities. Often these data are collected annually, which provides a source for trend data on the prevalence of impairments or disabilities. Ministries, government departments, advocacy groups, and service providers often maintain administrative records which they use to monitor and evaluate programs and services. Given confidentiality concerns, only aggregated data may be available, but this can still be useful. It must be kept in mind, however, that data from registration systems cannot provide information about persons with disabilities who need a service or programme but do not receive it. Data about unmet need has to come from other sources. Since the information collected in administrative data collections is limited to people receiving services, or otherwise known to the service agency (as in the case of people on a waiting list), these collections have limited coverage. Therefore administrative collections, despite all their benefits, are not good

- 40 -

WHO/ESCAP Training Manual on Disability Statistics

sources for estimating overall disability prevalence. 3.4.2 What are the benefits? As compared with censuses and sample surveys, administrative data collections have some advantages for disability data users: Data from administrative collections are generally available on a regular or on-going basis, as they are part of regularly updated information systems linked to a service. As administrative data are collected as part of the day-to-day running of a service, they use fewer resources than special purpose surveys. If the government or other fund provider requires data collection as a condition of on-going funding to a service, the data collected by that service will tend to be more reliable, complete and of higher quality. Information from administrative data collections is likely to be useful to a wide range of stakeholders such as service providers, higher-level bureaucrats, disability consumer groups, and researchers. 3.4.3 How can administrative data be used? Administrative data collections provide information on numbers and characteristics of service users, and the type, quantity and cost of services provided. Depending on the detail of the dataset and the complexity of the data collection format, a substantial amount of information about services, service users and service providers can be collected in this manner. Administrative data of this sort can then be adopted by service agencies, planners, consumer advocacy groups, and funding department for a variety of purposes: ¾ to provide an evidence base to support planning for future service delivery (e.g., by monitoring trends in client age or disability type); ¾ to indicate whether identified groups of people (e.g., particular ethnic groups) are accessing services as much as expected; ¾ to monitor the cost-effectiveness of service provision; ¾ to support a budget submission for increased funding. Moreover, with only minimal analysis, these data can answer basic administrative questions such as: ¾ How many people were supported by the service in a given time period, and what were their characteristics and support needs? ¾ What type of support was provided and received?

- 41 -

WHO/ESCAP Training Manual on Disability Statistics

¾ What quantity of support – e.g., in terms of staff hours – was provided and received? ¾ From whom was the support received (in terms of details of the service agency, such as size, staff profile, and hours)? ¾ What was the cost (total, per service type, per client) of providing these services? ¾ What were the outcomes for clients? (Examples of outcome questions include: Were clients satisfied with the services received? Were there increased levels of client participation in key life areas such as employment?) Box 3.4 gives an overview of the Australian National Disability Service data collection as a concrete example of an administrative data collection. It outlines the information collected, the method of collection, and how the data is used. Box 3.4: An Overview of the Australian Disability Services Data Collection In Australia, the Commonwealth/State/Territory Disability Agreement (CSTDA) funds a national program of disability support services for people with on-going support needs. Under the agreement, the Commonwealth Government is responsible for the overall planning and management of employment services, while the States and Territories are responsible for all other disability support services (including accommodation support, community access, community support, and respite services). All three jurisdictions share responsibility for advocacy, information, and print disability services. Services are provided by government and non-government service provision agencies. ‘Persons with disabilities’ are defined as people with a disability attributable to an intellectual, psychiatric, sensory, physical or neurological impairment or acquired brain injury (or some combination of these), which is likely to be permanent and to result in substantially reduced capacity in self-care, mobility or communication, requiring on-going or episodic support. National data on disability services From 1995-2002, the National Minimum Data Set (NMDS) collected data about services received on a single ‘snapshot’ day each year. Since then, data have been collected on a full-year, on-going basis, so that the collection now includes data on all persons with disabilities who receive a CSTDA-funded service during the year. The NMDS collects the minimal, essential set of data about disability services and clients. It is a set of nationally-agreed upon data items, and an agreed method of collection and transmission. The data collected includes: ¾ Demographic information, e.g., age, sex, indigenous status ¾ Support needs and whether the person has a carer

- 42 -

WHO/ESCAP Training Manual on Disability Statistics

¾ ¾ ¾ ¾

Primary disability and other disabilities Living arrangements and communication method Work, income, and funding, e.g., main source income Services the person receives

It also contains data items on service provider agencies, including the type of service provided, staff hours, operating hours per day, and number of service users. How the data is used The CSTDA-NMDS data provides valuable information about those who are receiving services, their characteristics (age, disability type, and support needs) as well as trends in types of services used. Data for nine years is now available. The data is used in funding negotiations between Commonwealth and State governments, and between departments and service providers. It is also used for developing national performance indicators, by which service outcomes can be monitored. The Australian Institute of Health and Welfare publishes a national data report on the collection each year. For more information on the Australian NMDS data collection see http://www.aihw.gov.au/disability/pubs.html

3.5

Joint use of census and sample survey – census screen

Most countries in the Asia-Pacific region do not have registers of the population to use as sample frames for social surveys and instead use areabased samples in their household data collections to select respondents for their surveys. Persons with disabilities are a relatively small population and so obtaining a sufficient sample can be very costly. Some countries use a small number of disability screening questions to provide the sampling frame for a follow-up survey. When this is done, it is important that the screening questions are effective in identifying as many persons with disabilities as possible. In particular, screening questions should attempt to minimize the number of “false negative” responses. Another method to find a targeted sample inexpensively is to use the census as a screening device to identify a population of persons with disabilities who are then the sample for a follow-up (or “post-census”) survey (see Box 3.5). As mentioned, relying on one or two screening questions will increase the number of false negatives. To compensate for this, it is wise to include in the follow-up survey a sufficient sample of negative responders to avoid biasing the survey population. Studies have shown that children with disabilities and people with mild disabilities are the most likely to be under-reported by the census screening questions. Whenever the census is used as part of a data collection strategy, timing becomes a factor. In order to add questions into a census, steps must be taken several years in advance of the actual data collection. Furthermore, it

- 43 -

WHO/ESCAP Training Manual on Disability Statistics

will take one or two additional years to have the census data processed and available for post-census surveys or other uses. Box 3.6 shows the different methods used by some countries in collecting disability data. Four of the nine countries (Cambodia, Fiji Islands, India, and Thailand) employed at least two methods of gathering disability data. (See also Appendix 1). Box 3.5: Use of the Post-Census Survey Approach to Collect Disability Data in Canada The post-census survey approach was used by Statistics Canada in its comprehensive surveys on disability following the 1986, 1991 and 2001 censuses. The censuses included two screening questions on disability and impairment. In 2001, the questions were: 1.

Does this person have any difficulty hearing, seeing, communicating, walking, climbing stairs, bending, learning or doing any similar activities? Yes, sometimes Yes, often

No 2.

Does a physical condition or mental condition or health problem reduce the amount or the kind of activity this person can do At home? At work or at school? In other activities, for example, transportation?

The purpose of these questions was not to estimate the prevalence of impairment or activity limitations. They merely defined a sample of individuals likely to have a disability. Following the census, a national sample survey based on census results was prepared. Those who screened positive in the census by answering yes to at least one of the questions were then asked detailed questions to confirm whether they had disability. A small sample of people who screened negative in the census was also included. The post-census survey results provided far more accurate information on the prevalence of disability than possible with a census. The survey also provided detailed data about the nature of the disability and other characteristics of persons identified. It has become the practice not to publish the disability data from the census at all, but only to use it to determine the sample frame for the follow-up survey.

- 44 -

WHO/ESCAP Training Manual on Disability Statistics

Box 3.6: Disability Data Collection Methods Practiced in Selected Countries Country

Method of Data Collection

Cambodia Fiji Islands Hong Kong, China India Indonesia Islamic Republic of Iran Mongolia Philippines Thailand

Social Economic Surveys and Population Census Census of Population and Household Income and Expenditure Survey Survey on Persons with Disabilities and Chronic Diseases Census, National Sample Survey Organization, Survey and Voluntary Registry National Socio-Economic Surveys Census Limited registry in capital city (voluntary, impairment-based registry) Population and Housing Censuses Disability Surveys, Bureau of Empowerment for Persons with Disabilities National Registry and Censuses

3.6 Strengths and weaknesses of the different data collection methods Tables 3.1 to 3.4 below summarise the salient advantages and disadvantages of census, sample surveys, and administrative data collections (both one-time and continuous) as approaches to the collection of disability data. Table 3.1: Population Censuses: Advantages and Disadvantages 2 Advantages

Disadvantages

Data can be tabulated for small, local areas.

Questions limited to basic socioeconomic and demographic characteristics, restricting the scope of disability questions.

Prevalence rates can be calculated for small geographical areas because data are also gathered for the population at risk.

Data collection is infrequent – usually every 10 years. The time between data collection and data dissemination can be considerable.

Detailed descriptive cross-tabulations are not subject to sampling errors.

In some censuses, populations in institutions with disabilities may not be included.

2

This information was taken from the United Nations’ 2001 Guidelines and Principles for the Development of Disability Statistics. See this document for further details.

- 45 -

WHO/ESCAP Training Manual on Disability Statistics

If disability questions remain comparable, they can be useful analysis of disability rates across time.

Subject to high non-response rates and under-enumeration because of the complexity and sensitivity of the question.

Identified set of persons with disabilities is usually large, allowing more detailed cross-tabulations and analyses.

It is costly and time-consuming to identify a relative small population of persons with disabilities by asking questions of the entire population.

Can provide a useful sampling frame for research on persons with disabilities who are otherwise difficult to find.

May be too costly to train enumerators in the specific guidelines required for disability questions.

Table 3.2: Sample Surveys: Advantages and Disadvantages

Advantages Flexibility in the depth and range of topics covered.

Disadvantages Limited sample size for small geographic area resulting to higher sampling errors, thus, limited ability to analyse prevalence rate for local areas.

Special probes can be used to ensure that persons with disabilities are identified.

Persons with disabilities account for less than 20 percent of any population, so unless the survey is very large, the sample size of persons with disabilities will be small Coverage poor for institutionalised persons, the homeless, refugees or nomadic populations. Time-series analysis of ad hoc surveys is uncertain.

If sampling frame and survey infrastructure in place, relatively easy to initiate. If comparable with census and other surveys, can be used for detecting change over time. Because of limited coverage and smaller sample, there is greater control over the conditions of observation and interviewing. Capacity to locate persons with disabilities can be increased with design modifications (e.g. co-coordinating probability sample selection with the census, using registered population lists, stratifying the sampling stage, or increasing the sampling fraction). Greater opportunity for field work supervision, specialised field training, question pre-testing.

Detailed surveys require close supervision of fieldwork and special disability training for field supervisors and interviewers.

- 46 -

WHO/ESCAP Training Manual on Disability Statistics

Table 3.3: One-time Registration Data Collection: Advantages and

Disadvantages

Advantages Easy to initiate.

Does not require a large, on-going staff.

Can provide a large number of cases for complex data analysis where extensive cross-tabulations and disaggregation are required. May be used to assist governmental and private agencies (e.g. social security and social welfare offices and medical and educational organizations) locating persons with disabilities in need of services. May be used to prepare sampling frame for research on populations with specific disabilities.

Disadvantages Extent of coverage and the degree of representation are unknown and sampling error cannot be estimated. Population at risk is unknown, so prevalence and incidence rates of disability cannot be calculated. Double counting is a problem, given many classes of informants and multiple registration lists of registered persons with disabilities. Findings can be easily or validly extrapolated to a larger population of persons with disabilities.

Table 3.4: Continuous Registration Data Collection: Advantages and Disadvantages

Advantages Tabulations can be prepared for small geographical areas. Detailed dis-aggregation is not subject to sampling error. Can provide numerator data for incidence and prevalence rates. Both short and long-term series are easier to compile and more reliable because of institutional continuity in the collection process. Data collection can be closely linked to the provision of special services. Provides a sampling frame for in-depth research on populations with specific disabilities.

Disadvantages Reporting based on when disability was diagnosed rather than onset. Population at risk must be determined independently from census data or from population estimates or projections. Registration systems are inflexible to changes to content and procedure. Organization and administration require well-trained statistical personnel.

Data collection and compilation over time is complex. Multiple registration, and double counting, is difficult to detect.

A comparative summary of the strengths and weaknesses of these four principal data collection methods for estimating and describing disability is given in Table 3.5.

- 47 -

WHO/ESCAP Training Manual on Disability Statistics

Like mortality and fertility, disability is a phenomenon that is neither uniformly nor normally distributed across the total population. Disability reporting has many of the same problems as in the reporting of mortality, such as the reliance on non-expert description of medical events, difficulty recalling the time of an illness or disability, and problems determining the most eligible and knowledgeable respondents. The 12 criteria for evaluating the data sources shown in the table below are particularly relevant to the collection of data about disability.

Table 3.5: Data Collection Methods for Disability Statistics: Comparative Summary

Criterion Ability to capture detailed disability characteristics Ability to capture topical detail (richness and diversity of nondisability characteristics)

Accuracy and coverage

Absence of sampling error Timeliness of data Geographical detail Ability to obtain information on the population at risk Ease of organization in a developing country Number of cases available for the analysis of selected disabilities Ability to estimate prevalence rates Can identify persons requiring special services Usefulness for community-based project planning for persons with disabilities

Population Census

Sample Surveys

One-time Registration

Continuous Registration

Weak

Weak to moderate

Moderate to strong

Moderate to strong

Moderate

Strong

Moderate

Weak

Weak to

Weak to moderate

Moderate

Strong

moderate

Strong

Weak

Weak

Weak

Weak

Moderate to strong

Strong

Moderate

Strong

Weak

Strong

Strong

Strong

Strong

Weak

Weak

Moderate

Strong

Strong

Weak

Moderate to strong

Weak to moderate

Moderate

Strong

Strong

Strong

Weak

Weak to moderate

Weak

Weak

Strong

Strong

Weak to moderate

Weak

Moderate to strong

Moderate to strong

- 48 -

WHO/ESCAP Training Manual on Disability Statistics

3.7

Relatable data

Table 3.5 illustrates that each data collection method has its strengths and weaknesses concerning disability statistics. If more than one method is used, in a coordinated fashion, then the strengths of each method will be preserved and the weaknesses of each avoided. Deficiencies with census data, for example, can be addressed through disability surveys or disability modules in other surveys. The inadequacies with survey data can be partly overcome by using censuses, registered populations, and other administrative data. Undoubtedly, a major challenge when assessing the life situation of persons with disabilities is locating and integrating data from various sources. First of all, there must be a variety of data sources to draw upon, and in many developing countries this is not true. Secondly, even if there are various data sources, the data must be relatable, that is based on a common conceptual framework and using comparable concepts and language. The ICF provides just what is required for relatability: a universal framework and a common language of disability. Box 3.7 gives an Australian example of the use of data from a variety of sources to examine the unmet needs for disability services, showing in particular how survey and administrative data can be analysed together to provide information regarding the lives of persons with disabilities.

Box 3.7: Using Data from a Variety of Sources: The Australian Disability Services Project In 2001, the Australian Institute of Health and Welfare was commissioned to estimate levels of unmet need for disability services among persons with disabilities in Australia. Specifically, the project aimed to: ¾ ¾

assess the effectiveness of previously allocated funding in reducing unmet need for disability services, by quantifying and describing additional services provided as a result of the funding; and identify any remaining unmet need for disability accommodation, in-home support, day programs, respite services, and disability employment.

The two sources of data used were the NMDS data on services and consumers (the source of information on the supply of services); and disability population survey data (a source of information on unmet need for services).In conducting the analysis, the ICF was used as a common framework to which concepts and data items from these two data sources were mapped. Overlapping ICF concepts could then be identified in both data sources: Disability services data collection Impairments (Need for assistance with…) Activities: Self-care, Mobility Communication, other activities Participation

Disability survey (Need for assistance with…) Activities: Self-care, Mobility Communication, other activities

Thus, a single ‘indicator’ of disability – “need for assistance” – was linked to the core purpose

- 49 -

WHO/ESCAP Training Manual on Disability Statistics

of a range of disability services. Estimation of the number of people in the Australian population with unmet needs for disability services was then based on an analysis of the survey data. The analysis involved: ¾ ¾

Using survey data to identify people who reported a need for help with self-care, mobility, or communication. This group corresponds to the ‘target population’ for funded disability services; and Applying ‘filters’ to refine the estimate, focusing on people who were living in households (not in institutional accommodation) and who reported unmet need for formal help in addition to a need for frequent assistance.

The analysis produced the following estimates of unmet need across Australia • 12,500 people needing accommodation and respite services • 8,200 places needed for community access • 5,400 people needing employment support Though thought to be too low, the findings informed negotiations between State, Territory and Commonwealth governments regarding funding for disability supports. This powerful analysis was only possible because common concepts were used defining the target population for disability services and the disability population survey. This was accomplished because both sets of concepts could be mapped to the ICF framework. The report on this study is available at http://www.aihw.gov.au/publications/index.cfm/title/7741

- 50 -

WHO/ESCAP Training Manual on Disability Statistics

Learning Objectives: Chapter 4 Using ICF in Survey and Census Design After reading this chapter, the reader should be able to:

1.

Describe issues in operationalizing disability for data collection needs.

2.

Distinguish and describe approaches to disability measurement.

3.

Describe issues in question development for disability data collection.

“Buddha and boatman” Serge Corrieras WHO Photo Contest “Images of Health and Disability 2002/2003”

- 51 -

WHO/ESCAP Training Manual on Disability Statistics

4.

USING ICF IN SURVEY AND CENSUS DESIGN 4.1

Operationalization of ICF in disability data collections

4.2 Key issues 4.2.1 Reduction of ICF items 4.2.2 Scope and coverage 4.2.3 Duration of disability 4.2.4 Use of assistive devices 4.3 Approaches to measurement 4.3.1 Impairment Approach 4.3.2 Activity and Participation Approach 4.3.3 Environmental Factors Approach 4.3.4 Comprehensive Approach 4.3.5 Advantages of the Comprehensive Approach 4.4 Question formats and response scales 4.4.1 Simple, list, multiple choices, open-ended, closed 4.4.2 Scaled response questions 4.5 Additional issues in question development 4.5.1 Language 4.5.2 Gender issues 4.5.3 Cultural issues 4.5.4 Context of disability questions 4.6 Census question formats 4.6.1 Single general question 4.6.2 Checklist questions 4.6.3 Including a response category on disability 4.7 International disability question sets 4.7.1 UN Guidelines for the Development of Disability Statistics 4.7.2 UN Global Census Recommendation 4.7.3 WHO/ESCAP’s pilot tested disability questions 4.7.4 Development of disability statistics from a census/survey: An example 4.8

Administrative data collections – an Australian example

- 52 -

WHO/ESCAP Training Manual on Disability Statistics

4.1

Operationalization of ICF in disability data collections

ICF makes it clear that disability is a multi-dimensional concept and cautions us to be clear about which dimension of disability we want information about. At the same time, the term comes with many different connotations and associated meanings with wide variations across time, culture, and age groups. Some people firmly believe that to be disabled is to be completely unproductive and useless in society, so will insist that they do not have a disability, even though they do have a functional restriction obvious to any observer. Some believe that disabilities are only physical, and refuse to believe that depression, or another mental disorder constitutes disability. Others insist that you cannot be disabled if your functional limitation is something many or even most people have, such as poor eyesight. In everyday language, disability means many different things to different people. Data collection developers face an enormous challenge: devise questions that avoid as many irrelevant beliefs or outright misconceptions about disability as possible. Even among experts, different ways of operationalizing disability, when incorporated in a data collection instrument, can yield very different prevalence results. A dated but useful example involves the three distinct operational measures of disability collected in the 1992/93 New Zealand Household Health Survey. The survey included data on self-reported disability, diagnosed disability and a measure of functional disability. The proportions of persons with disabilities varied widely across these three measures, ranging from 15 percent with a self-identified disability to 29 percent with a diagnosed disability. Given the disparity, these disability measures are no longer used in New Zealand. Recent surveys are designed to collect only measures of functional disability consistent with the ICF. Users’ needs must be clearly defined in order to determine how the population of persons with disabilities is to be measured. For example, if the users require small area data in order to have information for service planning at a local level, then the question is whether the population can be captured using one simple but broad question or whether several more specific questions would be better. Box 4.1 describes such a situation.

- 53 -

WHO/ESCAP Training Manual on Disability Statistics

Box 4.1: Australian Experience with Disability Census Questions During the development phases of its 1996 and 2001 population censuses, the Australian Bureau of Statistics (ABS) tried using a single question to identify a broad disability population. Testing showed that the disability population identified using this single census question was significantly different from that identified by a more detailed set of questions used in a national disability survey. In particular, the ABS noticed that a large number of people who reported disabilities in the survey questions did not report in the census, and people reporting disabilities in the census question did not always do so in the survey. Even though the proportions of persons with disabilities linked to the two question sets were very similar, the actual populations identified were different. In light of this, the ABS decided not to include a disability question in either the 1996 or 2001 census. In preparations for the 2006 census, the ABS looked at questions that focus on a different population. The new census questions attempt to identify people who need assistance with mobility, self-care or communication because of a disability or longterm health condition. Results were very promising and the disability-related module was included in the 2006 census.

ICF tells us that the first thing the data collection developer must do is to be absolutely clear which dimension of disability they are interested in. Is it impairment, activity limitation, participation restriction or environmental and social barriers? Unfortunately, although many national surveys, censuses or administrative data collection use these broad concepts of the ICF in their disability data collections, there is not yet a final international standard method or instrument for measuring disability linked to ICF categories. Such a method or instrument is still under deliberation by international experts on disability statistics. In this chapter, we focus on some basic issues of operationalization – the transition from ICF concepts and categories to the wording of questions usable for a variety of data collection vehicles. In the next chapter we look in more detail at these data collection instruments for collecting disability data, and later in Chapter 6 consider problems in developing and testing these instruments. 4.2

Key Issues

As it is true of statistics in any subject-matter area, there are a number of methodological issues involved in the production of disability statistics. Here we look at those issues related directly to the operationalization of ICF’s concept of disability.

- 54 -

WHO/ESCAP Training Manual on Disability Statistics

4.2.1 Reduction of ICF items Anyone opening the ICF ‘Red Book’ will be struck with one obvious problem: as a comprehensive classification of all dimensions of disability and environmental factors – with over 1400 separate categories – there are far too many domains to possibly be covered in a set of questions measuring aspects of disability. Even at the highest level of categories, the ICF gives nine domains of the activity and participation component. This number of domains is too many for data collection in most surveys, let alone in censuses. Even if a single domain is selected, for example, “self-care” , the ICF divides it into “caring for oneself”, “washing and drying oneself”, “caring for one’s body and body parts”, “dressing”, “eating and drinking”, and “looking after one’s health”. It is difficult to imagine how a single survey question could capture the complete range of activities covered by all of the activity and participation domains. There is no easy way to reduce the hundreds of body functions and structure codes to a small number of impairment questions; nor can the nine domains of activity and participation be easily reduced to a single ‘disability question’. Of course, users’ need is the key. But even so, considerable thought, and often testing, needs to go into the selection process. Examples of how Australia and Canada have made the selection of ICF Activity and Participation domains are given in Box 4.2.

Box 4.2: ICF Items in Australian and Canadian Measures of Disability In Australia, key disability data users wanted to define the disability population as broadly as possible. They were also interested in a specific population: people with a disability who need help or assistance in the areas of mobility, self-care and communication. These needs drove the decision to develop survey questions to identify these populations. In the early testing for the 1981 disability survey, a comprehensive set of screening questions was developed to cover all nine of the chapters of the ICF Activities and Participation classification. Analysis of test results showed that some of the screening questions were simply redundant, as people who had a disability in one area almost always had a disability in another area. Some screens were therefore dropped. Statistics Canada had a similar experience when developing the census screening items for their 2001 Participation and Activity Limitation Survey. They found out that there was very little additional benefit in including the term “toileting” in their list of activities, as almost everyone identified through this screening item also had difficulties with mobility. Therefore, toileting was dropped from the screening question.

- 55 -

WHO/ESCAP Training Manual on Disability Statistics

Although a country may want to identify disability as broadly as possible, such as activity limitations in any of the Activity and Participation domains of the ICF, it may be impossible to develop a small question set that would adequately identify everyone in the relevant population. The most sensible and practical solution is to go back to the data users and ask them which are the most important domains to measure, and use this information to include questions that would measure limitations in each of the high priority domains. Included in these data users groups are necessarily persons with disabilities themselves and persons with disabilities “Self-help Organizations”, i.e. NGOs run by persons with disabilities. The obvious reason for including persons with disabilities in disability collection design is the fact that they are not only important data users, but also represent those that will be most affected by use of the data. 4.2.2 Scope and Coverage An important step in the development of a statistical collection is to define the scope of the inquiry. Who is to be included in this data collection: children, adults, or older people? All of these people or only some of them? Should the data collection identify people living in private households, collective dwellings or institutions? Should the data collection cover all geographical areas, urban and rural, or some restricted part of a large urban centre? Care must also be given to the coverage of the survey, census, or administrative-based data collection. Serious consideration should be given to the following questions: Which dimensions of the concept of disability should be included – impairments, activity limitations, participation restrictions, social and environmental barriers or some combination? Should the data collection focus on selected disabilities (those associated with a specific disease or public health problem) or with all types of disabilities? Finally, should the data collection cover specific levels of severity of disability (severe, or moderate to severe) or all levels, including mild disability? The ICF is flexible and places no limitation on scope or coverage. Instead, the ICF provides a complete descriptive framework for all aspects of human functioning, a framework that systematically organizes these data. Nonetheless, decisions about scope and coverage are essential, unavoidable, and have a profound effect on the usefulness of resulting data. These decisions should be driven by the purposes of data collection, and therefore the needs of the ultimate data user. For example, if the objective is to produce national estimates of disability for different age groups in the population, people living in both private households and institutions should be included. If people living in institutions, particularly health institutions and institutions for the elderly, are excluded from the scope of the collection, the results are likely to be biased in favour of the younger population. Specifically, the data collected would tend to underestimate the true level of disability amongst older people and people with certain types of disability associated with higher levels of institutionalization, such as intellectual disabilities (see Box 4.3 for the New Zealand experience).

- 56 -

WHO/ESCAP Training Manual on Disability Statistics

Box 4.3: Inclusion of People Living in Institutions in Disability Surveys in New Zealand The total disability rate for the New Zealand population aged 75 years and over, is 66 percent. The rate is only 62 percent for people living in private dwellings. If the purpose of a survey in New Zealand was to get information about the frequency of severe disabilities, then people living in institutions should be included since they are generally more likely to have severe disabilities. Excluding the institutionalised population from the scope of the collection would result in an underestimation of the occurrence of severe disability.

People living in prisons or other custodial institutions are notably absent from most disability surveys. In many national disability surveys, this has led to underestimates of the prevalence of developmental disabilities, and intellectual and psychological disabilities. However, if the coverage of the survey is restricted to physical disabilities then, in part because these individuals make up a relatively small proportion of the total population, excluding them may not have a significant effect on the survey results. The level of severity of disabilities covered will affect assessments of progress made on the social goals of inclusiveness and full participation in economic activities. Very broad disability coverage is likely to show greater levels of employment of persons with disabilities as compared with disability populations made up of people with severe impairments. Any comparisons within countries over time, or across regions or countries, need to first identify the disability severity of the populations taken into account. The consistency of disability populations over time is also important when evaluating progress towards the achievement of social goals of inclusiveness and full participation. If the population coverage changes every data collection, then comparison is difficult or impossible. In Australia, for example, the measurement of changes in the disability population and their characteristics was identified as the highest priority need by the user community when developing the 2003 Australian disability survey. To meet this need, the Australian Bureau of Statistics made as few changes to the previous survey instrument and procedures as possible, to ensure as close a replication of the previous survey as possible. 4.2.3 Duration of disability As far as the ICF is concerned, the duration or chronicity of a disability is merely a matter of severity. When developing surveys or censuses, however, duration may pose a significant problem. Should a person who is completely unable to perform some activity, or go to school or work, because of a temporary condition or health problem (such as a broken leg) be counted as having a disability? Should a person who has the

- 57 -

WHO/ESCAP Training Manual on Disability Statistics

flu or some other debilitating, but short term, illness, be counted as disabled? Or, is a ‘person with a disability’ someone who has a chronic health condition and lives, and will always live, with the functional consequences of it? There is no standard solution to the duration issue used internationally. In most of the collections in the Asia-Pacific region, the issue is ignored and no time period is cited for an activity limitation to count as a disability. Some national collections use the phrase “long-term” health condition, or “a condition which has lasted or is likely to last for 6 months or more”. For example, the 2001 UK Census asked: Do you have any long-term illness, health problem or disability which limits your daily activities or the work you can do? Include problems which are due to old age ‰ ‰

Yes No

No guidance is given to respondents on what ‘long-term’ means, and presumably different respondents interpret the phrase differently – 6 months, a year, 2 years or longer. This ambiguity obviously affects the reliability of the data. Other national data collections are very clear about duration – 6 months in many cases (see the Canadian Health and Activity Limitation Survey, the New Zealand Household Disability Survey, and the Australian Survey of Disability, Ageing and Carers). Others have asked questions in the context of difficulties experienced on an average day e.g., the Canadian General Social Survey disability module. Two further complications make the issue of duration a concern: First, duration may apply either to the underlying health condition, or to the length of time that the person has been restricted in their activities. It is possible for a person to have a long-term illness that does not restrict their activities. This is particularly true of many progressive illnesses, such as Parkinson's Disease, which often do not limit people in the early stages of the illness. The second complication involves episodic and degenerative conditions. For example, some forms of schizophrenia are episodic, and the person’s life is unaffected when he or she is in the ‘up’ cycle of the disease. Is that person disabled all the time, or only when they actually experience the effects of the illness? In the case of progressively degenerative diseases, the concern is somewhat different. For example, in the case of HIV/AIDS the underlying health condition is long-term, but at any particular point in time, the disease may have more or less effect on functioning. As the disease progresses, the activity limitations

- 58 -

WHO/ESCAP Training Manual on Disability Statistics

and participation restrictions are likely to become more frequent and more extensive. Another example is arthritis. While the condition itself is longterm, many people have symptoms that cause limitations in their activities for short or long periods of time. These and other duration-related issues obviously need to be carefully thought through when designing questions. For some purposes, the duration question may not matter at all. For other purposes, duration may be an essential factor in determining the utility of the resulting data. 4.2.4 Use of assistive devices Should people who use assistive devices or aids – such as reading glasses, contact lens, or hearing aids – which completely eliminate any restriction to their activities be counted as having a disability? And when assistive devices do not eliminate, but merely reduce activity restrictions, such as in the case of a crutch, cane or wheelchair, how will the severity of the disability be assessed? Generally, including people who use assistive devices or technical aids within the scope of a statistical collection means that the prevalence of disability will increase, particularly among the older population with a higher proportion of people wearing glasses, hearing aids and other devices. The answer to the question: “Should assistive device use be part of the operationalization of disability or not?” depends on the purpose of the data gathering tool and the needs of the data user: If the purpose of collecting the data is to obtain prevalence data on impairments, then obviously it is important to include people who use assistive devices since these devices have absolutely no effect on the presence of an impairment. If wearing glasses puts a person’s visual acuity in the normal range, this does not mean that the underlying impairment or impairments have been corrected. If a data user wants to get a sense of the potential market for assistive devices – needs assessment for a sub-population, for example – then people who already have and use assistive devices ought not to be part of the scope of the data collection activity. If the purpose of collecting the data is to determine how many people have difficulties performing activities of everyday living (e.g. bathing, dressing, preparing meals, shopping) then, it is essential that the scope include people who rely on assistive devices and technical aids, since only then will the data reflect the actual level of participation restriction in this area. Again, there is no international standard practice in statistical collections. In some cases, the questions designed to detect the presence of an impairment or activity limitation do not mention the use of assistive devices or technical

- 59 -

WHO/ESCAP Training Manual on Disability Statistics

aids. This is particularly true of census questions where there is little scope for detailed qualifications or explanations of definitions and terms. In some detailed inquiries, people who wear glasses and contact lenses are excluded from the count of persons with disabilities. This is done by asking questions about levels of difficulty where, it is presumed, the person who wears glasses will indicate ‘no difficulty’ and so will not be counted. For example, the Australian and New Zealand surveys ask: Do you have any difficulty seeing ordinary newsprint, with glasses or contact lenses if usually worn? Do you have difficulty clearly seeing the face of someone across a room (that is from 4 metres/12 feet), with glasses or contact lenses if usually worn?

The Canadian Participation and Activity Limitation Survey (2001) adopts a different technique. For each activity domain (such as hearing), the survey first asks whether the respondent uses a common assistive device for that domain (hearing aids), then, depending on what answer is given, the survey asks about level of difficulty with or without the device: Do you use a hearing aid or hearing aids? (1) Yes (3) No (x) Don’t know (r) Refusal

With your hearing aid, how much difficulty do you

have hearing what is said in a conversation with ONE other person (1) No difficulty (2) Some difficulty (3) A lot of difficulty (4) You cannot hear

If hearing aid is not used:

Which of the following best describes your ability to hear? (1) You cannot hear (2) You have difficulty hearing (3) You have no problem

In this manner, data about activity limitations with and without the use of hearing aids is collected. Whichever technique is used, and whether the intention is to count people whose assistive device eliminates or lessens the severity of the impairment or

- 60 -

WHO/ESCAP Training Manual on Disability Statistics

not, the decision should be clearly documented in any output from the survey. Users can then understand how the numbers have been arrived at, and can take this survey decision into account when comparing the survey results with those from other collections.

4.3

Approaches to measurement

As mentioned in Chapter 1, analysis of UN DISTAT data shows disability prevalence rates ranging from less than 2 percent in Syria, Bangladesh and Tunisia, to about 20 percent in Australia and New Zealand. This enormous range of prevalence rates is obviously caused by different conceptions and measures of functional limitation, all described by the same, ambiguous term, ‘disability’. Different approaches to measurement produce different rates. The virtue of the ICF is that it clearly distinguishes the different dimensions of the overall disability experience so that ‘disability’ survey questions can refer to the same phenomenon. It also enables a comprehensive, multi-dimensional measurement strategy in which prevalence is determined by an active decision about the threshold value. Since this decision occurs after data collection and during analysis, it is transparent and open to debate. This is appropriate since prevalence rates directly affect policy decisions with respect to eligibility for pensions, access to resources, or other benefits. In this section, we outline three partial measurement approaches in terms of the ICF model of disability. Each approach has its virtues and drawbacks which we shall review. We conclude with what we call the 'comprehensive approach' in which the disability question set covers all dimensions of disability; and in terms of a set of activity and participation questions, produces a summary result that can be used to determine disability prevalence. The advantages of this approach will then be reviewed.

4.3.1 Impairment Approach This approach relies entirely on the Body Functions and Structures component of the ICF, the dimension of disability concerned with healthrelated problems in, to quote the ICF, “physiological functions of body systems, including psychological functions [and] structural or anatomical parts of the body”. The impairment approach would be appropriate if the aim of the data collection was to gather health data about the direct consequences of diseases, disorders and injuries. Such data can be used for tracking health service usages and need for medical and rehabilitative services. For determining disability prevalence what is commonly done is to add together positive response rates for each of the sets of impairments given to produce an overall 'prevalence' rate.

- 61 -

WHO/ESCAP Training Manual on Disability Statistics

Impairment measurement of disability is very common. It is straightforward and, on naïve and purely medical conceptions of disability, appears to capture all the data that one would need to estimate prevalence. Other than the challenge of determining which impairments to include and which to ignore, it is a simple approach to use for developing questions. An example of an impairment-oriented question is this from a Philippines census: Do you have any long-term impairments? Check all that apply: ‰ ‰ ‰ ‰ ‰ ‰ ‰ ‰ ‰ ‰ ‰ ‰ ‰ ‰ ‰ ‰ ‰ ‰ ‰

Missing limbs Unequal length of limbs Deformity of limbs Deformity of spine Joint/muscle pain Weakness/paralysis of limbs Impairment of sensation Abnormality in limb tone Abnormal movement of limb Weakness/paralysis of face Impairment of bowel/urinary control Impotence Hearing disorders Speech disorders Visual disorders Disfigurement Chronic respiratory disorders Mental impairments Others. Please specify

Other forms of impairment questions focus on a small set of severe impairments, such as blindness or deafness, total paralysis, or severe mental illness. This type of question has been used extensively in population censuses across the developing world. Only a very small number of people (usually less than 2% of the population) can be identified using this approach. Although the data collected from the impairment approach has important uses, as far as disability is concerned, the approach is severely limited, and indeed, leads to a highly distorted picture of disability at the population level. Disability is ultimately a matter of the lived experience of a person with a decrement in health, how they get on with their life, perform actions and tasks, and fulfil simple and complex social roles. As questions about impairment strike respondents as being about their medical condition, rather than their day-to-day lives, there is a tendency for impairments to be under-reported. An individual with a slight hearing disorder may think the impairment is not worth mentioning, or may even forget that they have the problem.

- 62 -

WHO/ESCAP Training Manual on Disability Statistics

However, even accurate information about incidence and prevalence of impairments will not tell us how the impairment affects people's lives. We cannot directly infer from the presence of an impairment, even if severe, the extent to which the person's life is actually affected by the impairment. The same degree of impairment in two individuals living with different demands and different environmental contexts may produce entirely different levels of disability. Not only do decrements in body functioning affect the person's capacity to perform actions differently, in different environments people's performance may be radically different. We can only understand disability itself if we have information about what a person can do, and what they actually do in their day-to-day environment. This, of course, is information about activities and participation. 4.3.2 Activity and Participation Approach The domains in the ICF Activity and Participation classification cover the full range of behaviours, actions, tasks, simple and complex activities, and roles that make up all of a person's active life, alone and in society. The Activity and Participation classification includes Activities of Daily Living (ADLs) such as the personal care activities of bathing, dressing, getting in and out of bed or a chair, using the toilet and eating, as well as Instrumental Activities of Daily Living (IADLs) or activities related to independent living such as preparing meals, managing money, shopping, performing housework, and using the telephone. In addition, the Activity and Participation classification adds the complex activities in interpersonal interactions and relationships, and major life areas such as school, work, family, community, social and civic life. Integral to the ICF is the view that disability is essentially and centrally a matter of how the interaction between a person's health and their personal and environmental context affects the person's life and living. In order to gather information about disability, therefore, it is essential to ask questions about these domains of living. That is the strength of the Activity and Participation Approach to measuring disability. Activity and Participation questions provide data about health-related functional limitations in a person’s ability to perform actions in one or another domain. Such data is essential to estimate the level of need for assistive devices, modifications of environments, training, or rehabilitative services to increase capacity. These questions give information about the day-to-day lives of persons with disabilities, information directly relevant to the disability experience, from an overall social, rather than merely medical or rehabilitative, point of view. In turn, these data are relevant to all aspects of disability social policy, from employment and education policy, to communication, transportation and social security. As a rule, Activity and Participation questions yield higher rates of disability prevalence compared to impairment questions. In part, this is because most impairments, if they affect any activity, tend to affect more than one. More

- 63 -

WHO/ESCAP Training Manual on Disability Statistics

importantly, prevalence is higher because activity and participation domains circumscribe an individual’s direct experience with disability. While people may not be completely aware of their impairments (such as those of the metabolic or endocrine systems, for example), people will usually be all too aware of difficulties they experience in these domains. Here are two examples of Activity and Participation questions. The first uses an ADL checklist to capture a portrait of a person's day-to-day disability experience: Is this person limited in his/her daily activities (at home, at work, at school) because of a long-term physical or mental condition (lasting six months or more)? No Yes -- Check all that apply: ‰ ‰ ‰ ‰ ‰ ‰ ‰ ‰ ‰

Seeing (even with glasses, if worn) Hearing (even with hearing aid, if used) Communicating (talking, conveying information, listening) Moving/mobility (walking, climbing stairs, standing) Body movements (reaching, crouching, kneeling) Gripping (using fingers to grip or handle objects) Learning (intellectual difficulties, retardation) Behaviour (psychological, emotional problems) Other. Please specify

The second question probes the particular domain of employment: Because of the condition that you have told me about, do you have any difficulties with employment such as these? No Yes -- Check all that apply: ‰ ‰ ‰ ‰

Restricted in type of work can do Restricted in the number of hours can work Difficult to change jobs or get a better job Other. Please specify

The strength of the Activity and Participation Approach to disability measurement is the range of questions about real-life domains of activity that it allows to be included in surveys and censuses.

- 64 -

WHO/ESCAP Training Manual on Disability Statistics

Impairment questions, no matter how long the list provided, will never be able to identify, or measure, functional difficulties that are experienced by persons with disabilities across the complete spectrum of health conditions. People with vastly different impairments may well experience the same kind of difficulty in their day to day life: a person who has depression may well share with the person who has a mobility problem, or a communication difficulty, difficulties in getting and keeping a job. This feature of disability can only be captured by asking questions about what is at the core of the concept, namely difficulties a person has in their day-to-day life. There is, however, an additional dimension of information that can be only caught by means of the Activity and Participation classification. This data is derived from the ICF constructs of capacity and performance (represented within the Activity and Participation classification by the qualifiers). For any domain of ADL, IADL or major life area, it is possible to ask whether a person's health condition includes a limitation in the person's inherent capacity to perform the required actions, or whether in the person's actual environment, there is a restriction in the performance of the actions. Information about capacity is information about the person's functioning, and so a matter of his or her state of health, whereas information about the person's actual performance goes beyond the person's intrinsic health state to consider the positive or negative impact of the physical, social and attitudinal environment on what they actually do. Strictly speaking, an objective measure of a person’s capacity to listen, read, speak, walk, carry objects, and so on, can only be determined in a standardised environment in which the person’s actual, inherent capacity is measured, not the effect of environmental facilitators (such as personal assistance or an assistive device) or barriers (such as stigmatising attitudes). In addition, because it is a difficult matter to determine one’s capacity to perform the complex activities required in working, going to school, being a parent, or being a citizen, activity limitation questions usually deal with ADL or IADL actions -- simple actions or simple clusters of action involved in sensing, learning, communication, mobility, self-care and domestic life. Still, with careful phrasing the distinction between capacity and performance can be captured in self-report questions. The aim is to direct the respondent's attention to both areas. The respondent is firstly asked to consider features of their own capacity to perform actions (capacity). They are then asked to consider in their actual environment, whether, in light of their inherent capacity, they have more or less difficulty performing the actions than would be expected. For the ADL of washing oneself, for example, a capacity question might be: In your present state of health, how much difficulty do you have washing yourself, without assistance?

- 65 -

WHO/ESCAP Training Manual on Disability Statistics

Whereas a question about the same domain, but from the perspective of performance, might be: In your own home, how much of a problem do you actually have washing yourself?

Box 4.4 shows the different methodologies used by nine countries in Asia and the Pacific. Only Fiji reported that the disability data were collected using the Activity and Participation Approach while Thailand used the Activity Limitation Approach. The rest of the countries used the Impairment Approach. The strengths of the Activity and Participation Approach for disability statistics are obvious. But there are challenges as well. The choice of impairments to use in a question can be motivated by available national and international health datasets about incidence and prevalence of diseases and other health conditions which involve those impairments. It is not so easy to determine which domains of the Activity and Participation classification should be used in the questions.

Box 4.4: Disability Data Collections Approach Employed in Selected Countries Country Cambodia Fiji Islands

Hong Kong, China India

Indonesia Islamic Republic of Iran Mongolia

Philippines

Method of Data Collection

Approach Used

Social Economic Surveys and Population Census Census of Population and Household Income and Expenditure Survey

Impairment-based questions focusing on “type” and “cause” of disability Three-question short set includes one activity and participation question referring to difficulties with communication and “other usual activities” Impairment-based questions and medical diagnosis; mental disability is not included in reported prevalence rate Impairment-based categorical model

Survey on Persons with Disabilities and Chronic Diseases Census, National Sample Survey Organization Survey, Voluntary Registry National Socio-Economic Surveys Census

Limited registry in capital city (voluntary, impairment-based registry) Population and Housing Censuses

- 66 -

Impairment model Yes/No question on physical impairment or handicap, impairment/categorical model to identify type of disability Voluntary, impairment-based registry

Impairment/categorical model; yes/no question on “physical or mental disability”

WHO/ESCAP Training Manual on Disability Statistics

Thailand

Disability Surveys, Bureau of Empowerment for Persons with Disabilities National Registry, and Censuses

Impairment and activity limitation questions

Ideally, extensive field testing across a sufficiently rich pool of items would be required to guide a data collection developer in the construction of a parsimonious list of essential Activity and Participation domains for data collection purposes. Finding such a list, however, is both an expensive and highly technical endeavour far beyond the resources of many countries in the region. The second challenge would be to devise a weighted summation of the scores on the Activity and Participation questions so that a determination of disability status can be interpreted from the responses to the questions. The impairment approach merely adds together the positive responses, treating the occurrence of any impairment equally. The same may not be true of Activity and Participation questions, since some difficulties (e.g., being employed) might be viewed as far more important for disability status than others (e.g., having difficulties grasping). Although the Activity and Participation Approach, its challenges notwithstanding, is far preferable than the purely Impairment Approach, it too has its limitations. Specifically, the approach does not provide us with enough information to account for the kind and degree of difficulty that responses to the questions would indicate. Even if capacity and performance questions are asked, the answers will still not reveal what has brought about the difficulty. The Activity and Participation Approach misses information, not only about impairments, but also about, the environment in which the actions and tasks are, or are not, satisfactorily performed. The Environmental Factors Approach, discussed below, aims to provide that data.

4.3.3 Environmental Factors Approach Environmental Factor questions ask, not about the disability itself, but about factors that make up the physical, social, and attitudinal environment in which people live and conduct their lives, and which in turn affect the presence, or the severity of the disability. On the ICF model of the disablement process, environmental factors can either facilitate functioning (hence alleviating or eliminating the functional difficulty) or can hinder functioning (hence exacerbating a functional problem or creating one). There are many uses for these kinds of questions. They provide information on needs for assistive devices or accommodations; on kinds of physical and attitudinal barriers that people with various kinds of impairments face in everyday life; and on the effect on participation in a domain of the introduction of barrier-removal policies (for example, anti-discrimination law) or facilitating

- 67 -

WHO/ESCAP Training Manual on Disability Statistics

policies (for example, policies to support accommodation at the workplace or school environment). Here is an example of an Environmental Factors question that collects data about the use of aids and equipment: Do you use any of these aids to help you move around? ‰ ‰ ‰ ‰ ‰ ‰ ‰ ‰

Canes Crutches Walking frames Walking sticks Wheelchair (manual or electric) Scooter Specially modified car or car aids Other. Please specify

Here is an example of an Environmental Factors question that asks about personal assistance: Do you receive assistance from any of these sources? ‰ ‰ ‰ ‰

Nurse Home care worker Voluntary worker Other. Please specify

This is another example asking information on external barriers of a person with disability: How much of a problem did you have because of barriers or hindrances in the world around you? Environmental Factors questions are less common than other kinds, but more and more national surveys are using them. Although they cannot be used to estimate disability prevalence, we know from the ICF that a person’s environment is an essential factor in the existence and severity of the participation restrictions they confront. Although these questions are generally not used for estimates of prevalence of disability, they can provide very useful information about the experiences of person with disability. The most common forms of environment questions relate to need for and receipt of assistance, including elements from the Products and Technology, Support and Relationships and Services, Systems and Policies chapters of Environmental Factors of the ICF. 4.3.4 Comprehensive Approach As outlined earlier, it is clear that, on their own, each of the approaches will be unable to create a disability question set for any data collection method that will adequately provide the data needed for a valid measurement of disability prevalence. Each approach provides relevant data, yet only a comprehensive - 68 -

WHO/ESCAP Training Manual on Disability Statistics

approach – in which all dimensions of disability, including Environmental Factors, are brought together – will provide valid measurement data. As we have discussed in Chapter 2, the model of disability given in the ICF makes it clear that valid disability prevalence data must arise from information on actual restrictions in an individual's performance of actions, tasks, and social roles in the actual physical, social, and attitudinal environment in which that person lives. This is the core data that determines disability prevalence. In addition, and in order to understand disability prevalence, data on impairments and limitations in a person's capacity to perform activities and tasks attributable to a health condition, on the one hand, and the facilitating or hindering features of the person's context on the other – both in terms of personal and environmental factors – is needed. To explain disability in an individual case, and to explain disability prevalence across a population, these data are essential. Therefore, a comprehensive approach to the development of a disability question set would have Activity and Participation questions at its core, combined with Impairment and Environmental Factor questions. Optimally, and for practical reasons, it is essential that each component be composed of questions about a well-tested minimal or parsimonious set of domains: there are far too many domains in the three classifications of the ICF for all of them to be used in a feasible survey/census/administrative-based data collection. An example of the Comprehensive Approach would be a combination of the Activity and Participation questions found in the WHO Disability Assessment Schedule II (WHO DAS II), coupled with the set of impairment questions used in the WHO World Health Survey. The relevant domains are as follows: WHO DAS II (Activities and Participation) DOMAIN 1 Understanding and Communicating (concentrating, remembering, analysing and finding solution to problems…) DOMAIN 2 Getting Around (standing, standing up, moving around inside your home…) DOMAIN 3 Self-Care (washing, getting dressed, eating…) DOMAIN 4 Getting along with people (dealing with people you do not know, maintaining a friendship…) DOMAIN 5 Life Activities (household responsibilities, getting household work done, doing work/school tasks well…) DOMAIN 6 Participation in Society (joining in community activities, doing things by yourself for relaxation or pleasure…)

- 69 -

WHO/ESCAP Training Manual on Disability Statistics

World Health Survey (Impairment elements) Aches and pains Discomfort Skin defect Using hands and fingers Seeing and recognizing a person Seeing and recognizing at arm's length Hearing someone talking Hearing what is said in conversation Urinating Defecating Shortness of breath at rest Shortness of breath with mild exercise Coughing or wheezing Sleeping Feeling sad Worry or anxiety

4.3.5 Advantages of the Comprehensive Approach The advantage of using questions from the WHO DAS II and the World Health Survey is that both were developed in multiple-year, international collaborative ventures. They have been exhaustively field tested and have been used in thousands of interviews. WHO DAS II questions have proven psychometric qualities (sensitivity 3 , specificity 4 , reliability and validity) demonstrated in general population surveys, in clinical sensitivity-to-change studies and costeffectiveness studies. Impairment or Activity and Participation Approaches in practice tend to assume what is sometimes called the 'minority model of disability', namely that disability status is dichotomous – everyone is either a person with a disability or not. The ICF rejects this approach in favour of a universal model, in which disability is a continuous state, which everyone shares. That is, everyone has some degree of functional limitation at the body, person or societal levels, people differ in the extent and range of functional limitation. The comprehensive approach is multidimensional and covers the full range of the disability experience. It does not directly yield a fixed prevalence value, nor does it directly identify the set of persons with disabilities. Rather, it provides data that can be analysed along a continuum. That is, prevalence depends on threshold decisions concerning where – on a continuum of difficulty or severity – to draw the line that separates disabled from nondisabled. Once such a threshold is established, prevalence can be determined on the basis of the data derived from the data collection activity. 3 Sensitivity: the probability that someone with a disability (according to a predetermined standard) will also identify themselves as having a disability according to the survey/census question. 4 Specificity: the probability that someone who does not have a disability (according to a predetermined standard) will also identify themselves as not having a disability according to the survey/census question. Detailed discussions about sensitivity and specificity are in Chapter 6.

- 70 -

WHO/ESCAP Training Manual on Disability Statistics

Diagram 4.1 graphically shows how threshold decisions can be used to produce a range of, in this case, age-distributed prevalence data sets. The results are from a 2003 Mexican survey, held in conjunction with WHO's World Health Survey. Four lines are displayed, representing age-distributed prevalence rates based on four severity threshold decisions: 65 percent, 80 percent, 90 percent, and 95 percent. As would be expected, age-related prevalence differs depending on level of severity: the lower the threshold, the higher the prevalence. In addition, the shapes of the lines differ, representing differences in how age-groups experience different levels of functional limitation. What is significant about this way of portraying disability prevalence is that the threshold decision represents a separate decision during analysis, and the data itself does not force or mandate that a particular threshold be used. This is important because decisions about thresholds are highly political. They have direct financial consequences when programming eligibility is determined by level of severity. A policy planner will need to know how severe the problem is (i.e., the level of prevalence), and can employ prevalence decisions to gauge the level of access to pensions, insurance, or other disability programme. The comprehensive, multidimensional approach to disability questions makes it clear that threshold decisions must be independently justified and are not simple consequences of prevalence rates. These decisions are politically important and should be transparently made, in light of social conditions, the state of the economy, or other issues that ought to be a matter of public knowledge and debate. Obviously, threshold decisions will always be controversial. Where the line is drawn between who qualifies as having a disability for the purpose of a social programme and who does not, will directly affect the lives of people. These decisions should not be disguised as statistical or technical matters. A comprehensive approach to disability statistics ensures that political and social issues remain in the political and social sphere, where they belong, while at the same time providing valid and reliable data that is directly relevant to decision-making.

- 71 -

WHO/ESCAP Training Manual on Disability Statistics

Diagram 4.1: Mexican Disability Evaluation Survey, 2003

100

90

80

70

65% threshold

90% threshold

60

50

95% threshold

40

30

80% threshold

20

10

0 20-24

25-29

30-34

35-39

40-44

45-49

50-54

per 60

55-59

per 80

- 72 -

per 90

60-64

per 95

65-69

70-74

75-79

80-84

85 y >

WHO/ESCAP Training Manual on Disability Statistics

4.4

Question formats and response scales

In addition to the content of the questions, there is, of course, the question of form and format. In this section we look at some basic formats for questions before turning to the more challenging issue of the uses of response scaling in disability statistics. 4.4.1 Simple, list, multiple choices, open-ended, closed Disability questions can take the form of a simple response, a list, or a multiple choices (or composite) question. Single response questions ask a single question involving a single concept, and offer simple response categories (‘yes’, ‘no’, ‘sometimes’) with only one output item. List questions (like the ones in the impairment approach above) pose a single question followed by a list of items, with responses recorded for each item. List questions can allow for multiple output items. Multiple choice or composite questions, on the other hand, ask about a string of characteristics, but allow a single, non-itemised response. The disability screen question of the New Zealand Census is an example: Does this person have any difficulty hearing, seeing, communicating, walking, climbing stairs, bending, learning or doing any similar activities? Yes, sometimes Yes, often No

For most uses of disability data, itemised responses are preferred to composite questions, as the value of much of the data is lost in non-itemised responses. Sometimes questions are used to lay out options for more detailed questions that follow, and in that case a list question makes sense and should be used. Open-ended questions are those that invite the respondent to reply without providing a set of options for selection or affirmation. Open-ended questions are most commonly used in qualitative research where it is deemed important to let the respondent use their own words to respond to a question and probe. In survey or census context, this luxury is not often possible and instead closed questions are asked, where one or more response option is specified and the respondent must select one or more of these. 4.4.2 Scaled response questions It is sometimes important to get information, not just about the existence of a disability (a dichotomous or ‘yes/no’ response), but also its severity. To do that, the question needs to use scaled response categories so that the respondent can select the appropriate level of difficulty experienced. As a rule, scaled response questions increase the number of positive responses, since the respondent has more choices that may fit their situation. For this reason, scaled responses are preferable to dichotomous ‘Yes/No’ responses in the identification of person with disability.

- 73 -

WHO/ESCAP Training Manual on Disability Statistics

This issue was the subject of a test for the 2001 Australian Census. The two questions tested were: A.

Does the person have difficulty in: doing everyday activities such as eating, showering or dressing? Yes/No hearing

Yes/No

… B.

How much difficulty does the person have in: doing everyday activities such as eating, showering or dressing?

None/A little/A lot

hearing

None/A little/A lot

…

The disability prevalence rate from question A was 15%, while that for question B was 22.4%. The tests were conducted in the same fashion in similar areas, the only difference being the format of the question. Given a scaled option, people seem much more likely to recognise difficulty with activities than with a dichotomous, ‘Yes/No’ response since respondents tend not to report mild, or even moderate disabilities, mentally reserving ‘yes’ for severe disabilities. There is a strong motivation, therefore, for data collection designers to include scaling options as responses. The severity or 'extent of' qualifiers used in the ICF offer a 5-point scale that is available for use as scaled responses in survey/census questions (and is used in the WHO DAS II): Table 4.1: Value of First Qualifiers in the ICF xxx.0

NO problem

xxx.1

MILD problem

xxx.2

MODERATE problem

xxx.3

SEVERE problem

xxx.4

COMPLETE problem

(none, absent, negligible,… )

0-4 %

(slight, low,…)

5-24 %

(medium, fair,...)

25-49 %

(high, extreme, …)

50-95 %

(total,…)

96-100 %

This five-point scale may be the most intuitive one for any data collection activity since it identifies and distinguishes the primary empirical values of an observable phenomenon: its presence and absence, and its presence in low, medium and high

- 74 -

WHO/ESCAP Training Manual on Disability Statistics

degrees. The ICF scale is also highly flexible as it can be easily mapped onto other intuitive scales that might be used to identify classes of responses. Diagram 4.2 shows how this mapping could be done for the examples of 2 (or dichotomous), 3, 4, 6, and 7-point response scales. The five-point scale used in the ICF provides a convenient and consistent technique for determining the extent or magnitude of the functional problem being codified. The qualitative description of the five points, and the associated percent scale, are heuristic suggestions for how to calibrate these levels. Different users may find it possible to calibrate these levels more precisely, depending on the requirements of the users. Whether ICF’s scaling response options are used or not, the data from these responses is only useful if the response options can be calibrated against a common metric or standard. In order words, it is not helpful that two respondents both answer that their disabilities are ‘mild’, if one respondent’s mental representation of ‘mild’ is more or less equivalent to what the other respondent would call ‘moderate’. Whenever scaled response questions are relied on in a disability survey/census, some technique must be used to transform the qualitative scale into a quantitative scale. This is called response calibration. There are several successful strategies that can be employed to calibrate response options to insure consistency of data. The primary focus of these strategies is to devise ways of linking levels of ‘difficulty’ with the response options available. The details of these strategies, and the levels of confidence that can be achieved in different circumstances, are beyond the scope of this manual. 4.5

Additional issues in question development

4.5.1 Language If you look at censuses in the Asia-Pacific region, the disability question most commonly asked is whether anyone in the household has “a disability”, although sometimes the question asks about “physical or mental disabilities”, and sometimes “handicap”, “impairment”, “impairments, disabilities or handicaps”. We know that when these general words are used, the results underestimate prevalence. The reason is that people object to being labelled or categorized – being ‘disabled’ or being ‘handicapped’ – and tend not to answer positively. Obviously, it is a bad practice to use words that offend people. But it is also bad science. If people do not – or will not – identify themselves and the problems in their lives by the language you use in questions, then your data will be useless. A person may flatly refuse or ignore a question that asked about being ‘cripple’ or ‘dumb’, even if they would readily answer a question about difficulties in walking or carrying on a conversation. Table 4.2 gives some examples of offensive or negative terms in English, with reasons why it is best to avoid them.

- 75 -

WHO/ESCAP Training Manual on Disability Statistics

Diagram 4.2: Response Scales

Complete

Complete

95-100%

Severe

Unable

A Lot

50-95%

Complete

Severe

? Yes

Moderate

Some

Moderate

25-49%

Some

Mild

? ?

? ?

Mild

5-24%

None 0-4%

No

ICF

2-point

None

3-point

None

4-point

- 76 -

None

5-point

6-point

7-point

WHO/ESCAP Training Manual on Disability Statistics

Table 4.2: Negative Terms Relating to Disability Term

Reason to avoid term

“Able-bodied,” “normal" or "ordinary" people vs “abnormal”people

Creates an unwarranted dichotomy: everyone has (or will have) some level of disability and everyone is, to some degree, able-bodied, normal or ordinary in some respect.

Disabled persons

Handicapped

The disabled

Your ‘problem’ Afflicted/suffering Cripple, mongol, dwarf, spastic, epileptic, deaf/ mute insane, crazy mental retardation Deaf and dumb Confined to a Wheelchair

This phrase emphasizes the disability rather than the person. A more acceptable alternative is "person with a disability". (Although, in England and throughout English-speaking Europe, disabled person is preferred. In the academic discipline of disability studies, “disabled persons” is preferred as it connotes that persons are disabled by society.) This term has a submissive "cap in hand" connotation which is offensive. ‘Handicap’ has long been used (e.g., in the ICIDH) to refer to the effects of an unaccommodating environment on persons with impairments. More recently, however, the term has regained its previous, more stigmatising connotation. Though historically, persons with disabilities used this term to raise their collective political voice, others object to being put into a broad and negative category. It is both more acceptable and more informative to describe people specifically, as, for example, a person with mobility difficulty, or intellectual disability. . Assumes that the disability is an individual problem, rather than one also created by the social environment. This also connotes that one is expected to fix the problem. These words reinforce the stereotype of a person with a disability as a victim or unfortunate sufferer. These words are examples of negative labelling, in which the whole person is regarded as those who are something wrong. It is better to refer to "a person with a physical disability", "a person with Down Syndrome", and so on. In addition, these terms are often used inappropriately as negative and often derogatory generalizations rather than as descriptions of specific impairments. Inability to hear and speak doesn't mean a person is stupid, as "dumb" suggests. Say "uses a wheelchair". ‘Confined’ implies suffering and victim status.

Language is often at the forefront of battles by advocacy groups to change social perceptions of marginalised people. Question designers must be aware that people will resist being labelled and slotted into boxes. The appropriateness of the language of interview questions should be checked in pre-testing, in consultation with persons with disabilities and amended where necessary. In the 2001 Australian Census, testing indicated that the term ‘mental condition’ in the disability screening question

- 77 -

WHO/ESCAP Training Manual on Disability Statistics

significantly reduced the number of positive responses, especially among older people. Focus group testing indicated that grouping together mental conditions with health or physical conditions offended people who refused to include themselves in that combined category. The ICF uses language that is as non-offensive and neutral as possible, and completely avoids insulting terms such as ‘handicap’ or ‘idiot’. More importantly, the ICF makes it clear why survey/census questions should stay away from the term ‘disability’, even when people are not offended by it. The problem is ambiguity. ‘Disability’ has many meanings and uses in everyday language. Most of these meanings fit into the three dimensions of the ICF: 1. problems with the body (impairments), 2. problems with a person’s capacity to do actions (activity limitations), and 3. problems a person has, in the environment, in performing actions and major life roles. When ‘disability’ is used, it is unclear to the respondent which dimension they are being asked about. That is a problem. It is far better to design the survey/census question explicitly to capture data about the dimension, or dimensions of disability of interest, and then ask questions that unambiguously elicit that information.

4.5.2 Gender issues Another unexpected consequence of survey/census screeners is that differences in the percentage of males and females with disabilities may be affected by whether impairment or activity or participation questions are used to identify persons with disabilities. As a rule, when a disability question is asked, the male to female ratios of the percentage with disabilities are close to 1, indicating that the same proportion of men and women self-identify as part of the disability population. By contrast, when impairment questions are used, the result is often a male-female percentage of disabilities ratios greater than 1.0, suggesting that more men self-identify as disabled. This may happen because the impairment approach often focuses on the more severe impairments, which tend to have higher rates for men. Disability questions should therefore be checked for gender bias. Bias need not be intentional; it can arise indirectly. An example of this is the 2001 Sri Lankan Census which defined persons with disabilities as those who are “unable or have limitations to carry on their own work, without the help of others, due to congenital or long term physical/mental disability…”. Were women who worked at home without pay” included in this description? The sex ratio of persons with disabilities reported from the 2001 census in Sri Lanka was 136.2, suggesting higher disability prevalence in men than in women. Unfortunately, since testing by focus group and cognitive interviewing was not done, we cannot be sure that the census questions were gender biased or not.

- 78 -

WHO/ESCAP Training Manual on Disability Statistics

4.5.3 Cultural issues Cultural differences in the perception of disability affect how it is reported. For international statistics, this is an issue of great importance. As has been mentioned, during its development, the ICF has undergone extensive cross-cultural applicability research to ensure that the concept of disability is culturally accessible, and that the meaning and significance of specific categories of impairments, activity, and participation are preserved through translation. There remain, however, cultural issues that are more difficult to solve. For example, whether someone reports having a difficulty with an activity will depend on their cultural expectations about whether they should be performing that activity or not. For example, women in some cultures would not be expected to work, but their failure to do so is not a disability. Other cultures are unwilling to report information about family members – because, for example, disability is viewed as a punishment for past sins or otherwise shameful. Sometimes there is a culturally based sensitivity on the part of men to answer questions about women in their household, or the other way around. The mere act of providing information to the government may be a problem for some cultures. Experience suggests that these cultural differences can be minimised in survey administration. It is recommended that: ¾ Respondents are informed about the potential benefits to themselves or others of high quality data about persons with disabilities. ¾ Associations for persons with disabilities are approached to support and promote the data collection and to explain to their members the need for the data and how it can benefit their lives. ¾ As discussed earlier, the language used in any data collection activity is acceptable and not offensive. ¾ Interviewers are trained to be aware of cultural differences and to conduct interviews in ways that encourage response. 4.5.4 Context of disability questions Disability prevalence rates may be affected by the context in which disability questions are embedded. For example, in the Australian Survey of Disability, Ageing and Carers, disability prevalence rates may have been affected by the fact that health status questions (from the Medical Outcomes Study 12 Item Short Form Health Survey) were asked before questions aimed at identifying the disability population. Perhaps focusing the respondent’s attention on their physical and mental health status immediately before asking about activity limitations may have led to more positive responses.

- 79 -

WHO/ESCAP Training Manual on Disability Statistics

4.6

Census format questions

Censuses are common tools for collecting a wide variety of data about the population of a country. Since they are used for many purposes, there is pressure to keep the number of questions low and their length short. Each of the approaches described in Section 4.3 above can be, and have been, used in the development of censuses. Instead, or in addition, a census may include a simple, one-line question asking whether the respondent has disabilities or whether disability is a reason for lack of employment or educational participation. Here are a few examples of questions that have been used in censuses to collect information about persons with disabilities: 4.6.1 Single general question Census disability questions that ask only a single question about disability are common in the Asia-Pacific region:

A.

Do you have any long-term disabilities or handicaps? No ‰ Yes ‰

B.

Is (name of person) limited in his/her daily activities (at home, at work or at school) because of a long-term physical or mental condition (lasting six months or more)? ‰ No ‰ Yes

C.

Do you have any difficulty hearing, seeing, communicating, walking, climbing stairs, bending, learning or doing any similar activities? ‰ Yes, sometimes ‰ Yes, often ‰ No

Although A, B, and C are similar, ‘Yes/No’ questions, there are important differences between them which result in very different populations being identified. As noted above, Question A is likely to pick up only a very small number of people since many people object to the labels of ‘disabled or handicapped’. Even if followed by a detailed question about types of impairments, activity limitations or participation restriction, unless people respond positively to the first question, they will not answer what follows. As a result, most useful disability questions do not use the term ‘disability’ in the question. Question B avoids the problems with A by being explicit about problems in daily activities. By adding that the limitations can occur anywhere in daily life – home, work or school – the question directs the respondent’s attention to severe or at least moderate disabilities. Also, the reference to ‘physical or mental condition’ makes it

- 80 -

WHO/ESCAP Training Manual on Disability Statistics

clear that the question is directed to disability, rather than other reasons why daily activities might be limited, such as gender discrimination, poverty, or lack of opportunity. Finally, Question C is a composite question, since a string of activities are listed, with a (slightly modified) ‘Yes/No’ response. This question has been used as a screening question to identify a sample population for a follow-up survey. The follow-up survey included much more detail about disability. In most cases, it is more useful to try to collect the detail rather than rely on a single, ‘Yes/No’ response. Roughly, the disability prevalence rates that one might expect from each of these questions are likely to be less than 2 percent for Question A, 8 -12 percent for B, and 20 percent for C. Results from New Zealand and Canada indicate that there are both false positive and negative responses to single general questions. In the case of Canada, 20 percent of people who responded 'yes' to the census questions were found not to have a disability after being asked a more rigorous set of questions, and 5 percent of those who answered 'no' to the census questions were found to in fact have a disability. This suggests that census questions provide at best a very crude indicator of disability prevalence. Mild disabilities, and disabilities among children and older people are especially under-reported in census questions. 4.6.2 Checklist questions Some countries have used the technique of setting out a short checklist of problems with regard to impairments, activity limitations, or participation restrictions. Here are two examples: A.

Does the person have any long-term impairments, disabilities or handicaps? No Yes - Check all that apply ‰ ‰ ‰ ‰ ‰ ‰ ‰ ‰

Incomplete use of legs, feet Incomplete use of arms, fingers Partial or total loss of sight even with glasses Partial or total loss of hearing Partial or total loss of speech Slow development/learning difficulties Behavioural problems/mental impairment Other. Please specify

- 81 -

WHO/ESCAP Training Manual on Disability Statistics

B.

How much difficulty does the person have in: Doing everyday activities such as eating, showering or dressing? Hearing? Learning, understanding or remembering things? Reading or seeing even with glasses? Walking, kneeling or climbing stairs? Living independently? Doing any other things people of the same age usually do (for example, working, studying, etc.) (Three scale response to each item: None, a little, a lot) What causes this difficulty for the person? ‰ ‰ ‰ ‰ ‰ ‰

Short term health condition (lasting less than 6 months) Long term health condition Disability Age Difficulty with English language Other cause – Please specify

A uses a checklist of impairments and B of activity limitations. The second part of B serves to identify positive responses that are associated with health conditions, and hence, true disabilities. These questions tend to obtain higher prevalence rates than simple screener questions, as respondents are forced to respond to each of the question items. 4.6.3 Including a response category on disability In a few national censuses – Myanmar, the Central African Republic, and Venezuela – the category of disability is added to an existing question on economic inactivity. In this way the respondent can cite disability as a reason for the inactivity (at par with being a homemaker, student or retired person). In a similar way, Mexico, Belize, and Cuba add disability as a possible reason for a child not attending school. This technique is unlikely to produce valid prevalence rates of disability in the population since the question is only asked of people who are experiencing economic inactivity or not going to school. It is perfectly possible to have a disability and not being restricted in these two domains. This kind of question also picks up on severe impairments associated with a complete participation restriction, rather than merely limited participation.

- 82 -

WHO/ESCAP Training Manual on Disability Statistics

4.7

International disability question sets

Below are outlined a range of suggestions for disability question short sets made by the Washington Group on Disability Statistics (WG), WHO and ESCAP. The reader will notice that establishment of a consensus on recommendations for operationalizing ICF is a minimal or core list of domains and questions. It should be noted that there are currently projects underway at both international (Washington Group) and regional (ESCAP) levels which aim to revise and improve these recommendations. 4.7.1 UN Guidelines for the Development of Disability Statistics The United Nations’ 2001 Guidelines and Principles for the Development of Disability Statistics, suggested the following ‘generic’ disability question for censuses: Because of a long-term physical or mental condition that has lasted or is expected to last six months or more, how much difficulty do you have? (check all that apply) ‰ ‰ ‰ ‰ ‰ ‰ ‰ ‰ ‰ ‰ ‰

Seeing (even with glasses, if worn) Hearing (even with hearing aid, if used) Speaking (talking) Moving/mobility (walking, climbing stairs, standing) Body movements (reaching, crouching, kneeling) Body movements (reaching, crouching, kneeling) Gripping (using fingers to grip or handle objects) Learning (intellectual difficulties, retardation) Behaviour (psychological, emotional problems) Personal care (bathing, dressing, feeding) Others. Please specify

If the respondent indicates one or more of the activity limitations in this list, then they are identified as having a disability for the purposes of the census. Compared to the cost of fielding a separate disability survey, including one or two questions on disability in a census questionnaire is a relatively inexpensive way of getting prevalence data on disability and type of disability. A major benefit from including disability in the census is that the data set can be analysed more extensively with other census variables (age, sex, marital status, income, labour force status, family status, and so on). In this way, the results for the disability population are easily comparable to those of the total population. These data can then be used for poverty analysis or detailed family analysis, in order to get a better picture of the situations of person with disability and their families. The UN global recommendations were updated in 2007 in the revised version of the Principles and Recommendations for Population and Housing Censuses (Rev. 2). Development of the UN recommendations on disability in censuses is based on the Washington Group’s suggestions.

- 83 -

WHO/ESCAP Training Manual on Disability Statistics

4.7.2 UN Global Census Recommendation After several discussions, the Washington Group on Disability Statistics (WG) which is an expert group to address the issues related to disability measurement and to report to UN Statistical Commission was able to devise the list of questions recommended for censuses below: Introductory phrase: The next questions ask about difficulties you may have doing certain activities because of a HEALTH PROBLEM. Question set: 1) Do you have difficulty seeing, even if wearing glasses? 2) Do you have difficulty hearing, even if using a hearing aid? 3) Do you have difficulty walking or climbing steps? 4) Do you have difficulty remembering or concentrating? 5) Do you have difficulty (with self-care such as) washing all over or dressing? 6) Using your usual (customary) language, do you have difficulty communicating, for example understanding or being understood?

Response categories: a. b. c. d.

No – no difficulty Yes – some difficulty Yes – a lot of difficulty Cannot do at all

WG performed pilot tests and analysis of the cognitive qualities of the questions, completed in 2006. The detailed recommendations and tabulation instructions are presented in Principles and Recommendations for Population and Housing Censuses (Rev. 2)5. 4.7.3 WHO/ESCAP’s pilot tested disability questions The WHO/ESCAP Project on Improvement for Disability Statistics and Measurement conducted pilot test of WHO, the Washington Group, and ABS question sets (see Appendix 2) in five countries, namely Fiji, India, Indonesia, Mongolia and the Philippines. The question sets from WHO for pilot test consisted of following modules: ‰

An impairment module derived from 17 ICF domains used in the WHO World Health Survey, for example ƒ ƒ ƒ ƒ

5

Seeing Hearing Pain Worry or anxiety

See: http://unstats.un.org/unsd/demographic/sources/census/census3.htm

- 84 -

WHO/ESCAP Training Manual on Disability Statistics

ƒ ‰

An activity and participation module based on the following six domains of the World Health Organization Disability Assessment Schedule II (WHO DAS II): ƒ ƒ ƒ ƒ ƒ ƒ

‰

Feeling sad or depressed

Understanding and communication Getting around Self care Getting along with people Life Activities Participation in Society

An environmental factors module which comprises the three “need for assistance” questions used by the Australian Bureau of Statistics in the 2006 census: ƒ ƒ ƒ

Need for assistance in self care activities Need for assistance in body movement activities Need for assistance in communication activities

Examples from the Washington Group, WHO, and ABS question sets are given in Table 4.3 below.

Table 4.3: Example of WG and WHO/ESCAP question sets

Impairment

Activity and Participation

Example of Washington Group question set

Example of WHO and ABS question sets

Do you have difficulty seeing, even if wearing glasses?

How much difficulty did you have in seeing and recognizing a person you know across the road? (take into account eye glasses, if you wear them)

Do you have difficulty walking or climbing steps?

How much difficulty did you have in walking a long distance such as a kilometre (or equivalent)? Do you ever need someone to help with, or be with you for, body movement activities?

Environmental Factors

For example: getting out of bed, moving around at home or at places away from home.

- 85 -

WHO/ESCAP Training Manual on Disability Statistics

The questions sets were comprehensively tested in five pilot countries of the ESCAP region using the following study protocols. The first study examined the sensitivity (ability to pick up true positive cases) and the specifity (ability to pick up true negative cases) of specified disability questions. The second study assessed the reliability of individual questions by a standard test-retest procedure. The aim was to flag unreliable questions for later attention in the production of a recommended question set. The third study aimed to record the respondents' understanding of the question's intent. The findings of the pilot test are summarized as follow: •

Validity

Generally, the construct validity of the Washington Group questions for the seeing, hearing, mobility and self care domains is good when WHO questions for the corresponding domains are used as a benchmark. This does not, however, apply to the questions for the cognition and communication domains. The Washington Group (WG) questions perform similar to corresponding WHO questions in terms of predictive validity6. The number of subjects reporting any difficulty in response to the combined WHO questions is larger than in response to the Washington Group questions for every domain. This is especially the case for the hearing, mobility and communication domains, where the combined WHO questions pick up more than twice the number of people with at least some difficulty than the Washington Group questions. Moreover, each individual WHO question for these domains, as well as the cognition domain, generates higher disability prevalence than the corresponding Washington Group question. WHO questions have five scale responses while WG questions have four choices. It could be that more scaled response questions increase the number of positive responses, since the respondent has more choices that may fit their situation. In terms of item properties, generally, the body functions and six domains (World Health Survey and WHO DAS II) questions that included thresholds (e.g., walking >1 km or seeing “across the road”) were more sensitive and therefore appear to result in less serious under-reporting. The WG Short Set of questions showed good specificity but low sensitivity. WHO DAS II domains like life activities, interpersonal relations, cognition, and mobility together with environmental and mental health related questions showed the highest power in explaining difficulties in participation. In terms of frequencies, within the WG set of questions, the domains of mobility, cognition, and vision had high endorsement rates. Body functions and the six domains had very high endorsement rates across the whole severity spectrum but in particular for Activity and Participation and Environmental Factor questions. 6

For details of empirical results and predictive validity, see Smit and Liu, 2007.

- 86 -

WHO/ESCAP Training Manual on Disability Statistics

Regarding utility, from the WG questions, one fixed prevalence rate (<5% for severe and complete difficulties, and >20% for any and mild difficulties) was generated. Body functions and the six domains allowed the production of multiple and scalable prevalence rates which in turn allowed the same data set to be used for various purposes. •

Reliability

All questions showed moderate to good reliability. Within a given domain differently phrased questions have similar levels of reliability. For example, Kappa statistics of W1 Seeing is 0.57; Kappa statistics of B1.6 Seeing short sighted is 0.65; and Kappa statistics of B1.7 Seeing long sighted is 0.53. •

Cognitive debriefing

Most questions were understood well. Approximately 95 percent of the respondents responded to the questions without any difficulties. The few cases where difficulties in question understanding were observed or reported involved questions that referred to mental health related domains (cognition, anxiety, and self esteem) and certain environmental factors. Respondents also had difficulties in understanding compound questions. Regarding the question wording, the inclusion of meaningful thresholds was found to be useful. Further studies were deemed necessary to determine the extent that translation difficulties influenced these outcomes. Some of the results of these pilot studies were presented at the 4th Workshop on Improving Disability Statistics held in Bangkok, Thailand (20-22 June, 2006). The workshop was attended by representatives from 16 countries in the ESCAP region. After discussing the results of the pilot studies, the participants at the workshop tentatively suggested that the following domains could be included in census. ¾ ¾ ¾ ¾ ¾ ¾

Walking Seeing Hearing Life Activities Self Care Communication

A working group was also formed at the workshop in order to follow up on remaining issues related to these tentative results through conducting more in-depth analysis of pilot study outputs. More detailed analysis of the results from the pilot studies is still ongoing by ESCAP, WG and WHO researchers and more work need to be done on development of the extended question sets for surveys, therefore complete recommendations on selecting among ICF domains is not yet concluded.

- 87 -

WHO/ESCAP Training Manual on Disability Statistics

4.7.4 Development of disability statistics from census/survey: An example In most countries, the development of disability statistics is a slow and step-by-step process. Once the need for disability data is identified and basic questions about coverage and scope are settled, the next task is to find the best data collection tool to use. This will be both an issue of statistical adequacy and of cost and feasibility. Most likely, for reasons already mentioned, the use of existing censuses, modified by the addition of a few disability questions, will be the first step. In time, these data can form the basis for a separate survey, dedicated to disability issues. Box 4.5 describes a hypothetical example of this process in action.

Box 4.5: Case Study of Middle Earth: Developing Disability Measures The situation Middle Earth is a land of groups of people at war, a war that has lasted for so many generations it seems to be entrenched. As a consequence of battle, many have severe impairments, such as lost limbs, severe hearing problems and burns to the skin. Middle Earth is a very poor land. Most inhabitants cannot afford to buy the aids and equipment that would overcome at least some of their disability, and the state lacks programmes for the provision of these. For example, some people have poor eyesight but simply cannot afford to buy glasses. The built environment of Middle Earth does not suit all of the people. Most buildings are suitable only for small ethnic groups and persons with disabilities often find that features of the built environment hinder their participation in a range of activities. Need for statistical information on disability In 1986, the state decided to collect information about the prevalence of disability in the population. They felt that information was needed for each of the country’s ethnic groups in order to understand both the prevalence and the experience of disability for each group. Planning for data collection Planning commenced immediately, with the intention of collecting disability data as soon as possible. Issues considered were: ¾ What collection vehicle to use ¾ How to define disability ¾ How to word the questions ¾ How aids and equipment should be taken into account ¾ How the environment should be addressed Collection of data It took several years to plan, but in 1990, the National Statistical Office (NSO) of Middle Earth included a single question on disability in the census: “Are you disabled?” with simple response categories of Yes/No. Middle Earth interviewers reported a widespread misunderstanding of the question and a reluctance to answer yes. The resulting estimates of disability for Middle Earth were less than 1%. Persons with disabilities and government disability policy staff did not believe the estimates from the census, and lobbied the NSO to improve the information. In the next Middle Earth Census in 1995, the NSO developed a disability question that was longer and collected more detail. The 1995 question covered a range of 6 impairments, mostly severe impairments such as blindness, deafness and loss of limbs. The key disability data users were happy to see disability prevalence increase to 2% of the population; but they were still concerned that the true picture of persons with disabilities was not being presented. In 2000, the NSO decided to conduct a dedicated survey of disability as well as the census. The

- 88 -

WHO/ESCAP Training Manual on Disability Statistics

census and survey questions used similar concepts of disability, based on the ICF. A combination of impairments, activity limitations and participation restrictions was used to identify the disability population in the broadest possible sense. Details of disabilities, including type, severity, cause and age of onset were collected in the dedicated survey. The census question focussed on activity limitations with the highest expected prevalence rates. The census question used 6 items, while the survey had a series of 15 screening questions. Disability estimates from the Census increased to 10% of the population, while the survey reported even higher rates of about 20%. Disability data users were very satisfied with the results of the 2000 census and survey program. Even though the estimates from the two sources were not identical, estimates of a similar concept (selected activity limitations) were similar, as were characteristics of the populations.

4.8

Administrative data collections, with an Australian example

Using ICF concepts in administrative data collections promotes data consistency and comparability. In developing data collections, and data items within collections, the ICF may be used in different ways, and at different levels of detail: ¾ The overall framework of the ICF can be used to guide the development of data items: what dimensions of functioning and disability should the data items cover – impairments, activity limitations or participation restrictions, or some combination? ¾ The classifications within each of the components of the ICF can be used as a ‘pick list’ from which relevant domains can be selected, either at broad chapter level, or at more detailed levels. ¾ ICF codes can be used as the basis for capturing and recording data. A good example of the use of ICF concepts in the collection of administrative data is that of the Australian experience developing a “support needs” data item. The new support needs question was developed as part of a major development of Australia’s data collection system. In consultation with stakeholders, it was agreed that a standard indicator was required to compare support needs of different clients at the national level. The data item is not a new assessment tool, but rather a framework into which assessment information already collected by agencies could be mapped. The data had to be comparable with disability population survey data, and it needed to relate to existing data standards (in particular, the ICF) and current practice in assessing support needs. Box 4.6 presents the data collection form that was used in Australia, with the new “needs support” Question 11. It can be seen that the question is set out: “How often does the consumer need personal help or supervision with activities or participation in the following life areas?” This is followed by nine life areas (based on chapter headings from the Activity and Participation component of the ICF) and a five-point scale for recording levels of supervision needed. As operationalised by this question, the concept of ‘need for help or supervision’ was perfectly compatible with measures used in the Australian Disability Population

- 89 -

WHO/ESCAP Training Manual on Disability Statistics

Survey, so that data from the two sources were consistent and could be used together. For analysis purposes, the nine activity areas where support is needed were then divided into three groups: ¾ Activities of Daily Living – self-care, mobility, and communication. ¾ Activities of Independent Living – interpersonal interactions and relationships; learning, applying knowledge, and general tasks and demands; and domestic life. ¾ Activities of Work, Education, and Community Living – education; community and economic life; and work. Data collected in 2002 could then be used to show some interesting facts about the support needs of persons identified as having a disability, based on category of activity area: ‰

42 percent always needed help or were unable to do Activities of Daily Living; 48 percent always needed help or were unable to do Activities of Independent Living;

‰

58 percent always needed help or were unable to do Activities of Work, Education, and Community Living;

‰

Those who used housing and accommodation support services had the highest support needs, while those using employment services had the lowest support needs;

‰

Persons with disabilities living with their family or spouse had higher support needs than those who lived alone;

‰

Among those living with family or spouse, 38 percent were unable to do or always needed help with Activities of Daily Living, 43 percent with Activities of Independent Living, and 48 percent with Activities of Work, Education, and Community Living;

‰

For persons with disabilities living alone, 17 percent were unable to do or always needed help with Activities of Daily Living, 23 percent with Activities of Independent Living, and 34 percent with Activities of Work, Education, and Community Living.

These conclusions are of considerable use in service planning and administration. These results were possible because data from the two administrative data sources were comparable since both rely on the ICF.

- 90 -

WHO/ESCAP Training Manual on Disability Statistics

Box 4.6: Australian National Minimum Dataset: Data Collection Form See Data Guide page 62 11. How often does the service user need personal help or supervision with activities or participation in the following life areas?

Please indicate the level of help or supervision required for each life area (rows a–i) by ticking only one level of help or supervision (columns 1–5). The person can undertake activities or participate in this life area with this level of personal help or supervision (or would require this level of help or supervision if the person currently helping were not available)

1) Unable to do or always needs help/ supervision in this life area

2) Sometimes needs help/ supervision in this life area

3) Does not need help/ supervision in this life area but uses aids or equipment

4) 5) Does not need Not help/ supervision applicable in this life area and does not use aids or equipment

LIFE AREA a) Self-care e.g. washing oneself, dressing, eating, toileting b) Mobility e.g. moving around the home and/or moving around away from home (including using public transport or driving a motor vehicle), getting in or out of bed or a chair c) Communication e.g. making self understood, in own native language or preferred method of communication if applicable, and understanding others d) Interpersonal interactions and relationships e.g. actions and behaviours that an individual does to make and keep friends and relationships, behaving within accepted limits, coping with feelings and emotions

1

2

3

4

1

2

3

4

1

2

3

4

1

2

3

4

NOTE: In the following questions ‘not applicable’ is a valid response only if the person is 0–4 years old. e) Learning, applying knowledge and general tasks and demands e.g. understanding new ideas, remembering, problem solving, decision making, paying attention, undertaking single or multiple tasks, carrying out daily routine

1

2

3

4

5

f) Education e.g. the actions, behaviours and tasks an individual performs at school, college, or any educational setting

1

2

3

4

5

g) Community (civic) and economic life e.g. recreation and leisure, religion and spirituality, human rights, political life and citizenship, economic life such as handling money

1

2

3

4

5

NOTE: In the following questions ‘not applicable’ is a valid response only if the person is 0–14 years old. h) Domestic life e.g. organising meals, cleaning, disposing of garbage, housekeeping, shopping, cooking, home maintenance

1

2

3

4

5

i) Working e.g. actions, behaviours and tasks to obtain and retain paid employment

1

2

3

4

5

- 91 -

WHO/ESCAP Training Manual on Disability Statistics

Learning Objectives: Chapter 5 Survey/Census and Administrative Data Collection Development: Planning and Measurement Issues After reading this chapter, the reader should be able to: 1.

Describe the planning cycle for censuses and surveys

2.

Outline issues in designing disability data collection

3.

Understand the issues in survey sampling design

“Blind moves” Main Uddin WHO Photo Contest “Images of Health and Disability 2002/2003”

- 92 -

WHO/ESCAP Training Manual on Disability Statistics

5.

SURVEY/CENSUS AND ADMINISTRATIVE-BASED DATA COLLECTION DEVELOPMENT: PLANNING AND MEASUREMENT ISSUES 5.1 Overview of the survey and census cycles 5.1.1 Survey cycle and costs 5.1.2 Census cycle 5.2

Planning for data collection

5.3

Asking the right questions

5.4 5.4.1 5.4.2 5.4.3

Determining data user needs Consult with data users Statistical advisory groups Consult persons with disabilities

5.5

Privacy implications of collecting data

5.6

Other basic management issues in planning

5.7

Quality of survey data and total survey error

5.8 5.8.1 5.8.2 5.8.3 5.8.4 5.8.5 5.8.6

Issues in collection design Context of disability questions Proxy versus non-proxy survey responses Face-to-face versus telephone interviewing Special questions for sub-populations Collection of disability data about children Collection of disability data about institutionalised people

5.9 5.9.1 5.9.2 5.9.3

Planning for administrative-based data collection Identifying information needs Developing data items Deciding on collection methods and materials

5.10 Sample Design 5.10.1 Sampling frame 5.10.2 Sample size and unit

- 93 -

WHO/ESCAP Training Manual on Disability Statistics

5.1

Overview of the survey and census cycles

5.1.1 Survey cycle and costs The development of most surveys follows the same cycle. Unless the survey has been conducted before, the survey cycle starts with the identification of the need for information by one or more clients or data user. This step is the most important since, without clear identification of user need, the purposes of the survey will be unclear and the development process will be flawed from the beginning. This starts the ongoing development process for data collection. User needs guide in the planning process that leads into development and design of the survey. When data is collected and processed, estimation of prevalence and other analyses begin. These are disseminated and evaluated by, among others, the data user. So the process comes full circle. Diagram 5.1 below illustrates the survey cycle. Diagram 5.1: The Survey Cycle

Data user Evaluation

Planning

Survey development

Dissemination SURVEY Analysis

Sample design Estimation/ Tabulation

Data Collection

Processing

Several of these phases, and in particular, those that raise issues of special concern to disability data collection, are discussed in this and the following chapters. Considerations of cost are always relevant to the development of surveys. National statistical offices must always be aware of whether the potential

- 94 -

WHO/ESCAP Training Manual on Disability Statistics

benefits of the survey compensate for the costs of developing a survey, putting it into the field, and collecting and analysing the data. National statistical agencies regularly record, not only total costs of data collection, but the costs of each phase. Cost information is essential for budgeting each phase of survey development within the organization, and may also be used to compare costs with those of other national statistical organizations. The most significant costs in any interviewer-administered data collection are the actual field costs to administer the survey and collect the data. The planning and development costs are significant and of course necessary to ensure high quality disability data. Costs of analysis and data dissemination are also major expenses. Usually, total development costs are roughly equal to total output related costs. 5.1.2 Census cycle The cycle of phases of censuses is similar to that of surveys, as Diagram 5.2 illustrates. In particular, the census cycle begins and ends with evaluation of previously collected data and user consultation. The major difference between the two is that far more time is required for all phases of census collection, especially for additional consultation, design and testing procedures, including topic selection, government endorsement of final design, and quality assurance. Diagram 5.2: The Census Cycle User consultation Topic selection Evaluation

Planning

Form design and testing

Dissemination of census products

Evaluation, Recommendations Government endorsement

CENSUS Analysis

Enumeration design Tabulation/ Results Output

Field Operations

Processing

Data capture Editing Imputation Quality Assurance

- 95 -

Mapping

WHO/ESCAP Training Manual on Disability Statistics

5.2

Planning for data collection

Given the complexity and cost of putting a survey or census into the field, and the necessity of having the data serve the needs of the data users, the planning stage is crucial. Some of the issues that must be addressed during the planning stage are: ¾ ¾ ¾ ¾ ¾ ¾ ¾ ¾

What are the objectives of the data collection? What are the questions to ask? What should be the scope and coverage? How should disability be operationalised? What should the screening questions look like? Should proxy interviews be used? Should we use telephone or face-to-face interviewing, or both? Should there be different questions or questionnaires for different sub-populations? ¾ How do we solve issues with the language and terminology we use? ¾ How do we solve basic methodology issues of sample design, sample frame, sample unit, stratification, weighting and so on? ¾ How do we preserve privacy and confidentiality or respondents’ response? Objectives should be stated as clearly as possible because they determine the scope, methodology and content of the survey/census, and ultimately, its statistical outputs. The first two of these questions can be ascertained through consultation with data users and those persons with disabilities. These will be discussed in detail in the next sections. The last question is discussed in Section 5.5. 5.3

Asking the right questions

Let’s take a look at a real case on how questions were formulated in designing a tool for gathering disability data. Consider an Australian survey conducted in 2003 called the Survey of Disability, Ageing and Carers (SDAC): From the outset it was agreed that there were fundamental gaps in information on three overlapping groups of people: persons with disabilities, older people, and those who provide care for other people because of their disabilities. What was needed was not only basic prevalence information, but also information that would identify unmet needs for persons with disabilities and the elderly -- in particular, health care and technical aids such as wheelchairs and hearing aids. For carers, what was needed was data on the effects of the caring role on their lifestyle.

- 96 -

WHO/ESCAP Training Manual on Disability Statistics

Data users expressed the need to identify changes over a period of time in the prevalence and characteristics of persons with disabilities. So the questions in the SDAC needed to be closely similar to the earlier survey conducted in 1998. Finally, in addition to the need for data at national and state/territory levels, for regional planning of disability services, users expressed the need for regional and small area data as well. The designers of SDAC sought answers to some very basic and common sense questions. ¾ ¾ ¾ ¾ ¾ ¾ ¾ ¾ ¾

Who needs the information? Why do the users need this data? Which populations should be surveyed? What questions need to be answered, and what information would answer them? What is the population of interest and what do we want to know about it? What is the geographical scope of the survey? Should our questions be statistically relatable to those in other surveys? When is the information needed, and how often does it need to be updated? What form should the results take, how much detail and crosstabulation are required?

During its discussions about a general disability measure for censuses, the Washington Group (WG) at its first meeting in 2002 identified the first two of these questions as particularly important for the task of finding out the purpose served by any disability data gathering instrument. The WG suggested the following possibilities: Who needs the information? ¾ ¾ ¾ ¾ ¾ ¾ ¾ ¾ ¾ ¾ ¾ ¾ ¾ ¾ ¾

Researchers The general public Policy makers Associations and NGO’s Trade unions Government agencies International organizations Health service organizations and providers Industry Device and equipment manufacturers Employers Disability consumer groups Insurance agencies Education planners Media

- 97 -

WHO/ESCAP Training Manual on Disability Statistics

Why do users need disability data? ¾ ¾ ¾ ¾ ¾ ¾ ¾ ¾ ¾

Planning Project and planning evaluation Marketing Policy development Advocacy and political action Prevention Tracking and enhancing participation Improving services Creating or improving standards

The answers to the other questions (which populations to survey, what to ask, geographical scope) were considered to be dependent on the needs of the data users in each individual country. This naturally leads to the question: How do we determine what the needs of disability information users actually are? In the next section we outline a series of practical steps that have been used with considerable success. 5.4

Determining user needs

Suppose your agency is planning to develop a survey relating to disability. There is a limit imposed by resources and the patience of respondents on the number of questions asked. However, you want all your questions to address the needs of the users. How should you proceed? 5.4.1 Consult with data users Data collections produced by statistical agencies that have not consulted, broadly and deeply, with those who would use the data are, in a word, useless. There are many techniques of consultation: ¾ Seek written submissions on data content and purpose of data ¾ Convene user groups to discuss data needs and survey/census issues ¾ After documenting data user needs, check back with the data users to confirm that they are correct. As there are limits to the number of questions that can be included in any data collection activity, not every data need can be addressed. The data collection developer should collect as much information as possible about users’ needs, and then check to see if some of these needs can be satisfied by alternative sources of data. Usually, it will be the difficult task of the data collector developer to prioritise needs, or to work out a compromise to satisfy as many users’ needs as possible.

- 98 -

WHO/ESCAP Training Manual on Disability Statistics

5.4.2 Statistical advisory groups It is a good practice to put into place an on-going and regular process of consultation through regular contact and discussion with the statistical user community. This establishes a rapport between data collectors and users, so that issues can be raised and addressed as they arise, leaving plenty of time for the data collection development process. The statistics user advisory group is one such mechanism. Advisory groups can be established for issues relating to a specific topic, or to provide advice about a data collection. Topic-based groups can meet two or three times a year to discuss emerging issues relevant to the survey/census topic. Collection-specific groups typically meet during the survey/census development stage. Membership in advisory groups should cover all stakeholders. In the case of disability survey/census, it should include representatives for all levels of government, academics and researchers, those with disabilities themselves and their advocacy groups. Box 5.1 describes the Australian terms of reference for its census advisory group for disability. Box 5.1: Australian Census Advisory Group on Disability The Australian Bureau of Statistics (ABS) convened the Census Advisory Group on Disability (CAGD) to advise it on the requirements for disability data from the 2006 Population and Housing Census in Australia. The terms of reference of the CAGD were: 1.

To identify and prioritise key requirements for disability data for small areas and for a range of population sub-groups, including relatively small subgroups.

2.

To guide the research, development, testing and validity analysis of (a) disability question (s) to provide acceptable disability data at a small area level for inclusion in the 2006 Census.

3.

To consider the results of testing and validation, and to recommend to the ABS action to be taken with respect to the inclusion of (a) disability question (s) to the 2006 Census.

The advice of this group assisted the ABS in the development of disability questions for the 2006 Census. During consultations, the group advised that the purpose of the disability questions in the census is to identify the disability population in need of assistance with the core activities of self-care, mobility and communication. The group argued that these data were the most important to collect at the fine geographic level available from the Census as they are different, but complementary to, the data derived from the disability survey.

- 99 -

WHO/ESCAP Training Manual on Disability Statistics

5.4.3 Consult persons with disabilities In addition to being important data users and those who most benefit from improved data, persons with disabilities and the organizations that represent them often know, better than anyone else, what data needs to be collected. They should be involved from the initial stage of collecting user needs and developing questions, through planning for information campaigns about survey/census goals and results. In the past, disability organizations have raised the following objections to surveys/censuses proposed by national and local governments: ¾ Not enough effort was made to use existing data to improve services; ¾ Survey/census question results seemed designed to make the case for more residential institutions, and other policies that go against the integration of persons with disabilities into society; ¾ Surveys/censuses violated the privacy of persons with disabilities, particularly those with psychiatric disabilities; ¾ Surveys/censuses were carried out without consultation with organizations of persons with disabilities. To help ensure that surveys and other disability data collecting tools are ‘fit for purpose’, it is therefore vital to engage in the process persons with disabilities and their organizations. This can be done in the following ways: ¾ Ensure that all advisory groups have representation from disability organizations. ¾ Provide copies of drafts of the survey/census objectives and draft questionnaires (in accessible formats and means) to disability groups. ¾ Invite disability groups to make submissions on the content of the survey/census or directly request them to provide advice on specific issues that arise in the survey/census development. All of these methods of consultation are worthwhile and help make a successful data collection. They help to ensure that the results will suit the needs of those disability data users who have direct experience of disability – persons with disabilities themselves.

- 100 -

WHO/ESCAP Training Manual on Disability Statistics

Box 5.2 provides some examples of countries that have consulted with disability groups when collecting disability data. Box 5.2: Consultation with Disability Groups: Examples New Zealand: During the development of the 1996 Household Disability Survey, the Deaf Foundation were concerned that the survey would not identify deaf people, because deaf people tend not to consider themselves as having a disability and so would not answer 'yes' to the screening questions in the Population Census (used to develop a sample frame for the survey). The Deaf Foundation had threatened to mount a media campaign to discredit the survey results. The government brought the Deaf Foundation into the development process and convinced them that the methodology they proposed would indeed identify deaf people. Thailand: The Thailand National Statistical Office (TNSO) had conducted five Health and Welfare Surveys since the 1970’s. Estimates of the percentage of the population with disability in Thailand had increased over this time from less than 0.5% to approximately 2% in 2001. The TNSO believed the true population of person with disability was still actually much higher than this. To improve their data, the Office consulted both the major users of disability data and persons with disabilities, who helped them to define disability more accurately. Australia: The Australian Bureau of Statistics has consulted with disability groups before each of its disability surveys, and consequently included many of the data requirements suggested by these groups. For example, the Head Injury Advisory Council lobbied the Bureau before the 1993 disability survey to include more information on people with head and brain injuries. They wanted to ensure that such individuals were identified in the general disability population and that the cause of their disability was correctly captured. The 1993 survey was changed to meet this need.

5.5

Privacy implications of collecting data

Prior to data collection, a plan has to be established as to how privacy and confidentiality will be guaranteed. These factors must be considered whenever data about individuals or service provider organizations are collected or disseminated. Privacy and confidentiality may be compromised by what information is collected, how it is collected (e.g., the manner in which the question is asked, and by whom), how the data is stored (e.g., who has access to the data and what protections are in place to stop unauthorised access), and how the data is analysed and reported. Convention on the Rights of Persons with Disabilities in its article 31 states the process of collecting and maintaining statistical data should comply with legally established safeguards, including legislation on data protection, to ensure confidentiality and respect for the privacy of persons with disabilities. Legislation or ministerial or departmental regulations may set out privacy requirements for data collections—it is important to be familiar with these rules as the development phase of the data collection begins. Even if no legislation or regulation exists, it is a good practice to follow the following principles:

- 101 -

WHO/ESCAP Training Manual on Disability Statistics

1. Agencies and other data-collecting organizations should inform respondents of the information they are collecting and the purposes and intended uses of that data. 2. The respondent’s consent – or in the case of someone unable to consent, the consent of a proxy – should always be obtained prior to collecting data, and the individual should be given the opportunity to refuse consent to the use or disclosure of information about them. 3. Agencies and other data-collecting organizations should give people reason to be confident that their privacy will be respected. Examples of Australian privacy legislation and guidelines relating to data collection can be accessed on the Internet via the following links: http://www.privacy.gov.au/health/index.html http://www.privacy.gov.au/publications/ipps.html http://www.privacy.gov.au/publications/nppgl_01.html See the following link for the Australian Institute of Health and Welfare’s document on privacy governing its national medical data sets.

http://www.aihw.gov.au/disability/csda_public/index.html 5.6

Other basic management issues in planning

During the planning stage, each phase of data collection, process and analysis, and dissemination should be planned in advance, with each phase flowing smoothly into the next. For example, data collection procedures should be developed in the light of the anticipated methods for processing and analyzing the data. And all stages of data collection must be reviewed in the light of the ultimate user and his or her data needs. Statistical information is useful only if it can help answer important research or policy questions. The succeeding chapters provide detailed guides in each planning stage. Planning also involves basic management issues, such as consideration of the following questions: What is the budget, and is it enough? How will the staff be selected, trained and managed? What project management mechanism will ensure that the project keeps to its timetable? The overall budget must be divided, and allocated to each phase of the collection. Inadequate monitoring of budgets may mean that data is collected, but no resources are available to produce and disseminate high quality statistical output. Staff resources need to be planned to ensure that people with the right skills are available at the appropriate times in the development, collection and output phases of the collection. If conducting an interviewer-administered collection rather than self-administered collection, the interviewer workforce is very likely to be the largest single cost in the collection phase.

- 102 -

WHO/ESCAP Training Manual on Disability Statistics

Project management is essential to ensure a smooth-running collection within acceptable time frame. In the case of disability household surveys, experience suggests that 12-18 months are required for the planning and development stages, with an equal time for data processing, analysis and preparation of survey output. Between these two phases is the data collection period, which, in the case of disability, is relatively short – perhaps only a matter of weeks or months. Disability data is generally unaffected by seasonal influences, and so collection over a longer period is not necessary. 5.7

Quality of survey data and total survey error

Disability surveys and questionnaires are subject to the same general rules about surveys one would find in any standard textbook on epidemiology or survey methodology. The two familiar textbook requirements of good survey data is that it should be, Valid (measures what it is intended to measure) and Reliable (gives consistent results over repeated measurements) Of course, assessing data is not a simple matter. Though reliability is relatively easy to assess, validity can only be definitely determined if there is a ‘gold standard’ against which the data can be measured. Yet in the case of disability data, other than impairment information, it is doubtful whether a suitable gold standard exists. This is in part why there are various, less demanding standards of validity (construct validity, being the most prominent) for assessing the quality of data. Surveys, by their nature, attract several varieties of potential error that affect both validity and reliability. There are two sources of error in survey data: sampling error and non-sampling error: Sampling error arises because survey estimates are based on a sample rather than a complete enumeration of the population, and the sample may not be, for a variety of reasons, representative of the whole population. Sampling error is minimised by increasing the sample size of collections and improving sample design. Some of these issues are discussed below. Non-sampling error is bias in survey estimates, not traceable to features of the sample that affect the validity of the data collected. Non-sample error is very difficult to measure, and can only be minimised by paying close attention to every step in the process, from survey development, question design, data collection and processing. In a census, since it has no sampling error, all errors are therefore attributed to non-sampling error. Diagram 5.3 sets out some major sources of non-sampling error, grouped by problem area.

- 103 -

WHO/ESCAP Training Manual on Disability Statistics

Diagram 5.3: Sources of non-sampling error in survey/census data

Non-sampling error

Specification error

Frame error

Nonresponse error

Processing error

Measurement error

Concepts

Omission

Unit

Editing

Information system

Objectives

Inclusion

Within unit

Data entry

Setting

Data elements

Duplication

Item

Coding

Mode of collection

Incomplete information

Weights

Respondent

Tabulation

Interviewer Instrument

Source: Research Triangle Institute

We do not have space in this manual to discuss all of these potential sources of error. Specialists in survey methodology are the best people to be on guard against errors associated with frame, non-response and processing. Nonsample errors associated with the specification of the underlying concepts, objectives and data elements are problems that arise in the early development phase, some of which we have already discussed. Finally, measurement errors linked to respondents’ characteristics, interviewers and instruments, are all familiar problems to statisticians and we will mention these only in passing in what follows. The development and implementation of any survey involves countless decisions, each of which have the potential to enhance or detract from the validity (or precision) of survey estimates. Not all decisions are a matter of choice, however. Better data usually costs more and takes more time and resources. This is especially true for specialised surveys, such as a disability survey. The more resources that can be invested in training interviewers in the concepts of disability, the better the response rates and data will be. Face-toface classroom type training is highly effective, but costly. When disability modules are added to existing household surveys, it may be possible to get away with less costly methods such as producing and distributing self-training manuals.

- 104 -

WHO/ESCAP Training Manual on Disability Statistics

In short, there is a trade-off between cost and high quality data. Inadequate funding is the primary cause of the following major predictors of poor quality survey data: ¾ ¾ ¾ ¾

Lack of adequately trained staff and interviewers; Lack of specialised statistical and methodological expertise; Failure to understand and define user needs for data; Inability to develop a data collection instrument tailored to meet user needs; ¾ Inability to put into place procedures to ensure adequate sample design. As we shall see in Section 8.1, interviewing skills are particularly important for surveys/censuses. Resources spent on training usually pay off in terms of increased quality of data. Additionally, pre-testing of collection instruments, discussed in Chapter 6, is a necessary part of any data collection in order to avoid errors. 5.8

Issues in collection design

Chapter 4 presented a variety of disability question types and made the point that different questions produce different disability statistics. Although determining which type of question is preferable for the purpose of the survey/census is a central design problem, other factors can equally distort estimates of disability prevalence. These are discussed in this section. 5.8.1 Context of disability questions When disability questions are embedded in non-disability surveys, the background context of the survey (health, income, employment) can affect the disability estimates. It has been shown that when disability data is collected in a health survey, the prevalence rates tend to be much higher than when it is collected in a census, a general social survey or even a survey devoted specifically to the topic of disability. This phenomenon is illustrated in Box 5.3 with the case of New Zealand. Box 5.3: The Context in Which Disability Questions are Asked The following were disability prevalence rates for New Zealand produced in the same year from three data sources: 1996 Population Census 1996 Household Disability Survey 1996/97 Household Health Survey

14.4 percent 19.1 percent 26.6 percent

The high prevalence derived from the health survey resulted from the fact that respondents, having been asked general questions about their health, health risk behaviours, and their use of health services, were sensitized to health problems and the difficulties they experienced, and were more likely to respond positively to the disability questions. One way to moderate this effect might be to ask the disability questions first, before any mention of health or health problems.

- 105 -

WHO/ESCAP Training Manual on Disability Statistics

5.8.2 Proxy versus non-proxy survey responses When information about one person is obtained from another, we call that a proxy response. Allowing or disallowing proxy responses is a basic decision about data collection methodology. Censuses usually allow proxy responses, while detailed surveys of disability often do not, requiring the information to come directly from the individual wherever possible. Persons with disabilities are suspicious of proxy responses since, historically, health professionals and others assumed they had the right to speak for persons with disabilities (it is not without reason that the rallying cry of Disabled Persons International is ‘Nothing about us, without us!’) In the Australian disability survey, proxy responses are allowed for the screening questions, but once a positive response has been made identifying someone as having a disability, further questions are asked of the individual concerned. Assistance in answering the questions is also allowed under the non-proxy reporting scheme. Statistics Canada, investigating the effect of proxy responses on prevalence, has concluded that proxy reporting tends to lower prevalence rates. Their analysis also showed that, while proxy and non-proxy prevalence rates were similar for moderate and severe disability, proxy-based prevalence rates for mild disabilities were lower than non-proxy rates. Proxy interviewing may also result in under-recording types of disability for which there is a high level social stigma, in particular disabilities associated with psychiatric illnesses. It is therefore recommended that direct, non-proxy interviews be used unless the respondent's health condition of intellectual disability makes it difficult to conduct the interview. Care must also be taken in those cases that the proxy is someone who genuinely knows the subject, not merely from a medical perspective, but as a complete individual. 5.8.3 Face-to-face versus telephone interviewing Face-to-face interviewing results in higher levels of disability self-identification than telephone interviewing. Table 5.1 shows this difference, by age group, for three Canadian disability surveys. It is generally safe to assume that higher levels of identification of persons with disabilities, if reliable, are closer to the population true value. Although the difference in prevalence rates may not be of enough concern to warrant the extra cost of face-to-face interviews, still it is recommended, as a result, that this method be used whenever feasible.

- 106 -

WHO/ESCAP Training Manual on Disability Statistics

Table 5.1: Face-to-face and Telephone Interviewing: Differences in Prevalence Interview Method

Canadian Health and Disability Survey

Testing of Activity of Daily Living Questions

Calibration Test

Telephone Interview 15-24 years 25-44 years 45-64 years 65 years and over

3.5 5.9 17.1 36.7

5.1 9.5 25.4 56.6

7.1 10.9 28.7 57.7

All ages Face-to-face 15-24 years 25-44 years 45-64 years 65 years and over All ages

11.7

17.9

4.4 8.4 23.5 44.3

7.0 10.6 31.3 58.8 16.6

20.5

6.5 14.9 35.1 55.8 21.6

21.3

5.8.4 Special questions for sub-populations All countries are composed of sub-populations, defined by culture, geographical area, age group or some other salient feature. Are there reasons to think that members of different sub-population will respond to disability questions differently than other sub-populations? This question was raised when Statistics Canada was asked to include disability questions on the Aboriginal Peoples Survey of Canada. On the advice of Canada’s aboriginal peoples, a few of the questions on agility were modified to reflect aboriginal conditions in order to validly test the same motor movement within that sub-population. The Australian Bureau of Statistics had a similar experience when testing the disability module to be used in the Australian National Aboriginal and Torres Strait Islander Survey. Testing showed that not all of the disability items in the standard module were relevant to the indigenous population. The module was improved to make the questions more understandable and relevant to the indigenous sub-population, while keeping the data as relatable to the general disability module as possible. 5.8.5 Collection of disability data about children Research suggests that, especially when activity or participation questions are asked, disabilities in children may go undetected. Standard activity questions asking about reading a newspaper, carrying a 4.5 kg object for 10 metres or more, or walking 350 metres may not be appropriate for children, especially young children.

- 107 -

WHO/ESCAP Training Manual on Disability Statistics

Recognition of this problem has lead to the development of age-specific questions, and in some instances, questionnaires. Though the inclusion of age-specific questions or questionnaires raises the cost of a survey, the alternative is not acceptable. Even when specially designed questions are included, there are systematic problems identifying children with disabilities. Fortunately, this is primarily an issue for pre-school age children: once the child reaches school age disabilities are easier to determine and identify. The source of the problem seems to be the fact that children develop at different rates. One child will be walking at 9 months while others will not take their first steps until well over a year old; some children can communicate quite well by the age of two, while others are three or older before they can talk or understand when spoken to. Thus, the parent – as proxy for the child – may not be able to tell the difference between a child whose activity limitation is due to normal variation in development, or a long-term disability or developmental delay. Equally, it may be difficult for a parent to determine whether a child has a condition that will correct itself over time, or one that can only be corrected by a technical aid, such as orthopaedic shoes or braces. Parents of children with disabilities may also simply refuse to admit that their child has a disability and dismiss it as slow development. Mild or moderate disabilities are less likely to be identified. Serious ones tend to be obvious to parents and professionals alike. Other problems may not be noticed until a child begins attending school – for example, behaviour problems associated with attention deficit disorders, or learning or reading difficulties. Unfortunately, once children are diagnosed and ‘labelled’ with some diagnostic term, this is also less likely to be reported during the interview. If children are included within the scope of a disability survey, it is therefore recommended that additional questions be added. These questions should be ¾ developmentally appropriate, given the child’s age and abilities expected at that age; ¾ culturally appropriate, asking about abilities expected of children in the child's culture; ¾ answered by an appropriate proxy (usually the parent) who knows the child well; ¾ worded in clear and unambiguous language easily understood by the respondent. If it is impossible or too costly to add questions of this sort, then the scope of the survey should be limited so that, say, children under the age of 5 are excluded.

- 108 -

WHO/ESCAP Training Manual on Disability Statistics

The Australian SDAC collects information about children from birth. Parents of children aged 0 to 5 are asked screening questions relating to their children in language modified for the young population group. The questions are part of the main questionnaire with special sequencing for children aged 0 to 5 years. The question on mobility for children aged less than 4 years, for example, reads: “Does (name) ever need more care or help than other children of (his/her) own age to get in or out of a bed or chair?” The question text refers to the same activity asked for the 5 years old and over population, but is modified to ask whether the need for assistance is greater compared to other children of the same age.

Diagram 5.4 presents disability prevalence rate data from the 2003 Australian survey, by age and sex. Disability prevalence rates increased between the 04 and 5-14 years age groups and then decreased again for the 15-24 years group. The disability prevalence rate for males aged 5-14 years (12.4%) was almost double that for 5-14 year old females (7.5%) Diagram 5.4: Disability rates by age and sex, Australia, 2003

Source: Disability, Ageing and Carers: Summary of Findings, 2003 (Cat. no.4430.0).

5.8.6 Collection of disability data about institutionalised populations The importance of disability information from people living in institutions is discussed in Section 4.2 in the context of the scope and coverage of disability collections. Excluding this population from surveys affects disability prevalence rates, especially for older people who are more likely to be living in institutions for the aged. Other sorts of institutions, such as prisons, are very rarely included in sample surveys. This is unfortunate since we have evidence that prison populations have high rates of persons with disabilities, especially developmental and intellectual disabilities. Yet, for most countries this population is small and its impact on national disability prevalence rates may not be great.

- 109 -

WHO/ESCAP Training Manual on Disability Statistics

Since there is usually good administrative data about people living in institutions, as long as this data includes disability information, it may be possible to combine administrative with survey disability data to yield a composite estimate of disability prevalence for the nation. 5.9

Planning for administrative-based data collection

In planning for administrative-base data collection, the same processes involved in any survey or census are also followed. This means that there should also be stakeholders’ consultation. The representatives, however, of advisory groups in this approach should include the agencies or service providers who will be providing the data as well those who will be using the data. It is strongly recommended that representatives of the group who are the target of the data collection – the recipients of the services or programme – be also represented, either individually or by organization. 5.9.1 Identifying information needs As with any data collection endeavour, the first question to ask is: What information do we need and who is going to use the data? As mentioned, typically, administrative data is collected for reasons other than getting disability information in general. Among the variety of planning, programme evaluation and monitoring activities for which administrative data is collected are the following: ¾ to gather information about clients and the services they use; ¾ to determine needs and unmet needs; ¾ to satisfy reporting requirements of government departments, boards or other regulatory bodies; ¾ to gather information about, and to report on, programme expenditures as related to populations served; ¾ to establish a benchmark for service providers, and to compare this with comparable benchmarks nationally or internationally. Knowing broadly what kinds of data are required, the next step is to get potential users of the data to be more specific about the information they need. If, for example, the data user really wants to know whether the number of people with intellectual disabilities accessing the service is increasing, or how many service users have a primary carer, the questions will have to be designed to identify these sub-populations. If the only people using the data will be policy planners, then more general information about numbers of service users, in broad categories, will suffice.

- 110 -

WHO/ESCAP Training Manual on Disability Statistics

If the data will be used by researchers who wish to explore service usage issues not usually highlighted, then perhaps more exploratory questions need to be considered. Once the overall purposes and the types of information needed are agreed on, the next question is whether this information already exists in some other form. Given the costs of data collection, it is well worth investigating this possibility before launching into another data collection. The next step is to define the scope of the collection – who or what you want to collect information about. Are you interested in tracking service usage? If so, the relevant population will be people with a disability who use the service. Do you want to monitor the hours staff spent caring for client? If so, your population should be the staff not the clients. 5.9.2 Developing data items Once the information needed has been clearly articulated, it is time to decide what data items will produce this information. Data items should focus on the information actually required. Try not to collect information that is not needed, and aim to collect the minimum amount of information required. Where possible, existing data standards should be used as a basis for developing data items. Think about the analyses you will want to conduct using the data since this should determine the data items you need, and the coding categories you will need to use to capture information. For example, if you want to know the age distribution of clients, you will need to record date of birth. If all you require is general age information, then it may suffice to use broad age groupings (‘0– 14’, ‘15–25’, and so on). Using age groupings is less invasive to privacy, but then again may not support the detailed analysis you would demand. 5.9.3 Deciding on collection methods and materials Consider the simplest methods of collection first. It might be easier to contact each service user directly and ask them one or two questions than to devise an elaborate three-page survey that is posted out for completion. The key considerations on data collection methods are: ¾ Should data be collected using pencil and paper (e.g., a standard survey form) or electronically? ¾ Will data collection be part of an on-going update of an information system or a periodic data capture? ¾ Who will collect the data (e.g., service users, service staff, or hired data collectors)?

- 111 -

WHO/ESCAP Training Manual on Disability Statistics

If you plan to collect a substantial amount of data, it is probably better to use an electronic tool to capture and store it. This may entail the use of an off-theshelf spreadsheet software or a purpose-built database. It is always advisable to test the tool to make sure it is fully operational, complies with your data needs, and is easy to use. Consider using existing systems, it may be possible to use information collected on standard client intake form, which uses an electronic system to collect data, and simply add questions about staffing, finances, or whatever additional data you need. Alternatively, the data required may not be collected by a single administrative agency, but can be pieced together by joining together several administrative collectives (for example, by utilising a single client identifier for all the systems). When developing your own administrative data collection service intake form, staff hours form, or whatever you may require, it is important to keep in mind that: ¾ you need to phrase the questions to make them clear and be able to gather what you are asking – try not to ask too much in a single question; ¾ you need to consider who will be asking the questions, how the questions will be asked and in what surroundings – tailor the data collection to the real-situation in which the data will be collected; ¾ all questions should be tested — no matter how obvious the meaning may seem to the person who wrote it, the question may still be ambiguous or vague to the respondent. You can only find this out by testing the questions with people who are part of your target population (detailed discussion on pre-testing is presented in the next chapter). 5.10

Sample design

To carry out a survey of persons with disabilities you have to find them. This is not always straightforward, and involves the fundamental issue of sampling. Sampling is a major consideration in the design of a reliable, and affordable, disability survey whose primary object is to determine prevalence. In this manual, it is only possible to review, in a general fashion, some considerations relevant to disability surveys. The reader should refer to the UN Guidelines and Principles for the Development of Disability Statistics, which reviews in far more detail the complex sampling issues raised by disability. 5.10.1 Sampling frame Since disability prevalence is relatively low, and certainly less than 20 percent, large samples are usually needed to measure it reliably. If the survey is to

- 112 -

WHO/ESCAP Training Manual on Disability Statistics

yield other information about distribution, types and causes of disability, or estimates for sub-populations, then the sample size has to be even bigger. Given cost considerations, most national statistical agencies are therefore forced to seek ways of increasing the efficiency of sample design. Usually this means making full and efficient use of existing sampling frames – that is, lists, registries or other materials from which samples are derived. The most common forms of sampling frames are those based on geographical area. A typical area frame is a set of enumeration areas developed out of the most recent population census, which supplies basic information about the number of people and households in the area. If one estimates a prevalence rate for disability, say 10 percent, then the size of the sample, for that area can be calculated easily. Applying conventional survey sampling techniques such as stratification, clustering and systematic random selection, can yield a national sampling strategy. Unfortunately, relying entirely on area frames is quite inefficient – and so expensive – for relatively rare phenomena such as disability. Many households have to be interviewed, on average, to locate a single person with a disability. To improve efficiency, it is highly recommended, wherever possible, to combine the use of area frames with others, and in particular list frames. List frames are derived from administrative and other data that specifically target sub-populations (e.g. the elderly, children in orphanages, the institutionalised, or long-term care service users) and are updated regularly. Persons with disabilities, compared to people living in households, are more likely to be included on these lists. Other lists may also be available, for example a registry maintained by a national organization for the blind. Since area frames tend to be complete but inefficient, and list frames are incomplete but highly targeted and so efficient, a sensible sampling strategy is to combine both frames. This is not without its own costs, however. For example, effort would have to be made to ensure that the list frames are upto-date, and some mechanisms for eliminating duplication would have to be designed and used. Another strategy is to conduct a small screening survey in an area to obtain a sample of households with persons with disabilities for a follow-up survey that is far more detailed. This strategy was used by Thailand and India to obtain a list of households with persons with disabilities. Unfortunately, sometimes the initial screening question used to identify households with persons with disabilities does not work well, and identifies only a small subset of the population of person with disability. Statistics Canada has developed an approach that improves efficiency of sample selection and reduces survey costs. They use the Population Census to identify respondents for a follow-up survey on disability. In order to use the census to filter a population, a suitable disability question was devised and workable mechanisms to carry out the follow-up survey were put into place.

- 113 -

WHO/ESCAP Training Manual on Disability Statistics

The census form was modified to inform the respondents that their responses would be used to select people for a follow-up disability survey, and they were asked to provide their telephone number in case they will be selected. The approval of the Canadian Privacy Commissioner had to be obtained to use census results in this way. (See Section 3.5 for more details of the Canadian strategy.) 5.10.2 Sample size and unit Once a successful sampling strategy has been adopted, the next question is: how big should the sample be in order to yield data with an acceptable level of sampling error? This is a highly technical matter that is the job of an experienced survey methodologist. The UN Guidelines and Principles for the Development of Disability Statistics has a discussion of sample size. In general, the sample size required to produce estimates of disability prevalence with acceptable levels of error is a function of the prevalence levels of the other characteristics or phenomena of interest to the survey designer, as well as the level of dis-aggregation of the data that is required. As a rough rule of thumb, surveys that have sufficient size to produce estimates of unemployment in a specified geographical area are likely to be large enough to produce rough disability estimates for the same area. The sample sizes of some recent disability surveys are provided in Table 5.2. Another important decision, concerning sample unit, is whether the survey should sample at the level of the person or the household (or institution). Canada and New Zealand used the person as the sample unit and they were able to identify persons with disabilities by their census screening. When this information is not available, then the sample unit is usually the household, which is suitable for area based samples.

- 114 -

WHO/ESCAP Training Manual on Disability Statistics

Table 5.2: Sample Sizes of Some Disability Surveys Country

Collection

Sample size

India

Survey of Disabled Persons

70,302 households.

Indonesia The Philippines

Australia

National Socio Economy Survey, 2003 2001-2003 Philippine Disability Survey 2003 Survey of Disability, Ageing and Carers

Comments The sample size is the number of households with someone with a disability identified in the listing sample.

67,000 households

59,443 households

14,000 households, 300 non-private dwellings, 500 establishments

- 115 -

Sufficient sample size to produce prevalence by State, as well as crosstabulations of population by age, sex and disability status.

WHO/ESCAP Training Manual on Disability Statistics

Learning Objectives: Chapter 6 Instrument Development After reading this chapter, the reader should be able to: 1.

Understand attributes of good questions

2.

Outline procedures for testing and validation of disability questions

“Mobility Scale” Krzysztof Kuczyk WHO Photo Contest “Images of Health and Disability 2003/2004”

- 116 -

WHO/ESCAP Training Manual on Disability Statistics

6.

6.1

INSTRUMENT DEVELOPMENT 6.1

Attributes of good questions

6.2 6.2.1 6.2.2 6.2.3 6.2.4

Testing and validation of data collection instruments Pre-testing Intermediate testing techniques Field testing Validation techniques

6.3

Data collection guide

Attributes of good questions

The development of disability questions for surveys or censuses is a difficult, but vitally important stage of any disability data collection. The data from any survey/census is only as good as the questions asked, and the ability of the interviewers who ask them. Ask poor questions or don’t train interviewers well and the results will be poor, unusable data. In Chapter 4 we described how the ICF can guide the question designer. The ICF model of disability makes it clear that, depending on the purposes of a disability survey/census, disability questions can adopt approaches based on the three dimensions of disability and the environmental factors component. The best approach of all, however, is to give a complete ‘portrait’ of disability by asking questions about all three dimensions of disability, and environmental factors. Good survey/census questions about disability or any other matter must be assessed in terms of their validity and reliability. Questions are valid to the degree to which they identify and provide the factors required to measure the phenomena that they are designed to, and nothing else. Disability questions, in a survey or census, should identify aspects of disability, and help us to measure prevalence of disability. Questions are reliable when they elicit the same response at different times, or with different respondents, when the levels of functioning and related circumstances are the same. Respondents should indicate by how they respond to a disability question whether or not they have a disability, and should do so consistently, when asked at different times, unless conditions have changed, Questions about disability – certainly as important as questions about income levels, education or employment – need to satisfy at least two other basic criteria. The first is cross-population comparability – the assurance that information about disability distilled from one sub-population will identify the same phenomena as the information distilled from another. Population comparability also applies between countries and across cultures, but, given the diversity in race, religion, and culture that exists within countries in the - 117 -

WHO/ESCAP Training Manual on Disability Statistics

Asia-Pacific region, comparability is very much an issue for surveys within countries. As has been noted above, the ICF provides a framework and a vocabulary that is ideally suited for ensuring cross-population comparability. The international revision process that led to the ICF included several years of extensive cultural applicability research designed to ensure that both the model of functioning and disability, and the individual categories within the classifications, are as culturally comparable as possible. The results of these extensive field trials formed basis for every step of the revision process and produced an international classificatory tool that retains its reliability across languages, cultures and societies. Questions that utilise the terminology of the ICF, and which reflect the ICF model of disability, can therefore be depended on to produce cross-population comparable data. The final criterion is technically the most difficult to satisfy. The ICF includes qualifiers for each of the dimensions of disability and environmental factors in order to codify information that is sensitive to extent or magnitude of the functional problem (or, in the case of environmental factors, whether the factor is a barrier or facilitator). The ICF, as mentioned in Chapter 4, provides a coding system for these qualifiers (NO, MILD, MODERATE, SEVERE, and COMPLETE). Also in Chapter 4, we outlined a number of types of questions that can be used in surveys and censuses to identify the population of persons with disabilities. We also looked at some issues of language, gender, and culture that can undermine the effectiveness of disability questions. But, regardless of the care taken in question design, questions developed by the data collection development team must be thoroughly tested to ensure that they are understood by respondents and consistently measure what they are intended to measure. We turn now to issues of testing and validation. 6.2

Testing and validation of data collection instruments

Data collection instruments should always be tested - a small investment in resources on testing can save a large amount of resources if errors are detected. Systematic errors, if undetected, can make the data collection activity unfit for the purposes it was meant to serve. Thorough testing minimises non-sampling errors, of the sort mentioned in Chapter 5. Additionally, testing provides management information useful for planning the conduct of any data collection undertaking. A complete testing strategy involves the following three components: Pre-testing – the investigation of possible data quality concerns and the identification of ways to minimise identified sources of non-sampling error. Field testing (or pilot testing) – systematic testing on a set of respondents to isolate potential bias and sampling errors. Dress rehearsal – a full-scale field test of the proposed final data collection instrument and procedures such as training, data processing, and data - 118 -

WHO/ESCAP Training Manual on Disability Statistics

analysis. The dress rehearsal provides valuable management information about interview time for cost estimates and the adjustment of sample size (for surveys) in order to fit a fixed budget.

6.2.1 Pre-testing The primary aim of pre-testing is to give a small convenient sample of respondents a chance to review the proposed questions and give their feedback. An interviewer or, if a focus group is used, a moderator, takes notes of the session indicating when questions are misunderstood, or when the respondents feel the language used is embarrassing, , inaccessible, insulting or otherwise unacceptable to them. Technically, the kinds of non-sampling error that can be elicited in this manner include: ¾ respondent biases that arise from the way the respondent interprets the questions and the cognitive processes used to answer them; ¾ interviewer effects – arising from the interviewer's ability to consistently deliver the questions as worded; ¾ mode effects – caused by the design and method of delivery of the data collection instrument; and ¾ interaction effects among or between the effects above. While pre-testing helps improve the questions and reduce errors, it cannot eliminate all non-sampling errors in the data. Pre-testing techniques usually involve convenience, or purposive sampling, rather than probability sampling. While pre-testing identifies issues that exist within the broader population and which may affect data quality, probability sampling is required to provide information about the magnitude of these effects on the final data. The most common pre-testing techniques used by national statistical organizations are focus groups and cognitive studies. Focus groups A focus group is an informal discussion between a small group of individuals led by a moderator or facilitator. The topic of the discussion is controlled by the moderator, but the discussion itself is allowed to follow its own path, within reason. The aim of any focus group is to allow the participants to freely express their views and perceptions in a discussion format. Focus groups between potential data users can be extremely helpful early in the development phase to explore conceptual and practical issues relevant to specific sub-populations. In particular, data user focus groups can help to: - 119 -

WHO/ESCAP Training Manual on Disability Statistics

¾ ¾ ¾ ¾ ¾

determine the feasibility of conducting the data collection activity; develop data collection objectives or data requirements; determine data availability and record keeping practices; explore and define concepts to be used in the questions; and clarify reference periods (e.g. ‘In the past 30 days, have you had any difficulties….’).

When used to gather input from people who are in the population to be covered, the objectives are primarily to determine the effectiveness of the questions and the appropriateness of the language used. The focus group may also be a setting to explore and evaluate alternative question wording, or interviewing formats, in order to make the data collection more successful. Focus groups are particularly useful when they allow a small or 'rare' segment of the population to be tested that is likely to be under-represented in a larger field test. Focus groups are generally unsuitable for highly sensitive or emotional topics as biases in intra-group behaviour are likely to distort the response of individuals with a tendency to give only acceptable or 'public' opinions. Focus groups are especially useful in testing disability questions because respondents with specific disabilities (which might be quite rare in the overall population) can be selected to provide feedback from their perspective. This was the objective in the use of focus groups to pre-test disability questions for the Australian Population Census. Older persons, people with physical disabilities, intellectual disabilities and carers of persons with disabilities were separately convened in focus groups to test the question sets. Sometimes problems with question wording can be quite unexpected. In the Australian example, one of the questions tested was: “Does a physical or mental condition or health problem (lasting 6 months or more) reduce the amount or kind of activity this person can do?” A number of people who were totally unable to perform the activity in question said they would not give a positive response since, from their perspective, being unable to do something was not a “reduction” in activity. Cognitive interviews A cognitive interview is an in depth, one-on-one interview in which trained interviewers ask volunteer participants probing questions about the questions being tested. Cognitive testing can be an iterative process, in which interviewers go back to the same participant and ask different questions and scripted probes, once questions have been modified. About 12 or 15 interviews at each round of questioning usually ensure that sufficient data is gathered. Cognitive interviews are usually filmed and, to avoid both interviewer and respondent fatigue, are limited to about 1 to1.5 hours per interview. The point of a cognitive interview is to understand the cognitive process that goes on when the respondent answers a question – what is he or she - 120 -

WHO/ESCAP Training Manual on Disability Statistics

thinking, what interpretation is given to the question, what assumptions are made. If properly conducted by skilled cognitive interviewers, the probes can identify and classify: ¾ difficulties the respondents may have with their comprehension of the concepts used in the question; ¾ the level of knowledge required to answer the question ¾ respondents’ ability to recall information; and ¾ respondents’ ability to form a response. As well, cognitive interviews can provide information on adverse respondent reactions to sensitive or difficult questions. In short, cognitive interviewing can identify both the source of and the reason for a non-sampling error in a set of questions. Cognitive interviewing is a sophisticated skill, requiring a talent in forming and asking spontaneous questions based on responses to scripted questions and aural and non-verbal cues. To increase the range of information that can be obtained from an interview, the cognitive interviewer can use some of the following specialised techniques: Paraphrasing invites the respondent to repeat the question in his or her own words. This allows the data collection designer to see if the wording of the question is being understood in the way intended. The paraphrase may also suggest alternative wordings. Vignettes are descriptions of hypothetical situations or levels of functioning. The respondent is asked how he or she would respond to the question if they were in the hypothetical situation or experienced the level of functioning. The vignette technique is especially useful for response calibration. Card sorting asks the respondent to sort through a stack of cards listing words or concepts and then group the cards together under some broader category or other arrangement. The results of card sorting indicate how respondents understand the broader concepts, which helps to determine how the question is interpreted. (This technique was extensively used during the development of the ICF to determine the correct grouping of classification items.) Other pre-testing techniques Several other common pre-testing techniques can be helpful at various stages during the pre-testing period: Literature reviews provide information about the tools that have been used to collect similar data – their costs, problems, and advantages. Expert reviews employ a group of survey/census design experts asked to critically appraise a survey/census or questionnaire to identify sources of non-sampling error and make suggestions for dealing with them. The experts are asked to analyse what is demanded of a respondent for each question, in terms of comprehension, knowledge, judgement, and ability to formulate a response. - 121 -

WHO/ESCAP Training Manual on Disability Statistics

Interviewer debriefings combine standardised interviewer debriefing questionnaires and focus groups to gather information from interviewers about a data collection instrument. They can also be used after field tests and data collection to provide information for later stages of survey/census development and future iterations. Observational studies are commonly used to test and evaluate surveys/censuses. In an observational study, a trained observer watches the survey/census process – either on the form completion or the responses within an interview – to better understand the respondent's line of thinking. These studies can locate problems in wording, question order, and presentation or layout, and are helpful in estimating the time needed to complete the questionnaire. Behaviour coding involves individuals, specially trained in behaviour coding, assessing respondent and interviewer behaviour during an interview according to a predetermined list of behaviours. The assumption is that any behavioural deviation on the part of the respondent or interviewer indicates a potential problem in the questionnaire. Selecting pre-testing strategies Given the range of available techniques, survey/census developers need to select the right combination of techniques to achieve the objectives of testing. The factors that shape the pre-testing strategy are primarily resources (including time and labour), how quickly results will be available, and at what stage during the pre-testing development phrase the technique is most effective. Table 6.1 summarises these factors for the pre-testing techniques discussed here.

Table 6.1: Attributes of Pre-testing Techniques

Technique Focus Groups Cognitive Interviews Literature Review Expert Review Interviewer Debriefing Observational Studies Behaviour Coding

Resources Medium High Low Low Medium High High

Timeliness of results Fast Fast Moderate Fast Fast Moderate Slow

Stage of development process Beginning or early on Middle Beginning Early on Beginning or early on Middle Later

A pre-testing strategy should combine a number of techniques to optimise the chances of identifying non-sampling error. As pre-testing focuses on issues of comprehensive and response error, the actual mode of administration of the final survey/census need not be decided during pre-testing, although it must be decided by the field testing phase.

- 122 -

WHO/ESCAP Training Manual on Disability Statistics

6.2.2 Intermediate testing techniques There are some techniques that are conceptually halfway between pre-tests and field tests. The following techniques are actually small-scale preliminary field tests (alternatively, they can be added as qualitative components of field tests). Skirmishes test two or three narrowly defined aspects of a survey/census, such as the effectiveness of introductory letters or a specific field procedure. They are small field studies that typically use about 150-200 completed questionnaires. Respondent debriefings are conducted on focus groups after a skirmish or field test, and are designed to provide information about the underlying causes of respondent misunderstandings, or other discrete features of respondent behaviour. Follow-up questioning is a technique which is also called a ‘post enumeration study’. This involves asking respondents other questions while the questionnaire is being administered to them. Follow up questions usually focus on respondents' thought processes as they complete the questionnaire, information useful for understanding what the respondent has in mind when answering the questions. Sometimes follow-up questions can be asked a couple of weeks after the respondents have taken the interview, using the original questions, or more detailed ones, and looking for differences in the two sets of responses. This is sometimes called “test/re-test” process. 6.2.3 Field Testing A field test involves carrying out a test of the actual survey/census using the designed instruments and procedures to check if the objectives of the survey/census are achievable. Field testing is generally conducted in a number of waves, with initial tests called pilot tests and the final test, the dress rehearsal. Pilot testing Pilot tests are a standard way of testing data collection instruments and procedures for both surveys and censuses. They can identify errors in the instruments that, if not corrected, could result in data being unfit for the purpose intended. The principal objectives of the pilot test are to test the questions and the overall instrument design. More specifically, pilot tests can be used to: ¾ ¾ ¾ ¾ ¾

refine the response categories of questions check question sequencing and flow evaluate the overall layout and design of the instrument check adequacy of written instructions for interviewers evaluate interviewer training manuals and the delivery of training - 123 -

WHO/ESCAP Training Manual on Disability Statistics

¾ estimate timing of survey/census and its components ¾ estimate the overall costing of conducting the survey/census. The pilot test can also test the effectiveness of the data processing procedures – where and how the data will be processed. This is not always done, but it is crucial to the overall success of the survey or census. The pilot test is the first step of the field-testing cycle. An evaluation of pilot test results should be done shortly after the test in order to identify any errors and deficiencies and make improvements prior to other field-testing. The evaluation can be both qualitative and quantitative, based either on cognitive debriefing of interviewers or statistical examination of pilot test data. Dress rehearsal The dress rehearsal is the last test before enumeration. It is an essential, final step in the development of any disability data collection. The dress rehearsal process is done in the same manner as a pilot test, but uses the final version of the instrument and procedures: there are no new questions or untested procedures. By the time the dress rehearsal is performed, it is presumed that all non-sampling errors have been identified and corrected. Nonetheless, this last test may pick up remaining problems that can be fixed, as long as doing so does not affect the rest of the instrument. 6.2.4 Validation techniques A key component of field trials for data collection instruments is validation: if the questions do not elicit data about relevant concept, such as disability, then the data is useless. The bulk of validation testing is qualitative, arising from the reports of respondents during cognitive debriefing, focus groups, and other techniques during which respondents report on what they had in mind when they answered the questions. Similar techniques can be used on interviewers as well, who report on what they believe the respondents had in mind when they answered the questions posed on them. A more rigorous method of validation involves comparing responses from survey question sets with responses to a ‘gold standard’ measure. The method requires the same set of respondents to answer two sets of questions: the ‘gold standard’ questions and those under study. A ‘gold standard’ is merely a measure that we have good reason to think is accurate, at least relative to the phenomena being measured. If we had faith in a particular screening question for disability, for example, and applied it to a sufficient large and representative sample of respondents, we would end up with values for ‘true positives’ (a) and ‘true negatives’ (d). We could now give our survey disability question to the same population, and compare their answers to b and c. The set of positive responses to our survey question that do not match up with our ‘true positives’, we call ‘false positives’ (b), and we similarly derive ‘false negatives’ (c). This produces the following table:

- 124 -

WHO/ESCAP Training Manual on Disability Statistics

Table 6.2: Comparison of ‘Gold Standard’ and Survey Question Responses

Survey Question Responses

‘Gold Standard’ Responses

+ -

+ (a) True Positive (c) False Negative

(b) False Positive (d) True negative

Two validation tests are now possible: Sensitivity: the probability that someone with a disability (according to the ‘gold standard’) will also identify themselves as having a disability according to the survey question, or: Sensitivity:

a a+c

Specificity: the probability that someone who does not have a disability (according to the ‘gold standard’) will also identify themselves as not having a disability according to the survey question, or:

Specificity:

d b+d

In both cases, a score of 100% indicates that the survey question perfectly matches the gold standard (that is, there are no false positives or false negatives). Sensitivity and specificity analysis is useful for understanding the quality of short question sets and their ability to identify true populations of persons with disabilities. Preparation for the 2006 Census by the Australian Bureau of Statistics provides an illustration of how this analysis can work. The Australian Bureau of Statistics wished to test proposed questions for its item on “disability – need for assistance.” This was thought to identify the population of people with a disability that require help with communication, self-care or mobility. The testing strategy used focus groups, cognitive interviewing, and pilot testing. In addition, a test/retest protocol was used in order to compare the census item with an equivalent item from a disability survey. This latter item was based on 17 disability screening questions and a ‘need for assistance’ module, and was thought to be a ‘gold standard’ statistic with respect to this population of people. The test/retest of the census and survey questions involved a total of 5246 respondents. The positive and negative responses from the two questions are given in Table 6.3. - 125 -

WHO/ESCAP Training Manual on Disability Statistics

Table 6.3: Comparison of Disability Census Items and ‘Gold Standard’ Survey Items Test question (census) Total ‘Gold Disability Not Disability standard’ (need for (need for assistance) assistance) ‘Gold standard’ measure (survey)

Disability Not Disability

231

136

367

53

4826

4879

The sensitivity score or (a/(a+c)), is (231/367) or 62.9%, and the specificity score or (d/b+d)), is (4826/4879) or 98.9%. These results indicated that the specificity of the test measure was very good, although some of the people identified by the survey as having a disability were not being picked up by the census questions. Feedback from interviewers and analysis of the data indicated that the people not being picked up were those at the less severe end of the disability spectrum. On the basis of this analysis, the “disability – need for assistance” questions were included in Australia’s 2006 census. 6.3

Data collection guide

Besides the format of the questions and the content of the questions themselves, additional material should be provided to support the collection of data and to ensure that the data will be of high quality. The people who will collect the data must fully understand the purposes of the collection and their role as data collectors. The best way to accomplish this is to provide them with a data collection guide and to discuss this during training (refer to the next chapter for the detailed discussion about training). Instructions to interviewers and supervisors about how to conduct data collection can help ensure that the data collected are of good quality. A data collection guide should contain the following: ¾ background information, which includes an explanation of the overall purpose, content and structure of the collection, objectives of the survey/census/administrative data collection, and information about the users of the data; ¾ information on the scope and coverage of data collection; ¾ definitions of concepts; - 126 -

WHO/ESCAP Training Manual on Disability Statistics

¾ detailed explanation of each disability question, instructions on how to ask these questions with some examples and illustrations, and coding classifications; and ¾ possible problems that may be encountered in data collection and suggestions to solve these problems. The data collection guide should be simple and easy to understand. Remember that this is the only reference instrument that the interviewers and supervisors will be referring to while they are in the field conducting the interviews. It is also important that the data collection guide clearly explores general and social perceptions of disability and persons with disabilities in order to expose underlying stereotypes and myths. As discussed in the earlier chapters, it is very common, in any culture, for stereotypes and generalizations about persons with disabilities to dominate public perception. Persons with disabilities are often viewed as totally incapable of participating in normal social life, as lacking skills or talents, needing charity or putting a strain on social resources. These views are rarely overtly expressed, but form part of the accepted social view, found in advertising and the media, in language, religion, or social policy. It is important in the interviewer instructions to review these myths and encourage interviewers to see if they unconsciously share them. To preserve the validity of the questionnaire, bias and presumptions about disability should be identified and corrected, wherever possible. Training sessions in which these issues can be discussed are the best way of getting myths and stereotypes out in the open where they can be most successfully dealt with. An example of a data collection guide provided by WHO and ESCAP to be used for pilot studies in five countries (Fiji, India, Indonesia, Mongolia and the Philippines) is included in Appendix 3. It is imperative that a data collection guide be pre-tested and pilot tested together with the questionnaire. This is to determine if the instructions are also adequately simple, clear, and complete.

- 127 -

WHO/ESCAP Training Manual on Disability Statistics

Learning Objectives: Chapter 7 Organizing the Fieldwork After reading this chapter, the reader should be able to: 1.

Outline the processes necessary prior to data collection

2.

Understand the basic guidelines for conducting census/survey interviews

3.

Outline responsibilities of the supervisor

“Encounter” Arthur Calasans WHO Photo Contest “Images of Health and Disability 2003/2004”

- 128 -

WHO/ESCAP Training Manual on Disability Statistics

7.

7.1

ORGANIZING THE FIELDWORK 7.1 7.1.1 7.1.2 7.1.3 7.1.4 7.1.5 7.1.6

Preparatory activities prior to conducting the fieldwork Publicity campaign Coordination with the organizations of persons with disabilities Coordination with local officials Printing and distribution of data collection instruments Recruitment Training

7.2 7.2.1 7.2.2 7.2.3 7.2.4 7.2.5

Guidelines for data collection Interviewing techniques Interviewing persons with disabilities – tips and traps Checking for completed questionnaires Responsibilities of the field supervisor Interviewing in institutions

Preparatory activities prior to conducting the fieldwork

Once the questionnaire has been finalized, the data collection developer should begin preparing for data collection. There are a number of preparatory activities that are advised prior to the commencement of fieldwork. Below are the activities that should be followed. 7.1.1 Publicity Campaign The purpose of the publicity campaign is to raise awareness among the population of the forthcoming data collection activity, to help them understand the significance of the activity, and to encourage them to answer all the questions honestly. Another important task of a publicity campaign is to introduce the agency responsible for the data collection. Such an introduction will help to reassure respondents about the legality of data collection, and consequently reduce the number of respondent refusals. Publicity can be done in a number of ways: 1. Conduct an awareness campaign. This can be done through participation in parades, programs held during holidays, and other related activities where lots of people gather together. 2. Have a regular column in a newspaper. This is one way of informing the public on the upcoming data collection activity. The answers to most frequently asked questions and some mechanics of data collection can be stated in the column. Of course, the explanation should be simple enough for the general public to understand. 3. Have regular segments in the broadcast media. Among the different media, radio is the most effective medium in developing and under-developed countries because it can reach remote - 129 -

WHO/ESCAP Training Manual on Disability Statistics

areas and those in the low-income groups. For a nationwide undertaking, the local officers of the data collection agency should coordinate with the local radio broadcasters. The cost to be incurred through this means is also low compared to the print and television media. 4. Produce calendars and posters for distribution. Calendars and posters can include information about the survey/census. These can be distributed in the provincial offices, towns, villages, or to the households. 5. Prepare brochures for distribution. Brochures containing facts about the different information to be collected and its uses can be prepared and distributed. 6. Prepare streamers and banners for display. Streamers and banners can alert the public to the upcoming survey/census. These should be displayed in areas where a lot of people can see them. 7.1.2 Coordination with the organizations of persons with disabilities As discussed in the earlier chapters, it is important that persons with disabilities are consulted early in the development stage of the data collection project, as they can provide valuable information about which data should be collected. During publicity campaigns, persons with disabilities should likewise be involved. Their involvement may encourage other persons with disabilities, and the organizations they represent, to participate either as respondents or as endorsers of the census or disability survey. They are the best source of information on how to conduct a campaign in accessible format and means. 7.1.3 Coordination with local officials It is essential that coordination takes place with the local officials. They need to be informed that the survey or census will be undertaken well in advance of the actual data collection. The local officials can assist in a number of ways, one of which is in the publicity campaign. They can also provide some assistance during enumeration such as security, transportation, accommodation, meals, guides, and translators. Gaining support from the local officials is also an effective way of gaining support from the respondents. In some instances, some respondents do not cooperate if they are not given notice from the local officials. Coordination or courtesy calls with the local officials on the first day of data collection is also important to make them aware that the data collection in their area is starting. This will also help to reduce suspicion on the part of the residents regarding the arrival of non-resident data collectors/supervisors in the area.

- 130 -

WHO/ESCAP Training Manual on Disability Statistics

7.1.4 Printing and distribution of data collection instruments Careful planning is required for prompt receipt of survey materials (e.g. questionnaires, manuals, administrative forms, and supplies) to the different areas covered in the data collection undertaking. Depending on the size of the sample households (survey) or of the total households (census), this may involve a distribution of a large volume of forms and manuals. As such, a large space for storage and a large group of individuals to do collating, sorting, and packing jobs, will be required. Problems with the freight can arise if the central office fails to select an agency capable of delivering the forms and manuals to the various destinations on time. A possible consequence of this is that data collection begins in some areas but is delayed in others because of the non-availability of the forms. Moreover, the distance separating some areas poses a problem in distribution whenever additional forms are requested. It often takes some time before the additional forms are received, with the worst case scenario occurring when the forms do not arrive until after the enumeration period. These problems are all associated with centralized printing of data collection materials. On the other hand, decentralized printing of forms can pose a problem regarding the standardization of the quality of printing. Due to the fact that different printers are contracted in different areas, it is possible that the quality of the printing will vary, even if the same specifications are given to the printers. This is particularly problematic if the technology for data capture is the Intelligent Character Recognition (ICR) or Optical Mark Reader (OMR) which rely on quality paper and ink for smooth operation (refer to Chapter 9 for a detailed discussion of this kind of technology). Another important point is to make sure that the data collection instruments are made accessible for persons with diverse disabilities. 7.1.5 Recruitment The quality of output from the field relies heavily on the effectiveness of the interviewers. Having permanent field staff may guarantee a higher quality than recruiting a new batch of field workers each time data is collected. Obviously, staff with more training and experience will be advantaged in terms of their understanding of important concepts, their ability to deal with unusual circumstances, and so on. Although interviewers do need to be sensitised to disability issues, they do not need medical training or backgrounds. The Philippines Disability Survey of 2001-2003 used nurses and other allied medical professionals to perform functional assessments using standardised tools highly specific for the type of disability. Generally, this level of medical sophistication is not required since disability questions should be written in language that is clear and accessible to respondents. However, interviewers should have at least completed the full period of schooling within their country and be fluent in the main language of the country. Individual countries must decide what further level of education is - 131 -

WHO/ESCAP Training Manual on Disability Statistics

required as well as what formal assessments will be carried out prior to selection. The characteristics of the interviewers (age, sex, education, professional training, employment status, past survey experience, and so on) should be recorded on a separate database. This information can then be linked to the identification numbers of interviewers for each questionnaire completed and an analysis can be carried out of individual interviewer performance. One common problem in recruitment is the reluctance of interviewers to be assigned to a remote area. This becomes a serious problem if no qualified personnel are found in the area. One solution is to offer incentives such as a higher travelling allowance and/or higher wages. In determining the number of interviewers needed for data collection activity, the following information is needed: ¾ the number of households/respondents to be covered or interviewed; ¾ the duration of data collection; and ¾ the average output of interviewer per day The number of supervisors will depend on the extent of supervision that they will perform. Some supervisory duties include editing questionnaires, attending to problems during enumeration (refusals, resignation of interviewers etc.) and monitoring the availability of funds. In some countries, the average ratio of interviewers to supervisors is five interviewers for every supervisor. 7.1.6 Training Thorough training is essential to ensure that both the interviewers and supervisors have the necessary knowledge and skills to collect valid and reliable data. The purpose of training is to: ¾ Ensure a standardized application of the survey materials ¾ Clarify the rationale of the study and study protocol ¾ Motivate interviewers ¾ Provide practical suggestions ¾ Improve the overall quality of the data In particular, for the disability statistics the following points are important: ¾ Ensure understanding of comprehensive concepts of disability ¾ Ensure sensitivity for persons with diverse disabilities and their families Detailed guidelines for training can be found in the WHO document, “Household Sample Surveys in Developing and Transition Countries”, which is located on the UN Statistics Division’s website: http://unstats.un.org/unsd/hhsurveys/index.htm. A summary of the key points is as follows:

- 132 -

WHO/ESCAP Training Manual on Disability Statistics

1. All the training should be carried out, as much as possible, by the same team to ensure consistency. 2. If possible, a booster session is recommended towards the middle of the data-collection period. The purpose of this session is to review various aspects of data collection, with a particular emphasis on those aspects that are proving complex and difficult, or those guidelines that are not being adhered to sufficiently by interviewers. 3. The training methods should include as much role playing in interviews as possible. For this to be effective, different scripts must be prepared in advance of the training so that the different branching structures of the interview, the nature of explanations that are permitted and anticipated problems during an interview with difficult respondents can be demonstrated. 4. There should be at least one opportunity, before starting the actual data collection, for trainees to conduct an interview with a real-life respondent outside of the interviewer group. The practice interviews should be tape or video-recorded as often as possible for review and feedback discussion during training sessions. 5. Training materials should be provided to all interviewers to use as reference material. 7.2

Guidelines for data collection

7.2.1 Interviewing techniques The interviewer should begin by trying to gain rapport with the respondent. With a friendly and respectful manner, the interviewer should introduce themselves by name, and as a representative of the organization, presenting his or her identification card (which should be worn at all times during the interview). The survey/census should then be introduced, with an explanation of its purposes and how the information will be used. The interviewer should be prepared to answer questions such as: ¾ ¾ ¾ ¾

Where did you get my name? Why was I chosen for this interview? What kinds of questions are you going to ask? I'm old. I'm not disabled. Why are you including me in your survey/census? ¾ Why does the government spend money on a survey/census instead of providing better services to those who need them? ¾ I am a member of (name of disability organization). Are they supporting this survey/census? ¾ What services are offered in my area?

Once the introduction and some explanation about the survey/census have been made, the interview can begin. The interviewer’s goal is to collect accurate information by using the questionnaire according to proven interview techniques. Since data users need to combine information collected from all interviews, the questions must be presented in a uniform manner. - 133 -

WHO/ESCAP Training Manual on Disability Statistics

During the course of the interview, a few simple rules should be followed: 1. Questions should be asked exactly as worded on the questionnaire. Research has shown that even slight word changes can alter the response obtained. If questions are paraphrased or rephrased the responses cannot be accurately combined with accurately asked questions. 2. Questions must be asked in the order they appear on the questionnaire. The sequence is planned for continuity so that early questions will not affect the answers given to later questions. 3. Every question in the questionnaire must be asked. Even if the respondent, while answering one question, answers another, goes out for a while, then appears later, the interviewer should ask the second question again, in sequence (perhaps with the apology ‘I know you've already told me something about this, but this next section asks...' – which acknowledges the earlier answer, but requests the respondent’s co-operation in answering it again). 4. It is important that all members of the household are identified. Some households might not volunteer the names or the existence of children or others with disabilities. Ensure that everyone in the household is listed when the household composition details are collected. 5. The interviewer should wait for the respondent to finish talking before starting to record their response. Failure to listen carefully can offend the respondent and result in errors. 6. The interviewer should not interrupt the respondent, even if he or she hesitates or is quiet for a while. Sometimes people initially answer, "I don't really know" when in fact they are thinking about their answer. 7. Answers should not be anticipated. No assumptions should be made about what the respondent is going to say. The respondent should be allowed to finish their sentence. 8. If the answer is dubious and the interviewer believes the respondent does not fully understand the question, the question should be repeated as written. If the interviewer is still doubtful, probing may be necessary (see below), but the interviewer should be careful not to antagonise the respondent by questioning their judgment. 9. Questions should be asked in a positive or neutral tone. Interviewers may feel uncomfortable about a question but should refrain from voicing this. Apologetic refrains ( e.g. “you might not want to answer this question, but…” or “this question probably won't make much sense to you…”), will only disrupt the flow of the interview and may affect how the respondent answers the question. 10. The interviewer should maintain a professional attitude and be upbeat even if a respondent is difficult or uncooperative. The interviewer - 134 -

WHO/ESCAP Training Manual on Disability Statistics

should not get upset by the respondent’s behaviour and be courteous throughout the interview. 11. At the end of each interview, the respondent should be thanked for their time and cooperation. 12. Ideally, the interview should be completed in one visit. With some respondents this may not be possible, and a second or even a third visit may be necessary. Please refer to Appendix 3 for an example of interview guidelines developed for pilot studies in 2005 by WHO and ESCAP.

7.2.2 Interviewing persons with disabilities – tips and traps Even well trained interviewers benefit from training specific to the problems and issues involved in interviewing persons with disabilities. Some tips and potential traps are listed below, grouped under common sense rules directed to the interviewer. 1. Treat persons with disabilities as you would treat anyone else Persons with disabilities have the same needs, desires, aspirations and goals as anyone else. They differ in how they can satisfy their needs and desires, or reach their aspiration and goals. It is a profound mistake to assume that disability changes everything about a person, or that their physical or mental differences would mean they will never understand you, or you understand them. As we know from the ICF, disability is a universal human phenomenon; limitations in functioning have been proven to be part of being human. Persons with disabilities usually refer to daily activities in the same way others do. Do not hesitate to use the words ‘look’ and ‘see’ with people who are blind or have visually impairment, ‘listen’ and ‘hear’ with people who are deaf or have hearing impairment, or ‘walk’ and ‘run’ with people with mobility impairments. It is also important to ask persons with disabilities on what communication needs and preference they have and accommodate the needs to the best possible extent. 2. Be respectful of persons with disabilities We have already mentioned the need to be sensitive to language, since in most cultures, the common perceptions of persons with disabilities are distorted by stereotypes and misunderstanding. At the same time, do not patronise persons with disabilities, or tell them that you admire their courage and strength. Establish trust and treat persons with disabilities as equal. - 135 -

WHO/ESCAP Training Manual on Disability Statistics

Never speak to a person with a disability in a tone of voice that suggests you think they are child-like or of limited intelligence. Use a normal tone of voice. If the respondent is accompanied by an attendant or a sign language interpreter, speak directly to the person with the disability. If a sign language interpreter is present, speak at a slower pace so that the interpreter does not miss what you say. Fatigue may be a problem for people with head injuries or other neurological conditions. When fatigued, they may become angry or frustrated. Suggest a break or at the beginning of the interview, tell the individual to let you know when s/he would like to take a break. 3. Be aware of, and accommodate communication differences Some impairments affect how people speak or communicate. A good interviewer will recognise these difficulties and, without being patronising, seek ways to increase communication. Some people with communication disabilities have learned to compensate for them: persons who are deaf may be able to lip-read and converse orally. A person with a speech impairment may have distinct speech patterns and what may appear at first to be uncontrolled can be understood after a few moments of listening. Be patient, but ask the person to repeat what s/he is saying if you do not understand. People who have unclear speech are used to people asking them to repeat things, so be honest about clarifying a response. 4. Accommodate the deaf respondent When interviewing a deaf person it is important to have his or her attention first before speaking. A deaf person may need a tap on the shoulder, a wave of the hand, or some other visual sign to get their attention. Speak clearly, not loudly, to a person who is hearing impaired; face them and make sure your face is well lit and not obscured by your hand or facial hair. The person will ask you to speak louder or slower if required. Do not exaggerate or overemphasize words. This distorts lip movements making lipreading difficult. Since some lip movements are difficult to lip-read, try to rephrase the question if the person who is lip-reading does not understand it after a couple of repetitions. It is also helpful to rephrase the question if the individual with a hearing impairment has difficulty in understanding you. Body language and facial expression are important factors in successful communication. It is a mistake to think that people who are deaf are also non-verbal. People who are deaf have normal vocal organs but may choose not to speak. Also, not all persons who have hearing impairments can lip-read. - 136 -

WHO/ESCAP Training Manual on Disability Statistics

5. Accommodate the blind respondent Announce your arrival and exit to a person who is blind or has a visual impairment. Identify yourself and let the person know that you are addressing them by using their name or touching their arm. Never distract a guide dog. The dog is working and is responsible for the safety of the person who is blind. Do not assume that all people who are blind can read Braille. If Braille materials are used, it is best to ask beforehand if the person can read them. 6. Accommodate the individual with intellectual disabilities Persons with intellectual disabilities may need more time to understand the question. Do not rush. Make sure the person clearly understands what is being asked. Persons with intellectual disabilities should not be treated like children; they should be treated like any other person of the same age. Sometimes persons with intellectual disabilities may be anxious to please and say what they think you want to hear. Bear this in mind when conducting the interview.

7.2.3 Checking for completed questionnaires After completing the interview, the interviewer should always check if all the questions are asked and if the answers are consistent. It is important to check the questionnaires before leaving the household so errors can be immediately rectified. It is clearly more difficult to correct an error after the interviewer has left the household as that involves another explanation to the respondent, as well as disruption to the interviewer’s schedule and additional cost.

7.2.4 Responsibilities of the field supervisor In any data gathering activity, supervisors play a major role in attaining high quality statistics. It is the duty and responsibility of the supervisor to: 1. Accompany interviewers, especially those new in the area of interview, in the first few days of data collection to assist them in following the proper procedures. The supervisor should observe how they conduct the interview and provide suggestions for improvement. It is important, however, for the supervisor not to intervene and correct the interviewer while the interview is being conducted, unless the error would have a major effect on the response of the respondents. In this case, the supervisor should ensure that the process of correcting the interviewer does not irritate the respondent nor reduce the credibility or morale of the interviewer. - 137 -

WHO/ESCAP Training Manual on Disability Statistics

2. Conduct spot-checking. This is the process of checking if the interviewers are actually in the area visiting the households and conducting the interview. Despite the fact that interviewers have gone through rigid screening and training procedures, it is always possible that some may be tempted not to visit all the households they are assigned to. 3. Re-interview a sub-sample of households. Supervisors should always re-interview some households to determine if the interviewers asked the correct questions to the respondents. The supervisors do not have to ask all the questions in the questionnaire, and the data collection developer may design a specific form intended for this activity. The contents normally consist of important data items in the questionnaire, such as age, sex, and few questions on disability. It is important, however, that supervisors provide clear explanation to the respondent why a re-interview is needed. It is crucial to emphasize to the respondent that they are being re-interviewed, not because the supervisor doubts the veracity of their answers, but merely to confirm some of the answers they provided. The supervisor should not tell the respondent that they are conducting a re-interview in order to check that the original interviewer was doing their job correctly, as this may call into question the interviewer’s credibility. 4. Review the completed questionnaires. Supervisors should review the work of the interviewers in order to correct errors at the earliest stage possible. Early detection of errors saves incorrect information being processed, and enables the supervisor to correct the interviewer’s mistake so it is not repeated in future interviews. 5. Attend to other problems during data collection such as refusals, lack of forms, supplies and materials, lost questionnaires, and replacement of staff. 6. Hold regular meetings with interviewers. Such meetings are vital as they provide a venue where problems and solutions can be discussed. They also provide an opportunity for the supervisor to discuss their observations made whilst conducting the spot-checks, re-interviews and reviews of completed questionnaires. 7.2.5 Interviewing in institutions Interviewing people residing in institutions presents additional challenges that need to be planned for in advance. Institution staff members (administrators or other contacts, such as head nurses and ward clerks) are busy professionals. Scheduling appointments for proxy interviews at a mutually convenient time will require flexibility. Interviewers must be prepared for the possibility that appointments may change at short notice.

- 138 -

WHO/ESCAP Training Manual on Disability Statistics

When interviewing in institutions, the interviewer must be prepared to adapt the interviewing techniques to meet the requirements of a different situation. If the interviewer senses that the respondent is becoming upset, or is incapable of completing the interview, it is best to terminate the interview and discuss with the administrator or other contact person how best to continue.

- 139 -

WHO/ESCAP Training Manual on Disability Statistics

Learning Objectives: Chapter 8 Processing the Data After reading this chapter, the reader should be able to: 1.

Understand the different processes involved in processing the data

2.

Understand the method and design of each processing activity

“Harvest” Robert Joiner WHO Photo Contest “Images of Health and Disability 2004/2005”

- 140 -

WHO/ESCAP Training Manual on Disability Statistics

8.

PROCESSING THE DATA 8.1

Processing procedures

8.2

Processing flow

This chapter is for subject matter specialists/statisticians, processors/editors, computer system analysts and programmers. 8.1

Processing procedures

Processing of data starts upon submission of the questionnaires to the office and ends with the compilation of these data into statistical tables. Processing of disability data follows the same process as that of any other data collected from survey, census or administrative-based data collection. The processes involved are: ¾ Receipt and Control ¾ General Screening ¾ Manual checking of the questionnaires for consistency and completeness of entries ¾ Data encoding ¾ Computer check of geographic identification ¾ Consistency edit check ¾ Imputation ¾ Generation of preliminary tables ¾ Evaluation of preliminary tables ¾ Finalization of tables Detailed instructions for those personnel who execute the above processes such as processors, editors, encoders, system analysts, programmers, and statisticians should be included in processing guides, separately for manual processing (from receipt and control to manual checking of questionnaires, and evaluation of tables) and computer processing (data encoding to generation of tables, and finalization of tables) The brief description of each process is given below: Receipt and control This is the process of receiving the questionnaires from the enumerators/supervisors in all areas (or sample areas in the case of a survey). Upon receipt, the geographic identification of the questionnaires is recorded in a control book with signature of the person who received and transmitted the questionnaires to the office. General screening General screening is done by going over the submitted accomplished forms and checking for the completeness of the geographic identification such as - 141 -

WHO/ESCAP Training Manual on Disability Statistics

region, province, municipality, village, and other information asked for in the cover page of the questionnaire. The codes in the cover page of accomplished questionnaires can be checked against the list of geographic identification of sample areas (for surveys or administrative-based data collection) or all areas (for censuses). If the codes are not yet written on the cover page of the questionnaire, the processors should translate the geographic identification into codes before these are encoded into the computer. Checking for consistency and completeness All accomplished questionnaires should be reviewed according to the instructions laid out in the processing instructions. In general, questionnaires are inspected to determine if all the required items have entries, if they are consistent with each other, and if the values are reasonable. It is also necessary that edited questionnaires be subjected to verification by the supervisor on a sample basis (approximately 20% of the total edited questionnaires). This procedure ensures that missed or overlooked items during the first phase of editing can still be corrected by the supervisor. Moreover, this procedure determines if the editor’s corrections in the questionnaires are reasonable and are in accordance with the instructions provided in the processing manual. If there is any error in the editor’s correction, the supervisor should call this to the editor’s attention so that the same error will not be repeated in other questionnaires. Data capture After the questionnaires have been edited, they are ready for the information to be transferred into the computer program. There are two common methods of capturing the data – data entry and scanning. Data entry This is done through the process of encoding or “keying-in” of the information from the questionnaires into the computers. This is sometimes called “traditional data entry”. Design of the data entry program In the design of the data entry system or program, some checks on the maximum or minimum values (range checks) can be included so that even during the data entry stage the validity of the data is protected. One way to guarantee the accuracy of the coding is to re-key-in the values and then match the keyed values during the first encoding to the values during the second encoding. If there are inconsistencies, it means that some values have been mis-keyed. In this case, the verifier should determine which values are correct by checking them with the original documents/questionnaires. It is not necessary, however, that all questionnaires be keyed-in twice. The sampling rate will depend on the degree of errors in the first data encoding stage. - 142 -

WHO/ESCAP Training Manual on Disability Statistics

Data scanning Another process of data capture is scanning. In this process, the questionnaires are fed into a copier-like machine where the image of the completed questionnaire can be viewed from the screen. Some of the common technologies in this method are: ¾ ¾ ¾

Intelligent Character Recognition (ICR) – this is used for handprinted data recognition. Optical Character Recognition (OCR) – this is used for machine printed data recognition. Optical Mark Recognition (OMR) – this is used for recognizing marks in circles, squares or ovals.

Each one has its own use depending on the format and content of the questionnaire. The advantage of scanning over data entry is the speed of capturing the data. However, while data entry has the problem of mis-keyed values, scanning also has problems regarding recognition. The system analyst as well as the data collection developer should therefore study the advantages and disadvantages of each technology in terms of accuracy, efficiency and cost. Geographic identification (GEO-ID) check After the data capture, the geographic identification of each questionnaire should be checked by the computer. It is true that these have been checked already during the receipt and control and general screening process (manual processing stage) but there are always human errors that we want to avoid. The GEO-ID check is very important because if it is incorrect then the information in that questionnaire may be erroneously attributed to the wrong area. This will lead to under or over estimation of values in the areas. GEO-ID check also identifies those questionnaires that were not keyed-in or were overlooked by the data processors. If there are errors identified in this kind of check, the computer should create output (reject listing) containing the list of errors for the processors/editors to verify, either from the questionnaire or from the list of geographic areas. Design of GEO-ID check program The GEO-ID codes of the keyed-in or scanned questionnaires are matched with the master file. The master file contains all the GEO-ID codes of all the expected areas. If there are any unmatched or incomplete questionnaires, the computer will display the unmatched information for verification by the processors. After verification, the corrected information is keyed into the computer for updating. After updating, the GEO-ID Check Program is again generated. The computer will display the same errors if the computer senses the same unmatched identification. The process will be repeated until all the errors are corrected. Structural check - 143 -

WHO/ESCAP Training Manual on Disability Statistics

This kind of computerised check is done after the GEO-ID Check. This determines if all the required sections or modules (for example, body function module, vision module or section) are encoded in the questionnaires. Design of the structural check program All the required section or module numbers (or items in the questionnaire) are searched by the computer. If any missing sections modules or items are detected, the computer displays these and the identification of the questionnaire so that the editor/processor can check with the actual questionnaire. Once the error is found, the correction is encoded in the computer and the same structural check program is run. This process is repeated until all errors are corrected. Consistency edit check After the data has passed the structural check, the next stage in computer processing is the consistency edit check. Even if there are instructions during manual processing on how to edit and verify the items in the questionnaire, there is still a need to do the same in the computer due to the possibilities of mis-keying data, incorrect editing and verification by processors. Design of consistency edit check In this stage, the relationship of one item to another is checked. The computer will identify a case as an error if the relationship does not agree with the relationship specified as correct by the statisticians. For instance, age is checked against the highest educational attainment of individuals. The computer will detect it as an error if, for example, the age is 10 years old but the education reported is college graduate. In this case, the computer displays the possible sources of error and the corresponding identification of that questionnaire for the processor to verify with the actual questionnaire. The database should be updated once the correct values are known. Several relationships between variables can be included in this program. Like in the GEO-ID and structural checks, this process should be repeated until all the errors have been corrected. In some instances, especially if there are few sections or modules in the questionnaire, the structural check and the consistency check can be combined in one program so as to minimize the number of times that the checking procedure needs to take place. In some cases, it is not possible to verify the data with the questionnaires, especially if there are a very large number of questionnaires, as in the case of a census. If this is the case corrections can be made using the output (or reject listing) generated by the computer. The output should include not only the error of the data but also the values of related items in the questionnaire so that the correct value can be determined from the values of related items in the questionnaire.

- 144 -

WHO/ESCAP Training Manual on Disability Statistics

Imputation Stage In this stage values are assigned to any remaining missing entries. These are assigned automatically by the computer program, on the basis of inputs provided by statisticians or subject matter specialists. An example is as follows: There is no reported age of a female respondent, but the age of her husband is reported. In this case, the subject matter specialist can ask the programmer to assign an age for the spouse in the imputation program. This is done on the basis of known facts about the relationship of the ages between husbands and wives in that country. For instance, if the average age of husbands is two years greater than their wives’, then the computer can assign the age of the female respondent to be the age of her husband minus two. Design of Imputation Program The design of the program depends on the kind of imputation procedure to be followed. One approach is to rely on known information, as in the case of the example given above - ages of husband and wife (this is called “cold deck approach”). Another approach is to use the valid information gathered from other questionnaires (“hot deck approach”). Generation of Preliminary Tables The last check in computer processing is the preparation of preliminary tables. It is advisable that the tables be generated at the lowest geographical subdivision, e.g, by province, so that the distribution can be easily studied and outliers can be easily traced. If there are outliers, a computer program can be developed to trace the causes of such outliers. As in the previous checks, this stage will be repeated until there are no more outliers in the tables. Generation of Final Tables Generation of final tables should be done only after the data has passed through rigid checking procedures. This ensures that the data provided to users is of high quality. 8.2

Processing flow

Diagram 8.1 shows the complete flow of processing survey/census/ administrative-based results. Some of the steps can be modified depending

on the scope and coverage of the data collection undertaking. The available resources (including the technical know how of the staff) will also influence whether some of the steps need to be modified.

- 145 -

WHO/ESCAP Training Manual on Disability Statistics

Diagram 8.1: Flow of processing survey/census/administrative – based results

Receipt and Control

General Screening

Manual Editing: Checking for Completion and Consistency

Verification

YES

With Error?

NO Data Encoding/Update

Geo-ID Check

YES

With Error?

Reject Listing Verification

A Continued on next page

- 146 -

WHO/ESCAP Training Manual on Disability Statistics

Continued from previous page

A Structural Check

Reject Listing Verification

YES

With Error?

NO Consistency Edit Check

Reject Listing Verification

YES

With Error?

NO Imputation

Generation of Preliminary Tables Check for Outliers

Reject Listing Verification

YES

With Error?

NO GENERATION OF FINAL TABLES

- 147 -

WHO/ESCAP Training Manual on Disability Statistics

Learning Objectives: Chapter 9 Analysing and Disseminating Disability Statistics After reading this chapter, the reader should be able to: 1.

Describe methods for presenting disability results

2.

Outline procedures for analysing disability data

3.

Describe dissemination strategies for disability information

“Weightlifting” Chris Melville WHO Photo Contest “Images of Health and Disability 2004/2005” - 148 -

WHO/ESCAP Training Manual on Disability Statistics

9.

ANALYSING AND DISSEMINATING DISABILITY STATISTICS 9.1 9.1.1 9.1.2 9.1.3 9.1.4

Planning the outputs Output items to meet user needs Output tabulations Graphical representation of data Map representation of disability data

9.2 Analysis – Turning data into information 9.2.1 High quality data 9.2.2 Pitfalls in analysing disability data 9.3 9.3.1 9.3.2 9.3.3 9.3.4 9.3.5

Forms of output and dissemination strategies Presentation of data collection detail with the results Accessibility of data to persons with disabilities Dissemination strategies Confidentiality Standard forms of output

9.4 Ensuring secure data 9.5 Documentation

9.1

Planning the outputs

Many developing countries do not put enough money into data compilation, analysis and dissemination. If the data collected is not analysed and disseminated, the resources used in planning, development and data collection are wasted. This is a problem of overall data collection planning, the key to which is effectively allocating resources to the design, collection, processing, analysis, and dissemination phases. Tabulation and analysis of data must be carefully planned during the early stages, not decided upon after the data has already been collected. The scope of analysis depends on the statistical variables (or data items) that the analyst can correlate. The output data items must be chosen at the planning stage of a collection. A well-planned data collection activity securely links the input of questions asked and data items derived with the output of statistical tables and usable information. As always, decisions about which outputs are appropriate, how the individual data outputs are to be correlated, and how this material should be presented, all centre on the needs of the ultimate user of the data.

- 149 -

WHO/ESCAP Training Manual on Disability Statistics

9.1.1 Output items to meet user needs What information is usable depends on the questions the data users want answered. In general, for disability policy, the questions range from basic prevalence to particular questions about the impact of disability in specific areas of life. Here are some examples, depending on the variables included in the data collection instrument: ¾ How many people are there with disabilities in the population? ¾ How does the prevalence of disability vary by age, sex and rural or urban residency? ¾ What is the prevalence of severe disability in the population? How does it vary by age, sex and area of residence? ¾ What proportion of households has a member with a disability? ¾ How do persons with disabilities compare with others in terms of major socio-economic indicators? ¾ To what extent are persons with disabilities receiving effective assistance services? ¾ To what extent do persons with disabilities experience participation restrictions, e.g. unemployment, exclusion from school, unable to use public transport? The kinds of data items that these and other policy questions require range from the most basic identification of populations of persons with disabilities, and their prevalence, distribution and characteristics, to highly detailed data sets which are possible mostly in a survey and not in a census. Some of the data items include: ¾ prevalence of specific activity limitations in seeing, hearing, walking about, moving from one room to another, speaking, communicating, learning, and so on; ¾ underlying cause of the impairments – congenital, disease or infection, injury or trauma; ¾ severity of the disability; ¾ age of onset of disability; ¾ need for and use of medical and rehabilitative services, and personal assistance; ¾ need for and use of assistive devices; ¾ quality of life (or socio-economic profile) of persons with disabilities, compared with persons without disabilities; and ¾ barriers to full and equal participation in society in areas such as education, work, housing, transportation and the political sphere. - 150 -

WHO/ESCAP Training Manual on Disability Statistics

9.1.2 Output tabulations Once the questions and kinds of information are settled on, it is possible to begin the task of identifying relevant cross-tabulations. Cross-tabulations can be specified in terms of a population, one or several output data items and, where appropriate, the counting unit to be used. For example, for the total population, one can identify the basic or minimum cross-tabulations based on age, sex and disability status. A set of disability statistical tables for census was suggested in the United Nations’ Principles and Recommendations for Population and Housing Censuses (Revision I). These are the following with age, sex, and urbanrural residence as the main variables: 1. Total population, by type of disability, geographical division, urban/rural residence, whether living in household or institution, age and sex. 2. Households with one or more persons with disability, by type, size of household and urban/rural area. 3. Total population 15 years of age and over, by type of disability, marital status, urban/rural area, age and sex. 4. Population with disability, by cause and type of disability, urban/rural area, age and sex. 5. Population 5 to 29 years of age, by school attendance, type of disability, urban/rural area, age and sex. 6. Population 5 years of age and over, by educational attainment, type of disability, urban/rural area, age and sex. 7. Population 15 years of age and over, by activity status, type of disability, urban/rural area, age and sex.

For data on persons with disabilities, cross-tabulations can be based on: ¾ Age, sex, cause and type of disability, ¾ Age, sex, and severity of disability, ¾ Unmet need for health services in the last 12 months, severity of disability, and so on. Similar data items can be applied to more narrowly defined populations, for example populations of people with specific impairments. However, too detailed cross tabulations should be limited for surveys, especially if the sample size is small that the sampling design is not appropriate for generating statistics at a lower geographical division. It is always advisable to examine each cell in the table. There may be a need to collapse some of the categories so that those cells with zeroes or small frequencies can be avoided. For surveys, it is always advisable to include a table on sampling error which is usually measured in terms of standard error and the confidence intervals. By providing this information, the users will be informed on the range in which the true value for the population could fall. - 151 -

WHO/ESCAP Training Manual on Disability Statistics

The counting unit is the unit used to quantify the cross tabulation data – most commonly in terms of appropriate population numbers (ones, hundreds, thousands, millions), or percentages and rates. For surveys, since the generated statistics are just estimates based on samples, percentages or rates are preferred. If the numbers are to be included they should be roundedoff to hundreds or thousands. The important thing to remember when designing cross-tabulations of data is that it is not an exercise in arithmetic. If a statistical collection uses six variables, then the challenge is not to present a series of tables in which all possible combinations of these six variables is displayed as two-by-three tables. Rather, the challenge is to produce tabulations that have a purpose and enable the end user to address issues of interest. Each tabulation should be there for a reason: to provide data relevant to a purpose or issue. To ensure this, it is recommended to state, under each tabulation, the purpose and underlying issue that it has been designed to address. 9.1.3 Graphical representation of disability data The final step is the presentation of the outputs for the ultimate data user. Graphical methods of presentation can provide the user with a clear picture of the significance of the data, highlighting aspects of the data that might otherwise be invisible. An example of a graphical output summarising disability data is presented in Box 9.1.

- 152 -

WHO/ESCAP Training Manual on Disability Statistics

Box 9.1: Summary of Disability Data in Australia, 2003

Source: Disability, Ageing and Carers: Summary of findings, Australia, 2003 (Cat. No. 4430.0).

The tree diagram in Box 9.1 shows how the two populations (those with and those without disabilities) are compared, which can be useful when investigating issues of equalization of opportunities. The further subdivisions create increasingly more refined subpopulations in terms of kind of restriction, severity, and, finally, whether the person is living in a household or in a cared accommodation. All these data are presented clearly and quickly in graphical format. There is often more than one way to present disability data. It is useful to think carefully about the aspect of the data that is important. For example, both diagrams in Box 9.2 present an age profile of person with disability, but emphasize different aspects of the data.

- 153 -

WHO/ESCAP Training Manual on Disability Statistics

Box 9.2: Age Profile of Persons with disabilities: Two Diagrams 2001 Census of Population and Housing, Sri Lanka

Survey of Disability, Ageing and Carers, Australia, 2003

The first diagram in Box 9.2 compares age distribution of the general population with that of person with disability. It shows that the distributions are very different and that, understandably, a large proportion of person with disability are in the higher age groups. The second diagram presents a graph of disability prevalence rates by age and sex. This diagram shows clearly that - 154 -

WHO/ESCAP Training Manual on Disability Statistics

the disability prevalence rate for both males and females aged 85 years and over is greater than 80 per cent. The same data is presented, but the impact is very different. 9.1.4 Map representation of disability data When comparing statistics by geographical subdivisions, e.g., regional, provincial, towns, and others, one very effective way of presenting the results is through the use of statistical maps. The map shows different colors that reflect different values. The colours are used for easy identification of areas with the highest and lowest values. For instance, the statistical map below that shows the data about disability rates in the different regions in the Philippines uses several colours with dark orange representing the highest disability rate and the lightest colour as the lowest. The regions with the highest disability rate are Eastern Visayas, Western Visayas, Mimaropa, and the Bicol Region (1.46 to 1.74 percent).

- 155 -

WHO/ESCAP Training Manual on Disability Statistics

Box 9.3: Disability Rates by Region, Philippines: 2000 0.54 - 1.09 1.10 - 1.25 1.26 - 1.45 1.46 - 1.74

Cordillera Region Cagay an Valley Ilocos Region

Central Luzon

Metro Manila CALABARZON

Bicol Region

Eastern Visay as MIMAROPA Western Visay as

Central Visay as

CARAGA Northern Mindanao Western Mindanao Central Mindanao Muslim Mindanao Southern Mindanao

Source: Philippines National Statistics Office, 2000 Census of Population and Housing.

- 156 -

WHO/ESCAP Training Manual on Disability Statistics

9.2

Analysis – Turning data into information

Statistical agencies are now taking a greater role in analysing the data their instruments collect. Rather than solely producing tabulations and handing over raw data to other agencies for analysis, statistical agencies are now more in the business of “adding value” to their data by engaging in preliminary analyses. This can be as simple as converting the number of persons with disabilities into a percentage of the overall population, or as complex as employing sophisticated mathematical modelling techniques to interpret the data. How do we turn disability data into disability information? 9.2.1 High quality data The value and usefulness of output information depends on the quality of the input data. Previous chapters in this manual have outlined the phases in the data collection process – from consultation with clients, to sampling design, testing, and derivation of tabulations – that are designed to yield good data. Disability statistics, perhaps more than any other area of social statistics, is vulnerable to distortions of data. Worldwide prevalence rates vary remarkably. Although in part this reflects real differences in chronic and infectious disease patterns, differential life expectancy, age structures, nutritional status, exposure rates to environmental hazards, war, and other public health problems, most of the differences can be traced to the quality of the data. Differences in the operationalization of disability, screening procedures, collection methods, and different methods of calculating disability rates, produce different prevalence estimates. Even within countries, different studies have produced widely different estimates of disability prevalence because they have used different definitions, instruments and procedures to collect the data. For instance, as earlier discussed, estimates of the percentage of persons with disabilities are lower when impairment questions rather than disability questions are used to identify persons with disabilities. This explains in part why the reported disability prevalence rates of Africa and Asia are lower than those of Europe and North America. In addition, when impairment questions are used for screening purposes, the resulting disability rates for men are higher than those of women. In contrast, when activity and participation screening questions are used, rates are similar for women and men, and in some cases disability rates for women are higher. It is widely believed, both among statisticians and persons with disabilities organizations, that the prevalence rates derived from impairment-based data collections largely under report actual disability incidence, thus compromising the quality and usefulness of the statistics.

- 157 -

WHO/ESCAP Training Manual on Disability Statistics

9.2.2 Pitfalls in analysing disability data Even with the highest quality disability data, the analyst must be aware of potential pitfalls in the analysis of this data. Many of the traps described below are standard and apply to the analysis of statistical data about any subject matter. Some of the traps, however, are particular to disability data analysis. Wrong population A common analytical mistake is to define the wrong population for the issue under analysis. A good example is the analysis of disability across the population in terms of data collected for people who access rehabilitation services. As mentioned in the earlier chapter, this population is likely to be composed of people with severe disabilities, and only those who receive services. Not included are those with severe disabilities who do not receive rehabilitation services as well as those with mild or moderate disabilities who do not require the specific types of support provided by these services. Additionally, as explained in the previous chapter, a survey of people living in private households (not institutions) yields understated data since older people and people with severe disabilities are more likely to be living in institutions. More generally, a common source of error in analysis comes from ignoring the effect of sampling variability. No sample is perfectly representative; there is bound to be some measure of difference between the estimate of prevalence of disability in a sample population, and that in the whole population. In a survey in New Zealand in 1996, for example, the disability rate was 19.1 percent in urban areas and 18.6 percent in rural areas. Was this a genuine difference, or merely a fluctuation caused by sampling variability? In this case, the difference was not found to be statistically significant; but even if it was, it would still be relevant to ask whether it represents the reality. Drawing unsupportable conclusions The task of a data analyst is to take valid, reliable, and high quality data then, draw conclusions about what the data mean and what they tell us about the disability. The process of drawing conclusions can go awry in many different ways, but these errors all share the same underlying problem: the data does not truly support the conclusion. The most common example is drawing conclusions that are plausible (because they appeal to unquestioned beliefs we hold) but which, when scrutinised, really do not have much data to support them. There are other cases which are less obvious and so potentially more dangerous.

- 158 -

WHO/ESCAP Training Manual on Disability Statistics

Ignoring the impact of other variables A standard problem in an analysis is assuming that one or more variables are responsible for an observed phenomenon, when in fact it is yet another, independent variable that accounts for all the data. For example, in New Zealand, the disability rate for the Maori population is lower than the non-Maori population. Should we conclude that if you are a Maori person you have a lower probability of having a disability than if you are a non-Maori person? No, we should not, since the higher disability rate for the non-Maori population is probably largely due to age: the Maori population is younger than the non-Maori population. Standardising for age, the two rates are roughly equivalent. Due to the nature of disability, age is almost always a relevant variable in analysis. In some social contexts, gender also matters. For example, men tend to work in jobs that have a high accident rate. Relevant cross tabulations are perhaps the best way to discover whether there are independent variables affecting disability prevalence rates. Endogenous factors A similar error can occur if the independent variable is a general social phenomenon that could easily go unnoticed, and would not normally be included in the analysis. These are sometimes called endogenous factors. For example, between 1986 and 1991 the number of Canadians reporting some degree of disability increased from 13.2 per cent to 15.5 percent. While the increase could be partially attributed to an ageing population and a change in the survey methodology, analysts suspected that these factors alone did not account for the increase. It was suggested that the increase in awareness of disability in Canadian society between the two survey dates made people more willing to respond affirmatively to questions about limitations in their activities and barriers they encounter in their everyday lives. Other potential endogenous factors might include promises of increases in welfare assistance and other programmes to those who identify themselves as having a disability, or outright payments to people who participate in the collection activity if they report their disability.

Masking If an analysis is conducted on highly aggregated data, trends of magnitude or direction may be masked unless the data is disaggregated by region, population group, or some other parameter. For example, the total disability rate in a country may not show any significant change over time, even though the rate may well have increased dramatically in a particular region because of a rapidly ageing population, natural disaster, or other factors. It is therefore a potential source of error not to consider conducting analyses on - 159 -

WHO/ESCAP Training Manual on Disability Statistics

disaggregated data to confirm the validity of trends at the aggregated level. Causal confusions Analytical conclusions that claim that the data supports a causal link between variables are subject to many pitfalls. The most obvious error is to claim that there is a causal relationship between variables on the basis of data that merely shows a correlation (that may well be coincidental). Causal errors are common in the analysis of disability data. For example, looking at the data for unemployment among people with intellectual disabilities, one might be strongly tempted to say that intellectual impairment might causally responsible for the low employment rates. Yet, although these variables are undoubtedly correlated, this may not be a cause at all. Recalling the mistake of ignoring independent variables described above, it may well be that the causes of unemployment are employers’ attitudes and behaviours, based on stereotypes and misunderstanding of the true work capacity of people with intellectual disabilities. Inconsistent definitions We have noticed on several occasions that disability statistics present special problems because the notion of ‘disability’ has been variously defined in surveys and censuses. The primary virtue of the ICF approach to disability statistics is that it makes absolutely clear that ‘disability’ is a complex term with three distinct dimensions, each of which can be precisely classified and measured. ICF makes it clear that the variability of definitions in statistical data collection has primarily been the result of data collection designers not clearly identifying which dimension, or which dimensions of disability, their collection is all about. The problem for the analyst, of course, is that for data that is not grounded in the ICF model, it is very difficult to determine what the answers to disability questions actually mean. Data with different definitions will not be comparable, and conclusions drawn will be unsupportable. Even if ICF terminology is used, some definitional issues still remain. For example, what do we mean by the disability rate for children? Do we mean children with disabilities aged 15 years or less, as a percentage of the total children aged 0-14 years, or the number of children aged 5-14 years with a disability as a percentage of the total children at these ages? Some disability surveys do not collect data about children under 5 years because of difficulties in identifying disability amongst children at these ages. The unit of analysis also needs to be defined. Is the analysis based on the number of persons with disabilities or the number of disabilities? If the latter is chosen, then the same individual may be counted more than once. This data cannot be used to determine the number of persons with disabilities, given the significant number of individuals with multiple disabilities.

- 160 -

WHO/ESCAP Training Manual on Disability Statistics

9.3

Forms of output and dissemination strategies

Data must be disseminated in a form that is both relevant and accessible to users. This requires an understanding of who the users are, and their needs, as well as a strategy for promoting the availability of data to maximise the use of the information. There are various audiences for disability data: the general public; the media; persons with disabilities and their advocacy and support organizations; policy makers, in both public and private sectors; universities and research institutions; and other statistical organizations, local and international. Each audience has different information needs, and the form of dissemination should take these into account. 9.3.1 Presentation of data collection details with the results Generally, statistical tabulations and analyses should be accompanied by sufficient technical detail to satisfy the needs of the data users. The general public will likely require less technical detail than researchers or other statistical agencies. But, in any case, survey results should always be accompanied by a description of the survey limitations – such as sampling errors, response rates, and others. This is especially important for disability data given the history of wide differences in definitions of disability, screening procedures, and collection methods. Box 9.4 provides a guide to what should be included in explanatory materials that will accompany survey results and analyses. 9.3.2 Accessibility of data to persons with disabilities An important consideration in the dissemination of statistical reports is their accessibility to person with disabilities. The relevant modes of presentation include large-type, Braille, audio formats, electronic tables on disk with computer programmes and interfaces for people with intellectual disabilities. It is best to consult with disabled people’s organizations to enhance further strategies for ensuring its accessibility. 9.3.3 Dissemination strategies In addition to hard copy publications, increasingly, there is a demand for electronic publication of statistical information, in CD-ROM format or via the Internet. Many statistical agencies have websites where they post reports and statistical information and this is becoming an important medium for disseminating statistical tabulations and analyses.

- 161 -

WHO/ESCAP Training Manual on Disability Statistics

Box 9.4: Explanatory Materials for Survey Analysis Presentation: A Guide The following information should be included: A statement of the objectives of the survey or data collection, including a definition of the target population. A description of its coverage in terms of inclusion or exclusion of geographical regions, particular social groups or age groups, and any other categories of the population covered. Collection procedures, such as sample frame used sample selection procedure expected sample size achieved sample size, including sub-groups response rates, and how they are calculated non-response methodology and suggested reasons for non-response collection procedure including type of interviews date and duration of the fieldwork quality control (e.g. efforts to reduce non-sampling errors, interviewer training, imputation procedures). Sampling errors For each estimate reported there should be an associated measure of the sampling error (and method used to calculate the error). Assessment Interpretation of the reasons for the results, and recommendations for future action, such as further research or policy implementation. Responsibility Who commissioned the survey, undertook the work, wrote the report.

A dissemination strategy should be worked out during the initial planning phase (and certainly before data is available), considered and agreed on by advisory groups. The strategy should respond to the information needs of the users, and address the following issues: Timing: when should the data and analyses be released? Type and range of output: what format should the output come in – paper, electronic or both? Should it be put in publicly accessible form or only in specialised and restricted formats (e.g., confidential unit record files)? Type of analysis: should the analysis be set out in tables only, or tables with summary commentary and projections, or modelled analysis and analytical commentary, or some other forms?

- 162 -

WHO/ESCAP Training Manual on Disability Statistics

Shells of tables: should blank tables showing proposed table contents including populations, output data items and cross-tabulations and counting units be included? Methods of access to data: should the data and analyses be given free of charge or sold, and if sold should it be available for purchase on-line as well as hard copy form in public libraries or other outlets? Should access to data be made confidential and restricted to a few users? Promotional activities: What types of media releases with summary statistics, fact sheets showcasing important findings, or brochures and user guides will be produced and distributed? Can advocacy groups and other associations of persons with disabilities be included in promotion and dissemination of the data? 9.3.4 Confidentiality Confidentiality of respondent data is an issue of great importance. Care must be taken to ensure that published data cannot be linked to particular individuals, either directly or by inference. The risk of releasing identifiable data is greatest when the data is very detailed or disaggregated, but even in these cases, procedures exist for guaranteeing anonymity. Furthermore, survey, census and registry respondents should always be made aware of confidentiality assurance policies as part of questionnaire introductions so that fear of disclosure of personal information does not affect results. 9.3.5 Standard forms of output Publications Paper or electronic publication is the conventional media of data dissemination. These may take the form of statistical compendia containing large numbers of statistical tabulations or reports containing descriptive commentary and graphics. Reports are effective means of communicating statistical information in an accessible form to people who do not have the skills to extract the key trends and patterns from statistical tables. They can either give a general descriptive overview of the results or present focused results on specific areas of interest and concern. Reports, however, require more resources to produce than statistical compendia, which can interfere with the timeliness of data dissemination. It is therefore advisable to choose cheaper and faster media – such as the Internet – to first release the data, with more detailed, and accessible, reports following later. Customised data service Many statistical agencies provide a service for clients that enables them to request their own datasets or tables derived from the survey data. This service can be extremely useful to technically-adept data users since all - 163 -

WHO/ESCAP Training Manual on Disability Statistics

combinations of useful data could not possibly be provided in a single publication. Highly specialised combinations of data may be of interest to only a few users, but of great importance to them. A customised service makes it possible for all users to request tables of data that meet their specific needs. Usually, statistical agencies charge for this service. Microdata files More and more data users are demanding access to unit record or raw data so they can carry out their own manipulations and analyses of data. Published tabulations of data may not allow some users to undertake sophisticated analyses using multivariate and other statistical modelling techniques. Where raw data is released care should be taken to ensure that the files do not have personal identifiers that might undermine confidentiality. When microdata files are provided, statistical agencies usually charge a fee. The fee may be quite substantial – for example, the cost of de-identified Unit Record Files for most Australian social surveys costs $AUD 8,000. The revenue from the fee can be used to offset the development of these files. To illustrate the diversity of dissemination approaches, Box 9.5 gives some examples of recent disability data releases. 9.4

Ensuring secure data or accomplished questionnaires storage

Once collected the data or completed questionnaires need to be managed and stored, at this stage, security and privacy issues become crucial since data either from survey, census or administrative-based data collections contain personal information such as age, sex or address which could be used to identify an individual. For paper-based data holdings, identifiable information should be kept securely locked away when not in use, and access should be limited to a small number of people directly involved in the data collection. Whenever possible, the questionnaires should be disposed of immediately after being encoded into the computer. Some countries, however, are required to wait a certain number of years before they dispose the accomplished questionnaires. In the case of the Philippines, for example, the retaining period of census questionnaires is five years. For data collected electronically, or where the data capture process includes the name, address and some detailed identification of individuals, a measure of security should be guaranteed by providing individual user accounts with password protection, and automatic screen shutdown or automatic log-off.

- 164 -

WHO/ESCAP Training Manual on Disability Statistics

9.5

Documentation

Documentation is the process of recording all the events that transpired during the data collection process. It enumerates and describes the different procedures employed and reports all problems encountered and solutions adopted. This documentation informs the public of how the operation was conducted, allowing them to analyse and interpret the results fully. Additionally, the information contained in the documentation report can serve as guide in planning for the next survey/census/administrative-based operation of the same type. Furthermore, it allows for international comparison for it provides the basis for an exchange of information on content and procedures. The following should be included when preparing a documentation report: a. b. c. d. e. f. g. h. i. j.

Description of the methods used Production schedules and size of staff Budget estimates Calendar of activities Forms and manuals used Organization of statistical agency Definition of geographic areas List and description of equipment and facilities used Quality control instituted Memoranda and other additional instructions not included in the manuals k. Other relevant information

Documentation is one of the important aspects in any data collection but much of the time it is neglected. In many cases, statistical agencies find that the persons who are supposed to document the data collection process become tied up with new data collection operations. Documentation should not be passed on to whoever is available, but should be completed by those who were actually involved in the data collection.

Box 9.5: Disability Data Statistical Releases: Some recent examples 1.

Two stage data release from the Sri Lankan Disability Census of Population and Housing

The Sri Lankan Department of Census and Statistics first issued a bulletin giving summary results of disability by age, sex and district, together with a short descriptive review. The bulletin was also made available on the Department’s website. This was followed by a detailed report containing 118 statistical tables covering six groups of disabilities. In addition to prevalence information for each type of disability – by age, sex and urban and rural sector – the report contains tabulations cross-classified by age of onset, cause of disability, whether the person was living in an institution, his or her educational attainment and employment status. The report contained a commentary and graphics highlighting major patterns in the data.

- 165 -

WHO/ESCAP Training Manual on Disability Statistics

2.

Release of New Zealand disability survey results

Statistics New Zealand issued a media release as soon as preliminary results from its disability survey became available. The release was issued four months after the fieldwork was completed. A second media release was issued when final results from the survey were available. In addition, a report containing a selection of tabulations from the survey was prepared for the survey sponsors, with tables specified by the survey’s sponsor. The report contained a selection of 50 tables from the survey and ten pages of commentary and graphics highlighting major trends and patterns in the data. Documentation of the survey methods, survey population, sample design, standard errors, and disability definition were included. A further publication is planned, aimed at a general audience and containing additional analyses.

3.

Release of Australian disability survey products

The Australian Bureau of Statistics released the results of its 2003 Survey of Disability, Ageing and Carers in several publications and formats:

‰

‰ ‰ ‰ ‰ ‰ ‰

A published summary of findings was released less than a year after the end of the survey enumeration, and contained a broad selection of national estimates of disability, ageing and caring, including detailed estimates of the number of persons with disabilities and their demographic and socio-economic characteristics. Simultaneously, separate sets of tables for each State and Territory were released, in paper and electronic forms. This was followed within six months by two special topic reports and electronic table sets. A de-identified Unit Record File was released for detailed analysis by government agencies and researchers. Special data tabulations were made available on request. Short analytical articles based on survey data were published in the statistical compendium Australian Social Trends, and made available on the website. A Statistical User Guide was produced describing the objectives and content of the survey, the concepts, methods and procedures used in the collection of data and the derivation of estimates.

- 166 -

WHO/ESCAP Training Manual on Disability Statistics

Learning Objectives: Chapter 10 Disability Statistics: Key to Sound Policy Formation After reading this chapter, the reader should be able to: 1.

Describe policies that can be formed from disability data

2.

Have a basic understanding of the guidelines to policy formulation and plan of actions relating to disability

3.

Be aware of some examples of disability policies from various countries

“Blind messaging” Raniel Jose Castaneda WHO Photo Contest “Images of Health and Disability 2004/2005”

- 167 -

WHO/ESCAP Training Manual on Disability Statistics

10. DISABILITY STATISTICS: KEY TO SOUND POLICY FORMULATION 10.1 10.2

Policies that can be formed out of the given disability data Disability guidelines for policy formulation and plans of action

10.1

Policies that can be formed out of the given disability data

Disability statistics are collected for the purpose of policy formulation, academic research, and the monitoring and evaluation of programs and projects. In earlier chapters, it was stressed that the choice of the domains and questions to be included in the questionnaire depends mostly on the needs of users. Moreover, in the cycle of the data collection activity shown in Diagrams 5.1 and 5.2, the first and the last stages are user consultation. Thus, the cycle is not complete if the provider of statistics does not face the users again to disseminate statistics. Depending on the coverage or scope of data items collected, disability statistics could offer a wide range of beneficial information for policy formulation, monitoring and evaluation. Cross-tabulation of disability prevalence rate by socio-demographic characteristics of the population, for instance, could show differences across the different age groups, sex, educational attainment, occupation, and others. These differences could then be used in order to identify priority groups of people for policy and program development, especially in developing countries where resources are limited. Over time such information would indicate if the policy, program, or project implemented is successful or not as far as the targeted persons with disabilities are concerned. Some of the specific information which could directly relate to the formulation of policies is listed below: 1. Differences in rates of disability between males and females Gender differences may indicate discrimination against a gender regarding service provision in some countries. Hence, a definitive policy regarding equal access to facilities may be indicated. At the same time, it is possible in some countries that males would tend to have higher disability rate than that of females especially when the cause of disability is related to occupation. It is important, therefore, that causes of disability be known, apart from the presence of disability, to be able to formulate policies related to provision of safety measures in the workplace, insurance, and financial assistance to mitigate effects of accident-related injuries.

- 168 -

WHO/ESCAP Training Manual on Disability Statistics

2. Data on the cause of disability Identification of the cause of disability may assist policymakers to make decisions about the type of preventive programs appropriate for their country. In the Asia-Pacific region, for instance, the underlying causes of disability are malnutrition, diseases, congenital factors, accidents and violence, inadequate hygiene, war and landmine explosions, lack of access to a health care system, exposure to chemical substances, stresses, and others (Takamine, 2003), most of which are preventable. Hence, in order to reduce the incidence of disability, strategies may be focused in these areas. 3. Differences in service utilization between the populations with and without disabilities Policies may be formulated to increase access of population with disabilities in the different services and facilities, such as mandatory provision of ramps instead of stairs for those on wheelchairs. 4. Identification of physical and social barriers Information on what types and causes of restrictions persons with disabilities are facing related to basic services is useful for determining the environmental and social barriers that most urgently need to be addressed. In countries where resources are limited, identification of the type of assistive device most needed is important for the best allocation of funds. 5. Access to and participation in education and employment The data may provide information on how persons with disabilities are integrated with the educational system and the labour market. This is a critical issue related to disability policy. Discrimination against persons with disabilities, and unwillingness to bear the costs of creating a more accessible environment e.g. at schools or workstations are key obstacles to the improvement of the lives of persons with disabilities. Disability data may encourage governments to construct better facilities or provide tax incentives to schools or firms that accept persons with disabilities. 6. Income by age and type of disability Poverty is one of the causes of disability. This is because the poor are more exposed to dangerous working and living conditions, including lack of access to healthcare facilities, and poor nutrition, among others. In the Asia-Pacific region, close to 40 percent of the persons with disabilities are poor (Takamine, 2003). On the other hand, disability can also be a cause of poverty. This is particularly true if the persons with disabilities, and their caretakers, do not have the capacity to generate income for the family. Moreover, there may also be financial constraints brought about by the expensive medical treatment or assistive devices needed by persons with - 169 -

WHO/ESCAP Training Manual on Disability Statistics

disabilities. Data on income as well as other information regarding the economic status of the household may provide insight into how poverty can affect disability and vice-versa. There are yet a number of policies which may be formulated to improve the circumstances of persons with disabilities. Several of these policies have been internationally recognized and incorporated in the national policy programs of various countries. The next section discusses the guidelines and plans of action that direct the disability policies of some countries. 10.2

Disability guidelines for policy formulation

As described in Chapter 1, the utility of disability statistics in fulfilling a global commitment to promote and protect human rights of persons with disabilities in economic, social, cultural, political and legal life cannot be overemphasized. The Convention on the Rights of Persons with Disabilities (CRPD) in its article 31 states that States parties should undertake to collect appropriate information, including statistical and research data, to enable them to formulate and implement policies to give effect to the Convention. CRPD covers the following areas: Article 6 - Women with disabilities Article 7 - Children with disabilities Article 8 - Awareness-raising Article 9 - Accessibility Article 10 - Right to life Article 11 - Situations of risk and humanitarian emergencies Article 12 - Equal recognition before the law Article 13 - Access to justice Article 14 - Liberty and security of person Article 15 - Freedom of torture or cruel, inhuman or degrading treatment or punishment Article 16 - Freedom from exploitation, violence and abuse Article 17 - Protecting the integrity of the person Article 18 - Liberty of movement and nationality Article 19 - Living independently and being included in the community Article 20 - Personal mobility Article 21 - Freedom of expression and opinion, and access to information Article 22 - Respect for privacy Article 23 - Respect for home and the family Article 24 - Education Article 25 - Health Article 26 - Habilitation and rehabilitation Article 27 - Work and employment Article 28 - Adequate standard of living and social protection Article 29 - Participation in political and public life Article 30 - Participation in cultural life, recreation, leisure and sport In Asia and the Pacific, the Second Asian and Pacific Decade of Disabled Persons (2003 – 2012) promotes an inclusive, barrier-free and rights-based society and its regional policy guideline, the Biwako Millenium Framework for Action (BMF) provides the following seven priority areas: - 170 -

WHO/ESCAP Training Manual on Disability Statistics

1. 2. 3. 4. 5. 6. 7.

Self-help organizations of persons with disabilities and related family and parent associations Women with disabilities Early detection, early intervention and education Training and employment, including self-employment Access to built environments and public transport Access to information and communications, including information, communications and assistive technologies Poverty alleviation through capacity-building, social security and sustainable livelihood programmes

Appendix 4 includes the different targets in this framework. Biwako Plus Five, the supplement to the BMF emphasizes the importance of international cooperation, mutli-stakeholders efforts including the private sector, local government and participation of persons with diverse disabilities in decision-making processes concerning disability and in the implementation and evaluation of projects and policies. Biwako Plus Five has a specific strategy ,” Improving the availability and quality of data and other information on disabilities for policy formulation and implementation. Appendix 5 describes the strategies. Several countries have formulated their own policies on the basis of the above mentioned guidelines. For instance, the Philippines adopted a Magna-Carta for Disabled Persons which specifically states that: .... Disabled persons are part of Philippine society, thus the State shall give full support to the improvement of the total well-being of disabled persons and their integration into the mainstream of society. Toward this end, the State shall adopt policies ensuring them to compete favourably for available opportunities....7

India's policy about persons with disabilities is embodied in Persons with Disabilities (Equality of Opportunities, Protection of Rights and Full Participation) Act of 1995 which stressed the equality of opportunities for persons with disabilities. The People’s Republic of China with an estimated 20 million persons with disabilities living in poverty, has a disability policy formulated as follows: ... for the purpose of protecting the lawful rights and interests of, and developing undertakings for, disabled persons, and ensuring their equal and full participation in social life and their share of the material and cultural wealth of society8

7

Robert L. Metts, Ph.d, Disability Issues, Trends and Recommendations for the World Bank. (February, 2000). Source of basic document: Philippines, “Magna-Carta for Disabled Persons”, Official Gazette of the Republic of the Philippines, 4 May 1992, Vol.88, No. 18, 2537-2556. 8 Ibid. Source of basic document: People’s Republic of China, Law of the People’s Republic of China on the Protection of Disabled Persons, Chapter 1, General Provisions (China, 1990). - 171 -

WHO/ESCAP Training Manual on Disability Statistics

The national disability framework of Bangladesh is embodied in the “Bangladesh Persons with Disability Welfare Act”9. The Act lists the following priority areas: Disability prevention Identification Curative treatment Education Health care

9

Rehabilitation and employment Transport and communication Culture Social Security Self-help organizations

The Danish Bilharziasis Laboratory for the World Bank, People’s Republic of Bangladesh, Disability in Bangladesh, A Situation Analysis, Final Report, (May 2004). - 172 -

WHO/ESCAP Training Manual on Disability Statistics

BIBLIOGRAPHY Australian Bureau of Statistics (2006) A Disability Question in the 2006 Census: Development, Testing, Analysis. Canberra. AIHW (2004). Disability and its relationship to health conditions and other factors, Australian Institute of Health and Welfare, Cat. No. DIS 37, Canberra. Black, Ken (2006). Testing A Disability Question for the Census. in Altman, B. M. and S. N. Barnartt (ed.) International Views on Disability Measures, Moving Toward Comparative Measurement, Elsevier. Dudzik, Pamela; Elwan, Ann and Metts, Robert. (2000) Disability Policies, Statistics, and Strategies in Latin America and the Caribbean: A Review. Philippines National Statistics Office (2003). Socio-Demographic Characteristics, Volume I. Census of Population and Housing. Manila, Philippines. Smit, Jan and Liu, Wei (2007). Comparing disability questions for censuses and surveys in Asia and the Pacific, Statistics Division, Discussion Paper no. 1, Bangkok, ESCAP. Takamine, Yutaka (2003). Disability Issues in East Asia: Review and Ways Forward. The Danish Bilharziasis Laboratory for the World Bank (2004). Disability in Bangladesh: A Situation Analysis, Final Report. People’s Republic of Bangladesh. United Nations (1998). Principles and Recommendations for Population and Housing Censuses. Revision I. Series M No. 76 / Rev 1.New York. United Nations (2001), Guidelines and Principles for the Development of Disability Statistics, Statistics on Special Population Groups, Series Y No.10, New York. United Nations (2007). Principles and Recommendations for Population and Housing Censuses. Revision 2. New York. WHO (1980), ICIDH International Classi…cation of Disability and Health, World Health Organization, Geneva. WHO (2001) International Classification of Functioning, Disability and Health. World Health Organization, Geneva.

- 173 -

WHO/ESCAP Training Manual on Disability Statistics

APPENDICES Appendix 1 – Selected Countries on Disability Data and Methodology Appendix 2 – Questionnaire Used in the Pilot Test in Five Selected Countries Appendix 3 – Sample Data Collection Guide for WHO/ESCAP 2004 Pilot Studies Appendix 4 – Biwako Millennium Framework for Action Targets Appendix 5 – Biwako Plus Five

- 174 -

WHO/ESCAP Training Manual on Disability Statistics

Appendix 1: Selected Countries on Disability Data Collection and Methodology

CAMBODIA Source Statistical Indicators GDP per capita (PPP, USD), 2003 HDI ranking, 2003 Total Population (in thousands) National disability prevalence rate (%)

2,078 UNDP HDR, 2005 130 of 177 UNDP HDR, 2005 13,807 ESCAP Population Data Sheet, 2005 1.5 Cambodia SocioEconomic Survey, 1999

Disability Data Collections Main Instruments Social Economic Surveys (SES), Population Census Published Sources SES reports, “Social Statistics of Statistics Yearbook” Active Participating National Institute of Statistics, Organizations ESCAP, WHO Mode of Collection Face-to-face interviews for all surveys and censuses Coverage Complete only in urban areas due to high costs, low awareness, and other restrictions in rural areas. Covers households only; institutions’ registries also considered. Main To identify prevalence and spread Purpose/Users of and promote awareness for policy Disability Data makers, treatment/assistance providers, and persons with disabilities Future Plans Population Census in 2008, Continuous Cambodian Socioeconomic Survey (CSES) 2006-07, and Cambodia Accident and Injury Survey 2006 all may include modules for disability using a traditional, impairment approach. Methodology National Definition “A restriction or lack of ability to perform an activity in the manner of “Disability” or “Disabled Person” or within the range considered normal for a human being…It describes functional limitation or activity restriction caused by impairment.” Methodological Impairment-based questions Approach focusing on “type” and “cause” of disability

- 175 -

Country Paper for “Improving Disability Data for Policy Use” ESCAP Workshop, Sept 2003

Country Paper for “Improving Disability Data for Policy Use” ESCAP Workshop, Sept 2003 National Institute of Statistics

Country Paper for “Improving Disability Data for Policy Use” ESCAP Workshop, Sept 2003

Social Economic Survey (SES), 2003

WHO/ESCAP Training Manual on Disability Statistics

FIJI ISLANDS Source Statistical Indicators GDP per capita (PPP, USD), 2003 HDI ranking, 2003 Total Population (in thousands) National disability prevalence rate (%)

5,880 UNDP HDR, 2005 92 of 177 UNDP HDR, 2005 848 ESCAP Population Data Sheet, 2005 0.01 1981 Employment/Unemployment Survey

Disability Data Collections Main Instruments Census (1996), and HIES Survey (2003) Published “Disability Counts in Fiji”, 1996 Sources of Census Analysis Statistics Fiji Islands Bureau of Statistics, Active Fiji National Council of Disabled Participating Peoples (FNCDP), ESCAP, WHO, Organizations ABS Mode of Face-to-face interviews Collection Coverage Census and surveys cover urban and rural areas, registries are mainly urban only Main To create awareness of related Purpose/Users issues and provide information for of Disability Data policy makers Future Plans

Methodology National Definition of “Disability” or “Disabled Person” Methodological Approach

2007 Census will potentially include ICF-based disability module Must be long-term (more than 6 months) physical or mental difficulty categorized by” “sight, intelligence, hearing, physical, age, or other”; must be reason for unemployment (EUS) 3-question short set includes 1 activity and participation question referring to difficulties with communication and “other usual activities”

2002/3 HIES Survey, 2004/5 EUS

Country Paper for “Improving Disability Data for Policy Use” ESCAP Workshop, Sept 2003 Fiji Islands Bureau of Statistics

1996 Census, HIES 2002/3, EUS 2004/5

1996 Census

Hong Kong, CHINA Source Statistical Indicators GDP per capita (PPP, USD), 2003 HDI ranking, 2003 Total Population (in thousands) National disability prevalence rate (%) - 176 -

27,179 UNDP HDR, 2005 22 of 177 UNDP HDR, 2005 6,889 ESCAP Population Data Sheet, 2005 4.0 Survey on Persons with

WHO/ESCAP Training Manual on Disability Statistics

Disabilities and Chronic Diseases, 2000 Disability Data Collections Main Instruments Survey on Persons with Disabilities and Chronic Diseases (ad hoc) Published Survey Reports Sources of Statistics Census and Statistics Department Active (C&SD) Participating Organizations Mode of Face-to-face interviews Collection Coverage

Main Purpose/Users of Disability Data

Future Plans

Methodology National Definition of “Disability” or “Disabled Person”

Methodological Approach

Persons of all ages in households and most institutions (excluding prisons) and persons living on board marine vessels To estimate the prevalence rate for all disabilities and for selected categories, and for information on the characteristics of persons with disabilities New disability survey scheduled for 4th quarter 2006-3rd quarter 2007 Those who either (i) have been diagnosed by qualified health personnel as having one or more of the 7 conditions listed below, or (ii) had perceived themselves as having one or more of the first 4 conditions for at least 6 months: 1.restrictions in body movement; 2.seeing difficulty; 3.hearing difficulty; 4.speech difficulty; 5.mental illness; 6.autism; 7.mental handicap Impairment based questions and medical diagnosis; mental disability is not included in reported prevalence rate

C&SD

C&SD

Survey on Persons with Disabilities and Chronic Diseases, 2000 Survey on Persons with Disabilities and Chronic Diseases, 2000 C&SD

C&SD

Survey on Persons with Disabilities and Chronic Diseases, 2000

Survey on Persons with Disabilities and Chronic Diseases, 2000

INDIA Source Statistical Indicators GDP per capita (PPP, USD), 2003 HDI ranking, 2003 Total Population (in thousands) National disability prevalence rate (%) Disability Data Collections

2,892 UNDP HDR, 2005 127 of 177 UNDP HDR, 2005 1,103,371 ESCAP Population Data Sheet, 2005 1.8 NSSO Survey, 2002

- 177 -

WHO/ESCAP Training Manual on Disability Statistics

Main Instruments Published Sources of Statistics Active Participating Organizations

Mode of Collection Coverage Main Purpose/Users of Disability Data

Future Plans Methodology National Definition of “Disability” or “Disabled Person”

Methodological Approach

Census, NSSO Survey, voluntary registry Census and survey reports

Central Statistical Organization (CSO), Ministry of Social Justice and Empowerment, National Institute of Hearing and Handicapped, ESCAP, WHO, ABS Face-to-face interviews

NSSO Surveys

Households only For information on prevalence and cause and socio-economic characteristics of persons with disabilities for policy and service planning Census in 2011, NSSO Survey in 2012

NSSO Surveys, Census “Statistics on the Employment Situation of Persons with disabilities: A Compendium of National Methodologies” ILO, 2004 CSO

5 forms: sight, hearing, speech, movement, and mental; “A person with restrictions or lack of abilities to perform an activity in the manner or within the range considered normal for a human being…excludes illness/injury or recent origin (morbidity) resulting into temporary loss of ability” Impairment-based categorical model; 1 question in Census

2001 Census, 2002 NSSO Survey

2001 Census, 2002 NSSO Survey

INDONESIA Source Statistical Indicators GDP per capita (PPP, USD), 2003 HDI ranking, 2003 Total Population (in thousands) National disability prevalence rate (%)

3,361 UNDP HDR, 2005 110 of 177 UNDP HDR, 2005 222,781 ESCAP Population Data Sheet, 2005 0.7 National Socio-Economic Survey, 2003

Disability Data Collections Main Instruments National Socio-Economic Surveys Published Survey reports Sources of Statistics BPS-Statistics Indonesia, Ministry Active of Health, Ministry of Social Participating Affairs, PPCI disabled persons Organizations organization Mode of Face-to-face interviews Collection - 178 -

National Socio-Economic Surveys

WHO/ESCAP Training Manual on Disability Statistics

Coverage Main Purpose/Users of Disability Data

Future Plans

Methodology National Definition of “Disability” or “Disabled Person” Methodological Approach

Surveys include all provinces To identify the existence of disability, type of disability (blind, deaf, mute, deaf and mute, handicap, mental defect, psychotic) and cause of disability (nature, accident, disaster, disease, disturbance) 2006 National Socio-Economic Survey will include 3 ICF-based questions from self care, communication, and movement domains in addition to questions on types of difficulties, and the need for and use of assistance “A person with loss or abnormality in body structure or physiological function that influences or causes restriction on activity”; 3 categories: physical, mental, and physical/mental Impairment model

National Socio-Economic Surveys

BPS-Statistics Indonesia

Law No.4 (1997)

National Socio-Economic Survey, 2003

ISLAMIC REPUBLIC OF IRAN Source Statistical Indicators GDP per capita (PPP, USD), 2003 HDI ranking, 2003 Total Population (in thousands)

6,995 UNDP HDR, 2005 99 of 177 UNDP HDR, 2005 69,515 ESCAP Population Data Sheet, 2005 N/A

National disability prevalence rate (%) Disability Data Collections Main Instruments Census (1976, 1986, and 1996) Published 1986 Census Report Sources of Statistics Active Statistical Centre of Iran (SCI) Participating Organizations Mode of Face-to-face interviews Collection Coverage 5 districts in 5 provinces Main Welfare organizations Purpose/Users of Disability Data Future Plans May use census to design a sampling frame for follow-up survey Methodology 2 working definitions: (1) National categorical (blind, deaf, voice and Definition of speech disorder, amputation, “Disability” or - 179 -

Censuses SCI

SCI

2005 Census pre-test

WHO/ESCAP Training Manual on Disability Statistics

“Disabled Person”

Methodological Approach

mental disorder, etc.) and (2) need for assistance or aid devices for self care, movement, or communication activities Yes/No question on physical impairment or handicap, impairment/categorical model to identify type of disability

2005 Census pretest

MONGOLIA Source Statistical Indicators GDP per capita (PPP, USD), 2003 HDI ranking, 2003 Total Population (in thousands) National disability prevalence rate (%)

1,850 UNDP HDR, 2005 114 of 177 UNDP HDR, 2005 2,646 ESCAP Population Data Sheet, 2005 3.5 NSO-Mongolia Official Statistics

Disability Data Collections Main Instruments Limited registry in capital city Published N/A Sources of Statistics National Statistical Office, ESCAP, Active WHO, ABS Participating Organizations Mode of Collection Coverage Registry system records Main information on the disability type, Purpose/Users of Disability Data cause, and age group Future Plans Disability module in 2010 Census, a national disability survey, and revision to the current registry Methodology National Persons with “physiological and Definition of mental impairment” as certified by “Disability” or Medical and Labour Commissions “Disabled Person” Methodological Voluntary, impairment-based Approach registry

- 180 -

Provision No.3 of law on “Social security of the persons with disabilities” and working definition for NSO

WHO/ESCAP Training Manual on Disability Statistics

PHILIPPINES Source Statistical Indicators GDP per capita (PPP, USD), 2003 HDI ranking, 2003 Total Population (in thousands) National disability prevalence rate (%)

4,321 UNDP HDR, 2005 84 of 177 UNDP HDR, 2005 85,237 ESCAP Population Data Sheet, 2005 1.23 Population and Housing Census, 2000

Disability Data Collections Main Instruments Population and Housing Census Published Census reports Sources of Statistics Active Philippines National Statistics Participating Office (PNSO), National Statistical Organizations Coordination Board (NSCB), National Council for the Welfare of Disabled Persons (NCWDP), ESCAP, WHO, ABS Mode of Face-to-face interviews Collection Coverage Total population Main Purpose/Users of Disability Data

Future Plans

Methodology National Definition of “Disability” or “Disabled Person”

Methodological Approach

To enable planners to prepare education, development, and health programs geared towards prevention, rehabilitation, and the integration of persons with functional difficulties into mainstream society New, ICF-based module in a regular survey and revised question set in Population and Housing Census Any restriction or lack of ability (resulting from an impairment) to perform an activity in the manner or within the range considered normal for a human being. Impairments may be physical, mental, or sensory/motor. Impairment/categorical model; yes/no question on “physical or mental disability”

- 181 -

Population and Housing Census, 2000 Population and Housing Census, 2000 Population and Housing Census Enumerator’s Manual, 2000

PNSO

Population and Housing Census, 2000

Population and Housing Census, 2000

WHO/ESCAP Training Manual on Disability Statistics

THAILAND Source Statistical Indicators GDP per capita (PPP, USD), 2003 HDI ranking, 2003 Total Population (in thousands)

7,595 UNDP HDR, 2005 73 of 177 UNDP HDR, 2005 64,261 ESCAP Population Data Sheet, 2005 1.7 Disability Survey, 2002

National disability prevalence rate (%) Disability Data Collections Main Instruments Disability Surveys (every 5 years), Bureau of Empowerment for Persons with Disabilities national registry, Censuses Published Survey reports, regular reports Sources of from Bureau of Empowerment for Statistics Persons with Disabilities NSO, International Health Policy Active Program (IHPP), Sirindhorn Participating Medical Rehabilitation Centre, Organizations Bureau of Empowerment for Persons with Disabilities, Ministry of Social Development and Human Security, ESCAP, WHO Mode of Face-to-face interviews Collection Coverage Persons 7 years of age and older in households To collect data on persons with Main disabilities and provide adequate Purpose/Users of Disability Data services Future Plans Methodology National Definition of “Disability” or “Disabled Person” Methodological Approach

Disability Survey, 2007 Disability Survey, 2007 Disability Survey, 2007, Bureau of Empowerment for Persons with Disabilities Registry

N/A Impairment and activity limitations (6 months or more)

Disability Survey, 2007

Impairment and activity limitation questions

Disability Survey, 2007

- 182 -

WHO/ESCAP Training Manual on Disability Statistics

Appendix 2: Questionnaire Used in the Pilot Test in Five Selected Countries

ESCAP/WHO Project on Health and Disability Statistics

Disability Question Set Testing Study One: Specificity and Sensitivity Testing Study Two: Test-Retest Reliability Study Three: Cognitive Interview

Questionnaire Version A

May 2005

- 183 -

WHO/ESCAP Training Manual on Disability Statistics

SECTION 1. Face Sheet

ITEMS F1- F5 ARE TO BE COMPLETED BY INTERVIEWERS PRIOR TO STARTING EACH INTERVIEW F1

RESPONDENT I.D . #

Centre # - Subject # - Interview time point

F2

INTERVIEWER I.D. #

Centre # - Interviewer #

F3

INTERVIEW TIME POINT (1, 2, ETC.)

F4

a) INTERVIEW DATE

___ ___/___ ___/___ ___ month

b) STARTING TIME

day

year

___ ___/___ ___ hrs min

c) TIME INTERVIEW ENDED

___ ___/___ ___ hrs min

d) TOTAL DURATION

___ ___/___ ___ hrs min F5

F6

LIVING SITUATION AT TIME OF INTERVIEW (CIRCLE ONLY ONE)

SAMPLE (CIRCLE ONLY ONE)

Independent in Community

1

Assisted Living

2

Hospitalized

3

General population

1

Other (specify)

6

__________________________

- 184 -

WHO/ESCAP Training Manual on Disability Statistics

SECTION 2. DEMOGRAPHIC AND BACKGROUND INFORMATION PREAMBLE

SAY TO RESPONDENT: This interview has been developed by the WHO / ESCAP Project on Health and Disability Statistics to better understand the difficulties people may have due to their health conditions. The information that you provide in this interview is confidential and will be used only for research. FOR RESPONDENTS FROM THE GENERAL POPULATION SAY: Even if you are healthy and have no difficulties, it is necessary that I ask all of the questions for completeness. I will begin with some background questions. A1

Female Male

RECORD SEX AS OBSERVED

1 2

A2

How old are you now?

___/___ years

A3

How many years in all did you spend studying in school, college or university?

___/___ years

A4

What is your current marital status? (SELECT THE SINGLE BEST OPTION)

Never married

1

Currently married

2

Separated

3

Divorced

4

Widowed

5

Cohabiting

A5 Which describes your main work status best? (SELECT THE SINGLE BEST OPTION)

Paid work

1

Self employed, such as own your business or farming

2

Non paid work, such as volunteer or charity

3

Student

4

Keeping house/Homemaker

5

Retired

6

Unemployed (health reasons) 7 Unemployed (other reasons)

8

Other (specify)

9

_________________________

- 185 -

WHO/ESCAP Training Manual on Disability Statistics

SECTION 3. DISABILITY QUESTION SETS

Question Set 1

The next questions ask about difficulties you may have doing certain activities because of a HEALTH PROBLEM W1 W2 W3

No

Some

A lot

Unable

Do you have difficulty seeing, even if wearing glasses?

1

2

3

4

Do you have difficulty hearing, even if using a hearing aid?

1

2

3

4

Do you have difficulty walking or climbing steps?

1

2

3

4

1

2

3

4

1

2

3

4

1

2

3

4

W4

Do you have difficulty remembering or concentrating?

W5

Do you have difficulty (with self-care such as) washing all over or dressing?

W6

Because of a physical, mental, or emotional health condition, do you have difficulty communicating, (for example understanding or being understood by others)?

- 186 -

WHO/ESCAP Training Manual on Disability Statistics

Question Set 2 Part 1: Introduction

SAY TO RESPONDENT: The interview is about difficulties people have because of health conditions. (HAND FLASHCARD #1 TO RESPONDENT). By health condition I mean diseases or illnesses, other health problems that may be short or long lasting, injuries, mental or emotional problems and problems with alcohol or drugs. I remind you to keep all of your health problems in mind as you answer the questions. When I ask you about difficulties in doing an activity think about (POINT TO FLASHCARD #1). • • • •

Increased effort Discomfort or pain Slowness Changes in the way you do the activity

(POINT TO FLASHCARD #1). When answering, I’d like you to think back over the last 30 days. I also would like you to answer these questions thinking about how much difficulty you have, on average over the past 30 days, while doing the activity as you usually do it.

(HAND FLASHCARD #2 TO RESPONDENT). Use this scale when responding. (READ SCALE ALOUD): None, mild, moderate, severe, extreme or cannot do.

(FLASHCARDS #1 AND #2 SHOULD THROUGHOUT THE INTERVIEW. )

REMAIN

- 187 -

VISIBLE

TO

THE

RESPONDENT

WHO/ESCAP Training Manual on Disability Statistics

Part 2: Questions on body functions I am going to ask some questions about your body functions. Please remember that I am asking only about difficulties you experienced in the last 30 days that are due to health problems. POINT TO FLASHCARDS #1 AND #2 None

Mild

Moderate

Severe

Extreme /Cannot Do

B1.1

How much of bodily aches or pains did you have?

1

2

3

4

5

B1.2

How much bodily discomfort did you have?

1

2

3

4

5

B1.3

Have you had a problem with a skin defect of face, body, arms or legs?

1

2

3

4

5

B1.4

Have you had a problem with your appearance due to missing or deformed or paralyzed arms, legs, feet?

1

2

3

4

5

1

2

3

4

5

1

2

3

4

5

1

2

3

4

5

1

2

3

4

5

1

2

3

4

5

B1.5

B1.6

B1.7

B1.8

B1.9

How much difficulty did you have in using your hands and fingers, such as picking up small objects or opening or closing containers? How much difficulty did you have in seeing and recognizing a person you know across the road? (take into account eye glasses, if you wear them) Read the brackets if you see respondent wearing glasses. How much difficulty did you have in seeing and recognizing an object at arm’s length or in reading? (take into account eye glasses, if you wear them) Read the brackets if you see respondent wearing glasses. How much difficulty did you have in hearing someone talking on the other side of the room in a normal voice? (take into account hearing aids, if you use them) Read the brackets if you see respondent using hearing aid.. How much difficulty did you have in hearing what is said in a conversation with one other person in a quiet room? (take into account hearing aids, if you use them) Read the brackets if you see respondent using hearing aid..

B1.10

How much of a problem did you have passing water (urinating) or in controlling urine (incontinence)?

1

2

3

4

5

B1.11

How much of a problem did you have with defecating, including constipation?

1

2

3

4

5

How much difficulty did you have with shortness of breath at rest?

1

2

3

4

5

B1.13

How much difficulty did you have with shortness of breath with mild exercise, such as climbing uphill for 20 meters or stairs (such as 12 steps)?

1

2

3

4

5

B1.14

How much difficulty did you have with coughing or wheezing for ten minutes or more at a time?

1

2

3

4

5

B1.15

How much of the time did you have a problem with sleeping, such as: falling asleep, waking up frequently during the night or waking up too early in the morning?

1

2

3

4

5

B1.16

How much of a problem did you have with feeling sad, low or depressed?

1

2

3

4

5

B1.17

How much of a problem did you have with worry or anxiety?

1

2

3

4

5

B1.12

- 188 -

WHO/ESCAP Training Manual on Disability Statistics

Part 3: Questions on Activities and Participation DOMAIN 1 Understanding and Communicating I am going to ask some questions about understanding and communicating. Please remember that I am asking only about difficulties you experienced in the last 30 days that are due to health problems. POINT TO FLASHCARDS #1 AND #2

None

Mild

Moderate

Severe

Extreme /Cannot Do

D1.1

How much difficulty did you have in concentrating on doing something for ten minutes?

1

2

3

4

5

D1.2

How much difficulty did you have in remembering to do important things?

1

2

3

4

5

D1.3

How much difficulty did you have in analysing and finding solutions to problems in day to day life?

1

2

3

4

5

D1.4

How much difficulty did you have in learning a new task, for example, learning how to get to a new place?

1

2

3

4

5

D1.5

How much difficulty did you have in generally understanding what people say?

1

2

3

4

5

D1.6

How much difficulty did you have in starting and maintaining a conversation?

1

2

3

4

5

DOMAIN 2 Getting Around I am now going to ask you about difficulties in getting around. Please remember that I am asking only about difficulties you experienced in the last 30 days that are due to health problems. POINT TO FLASHCARDS #1 AND #2

None

Mild

Moderate

Severe

Extreme /Cannot Do

D2.1

How much difficulty did you have in standing for long periods such as 30 minutes?

1

2

3

4

5

D2.2

How much difficulty did you have in standing up from sitting down?

1

2

3

4

5

D2.3

How much difficulty did you have in moving around inside your home?

1

2

3

4

5

D2.4

How much difficulty did you have in getting out of your home?

1

2

3

4

5

D2.5

How much difficulty did you have in walking a long distance such as a kilometre [or equivalent]?

1

2

3

4

5

- 189 -

WHO/ESCAP Training Manual on Disability Statistics

DOMAIN 3 Self Care I am now going to ask you about difficulties in taking care of yourself. Please remember that I am asking only about difficulties you experienced in the last 30 days that are due to health problems. POINT TO FLASHCARDS #1 AND #2

None

Mild

Moderate

Severe

Extreme /Cannot Do

D3.1

How much difficulty did you have in washing your whole body?

1

2

3

4

5

D3.2

How much difficulty did you have in getting dressed?

1

2

3

4

5

D3.3

How much difficulty did you have in eating?

1

2

3

4

5

D3.4

How much difficulty did you have in staying by yourself for a few days?

1

2

3

4

5

DOMAIN 4 Getting along with people I am now going to ask you about difficulties in getting along with people. Please remember that I am asking only about difficulties you experienced in the last 30 days that are due to health problems. POINT TO FLASHCARDS #1 AND #2

None

Mild

Moderate

Severe

Extreme /Cannot Do

D4.1

How much difficulty did you have in dealing with people you do not know?

1

2

3

4

5

D4.2

How much difficulty did you have in maintaining a friendship?

1

2

3

4

5

D4.3

How much difficulty did you have in getting along with people who are close to you?

1

2

3

4

5

D4.4

How much difficulty did you have in making new friends?

1

2

3

4

5

D4.5

How much difficulty did you have in sexual activities?

1

2

3

4

5

- 190 -

WHO/ESCAP Training Manual on Disability Statistics

DOMAIN 5 Life Activities I am now going to ask you about difficulties involved in household and work or school related activities. Please remember that I am asking only about difficulties you experienced in the last 30 days that are due to health problems. POINT TO FLASHCARDS #1 AND #2

None

Mild

Moderate

Severe

Extreme /Cannot Do

D5.1

How much difficulty did you have in taking care of your household responsibilities?

1

2

3

4

5

D5.2

Doing your most important household tasks well?

1

2

3

4

5

D5.3

How much difficulty did you have in getting all the household work done that you needed to do?

1

2

3

4

5

D5.4

How much difficulty did you have in getting your household work done as quickly as needed?

1

2

3

4

5

D5.5

How much difficulty did you have in your day to day work/school?

1

2

3

4

5

D5.6

How much difficulty did you have in doing your most important work/school tasks well?

1

2

3

4

5

D5.7

How much difficulty did you have in getting all the work done that you need to do?

1

2

3

4

5

D5.8

How much difficulty did you have in getting your work done as quickly as needed?

1

2

3

4

5

DOMAIN 6 Participation in Society Now, I am going to ask you about your participation in society and the impact of your health problems on you and your family. Some of these questions may involve problems that go beyond the last 30 days, however in answering, please focus on the last 30 days. Again, I remind you to answer these questions while thinking about health problems. None

Mild

Moderate

Severe

Extreme /Cannot Do

D6.1

How much of a problem did you have joining in community activities (for example, festivities, religious or other activities) in the same way as anyone else can?

1

2

3

4

5

D6.2

How much of a problem did you have because of barriers or hindrances in the world around you?

1

2

3

4

5

D6.3

How much of a problem did you have living with dignity because of the attitudes and actions of others?

1

2

3

4

5

D6.4

How much time did you spend on your health condition, or its consequences?

1

2

3

4

5

D6.5

How much have you been emotionally affected by your health condition?

1

2

3

4

5

D6.6

How much has your health been a drain on the financial resources of you or your family?

1

2

3

4

5

D6.7

How much of a problem did your family have because of your health problems?

1

2

3

4

5

- 191 -

WHO/ESCAP Training Manual on Disability Statistics

D6.8

How much of a problem did you have in doing things by yourself for relaxation or pleasure?

1

2

3

4

5

Part 4: Questions on need for assistance I am going to ask some questions about your need for assistance.

No

Yes, sometimes

E1

Do you ever need someone to help with, or be with you for, self care activities? For example: doing everyday activities such as eating, showering, dressing or toileting.

1

2

3

E2

Do you ever need someone to help with, or be with you for, body movement activities? For example: getting out of bed, moving around at home or at places away from home.

1

2

3

E3

Do you ever need someone to help with, or be with you for, communication activities? For example: understanding, or being understood by, others.

1

2

3

E4

What are the reasons for the need for assistance or supervisions shown in questions 1, 2, and 3? (Mark all applicable reasons)

a) No need for help or supervision b) Short-term health condition (lasting less than six months) c) Long-term health condition (lasting six months or more) d) Disability (lasting six months or more) e) Old or young age f)

Difficulty with English language

g) Other cause

This concludes our interview, thank you for participating. INTERVIEWER: Return to page 1 and record TIME INTERVIEW ENDED (Question F4c) and calculate TOTAL DURATION of interview (Question F4d).

- 192 -

Yes, always

WHO/ESCAP Training Manual on Disability Statistics

Flashcard #1

Having difficulty with an activity means: • • • •

Increased effort Discomfort or pain Slowness Changes in the way you do the activity

Think about the past 30 days only

Health Conditions: • Diseases, illnesses or other health problems • Injuries • Mental or emotional problems • Problems with alcohol • Problems with drugs

- 193 -

WHO/ESCAP Training Manual on Disability Statistics

Flashcard #2

1 None

2

3

Mild

Moderate

- 194 -

4 Severe

5 Extreme / Cannot Do

WHO/ESCAP Training Manual on Disability Statistics

Appendix 3: Sample Data Collections SECTION I

HOW TO ADMINISTER THE QUESTIONNAIRE 1. OBJECTIVES The questionnaire has been developed by WHO and ESCAP for testing of disability questions sets for use in census and surveys. The data collected will help to identify the best set of questions that explain the greatest variance in disability with the smallest number of questions. The instrument is intended to be used in different cultures and a variety of settings. The questionnaire is divided into the following sections: Coversheet Section 1: Face Sheet Section 2: Demographics and Background Information Section 3: Question Sets Question Set 1 Question Set 2 Part 1: Introduction Part 2: Questions on body functions Part 3: Questions on activities and participation Part 4: Questions on need for assistance The appendix includes: Appendix A: Flashcard #1 Appendix B: Flashcard #2 To avoid a response bias due to an order effect, the questionnaire comes in two versions. Version A lists Question Set 1 followed by Question Set 2, Version B starts with Question Set 2 and then proceeds to Question Set 1. This manual is a training tool for interviewers who will administer the questionnaire.

2. RESPONDENTS The questionnaire is to be administered to adults aged 18 or older from different educational and cultural backgrounds. Respondents do not need to be literate, although, depending on age, maturity, cognitive ability, and place of origin, respondents may find some questions or concepts more difficult to understand than others. Cards (i.e. the flashcards found in the Appendix) should be provided to respondents as a memory aid. From the total of all respondents 50% should be interviewed with Version A and the other 50% with Version B of the questionnaire.

- 195 -

WHO/ESCAP Training Manual on Disability Statistics

3. DURATION OF INTERVIEWS The interview is expected to last 40 minutes on average, but may take longer depending on the comprehension and literacy level of the respondent. Respondents who have problems with language, who are less educated, who are very talkative or who have many difficulties because of their health condition may take longer to answer questions.

4. CONDUCTING THE INTERVIEW IN PRIVATE The respondent should be interviewed in private and no other member of the household should be present. If total privacy is not possible, the respondent may have to be interviewed outside the house.

5. TYPES OF QUESTION FORMAT Two types of question format are used in the questionnaire. •

Matrix questions

These are questions that use a common set of answer categories. These questions are asked to obtain information about degrees of difficulties, frequency or related qualities. D1.1.

How much difficulty did you have in concentrating or doing something for 10 minutes? None Mild Moderate Severe Extreme/Cannot Do

1 2 3 4 5

W1. Do you have difficulty seeing, even if wearing glasses? No 1 Some 2 A Lot 3 Unable 4 E1. Do you ever need someone to help with, or be with you for, self care activities? For example: doing everyday activities such as eating, showering, dressing or toileting. No Yes, sometimes Yes, always

1 2 3 - 196 -

WHO/ESCAP Training Manual on Disability Statistics

•

Multiple-choice questions

In this type of question, the respondent is asked to choose one from several options. A4. What is your current marital status? Never married Currently married Separated Divorced Widowed Cohabiting

1 2 3 4 5 6

6. RECORDING TIME There are several occasions in which the interviewing is asked to record the time. The time variable is stated as “Starting Time” to mark the beginning of the interview and “Time Interview Ended” to mark the end. Time should be recorded following international conventions. 00 = MIDNIGHT

12 = 12PM (Noon)

01 = 1AM

13 = 1PM

02 = 2AM

14 = 2PM

03 = 3AM

15 = 3PM

04 = 4AM

16 = 4PM

05 = 5AM

17 = 5PM

06 = 6AM

18 = 6PM

07 = 7AM

19 = 7PM

08 = 8AM

20 = 8PM

09 = 9AM

21 = 9PM

10 = 10AM

22 = 10PM

11 = 11AM

23 = 11PM

Time should be recorded using 4 digits: 0 9/0 0 hrs/mins

- 197 -

WHO/ESCAP Training Manual on Disability Statistics

SECTION II GENERAL INTERVIEWING GUIDELINES

A. GENERAL OVERVIEW

1. STANDARDIZATION Standardization means that every interviewer must conduct the interview the same way with each respondent. This is done to minimize differences in responses that might occur if formatting conventions or interviewing techniques changed with every respondent. A respondent might answer very differently if he was to be interviewed with other people in the room or if the interviewer was unpleasant or patronizing. Likewise, a rushed interview or the interviewer’s lack of interest may greatly affect responses. Research has shown that the interviewer can have an effect on the data collection because of four factors: socioeconomic background, unconventional administration of the questionnaire, wording and intonation, and reaction to respondent’s difficulties in understanding the questionnaire. Except for the interviewer’s background, the other three factors can be addressed in training. 2. ROLE OF INTERVIEWER The interviewer is responsible for asking questions, answering the respondent’s queries, recording answers and editing the questionnaire. He must check that the respondent has understood the questions by using interviewing techniques, such as neutral probes, clarification and appropriate feedback, and determine whether the answer given is appropriate. Listening to what the respondent is communicating, both verbally and non-verbally, will ensure that the information is correct. The interviewer must set the pace of the interview and keep the respondent focused and interested. The atmosphere should be comfortable and pleasant at all times. Before going to the field, the interviewer must know the questionnaire and how it is to be administered. A thorough preparation as well as extensive practice will guarantee that this is achieved.

3. ROLE OF RESPONDENT The role of the respondent is to cooperate with the interviewer and follow his instructions. He must listen to questions attentively without interrupting, take his time before answering, and try to give an accurate and complete response as much as possible. The respondent should ask for clarifications whenever a question seems unclear to him and ask the interviewer to repeat or rephrase it. Trying to answer an unclear question is likely to lead to an incorrect response. - 198 -

WHO/ESCAP Training Manual on Disability Statistics

4. ROLE OF SUPERVISOR The role of the supervisor is to check that interviewers perform their work well. This includes handling the logistics of the survey, coordinating with other staff, recruiting and training additional interviewers, and supervision. Supervision must be given to interviewers before, during, and after the interview. Supervisors must not only check that contact procedures are followed correctly but that interviews are conducted appropriately. They should be present for 10 interviews and ensure that standardized interviewing techniques are observed when asking questions, clarifying, probing and giving feedback in a nondirective manner. After the interview has been conducted, they must also check that data is coded and entered correctly. Supervisors must give feedback and debrief on a regular basis, in order to update the organization responsible for the study on the progress of the survey and any problems that have arisen.

B. QUESTIONNAIRE CONVENTIONS

Objectives: • • •

Identify and use interviewer instructions correctly throughout the questionnaire Recognize typographical conventions and what they mean Learn how to use visual aids

1. INTERVIEWER INSTRUCTIONS •

Anything written in standard print is to be read to the respondent.

Example: The first questions are about your overall health, including both your physical health and your mental health. •

Anything written in italics and in bold (uppercase or lowercase) is an interviewer instruction and should not be read aloud

Example: Question Set 2 Part 1. Introduction SAY TO RESPONDENT The interview is about difficulties people have because of health conditions. (HAND FLASHCARD #1 TO RESPONDENT). By health - 199 -

WHO/ESCAP Training Manual on Disability Statistics

condition I mean diseases or illnesses, other health problems that may be short or long lasting, injuries, mental or emotional problems and problems with alcohol or drugs.

2. UNDERLINED TYPE •

Words which are underlined within questions are key words or phrases that need to be emphasized when read to the respondent. Example: D2.3 How much difficulty did you have in moving around inside your home?

3. VERBATIM ENTRIES •

A line beside the question indicates where the response must be recorded. Example: How old are you now?

•

_/_ years

An option which contains a line is meant to be filled in.

4. PARENTHESIS •

Items in parentheses ( ) contain examples to illustrate a point and are to be read to the respondent. Example: W6. Because of a physical, mental, or emotional health condition, do you have difficulty communicating, (for example understanding or being understood by others)?

5. BRACKETS •

Items in brackets [ ] contain:

Instructions to translators. English-speaking interviewers may also use these guidelines, whenever needed, to clarify a question and/or make it relevant to the respondent’s culture. Example: D2.5. How much difficulty did you have in walking a long distance such as a kilometre [or equivalent]?

- 200 -

WHO/ESCAP Training Manual on Disability Statistics

6. VISUAL AIDS •

The function of visual aids is to help respondents remember important information while answering questions and rate different items.

Visual aids are cards that describe terms (e.g. difficulties, health condition etc.) and illustrate scales (e.g. severity response scale) Enough time must be allowed for the respondent to examine the cards. Interviewer instructions are given throughout the questionnaire so the interviewer knows when to produce the visual aids and how to use them. Practice is recommended beforehand in order to facilitate the flow of the interview.

C. INTERVIEWER INSTRUCTIONS

Objectives • •

Learn how to introduce yourself well Learn how to use interviewing techniques in a standardized way

1. INTRODUCTION TO THE INTERVIEW The interviewer must clearly communicate the objectives of the survey to the respondent. Knowing what is expected of the respondent will contribute to the accuracy in responses. The interviewer should establish a good rapport by introducing himself and the survey clearly and completely. •

Make a good impression

1. You are a professional interviewer from a legitimate and reputable organization. 2. The questionnaire is for gathering data for important, worthwhile research. 3. The respondent’s participation is vital to the success of the research. 4. The responses given will be confidential and will only be used for research purposes. Example of introduction: The interviewer might used the following introduction (or find something that works better):

“Hello my name is…and I work for…. The reason I am contacting you is because we are conducting a study on health in our country and I would like to - 201 -

WHO/ESCAP Training Manual on Disability Statistics

ask you a few questions. Let me assure you that whatever information you tell us will not be disclosed to anyone and will only be used for research purposes.” •

Conduct the interview clearly and pleasantly

1. You should be pleasant and assertive, and make the respondent feel at ease. 2. You should know the questionnaire thoroughly and be well prepared to answer any questions. 3. You should speak slowly and clearly to set the tone for the interview. 4. You should adapt your introduction to the respondent, as different respondents require different amounts of information. 5. You should be motivated and interested in the interview. 2. ASKING QUESTIONS When asking questions, the interviewer should point out that there are no right or wrong answers and that the interview is not a test. There are rules in interviewing that should be followed to avoid biased answers and to ensure comparability of data. Options must be read aloud to the respondent, except for Don’t know (DK), Refuse, Not Applicable (NA) and other. • • • • • • • •

Read questions as they are written in the text and do not change the wording (paraphrase or rephrase). Do not change the order of the questions. Read the questions slowly and clearly, emphasizing key words. The pace should be about two words per second. Read the questions in a pleasant voice that conveys interest, assurance and professionalism. Maintain eye contact or whatever is culturally appropriate. Read the entire question to the respondent making sure that he has heard all of it. If respondent interrupts before the end, the question should be repeated. Do not skip questions even if the respondent has given the answer earlier or if one answer applies to questions that are similar, unless you are certain of the answer. Verify information volunteered by respondent. The respondent may volunteer information before a question is asked. If the interviewer asks that question when it comes up and ignores what the respondent has said earlier, he may be annoyed because he thinks that the interviewer was not listening to him.

Two approaches can be taken: A. Ask questions with a preface to acknowledge the fact that the respondent has already provided information relevant to a particular question. The interviewer may read the question with a preface. - 202 -

WHO/ESCAP Training Manual on Disability Statistics

“You told me that before…but I still need to ask you this question as it is written”. B. Skip questions only if the interviewer knows with certainty that a question has already been answered. This is not recommended. •

Use lead-in statements, such as “how much difficulty did you have in…” to help the respondent answer the question and keep the interview moving.

The interviewer should not assume what the respondent will say, or think, because of health condition or lifestyle, that the respondent is bound to answer one way rather than another. The interviewer may be tempted to skip questions because or make comments such as “I know this probably doesn’t apply to you, but...” This practice may prevent getting accurate and unbiased information or learn to what extent answers to earlier questions actually do predict answers to later ones. The interview should not be rushed and the respondent must be allowed enough time to understand and answer a question. If the respondent feels pressured to give a quick reply, he may answer anything that crosses his mind or say that he “doesn’t know”. In addition, trying to have a rushed interview will slow things down, as questions will need to be repeated a second time. 3. CLARIFICATION Clarification is needed when the respondent is unable to answer a question because he does not understand it completely or at all. When to clarify: • • • •

Whenever the respondent does not seem to understand the question and gives an inappropriate reply. When the respondent does not seem to have heard the question. When the respondent takes a lot of time to think about his answer. When the respondent asks about a specific part of the question. In this case it is acceptable to repeat only that part.

When the respondent asks for one option to be repeated, all the options should be read again. Assumptions should not be made about how the respondent is going to respond. An option may be omitted if the respondent has clearly eliminated it. When the respondent asks for one term to be clarified, the interviewer should refer to the Question by Question Specifications (see Section III). If the definition does not exist, he should ask the respondent to answer the question according to whatever the question means to him.

4. PROBING Probing is needed when the respondent seems to understand the question but gives a response which does not meet the objectives of the question. It is mainly used to encourage the respondent to expand on what has been said or - 203 -

WHO/ESCAP Training Manual on Disability Statistics

to clarify his response, while keeping him focused to avoid incomplete answers or irrelevant information. When to probe: • • • • •

The respondent does not seem to understand what is asked, misinterprets the question, cannot make up his mind, or digresses from the topic. The respondent seems to have understood the question but has not answered appropriately. The respondent has replied correctly but may have other things to say. Silence may be the best probe and will give him time to think and expand on his answer. A look or a nod may also encourage communication. The respondent has not given a complete answer or his answer is unclear. The respondent needs to give more information.

Probing techniques • •

• •

Just repeat the question. The respondent may come up with the right answer if he hears the question a second time. Pause. This gives the respondent time to collect his thoughts and expand on his answer if he has more to say. The interviewer must be sensitive enough to know when to use a pause and for how long. Usually a pause together with an expectant look or a nod will encourage communication. Repeat the respondent’s reply. This is often a very effective way of having the respondent reflect of the answer he has just given. The interviewer can repeat the question as he is recording it. Use neutral introductions to avoid biasing responses. Do not ask leading questions or suggest answers such as “I guess you mean…”, as they may influence the respondent. Instead say: overall, generally speaking….The interviewer should never give the impression that he approves or disapproves what the respondent says, or that his answer is right or wrong. If the respondent asks for his opinion, the interviewer should say that he is interested in what the respondent has to say and that he needs to keep the interview going. (See Table 1.)

Common probing situations • Don’t know (DK) When the respondent says “I don’t know” to a question, the general rule is to repeat the question. If the respondent still does not know, the interviewer should probe once before recording (DK). An effort at recall should be encouraged with a probe such as: Could you give me your best estimate? Which would be closer? If there is no DK option, DK should be recorded in the right margin. If after probing the answer is still “don’t know”, it may mean that the respondent: - 204 -

WHO/ESCAP Training Manual on Disability Statistics

- Does not understand the question but is afraid of saying it - Is taking time to think and wants to gain time - Does not want to answer because of personal reasons

•

Not applicable (NA)

Occasionally, the interviewer may ask a question that the respondent feels does not apply to him. The interviewer should ask him why he thinks this, and write down NA if it is clear that the question is irrelevant. If this option is not available, it should be recorded in the right margin. In order for probing to be successful, the interviewer must be able to see why the respondent’s response is inadequate and does not answer what is asked. To avoid hurting the respondent’s feelings when he has been unclear, the interviewer should mention that perhaps he has misunderstood him. 5. FEEDBACK It is important that the interviewer tells the respondent when he/she is doing well. It is also a way of maintaining control over the interview. This will keep the motivation going and encourage good performance. Acceptable performance includes the following: • • • •

Listening to the whole question without interrupting. Giving appropriate and complete answers to the questions. Answering in a way that meets the objectives of the questions. Avoiding digression.

Rule for giving feedback • • • • • • •

Feedback should be used to keep the respondent focused, discourage digression or inappropriate enquiries. Positive feedback must be given for good performance and not good content. Feedback can be verbal as well as non-verbal, such as a smile or a nod. Short feedback sentences should be used for short responses and longer feedback sentences for longer responses. There should be a brief pause after feedback for more effect. Vary the type of feedback by using different phrases. Certain comments such as “let me make a note of this” can motivate the respondent if he feels that what he is saying is important.

Feedback must always be neutral and the interviewer should acknowledge the respondent’s performance by using the following feedback sentences: Short feedback: to acknowledge responses to close-ended questions - 205 -

WHO/ESCAP Training Manual on Disability Statistics

Thank you/Thanks I see All right Long feedback: to reinforce respondent motivation and attention on long series of questions, open-ended questions, or questions that are difficult for the respondent. That is certainly useful/helpful information. It is useful to get your ideas on this. I see, that’s helpful to know. That can be difficult to remember/answer. Task-related phrases: to acknowledge that what the respondent has said is important and worth recording. Let me get that down. Let me make sure I have got that right (repeat answer). Let me go over what you have just told me. In addition to listening to what the respondent is saying, it is useful to pay attention to the gestures and tone of voice, as they can often give a better indication of what the respondent is trying to say if his verbal answer is confusing or inarticulate. Anger or frustration may not come through verbally but may be communicated non-verbally. Situations requiring feedback •

Respondent makes inappropriate enquiries and asks for advice or information, or wants to know about the interviewer's personal experiences. Suggested phrases: In this interview, we are really interested in learning about your experiences. When we finish, let us talk about that. We will come to that later.

•

Respondent digresses from the questions by giving lengthy responses or unnecessary information. I have many more questions to ask so we should really move on. If you would like to talk more about that, perhaps we can do it at the end of the interview.

•

Respondent gives inappropriate responses or feels like conversing. Silence can be quite effective in this case.

- 206 -

WHO/ESCAP Training Manual on Disability Statistics

Table 1 - STANDARD INTERVIEWING TECHNIQUES Clarification

Probes

Overall …

Neutral prefaces

Can you be more specific? Generally What is your best speaking … estimate? Whatever … What do you means to you mean by that? Whatever you In what sense are think is … you saying that? Let me repeat the question again Let me repeat the different options again The definition for … is

Feedback

Overall …

I see …

Well, in general … Generally speaking … In the country as a whole

I get your point

Neutral prefaces

Feedback

That is useful information It is important to know what your opinion on this is What do you Let me repeat the Thank you for think? question … your clarification on this Which would be Yes, but … I understand what closer to your you are saying condition? Would you say Of course, it is Your comments that you strongly difficult to know are very helpful agree or disagree?

Clarification

Probes

Can you tell me There are no right more about that? or wrong answers … Can you think of We are just any other interested in your examples? opinion … How is that? We all hope that In what way? … Anything else? Can you explain?

Let me make a note of what you have just said Let me make sure I understand correctly

6. RECORDING DATA Rules for data entry The questions must not only be asked correctly but also recorded correctly in order to have unbiased and reliable data. The respondent’s answers must all be noted down. It is important to record not only what the respondent said, but also how he said it to be able to recreate the atmosphere of the interview and get an idea of the respondent’s personality. A number of things can be done to achieve this:

- 207 -

WHO/ESCAP Training Manual on Disability Statistics

A. Write down the responses during the interview •

Responses must be transcribed while the respondent is talking, as the interviewer might not remember exactly what was originally said at a later stage.

B. Report what is said, the way it is said (verbatim reporting) • •

• • • •

To preserve the flavour of each interview, responses must be recorded word-for-word, with no attempt to correct grammar, or alter phrases or expressions. Responses must not be summarized or paraphrased as data may be misreported or missed out. The length of the response, the words, examples and expressions used by the respondent, all provide valuable information. Anything that is related to the question should be noted even if the answer is long. Qualifications such as “if” and “but” should be especially recorded, as well as expressions showing intensity of feelings e.g. “very”, “a little”. Hesitations, mumbling or irrelevant feedback should not be recorded. Comments or explanations must be recorded in brackets in the questionnaire next to the corresponding question. Keep the respondent’s interest. If the interviewer is too absorbed recording the answers the respondent may get bored. It is useful to say the respondent’s response aloud as the interviewer writes it down. This will allow the respondent to modify his reply or expand on it once he hears it.

Tips for taking notes • •

It is important to find a comfortable place for writing. The interviewer should sit in front of the respondent or in a place where what he writes cannot be seen. As soon as the respondent starts talking, the interviewer should start writing in order to reduce the waiting time for the next question.

Techniques for data entry •

• •

•

Use a pencil for writing. It will be easier to erase any information from the questionnaire or rewrite words or sentences. The interviewer should not erase any notes made, as they can be useful. Red ink is not to be used as it is for editor’s corrections. The information must be legible. Illegible handwriting is not of much use if only the interviewer can read it. Check that all the questions have been asked. If a question has been skipped by mistake, it can be corrected. If the respondent decides to change his mind on one of the options, the new answer must then be recorded. Each interview must be identified and the following information must be noted on each questionnaire: interviewer’s name, project number, sample ID, interview number, and date of interview.

- 208 -

WHO/ESCAP Training Manual on Disability Statistics

How to record data A. Closed-ended questions When a question requires that one option be circled, the interviewer must neatly circle one number and make sure that no other number is. If an incorrect answer is circled because the respondent has changed his mind or because the interviewer has made a mistake, the answer should be crossed out with a (/) over it, and the correct option circled. B. Open-ended questions These questions should be filled in capital letters. C. Fill-in questions If entering a number, answer should be “right justified” such as AGE ___/___. D. Marginal notes When the respondent gives an option, as well as additional descriptions such as “if”, “except” or “but”, such qualifications should be recorded in the right margin since they may give important information to the researchers. E. Uncertainty about the respondent’s answer If the interviewer is uncertain about a respondent’s answer, the question should be repeated and the answer recorded exactly (but remember that paraphrasing a response when in doubt is not permitted). If the interviewer understands the response but is not sure about the coding, he should record enough information in the left margin for the coder to decide what it should be. He should also use a question mark (?) in the left margin to indicate the uncertainty to the editor. If a question does not apply to a respondent and “NA” is not an option in the questionnaire, “NA” will then be recorded on the left margin and entered in the data entry program. Most questions will have “NA” in the program. The same thing will be done for the response “DK”. If after probing, the respondent is unable to give an answer, “DK” should be recorded and later entered in the program. F. Missing data If the interviewer accidentally misses any question, the interviewer enters MISSED in the right margin of the form. This indicates to the coder that the question was not asked. During an interview, if an interviewer notices that he missed a question, he should go back and ask the question, making a note in the margin that the question was asked out of sequence. If the missing data is not discovered until after the interview, the researcher must re-contact the respondent.

- 209 -

WHO/ESCAP Training Manual on Disability Statistics

Refusals to answer questions should always be recorded. The interviewer should write “Refuse”. Before accepting a refusal, the interviewer should explain the objective of the question to the respondent.

7. EDITING Before leaving the household, the interviewer should review the questionnaire to check that it is complete and that no questions have been omitted. Right after the interview, the interviewer should spend time checking the questionnaire to ensure that: • • • • •

All the questions have been answered. The information recorded is clear and legible for others to read. Comments are indicated by putting them between slashes. Open-ended questions are written in full sentences and not in abbreviations. The Interviewer Observations and Report is completed.

The interviewer should also edit the coversheet and check that: • • •

There is no missing information, such as interviewer’s name, interview number, date, length of interview and length of editing. The address is correct on the sample label. Every attempt call to the household is recorded, as well as the re-contact information.

The interviewer should submit the completed form to the study supervisor promptly, so that any errors in administration can be noted and procedures corrected before other interviews are completed incorrectly.

SECTION III QUESTION BY QUESTION SPECIFICATIONS This purpose of this section is to indicate what is intended by each question in the WHO/ESCAP Questionnaire for Disability Question Set Testing. Interviewers should use this information when respondents request clarification about specific questions and should refrain from offering their own interpretations. Each section is listed alphabetically based on the letter that precedes the number of the question. SECTION 1. FACE SHEET QUESTIONS F1 – F6 This section should be completed by the interviewer. - 210 -

WHO/ESCAP Training Manual on Disability Statistics

F1.

Respondent I.D. The Centre # is the three-digit number assigned by the study or project coordinator. If you have forgotten your centre number, please contact your study or project coordinator The Subject # is the unique four-digit number you should assign to that respondent. It is imperative that the number assigned to the respondent be recorded exactly the same way for all interviews, to ensure reliability. The Interview time point indicates whether the interview is the first or second time the respondent has been interviewed. Enter "1" if the respondent is interviewed for the first time (initial interview). Enter “2” if the person is interviewed for the second time (retest interview). The retest should be done by a different interviewer

F2.

Interviewer I.D. Under Centre # enter the three digit number that identifies your center. Under Interviewer I.D. # enter the three digit number that identifies the interviewer.

F3.

Interview Time Point Indicate whether the interview is the first or second time the respondent has been interviewed. Enter "1" if the respondent is interviewed for the first time (initial interview). Enter “2” if the person is interviewed for the second time (retest interview). The retest should be done by a different interviewer. Check entry on the last box of F1.

F4. Interview date, starting time, time interview ended and total duration Enter the date the interview took place. Write Month/Day/Year, and fill in the blanks with zeros. For example, February 4, 2004 would be written as 02/04/04. Record when the interview begins and ended and calculate the duration of the interview. F5.

Living situation at the time of the interview Indicate the type of residence in which the respondent resides. "Independent in Community" means the respondent lives on his/her own, with family, or friends in the community. "Assisted Living" refers to a respondent who lives in the community but receives regular assistance with at least some daily activities (e.g., shopping, bathing, meal preparation). This may include physical help, verbal reminders or cues, supervision, or psychosocial assistance. Assistance may be provided from a family member, friend, or professional care.

- 211 -

WHO/ESCAP Training Manual on Disability Statistics

"Hospitalized" should be indicated if the respondent resides in a 24-hour supervised setting (e.g. nursing home, hospital, rehabilitation facility). F6.

Sample Sample refers to the primary population category of the respondent: General population Other (specify) ______________

Write the name of respondent and circle code 1 if respondent is original or 2 if proxy. Proxy respondents are those who assist the original respondents in communicating directly to the interviewer. Fill in the name of interviewer and the supervisor shall also fill in his/her name and affix respective signature. SECTION 2. DEMOGRAPHIC AND BACKGROUND INFORMATION QUESTIONS A1 – A5 Read the preamble first. SAY TO RESPONDENT: This interview has been developed by the WHO / ESCAP Project on Health and Disability Statistics to better understand the difficulties people may have due to their health conditions. The information that you provide in this interview is confidential and will be used only for research. FOR RESPONDENTS FROM THE GENERAL POPULATION SAY: Even if you are healthy and have no difficulties, it is necessary that I ask all of the questions for completeness. I will begin with some background questions. Caution: For cases when respondents are obviously not healthy, refrain from mentioning disability and/or difficulties in the preamble to avoid embarrassment on the part of the respondent. This section should be completed with reference to the person completing the interview. A1.

Record sex as observed Circle option 1 or 2.

A2.

How old are you now? Record age.

A3.

How many years in all did you spend studying in school, college or university? The term “school” refers to any kind of formal school and excludes short courses (typing, sewing) or religious education such as Bible school or - 212 -

WHO/ESCAP Training Manual on Disability Statistics

Koran school. It includes technical or vocational training beyond formal school. If a person dropped out of school or university, partial years should not be counted. If someone has been in school both full and parttime, record the number of years at full-time education. A4.

What is your current marital status? Ask this question without reading the options to the respondent. If the response does not match any of the options e.g. “single”, read the options to the respondent and ask him/her to choose the option that best reflects his/her current marital status. For example, if the respondent is currently married (but was divorced in the past), the option “currently married” should be circled.

A5.

Which describes your main work status best? We are interested in the option that best reflects the current main work status. If the respondent gives two options e.g. “homemaker” or “unemployed” ask the respondent to pick the one that best describes their work status. There is no minimum number of hours per week that a respondent must work to qualify for the paid work category. Similarly, students need not be full time in order to endorse that choice. If the respondent reports being unemployed, interviewers should ask: “is this for health reasons or for other reasons” and record the appropriate option. The option “nonpaid (volunteer)” also includes work, which is non-paid, that a person does to help the family. Examples of this kind of work may be farming or helping run the family business.

SECTION 3. DISABILITY QUESTION SETS QUESTION SET 1 The questions ask about difficulties you may have doing certain activities because of a HEALTH PROBLEM. Choices of responses should always be mentioned to the respondents. These choices are No, Some, A Lot and Unable. QUESTIONS W1 – W6 W1.

Do you have difficulty seeing, even if wearing glasses? The purpose of this question is to assess a person’s vision taking into account the use of an assistive device.

W2.

Do you have difficulty hearing, even if using a hearing aid? The purpose of this question is to assess a person’s hearing taking into account the use of an assistive device. - 213 -

WHO/ESCAP Training Manual on Disability Statistics

W3.

Do you have difficulty walking or climbing steps? This question refers to walking or climbing steps at home or outside the home. Any assistive devices or personal help that is usually in place should be taken into account.

W4.

Do you have difficulty remembering or concentrating? The purpose of this question is to assess a person’s memory and difficulties in concentration. Any memory or concentration aids (e.g., writing things down, relying on electronic reminder systems) should be taken into consideration when rating this question.

W5.

Do you have difficulty (with self-care such as) washing all over or dressing? This question refers to washing one’s entire body, as it is customary for the culture. The question also includes all aspects of dressing the upper and lower body. Any assistive devices or personal help usually in place should be taken into account.

W6.

Because of a physical, mental, or emotional health condition, do you have difficulty communicating (for example, understanding or being understood by others)? The respondent should consider his/her usual mode of communication (e.g. spoken language, sign language, with an assistive device such as a hearing aid) and rate the degree of difficulty in understanding messages of others.

QUESTION SET 2 PART 1. INTRODUCTION Read the introductory part of this section. QUESTION SET 2 PART 2: QUESTIONS ON BODY FUNCTIONS B1.1 – B1.17 The respondent is asked to think of the last 30 days. A period of 4 weeks is an optimum period for comparison. Studies have shown that beyond 4 weeks there is a very steep fall in recall. The notion of difficulty is emphasized and the respondent is asked to describe his condition rather than say how he feels about it. B1.1. How much of bodily aches or pains did you have?

- 214 -

WHO/ESCAP Training Manual on Disability Statistics

This refers to any form of pain that interferes with a person’s usual activities. By pain we mean “physical” pain for a short or long period of time over any part of the body. B1.2. How much of bodily discomfort did you have? This refers to any form of discomfort that interferes with a person’s usual activities. By discomfort we mean “physical” low threshold pain for a short or long period of time over any part of the body.

B1.3. How much have you had a problem with a skin defect of face, body, arms or legs? Problems include pimples, warts, or changes in colour or scarring over large portions of face, body, arms or legs (due to burns, wounds), surgery, skin diseases, skin infections, etc. B1.4. How much have you had a problem with your appearance for example missing or deformed or paralyzed arms, legs, feet? The respondent should consider problems he/she has had in living with his/her appearance in the community. Missing, deformed or paralyzed limbs either from birth or due to an accident, a disease. B1.5. How much difficulty did you have in using your hands and fingers, such as picking up small objects or opening or closing containers? This includes using hand, fingers and thumb to handle objects, picking up, manipulating and releasing them, such as required to lift coins off a table or turn a dial or knob. B1.6. How much difficulty did you have in seeing and recognizing a person you know across the road? (take into account eye glasses, if you wear them) Read the brackets if you see respondent wearing glasses. The respondent should answer this question taking into account any assistive device (glasses or contact lenses) she may be using. “Seeing and recognizing a person” under normal circumstances. One would not be expected to recognize a person on a foggy or stormy day for example. B1.7. How much difficulty did you have in seeing and recognizing an object at arm’s length or in reading? (take into account eye glasses, if you wear them) Read the brackets if you see respondent wearing glasses. The respondent should answer this question taking into account any assistive device (glasses or contact lenses) he/she may be using. - 215 -

WHO/ESCAP Training Manual on Disability Statistics

B1.8. How much difficulty did you have in hearing someone talking on the other side of the room in a normal voice? (take into account hearing aids, if you use them) Read the brackets if you see respondent using hearing aid. By “normal” voice we mean not speaking loudly or shouting. Any hearing aid should be taken into account. B1.9. How much difficulty did you have in hearing what is said in a conversation with one other person in a quiet room? (take into account hearing aids, if you use them) Read the brackets if you see respondent using hearing aid. This refers to hearing in a quiet room with no background noise. Any hearing aid should be taken into account. B1.10. How much of a problem did you have passing water (urinating) or in controlling urine (incontinence)? This includes any problems urinating -- such as urine retention, inability to control urine, increase of the frequency of urinating or feeling pain during urinating. B1.11. How much of a problem did you have with defecating, including constipation? This may be diarrhoea as well as constipation. B1.12. How much difficulty did you have with shortness of breath at rest? Difficulty in shortness of breath without having done any exercise. B1.13. How much difficulty did you have with shortness of breath with mild exercise, such as climbing uphill for 20 meters or climbing stairs (such as 12 steps)? Difficulty in shortness of breath after having done some exercise. B1.14. How much difficulty did you have with coughing or wheezing for ten minutes or more at a time? Coughing or wheezing, without having done any exercise or without any reasons that would make a person cough or wheeze, e.g. smoky environment. B1.15. How much of the time did you have a problem with sleeping, such as: falling asleep, waking up frequently during the night or waking up too early in the morning? Inability to fall asleep, interrupted sleep or waking up earlier than usual.

- 216 -

WHO/ESCAP Training Manual on Disability Statistics

B1.16. How much of a problem did you have with feeling sad, low or depressed? Feeling tearful, having lost interest in things, feeling low and tired, not looking forward to anything. B1.17.How much of a problem did you have with worry or anxiety? Preoccupied more than usual with things that one would normally not worry about. QUESTION SET 2 PART 3: QUESTIONS ON ACTIVITIES AND PARTICIPATION D1.1 – D6.8 DOMAIN 1: UNDERSTANDING AND COMMUNICATING Domain 1 includes questions about communication and thinking activities. Specific areas that are assessed include concentrating, remembering, problem solving, learning and communicating. D1.1. How much difficulty did you have in concentrating on doing something for ten minutes? This question is intended to determine the respondent’s rating of difficulty with concentration for a short period, defined as 10 minutes. Generally, respondents have no problem understanding this item. If clarification is requested, however, respondents should be encouraged to think about their concentration in usual circumstances, not when they are preoccupied by a problem or in an unusually distracting environment. They may be cued to think about concentration while doing work tasks, reading, writing, drawing, playing a musical instrument, assembling a piece of equipment etc. D1.2. How much difficulty did you have in remembering to do important things? This is a question about memory for matters of day to day importance. It does not refer to memory for irrelevant content or to memory for detailed information from the past. Respondents should consider how well they remember to do things that are important to them or to their family. If note taking, electronic reminder systems or verbal cuing from personal assistants is usually in place as a memory aid, performance should be rated with this help taken into consideration. D1.3. How much difficulty did you have in analysing and finding solutions to problems in day to day life? This item refers to a complex activity involving many mental functions. If respondents are unsure, ask them to think about a problem they encountered in the past 30 days. Once a problem is identified, respondents should be asked to consider how well they: - 217 -

WHO/ESCAP Training Manual on Disability Statistics

• • • • • • •

identified that a problem existed broke it down into manageable parts developed a list of possible solutions determined pros and cons of each solution determined the best solution given all considerations executed and evaluated the chosen solution selected an alternate solution if the first choice was not successful

D1.4. How much difficulty did you have in learning a new task, for example, learning how to get to a new place? Learning a new route is offered as an example in this question however, respondents should not limit themselves to only this situation. If further clarification is requested or if the interviewer feels the respondent has difficulty to answer, the interviewer may ask the respondent to think about a situation in the past month where learning something new was required, such as learning a task at work (such as a new procedure or assignment), or a new activity (cooking, learning a language, a new sport). When making the rating, respondents should consider how easily new information was acquired, how much assistance or repetition they needed in order to learn and how well what was learned was retained. D1.5. How much difficulty did you have in generally understanding what people say? The respondent should consider his/her usual mode of communication (i.e. spoken language, sign language, with an assistive device such as a hearing aid) and overall, rate the degree of difficulty in understanding the messages of others. All situations encountered in the past 30 days by the individual should be considered, such as: • when others speak quickly • with background noise • with distractions Difficulties due to the fact that the language spoken is not one's mother tongue should be excluded when rating this question.

D1.6. How much difficulty did you have in starting and maintaining a conversation? Both starting and maintaining conversation should be rated. If a respondent states that they have more trouble with starting than maintaining a conversation (or vice versa), they should average the amount of difficulty experienced with both activities to determine the final - 218 -

WHO/ESCAP Training Manual on Disability Statistics

difficulty rating. Conversation includes use of whatever is the usual mode of communication (spoken, written, sign language, gestural). If assistive devices are used by the respondent for communication, the difficulty rating provided should take into account conversation while using those devices, assuming they are usually present. The respondent should consider any and all other factors of relevance to them in starting and maintaining a conversation such as hearing loss, language problems as after a stroke, stuttering, anxiety or any other factor related to a health condition. DOMAIN 2: GETTING AROUND Activities discussed in Domain 2 include standing, moving around inside the home, getting out of the home and walking a long distance. D2.1. How much difficulty did you have in standing for long periods such as 30 minutes? This question refers to situations where the person has to remain in a standing position for long time without leaning on something. For example, standing in a queue or waiting for a train. D2.2. How much difficulty did you have in standing up from sitting down? This question refers to standing up from sitting in a chair, on a bench or toilet. It does not refer to standing up from sitting on the floor. D2.3. How much difficulty did you have in moving around inside your home? This item refers to moving from room to room, and within rooms, using assistive devices or personal help that is usually in place. If the respondent lives in a house with multiple floors, this question also includes getting from one floor to another as needed. This will give a good idea of a person’s mobility. A person who has problems moving inside the house is likely to have problems outside as well. D2.4. How much difficulty did you have in getting out of your home? This question seeks information about: • mobility aspects of getting out of the home • emotional or mental aspects of leaving the home (depression, anxiety, etc.) For this question, “home” means the respondent’s current dwelling, which might be a house, apartment, or nursing home. D2.5. How much difficulty did you have in walking a long distance such as a kilometre [or equivalent]? Distances should be converted into imperial measure where necessary. - 219 -

WHO/ESCAP Training Manual on Disability Statistics

DOMAIN 3: SELF CARE Domain 3 asks about bathing, dressing, eating and staying alone. D3.1. How much difficulty did you have in washing your whole body? This question refers to washing one’s entire body in the usual manner for the culture. If respondents report that they have not washed their bodies in the past 30 days, interviewers should ask whether this is due to a health condition. If respondents report that it is due to a health condition, then the item should be coded “5” for extreme/cannot do. On the other hand, if respondents report that it is not due to a health condition, then the item should be coded “NA” for not applicable. D3.2. How much difficulty did you have in getting dressed? This question is intended to include all aspects of dressing the upper and lower body. Activities such as gathering clothing from storage areas (i.e. closet, dressers) and securing buttons, tying knots, etc., also should be considered when making the rating. D3.3. How much difficulty did you have in eating? This item refers to both feeding oneself (i.e. cutting food, getting food or drink from plate or glass to mouth) and swallowing (both food and drink). It also includes mental/emotional factors contributing to difficulty eating such as anorexia, bulimia, or depression. This question does not refer to meal preparation. If the respondent uses non-oral feeding (e.g.. tube feedings), this question refers to any difficulties experienced in self administering the non-oral feeding (e.g. feeding pump set up and cleaning). D3.4. How much difficulty did you have in staying by yourself for a few days? The intent of this question is to determine any difficulty staying alone for an extended period and remaining safe. If the individual did not experience this situation in the past 30 days, “not applicable” is the correct rating. It is important for interviewers to probe responses of “none” for this question. This will help differentiate situations that were experienced without difficulty (in which case “1” is correct) from situations not experienced at all (in which case “NA” is correct). DOMAIN 4: GETTING ALONG WITH PEOPLE Domain 4 assesses “getting along with people” and difficulties that might be encountered with these activities due to a health condition. In this context, - 220 -

WHO/ESCAP Training Manual on Disability Statistics

“other people” may be those with whom one is intimate or one knows well (e.g. one’s spouse or partner, one’s family members or close friends), or those whom one does not know at all (e.g. strangers). D4.1. How much difficulty did you have in dealing with people you do not know? This item refers to interactions with strangers in any situation, such as: • shop keepers • service personnel • asking someone for directions When making the rating, respondents should consider both approaching such individuals, and interacting successfully with them to obtain their desired outcome. D4.2. How much difficulty did you have in maintaining a friendship? This includes: • staying in touch • interacting with friends in customary ways • initiating activities with friends • participating in activities when invited If respondents report that they have not engaged in friendshipmaintenance activities in the past 30 days interviewers should ask whether this is due to a health condition. If respondents report that it is due to a health condition, then the item should be coded “5” for extreme/cannot do. On the other hand, if respondents report that it is not due to a health condition, then the item should be coded “NA” for not applicable. D4.3. How much difficulty did you have in getting along with people who are close to you? The respondent should consider those relationships that he or she defines as close. These may or may not be family relationships. D4.4. How much difficulty did you have in making new friends? This includes: • seeking opportunities to meet new people • following up on invitations to get together • social and communication actions to make contact and to develop a friendship On occasion, participants will report that they have not engaged in friendship-making activities in the past 30 days. In this case, interviewers should ask whether this is due to a health condition (as defined by the WHODAS II). If respondents report that it is due to a health condition, then the item should be coded “5” for extreme/cannot - 221 -

WHO/ESCAP Training Manual on Disability Statistics

do. On the other hand, if respondents report that it is not due to a health condition, then the item should be coded “NA” for not applicable. D4.5. How much difficulty did you have in sexual activities? The respondent should consider what he/she considers to be sexual activity when answering this question. However, if clarification is requested, this question refers to: • • • • •

sexual intercourse hugging kissing fondling other intimate or sexual acts

If respondent says “none”, the interviewer should probe this response to determine whether there was no difficulty when engaging in sexual activities (“none” should be coded) or no difficulty because the respondent never engaged in such activities (“NA” should be coded). DOMAIN 5: LIFE ACTIVITIES This domain includes questions about difficulty in day to day activities. These activities are those that people do on most days and include household, work and school activities. D5.1. How much difficulty did you have in taking care of your household responsibilities? This is a global question intended to elicit the respondent’s appraisal of difficulty encountered in maintaining the household and in caring for family members or others with whom one is close. Activities include: • physical, • emotional, • financial, • psychological needs of the household or family. In some cultures, males may indicate that they do not have household responsibilities. In this situation, it should be clarified that Household responsibilities include: • managing finances • car and home repairs • caring for the outside area of the home • picking up children from school • helping with homework • disciplining children and • other examples that interviewers believe elucidate the household responsibilities of males in the culture. - 222 -

WHO/ESCAP Training Manual on Disability Statistics

The term “household” is very broad. In the case of respondents who do not have a stable dwelling place, there are still activities surrounding the upkeep and maintenance of their belongings. This question refers to those activities. D5.2. How much difficulty did you have in doing your most important household tasks well? The respondent provides ratings based on his/her own appraisal of the quality in which household tasks and responsibilities are carried out.

D5.3. How much difficulty did you have in getting all the household work done that you needed to do? The respondent provides ratings based on his/her own appraisal whether all needed household work gets done (quantity of work). If necessary, interviewers may remind respondents that they are to report difficulties due to the health condition, not those that may be experienced for other reasons such as not having enough time (unless this reason is somehow linked back to a health condition). D5.4. How much difficulty did you have in getting your household work done as quickly as needed? This question refers to the timely meeting of expectations and needs of those with whom one lives (or is close) with respect to household tasks and responsibilities. D5.5. How much difficulty did you have in your day to day work/school? This is a global question intended to elicit the respondent’s appraisal of difficulties encountered in day to day work or school activities. This might include but is not limited to attending on time, responding to supervision, supervising others, planning and organizing and meeting expectations in the workplace. D5.6. How much difficulty did you have in doing your most important work/school tasks well? Doing work or school tasks “well” refers to completing them as expected by one’s supervisor or teacher, by one’s own standards or as specified in job or school performance criteria. D5.7. How much difficulty did you have in getting all the work done that you need to do? This question refers to meeting work expectations in terms of amount of work (quantity). D5.8. How much difficulty did you have in getting your work done as quickly as needed? - 223 -

WHO/ESCAP Training Manual on Disability Statistics

This question refers to meeting time deadlines related to the work tasks. DOMAIN 6: PARTICIPATION IN SOCIETY Respondents are asked to consider how other people and the world in which they live restrict them in social participation, such as laws or other features. They must understand that they are not to report their own difficulties and activity limitations but rather problems encountered because of the society in which they live. In addition, questions are asked regarding the impact of the health condition. Respondents should be reminded that the focus is on the past 30 days. D6.1. How much of a problem did you have in joining in community activities (for example, festivities, religious or other activities) in the same way as anyone else can? Examples of community activities include attending town meetings, fairs, leisure or sporting activities in the town, neighbourhood or community. The relevant issue being asked in this question is whether the individual is facilitated to participate in these activities or whether there are inhibitors to doing so. If respondents are confused by the phrase “in the same way as anyone else can”, they should be asked to use their best judgment to assess the extent to which average people in their community are able to join community activities, and then to compare their degree of difficulty in joining community activities in relation to this assessment. If a person does not usually join in community activities “NA” will be recorded next to the question. This question will allow to see if there is an element of exclusion because of the nature of a health condition. D6.2. How much of a problem did you have because of barriers or hindrances in the world around you? The intent of this question is to determine how much has stood in the way of the respondent being able to realize aspirations and plans as other people can. The concept here is external interference faced by the individual as created by the world or other people. Barriers could be physical such as the lack of ramps to get into church, social such as laws that discriminate against persons with disabilities and/or the negative attitudes of people that create the barrier.

D6.3. How much of a problem did you have living with dignity because of the attitudes and actions of others? The respondent should consider problems he has had in living with stature or pride in who he is, what he is doing, and/or how he lives his life. - 224 -

WHO/ESCAP Training Manual on Disability Statistics

D6.4. How much time did you spend on your health condition, or its consequences? This question intends to capture an overall rating or snapshot of the portion of the past 30 days spent by the respondent in dealing with any aspect of the health condition. This may include time spent for visits to a treatment centre, time spent managing financial matters related to the health condition, such as payment of bills, reimbursement of insurance or benefits, time spent in obtaining information about the health condition or in educating others about it, etc. D6.5. How much have you been emotionally affected by your health condition? This question refers to the degree to which the respondent has felt an emotional impact due to the health condition. Emotions may include anger, sorrow, regret, thankfulness, appreciation, or any other positive or negative emotions. D6.6. How much has your health been a drain on the financial resources of you or your family? Family is broadly defined to include relatives as well as those to whom the respondent is not related but considers to be like family, including those who may be sharing in the financial aspects of the health condition. The focus of this question is on the depletion of personal savings or current income to meet the needs created by a health condition. If a respondent has experienced a significant financial drain but the family has not, or vice versa, she should respond to the question based on the drain experienced by either party. D6.7. How much of a problem did your family have because of your health problems? The focus here is on problems created by the interaction of this health condition with the world in which the person lives. The question seeks information on problems that are borne by the family, which might include financial, emotional, physical problems, etc. Note the definition of family as stated in D6.6. D6.8. How much of a problem did you have in doing things by yourself for relaxation or pleasure? The respondent should consider leisure interests which are currently pursued and those that he/she would like to pursue but cannot due to the health condition and restrictions imposed by the society. Examples might include a respondent who has the desire to read novels but is restricted from doing so because the local library does not carry large print books for use by individuals with low vision or a respondent who enjoys watching movies on video but cannot do so because very few are produced with subtitles for the deaf. An overall rating of problems encountered should be provided. - 225 -

WHO/ESCAP Training Manual on Disability Statistics

QUESTION SET 2 PART 4: QUESTIONS ON NEED FOR ASSISTANCE E1 – E4 These questions ask whether the respondent is in need of a personal assistance in order to carry out certain activities, such as self care, movement and communication. E1.

Do you ever need someone to help with, or be with you for, self care activities? For example: doing everyday activities such as eating, showering, dressing or toileting. Examples of personal assistance for self care may include family members or professional caretakers who assist the person in washing or dressing.

E2.

Do you ever need someone to help with, or be with you for, body movement activities? For example: getting out of bed, moving around at home or at places away from home. This question refers to personal assistance for movement activities such as a family member who helps the person moving inside or outside the home or a passenger who helps the person getting in and out of the bus.

E3.

Do you ever need someone to help with, or be with you for, communication activities? For example: understanding, or being understood by, others. Personal help for communication activities include family members, friends or professional helper (e.g. sign language interpreter) who assist the person in communicating.

E4.

What are the reasons for the need for assistance or supervision shown in questions E1, E2, and E3? This question invites the respondent to provide the underlying reasons for the need for assistance. The response categories of 'Long-term health condition', 'disability' and 'old age (old or young age response combined with respondent age data)' are considered as indicative for having a need for core activity assistance. If only other reasons are given (e.g. difficulty with English language), the respondent is not viewed as having a need for core activity assistance.

- 226 -

WHO/ESCAP Training Manual on Disability Statistics

Appendix 4: Biwako Millennium Framework for Action Targets 1.

Self-help organizations of persons with disabilities and related family and parent associations

Target 1. Governments, international funding agencies, and non-governmental organizations (NGOs) should, by 2004, establish policies with the requisite resource allocations to support the development and formation of self-help organizations of persons with disabilities in all areas, and with a specific focus on slum and rural dwellers. Governments should take steps to ensure the formation of parents associations at local levels by the year 2005 and federate them at the national level by year 2010. Target 2. Governments and civil society organizations should, by 2005, fully include organizations of persons with disabilities in their decision-making processes involving planning and programme implementation which directly and indirectly affect their lives. 2.

Women with disabilities

Target 3.

Governments should, by 2005, ensure anti-discrimination measures, where appropriate, which safeguard the rights of women with disabilities. Target 4. National self-help organizations of persons with disabilities should, by 2005, adopt policies to promote the full participation and equal representation of women with disabilities in their activities, including in management, organizational training and advocacy programmes. Target 5. Women with disabilities should, by 2005, be included in the membership of national mainstream women’s associations. 3.

Early detection, early intervention and education

Target 6. Children and youth with disabilities will be an integral part of the population targeted by the millennium development goal of ensuring that by 2015 all boys and girls will complete a full course of primary schooling. Target 7. At least 75 per cent of children and youth with disabilities of school age will, by 2010, be able to complete a full course of primary schooling. Target 8. By 2012, all infants and young children (birth to four years old) will have access to and receive community-based early intervention services, which ensure survival, with support and training for their families. Target 9. Governments should ensure detection of disabilities at as early an age as possible. 4.

Training and employment, including self-employment

Target 10. At least 30 per cent of the signatories (member States) will ratify the International Labour Organization Vocational Rehabilitation and Employment (Disabled Persons)

- 227 -

WHO/ESCAP Training Manual on Disability Statistics

Target 11. By 2012, at least 30 per cent of all vocational training programmes in signatory countries will be inclusive of persons with disabilities and provide appropriate support and job placement or business development services for them. Target 12. By 2010, reliable data that measure the employment and selfemployment rates of persons with disabilities will exist in all countries. 5.

Access to built environments and public transport

Target 13.

Governments should adopt and enforce accessibility standards for planning of public facilities, infrastructure and transport, including those in rural/agricultural contexts. Target 14. All new and renovated public transport systems, including road, water, light and heavy mass railway, and air transport systems, should be made fully accessible by persons with disabilities and older persons; existing land, water and air public transport systems (vehicles, stops and terminals) should be made accessible and usable as soon as practicable. Target 15. All international and regional funding agencies for infrastructure development should include universal and inclusive design concepts in their loan/grant award criteria. 6.

Access to information and communications, including information, communications and assistive technologies

Target 16. By 2005, persons with disabilities should have at least the same rate of access to the Internet and related services as the rest of citizens in a country of the region. Target 17. International organizations (e.g., International Telecommunication Union, International Organization for Standardization, World Trade Organization, World Wide Web Consortium, Motion Picture Engineering Group) responsible for international ICT standards should, by 2004, incorporate accessibility standards for persons with disabilities in their international ICT standards. Target 18. Governments should adopt, by 2005, ICT accessibility guidelines for persons with disabilities in their national ICT policies and specifically include persons with disabilities as their target beneficiary group with appropriate measures. Target 19. Governments should develop and coordinate a standardized sign language, finger Braille, tactile sign language, in each country and to disseminate and teach the results through all means, i.e. publications, CD-ROMs, etc. Target 20. Governments should establish a system in each country to train and dispatch sign language interpreters, Braille transcribers, finger Braille interpreters, and human readers, and to encourage their employment. Target 21. Governments should halve, between 1990 and 2015, the proportion of persons with disabilities whose income/consumption is less than one dollar a day.

- 228 -

WHO/ESCAP Training Manual on Disability Statistics

Appendix 5: Biwako Plus Five Further Efforts Towards an Inclusive, Barrier-Free and Rights-based Society for Persons With Disability S in Asia and the Pacific Strategy C. Improving the availability and quality of data and other information on disabilities for policy formulation and implementation Strategy 13 38. The importance of collecting data on disabilities should be stressed and advocated not only within the United Nations system but also among decision makers at the national level, including national statistical offices, as well as academic institutions, self-help organizations and other civil society organizations. Strategy 14 39. Governments are encouraged to develop policies or laws to mandate the collection of data on disabilities, as well as the allocation of the requisite resources. Such policies and laws should, among other things, respect the privacy of persons with disabilities. Strategy 15 40. As far as possible, data should be classified by the socio-economic status of persons with disabilities, including the type of impairment, sex, age, education, employment and income. Strategy 16 41. Governments should build national capacity so that data on disability can be regularly collected through population censuses and surveys and disseminated. Strategy 17 42. Governments are encouraged to develop innovative methods of data collection in order to capture the needs of persons with disabilities, in particular those who are illiterate or who live in remote areas. Strategy 18 43. Governments are encouraged to undertake regular assessments of the impact of policies and programmes which are intended to improve the situation of persons with disabilities and to ensure that they fully enjoy their human rights and fundamental freedoms. Strategy 19 44. Governments, in cooperation with ESCAP, should, as appropriate, take measures to ascertain the concerns of disabled persons and to develop future action plans through questionnaires and surveys, depending on the availability of resources. Strategy 20 45. ESCAP, other United Nations organizations, agencies and intergovernmental organizations, should, upon request, assist Governments in setting statistical standards and in formulating policy regarding persons with disabilities. - 229 -

WHO/ESCAP Training Manual on Disability Statistics

United Nations publication The publication herein is copyright © 2008 United Nations. Printed in Thailand ISBN: 978-92-1-120548-0 ST/ESCAP/2499 http://www.unescap.org/stat/disability/index.asp

- 230 -