Vous êtes sur la page 1sur 42

Lecture 7

Data collection

Introduction

Data Collection is an important aspect of any type of research study.

Inaccurate data collection can impact the results of a study and ultimately lead to invalid results.

Data collection methods for impact valuation vary along a continuum.

At the one end of this continuum are quantitative methods

and at the other end of the continuum are Qualitative

methods for data collection .

Data Collection Methods

  • 1. Observation

  • 2. Interview

  • 3. Survey

  • 4. Content analysis

  • 5. Case study

Data Collection Methods 1. Observation 2. Interview 3. Survey 4. Content analysis 5. Case study

1. Observation

The purpose of observation:

  • - to describe the setting that was observed,

  • - the people who participated in those activities, and

  • - the meanings of what was observed from the

perspective of the those observed.

Variations in observer involvement: participant or onlooker?

The extent of participation is a continuum that varies from complete immersion in the setting as full participant to complete separation from the setting as spectator

To understand a society, the anthropologist has traditionally immersed himself in it, learning, as far as possible, to think, see, feel and sometimes act as a member of its culture and the same time as a trained anthropologist from another culture.

Types of observations

o

Overt observations:

when the people in the setting are aware they are being studied.

o

Covert observations:

to capture what is really happening.

This is based on the basic notion that people may behave

quite differently when they know they are being observed

compared with how they would behave if they were not aware of being observed.

(Hawthorne Effect)

o Variations in duration of observations (the length of time devoted to data gathering )

i.e.

to develop a holistic view of an entire

culture takes a great deal of time; in the

anthropological tradition of field research, a participant observer would expect to spend six months at a minimum, and

often years, living in the culture being

observed.

o Variations in observational focus

The scope can be broad, encompassing virtually all aspects of the setting, or it can be narrow, involving a look at only some small part of what is happening.

o Variations in observational focus The scope can be broad, encompassing virtually all aspects of the

Strategies For Analyzing Observations

Chronology

Describe what was observed chronologically, over time, to tell the story from beginning to end

Key events

Present the data by critical incidents or major events, not necessarily in oder of occurrence

but in order of importance

 

Describe various places, sites, setting, or

Various settings

locations

People

If individuals or groups are the primary unit of analysis, then case studies of people or groups may be the focus for case studies

Processes

The data may be organized to describe important processes (e.g. recruitment, decision

making, communication, etc.)

 

The observation may be pulled together to

Issues

illuminate key issues, often the equivalent of

the primary evaluation question, such as how

did participants change?

Case analysis

Writing a case study for each person interviewed or each unit studied (e.g. each critical event, each group, etc.)

Cross-case

Grouping together answers from different people to common question or analyzing

analysis

different perspectives on central issues

Participant

Requires researcher to take an active role in

observation

the research setup, aimed at making significant change to the way things are done. Suitable for action-oriented research.

Example: To study the effectiveness of using e-book as alternative to regular textbook teaching in the classroom.

Define the problem.

Develop the hypothesis.

Decide what constitutes effective teaching (domain expertise).

Units of observation include number of times difficulty encountered in e- book operation; number of students who find difficulty in operating the device; time taken to access a particular chapter/section; number of times

positive/negative remarks/gestures made regarding the e-book; level of

interest in e-book content.

Define a precise time unit or total length of the study. Eg. 5 lessons of 1 hour.

  • a. Select/develop an Observational Form.

Develop a schedule/form in the form of a checklist.

Include scores/rating scale to indicate presence and

absence, frequency of occurrence etc. Try it out in a pilot test.

Modify if any weaknesses are found.

  • c. Train Observers

To ensure objectivity of recording, it is important to train the observers who are to use the observation form.

Discuss details of observation with observers, preferably

on a pre-recorded example using as videotape.

Conduct practice sessions to ensure high agreement on descriptive variables to be observed between observers (not less than 90%). Not less than 70% on inferential and

evaluative variables (also known as interrater reliability).

2. Interview

  • The purpose of interviewing is to find out what is in and on someone else’s

mind.

  • The quality of the information obtained during an interview is largely dependent on the interviewer.

  • Inner perspectives:

we interview people to find out from them those things we cannot directly observe. We cannot observe feelings, thoughts, and intentions. We cannot observe behaviors that took place at some previous point in time. We cannot observe situations that preclude the presence of an observer. We cannot observe how people have organized the world and the meanings they attach

to what goes on in the world. We have to ask people questions about those things. The purpose of interviewing then, is to allow us to enter into the

other person’s perspective.

(Patton, p. 278)

i. Face-to-face interviews

Enabling the researcher to establish rapport with potential participants and therefore gain their cooperation.

These interviews yield highest response rates in

survey research.

They also allow the researcher to clarify ambiguous

answers and an appropriate, seek follow-up

information.

ii. Telephone interviews

  • - are less time consuming and less expensive

  • - the researcher has ready access to anyone on the planet who has a telephone.

  • - disadvantages are that the response rate is not as high

as the face-to- face interview

Variations of Interview

Four basic approaches to collecting qualitative data through open-ended interviews:

  • - The informal conversational interview

  • - The general interview guide approach

  • - The standardized open-ended interview

  • - Closed, fixed response interview

Content of Interviews

Six kinds of questions that can be asked of people:

  • 1. Experience/behaviour questions

  • 2. Opinion/values questions

  • 3. Feeling questions

  • 4. Knowledge questions

  • 5. Sensory question

  • 6. Background/demographic questions

The Content of Interviews:

Questions To Ask

TYPES OF

EXAMPLES

QUESTION

Experience/

Aimed at eliciting description of experiences, behaviors, actions and activities that

behavior

would have been observable had the observer been present

questions

  • 1. If I had been in the program with you, what would I have seen you doing?

  • 2. If I had followed you through a typical day, what would I see you doing?

  • 3. What experiences would I observe you having?

Opinion/ values Questions

Aimed at understanding the cognitive and interpretive processes of people. 1.What do you believe……… 2.What do you think about… .. 3.What is your opinion of… ..

Feeling question

Aimed at understanding the emotional responses of people to their experiences and thoughts 1.How do you feel about… .. 2.To what extent do you feel….

The Content of Interviews:

Questions To Ask

TYPES OF QUESTION

EXAMPLES

Knowledge questions

To find out what factual information the respondent has.

1.What are some of the rules and regulations of the program? 2.What kind of services are provided in the program

Sensory questions

The purpose is to allow the interview to enter into the sensory

apparatus of the responden.

  • 1. Describe to me what I would see if I walked through the doors into the program?

Background/

Concern the identifying characteristics of the person being

demographic

interviewed

questions

  • 1. Age, education, accupation, residence/ mobility, etc.

3. Survey

  • 1. A survey study will be most appropriate under the following conditions:

    • a) To describe existing phenomenon by collecting detailed factual information;

    • b) To justify current conditions and practices or identify the problems;

    • c) To make comparisons and evaluation;

    • d) To determine what others are doing with similar problems and benefit from their experience in making future plans and decisions.

2. The approach to survey research consists of the following

steps:

  • a) Decide the technique and procedure for data collection;

  • b) Select the subjects;

  • c) Select/develop the instruments;

  • d) Identify procedures for data collection and data analysis.

2a. Decide the technique and procedures for data collection.

  • i. Technique for Data Collection.

Use questionnaires, interviews and/or observations.

Questionnaire is recommended for large number of

respondents.

Interviews would be preferred for small sample size

and when in-depth data are required.

Observations are ideal for real-life phenomenon.

2a (ii) Procedure for Data Collection.

For questionnaires, decide whether through mails or visits.

For

interviews

or

observations, consider

unstructured or semi-structured questions. Require research protocol and well trained interviewers.

2b.

Selecting the subjects.

i.

Define the population.

ii.

Sample should be representative of the population. This would require selection of

iii.

appropriate sampling technique. Types of sampling techniques include:

Convenience sampling nonprobability samples that are unrestricted, cheapest

and easiest to conduct but the least reliable. Purposive sampling nonprobability sample that conforms to certain criteria.

Simple random each population element has an equal chance of being selected into the sample. Sample drawn using random number table/generator.

Systematic sampling selects an element of the population at the beginning with a random start and following the sampling fraction selects every kth. element. Eg. firm’s size, age etc.

Stratified sampling for each subpopulation or stratum use the simple random sampling. Within each stratum is homogeneous, between

strata are heterogeneous. Results may be weighted and combined. Eg. sectors of industry.

Cluster sampling population is divided into internally heterogeneous subgroups. Within subgroup is heterogeneous, between subgroups are

homogeneous. These subgroups are randomly selected for further

study. Eg. the faculty system in universities.

2c.

Select/develop the instruments.

  • i. Look for available questionnaires or interview guides previously used to conduct similar studies.

ii.

Ensure instrument is valid and reliable. Valid means what it is intended to measure. Reliable means

producing consistent results/responses.

iii.

If instruments not available, they need to be developed. Instruments developed should be attractive, brief, easily answered, valid and reliable.

Four steps involved in development of instruments:

1)

Concept Development.

Conceive constructs of interests based on sound knowledge of domain area, extensive literature and/or seek opinions from domain experts.

2)

Concept Specification.

Break down original concept/constructs into several components. Employ statistical techniques such as factor analysis or cluster analysis to determine concept components.

3) Indicator Selection.

Once the constructs/dimensions have been set, the indicators by which to measure each concept must be developed. Indicators can be questions, statistical measures, or other scoring devices. Eg. single- scale or multiple-scale indexes. Single-scale may fall short of the variety of dimensions that could be included.

4) Formation of Indexes.

Combine the various dimensions or different measures into a single index. Possibly assign weights to the different dimensions. Eg. Software Sophistication Index, User Satisfaction Index.

2d.

Data Collection and Data Analysis.

i.

Data collection through mailed questionnaires expects low rate of return,

ii.

respondents skipping some questions and responding to questions very casually (informal). Data collection through interviews require training for interviewers and a set

of interview protocols. Large sample size will not be practical and

economically feasible.

iii.

Data analysis depends upon nature of research questions, hypothesis and scales of measurements used for measuring variables.

iv.

Scales include nominal, ordinal, interval and ratio

Questionnaires is the most popular instrument used in survey research

Questionnaires can be handed out or

sent by mail and later collected or

returned by stamped addressed envelope. This method can be

adopted for the entire population or

sampled sectors.

Questionnaires may be used to collect regular or infrequent routine data, and data for specialized studies.

• Questionnaires is the most popular instrument used in survey research • Questionnaires can be handed

Some of the data often obtained through questionnaires include demographic characteristics, opinions of stakeholders

etc.

A questionnaire requires respondents to fill out the form

designed to be as simple and clear as possible, with targeted

themselves, and so requires a high level of literacy. In order to maximize return rates, questionnaires should be

sections and questions. Questionnaires, like interviews, can contain either structured

questions with blanks to be filled in, multiple choice

questions, or they can contain open-ended questions where

the respondent is encouraged to reply at length and choose their own focus to some extent.

4. Content Analysis Document Reviews

Content analysis is a technique used to systematically make an objective and qualitative description of any content of communication.

Communication could be verbal or non-verbal: such as content of speeches, letters,

essays, technical reports, white papers, research articles, journals, films, music, textbooks, interviews, examination papers, questionnaires, course content, program codes, requirements specifications.

Purpose:

Analyze existing program records and other documents.

Examples include recruitment and attendance records, budget, staff records, and manual reports.

They are particularly useful for documenting implementation.

Type of Information Collected:

Youth, family and community demographics. Program characteristic.

Advantages:

Record are tailored to programs. Save on evaluation time and costs.

Approach to Content Analysis:

  • 1. Establish Purpose to be Achieved.

Eg. Readability of content, description and classification of content, relevance of content to a situation related to the problem, or classification of conceptual information.

  • 2. Develop Hypotheses or Research Questions.

The purpose of the study is analysed and expressed in terms of testable hypotheses or verifiable research questions.

  • 3. Sample the Content.

Three possible populations from which content can be sampled:

i.

Titles.

ii.

Issues or dates of titles.

iii.

Content within issues or titles.

All content specifically related to the research problem is studied.

Sampling needs to be done when content analysis has to be performed for a large body of documentary materials. Eg. Brancheau and Brown (1993) study of EUC Management Research Model. Sampling technique can be random sampling or purposive sampling.

4. Develop the Classification System for Analysis

Define categories of content that measure the

variables indicated in the research questions or hypotheses.

Example: Content analysis on effectiveness of e- book, categories include behavioral issues associated with e-book operation, user interface, ease of use, animation.

  • 5. Ensure High Degree of Interrater Reliability.

Interrater reliability refers to the rate of agreement between the scribers (ie. persons recording the information)

90% or more for descriptive variables.

70% or more for inferential or evaluative variables.

5. Case Study

Case study methods involve an in-depth, longitudinal examination of a single instance or event, an institution or a group.: a case o that investigates a phenomenon within its real-life context, o be based on any mix of quantitative and qualitative evidence

“A case study is an empirical inquiry that investigates a contemporary phenomenon within its real-life context, especially when the boundaries between phenomenon and context are not clearly evident” (Yin 1994, p.13).

According to Benbasat et al. (1987), case study research method is particularly well- suited to IS research, since the object of the IS discipline is the study of IS in organizations.

They provide a systematic way of looking at events, collecting data, analyzing information, and reporting the results.

As a result the researcher may gain a sharpened understanding of why the instance happened as it did, and what might become important to look at more extensively in future research

Types of case study:

o

Single Case Study

o

Multiple Case Studies

Characteristics of the Case Study Method:

  • 1. Case study is an intensive study of a social unit.

  • 2. Observation is the primary technique of gathering information.

  • 3. A case study examines a small number of units but the number of variables is large.

Approach to Case Study:

  • 1. Select Units of Analysis. Select cases which typically reveals the major dimensions of the problem. That is cases that represent extreme positions regarding the phenomena being studied.

  • 2. Identify Available Sources of Data. Sources of data could be interviews with the persons concerned with the phenomena being studied; results of psychological tests; analysis of documents; and observations.

  • 3. Select/develop Data Collection Procedures The primary technique used in data collection for a case study is the observational technique. This can also be supplemented by interviews and questionnaires.