Vous êtes sur la page 1sur 52

PART 2

FOCUSING YOUR RESEARCH EFFORT (CHAPTER 5)

Planning the Research Project Successful research is a planned research Fail because plunged into activity with only a partially thought-out plan & inconclusive design Research requires a conceptualisation of overall organization and detailed plan Should be no less totally visualized and precisely detailed

Planning the Research Project A general strategy = research design (RD) RD provides
Overall structure for the procedures The data the researcher collects The data analyses the researcher conducts

Must plan the overall design carefully to be successful Wasted effort if half-prepared (vague ideas) Must identify RESOURCES, PROCEDURES, and DATA always with the goal from the very beginning

All Research has a Basic Format No matter what the academic discipline Not parochial (narrow, limited) and seldom fits into neatly packaged academic disciplines Software engineer may be compelled to stray far from the field of SE in pursuing a research problem in user interface design The ramifications of problem may lead into fields far from that of the original problem Err in thinking too narrowly, in restricting problems for research to one academic area

The Basic Format of the Research Process 1. A question is posed. In the mind of the researcher, a question arises that has no known resolution 2. Its a matter of words. The researcher converts the question to a clearly stated research problem. 3. Its worth a guess! The researcher poses a temporary hypothesis or series of hypotheses. 4. The search is on! The researcher searches the literature for ideas that shed light on the problem and for strategies that may help to address it.

The Basic Format of the Research Process 5. Data! Hard Data! And nothing BUT the data. The researcher collects data that potentially relate to the problem. 6. How do the data fit together? The researcher arranges the data into a logical organizational structure. 7. The data speak! The researcher analyzes and interprets the data to determine their meaning 8. Its either or Either the data seemingly resolve the research problem or they do not. Either they support the hypotheses or they do not.

All Research has a Basic Format


Might do better to think of problems as arising out of broad generic areas within whose boundaries all research falls:
People research problems relating to individuals, groups, populations, nationalities, families, ... Things biological life, matter, Records letters, legal documents, lists, journals, Thoughts and ideas opinions, reactions, concepts, theories, Dynamics and energy human energy and activity, metabolism, gravity, hydrologic cycles,

Above suggestions are not all-inclusive, merely suggest the broad ramification of research possibilities Substratum (an underlying layer or foundation) of research principles is all-inclusive

All Research has a Basic Format Physicist exploring subatomic particles and sociologist exploring social behaviour are both employing the same research principles, although tools may differ, interpretational techniques may have little in common approach identical Research approaches all problems through certain methodological channels appropriate to the nature and type of data Must differentiate research design VS research methodology

Research Planning Versus Research Methodology


Many people are confused between the two Example, statement research in physics is different from research in philosophy or history Confusion between research as a process and the methodology employed by separate academic disciplines in collecting and processing data Genuine research follows the scientific method and exhibits the EIGHT characteristics

Research Planning Versus Research Methodology


Data vary widely, therefore cannot employ the same methodology, e.g. blood cells vs historical facts In planning, the researcher must choose a viable research problem and consider the nature of data and the feasible means of collecting and interpreting those data Comparing the brain wave patterns of gifted and nongifted children may be an engaging project for research, BUT consider the following issues:

Research Planning Versus Research Methodology


Who are willing to participate? Do we have the equipment? If so, do we have the technical skills to use it? Are we able to interpret the tracings? Do we know how to draw conclusion? It must be a practical research something that we must be able to handle

General Criteria for a Research Project Universality Could be carried out by any competent person other than the researcher, same results without prejudice to project and validity Replication Should be repeatable by other researchers under the same circumstances, within identical parameters and achieve comparable results Control All researches are conducted within an area sealed off by given parametric limitations

Criteria for Research Project Measurement Data should be susceptible to measurement. Easily accomplished in the physical sciences. In humanities and social sciences research, it is more difficult to quantify, measure, or evaluate Many research efforts are wasted by going-off HALFPREPARED Must bring all together, early inventory of resources, your problems, and the source of data, in order to save time, money and effort

Data: Their Relationship to Design and Methodology


Without facts (data), inductive reasoning vanishes and scientific method collapses The lifeblood of research Data is a Latin verb dare, meaning to give give information, impression or other factual data to an observer Facts from Latin word facere, meaning to make what the situation makes or manifests to the observer

What is Data? Data are manifestation of the truth rather than the truth itself no one has ever look upon truth itself pure, undisguised, naked Truth Data are merely representative, intermediate, elusive surrogate of Truth reflects Truth as a mirror which reflects sunlight Researchers are like those who live in dungeon seeing a beam of sunlight passes across the floor and get the idea of what the sun must be like

What is Data? Factual dungeon never be able to see the original source of the data see what other people do the behaviours they exhibit, the things they create, and the effects of their actions on others. But the actual people inside those individuals we shall never know! Research seeks, through data, what is true absolutely the complete meaning of the data Experienced researchers are constantly aware that the ultimate goal of research (Truth) is forever just beyond what is represented by the data, and, hence, just beyond human grasp

What is Data? The relationship between data and the ultimate truth is shown in the diagram - various states of data Realm of ultimate truth, the most inaccessible can be approached by means of passing through two intermediate areas labeled The Realm of the Data

? ? ? The Realm of the Inquisitive ? ? ? ? ? ? ? ? Mind of the Researcher ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? o o o o o o o o o o The Region of the o o o o o o o o o o Secondary Data o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o
The Region of the Primary Data

The Barrier of the Human Senses Skills of Reading and Writing, Language Capability, and the Channels of Communication
The Dividing Line Between Primary and Secondary Data

The Impenetrable Barrier Beyond Which Lies the Absolute Truth

THE REALM OF ABSOLUTE TRUTH

Data are Transient and Ever Changing Data are not only elusive but also transient exist in a split second Example, an opinion survey on a population tomorrows result will not be the same as now, people move out, move in, died, . Pretest and posttest conducted by a teacher in a class may not be the same class at all in six weeks, they have been growing, learning, maturing, changing

Data are Transient and Ever Changing Example, asking people about their interests can we be sure what they tell us is true? They may answer honestly that represents the truth as they perceived it at that instant May change because of factors: age, maturity, newscast, or items in newspaper Researcher should recognize that even the most reliable, refined, carefully controlled data may have elusive (hard to find, describe or remember) quality very volatile, evaporate

Primary Data Lie closest to the source of The Absolute Truth underlying the phenomenon Reflect truth more faithfully than any other approach most valid, most illuminating, the most truthmanifesting Impenetrable barrier exists between the Realm of Absolute Truth and the region of primary data A beam of sunlight does not compare with seeing the sun itself

Car Accident I witnessed the full occurrence of a car veer off the highway and careen into the ditch The driver told me that he had no awareness of the possibility of an accident Neither of us will ever be able to determine the absolute truth underlying the accident What are the factors? Momentary seizure? Mechanism imperfections? Answers lie beyond an impenetrable barrier, may never be known

Secondary Data Beyond the region of primary data lies the region of secondary data In the dungeon not a direct beam of sunlight but simmering light upon the wall, reflected by mirror, distorted by imperfections In the case of the car accident, I wrote an account stating precisely what I observed and confirmed by the driver readers of the next days newspaper get the similar reflection of the sunlight

Barriers of The Truth The data/facts are of necessity distorted albeit by the channels of communication, my writing skills, reading skills, and the inability of language to produce every detail Many more barriers than in the diagram
The acuity of human senses The sensitivity of instruments The failure of language to communicate thought exactly The inability of two persons to witness the same event and report it precisely as duplicate accounts

Unwarranted Conclusion Even in exuberant moment of discovery, no one has ever glimpsed the Absolute Truth, nor can we come to a knowledge of the data that reflect that Truth except through the gross and shadowy channels of our dull and imperfect senses Be cautious and respect for words such as perhaps, it seems, one might conclude and it would appear to be

Criteria for Admissibility of Data Not all data are acceptable can be defective affect the validity of the conclusion Stem from imperfections and irregularities of nature Research should be replicable under precisely the same conditions To regulate the precision of conditions, criteria must be adopted, limits established, standards set up that all data must meet

Criteria for Admissibility of Data Non-conformance excluded These controls are easy in sciences Standardize data, admit only compliance can control the research effort and conclude with greater certainty The criteria must be set forth clearly in proposal as well as report to ensure integrity of research So the consumer can come to an intelligent understanding of what is being studied

Identifying Appropriate Measurement Instruments Sometimes we will be able to use one or more existing measurement instruments OR have to develop our own instruments Measurement instruments provide a basis on which the entire research effort rests Describe any instrument used in explicit, concrete terms Provide evidence that the instruments you use have a reasonable degree of validity and reliability for your purposes Validity = the extent to which the instrument measures what it is actually intended to measure Reliability = the extent to which it yields consistent results when the characteristic being measured hasnt changed

Determining the Validity of Measurement Instruments Face validity The extent to which, on the surface, an instrument looks like it is measuring a particular characteristic Useful for ensuring the cooperation of people who are participating in a research study Since it relies entirely on subjective judgment, it is not, in and of itself, terribly convincing evidence that an instrument is truly measuring what the researcher wants to measure

Determining the Validity of Measurement Instruments Content validity The extent to which a measurement instrument is a representative sample of the content area (domain) being measured Used to assess peoples achievement in some area for instance, the knowledge they have learned during classroom instruction or the job skills they have acquired in a rehabilitation program High content validity means that its items or questions reflect the various parts of the content domain in appropriate proportions and requires the particular behaviours and skills that are central to that domain

Determining the Validity of Measurement Instruments Criterion validity The extent to which the results of an assessment instrument correlate with another, presumably related measure (the criterion) Example, a personality test designed to assess a persons shyness or outgoingness (introvert or extrovert) has criterion validity if its scores correlate with other tests of introversion versus extroversion An instrument designed to measure a salespersons effectiveness on the job should correlate with the number of sales the individual actually makes during the course of a business day

Determining the Validity of Measurement Instruments


Construct validity The extent to which an instrument measures a characteristic that cannot be directly observed but must instead be inferred from patterns in peoples behaviour (such a characteristic is a construct) Motivation, creativity, racial bias, bedside manner all of these are constructs, in that none of them can be directly observed and measured Sometimes there is universal agreement that a particular instrument provides a valid instrument for measuring a particular characteristic We could all agree that a ruler measures length, a thermometer measures temperature, and a barometer measures air pressure But whenever we do not have such universal agreement, we must provide evidence that an instrument we are using has validity for our purpose

Determining the Validity of Measurement Instruments


To demonstrate that the measurement instruments used by researchers have validity for their purposes, the following examples are used: - A multitrait-multimethod approach Two or more different characteristics are each measured using two or more different approaches. The different measures of the same characteristic should be highly correlated. The same ways of measuring different characteristics should not be highly correlated - A table of specifications To construct a measurement instrument that provides a representative sample of a particular content domain in other words, to establish content validity the researcher often constructs a two-dimensional grid (table of specifications) listing the specific topics and behaviours that reflect achievement in the domain. In each cell of the grid, the researcher indicates the relative importance of each topic-behaviour combination. He or she then develops a series of tasks or test items that reflects the various topics and behaviours in appropriate proportions - Judgment by a panel of experts Several experts in a particular area are asked to scrutinize an instrument to ascertain its validity for measuring the characteristic in question Although none of the approaches just described guarantees the validity of a measurement instrument, each one increases the likelihood of such validity.

Determining the Reliability of Measurement Instruments Interrater reliability the extent to which two or more individuals evaluating the same product or performance give identical judgments Internal consistency reliability the extent to which all the items within a single instrument yield similar results Equivalent forms reliability the extent to which two different versions of the same instrument (e.g., Form A and Form B of a scholastic aptitude test) yield similar results Test-retest reliability the extent to which the same instrument yields the same result on two different occasions

Reliability and Validity A researcher can enhance the reliability of a measurement instrument in several ways. 1) The instrument should always be administered in a consistent fashion; in other words, there should be standardisation in use of the instrument from one situation or person to the next 2) To the extent that subjective judgments are required, specific criteria should be established that dictate the kinds of judgments the researcher makes 3) Any research assistants who are using the instrument should be well trained so that they obtain similar results

Reliability and Validity We can measure something accurately only when we can also measure it consistently In other words, in order to have validity, we must also have reliability The more valid and reliable our measurement instruments are, the more likely we are to draw appropriate conclusions from the data we collect and, thus, to solve our research problem in a credible fashion

Linking Data and Research Methodology Data are like ore contain pieces of the truth but are in a rather unrefined state To extract meaning from the data, we employ what is commonly called research methodology (RM) Data and methodology are inextricably interdependent The RM to be used for a particular research problem must always take into account the nature of the data that will be collected in the resolution of the problem RM is merely an operational framework within which the facts are placed so that their meaning may be seen more clearly

Data and RM Different questions yield different types of information Different research problems lead to different research designs and methods, which in turn result in the collection of different types of data and different interpretations of those data Many kinds of data may be suitable only for a particular methodology

Data and RM To some extent, the data dictate the research method Historical data from written records of past event cannot extract meaning by laboratory experiment. An experiment is simply not suited to the nature of the data Not only a true experiment constitutes a research

Nature of Data Two types, writings and observations


Written records and accounts of past happenings Observation for whose transmission description is the best vehicle at the scene of occurrence Observation that are quantified and exist in the form of numerical concepts in the language of mathematics Observation of certain differences and likeness that arise from comparison or contrast of one set of observations with another set of similar observations

Research Methodology (RM) Four kinds of data demand four discrete and different research methodologies The data dictate the RM Unless fits a given methodology, it fails to be a research

Research Methodology (RM) 1-1 correspondence between data and RM


Historical method for documentary data or literary Descriptive survey method (normative survey method) data from observational situation physically observed or through questionnaire or poll techniques
Analytical survey method quantitative data that need statistics to extract their meaning Experimental method appropriate for data derived from experimental control or pretest-posttest design, two separate groups or one group from which data are derived at two separate intervals

Quantitative vs. Qualitative Research Quantitative research is used to answer questions about relationships among measured variables with the purpose of explaining, predicting, and controlling phenomena. This approach is sometimes called the traditional, experimental, or positivist approach Qualitative research is used to answer questions about the complex nature of phenomena, often with the purpose of describing and understanding the phenomena from the participants point of view. This approach is also referred to as the interpretative, constructivist, or postpositivist approach

Quantitative vs. Qualitative Research Both approaches involve similar processes (e.g., formation of one or more hypotheses, review of the related literature, collection and analysis of data). Yet these processes are often combined and carried out in different ways, leading to distinctly different research methods Quantitative researchers usually start with a specific hypothesis to be tested, isolate the variables they want to study, control for extraneous variables, use a standardized procedure to collect some form of numerical data, and use statistical procedures to analyse and draw conclusions from the data

Quantitative vs. Qualitative Research Qualitative researchers often start with general research questions rather than specific hypotheses, collect an extensive amount of verbal data from a small number of participants, organise those data into some form that gives them coherence, and use verbal descriptions to portray the situation they have studied

Quantitative vs. Qualitative Research A quantitative study usually ends with confirmation or disconfirmation of the hypotheses that were tested A qualitative study is more likely to end with tentative answers or hypotheses about what was observed. These tentative hypotheses may form the basis of future studies (perhaps quantitative in nature) designed to test the proposed hypotheses In this way, qualitative and quantitative approaches represent complementary components of the research process appropriate for answering different kinds of questions As a result, we learn more about the world when we have both quantitative and qualitative methodologies

Quantitative vs. Qualitative Research Summary of Qualitative vs. Quantitative Approaches Please refer to Table 5.1 on page 96 of the textbook Deciding whether to use a quantitative or qualitative approach Please refer to Table 5.2 on page 106 of the textbook

Other RMs Action Research Case and Field Study Research Correlational Research Developmental Research Ex Post Facto or Causal-Comparative Quasi-Experimental Research

Other RMs Please refer to Table 5.3 on page 108 of the textbook

Question What is the purpose of the research?

Quantitative To explain and predict To confirm and validate To test theory Focused Known variables Established guidelines Predetermined methods Somewhat context-free Detached view

Qualitative To describe and explain To explore and interpret To build theory Holistic Unknown variables Flexible guidelines Emergent methods Context-bound Personal view

What is the nature of the research process?

Question What are the data like, and how are they collected?

Quantitative Numeric data Representative, large sample Standardized instruments Statistical analysis Stress on objectivity Deductive reasoning

Qualitative Textual and/or image-based data Informative, small sample Loosely structured or nonstandardised observations and interviews Search for themes and categories Acknowledgment that analysis is subjective and potentially biased Inductive reasoning Words Narratives, individual quotes Personal voice, literary style

How are data analyzed to determine their meaning?

How are the findings communicated?

Numbers Statistics, aggregated data Formal voice,scientific style

Research Design Must answer four fundamental questions with respect to data no reservation
What are the data needed? What data are mandatory? What are their nature? Documentary? Statistical? Interview? Questionnaire? Where are the data located? How will the data be secured? How will the data be interpreted?