UNIT-3 DATA COLLECTION 3. QUESTIONNAIRE SCHEDULE, (QUESTIONNAIRE “A questionnaire is a document used as a data capture instrument and does ‘wo things: + Ie lists all the questions 2 researcher wishes to address to each respondent and + Itprovides a space or some mechanism for recording the responses” + Asset of Questions designed to generate the statistical information ‘needed to accomplish the research objectives Purposes of the Questionnaire + Ensures standardization and comparability of the data across interviews - everyone is asked the same questions + Increases speed and accuracy of recording Facilitates data processing + Allows the researcher to collect the relevant information necessary to address the research problem Survey Research Processes 1 Define the research aims/objectives 2. Identify the population and sample 43. Identify Data collection methods 4. Design the data collection method 5. Runa pilot survey 6.Carry out the main survey 7. Analyse the data Data Collection Decide how to collet replies: 1) Self administered (online, postal ete) - ie completed by the respondent- Questionnaires can be delivered tothe respondent by post, & ‘mall attachments or publishing on a website for interactive completion, 2) Interviewer administered (face to face) ‘Questionnaire Survey -Steps 1. Define the preblem/ survey topic 2. Use previous studies &expertadvice 3. Define research questlons/ objectives 4. Define study population and sample size 5, Develop a preliminary questionnaire 6. Pretest the questionnaire 7. Make necessary alterations and corrections 8. Cover letter {introducing the study, deadline for return, guarantees for data anonymiy) 9, Stamped & se Faddressed envelop 410,Send to respendent11 Follow-up letter for non-respondents Questionnaire Types Questionraires can be: + Unstructured/semi-structured ~ qualitative + Structured - quantitative + Hstall questions put in logical sequence specify precise wording tobe used provide pre defined categories for recording the replies Main aim is that questions are standardised so individual responses can be counted up and compared Questionnaire Design Designingan effective series of questions is never easy: ‘+ respondents must beable tounderstand the questions and understand them in the same way as other respondents + respondents must be able to provide the answers ‘+ respondents must be willing to provide the information, Improper design can lead to? "Incomplete information| + Inaccurate data + Higher costs Criteria te consider in Questionnaire Design oes It provide the necessary information Does tconsider the respondent Does it meet editing, coding and data processing requirements The crucial part of good research design is making sure that the ‘questionnaire addresses the objectives ofthe research * Fulfiling these aims should drive the design of the questionnaire and help With the selection of questions that are relevant, clear, concise and efficient Avoid asking too many questions Questionnaire introduction ‘+ Its important to explain to the potential respondents why you believe they should answer the questions ~this can be in the introduction or an accompanying message (email) or letter (postal) + Introductory letters, reply paid envelopes and follow-up telephone calls also help to raise the response rate for self-administered ‘questionnaires, Self-Administered Questionnaires *+ Respondents are more likely to commit to answer a questionnaire when they see it as interesting of value, shor, clearly thought through, and well presented. + Inducements to complete surveys can be used,g.offering respondents a report ofthe survey findings or entry ina prize draw. * Ibis good practice to send questionnaires to an identified individual which should lead toa higher response rate. + Questionnaires can be delivered to the respondent by various means Including post, e-mail attachments or via publishing on a web site for interactive completion Design ofthe questionnairecan be split into three elements: + Select the question type foreach question and specify the wording + Design the question sequence and overall questionnaire layout Determine the questions tobe asked + Key link needs to be established between the research aims and the individual questions Question Types Different types of questions can be used + openvs.closed + single vs. multiple responses + ‘tickone box only’ vs. "tick all boxes that apply” + Ranking (indicate by numbering from 1-4 in order where 1 isthe most important) ‘Sealing eg, Likert scale Level of agreement witaa statement + Can also include the catch-all option ‘Other’ + Include N/A for those who may not have had a particular experience Question Wording ‘Some general rules can be stated on question wording: ‘Be concise and unambiguous + Avold double questions + Avoid questions involving negatives + Ask for precise answers + Avoid leading questions Layout and Sequence es good practice for he quesionalreto: Be of reasonable length + Flow logically and very simply, ic. avoiding complex routing + Begin with questions that will raise interest (especially for self ‘completion questionnelres) Havea title Havea briefintroductory statement Include contact and return information Have numbered questions for reference purposes Have questions and answer choices attractively and neatly laid out Be consistent in wording Be standardised by using as few question types as possible+ Bewritten ina legible font Make good use of ltalles and bold types eg, use Italics consistently to give Instructions suchas ‘please tick the box’ Confidentiality and Ethics ‘Allow for privacy and do not ask questions which may offend, or ask for data that Is not essential. Apart from anything else, the response rate will sufer: + In surveys promises of confidentiality are often made to the respondents to reassure and encourage replies. + Aside from the legal issues, it is unethical to obtain data from respondents by misteading them about the survey purpose and the ‘method of analysis. Survey The alm of a pilot survey is to detect any flaws in the questionnaire and correct these prior to the main survey. ‘+ The questionnaire should be tested ona small sample of subjects first. + If this is not possible, it should at least be tested on some ‘colleagues or friends. Having undertaken a. pilot survey, amendments can be made to the ‘questionnaire which will help to maximise the response rate and minimise the Main Survey +” Identify Respondents and Keep Track of Status {simple database is useful to keep track of when questionnaires are sent out and to whom. This can be updated with detalls of follow-up contact and dates ofreminders. + Number Each Questionnaire Each individual questionnaire must be uniquely identified by you, either before it is distributed or, to preserve anonymity, afterwards. This identification could be viaa number, or letters and numbers. ‘This code should be transferred to all the computer records that will be produced from the questionnaire, + Deliver the Questionnaire Follow through on the decisions made earlier about how the questionnaires should be administered and who the recipients should be. Deadlines for distribution should have been decided earlier with allowances for sending out reminders or carrying out follow-up calls Data Analysis Review research aims/objectives ‘This helps to focusthe analysis + Interpret results with eare Consider that some people may distort answers to attect the outcome + Look for facts an¢ patterns+ Don't ignore negative results + Take time when analysing results Data Analysis (Quantitative Questions + Collect data and inputinto spreadsheet + Display data using charts and graphs + Use graphs and charts that are clear and easy to understand ‘+ Interpret data and explain any patterns and trends + May have to break down analysis to find patterns Qualitative Questions ‘Open questions are harder to interpret as they give unique and wide ranging answers No common way of analysing + Organise comments into similar categories + Attempt to identify patterns, or associations and relationships + To analyse opinions on a spreadsheet it's best to use rating questions Questionnaire Development + Requires: + Consideration + Preparation + Time + Effort + Can be a long and frustrating process, but Is worth it to achieve a ‘questionnaire that will deliver the required cesults (Questionnaire Terms + Attitude Questions Questions that ask respondents about their IIkes/dislikes, preferences, ‘evaluation of objects, persons, organisations, events or situations + Behaviour Questions Essentially factual and relate to what respondents did in the past, are currently doing or may doin the future ‘+ Classifications Questions Usually factual and relate to demographic characteristics + Pre-coding [Numbering all response categories on a questionnaire so all responses receive a number as they are given + Routing Guiding the respondent or interviewer in @ questionnaire to the next relevant {question depending on the answer given‘Questionnaire Survey ‘Questionnaires - rmall ‘web based op Questionnaire - Advantages Broad coverage (loca, national, international) Frank and anonymous answers Economical fr large population Less Time consuming Quantitative data (easy to analyze) Fling at a convenient time Easy to respond (fixed responses) Good for sensitive & ego-related Qs. Questionnaire - Disadvantages No clarification for ambiguous questions Taadequate motivation to respond Poorly worded or direct questions Unattractive style and format Low response rate + Inaccurate responses - no verification + Cannot uncover causes or relationships for attitudes, beliefs, actions ‘Type of Questions Factual: Age, gender, education, experience (often used to investigate relationship) + Informative: ‘What respondents know about a given topic + Astitudinal/opini ‘To obtain info about respondents’ belts, feelings, values + Seltperception: Allows subjects to compare thelr ideas or actions with others + Standard of Action: How respondents willactina situation + Projective Questions: ‘Allow respondents to answer in an {direct manner by imposing their felings, attitudes or beliefs on others Structured/Unstructured Qs Unstructured Questions: * Allow respondents to reply freely without having to select one of several provided responses (also called open-ended questions)‘+ Useful for exploratory studies in which various dimensions and facets of problem are examined + Usually difficult to analyze responses, + Usually ow response rate ‘Structured Questions Provide a group of fixed responses + Dichotomous: yes/no, true/false, agree/disagree + Multiple Choice: ‘What sources do you use for writing term reports? (check all that apply) How many hours do you exercise per week (check only one response) + Contingency: Determine if the respondent is qualified to answer a subsequent question) Do you use .n-? (Ifno, please move to question 15) * Have you participated in... (Yes/No) Ifyes, how effective was ....? (Very effective, effective, ineffective . Scaling Responses + Likert-type Scale: Each response is assigned a numeric ranking based on ‘a continuum that contains predetermined units of measurement + Designed to showa differentiation among respondents opinions How important i... (very important, important.) How adequate are. (v. adequate ..Inadequate) How frequently do you. (frequently .. infrequently) + Semantic Differential Scale: Provides a set oftipolar adjective pairs How would you rate performance of the Income Tay staff? 1234567 ad v= Good Unfair Zl pair Harsh Gentle Rankcorderor Comparative Ranking Seale Respondents are expected to rank resporses according to their Importance/ preference + Should be used carefully as many respordents face difficulty in understanding/responding Only usea shortlist of responses (@ What sources do you prefer for writing term reports? Please rank them accordingto...(1= most preferred; 7=least preferred) Books‘SCHEDULES AND QUESTIONNAIRES + Schedule v. Questionnaire + Schedules and questionnaires are the most common instruments of data collection. These two types of tools have much in common. Both of them ‘contain a set of questions logically related to a problem under study; both aim at eliciting responses from the respondents; in bth cases the content, response structure, the wordings of questions, question sequence, etc. are the same forall respondents. Then why should they be denoted by the different terms: ‘schedule’ and “questionnaires”? This is because the ‘methods for which they are used are different. While a schedule is used ‘25a tool for interviewing, a questionnaire is used for mailing, + This difference in usage gives rise to a subtle difference between these two recording forms. That is, the interviewer in a face-to-face interviewing fils a schedule, whereas the respondent himself fills in a questionnaire, Hence the need for using two different terms. +The tool is referred to asa schedule when itis used for interviewing: and itis called a questionnaire when its sent toa respondent for completion and return Summary. ‘Surveys are more than a collection of unambiguous questions How questions are specified and put together will influence the respondents’ willingness to participate & the responses they provide Question Placement * Opening questions - start with easy non threatening questions + Personal Data ~ age, ender, qualification, experience + Sequence according to the research topic logical flew + Sensitive questions - should be asked only after trustis developed + Difficult questions - should be asked atthe end ‘The Major Decisions in Questionnaire Design 4. Content - What should be asked? 2. Wording - How should each question be phrased? 3. Sequence - In what order should the questions be presenced? 4. Layout - What layout wil best serve the research objectives? Content: Questions must meet 5 requirements ‘LAr the questions relevant. Do they pertain to the research problem 2. Ave the questions accurate, Do they accurately depict the altitudes, Dbehaviors, etc intended to investigate 3. Do respondents have the necessary information? + Qualify respondents 4, Do respondents understand and interpret the question correctly ‘5. Will respondents give the information?‘Question Wording + Camhave major impact on how respondent interprets question + llrespondents should interpret in the same way How should the questions be asked? Format; How much freedom do we give respondents in answering questions Open-ended questions =" What do you look for most in an architects service? = Is there anything else you would like to add about the finishes preferred? Open Ended Questions: key advantages ‘Wide range of responses and information can be obtalned + Answers based on respondent's not researcher's frame of reference + Lackof influence. Don't channel respondents thinking + Can help interpret closed-ended question's - why factor ‘Particularly useful as introduction to survey or topic ‘When it’s important to measure the salience ofan issue + When too many possible responses tobe listed or unknown, Open-ended questions: Key disadvantages ‘Ability and/or willingness of respondent to answer Interviewer's ability to record answers quickly or summarize accurately & probe effectively + “Toterviewer's attitude influences response + Time consuming (interview sessions, tabulation, classification, assignment, validation) + Difficulty in coding ‘Require respondents tobe articulate ‘Respondents may miss important points ‘= Non-response losed-ended questions (Fixed-alternative responses) What do you look for most ina job? Work that pays well Work that gives a sense of accomplishment Work where you make most decisions by yourself Werk that is steady with tte chance of being laid off ‘Advantages + Base of understanding‘Requires less efforton partof interviewer and respondent Ease of tabulation & analysis Less error prone Less interviewer bias Less time consuming “Answers directly comparable from respondent to respondent Disadvantages ‘+ Middle/Neutral categories often selected inappropriately (ignorance, safety) ‘+ Less opportunity for selfexpression + Less involving for respondents + Order of response categories can have major impact on results Dichotomous Questions ‘Agree/Disagree ‘Advantages + Easy wadminiserand tabulate Disadvantages + Prone to large amounts of error since polarized responses prevent ining information on the range of variation + “Failto communicate any intensity of feeling ‘Multiple Choice Questions ' "Are all possible alternatives included? + Too many alternatives, + Position Bias Scaled Response Questions = Closed ended questions where the response choices are designed to ‘capture an intensity offeeling (Likert, Staple, Semantic differential) ‘= Basy to code and more powerful statistical tools ‘+ Main problem: Respondent misunderstanding ‘Things toavold * Complexity: use simple, direct, conversational language + leading questions ~ that suggest or imply certain answers * Jaded questions — suggest social desirability, or are emotionally charged. Have you purchased a high quality Sony TV this year 10‘© Ambiguity and vagueness: Words such as “often, ‘occasionally’ usually’, “regularly, "frequentl’, “many”, should be used with caution. IF these words have to be used, thelr meaning should beexplained properly, Which state is bigger? - would your answer be based on population or + long.worded questions * double-barreled questions. Questions that refer to two or more issues within the same question. Where respondent may agree with only 1 part of multipart statement. £g.Do you think Nike offers better pricing and variety than other brands = Making implicit assumptions Jargon burdensome questions - that may tax the respondents memory ‘+ Embarrassing, sensitive, or threatening questions: ave you charged more on your credit ard than you should ‘Question Sequence & Layout Decisions nial stages ‘Screening or qualifying Questions: ‘= Need to gain & maintain respondent's cooperation ‘+ Make questionnaire simple for interviewer to administer ‘opening questions should be interesting, simple, and easy to answer. Which theatre did you goto last? + Whataspects ofthe theatre did you like best + Questions directly related to research objectives which require ‘mors effort and get respondent thinking abou: topic Difficult and Complicated Questions ‘Whats your averag? annual household income + May not be answered (sensitive or threatening) sut most ‘questions have been answered initial questions - screening and rapport questions Middle - product specific questions 3. End demographic questions. (Questions should flow logically from one tothe next General questions should be asked before more specific ones Earlier questions should not influence response to later ones Questions should flow from factual and behavioural questions to attitudinal and opinion questions {Questions should low from the least sensitive tothe most sensitive. {Questions should flow from unaided (which brand do you prefe:) to aided ‘questions a* Demographic questions should come atthe end. ‘Use multiple questions instead of one ‘Similar questions together ‘consistent mindset for respondents + Develop a logical flow ‘Use transitions between sections (Fg, In this section we ask questions about) * Distinguish between instructions, questions and responses ‘Distinguishing question and responses: INSTRUCTIONS LAYOUT A: Do you agree, disagree of have no opinion that this company has: * Agood vacation policy -disagree/notsure/agree. + Good management feedback - disagree/not sure/agree. + Good medical insurance -disagree/not sure/agree. INSTRUCTIONS LAYOUT: Does this company have: Disagree NotSure Agree, * Agood vacation policy 1 2 + Goodmanagemertfeedback © 1 + Good mediealinewrance 1 92 3 Filter and pivot questions should be used as necessary. ‘+ -AFFILTER question is one that screens out respondents who are not ‘qualified to answer a second question. ‘+ APIVOT question (sa type of filter question that is used to determine ‘what version ofa second question to ask. Developing logical low TF the questionnaire deals with several topes, complete questions on a single topic before moving on toa new topic + If topics are related, ask questions on related toples before asking. questions about unrelated topics + IF you ask questions about behaviors over some time period, follow chronological order backward in time ‘When changing topics, use some transitional phrase QUESTIONNAIRE LAYOUT ‘The layout and physical attractiveness ofa questionnaire ae important aspects ‘Questionnaires should be designed to appear as short aspossible * Questionnaires should not appear overcrowded + Leave lets of space for open ended questions * Questionnaires in booklet form are often recommended OPENING + Provide irst name (atleast) 12‘Provide name of company doing research ‘+ Provide reason for survey and topic State that no selling willbe involved and no personal data other than for statistical purposes ‘Tell respondent approximate time to complete ‘Reinforce that respondent's time is appreciated Invite to participate AND CLOSING Thank for time + Askif they had a positive experience and remind them that their opinions count Pretesting and Correcting Problems Purpose of pretest: To ensure that the questionnaire meets the expectations in terms ofthe information that will be obtained * Is question necessary - does it serve a purpose, will info be used ‘Missing important variables ~ does it provide the info needed ‘Match questions to objectives + Protest Specific Questions For + Variation + Meaning + Task difficulty + Respondent interest and attention + Ambiguous il-defined, loaded, double-barreled questions Pretest the Questionnaire ‘+ Flow ofthe questionnaire + Skip patterns + Length + Reepondent Interest and Attention ‘Questionnaire Design Flow Chart Step 1; Specify what information will be sought ‘Step 2: Determine type of questionnaire and method of administration ‘Step 3: Determine the content of individual questions ‘Step 4: Determine form of response for each question Step 5: Determine wording for each question Step 6: Determine sequence of questions ‘Step 7: Determine physical characteristics of questionnaire Step 8; Revie etope 1-7, revieeifnecessary Step 9: Pretest questionnaire, revise if necessary ‘Step 10: implement 13