10/10 Chapter 7 Experiments Experimental research is most appropriate for answering research questions about the effect of a treatment

or some other variable whose values can be manipulated by the researcher o Most powerful design for testing nomothetic True experiments o Have 3 things Atleast 2 groups (in the simplest case, an experimental and a control group) Variation in the independent variable before assessment of change in the dependent variable Random assignment to the 2(or more) comparison groups o The combination of these features permits us to have a much greater confidence in the validity o Our confidence in the validy of an experiments findings is enahcned by 2 more things Indication of the causal mechanism Control over the context of an experiment Experiments and comparison groups o True experiments must have at least one experimental group (subject tht get treatment) and a comparison group (subjects who get no treatment o Experimental group- in an experiment the group of subjects that is manipulated o Comparison group- a group that is exposed to a different treatment or no treatment at all Pretest and posttest measures o All true experiments must have a posttests- that is, measurement of the outcome in both groups after the experimental group has received the treatment o Many true experiments also have pretests that measure the dependent variable prior to the experimental intervention A pretest is exactly the same as a posttest, just administed at a different time Pretest scores permit a direct measure of how much the experimental and comparison groups change over time o A randomized experimental design with a pretest and a posttest is termed a pretestposttest control group design An experiment may have multiple posttests and perhaps even multiple pretests Multiple posttests can identify just when the treatment has its effect and how long This is particularly important when treatments are delivered over a period of time Randomization o Randomization or random assignment, is what makes the comparison group in a true experiment such a powerful tool for identifying the effects of the treatment o

o o o o o 10/15/12 -

A researcher cannot determine what the unique effects of a treatment are if the comparison group differs from the experimental group in any way other than no receiving the treatment Assigning subjects randomly to the experimental and comparison groups ensures that systematic bias does not affect the assignment of subjects to groups What random assignment does create 2 (or more) equivalent groups is useful for maximizing the likelihood if internal validity, not generalizability Matching is another procedure sometimes used to equate experimental and comparison groups, but by itself it is a poor substitute for randomization matching of individuals in a treatment group with those in a comparison group might involve pairing persons on the basis of similarity, gender, age etc tht can be a problem

limitations of true experimental design o the distinguishing features of true experimentsexperimental and comparison group, pretests (which are not always used) and posttests, and randomization do not help researchers identify the mechanisms by which treatments have their effects o in fact, this question of causal mechanisms often is not addressed in experimental research o the hypotheses test itself does not require any analysis of mechanism, and if the experiment was conducted under carefully controlled conditions during a limited span of time that causal effect (if any) may seem to be quite direct o but attention to causal mechanisms can augment experimental findings o true experimental designs also do not guarantee that the researcher has been able to maintain control over the conditions to which subjects are exposed after they are assigned to the experimental and comparison groups o examine the criteria for a nomothetic cause association time order of effects of one variable on the others nonspurious relationships between variables mechanisms that creates the causal effect context in which change occurs quasi experiments o testing a hypothesis with a true experimental design is not feasible with the desired subject and in the desired setting o such a test may be too costly or take too long to carry out; it may not be ethical to randomly assign subject to the different conditions; or it may be too late to do so o researchers may instead use quasi-experimental designs that retain several componenets of experimental design but differ in important details o aquasi-experimental design is one in which the comparison group is predetermined to be comparable to the greatment group

these research designs are only quasi experimental because subjects are not randomly assigned to the compensation of others o we cannot be sure of the comparison group to other groups 2 major types of quasi experimental o Nonequivalent control group deisgns- experimental and comparison groups that are designated before the treatment occurs not created by random assignment o Before-and-after designs no control group, look at subjects, undergo something, measure at time 2. Change in behavior shows comparison Nonequivalent contorl group design* o comparison group is selected but not randomly assigned o 2 ways to do that: Individual matching* In some situations this can create a comparison group which is similar to the experimental. When we cannot we use aggregate matching Aggregate matching* Finding comparison group that is similar on key aspects Still not randomly assigned Before and after designs o No comparison groups o Way to compare is pretest against posttest measures o These designs thuse useful for studies of interventions that are experienced by virtually every cases in some population o Types of before-and after design Multiple group before and after design- multiple tests and comparisons by different groups Repeated measures panel designs Time series designs Causality in quasi-experiments o Association between the hypothesized independent and depdent variables o Time order of effects of one variable on the others (cannot be as certain of time-order of some effects) o Nonspuriousness, we cannot be sure to eliminate extraneous influences However since we chose those groups, we should be able to rule out extrat\neous influences o Mechanisms that creates causal effect Features of QE do not in themselves allow identification of causal mechanisms However the repeated measures does provide a means for testing the hypothesis o Context in which change occurs* Nonexperiments o All of the other research designs we study are, of course, nonexperimental

Generalizability o The need for generalizabile findings can be though as the fatal flaw of true experimental designs o Components essential for true experiments and that minimize threats to causal validty make it more difficule to achieve sample generalizability (being able to apply the finding to some clearly defined larger population) and local generalizability (apply to individuals) conclusions o true experimental play 2 critical roles in social research first, they are the best research design for testing nomothetic causal hypotheses 2nd, true experiments also provide a comparison point for evaluadting the ability of other research designs to achieve causal valid results o True experiments are used infrequently for these 3 reasons: Experiments required to test many important hypotheses require far more resources than most social scientists have access to. Most of the research problems of interest to social sceints simple are not amenable to experimental designs, for reasons ranging from ethical considerations to the limited possibilities for randomly assigning people to different condtions in the real world The requirements of experimental design usually precule lare-scale studies and so limit generalizability to a degree that is unacceptable to man social scientists

10/22/12 Survey research Intro o

Survey research involves the collection of info from a sample individuals through teir response to question o Most popular because they are versatile, reliable, etc. Errors in survey o 4 types Poor measurement Nonresponse Inadequate coverage of the population Sampling error Writing question o Centerpiece of surveys o They way they are worded have heavy weight o All hope for achieiving measurement validty is lost unless the question in a survey are clear o Adherence to a few basic principles will go a long way toward ensuring clear and meaningful question 1. avoid using confusing phrasing- simple direct approach Use shorter rather than longer words and sentences

Breading up complex issues into simple parts Avoid double negatives Avoid double-barraled question (question tht ask 2 things at once) 2. Minimize the risk of bias. Specific words in survey questions should not trigger biases, unless that is the intent Produce misleading answers Cause subtle errors 3. Avoid making either disagreement or agreement disagreeable. people then to agree with a statement just to avoid seeming disagreeable. 4. Minimize fence-sitting and floating. 2 related problems in question wring also stem from peoples desire Fence sitters- people who see themselves as being neutral, may skew the result if you force them to choose between 2 opposites Even more ppl can be termed floaters: respondents who choose a substantitive answer when they really dont know or have no opinion o Maximize the utility of response categories Questions with fixed response choices must provide one and only one possible response for everyone who is asked the question (exhaustive and mutually exclusive) 2 exceptions Filter questions- have ppl answer a question first to weed out misleading answers Check all that apply o Vagueness ion the response choices should be avoided o Questions about thoughts and feelings will be more reliable if they refer to specific times or events o Sometimes, problems with response choices can be corrected by adding questions. o How many response categories are good? (. Find notes) Combining question in indexes o develop multiple question about a concept and then to average responses, in an index or scale Index- the sum or average of responses to a set of questions about a concept Idea is idiosyncratic variation in the response will avg out Index can be considered a more complete measure of the concept o Because of the popularity of survey research indexes already been developed to measure many concepts o It usually is much better to use such an index to measure a concept rather than make a new index o Use of a preexisting index both simplifies the work involved in designing a study and facilitates comparison of findings to those obtained in other studies o 3 cautions in order of using indexes:

Make sure each question is measuring the same concept you want to Combining responses to specific questions can obscure important differences in meaning among the questions The questions in an index may cluster together in subsets Designing questionnaires o Questionnaire- the survey instrument contain the questions o Individual schedules? o Build on existing instruments o Refine and test questions Order the questions o The way the questions are ordered will influence the respondents o 1st step- individual question should be in broad categories o Throughout the design process..(find notes) o One or more filter or screening questions may also appear early in the survey in order to identify respondents for whome the questionnaire is not intended or perhaps to determine which sextions of the multipart questionnaire a respondent is to skip o Context effects- one or more questions influence how subsequenmce question are interpreted Prior questions can influence how questions are comprehended, what beliefs shape responses, and whether comparative judgements are made o Should the survey be translated into one or more languages? In the 21st century, no survey plan in the united states or many other countries can be considered complete until this issue has been considered When immigrants are a sizeable portion of a population, omitting them from a survey can result in a misleading description of the population Organizing surveys o 5 basic designs Manner of administration way they are given to respondents Questionnaire structure- how survey is laid out Setting- where they take the survey Cost-expense of different types of survey cary greatly Comparison of survey designs o Various points about the different survey designs lead to 2 general conclusions 1st in-person interviews are the strongest design and generally preferable when sufficient resources and a trained interview staff are available: telephone surveys have man of the advantages of in-person interviews at little cost, but response rates are problem 2nd best design for and study will be deteremined by the studys unique features and goals rather than by any absolute standard of what the best survey design is Ethical issues in survey research o Less ethical problems than experiment and field research o Potential respondents can decline

Mixed mode surveys o Allow the strengths of one survey design to compensate for the weaknesses of another and they can maximize the likelihood of securing data from different types of respondents Not a perfect solution because u can get different answers from the survey mode, not because respondents have different opinions Ethical issues in survey research (cont.) o Confidentiality is most important focus of ethical concern Many surveys include some essential questions that might, in some way, prove damaging to the subjects if answers are disclosed Prevent any possibility of harm to subjects due to disclosure of such info, the researcher must preserve subject confidentiality Conclusions o Survey research is exceptionally eficnet and productive method for investigationg social research questions o It is easy to do, but there is training and education required There has been worthless survey results You must be prepared to examine carefully the procedures used in any survey before accepting its findings as credible And if you decide to conduct a survey, you must be prepared to invest the time and effort required to propoer procedures

11/7/12 Chapter 9 qualitative methods o Participant observation o Intensive (depth) interviewing o Focus groups Feature of qualitative research o Exploratory research questions, with a commitment to inductive reasoning o Look this shit up History pf qualitative research o Anthropologies and sociologists laid the foundation for modern qualitative methods while field research in the early devades of the 20th century o The case study o Qualitative research project often have the goal of developing and understanding of an entire slice of the social world. Not just discrete parts of it o Case study is not so much a single method as it is a way of thinking about what qualitative data analysis can, or perhaps should, focus on o The case may be an organization, community, social group. Family. Or even an individual; as far as the qualitative researcher is concerned, it must be understood in its entirety.

The idea that the social world really functions as an integrated whole, so that the focus in quantitative research o Thick description- a description that provides a sense of what it is like to experience that setting from the standpoint of the nnatural actors in that setting o Participant observer- refers to several specific roles qualitative research can adopt Covert observer Complete observer Covert participant Overt participantTaking notes o Written notes are the primary means of recording participant observation data o It is almost always a mistake to try to take comprehensive notes while engaged in the field o Look this shit up too