CHAPTER 1

Scientific Method

1.1
1.2
1.3
1.4
1.5

Introduction to Scientific Method

The Good Experiment
Evaluating Scientific Information
Model Organisms
Some Common Mistakes

1.1 Introduction to Scientific Method

Figure 1.2 Example: computer woes

We all use the scientific method in our daily lives - whenever we test possible causes of an observed phenomenon. It is composed of these events:

making an observation

thinking of a possible explanation (hypothesis)

thinking of an outcome that would follow (prediction) if your hypothesis

were true

testing your prediction (this is the hard part!)

Based on your result, you may need to revise your hypothesis or come up with
an alternate.

What is a scientific hypothesis? A scientific hypothesis:

Figure 1.1 The Scientific Method

is based on observations and previous knowledge

proposes an explanation

is testable (it leads to predictions with measurable outcomes)

If a hypothesis is not testable, it is not a scientific hypothesis. For example, you

might observe your friend acting strangely and hypothesize that his behavior is
being influenced by invisible aliens. This is a proposed explanation based on observations and thus fulfills the first two criteria above. However, to be truly testable it must lead to two opposing predictions with measurable outcomes:
a) If Ed is being controlled by invisible aliens, then....
b) If Ed is not being controlled by invisible aliens, then....
As you can see, the second one is even harder than the first to measure. How
can one disprove the influence of invisible aliens? You cant. Therefore, this is
not a scientific hypothesis.

Note that the final step does not say accept hypothesis. Why not? Well, thats
the one step that might differ between scientists and nonscientists. Any normal
person would say Yay, I was right! and move on. Scientists will say It looks
like we might not be wrong - lets put it to another type of test! In other
words, well take that same hypothesis and come up with another prediction
that should follow from it, then test that prediction. Well keep doing that over
and over until we have enough evidence to publicly put forth our hypothesis
and our data. Results are useful only if we share them!

We all do hypothesis testing of some form. For example:

Observation: I always get a cold when I go home for the holidays.
Your hypothesis might be one of the following:

I catch a cold because of the cold weather at home.

I catch a cold during the holidays because I am around children, who

tend to transmit colds.

I catch a cold because I visit home right after exams, and the increased
stress suppresses my immune system.

Your prediction would correspond to something like:

If cold weather causes colds, then I should not catch cold when it is unusually warm.

If being around children causes my colds, then I should not catch a cold
when my nieces and nephews are absent.

If its exam-related stress that causes my colds, then I should not catch a
cold after an unusually easy semester.

catching a cold depends on one of the other variables. Therefore, catching a

cold is the dependent variable.

1.2 The Good Experiment

Say we wanted to rigorously test the hypothesis that people who take
echinacea supplements are less susceptible to catching colds than those who do
not take echinacea. A good experiment:

Testing your prediction (this is the tricky part):

You keep notes on the weather each time you go home for the holiday,
and keep track of whether or not you catch a cold.

You note whether or not you catch a cold the one time that your brothers
family (and hence your nieces and nephews) did not visit home for the
holiday.

Tests one independent variable at a time

Minimizes bias (the experiment is performed blind)

Includes sample sizes that are large enough to reach statistical significance

1. Testing one independent variable

First, we need to make sure we are
testing ONLY the effect of echinacea (our independent variable) on the susceptibility to catching cold (the dependent variable). This means we need to split our
test subjects into two groups that are very similar in all reasonable respects:
the age distribution should be similar in both groups, as should the weight, gender, exercise habits, alcohol intake, etc. Then we randomly decide which group
gets the experimental treatment. The group that does not receive the treatment is called the control group. If the treatment is a medical one, such as a
new drug, our experiment is called a clinical trial. So this is what they are talking about when you hear the words randomized clinical trial. Once we have
randomly chosen which group gets the treatment, we can impose our intended
difference - for example, having the group take echinacea for a week. Then we
are ready to expose the subjects in both groups to the exact same dose of virus
and note who gets sick and who does not.

You note whether or not you catch a cold following an unusually light semester, when you had only one (easy) final exam.

Based on your results, you might either turn to an alternate hypothesis, or further test your idea. For example, if your hypothesis had been being around children causes me to catch colds, and you DIDNT catch a cold that one time that
your brothers family was absent, then you might further test this, perhaps by
visiting your brothers family at another time of year and seeing if you get sick.
The above type of hypothesis testing, while useful, is not very rigorous. It is
based on a small number of events (trips home), and there are a lot of other
things that might vary besides the thing you are trying to test. For example,
what if it is unusually warm AND your brothers family did not visit home? In
this case, you cant attribute your result to any one variable in particular.

2. Minimizing bias
Now, if you tell the test subjects everything about the
nature of the hypothesis being tested, there is danger that those in the echinacea group will EXPECT that they should stay healthy, and so they may be less
likely to FEEL sick and report symptoms. Those who are knowingly in the no
echinacea group may expect to get sick and will be more likely to report symptoms. How do you get around this? Well, you dont tell the subjects which group
they are in! This is known as a subject blind experiment. In order for this to
work, you need to give something that looks like the treatment to everyone,
but only one group is really getting it - the other group is getting a placebo - a
fake treatment for the purposes of making the subjects blind to which group
they are in. Great, now the subjects are blind, is that all we need to worry
about? Not quite. What if the determination of illness is done by an experimenter who knows which subjects are in which group, and expects the no echinacea group to get sick? Without intending to, she might convey this bias during the interview, perhaps by walking into the room with a sympathetic look on

_________________________________________________________

Question 1. Which of the following hypotheses are testable and therefore scientific? (ie- there is a possible way to disprove it)
a) Your luck is influenced by your actions in a previous life.
b) Embryonic stem cell research is unethical.
c) The Dinosaur T. Rex used a distinct call to attract mates.
d) Singing can decrease your cholesterol levels.
_________________________________________________________
In science, we take great efforts to get data from a large number of events and
to test just one independent variable at a time. In the previous example, the
independent variables are: a) weather b) children and c) stress. The dependent variable is "catching a cold". Just remember, youre testing whether or not

her face, thus making it more likely for the subject to report symptoms. To get
around this, we can keep the experimenter blind to the groupings until all data
have been collected. If both the subject and the experimenter are blind, we call
it a double blind study. This is the gold standard in minimization of experimental bias.

c) You could take a large group of subjects and randomly divide them into two
groups. Then have one group take vitamin C supplements for two weeks.
Then expose both groups to a mild virus and see who catches cold; note the
difference between groups.
_________________________________________________________

3. Using statistics
Let's say you found a difference in your two groups that
appears to support your hypothesis: 60% of the no-echinacea group showed
cold symptoms and 40% of the echinacea group showed symptoms. Can you
conclude that your hypothesis is supported? Not so fast. Think about this: what
if you had only 10 subjects in each group? That means 6/10 vs. 4/10 subjects
showed symptoms, a difference of only two people. Do you think this difference
is due to the echinacea, or could this arise by chance?
Think of it this way. If you were to have two groups of ten people flip a
coin, do you think it's possible that one group might get as many as 6/10
heads? And that the other group might get as few as 4/10 heads? Certainly! Applying statistical methods (which we won't go into), we can calculate that a difference of 2 or greater will arise by chance in 66% of trials.
Now, let's say you had a group of 100 people flip a coin. Do you think it's
possible that one group might get as many as 60 heads? Maybe. And the other
group as few as 40 heads? It's possible, but not likely. Statistical methods tell
us that the probability of that great a difference arising by chance is less than
1%.
Even without statistics, I think we can all agree that the 100 subject
study is more likely to be informative than the 10 subject study. But statistics
tells us exactly how large our sample sizes need to be in order to be confident
that the differences that we observe are not due to random variation.

As you can see, a lot goes into an experiment. Even so, we do not take
the results of an experiment as truth, but rather as being consistent with a particular hypothesis. If a hypothesis is rigorously tested again and again by many
independent researchers, AND it is capable of explaining a wide range of observations, then it may become a scientific theory. In daily life, we often misuse the word theory when we really mean hypothesis. If you ever
hear someone say I have a theory.... you can feel free to correct them. An individual can have a hypothesis, but no single person can have a theory: a theory is derived from rigorous testing by a large community of scientists.

Figure 1.3 Hypothesis vs. Theory

An example of a theory is the germ theory of disease, which states that many
illnesses are caused by microorganisms. It explains a large set of observations
and has been rigorously tested (first by Louis Pasteur in the 1800s, and thousands of times since) without cause for rejection. This is as good as it gets.
When a hypothesis becomes a theory, thats as far as it will ever go. There is a
common misconception that a theory can become a law. It cant.

Statistics: size matters!

(sample size, that is)
_________________________________________________________

Question 2. Say you want to test the hypothesis that: Vitamin C causes in-

A very well-tested hypothesis can become a theory.

A reproducible set of observations can become a law.

creased resistance to catching colds. What do you think about each of these
methods?
a) You could do a survey to ask who takes vitamin C and how often they catch
colds, and see if there is a relationship between vitamin C and cold resistance.

For example, Newtons law of universal gravitation states:

Every point mass attracts every single point mass by a force pointing along
the line intersecting both points. The force is directly proportional to the product of the two masses and inversely proportional to the square of the distance
between the point masses.

b) You could do a survey to ask who takes vitamin C, then put the vitamintakers into one group and the rest into another group. Then expose each group
to a mild virus and see who catches cold; note difference between groups.

This law describes what will happen between two objects, such as the moon
and Earth, or between Earth and object that you drop. However, it does not tell
us anything about why it happens. A hypothesis to explain this was proposed by
Einstein, and after rigorous testing it eventually became the Theory of General
Relativity.
_________________________________________________________

Most non-scientists learn about scientific discoveries via a secondary source

of scientific information, such as:

a summary of the study written by someone other than the original researchers
a newspaper clip, a blog, a video, Wikipedia

an interview with the researchers

Question 3. A scientific theory is

1.4 Model Organisms

a) an absolute truth that can never be changed

b) an educated guess
c) an explanation that has been very well tested and is supported by all data
d) an explanation that is supported by most, but not all, available data
_________________________________________________________

Let's say you wanted to test the hypothesis that a given chemical could
cause cancer. It would clearly be unethical to test this on human subjects. So,
how do we know which chemicals are cancer-causing (carcinogenic) and which
are not? For these types of studies, model organisms are very useful. We
need to choose an organism whose biology is similar to ours in the respect that
we are interested in. As we'll learn later, cancer-causing agents act by disrupting DNA replication. Since all organisms on earth, even bacteria, replicate their
DNA in a similar manner, we can use something as simple as bacteria to test
potential human carcinogens. Other model organisms include yeast, worms,
flies, fish, and mice, to name a few. The ideal model organism in each case depends on the question we are trying to answer. For example, the question of
why we sleep remains a great mystery.
Not only do we humans need sleep,
but apparently so does any organism
with a nervous system. By studying
sleep in the simplest organisms that
show this behavior, we can begin to
understand why sleep is universally
Play Video
important.

1.3 Evaluating Scientific Information

So, weve performed our echinacea study with a large sample size and
weve repeated our experiments several times. Weve followed up with further
testing of our hypothesis using new predictions. Weve performed statistical
analysis of all of our data, and weve made some lovely figures. Were going to
send our results in the form of an article to a scientific journal (such as Science
or Nature), which will then distribute it to other scientists for peer review. During peer review, other scientists will check our experimental methods and make
sure weve performed well-controlled experiments that test one variable, minimize bias, and use sound statistical analysis. If it looks good to them, they will
give your article a favorable review, and the journal will publish it. If they find
errors in our scientific methods, then its back to the drawing board. Maybe we
need to eliminate a particular variable from our experimental setup - perhaps
the reviewers noticed that the average ages of our groups were different. We
can look through our data again and discard the youngest subjects from one
group so that the average age of the two groups is now the same. We say that
weve controlled for age. If we still see a statistically significant difference, we
can say that it is not due to age. We send it for a second round of peer review and its accepted. Yay!

YouTube:

and song about a sleep research lab here at CSUN.

1.5 Some Common Mistakes

Lets finish this chapter by discussing some common mistakes that are
made at the start of the scientific method: observation. Types of observation
can include a single isolated event, or an observed trend. A common mistake is
to take an observation and directly making a conclusion from it, without hypothesis testing. For example, youll often hear people say something like Well,
you may have read a study that says echinacea does not protect against colds,
but my Aunt Mabel takes echinacea every day and has never had a cold in her

This published article is considered a primary source of scientific information, because it:

was written by the researchers who performed the experiments

shows original data

was peer-reviewed

Check out this short video

life! This type of statement usually comes with the attitude that it somehow
refutes the entire study. Does it? Where does this type of statement fit into the
scientific method?

The popularization of rap music in the late 1980s coincided with a

rise in narcotics use and violent crime.

Well, its an observation, so it is an important part of the first step of the scientific process.

Notice that while the statement does not state that rap music causes crime, we
feel compelled to assume that it does. Be aware of this tendency! Its human
nature to assume causation when faced with a correlation. When you hear a correlation, it's important to recognize it as such and to think of potential alternate
explanations. In the above example, the obvious assumption to make is that
rap music causes crime. But what about the other way around? That music
tends to reflect what is already happening, and thus it was the increase in drug
use and crime that caused a popularization of music about these topics?

An anecdote is an isolated, informal observation.

Aunt Mabel has never had a cold, and she takes echinacea.
Observations like this peak our curiosity and lead us to develop hypotheses,
from which we can then make predictions that can be tested rigorously. If the
results of these experiments do not support the hypothesis that echinacea protects us from colds, then we must reject this hypothesis, no matter what Aunt
Mabel does.

Heres another example. People who take vitamin C catch fewer colds than
those who do not take vitamin C. The most obvious assumption to make is that
vitamin C causes us to be resistant to colds. This might be true. But are there
alternate hypotheses that may explain this observed correlation? What if the
type of people who take vitamins are also the type of people who exercise?

A correlation is an observed relationship between two variables.

Figure 1.4 Correlation: the causation could go either way!

Figure 1.5 A third variable, C

Could it be that exercise boosts our immune system, and not vitamin C?

Well, it turns out that this is actually the case. If you control for exercise habits,
the correlation disappears between Vitamin C and resistance to colds.

Figure 1.6 Think critically about observed correlations

Figure 1.7 The Correlation Song

Keep a critical ear out for correlations.

Once you know how to recognize them, you'll hear them often!

Answers
1d. How might you set up this experiment?
2c. This is why there are so few theories!
3c. Answer a simply establishes a correlation. Answer b at least involves an experiment, but it lets the two groups self-select, meaning they may bring other
variables with them. Answer c randomly assigns subjects to each group, making
it the best choice.