Some of this document is reasonably polished, but other parts are rough notes only.

I have also put up a webpage on "How to study statistics", aimed particularly at those
taking their first statistics course: click here.
My name is Paul Hutchinson, and I work in the Department of Psychology, Macquarie
University. However, this document is not really specific to psychology --- in previous
existences I've worked in Statistics and Civil Engineering departments, and I think
most of what is said below is quite widely true. Anyway, here is my advice on
How to do research
The first thing to say is that if it has to be done, it's you who's got to do it.
You. There's no-one else to do the project for you. Very likely, there's
no-one who will give you any help. You're the one who cares, the drive has
got to come from within you. You're the one who knows how to answer the
question, no-one else knows as much.
The above is not the whole truth --- as your project evolves, you'll find
yourself receiving technical help, wisdom, and emotional support from a
great many people --- but you need to be aware that self-reliance is vital.
There may be other types of research, e.g., in which a neophyte is fitted
into a slot in the God Professor's program. I don't have experience of
them.
Do. It is important that you start the main activities of the research --experimentation, or survey, or simulation, or data analysis --- sooner rather
than later. A common mistake is to think and plan and read, and never get round
to the main game. When you start, you may find that the real difficulties and
issues are quite different from what you thought in the abstract.
Research has several components, for example: thinking of the question, answering the
question, communicating the answer. It may be worth mentioning that not all of these
are essential.
Even without much of a question, one can collect some data, summarise it, and
report the results. Certainly if you're the first to think of this kind of data,
that's all you need to do. Certainly if the data keeps changing from year to year
(e.g., accidents, and many other society-level phenomena), this type of study is
always useful to a greater or lesser extent.
Even without an answer, the question is sometimes such a good one that it is
worth telling others about. (It might be that you don't have time to answer it,
or are uninterested, or not competent in the necessary techniques.)
Sometimes you don't want to communicate your question and answer. They might have
turned out to be less interesting than you thought when you started. Or you might
want to wait until you've done something additional.
Nevertheless, most research has all three components. The answering and the
communicating can be dismissed quite briefly.
The fact that you're doing research is strong evidence that you have the skills
necessary to answer the question. Not all of them, perhaps, but enough to work
out a way of attacking the research and to serve as a foundation for additional
skills to be learnt along the way. Incidentally, as a rule of thumb, if there is
a course on something, people think of it as a low-level skill --- it is not
necessarily below your dignity as a researcher, but you don't get any merit
badges for mastering it. (I am a little concerned at the increasing amount of
structured studies that universities are prescribing for their first-year
research students. To my mind, the overwhelming priority at that stage is to do a
doable piece of research, not attend lectures on bibliographic tools and the
institution's policy on intellectual property.) As to non-technical aspects of
answering the question, there is some advice below.
There are lots of textbooks and courses to aid you in your scientific
writing. Essentially, though, it is all a matter of practice.
One of the reasons I recommend psychology to students is that you can't
escape either the use of numbers or the use of words. (Economics is another
good subject from this point of view, but it doesn't have much of an
experimental component.)
As soon as you've got a message to communicate, you should write a report
on it, not wait until the whole project is finished. That probably means
writing 1000 words per week, every week. Just occasionally, supervisors
forget to tell new writers the obvious --- the easiest sections to write
are the descriptions of the results, and of what you did (because these
sections are highly constrained by the facts), and you should write these
first; what the results mean, and why you did what you did, are altogether
more problematic, so the Discussion and Introduction sections should be
written later. If you're desperate, imagine you're speaking to a friend
about your work, and write down what you would say. (Then, as always,
revise and edit, revise and edit, revise and edit.)
Some research has to be done to a tight deadline --- for example, the final-year
projects carried out by undergraduates. The big difference this makes is that you don't
have time (which you do in most situations) to get things wrong and then do them again.
The consequence is that there's a good deal of luck involved at this stage, so I think
that if your final-year project falls flat on its face, it doesn't mean you won't be
good at research in a more normal environment.
It is only natural for a student to ask "What will get me good marks?" In assessing
theses, some universities have marking schemes --- u marks are allocated for the
project design, v marks are allocated for the conduct of the experiment or survey, w
marks are allocated for the general writing of the thesis, x marks are allocated for
the review of previous literature, y marks are allocated for the presentation of the
results, z marks are allocated for the interpretation of the findings, and so on. Other
universities have no such marking scheme. (Even where one exists, it is very often
impracticable to use it strictly.) My advice in such a situation may be different from
that of others. But, for what it's worth, here it is. My impression is that students
typically spend too much time reviewing past literature, and not enough time thinking
about it and criticising it. The student's ideas need to be firmly grounded in
established wisdom, not plucked out of cloud cuckoo land. But the thing that will most
impress the marker of a thesis is a focus on the central ideas, a willingness to
criticise imprecise thinking by previous authors, the dissection of what is essential
from what is less relevant, and the demonstration of how the student's ideas sharpen
what has previously been blunt. In other words, intellectual oomph.
An important issue is that of what to study. If you have ideas, there's no problem.
But, is there anything you can do to prompt ideas to come to you? Or, in the absence of
ideas, how can you do something worthwhile? I don't suppose there's a complete answer
to these questions, but the following comments are intended to be helpful.
To get ideas, expose yourself to them.
Talk to someone about your research. Your mum or dad, for example. Or
(though I've never tried this) the top stream of 13-year-olds at your local
selective high school. Brainstorming with your colleagues is better than
nothing, but you're not really forced to decode your talk sufficiently.
Anyway, there are other uses for your colleagues, such as being rude to
you.
It is tremendously valuable if there's someone in your circle who can bring
off the trick of being rude without causing offence. Suppose you are a
cognitive psychologist, and in a seminar you claim that "activation flows
through the levels" of your model. You benefit if someone from another
tradition confronts you with the claim that "activation", "flows", and
"levels" are all meaningless. It is generally up to the young to do this.

Informal seminars, where you listen to others, and talk yourself, are an
important part of this.
Read widely. (But remember what I said earlier: reading is not what research is
about. Indeed, I would say there is a type of personality for whom too much
reading is a major danger to their ever doing anything worthwhile.)
o How to read: read the title, skim the abstract, look at the pictures and
maybe the tables, and if there's anything interesting, then consult the
text, looking for that specific point. (No-one starts at the beginning of a
paper and works their way through to the end.)
o What to read: many of the "prestigious" journals are best avoided, as they
have got into the habit of attending to a lot of details that everyone
knows are unimportant, thus diluting the real message. The papers are
polished and bland, and the reader is sucked into taking them on their own
terms. You need something rougher, that you can get to grips with.
Comments and letters in any journal. Whatever the point is, it is
made quickly. Furthermore, an area of controversy may be highlighted
for you.
Journals from outside your main discipline that are relevant to your
topic. For example, if your own discipline is psychology, see what
the management, sociology, medicine, and engineering journals are
saying about your topic. They may very well take sufficiently
different an attitude to provoke thought. And interdisciplinary
journals, e.g., on the borders of music and psychology, or law and
statistics.
For the same reason, journals from countries that Americans have
never heard of (such as Germany, India, and Japan).
Low prestige journals sometimes publish simple data on unusual
questions --- if the topic attracts you, ideas for improving the
method or generalising the results will soon come.
For the same reason, you can often find mental stimulation in
journals from 60 or 80 years ago.
o Your attitude: sympathetic criticism. So much junk is published
that a degree of sceptical hostility is appropriate when
approaching most papers (unless the author is on the Approved
List). But temper this with sympathy: a paper may be useless to
you, and it may even be clear that by any reasonable standard
the research was a waste of time, but yet the method may be
useful to one reader, a detail in the results may be just what
a second reader is looking for, and a third reader may find her
thoughts clarified by a point in the discussion.
If someone says your paper is trash, retort that anyone who writes a
decent paper can get it published, but it takes real talent to get
rubbish into print.
o
Forget harsh realities for a moment, and think seriously about what question
really interests you, gets you passionate. Even though you're trying to keep your
mind on the ideal, you will find practicalities obtrude themselves. Some of these
practical problems may turn into research projects.
Look to a leisure interest of yours for inspiration. Many students are interested
in at least one of music, sport, and politics; well, if their academic subject is
psychology, surely this has some interesting things to say about all of these?
The same goes for many other academic subjects.
Many of us are lucky enough to have easy access to computerised databases. Try
searching them with an unusual combination of words, e.g., statistics and music,
or laughter and Wagner, or measurement and theology.
Ask successful researchers in your department whether they have any particular
methods for getting ideas to come.
I find making lists, and then organising them, often useful. Lists of what? Of
questions, possible experiments, possible surveys, sources of data, ways of
operationalising a concept, ways in which your supervisor could be more
helpful,...
Draw an analogy. (Theoretical, or method of analysis.)
Thought experiments. If the results were to be ..., then we would conclude ....
On the other hand, if they were to be ..., our conclusion would be ....
Suggestions for what to do if you don't have any ideas.
Criticise a paper by someone else.
o Can I improve the experiment?
o Can I improve the theory? In statistical modelling, one might replace a
discrete variable with a continuous variable, or vice versa --- e.g., if
"blue" and "white" types have been proposed, replace them with a trait
("blueness").
o Can I refine the definition?
o Can I broaden the circumstances in which the effect occurs?
o Can I simulate this on the computer?
o Can I improve the statistical analysis? ("Two of a trade never agree", it
is said, and certainly no two statisticians have agreed with each other
this
century.)
Improvement
does
not
necessarily
mean
additional
complication. Sometimes, simple descriptive results are overlooked in the
rush to perform a complicated hypothesis test. If nothing better comes to
mind, try confidence intervals in place of hypothesis tests, or
nonparametric methods in place of parametric, or Bayesian weight of
evidence in place of a p-value.
If you do comment on a published paper, you're actually paying it quite a high
compliment. (If you come across 24-carat dross, you should follow the convention
of politely ignoring it.)
You can always write a paper on:
o The useability of university Calendars.
o The comprehensibility of ergonomics textbooks.
o The rise and fall of the use of particular cliches in the titles of learned
articles.
o Do readers want more raw data in journal articles?
o Bayesian weight of evidence in psychology journals, 1970--1996.
o How first-year students view research participation.
o Comparison of several fields in respect of their tolerance of low survey
response rates.
o Why do students hate statistics?
Get the idea?
Think of an experiment that no-one else has done, and do it. (The emphasis here
is on doing something because it can be done, not because you actually understand
the implications.)
The different packaging of something unoriginal.
Go to a journal that is less quantitative than you are, and teach the readers
something (be tactful, now).

General advice.
If you have a rigid deadline to work to, it is vital that you are not reliant on
someone else for anything important. Among the possible areas of difficulty are:
administrative approval from an outside body, construction of equipment,
computing. You must have a workable plan that can be implemented if approval is
refused/the equipment isn't delivered/the computing expertise is unavailable.
Most people beginning research underestimate the importance of the physical
aspects --- going through 500 files a second time because you didn't extract a
particular item of information the first time, coming in to the lab on Sunday
afternoon to try out an idea you've just had, driving at 4am to another city
because the evening news told you of a "natural experiment" there that day,
taking 60 journal volumes off the shelf to skim through for the key paragraph you
wish you had made a note of, and so on.
Write down an idea when you have it, because you may easily have forgotten it by
the time you get to the office. Some people even keep an ideas book for this
purpose; but the trouble with this is that it really need to be physically very
small, so that it can be always with you.
Work around the problem, if you can't solve it.
Sometimes a problem is best handled by defining it away.
There is some research that simply isn't worth doing. It may be obvious to the
drover's dog roughly what the problem is... and that the institutional barriers
to solving it are insurmountable. Surveys, especially, can be a substitute for
actually doing anything about the problem.
Importance of flexibility.
Luck. Have several projects going --- diversification --- they won't all be
disasters.
Don't be afraid to admit your ignorance. The nature of academic research is that
one is continually venturing into areas where one is an ignoramus. It is
unpleasant to admit this. But the earlier you do, the sooner you get an
explanation and the quicker you can put that little bit of extra learning to use.
There can be a danger in overmuch personal involvement. Some people love people
with Alzheimer's disease, or autism, or depression, and want to research the
condition. For a beginning researcher, the problem of separating scientific
evidence about the mass of people with the condition from the loved one's
personal experience is one problem too many --- it is better to select another
subject for research.
Perfectionism and attention to detail is a virtue. Taken to excess, it is a vice.
Some researchers seem to suffer from a "fear of success", which I think is
different from obsessive perfectionism. I think what happens is that they
perceive the successful completion of their research as the end of one stage in
their life and the beginning of another, and they are reluctant to move on to the
new stage.
Universities have become very bureaucratic about documenting the progress
of research students. One can appreciate the reasons for this, and yet
there is a sense in which it is all a waste of time --- if the student is
successful, there's no need to have a file of routine documentation; if he
or she isn't, no amount of paperwork will rescue the situation.
The supervisor may feel the student is wasting his or her time, and yet be
reluctant to say this (or say it forcefully enough) because it may actually
be wrong and because in any case it will be discouraging. The supervisor
may proffer clear advice, and yet the student be stubborn. If I had an
answer, I'd give it here...

All sorts of stuff has been written about the role of the supervisor. Much of it
is sensible. But it's all a long-winded way of saying that it is reasonable for
the student to expect the supervisor to listen to him or her for something like
20--50 hours per year. Anything more than that is taking a responsibility away
from where it properly belongs (i.e., with the student).
Don't miss any deadlines that have been agreed with your supervisor. If you do,
then (depending on the nature of your excuse) you'll go on to either the list of
people who probably won't make it, or the list of people who probably won't make
it but who deserve sympathy. You do not want to be on either of these lists. And
don't be late for meetings with your supervisor.
Don't expect much praise from your supervisor. If your supervisor thought about
it, he or she would realise that a few words of praise were both justified (by
the brilliance of your work) and desirable (for the benefits of positive
reinforcement). Sad to say, he or she doesn't think about it: from about your
second week of research, you've been accepted as a grown-up and been judged by
the same standards as the professors.
You might find it worthwhile reading How to get a PhD. Managing the
peaks and troughs of research by E M Phillips and D S Pugh (Milton
Keynes: Open University Press, 1987).
Here are three points that Phillips and Pugh make, all of which I mildly
disagree with. (i) Intelligence-gathering (i.e., description of the facts)
is not research. (ii) Research of the testing-out type (as contrasted with
exploratory research and problem-solving research) is much the most
appropriate for a PhD student. (iii) It is necessary for your thesis to
have a "thesis", in the sense of a connected argument and message.

It may be that Phillips and Pugh are substantially correct in the context
of relatively mature fields of study; and that my opinion that (i)
intelligence-gathering may be research, (ii) exploratory research is useful
and appropriate, and (iii) the research may be too messy to have much of a
"thesis" to it, is formed by my experience being mostly in relatively
immature fields of study.

Taste. Is it permissible to use phrases like "glass ceiling" or "evidence-based

medicine" or "gold standard" in reporting your research? Fastidious writers avoid
cliches most of the time, but not every use of them should be condemned. (In some
allegedly scholarly fields of study, they seem to be compulsory.)
Don't worry about the possibility of someone else duplicating your research. It
is rare for this to happen, and, when it does, it is stimulating to examine the
similarities and the differences.
Don't despair if it becomes necessary to tear up two years' work. This is quite
common, and often means you are so expert and can see things so clearly that you
can finish the whole project in only six more months.
If you do happen to discover something important, be sure to do it at the right
time and place. Perhaps you remember the inventor of the Infinite Improbability
Drive: "just after he was awarded the Galactic Institute's Prize for Extreme
Cleverness he got lynched by a rampaging mob of respectable physicists who had
finally realized that the one thing they really couldn't stand was a smartass"
(The Hitch Hiker's Guide to the Galaxy, Chapter 10). Read here about an earlier
savant who suffered an equally dire fate.
Applying to do research.

There is quite a lot of advice available on the WWW about how to choose which
departments to apply to (for a research degree), and how to increase your chances of
success. I'm a bit sceptical about the need for this --- I think that by the end of
their undergraduate careers, most students know in what sub-field of the subject they
want to work, or what style of approach they want to use. Furthermore, they know which
departments at which universities study that sub-field or take that style of approach.
Consequently, their list of desirable departments is already a short one; if they have
made personal contacts, the list will be very short indeed. However, if you think you
need advice, here is mine.
You should appreciate that there are an enormous number of specialisations
available in the scholarly world. Flick through one of the directories of
scholarly societies if you don't believe me. For example, there is a Colour
Society of Australia, an International Association for Cross-Cultural
Psychology, and a Stress and Anxiety Research Society. If some
specialisation appeals to you (surely you'll like something in all your
undergraduate
studies!),
then
sometime
about
the
middle
of
your
undergraduate career, you should stop thinking of yourself as (for example)
an embryo psychologist, and start thinking of yourself as (for example) an
embryo colour psychologist. You will then only be interested in departments
that are strong in colour studies.
If you are in this position, do not limit yourself to departments of
psychology. You might find the most suitable place is in a group within a
chemistry or physiology department.

Make personal contacts. If colour psychology really does fascinate you, make
contact with people in the field. Academics are enthusiasts for their own little
patch, and will be delighted to find someone who shares that enthusiasm. If
possible, get some lowly job in their department during the vacation. But be
warned --- I would say it is impossible to successfully fake this. Alleged
enthusiasm without an appropriate level of specialist knowledge and native
intelligence will get you nowhere.
If you really are unsure --- if you just feel you'd like to do research in
"psychology" or "chemistry" --- you may do better to delay your entry into
research until your ideas have matured a little further. (You cannot be expected
to have a fully-thought-out research project at this stage; what I mean is that
you should know (for example) that your interest is in cognitive psychology
studied by computer-controlled experiments on normal humans.)
A little advice of a statistical or technical nature.
This is not the real theme of the present document, but a few comments of this type may
be worthwhile. You might also like to read Pitfalls of data analysis by Helberg.
There is often tension between the respective advantages of
standardising on one particular method of answering a question, and
of using a variety of different methods. Think of measuring
unemployment, or television viewing, for instance. On the one hand,
one wants a standard, reproducible, method --- comparisons from month
to month are important, and one has reason to think that whatever
defects are present remain pretty constant over time. On the other
hand, any single practicable method is bound to have peculiarities
and biases, and the "real, true" answer should probably be
synthesised from several different methods, even if each of them
makes use of only a small sample size.
When controversies break out, one factor is often the intrusion of a new
player into the system: the existing interests have learned to co-exist
with each other and with the defects of a question-answering method, but
this doesn't suit someone new.
The following is very much a broad-brush statement, to which there
must be numerous exceptions: too often, a standard method is adopted
too early in the development of a subject. In research, one usually
wants the "real, true" answer, whereas for administration, one may
prefer a standardised, reproducible, answer.
"Figures often beguile me, particularly when I have the arranging of them
myself" (Mark Twain). Presumably sociologists have studied the use of
statistics in the wielding of power?

Meta-analysis: this term refers to quantitatively combining the results of

previous studies of a subject, in order to arrive at the right answer. I'm not
terribly keen on the idea, but this may be because I have not worked in fields
where there have been lots of previous studies of the same question. There are
some difficult principles and practicalities involved, and inadequate handling of
these led critics to refer to meta-analysis as being "meta-silliness". I think
there have been substantial improvements in the methodology, and these were
winning me over, but then I read a paper by Sohn (Clinical Psychology Review, 16,
1996, 147--156), which argues that publication bias is so serious as to vitiate
all conclusions from literature reviews in the field he examines, effectiveness
of psychotherapy (and I think the implication of the paper is that publication
bias has a more serious impact on a meta-analysis than on a review in traditional
narrative format).
Contrast between exploring data, and using data for hypothesis testing.
Maybe you want to do both. One easy technical solution is to randomly split your
data into two portions, explore one half and generate hypotheses from it, and
then test the hypotheses using the other half.
Importance of a random sample, or random allocation. I have the feeling
that often it is foolish to prefer a rubbishy sample of 200 to a random
sample of 20, yet often this is done.
It might be possible to argue that a rubbishy sample of 200 is the better
for hypothesis generation, whereas a random sample of 20 is the better for
hypothesis testing.

My impression is that the subject which is leading the way as regards strictness
of methodology is medicine. Many people will be familiar with the basics. In
order to measure the effect of something, it is necessary to compare it with
something else. That is, we have a treatment group and a control group. Patients
are assigned at random to the one group or the other. It is desirable that
neither the patient nor the doctor who evaluates the patient's condition know
whether the patient received the drug or not. (This is termed a double-blind
experiment.) But did you know this idea is being taken so far that Gotzsche
(Controlled Clinical Trials, 17, 1996, 285--293) seriously suggests that the data
should be blinded during statistical analysis? That is, two reports are written.
The first assumes A to be the treatment and B to be the control, and the second
is based on the reverse assumption. Both are completed before the code is broken.
This is to avoid bias creeping in during the data analysis and writing.
Moderate-sized effects imply large studies. What may turn out to be
the most important statistical paper of 1996 was published in the
Oxford Textbook of Medicine, of all places: the convincingly-written
article by Collins et al (1996) will surely encourage tens of
thousands of medical professionals towards good statistical practice
in their research. One of the messages is that there may be a number
of treatments that are only moderately beneficial, but yet would be
very worthwhile because the disease is common or the treatment is
cheap or both; and which, because the treatment vs. control

difference is not great, require large sample sizes in order to

establish statistical significance of the difference. These sample
sizes are so large that they imply collaboration between many
investigators. Collins et al give clear and interesting examples of
both large single clinical trials (e.g., aspirin in treating heart
attacks), and of meta-analyses of many small trials (e.g., adjuvant
therapy for early breast cancer treated surgically). The effect sizes
in these studies were large enough to be well worth having --- a
reduction in 35-day mortality from 11.8% to 9.4% among patients with
acute myocardial infarction who were treated with aspirin, and a
reduction in 10-year mortality from 48% to 38% among women aged 50+
with stage II breast cancer who had tamoxifen included in their
treatment --- yet were also small enough that clinical trials of the
usual sizes were too small to detect them.
It would not surprise me if there were to be a trend towards simple, large,
collaborative studies in many areas of scientific enquiry besides medicine.
Now, consider that very many students conduct some sort of small research
project as part of their studies. I suggest that if different students in
different universities together decided on a common topic and worked out a
common methodology (and stuck to it), then (i) the results would be of much
greater value than results from a single institution, and (ii) the
experience of collaborative work would itself be valuable training, if
indeed we are entering an era when this will be increasingly common.

Collins R, Peto R, Gray, R, Parish S (1996). Large-scale randomized

evidence: Trials and overviews. In Weatherall D J, Ledingham J G G, Warrell
D A (Editors), Oxford Textbook of Medicine. 3rd Edition, pp. 21--32.
Oxford: Oxford University Press.

Don't get carried away with the statistical analysis. Tell the story
of your research using the tools from your first course in
statistics, not your last --- that is, using nothing more complicated
than well-chosen descriptive techniques plus the concept of standard
error. As much as is practicable, present your results in such a way
that the reader can see that the conclusions from your more
complicated calculations are probably correct.
Suppose your research is experimental; and involves obtaining a measurement
(e.g., an accuracy score) from each subject in each of four conditions;
that the four conditions consist of each combination of two levels (e.g.,
big and small) of one factor with two levels (e.g., light and dark) of
another factor; that the subjects are in two groups (e.g., males and
females); and that interest centres on whether the average interaction
effect is different for males from what it is for females. Such an
experimental question would be recognised by some people as being about the
three way interaction in a three-factor experiment, one factor being a
between-subjects factor and two of the factors being within-subjects
factors. Such people would know that a concise way of analysing the data
and presenting the results would be via Analysis of Variance. I recommend:
(a) Yes, do that. (b) But also, present your results in a way that enables
them to be roughly checked. (Otherwise, your readers will presume you've
got them wrong.) In the example, the interaction can be calculated for each
subject. It is the difference between two differences, [(big, light) (big, dark)] - [(small, light) - (small, dark)]. So calculate this for each
subject, and summarise the values for the males by their mean and s.d., and
the values for the females likewise. It will then be clear how similar or
different are the interactions for the two genders.

It might be worth going to quite some trouble to make sure you, your
supervisor, and your examiners agree on this one. Having gone to the
trouble of really understanding your data and then explaining it
comprehensibly, it would be vexing to come across an examiner who wants to
see 20 pages of ANOVA tables. Or vice versa.

Just occasionally, it may happen that careful attention to notation in

writing an equation will reveal that you really did not understand
something you thought you did (or even that everyone else did not
understand it).
This can happen with any type of mathematical equation, not necessarily one
involving random variables. But, to give a simple example of this latter
type, there might be an averaging process somewhere, and one can sometimes
get confused about what domain it is that one is averaging over.
T P Hutchinson, 21.July.97: phutchin@bunyip.bhs.mq.edu.au
For details of my books, click here.
Some of this document is reasonably polished, but other parts are rough notes only.
I have also put up a webpage on "How to do research", aimed at students beginning
research (e.g., for Honours or PhD degrees): click here.
My name is Paul Hutchinson, and I work in the Department of Psychology, Macquarie
University; but in previous existences I've worked in Statistics and Civil Engineering
departments, and I think most of what I say here is quite widely true. This document is
aimed particularly at those who are taking their first course in statistics (most of
whom are specialising in some other subject). See also a page by Michael Hughes of
Miami University, Ohio: click. Anyway, here is my advice on
How to study statistics
Do well in it --- if you do well, you are much more likely to enjoy it. (And in later
years you may even make some money tutoring other students!)
If you have to work hard in order to do well --- so be it. Finding subjects difficult
and needing to put in a lot of hours of study is usual.
Perhaps you're really interested in some other subject, and are only taking a
statistics course because you have to. That's a nuisance, and I sympathise with you,
but remember this: statistics is not something imposed from outside (by mathematicians,
perhaps); statistics as a subject exists because psychologists and agriculturalists and
engineers and medical researchers and economists built it, and they built it because
they needed it. You will find that you need it, too.
Overview of introductory statistics.
A typical introductory statistics course is in three parts.
Data description. For example:
o Pictorial presentation of data (principally, a single numeric variable).
o Calculation of summary statistics (principally, a single numeric variable).
o Two numeric variables: scatterplots, correlation, regression.
Probability. For example:
o Rules for doing calculations with probabilities. Diagrams that help with
the calculations.
o Discrete probability distributions. The binomial distribution. The Poisson
distribution.
o Continuous probability distributions. The uniform distribution. The
exponential distribution. The normal distribution.
o Expectations and variances of random variables.

Inference. Here, we try to draw conclusions about the population from which our
sample came.
o The standard error of the mean. The Central Limit Theorem.
o The concept of using a sample statistic to estimate a population parameter.
Criteria for choosing a good method.
o The general idea of hypothesis tests. Examples of hypothesis tests for
particular situations.
o The general idea of confidence intervals. Examples of constructing
confidence intervals.
o Inference in the linear regression context.
Hypothesis testing can come to dominate a statistics course, and even whole areas
of the application of statistics. This is rather a pity, (a) because students
ought not to get into the habit of thinking that hypothesis testing is the be-all
and end-all of statistical analysis, and (b) because it is quite controversial
just what hypothesis tests mean and how they should be used.
(These few sentences are rather advanced, don't worry if you're not yet at
the point where you appreciate them.) In particular, the output from a
calculation is an indication of the probability of the data conditional
upon a particular (null) hypothesis being true. What an investigator wants
is not this, but an indication of the probability of the hypothesis being
true, conditional upon the data that was observed. A good many philosophers
and statisticians would say this latter "probability" is a meaningless
concept.
One can imagine statistical inference in its present form going entirely out of
fashion, and being replaced by strength of evidence or cost-benefit analysis or
something else. (It is much harder to imagine data description or probability
becoming obsolete.)
Missing topics --- but important.
Notice a couple of things that are absent from the above list, though they are
important when doing statistics "for real".
Rather little is usually said about the process of collecting the data. I think there
are a variety of reasons for this, some of them good and some of them bad. One good
reason is that there tends to be a lot of detail that is specific to the variable being
measured and the purpose for which the measurement is made. Another is that students
often do not appreciate the rights and wrongs of data collection (it all appears too
easy, for one thing) until they have had some experience with describing data and
drawing inferences.
Nor is there much said about the strategy used when approaching a dataset. By this, I
mean things like:
Quality control of the data,
Building one's understanding of the data by:
o Looking at variables one at a time (graphical and numerical summaries,
o Deciding just what are your research questions, and which variables should
be thought of as independent variables and which ones should be thought of
as dependent,
o Looking at variables two at a time (scatterplots, correlation coefficients,
etc.),
o Looking at variables three at a time (e.g., plotting y vs. x with different
values of z distinguished),
o and so on.
Possibly, getting rid of interaction by transformation of the dependent variable,
Considering whether it is appropriate to test hypotheses, or whether this is
impracticable (because, for instance, the sample is too far from random),
Possibly, deciding to explore only a randomly-selected half of the dataset, using
this to generate hypotheses, and using the other half to test these hypotheses.
(It is common to find most of the above in an introductory course, the point I am
making is that it is rare to find much emphasis on putting them together.)
Two directions of approach.
Much of the statistical work associated with scientific research can be put into one of
two categories --- modelling the mechanism and data analysis.
With modelling, assumptions and deductions from those assumptions are
prominent. Many simple "models" have been invented, and you need to be
familiar with the basic repertoire. Some of them refer to the deterministic
aspect of the situation, and some of them refer to the stochastic (random)
aspect.
Suppose we are concerned with the number of events of a particular type.
The deterministic part of the model might say that the expected (average)
number of events is obtained by adding together two independent variables,
appropriately weighted. The stochastic part might say that we can assume
(a) the probability of an event happening is constant, and (b) the
occurrence of events are independent. (It is useful to know that the
binomial and Poisson probability distributions follow from this pair of
assumptions --- it is unnecessary to re-invent these distributions every
time these assumptions are appropriate!)

With data analysis, the emphasis is on trying to understand what the data is
telling us. We try to follow passively, without preconceptions about what is the
appropriate model, where the data leads us.
Good modelling keeps in close touch with data, good analysis leads us to models
of what is going on, and the statistician is constantly switching from one to the
other.
General advice.
My feeling is that there is rather too little probability in introductory statistics
courses these days. It could well be useful for you to take a course specifically in
the subject. I know in practice a lot of students select courses partly by eliminating
the subjects they hate (or think they hate). Try to overcome this as much as you can.
Even if you hate mathematics, make sure you have a good grasp of algebra --without it, you'll always be struggling in your statistics. (Many universities
have some sort of bridging course available for students who have neglected or
forgotten their algebra, and then find they need it.)
Even if you hate the humanities, make sure you can write well in English --- an
important part of statistics is the presentation of your results to your
audience, and this necessitates explaining yourself.
Textbooks? Rather to my regret, it seems to be the fashion these days for students to
only study the textbook that the lecturer recommends, rather than finding one that
suits the way they themselves think. So there doesn't seem to be much point in my
putting down some notes on texts that I like. For a description of a booklet that I've
written, that is intended as a supplement to any conventional introductory text and as
a revision aid, click here.
Should you work alone, or collaborate with your classmates? My view is that everyone
needs to find their own individual way of understanding things, and this can only be
done by struggling on their own. But I have met people whom I respect who say that for
them the interchange with others is a vital part of learning. (I have no doubt that the
good student benefits from explaining something to poorer students. What worries me is

a feeling that this may actually do more harm than good to the poorer students, because
they haven't struggled through to the answer themselves.)
Perhaps the most obvious point of all --- to even start to learn something, you need
four exposures to it: the lecture, the textbook, the tutorial, and the homework. So go
to all lectures, tutorials, and practical classes, do all the homework assigned to you
and a bit more. Your course is designed on the assumption you do all of these things
(and is designed to be passable by the average student who does them, not only by some
genius). It will very quickly become very difficult if you start missing things.
"Just put the numbers into the formula." This is a very undesirable way of doing
statistics, but if all else fails, it may be better than nothing. For one thing, you
might choose the right formula and get the right answer and pass the exam. For another,
doing things without understanding sometimes leads later to understanding.
Most statistics lecturers are pleasant enough, and you should not be afraid to go to
them for help with the course, if you need it. They may even announce certain hours
when they'll be in their office and available to assist. But be reasonable about this
--- if you have missed a class and therefore haven't got whatever was handed out then,
you're responsible for your problem, and you should solve it by copying the paper from
a friend.
In the olden days, poor lecturers used to defend themselves by saying that by teaching
badly, they forced students to think for themselves and learn. This is mostly nonsense,
but nevertheless does contain just an element of truth, I feel. There are at least two
dangers with a course that is too well-organised and slickly presented. (a) There may
be an over-emphasis on training, as contrasted with education. For example, students
may learn how to smoothly and competently tackle a problem of a certain type, yet not
recognise it when the wording is changed slightly. (b) Students may overestimate how
much they are learning and understanding.
When you're doing an assessed piece of work, what you do will naturally be driven by
what will get you the marks. But at an earlier stage, when you're actually learning the
stuff, it will be helpful to think of partial answers that you could give if you didn't
know the full answer. For example, suppose you recognise a particular question as
requiring you to perform a nonpaired (that is, independent groups) t-test; as well as
doing this, think about the partial answers you could have given at earlier stages of
your course.
If you only knew about techniques of descriptive statistics, you might draw
a box-and-whisker plot to compare the two samples.
If you knew about the standard error of the mean, you might draw a picture
showing mean plus or minus 2s.e. of one sample beside mean plus or minus
2s.e. of the other sample.
If you knew that the variance of a difference is the sum of the variances
(provided the random variables are independent), you might work out that
the difference between the two means is such-and-such, and the
corresponding standard error is so-and-so, and notice that the difference
is or is not much larger than its s.e.
It is curious that even quite experienced teachers of statistics can find first-year
exam questions set by someone else quite difficult --- the language is just
sufficiently different that they're not altogether sure what the question is getting
at. (I'm not sure what to make of this, except perhaps statistics is a difficult
subject to teach and therefore statistics lecturers deserve a pay rise.)
Finally, try not to be too hard on your lecturers: "The young have aspirations that
never come to pass, the old have reminiscences of what never happened. It's only the
middle-aged who are really conscious of their limitations --- that is why one should be
so patient with them."
T P Hutchinson, 7.Mch.97: phutchin@bunyip.bhs.mq.edu.au