What are Online Research Methods?

Tristram Hooley & Jane Wellens

What are online research methods?

Research methods that utilise the internet as
medium for research
Sometimes referred to as internet mediated
research
Includes research methods designed to
investigate both online & offline phenomenon

Types of ORM

online questionnaires
synchronous and asynchronous interviews
virtual ethnographies
online experiments
web analytics & content analysis
+ others

Online Research Methods Challenges

Draw on a wide range of methodological, theoretical
and disciplinary traditions
Understanding the range of tools, environments and
online cultures is a precursor to successful research
Technological and social environments change year on
year
Increased blurring of the online and offline
environments creates a need for new composite
methodologies
Potentially poses new and different ethical
considerations

Advantages of online research?

Huge uptake of ORM, especially online questionnaires
Useful in particular situations e.g. researching online
populations and previously difficult to contact groups
Can mitigate distance and space and so
internationalise research
Can save time and money
Decrease personal risk
Different dynamics of communication:
- more neutral venue?
- more thoughtful responses?
- different participants open up/contribute?

But.disadvantages
But issues of digital access and digital literacy can limit
who we talk to
Drop off rates for questionnaires high
Issues of identity verification
Challenges to building rapport in interviews
Circumstances of interview beyond interviewers control
Technical obstacles
Still need degree of technical competence and
institutional support (although this is getting easier all
the time)

Online Questionnaires - key

questions

What are
your
research
questions/
aims?

Who are
you talking
to?

How are
you talking
to them?

Sampling
Recruitment
Non-response

Question type
Survey length
Language

Advantages of online
questionnaires
Increasingly common approach (familiar to
respondents)
Speed and volume of data collection
Savings in costs (to researcher)
Flexible design
Data accuracy
Access to research populations
Anonymity

Online questionnaires - challenges

Sample bias
Measurement error (Sax et al, 2003)
Non-response bias
Length, response and dropout rates
Technical problems
Ethical issues
Recruitment
Identity verification
Response rates
Identifying appropriate tools & implementation

Design issues
Will your questionnaire be easy to use?

Usability

Or will the interface, colours and question

types frustrate people and put them off?
Will your questionnaire be accessible?

Accessibility

Or will it crash on old computers, render

weirdly on different operating systems and
be impossible to use with a screen reader?
Will your questionnaire be doable?

Doability

Or are your multi-media dreams running

ahead of your technical ability and are your
data arriving in an unusable format?

Improving response rates

Make contact before mailing the survey e.g. an introductory

letter
Provide information that builds trust e.g. names and photos of
the researchers
Engage gatekeepers and encourage them to endorse the survey
Think about how your brand (e.g. the university) will be
perceived by the subjects
Provide clear instructions on how to complete the questionnaire;
Limit the amount of personal information you request
Use simple questionnaire format and avoid complex or openended questions
Design survey so it takes approximately 10 minutes to complete;
Do not include more than 15 questions
Send one or two follow up reminders
Emphasise confidentiality (if appropriate)

Key resource: Fan, W., & Yan, Z. (2010). Factors affecting response rates of
the web survey. Computers in Human Behavior , 26 (2), 132-139.

Some Software Options

Google Forms http://docs.google.com FREE

Lime Survey http://www.limesurvey.org FREE
BOS http://www.survey.bris.ac.uk CHEAP
SurveyMonkey http://www.surveymonkey.com
Snap http://www.snapsurveys.com
Using the functionality of your website or VLE
Bespoke systems

So.
The quantity of information that may be
generated, and the speed at which responses
can be collected, can result in pleasing piles of
data- but we should be wary of being seduced by
sheer quantity; data is only useful if it is
representative of the larger population.
Wakeford (2000, 33)

Online Interviews key questions

Why use online interviews?
What type of interview (synchronous or
asynchronous)?
One-to-one or group?
Text based or using audio or video?
What software/service will you use?
How are you going to record, retrieve and
analyse data?

Online Interviews: advantages

Carry out interviews with a very geographically
dispersed population
Interview individuals or groups who are often difficult to
reach, such as the less physically mobile or the socially
isolated or those living in dangerous places.
Provide savings in costs to the researcher
Supply ready transcribed interview data, quickly,
providing fast and cheap alternatives to face-to-face
interviews
Reduce issues of interviewer effect as participants
cannot 'see' each other

Online interviews - challenges

Trust and rapport building

Design of interview schedule
Keeping people on topic
Dealing with technical hitches
Maintaining momentum
Dealing with silence
Guaranteeing the ethical rights of respondents
including informed consent, confidentiality and
privacy

Key resource: Salmons, J. (2010) Online Interviews in Real Time, Sage

London

Some software options

Adobe ConnectNow
http://www.adobe.com/acom/connectnow/
TinyChat http://tinychat.com/ (probably need to
upgrade for anything serious)
GoogleTalk http://www.google.co.uk/talk/
Many VLEs also have chat and virtual
classroom tools that can be used for this kind
of purpose.

So...
The data collected by virtual interviews can be rich and
valuable to the researcher, but the potential of on-line
research should not be exaggerated: many of the
issues and problems of conventional research methods
still apply in the virtual venue
Moreover, it is unlikely that online interviewing is going
to replace face-to-face interviewing but rather it is
another option in the methodological toolkit

Online research ethics

There is currently a big push to embed research ethics
more formally in the culture of the social sciences.
Yet there is far less of a consensus about online
research ethics.
This asks the question is there anything special about
the online environment that requires new set of ethical
guidelines?

Key resource: Economic and Social Research Council (2005). Research

Ethics Framework.

Ethical decision making and Internet

research
Ess & AoIR (2002) stresses Ethical pluralism and
argues that there is more than one ethical decisionmaking framework used to analyze and resolve those
[internet research] conflicts.

Note: the world of chatrooms, MUDs and MOOs and

USENET newsgroups, described in this report in 2002
is now radically transformed.

Key resource: Ess, C. and AoIR Ethics Working Committee (2002) Ethical
decision-making and internet research http://www.aoir.org/reports/ethics.pdf

Where does it fit in?

One of the problems of mapping existing research ethics
onto the online world is the difficulty of deciding how to
categorise the spaces that you are observing/interacting
with.
Public

Private

Published

Informal

Writing

Speech

Personal
Anonymous

Broadcast
Identified

Key resource: Internet Research Ethics: Digital Library, Resources Centre,

and Commons - http://internetresearchethics.org/

Ethical Challenges
The great variety of human inter/actions
observable online and the clear need to study
these inter/actions in interdisciplinary ways
This interdisciplinary approach to research leads,
however, to a central ethical difficulty: the primary
assumptions and guiding metaphors and
analogies - and thus the resulting ethical codes can vary sharply from discipline to discipline,
especially as we shift from the social sciences.
Ess, C. and AoIR Ethics Working Committee (2002)

Careful and critical use ORMs

So ORMs are not a shortcut `easy option

Many issues and problems of onsite research remain
Divide between onsite/online methods inappropriate
Must be used, carefully and critically and appropriately
in light of each specific research topic

Attempting to undertake online data collection is far easier than

successfully accomplishing it. For those who chose to perform it, they
must do so deliberately and cautiously.
(Best and Krueger, 2004)
While online methodological frameworks are in constant flux, change
is not necessarily always progressive: there is a need for online
researchers to practice their 'craft' with reflexivity.
(Madge and OConnor, 2005)

Further reading
Exploring online research methods
http://www.geog.le.ac.uk/orm/
Tristrams ORM bibliography on CiteULike
http://www.citeulike.org/user/pigironjoe/tag/online_research_methods

Tristrams blog
http://adventuresincareerdevelopment.posterous.com/
I talk about ORM sometimes and technology often on this
blog.