Research skills

Unit 5 Research skills.....................................................................................................2

1 Five stages of the research process.............................................................................2

1.1 Conception...........................................................................................................2

1.2 Realisation – making research happen.................................................................2

Description.............................................................................................................3

1.3 Investigation – gathering information..................................................................3

1.4 Interpretation – making sense of data..................................................................4

1.5 Utilisation of research findings............................................................................4

1.6 Managing research...............................................................................................5

2 Designing research questions......................................................................................5

Discussion..............................................................................................................7

3 Research design...........................................................................................................8

Discussion..............................................................................................................9

4 Triangulation.............................................................................................................10

Description...........................................................................................................11

Discussion............................................................................................................12

5 Critical review of research approaches.....................................................................12

Conclusion....................................................................................................................15

Next steps.....................................................................................................................15

References....................................................................................................................15

Notes............................................................................................................................16

1
2
Unit 5 Research skills
This final unit introduces the research process and identifies some key skills necessary
to carry out a successful research project. Five stages of the research process are
identified: conceptualisation, realisation, investigation, interpretation, utilisation. We
consider each stage before considering the higher level skills needed to effectively
carryout research such as designing research questions, research design, triangulation
and critical review of research approaches.

1 Five stages of the research process

In this section you will be exploring each stage of a research process in detail.

1.1 Conception
At its most basic, this is the ideas stage, when you decide what interests you. It is
worth keeping a journal to record your thinking and reflections on intellectual issues
as a repository of possible research ideas and problems. But as well as coming from
personal reflection, research ideas are also generated by people and organisations you
work with, or groups you want to help in some way.

Focussing your research question is a continual process during the first phase of a
research project, but it doesn’t usually end there. You often have to come back to
refine your research question during the course of research as findings emerge and are
interpreted. This was demonstrated in the Massey and Wield paper on science parks
studied in unit 4. Their paper argues that conceptualization and reconceptualization is
possible and required throughout research. The main points of reference in
conceptualisation and reconceptualisation are the literature (what theories are
relevant), the methodology you are using, and for practical and policy oriented
research, the concerns of potential client or user groups. The other central
considerations are scale, resources (people and time) and access. In other words, is it
doable with the resources available, and can you get the information required? You
can start with a preliminary question, then cycle between each of these until you have
refined a question where the theory and method fit well together. It may help to break
your central question down into a hierarchy of questions, with the answers to the sub-
questions leading to the central question.

1.2 Realisation – making research happen

This is often not accorded the importance it deserves. Research design is about
organising your research approach in order to answer your research questions, or at
least generate reliable valid information to interpret for your research questions.
Typical considerations are quantitative or qualitative or a combination of the two.
Another issue is what data sources to use? For example, in survey research, how do
you determine the structure of a sample? Or in qualitative case study research, how do

3
you decide which cases to research? Triangulation helps to ensure the validity of
findings which we will consider shortly.

When it comes to resourcing research, a key resource is clearly you and your time.
But other vital resources you may need could be finance for travel, fieldwork,
facilities and equipment, interviewees and interpreters expenses etc. Planning and
organising all these is important and it is worth drawing-up plans and schedules to
keep track.

Figure 5 Research process in a typical project cycle

Description

Figure 4 Research process in a typical project cycle

End of description

1.3 Investigation – gathering information

4
Key issues here are interpersonal skills, such as teamwork and communication. There
is also understanding responsibilities (honesty and integrity of researcher), rights of
informants (including data protection), and ethical issues (including confidentiality).

Interview skills need to be rehearsed and practiced. Other important issues are getting
the setting right, making people feel comfortable, and ensuring you have a well
organised approach to recording what is said accurately. A tape recorder is the best
way of doing this, but sometimes this isn’t appropriate as we saw in unit 4, when the
researcher had to abandon using a tape recorder because of the unwanted attention it
generated which compromised the integrity of the record. Consequently, she had to
change her research tool from an open interview to a semi-structured interview in
order to aid her recall of the answers to her questions. Answers had to be memorised
as note-taking wasn’t an option because the respondents would have felt
uncomfortable with this.

The health and safety of yourself, any assistants and respondents should also be
considered. It applies to fieldwork as much as to laboratory work. Try to think through
the risks in each case and take advice on the best ways of minimising them.

1.4 Interpretation – making sense of data

We’ve all heard stories of researchers losing all their fieldwork notes either in hard or
soft copy. But there are many other less publicised stories of people spending hours
searching for notes and computer files that they need when writing up. Remember to
write up field work as you go, and as soon as you can. Keep copies in separate places
– not all on a computer hard drive. Build up a filing structure for your data; for
example, with a spreadsheet. Bear in mind the structure will almost certainly change
as you go along.

There are a couple of important points to consider when beginning analysis. The first
is that you shouldn’t leave analysis till you have collected all the data. Do as much as
you can of preliminary analysis or reflections in a research journal as you go along.
Secondly, recognise that a substantial amount of analytical thinking also takes place
as the writing up stage, so start doing this from day one. Thirdly, the chances are that
you will collect more data than you actually need. You need to be prepared to select
data that is relevant for answering your questions rather than including it all. The
latter would simply undermine the sharpness and coherence of your results and
conclusions.

1.5 Utilisation of research findings

As said above, begin writing from day one, both in your research journal and on the
different stages of work as your research develops. As early as you can, map out the
structure of the thesis, dissertation or report down to chapter headings so that you
begin to slot pieces into this structure early on. Developing writing skills is about
doing it regularly. It is also about doing manageable chunks – Somerset Maughan
used to write 3000 words before breakfast, but for more pedestrian writers you need
to think about chunks within a structure, and setting targets (words/timescale). The
structure may change as you write more and more, but time thinking through a good

5
overall structure is time very well spent – and the same applies to working out a
structure for each chunk before you start writing. Ensure you get feedback, and give
yourself time for editing and for all the apparently insignificant detail like notes and
references which can take far longer than you expect! Finally, using your research to
make a difference is not just about how you report and communicate your findings –
important though these are; it can also be about how you involve stakeholders in the
research process.

1.6 Managing research

It is vital to plan with a chart for all activity giving start and finish times. This can be
done as a table on a computer or drawn up on paper. Setbacks are bound to happen,
face up to them and try to address them before they turn into disasters! If a major
setback or disaster does occur, remember one thing: the fact that someone has
probably been there before, so through talking to supervisors and colleagues you may
get advice and support. Some disasters are difficult to discuss, but it is really
important you don’t push them aside; rather, concentrate on finding the right person to
talk to, such as a counsellor.

Reflecting on progress (and the lessons from setbacks) is essential if you want to learn
how to be a better researcher. This often takes place naturally, in quiet moments, but
more and more people are coming to recognise that formally putting aside time for
yourself to do this is important, and trying to encapsulate your reflections in a journal
or notebook is also very valuable.

As you can see from this brief overview of the components of the research process,
skills around organisation, time management, rigour and routine are necessary to
conduct a research project. But there are also some higher level skills which you will
need to actively develop if your research is to be effective. These include generating
research questions to investigate, research design skills, triangulation and critical
review of research approaches which we now consider.

2 Designing research questions

You may already know the area or problem you wish to research. However, rendering
it researchable is not always straightforward. Critical to the success of the research
project is generating research questions that have boundaries and are manageable. The
following discussion explores how an interest in the role of institutions was
conceptualised to render it researchable.

As you should recall, the concept of ‘institutions’ has a dual meaning:

 as ‘complexes of norms and behaviours that serve a collective purpose’ (de

Janvry, 1993); and

 as organizations, when they are established (‘institutionalised’) and embody

established norms and roles.

6
All innovation takes place in an institutional setting and must to some extent imply a
deliberate attempt to change institutions: either the promotion of change within an
institution or the establishment of a new institution. For example, the spread of an
innovative use of IT will require organizational change as well as changes to the
norms and roles within organizations. A significant change in, say, environmental
policy will at least require change in the roles played by existing agencies if not more
sweeping organizational changes and changes in underlying values. Hence an
important aspect of innovation research is research about deliberate attempts at
change in institutions.

Many questions about how innovation can be promoted (or ‘institutionalised’) can be
stated in the following general form:

How can change X be implemented successfully in circumstances Y?

For example:

How can IT be used innovatively in primary schools?

How can skills and capacities be built in developing countries?

How can innovation be promoted in small companies?

How can local authorities promote sustainable development?

How can NGOs influence environmental policy?

These questions, stated in this way, are policy questions rather than research
questions. In order to turn them into questions which are researchable one has to ask
more about the ‘circumstances’. For example:

Who (what organization or agency) is trying to implement this change?

What criteria will be used to judge what is ‘successful’? What values underlie these
criteria?

What institutions (established organizations, networks, norms) is the change to be

effected within?

One problem about researching ‘How can…?’ questions like those above is that one
cannot find out about things which have not yet happened. One can only speculate. So
it is important to investigate cases where innovations have occurred or at least where
there have been clear-cut attempts at innovations. Yet the likelihood, particularly in
the early days of an attempted institutional change, is that there are few clear
successes and perhaps many more examples of failure or no attempted change at all.
This points to the use of case study as the most appropriate method, focusing on
whatever successful cases exist, together perhaps with one or two cases where definite
attempts failed in a way which promises to be illuminating.

7
So the general ‘How can…?’ question can be turned into a researchable question of
the following general form:

In those cases where attempts at X have been successful, how were they done and
why have they been successful?

Turning such policy questions into researchable questions will usually entail bringing
in some specificity about the ‘circumstances’. For example, are we interested in the
efforts of particular implementing agencies, or in attempts at innovation in a particular
region or sector?

Activity 33

Think of an innovation, the possible implementation of which is close to your own

research interests. Try going through the following sequence:

Question 25

Write a policy question of the general ‘How can…?’ form above which relates to the
innovation you are interested in.

Question 26

Answer (briefly) the above questions about the ‘circumstances’.

Question 27

What problems do you see in trying to research this policy question directly? How do
they relate to the brief discussion above?

Question 28

Now turn your policy question into a researchable question of the form suggested
above.

Discussion

Not everyone’s research interests can be turned into a question of exactly the same
form. But I hope you were able to see that there are common kinds of problems in
researching innovation or change directly which may be overcome by concentrating
on cases of success. Indeed, even with a research question which prioritises cases of
success like this, one can still include other cases in one’s investigations. The general
form of research question put forward here is only one way of investigating
innovation in institutions – although I would suggest it is an important one.

End of discussion

This example demonstrates how to move from an open and broad question, to a series
of more discrete researchable questions. Armed with your research questions you can
then identify what your knowledge needs are in order to answer them. Each research

8
question may need a particular research method to arrive at the data evidence you
require. Thus, you will have to consider in some detail your overall research design,
informed by what is manageable from the perspective of your resources and time.

3 Research design
One of the most difficult parts of doing research is designing the research process so
that the thinking element melts naturally into the practical element or how to address
the overall mix of theory and practice in the research process. One difficult issue is
how to move from a big question or issue to be able to design a piece of doable
research which we discussed above. Usually a range of methods can be tailored to a
particular research question - a choice of methods is available for each research
question. This is explored in the next reading which is an extract from a paper on a
clothing cluster in New York City. The research is about exploring the concept of
embeddedness and how that changes economic relations and outcomes, or to put it
another way: how social relations change market transactions. The network studied is
not highly innovative, but joint problem solving arrangements and complex
adaptation are present as foci for knowledge production. The concept of
embeddedness helps explain how trust develops in networks and is relevant to study
of the way clusters operate as you may recall from your study of clusters in unit 3.
The paper, however, adopts quite a narrow view of embeddedness based only on
structural relations between economic actors, and ignoring (deliberately) other
dimensions of embeddedness such as institutions and culture, which was embraced by
the theoretical framework in the unit 3 paper on north Italian clusters. Nonetheless
this simplifying of the research field allows a very focused and fine-grained analysis
through an ethnographic approach, with clear and important findings (including the
risks of over-embeddedness, leading to reduced exposure to information external to
the network).

The extract is particularly interesting because of its detailed exposition of the

methodology used, the boundaries set around the investigation and the rationale for
doing what was done. The appendix is also worth looking at as it lists the questions
that informed the open-ended interviews and provides a useful example of how the
big question of how embeddedness changes economic relations and outcomes was
deconstructed and reframed to provide a manageable framework for research.

Activity 34

Read pp. 38-42 (starting Research methodology to include Appendix) of Brian Uzzi
(1997) ‘Social structure and competition in interfirm networks: the paradox of
embeddedness’, along with the appendix on p.67.

PDF content unavailable

Question 29

How does the author justify his choice of field?

Question 30

9
Do you have any comments on the choice of firms and interviewees? Can you suggest
any variations on this aspect of research design that might have been worth
considering?

Question 31

How were interviews and field observations used to improve the validity of the
findings?

Question 32

What was the purpose of the ‘cross site display’?

Question 33

Note the different phases in the study, and the gradual development of a framework
for interpreting the data, and that this was initially based on theory. How was
‘construct validity’ tested, and do you have any comment on this approach to validity?

Question 34

The questions at the end, clearly relate to complex theoretical terminology, but Uzzi
has been quite successful at pitching them at a level that the ordinary senior manager
would understand. How does Uzzi guard against response bias?

Discussion

Question 29

Classic ‘arms length’ market relations ought to apply in this type of highly
competitive field; an ethnographic approach provides rich data for theorising and
analysis, but with 23 cases generalisability is only moderate.

Question 30

Considerable attention has been paid to the issue of representativeness, and to the
potential influence of size and ethnicity, and firms were not chosen to be part of the
same trading network. This suggests some design variations: either examining a
trading network, or examining pairs (firms that trade with each other) – this would
allow triangulation on economic transactions from the perspective of each partner
pair. However this would have required a larger sample to address the issue of
representativeness. There is considerable variation in size, from 2-182 employees.
This does raise questions about the interview strategy in the larger firms, so that
relevant social networks are examined, by selecting appropriate interviewees.

Note that the unit of analysis was the interfirm relationship, and that various sources
of information were used to gather background statistics and provide a sampling base.

Question 31

10
Ethnographic interview accounts were compared with field observations to check for
differences.

Question 32

It is often the case in ethnographic studies that researchers quantify their data in some
respects to estimate frequency and weighting of data across cases (as here).

Question 33

Construct validity has been tested by getting industry experts to check if the
interpretation (based on the conceptual framework) is plausible. There is considerable
debate amongst some ethnographic researchers regarding validity (with some arguing
there is a crisis of representation, based on the perspective that interpretations are just
constructs of the ethnographers mind).

Question 34

Uzzi partly addresses response bias by non-directive questioning; there may be a

tendency for respondents to cast themselves in a good light; also his triangulation of
field observation and ethnographic data would help remedy this.

End of discussion

4 Triangulation
The concept of triangulation suggests that this broadness is a strength; that greater
confidence can be placed in research findings based on the points of agreement
between the results of diverse methods. Triangulation involves applying a range of
different perspectives on an issue in order to reduce uncertainty about it. The greater
the differences between the perspectives, the more the confidence that can be placed
in any agreement between their findings in a particular case. Triangulation therefore
relies on an understanding of the sources of bias embodied in the different methods –
their strengths and weaknesses as vehicles for engaging with the research context.

Triangulation is originally a term from navigation or cartography. A navigator can

pinpoint their position by taking bearings on three features whose identity and
position are known. Theoretically the lines should intersect one another at a single
point, but in practice there are almost always errors in measurement. Plotting lines on
a map on these bearings from the location of the three features will produce a triangle.
The interior of the triangle is the best available approximation for the navigator’s
position. The reason that three bearings are taken, rather than two is because the third
line gives an idea of the magnitude and direction of the errors introduced by the other
two.

11
Figure 6 An example of navigation by triangulation

Description

Figure 6 An example of navigation by triangulation

End of description

The metaphor carries over to research, where the term triangulation is used in two
different senses. In survey and interview methods, triangulation refers to asking the
same question in different ways, or cross-checking responses against one another. In
semi-structured interviewing, for example, it can be helpful to point out any perceived
discrepancies between replies to different questions, and ask the interviewee to
explain them or to clarify any misunderstandings. The other sense refers to the use of
multiple perspectives in order to converge on an understanding of a particular issue or
research question. The different perspective may be embodied in different methods,
data, theories, disciplines and/or researchers. Multidisciplinary research can involve
some or all of these elements.

In both cases, information from one perspective is compared to one or more other
perspectives. The comparison helps to reduce the uncertainty inherent in taking a
particular perspective (whether that is asking a question in a particular way,
employing a particular methodology or adopting a particular theoretical standpoint),

12
in as much as the different views agree. It should be noted that even though
navigational triangulation generally relies on taking three bearings, it is not necessary
to use three questions or three different data types etc. The main idea is to compare
between as many divergent perspectives as is reasonable, whether that is two or
twelve.

In the next reading, Roe discusses the use of triangulation in policy research on
sustainable livelihoods. Sustainable livelihoods for the world’s population are
arguably necessary in order to reconcile conservation of the environment with social
and economic development giving rise to sustainable development. The example he
gives of a triangulation is one based on different theoretical perspectives of
sustainable development, and the interconnections between them. Thus, he adopts an
approach which draws on the perspectives and methods of more than one traditional
academic discipline, by taking an interdisciplinary or multidisciplinary approach
[Note 1] . But much of what he says can be applied to triangulation on different
methods too as discussed in unit 4.

Activity 35

Read pp. 35-37 (starting ‘Complex policy analaysis’ up to ‘Starting the complex
policy analysis’) of Roe (1999). Why does Roe argue that elements of a triangulation
be as different as possible?

PDF content unavailable

Discussion

Roe argues that the greater the divergence between the viewpoints upon which a
triangulation is based, the greater the confidence that can be placed in its results.
Divergent perspectives are ones which introduce opposing biases, and these will tend
to cancel one another out. Note that he does not claim that triangulation brings a
researcher nearer to the truth. Convergence between different viewpoints is merely
taken as a good starting point for pursuing an issue further. In Roe’s terms, bias has
been reduced (but not eliminated).

End of discussion

5 Critical review of research approaches

A critical review involves looking at the research process with an impartial eye,
stepping back from the detail and considering the research as a holistic investigation
and the plausibility of the conclusions and recommendations drawn from it. As such
you are assessing the construct validity of the research. This involves evaluating
whether there are good operational definitions and measures, multiple sources of
evidence and a chain of evidence which establishes links between the research
questions, research methods, data and conclusions. This is complemented by judging
three aspects of the work: the academic rigour brought to bear on the investigation;
the contribution to understanding of technology policy and innovation and the
accessibility and coherence of the project reporting style.

13
Key to this will be the conceptualisation of the problem. As Mukherjee and Wuyts
say, ‘...good conceptualisation is akin to an opening in a chess game: it sets the stage
for the game but does not predetermine its final outcome.’(in Thomas, Chataway and
Wuyts, 1998 Fast, p.257). Conceptualising your research question will determine your
approach to the data. But even when you have your data, data does not speak for
itself. You will need to approach and interrogate the data from different angles, some
of which may lead nowhere. Even where some results are disappointing or
unexpected, it is worth thinking critically about why this was and why it failed to
support the idea you had. Indeed, this may suggest an alternative idea that you may
not have uncovered but for this more interrogative approach. So all data is
meaningful, both in terms of what it tells you and what it doesn’t tell you. As
Mukherjee and Wuyts say,

‘You approach data from different angles because you want to investigate rival
notions in the light of evidence each brings to bear on the problem at hand. Through
contrastive inference you seek to arrive at that notion which appears more plausible in
the light of the overall evidence. Your interest is not just to test a particular idea in
isolation ...What matters is to compare ideas. The implications are that you do not
merely accept an idea because some evidence points towards it, because it may
support other ideas equally well, and you pay particular attention to evidence which
goes counter to an idea, because it may point towards a superior notion.

In policy analysis, you may make extensive use of secondary data relevant to your
research question. It is important, therefore, to reflect carefully on the limitations
inherent in the data. This is not just a question of checking the quality of the data.
What matters as well is to question the type of questions data allow you to answer and
which questions cannot possibly be tackled with the data at hand. This point matters
equally when you collect your own data (for example, through survey research). The
design of a survey is predicated by the type of questions you seek to answer.

Analysing data in relation to a specific question almost invariably implies further data
manipulation...Data manipulation are the ‘bread and butter’ of good applied research.
Good conceptualisation of research often reveals itself first by the quality of data
manipulations and, conversely, careless thinking often shows itself in the way data are
organized or manipulated.

The question you seek to answer, the type of data at your disposal, and the way you
manipulate and transform them, jointly set the stage for the kind of techniques best
suited to address the question. This is not a one-way street. Often data are
manipulated in certain ways so as to make them amenable to the use of a particular
technique. Nor is it a one-shot effort.

In general, trying out different techniques using the same data to answer the same
question is a good thing. This way you avoid turning technique into a fetish: a tool to
impress rather than to gain insights.

Numerical skills matter provided they are not used in a mechanical way. They should
be used in combination with, and not as a substitute for, clear conceptualisation of the
problem...Numerical skill in itself does not provide magical answers to problems.
Good conceptualisation of the problem determines whether numerical skills are used

14
appropriately or foolishly...Few patterns in the data can be seen by merely staring at
them. It is through data manipulations, data transformations and the calculation of
numerical summaries or the use of graphs that conceptualisation interacts with
numerical skill to bring out patterns in data which either test or generate ideas. There
is no easy shortcut, but fortunately much mileage can be obtained with relatively
simple techniques and extensive practice.

Finally, good data analysis should be fun, although it may involve hard work. There is
something very exciting about ‘finding out’. This is why data analysis should never
just consist of testing old ideas with new data, but new data should also generate new
ideas (Heckman, 1992, p.884). Do not hammer your data into submission in a
mindless exercise of number crunching. Nor turn your back to them in an attack of
data phobia. It is worth learning to converse with quantitative data in a genuine
theory-driven dialogue (in Thomas, Chataway and Wuyts, 1998, pp.257-259)

This quote relates to quantitative data, but I think the guidance is sound for qualitative
data too. That is, that data evidence cannot speak for itself, that it has to be explored,
interrogated and cross checked if meaningful results are to be derived. Finally, I
would like to leave you with a checklist for evaluating research which has utility both
for evaluating the research of others when carrying out a literature survey for
example, and for producing your own piece of research to ensure it is robust and
authoritative.

1. Is the research topic or problem clearly stated and its significance for
technology policy and innovation clear?
2. Are the research questions unambiguous and appropriate?

3. Is the research design strong with a range of meaningful and trustworthy

methods to facilitate triangulation and a consideration of the ethical
dimensions?

4. Does ‘pragmatic opportunism’ inform the research in order to set manageable

boundaries?

5. Does the research demonstrate competent and confident critical reflection on

research practice with steps taken to eliminate the effects of the researcher and
the research process on respondents?

6. Is the presentation of results innovative, interesting, unambiguous and free

from distortion?

7. Are the conclusions and recommendations plausible and convincing?

Conclusion
This unit has considered the stages involved in the research process and explored
some key skills for effective research. What emerges is that there is no one way of
doing research, and that iteration within the research process is necessary to render
results, conclusions and individual learning more robust and meaningful. Moreover,

15
the element of individual judgment is critical to research. Researchers may not always
get it right, but if the reasons for their decisions are transparent and ethical, the
researcher, their stakeholders and audience can usefully learn. Lastly, research results
are never an end point or the definitive truth, merely a landmark on an intellectual
pathway to deeper enlightenment.

Next steps
After completing this unit you may wish to study another OpenLearn Study Unit or
find out more about this topic. Here are some suggestions:

Science, maths and technology

If you wish to study formally at The Open University, you may wish to explore the
courses we offer in this curriculum area:

Engineering and technology

The Open University Postgraduate Programme in Development Management

or find out about studying and developing your skills with The Open University.

OU study explained

Skills for study

or you might like to:

Post a message to the unit forum, to share your thoughts about the unit or talk to other
OpenLearners

Review or add to your Learning Journal

Rate this unit

References
de Janvry, A., Sadoulet, E. and Thorbecke, E. (1993) ‘State, market and civil
organizations: new theories, new practices, and their implications for rural
development’, World Development 21, 4: 565-689.

Mayers Uzzi, B. (1997) Social Structure and Competition in Interfirm Networks: The
Paradox of Embeddedness, Administrative Science Quarterly, 42 March pp35-67.

Roe, E. (1994). ‘Narrative Policy Analysis: Theory and Practice’, Durham and
London, Duke University Press.

16
Thomas, A, Chataway, J. And Wuyts, M. (1998) ‘Finding out Fast. Investigative skills
for policy and development.Open University Press, Milton Keynes.

Notes
1. Although these terms are often used interchangeably, it is worth noting that
sometimes a distinction is made between them. For example O’Riordan (1995) uses
‘multidisciplinary’ to refer to a discipline which draws on two or more others in a
fairly fixed relationship. He uses ‘interdisciplinary’ to refer to a research approach
which is committed to bringing appropriate disciplinary perspectives together in
particular cases, without having to specify which ones in advance of engaging with a
given research problem.

17