Linnaeus University

Informatics
Programme: Master of Information systems


October 12
th
2013





Assignment 3 Presentation of an Article and Comments to an
Article

Choosing Evaluation Models - A Discussion on
Evaluation Design, Hanne Foss Hansen, 2005,
Evaluation Journal, vol. 11 (4), pp. 447-462.

(Athens Group 11a)

Authors: Kanelakopoulou Marianna, Manikas Konstantinos, Rekleitis Dimitrios
E-mails: kanel_marianna@hotmail.com / mk222zq@student.lnu.com
konstantinosmanikas@gmail.com / km222fd@student.lnu.se
dreklitis@yahoo.com














Course: 4IK003
Teacher / tutor: Christina Mrtberg / Thomas Ivarsson



1
PAPER REVIEW

WHAT: What is the key question that the author addresses in her text?
In the evaluation literature, many evaluation models are presented. The most
discussed are six and some subcategories. So, the logical question that arises is: "How
can evaluation sponsors and evaluators decide how to design an evaluation with so
many models to choose from?".

WHY: How does she motivate (i.e. justify) the importance of the question?
This question is crucially important because the volume of evaluation models
is so large that causes confusion to sponsors and evaluators in which one to choose in
order to have the desired effect. Moreover, the authors of evaluation literature claim
that their own work is the most appropriate, but the author of the paper suggests that
alternative approaches of evaluation should be studied comparatively and meta-
models should be developed in order to help us consider the designing evaluations.

HOW: How does she go about to produce her arguments to the question addressed?
Firstly, the author presents the Typology of Evaluation Models which includes
Result models, Explanatory process model, System model, Economic model, Actor
model and Programme theory model, and then she escalates her reasoning by
presenting three Criteria which are three different types of logic. The first Criterion
suggests that The purpose of Evaluation should Determine Design. The second
Criterion suggests that Characteristics of the Evaluand should Determine Design. In
this reasoning there are two variants. The Possibility Reasoning, which means that the
characteristics of the evaluand should be determined in order to select a model, and
the Legitimacy and Justice Reasoning, which means that the characteristics of the
evaluand should be considered, as well as what they justify, in order to select a model.
The third Criterion suggests that The problem to be solved by the Evaluated Object
should Determine Design. Apart from the Criteria presented, the author claims that
there are also some other factors that influence evaluation design. These are the
Negotiation Process, namely the different requests and interests of the actors can
become objects of negotiation, the Appropriateness, namely what the designers expect
to be appropriate considering the image of the actor and the situation, the What is
Usually Done, namely past experiences that influence the evaluation sponsors and the
evaluands choices and opinions, and the What Can be Done, namely the designers do
this in which they are competent.

MESSAGE: What is the answer to the question stated, or what is the key message
of the paper that the author tries to send to the readers?
In the article we can see that by different types of reasoning are produced
different recommendations. Different recommendations are produced when design is
determined by the purpose of evaluation, when design is determined by the
characteristics of the evaluand and when design is determined by the problem to be
solved.
BENEFI T: What are the potentially positive benefits, that the author state herself,
of the key message of the text?
The evaluation literature is large, chaotic and confusing and it needs further
clarifications. Through the authors' research and her suggestions, evaluators can be
benefited in order to choose the most appropriate evaluation models.

2
CRITICAL ASSESSMENT

What are the strengths and the limitation of the paper that you as the student would
like to bring forward? (i.e. what do you think is clear and unclear with the papers
content?)

The article of Hanne Foss Hansen is quite important because it demonstrates
the problems in the existing evaluation literature. The author performs a good analysis
of the models and the criteria which must be set for the models choice, and the other
factors which affect the evaluation design. However, we have also noticed some
drawbacks. The first is that the writers reasoning rests on a theoretical level and has
not been implemented in practice. The same observation is mentioned by her in her
conclusion by saying that we do not have systematic knowledge in relation to how
important these logics and procedures are in practice. The second drawback of this
article is that, in some parts of it, the writer widens the theories more than needed and
it s difficult for the reader to follow her thoughts and stay focused on the main issue.
Therefore, the article could be more readable and keep the reader in a logical order. In
general, the text is quite interesting and helps the reader to understand the complexity
of selecting and designing an evaluation model based on the different needs.
































3
REFERENCES

Cameron K. (1986). Effectiveness as Paradox: Consensus and Conflict in Conceptions
of Organizational Effectiveness, Management Science, 32(5):539-553.
Hansen Hanne Foss (1999). Organisatorisk effektivitet, in Torben Beck Jrgensen
and Preben Melander (eds) Livet i offentlige organisationer, pp. 277-313.
Copenhagen: Jurist- og konomforbundets Forlag.

Hansen Hanne Foss (2003). Evaluering i staten. Frederiksberg: Samfundslitteratur.
Hansen Hanne Foss (2005). Choosing Evaluation Models: A Discussion on
Evaluation Design. Evaluation. 11(4):447-462.
March, James G. and Johan P. Olsen (1989). Rediscovering Institutions. New York:
Free Press.

Mintzberg, Henry (1983). Structure in Fives: Designing Effective Organizations.
Englewood Cliffs, NJ: Prentice-Hall.

Rossi, Peter H., Mark W. Lipsey and Howard E. Freeman (2004). Evaluation: A
Systematic Approach. Thousand Oaks, CA: SAGE.

Scriven, Michael (2003). Evaluation Theory and Metatheory, in Thomas Kellaghan,
Daniel L. Stufflebeam and Lori A. Wingate (eds) International Handbook of
Educational Evaluation. Dordrecht: Kluwer Academic Publishers.

Stufflebeam, Daniel L. (2000). Foundational Models for 21st Century Program
Evaluation, in Daniel D. Stufflebeam, George F. Madaus and Thomas Kellaghan
(eds) Evaluation Models: Viewpoints on Educational and Human Services
Evaluation, pp. 33-84. Boston, MA: Kluwer Academic Publishers.

Vedung, Evert (1997). Public Policy and Program Evaluation. New Brunswick, NJ:
Transaction Publishers.