Introduction to Evaluation Studies

Blaga Ioana-Iulia SD II

Romanian evaluation system versus Swiss evaluation system

Being a discipline with inchoation from social science research, Evaluation has its
roots of not being a unified practice or derived from a single set of tradition, but based on
scientific methods, dating from Ancient times. The sources of evaluation thinking during
modern times can be found in the 1960s Great Society initiatives in the United States; in the
educational innovation and curriculum innovation in schools; and in the budgetary control and
efficiency systems. To those, we could add management by objectives, participative research
in community and rural development, results based management and many more. The four
main set of ideas in Evaluation can be found in scientific research and methods, economic
theory and public choices, organization and management theory, political and administrative
science.
As any other discipline from social science, it has many understanding and definitions.
It is notable that most of them have a common aspect the involvement of the judgement of
the value of the subject of the evaluation. According to the Organisation for Economic Cooperation and Development, Evaluation refers to the process of determining the worth or
significance of an activity, policy, or program. [It is] as systematic and objective as possible,
of a planned, on-going, or completed intervention.1
Even if there are different views a about what the purpose or goal of evaluation should
be in a given context, most of them imply that it has four distinct purposes: the ethical
purpose to report to political leaders and citizens on how a policy or program has been
implemented and what results have been achieved (it combines the objectives of using better
accountability, processing information, and serving of democracy; the managerial purpose
to achieve a more rational distribution of financial and human resources among competing
programs, improve program management, and increase program benefits; the decisional
purpose to pave the way for decisions on the continuation, termination, or reshaping of a
policy or program; educational and motivational purpose to help educate and motivate
1 Development Assistance Committee DAC Glossary, OECD 2000, p. 21;

Introduction to Evaluation Studies

Blaga Ioana-Iulia SD II

public agencies and their partners by enabling them to understand the processes in which they
are engaged and to identify themselves with their objectives.2
Prominent evaluators in the field describe the following purposes of evaluation: obtain social
betterment; promote the fostering of deliberative democracy; provide oversight and
compliance; ensure accountability and transparency; build, share, and manage knowledge;
contribute to organizational improvement; promote dialogue and cooperation among key
stakeholders; determine project, program, or policy relevance, implementation, efficiency,
effectiveness, impact, and sustainability; generate lessons learned.
Ultimately, the purpose of any evaluation is to provide information to decision makers
to enable them to make better decisions about projects, programs, or policies. Evaluation
should help decision makers understand what is likely to happen, is happening, or has
happened because of an intervention and identify ways to obtain more of the desired benefits.
The contribution of evaluation is potentially greatest in innovative policy areas where
achieving success cannot be taken for granted and where implementation is not always
straight forward. There is a need for sophisticated management and planning. When properly
applied, evaluation can help make manageable some of the unavoidable uncertainties of
complex situations. Socio-economic development is certainly complex and often faces many
uncertainties: it is not a precise science. Choosing goals and measures, designing programmes
and policies, implementing and sustaining a development dynamic, all require analysis,
anticipation, establishing feedback systems and mobilizing different institutions, agencies and
population groups.
It is because evaluation know-how and practice has been shown to make a contribution
to these processes that it has become such a key component in so many socio-economic
development initiatives.
For my final assignment, I chose to compare to very different countries, Romania and
Switzerland. They are, if I may say, on opposite places on any scale that assesses almost any
indicators. Switzerland is and always was at the top of the most ranking lists in the world,
while Romania is merely there, and when it is, it`s situated and the bottom of the list.
Therefore, I tried to compare these two very different systems and see how their evaluation
works.
2 Scientific and National Councils of Evaluation 1999;

Introduction to Evaluation Studies

Blaga Ioana-Iulia SD II

The comparison between the two systems will analyse the strategies of the two
countries, the institutions involved, how are the institutions inter-related, how did the
evaluation theory come into practice, evaluation programs, how the public policy is made,
related to evaluation, compared to other public policies.
Romania has a quite new evaluation culture, and it only began to evaluate its system
after the submission its official application for membership in the EU in 1995. Even from the
beginning of the evaluation processes, the request to evaluate policies, strategies, systems and
projects in Romania, didn`t came from the inside, the state started to evaluate because of the
prospective to become an EU Member State and it was a requirement from the EU. Until
today, most of the evaluation requests come from outside manly because of our
memberships in NATO, EU, World Bank, UN; also the experts and analysis come from
external sources.. In order to be a valuable member and partner of the most important
organizations in the world, Romania had to have the ability to evaluate and to report.
But, even if it needed to have an evaluation culture, national strategy, public policies,
evaluation institutions and experts doesn`t necessarily imply the fact that Romania has all of
them or that it is efficient in evaluating. In an Analysis of the evaluation culture in Romania
made in 2006, Hilary Curley and Eugen Perianu state that the appearance the practice of
evaluation was unlikely so far. Probably evaluation practice requires stable and consistent
processes and procedures established in the public administration mechanism, which have
little chance of success in a scanty and unstable (yet) institutional environment.3
Romania made its first National Evaluation Strategy in November 2006. It was
designed by Ministry of Public Finance Managing Authority for Community Support
Framework Central Evaluation Unit in the same context, of the candidacy for EU
membership. To have a coherent strategy, an analysis of the evaluation culture in Romania
was conducted in June 2006 by two short-term experts. Terms of Reference for this study
included: analysis of the current and potential evaluation culture in Romania. The analysis
was on the evaluation culture 9 + 3 (adopted by the Evaluation Advisory Group, established
in 2003, which included experts from the European Commission, EU Member States and
Candidate) criteria to assess the extent to which there is a culture of evaluation
3 Analiza culturii de evaluare din Romnia, Hilary Curley (Curley Consulting),

and it

Eugen Perianu (Consilium), 25 iulie 2006

4 Furebo, Rist, and Sandahl, International Atlas of Evaluation, Transaction Publishers, New Brunswick, New Jersey, 2002, USA.

Introduction to Evaluation Studies

Blaga Ioana-Iulia SD II

presented a picture of the desideratum evaluation culture in Romania, looking at possible

ways to move from the current situation as desired. Stated from the beginning, the evaluation
culture was absent at that time, but now, it is on its initiation path, being still young and
unskilled in evaluation or having an evaluation culture.
Romania`s next National Evaluation Strategy had as overall objective to achieve a
functioning national evaluation system, the parties mutually reinforcing, including public and
private sector and civil society to contribute to the effective management of public
interventions and accountability promoters policy and civil servants. It will recognize the
importance of evidence-based policy making.5
The lack of culture and capacity in program monitoring and evaluation involves the
lack of tradition to assess performance in the public sphere. Romania has been characterized
by an increased dynamic of legislative change during the past 20 years. But, willingly or not,
the regulations concerning Program Evaluation field has been left aside. The Romanian
National Evaluation Strategy is an important step forward in acknowledging the importance
of Program Evaluation in the decision-making process. But further steps are required in order
to turn strategic thinking into reality. Internationally, Results-Based Management and Results
Based Reporting are presently on focus. Successful reform of public administration
necessarily involves the evaluation of programs and performance. There has been no regime
able to manage its fiscal resources effectively if its programs and its performance were not
constantly measured, evaluated and improved.
When it comes to evaluation and public policies in Romania, there is no coherent
system, a clear line. Mostly because of the decision of not organizing a Program Evaluation
System at country level government and has many negative implications as far as the
decision-making process is concerned. The lack of political responsiveness, fiscal discipline
and institutional effectiveness are part of the effects. The government does not require a
coherent, solid evaluation system and, in exchange, it gets Bleak House- type reports.
Program evaluation offers the adequate tools to do evidence-based decision-making on public
policy priorities and public resource allocation.
The evaluation processes in Romania is mostly required by external actors and, almost
any Romanian institution, ministry, NGO, expert, committee, organization, enterprise,
authority, general secretariat, national direction, program direction, administration, unity,
5 National Evaluation Strategy 2007-13, Ministry Of Public Finance Managing Authority for Community Support Framework Central
Evaluation Unit

Introduction to Evaluation Studies

Blaga Ioana-Iulia SD II

administrative board, commission and so on and so forth, has the capacity to evaluate,
regardless of their domain.
The formative, summative, or prospective evaluation is in general for programs that
involve huge amounts of money coming from international organizations, and their goal is to
see how Romania manages to be efficient with that total. For example, the first evaluations in
Romania were on the EU-PHARE, ISPA, POSDRU, SAPARD Programs. Romania doesn`t
only evaluate EU programs, but anything beginning with the human performance and
finishing with state`s performance. Examples of institutions that perform evaluations in
Romania: Central Evaluation Unit (nowadays ACIS) part of Ministry of Finance, Evaluation
Unit of Directorate General for Administrative Capacity Development, Ministry of Interior
and Administrative Reform, National Authority for Scientific Research is the state authority
for research and development in its relations with third parties and is charged with monitoring
and evaluation process of validating the results, Advisory Board for Research and
Development and Innovation, Monitoring Committee, Romanian Unit of the Directorate
General for Regional Policy, Evaluation Unit of DG Regio / DG Employment, other units of
the MA, the competent authority for the accreditation of paying agencies and the
Coordinating Body of the Ministry of Agriculture and Rural Development, Internal Audit
Department of the Ministry of Culture. Even if there is a big range of domains and activities,
the insitutions are inter-ralated and they cooperate on different matters, because some of them
are part of the same governmental bodies and it is mandatory to do so.
Being mostly required by external actors, they are those who perform most of the
evaluations about and within Romania. For example, EU, World Bank, International
Monetary Fund and NATO evaluate Romania and its performance periodically.
Therefore, Romania doesn`t have a clear evaluation structure, evaluation culture, a
public policy or national program regarding evaluation, but it has a National Evaluation
Strategy which has been evaluated and re-evaluated, but still not practically and correctly
applied. It has quite many (too many) institutions that evaluate anything because they are
forced to do so, creating a big chaos and confusion in measuring performance within the
country. Because of the lack of a clear set of rules and measuring tools, the best evaluation
and monitoring processes come from outside, from those international bodies that can`t afford
to see Romania underperforming or not being at the desired level.

Introduction to Evaluation Studies

Blaga Ioana-Iulia SD II

Switzerland, on the other hand, is on the opposite side of the scale, right on top of it.
In a comparative cross-country analysis of evaluation culture and the institutionalization of
evaluation, Switzerland ranks second in new Atlas of Evaluation 2015. The countries
included in this research are the 19 OECD countries examined by the authors of the
International Atlas of Evaluation 10 years ago. 6The analysis is based on the results of an
expert survey of four to five evaluation experts from different backgrounds for each country,
as well as additional information from the literature. Using the nine indicators from Furubo et
al. (2002) with a focus on the institutional characteristics of reforms, trends in evaluation
culture over the last decade have been identified..7
Switzerland is the first country in the world to have an evaluation clause at the
constitutional level. There are two main actors in evaluation domain in Switzerland, i.e.
the Parliamentary Control of the Administration and the Swiss Federal Audit Office. Swiss
Federal Administration spends annually approximately 15 million Swiss francs on
evaluation. In spite of that, the evaluation supply is still more developed than evaluation
demand due to the peculiarities of the Swiss political system, i.e. federalism and semidemocracy. The quality of evaluation is high and the process for the handling of
evaluations are structured in a sensible and appropriate manner; however, yet evaluation
findings and recommendations are used only rarely in the decision-making process, and
when they are used, it is then often as a means of legitimation.
But, it isn`t a country with a long tradition in evaluation culture. The origins of
evaluation activity in Switzerland lie on one hand in the national research program The
Effectiveness of Public Measures. It was funded by the Swiss National Science Foundation,
the national body for the promotion of scientific research in Switzerland. This programme
dealt with the overall conditions for government programs, in terms of organization and
interests, between 1976 and 1986. In view of the increasing difficulties involved in executing
the law, the problem of the implementation of legislationwhich had undergone virtually no
systematic examination in Switzerland until then emerged as a focal point. On the other
hand, the roots of evaluation in Switzerland can be traced back to the activities of a Working
Group on Legislative Evaluation (AGEVAL), which was set up in 1987 by the Federal
6 Furubo et al., 2002
7 Steve Jacob from Laval University, Canada; Sandra - Speer Independent evaluator, Germany ; Jan-Eric Furubo National Audit Office,
Sweden, The institutionalization of evaluation matters: Updating the International Atlas of Evaluation 10 years later

Introduction to Evaluation Studies

Blaga Ioana-Iulia SD II

Department of Justice and Police (Federal Office of Justice) and was active by 1991.
AGEVAL was composed of federal and cantonal officials as well as academic scholars, and it
promoted evaluation through a series of commissioned evaluation studies. As part of a second
national research program, these activities were paralleled by a systematic scientific
examination of the subject of policy evaluation. One important consequence of this national
research program was the establishment of the Swiss Evaluation Society and the network
Evaluation of Federal Administration.
Switzerland has an active and dense network of evaluators and circles interested in
evaluation in the civil service and in academia. It is achieved by well-functioning evaluation
association and network. Besides SEVAL, there is also the Geneva Evaluation Network
(GEN) , which was established in 2006 as an informal network by Mr. Craig Russon. It
consists of 95 members and aims to strengthen the individual evaluation capability of its
members through informal presentations/seminars and enhance networking among the local
evaluation community in the Geneva area.
There are two main institutions in evaluation domain in Switzerland, i.e. the
Parliamentary Control of the Administration and the Swiss Federal Audit Office.
In Switzerland evaluation has developed steadily since 1980th. Swiss NEP is focused
on evaluation of public policy effectiveness. Evaluation function is laid down in the
Constitution. Evaluation has been integrated into the planning and management instruments at
the federal level. Parliament plays an important role in evaluation, which corresponds to its
lawmaking function. The mandate of carrying out evaluation relating to the results and
performance of public policies and services is given to the Parliamentary Control of the
Administration. Meanwhile, the Competence Centre Performance Audit and Evaluation of
the Swiss Federal Audit Office undertakes assessments of the implementation and
effectiveness of federal policies that have significant financial implications, and then to
recommend changes for enhancing their efficiency. Federalism and direct democracy are two
main factors of the Swiss political system that influence the policy evaluation in the country.
Evaluation activities in the federal administration are very varied. Switzerland has an active
and dense network of evaluators and circles interested in evaluation in the civil service and in
academia. Swiss Federal Administration invests into evaluation around 15 million Swiss
francs per year. Yet evaluation findings and recommendations are used only rarely in the
decision-making process, and when they are used, it is then often as a means of legitimation.

Introduction to Evaluation Studies

Blaga Ioana-Iulia SD II

Federal evaluation activities are strongly related to power games: evaluation often is seen as
a way to gain power and influence within the administration. Because of this, evaluation coordination has to be considered as very weak. Prospective planning of evaluation activities
synchronized with other administrative units does not usually occur.
In conclusion, Romania and Switzerland lie on the opposite side of the evaluation
scale, the first one being close to the bottom, while the last one is on 2nd place in the world.
Bibliography

OECD 2000;

Scientific and National Councils of Evaluation 1999;

Analiza culturii de evaluare din Romnia, Hilary Curley (Curley Consulting),
(Consilium), 25 iulie 2006

National Evaluation Strategy 2007-13, Ministry Of Public Finance Managing Authority for Community
Support Framework Central Evaluation Unit
National Evaluation Strategy 2006, Ministry Of Public Finance Managing Authority for Community
Support Framework Central Evaluation Unit
Thomas Widmer, Charles Landert and Nicole Bachmann, Evaluation Standards of SEVAL, The Swiss
Evaluation Society , 5 December 2000
Thomas Widmer and Peter Neuenschwander,
Embedding Evaluation in the Swiss Federal
Administration Purpose, Institutional Design and Utilization
http://www.evaluare-structurale.ro/

http://www.seval.ch/de/index.cfm

Eugen Perianu