Expert Evaluation (EE)

Tool within Governance

project

Dr. Ludger Deitmer,

Dr. Wolfgang Wittig
University of Bremen,
ITB
Germany 1
 EE tool application in four expert
workshops in Berlin, Copenhagen, Vienna
and Zurich in November 2007
• Tool consists of a questionnaire in which
seven main criteria are operationalised
by roughly 30 sub-criteria
• Respondents are asked to evaluate the
sub-criteria on a scale from 1 (=
criterion is not realised) to 10 (= fully
realised)
• In addition they are asked to assess the
relevance of the main criteria for the
adequate performance of the VET
system by weighting them in per cent.. 2
7 main criteria: five on integration
& two input and output

1. Consistent legal framework

2. Cooperation of the various bodies
3. Innovation strategies
4. Balance of relevant policy areas
5. Allocation of strategic and
operational functions
6. Outcome orientation
7. Input orientation
3
 Guiding principles for EE tool in
the expert workshops:
• Participative: active national VET system
experts: social partner, research and VET
practise etc.
• Discoursive: focus group discussion
• Multiple perspectives: individual and
collective assessment

4
 Rationale of the evaluation concept
• The method has to be understood as a “formative” or
in other words “qualitative evaluation” process.
• Those ones who are involved in Vet decision making
play the most important role.
• Experts with different backgrounds in national VET
systems express individually and jointly what the
actual governance situation in their VET system is.
• The function of the moderator is to implement the
self-evaluation process in a systematic way and to
moderate the discussion in such a way that all
persons get a voice and can express their wishes,
needs and assessments.
• Our experiences from former evaluation activities of
this kind show that the opinions are becoming clearer
during the process of their formulation and
discussion.
5
 Advantage/Challenges of
focus group instrument
Adavantages Challenges
• A topic can be • Can be hard to
discussed indepth analyse responses
• Common • Some dominating
understanding and partners can
complaints get on influence the result
the table: learning • Composition of
can be triggered discussion group is
• A complex topic can important
be assessed quickly • Diffcult to schedule 8
• Effective way of to 15 busy experts
getting a range of
information
• Good on
heterogenous groups 6