Using Tracer Surveys to Track

Impact of Training & Skills

Programs
Presentation at the worshop
“ Let’s Work - Pillar 2: Measuring the Impact of Private Sector Investments on Jobs”
London 16-17 September, 2015 (hosted by CDC)

By Harald Schomburg
Outline

1. Background – INCHER Kassel

2. Theoretical framework
3. Key aspects of the methodology of tracer
studies
4. Conclusions
Outline

1. Background – INCHER Kassel

2. Theoretical framework
3. Key aspects of the methodology of tracer
studies
4. Conclusions
 4 Background: International Tracer Study Cooperation
Harald Schomburg: Using Tracer Surveys to Track Impact of

Africa
Cameroon, Ghana, Kenya,
Malawi, Namibia, Nigeria, South
Africa, Uganda, Tanzania
Training & Skills Programs

Central America Asia

Costa Rica, El Salvadore,
Honduras, Guatemala, INCHER- Indonesia, Iran,
Mexico, Nicaragua, Japan, Kyrgyzstan,
Panama Kassel Oman, Philippines,
Vietnam

Europe
Austria, Belgium, Czech Republic, Estonia, Finland,
France, Italy, Netherlands, Norway, Portugal, Romania,
Serbia, Slovenia, Spain, Sweden,
11/09/2015 Switzerland, United Kingdom
 INCHER - University Kassel:
5
in the Heart of Germany, in the State of Hesse
Harald Schomburg: Using Tracer Surveys to Track Impact of

Located in the center of

Germany and Europe
Training & Skills Programs

Unique unity of urban

culture and landscape

City of the documenta

Parks and recreational areas

Center for industry, services

and civil administration
of nothern Hesse

11/09/2015
Outline
1. Background – INCHER Kassel
2. Theoretical framework
3. Key aspects of the methodology of tracer
studies
4. Conclusions
 7 What are (Graduate) Tracer Studies in Education?
Harald Schomburg: Using Tracer Surveys to Track Impact of

Similar terms
Graduate survey
Tracer study
Training & Skills Programs

Alumni research
Follow-up study
Graduate career tracking
Target population
Graduates of an institution of VET/HE (regardless the kind of
degree)
Surveyed after graduation

11/09/2015
 8 High Demand on Tracer Studies
Harald Schomburg: Using Tracer Surveys to Track Impact of

Tracer Studies are very often a key instrument for

monitoring and evaluation in projects of donor
agencies (like ADB, GIZ, JICA) in the VET and HE
Training & Skills Programs

area
Accreditation and accountability
Labor market relevance as a key indicator of quality
Assessment and further development of
study/traing programmes based on LM feedbacks

11/09/2015
 9 Objectives of Institutional VET/HE Tracer Studies
1. Development of the 2. To evaluate
Harald Schomburg: Using Tracer Surveys to Track Impact of

VET/HE institution the relevance of

(e.g. curriculum) VET/HE (assessment)
Training & Skills Programs

3. To contribute to the 4. To inform students,

accreditation process parents, lecturers and
(accountability) administrators
11/09/2015
 10 The Conceptual Framework of Tracer Studies
Tracer study
Harald Schomburg: Using Tracer Surveys to Track Impact of

VET/HE Input
Study conditions Labor market
and provisions
Training & Skills Programs

signals

Student Input
Students‘ bio
Output Outcomes
Kowledge, skills, Transition,
data, experiences,
competences, employment, work,
motives
motivation, professional success
grades service to society
Process
Teaching and learning

Context: Labour market – region – country; socio-economic

11/09/2015 development and personal development
 Socio-biographic background
Parents‘ Prior
11 education Gender Motives and abilities education
Harald Schomburg: Using Tracer Surveys to Track Impact of

VET/higher education
Study
Structures Study conditions Curricula behaviour
Training & Skills Programs

Socio- Competences
cultural
conditions
Labour
Transition Employment market
Country
process and work conditions
Region

Knowledge Globalisation, Life

11/09/2015
society internationalisation course
 12 Key Theoretical Problems of Graduate Surveys
Harald Schomburg: Using Tracer Surveys to Track Impact of

1. Which indicators do we need?

2. How to measure the labour market signals?
Training & Skills Programs

3. Which factors are relevant to explain the success of

the graduates besides education?
4. To what extent is the professional success/career
caused by VET/HE?
5. Which elements of VET/HE (study conditions and
provisions, etc) have an impact?
11/09/2015
 Common Indicators of Professional Success – Labor
13
Market Signals – Employment Outcomes
Objective indicators
Harald Schomburg: Using Tracer Surveys to Track Impact of

Short search duration

Employed or further study
High income
Appropriate position regarding level of education Professional
success:
Training & Skills Programs

(job title; ISCO)

Work tasks closely related to field of study
(description of work tasks)
A multi-
Subjective indicators
Perceived links of kind of job and own level of
dimensional
education concept
Perceived high use of competences
Perceived high status (income, career perspectives)
Perceived high work autonomy
Perceived high job satisfaction

11/09/2015
Outline

1. Background – INCHER Kassel

2. Theoretical framework
3. Key aspects of the methodology of tracer
studies
4. Conclusions
 15 Traditional and New Type of Graduate Surveys
Harald Schomburg: Using Tracer Surveys to Track Impact of

Traditional type – representative sample survey to

inform the Ministry and other key stakeholders
Since more than 30 years
E.g. France, Germany, Italy, Norway, Switzerland
Training & Skills Programs

Irrelevant for individual institution

New type – institutional graduate surveys
(all graduates from one institution of VET/HE)
(France), Germany, Hungary, Italy, Indonesia, the Netherlands,
Romania
Combination of a National Monitor and a feedback
instrument for VET/HE institutions
Relevant for individual institutions (reports for individual
institutions, study program level breakdown of results)
11/09/2015
 The Old Type of (National) Tracer Studies:
16
the School/University Delivers Data (e.g. Addresses)
Harald Schomburg: Using Tracer Surveys to Track Impact of

Consultants; Ministry of HE:

Research institute Donor agency
Researcher
Researcher
Training & Skills Programs

Researcher
Data

Data

Data
University A University B University C

11/09/2015
 The New Type of Graduate Surveys:
17
the Network Approach (KOAB)
Harald Schomburg: Using Tracer Surveys to Track Impact of

2012: About 80
Uni 1
institutions of HE
in Germany are Uni
Uni 2
Training & Skills Programs

80
cooperating in
conducting regular
tracer studies INCHER-
Uni … Kassel Uni 3

Uni 6 Uni 4

Uni 5

11/09/2015
 The Bridge Concept – Institutional
18
Development and Research
Harald Schomburg: Using Tracer Surveys to Track Impact of

Quality Vertical match –

assurance overeduction/
(accreditation) undereducation
Training & Skills Programs

Skills
Alumni and mismatch -
career Institutional study and
service Tracer Study work

Evaluation, Study
conditions and
information, professional
marketing success

11/09/2015
Institutional Development Research
 19 The New Approach of Tracer Studies
Harald Schomburg: Using Tracer Surveys to Track Impact of

Long term perspective: implementation of regular tracer

studies
Capacity buildung
Training & Skills Programs

• Tracer Studies should be organized as a learning process

• Training and cooperation (network) is needed
High quality of data (standardisation of processes)
International standards (design, use of instruments,
comparison of results)

11/09/2015
 20 Key Issues of the KOAB Tracer Studies
Harald Schomburg: Using Tracer Surveys to Track Impact of

Every university has an individual questionnaire

(2009: 160 surveys were administrated at the same
Training & Skills Programs

time by INCHER-Kassel)
Every university has a project coordinator
(= researcher)
At least 6 training workshops per survey + national
and international conferences
Response rate: 50 %
11/09/2015
 The New Tracer Study Network Approach:
21
Capacity Building for a Learning Organisation
Harald Schomburg: Using Tracer Surveys to Track Impact of

Staff
training

Field
Training & Skills Programs

Concept and
planning A cycle of phase/data
12 – 24 collection
months
Staff Staff
training training

Data analysis
and
interpretation
11/09/2015
 22 Multifunctional Tracer Surveys – the Netherlands
Harald Schomburg: Using Tracer Surveys to Track Impact of

Quality assurance (and increasingly accreditation)

Alumni-tracking
Training & Skills Programs

National monitoring of transition from higher

education to work
Study choice information for school-leavers (both
at national and institution level)
Input for labour market forecasting
Scientific and policy analyses

11/09/2015
Source: Jim Allen (ROA, University of Maastricht, Netherlands), Kassel 2008
 23 Multifunctional Tracer Surveys - Switzerland
Harald Schomburg: Using Tracer Surveys to Track Impact of

Information for political authorities

(education/employment sectors)
Training & Skills Programs

Benchmarking (system of indicators of Swiss higher

education institutes)
Evaluation and information tool for higher education
institutes
Information for career/study counselling services

Source: Andrea Witmer (Swiss Federal Statistical Office), Kassel 2008

11/09/2015
 24 Design of Graduate Surveys
Harald Schomburg: Using Tracer Surveys to Track Impact of

Student survey Graduate survey

Seldom Most frequent Less frequent
Training & Skills Programs

Exit Survey Graduate Graduate

Survey I Survey II

1- 2 years 4 - 5 years

Time After Graduation

11/09/2015
 University of Indonesia:
25
High Response Rate – High Use of the Results
Harald Schomburg: Using Tracer Surveys to Track Impact of
Training & Skills Programs

Source: Fikawati/Syafiq: Sustaining Tracer Study at University Level: Universitas Indonesia Experience, EXLIMA Conference Bali, 2012
11/09/2015
 Harald Schomburg: Using Tracer Surveys to Track Impact of
Training & Skills Programs
26

11/09/2015
University of Indonesia Tracer Study
 Harald Schomburg: Using Tracer Surveys to Track Impact of
Training & Skills Programs
27

11/09/2015
Successful Methodology
Low Budget Example Guatemala (UNITRACE):
Outline
1. Background – INCHER Kassel
2. Theoretical framework
3. Key aspects of the methodology of tracer
studies
4. Conclusions
 29 Low Quality of Many Tracer Studies
Harald Schomburg: Using Tracer Surveys to Track Impact of

Tracer Studies are very often done by local consultants

with little experiences
High costs and low quality
Training & Skills Programs

low response rate

misinterpretation of findings
Local consultants are often not following international
standards – e.g. they do not compare their study with
others
Only a few studies are published

11/09/2015
 Lessons Learned from Tracer Studies
30
in Different Countries
Harald Schomburg: Using Tracer Surveys to Track Impact of

High response rates are possible to achieve in almost

all countries
High quality of the data is a pre-requisite of the
Training & Skills Programs

relevance of the findings for research and institutional

development
Availability and quality of addresses (methods used to
update addresses) seems not to be a real problem
Documentation of survey procedures and data editing
is necessary – culture of critique must be developed

11/09/2015
 31 Open Questions
Harald Schomburg: Using Tracer Surveys to Track Impact of

How to ensure comparability of Tracer Studies?

Who collects the data and is responsible for data quality?
How to build a system of Tracer Studies which is flexible and
Training & Skills Programs

relevant for the institutions/sectors and for researchers

(and the Ministry of Education/Employment) at the same
time?
How to exchange systematically the experiences to improve
the future Tracer Studies (building a learning culture)?

11/09/2015
 32
Harald Schomburg: Using Tracer Surveys to Track Impact of
Training & Skills Programs

Tracer Study Researchers from Africa, America, Asia and Europe

International Conference „Experiences with Link and Match in Higher

Education: Results of Tracer Studies Worldwide“ (Bali, 2012)
11/09/2015
 www.exlima.org

11/09/2015
 Thank you very much for your attention

Further information and tools:

Web site http://ingradnet.org

Harald Schomburg (2014): Carrying out tracer studies.

In: ETF/CEDEFOP/ILO (eds.): Guide to anticipating and matching skills and jobs.
Volume 6. Torino: ETF (in print).