Problems of Economic Transition

ISSN: 1061-1991 (Print) 1557-931X (Online) Journal homepage: http://www.tandfonline.com/loi/mpet20

Are Retraining Programs a Panacea for the Russian

Labor Market?
I. Denisova & M. Kartseva
To cite this article: I. Denisova & M. Kartseva (2010) Are Retraining Programs a Panacea for the
Russian Labor Market?, Problems of Economic Transition, 53:1, 5-16
To link to this article: http://dx.doi.org/10.2753/PET1061-1991530101

Published online: 08 Dec 2014.

Submit your article to this journal

Article views: 3

View related articles

Full Terms & Conditions of access and use can be found at

http://www.tandfonline.com/action/journalInformation?journalCode=mpet20
Download by: [Library Services City University London]

Date: 17 April 2016, At: 17:41

Downloaded by [Library Services City University London] at 17:41 17 April 2016

Problems of Economic Transition, vol. 53, no. 1, May 2010, pp. 516.
2010 M.E. Sharpe, Inc. All rights reserved.
ISSN 10611991/2010 $9.50 + 0.00.
DOI 10.2753/PET1061-1991530101

I. Denisova and M. Kartseva

Are Retraining Programs a Panacea

for the Russian Labor Market?
The article analyzes retraining programs provided by public employment
offices in Russia with an emphasis on their potential role during economic crisis. Net impacts of the programs are estimated using a rigorous
quasi-experimental evaluation technique. It was found that the impact of
retraining the unemployed is statistically insignificant both for wages and
employment. Neither a positive nor a negative overall effect of the programs
was found.

Russian society puts high hopes on programs for retraining the unemployed as
a way to overcome the negative consequences of the global economic crisis.
These programs are at the top of the list of measures announced by officials
of the Ministry of Health and Social Development to counteract outright unemployment. Moreover, increased funding for active employment programs
is planned. to what extent are these hopes justified?
Active employment programs are known as a tool for reducing unemployment. Furthermore, the overall system for retraining personnel certainly
includes programs that are carried out as part of the government employment services projects. At the same time, the appropriateness of widespread
English translation 2010 M.E. Sharpe, Inc., from the Russian text 2009 Voprosy
ekonomiki. Programmy pereobucheniiapanatseia dlia russkogo rynka truda?
Voprosy ekonomiki, 2009, no. 4, pp. 10412. A publication of the NP Editorial
Board of Voprosy ekonomiki and the Institute of Economics, Russian Academy of
Sciences.
I. Denisova is lead research fellow at the Center for Economic and Financial Research and Development and senior researcher at Central Mathematical Economics
Institute. M. Kartseva is a research fellow at the Center for Economic and Financial
Research and Development.
Translated by James E. Walker.
5

Downloaded by [Library Services City University London] at 17:41 17 April 2016

6 PROBLEMS OF ECONOMIC TRANSITION

use of programs for retraining the unemployed during an economic crisis is

questionable.
First, these programs are potentially effective in combating structural
unemployment, while what we see in a decline in overall demand, that is,
growth of cyclical unemployment in the current stage of development of the
crisis. The economic crisis will obviously lead to a change in the structure
of the economy and, in particular, the structure of demand for labor, which
will require the considerable sectoral and regional redistribution of human
resources. Retraining programs will play an important role in counteracting
structural unemployment. In the active phase of crisis development, however, it is not possible to predict in detail the scale and direction of structural
changes, so we cannot know what occupations will be in greatest demand in
the postcrisis labor market.
Second, the current organization of programs for retraining the unemployed
in Russia is questionable in terms of their effectiveness, which requires a
detailed analysis of the procedures for selection and self-selection of the
programs participants, choice of the proposed directions of retraining, and
other program procedures. In most developed countries, evaluation of the
effectiveness of government programs is considered the most important part
of the mechanism for ensuring efficient resource allocation. The importance
of such research is increasingly recognized in transition economies.1 Little
is known, however, about the effectiveness of active employment programs
in Russia,2 and even less is known about the effectiveness of retraining programs. In fact, such analysis requires special data and inevitably involves
the collection of additional information, which requires expenditures of time
and money.
This study is one of the first attempts to evaluate the effectiveness of programs for retraining the unemployed.3 To reveal the effect of these programs,
we surveyed formerly unemployed people, some of whom participated in
them, in two Russian regions.
Study methodology
Correctly determining the effect of participation in a program requires the
comparison of two potential outcomes: with and without the participation of
subject i in the program (measuring the effect, for example, by the potential
earnings of i in both cases). The potential outcome implies the possibility of
events opposite to those that actually occur. For example, for an individual
who did not participate in the program, a potential outcome is his potential
earnings after taking the training course, Y1i, that is, if this individual had been
a participant in the program. Similarly, for an individual who did participate

Downloaded by [Library Services City University London] at 17:41 17 April 2016

May 2010 7

in the program, a potential outcome is the wages, Y0i, that this individual
would have received if he had not been a participant in the program. As a
result, we cannot observe both potential outcomes, Y1i and Y0i, for any specific
individual. We can only observe the realized outcome Yi. This gives rise to
the so-called fundamental problem of identifying the effect of a program:
we cannot observe both values Y0i and Y1i for the same subject, and we cannot calculate the individual effects Y1i Y0i. If the subjects (individuals) are
homogeneous, this problem can be solved by comparing the outcomes for
those who did and did not participate in the programs (or the outcomes for
the same individual who first took part in the program and then dropped out
of it). Unfortunately, there is every reason to believe that the consequences
of participation (or nonparticipation) in programs varies for different people,
and all that can be calculated, at best, is some average effect of participation
in a program. The basic approach here is to select an adequate replacement
for one of the potential outcomes.
If the participants in a program and the control group differ only in observable characteristics, we can control for such differences. In this case, the
strategy is to find a control group that would be maximally comparable with
the group of participants, at least with respect to observable parameters. Such
an approach is called the selection of observable characteristics, in the course
of which the evaluations are substantiated by matching. This methodology is
based on collecting as many covariants (variables) as possible and adjusting
the calculated differences by regression or matching (correspondence). We
apply the correspondence method, using the propensity score matching approach to reveal the average overall and average group (with respect to age
groups, categories of education, place of residence, and status in the labor
market during the previous period) effect of participation in a retraining program. In the course of selecting control groups by scores, we evaluated the
probability that an agent would be sent to a retraining program depending on
the individuals age, education, state of health, job history prior to termination, and region of residence (logit regression). Using coefficients computed
as a result of estimating the logit model, we could derive predicted values
of the probability (propensity score) of getting into the program. Thus, the
propensity score reflects all of the observable characteristics determining the
probability of being selected for the retraining program. Using the propensity
scores that were derived, we selected a control group for the participants in
the retraining program.
One of the most important stages in evaluating programs is the determination
of their outcomes: the indicators according to which their effectiveness will
subsequently be determined. In our study, we used two classes of such indicators. The first class is evaluation of the programs quality and effectiveness

8 PROBLEMS OF ECONOMIC TRANSITION

Downloaded by [Library Services City University London] at 17:41 17 April 2016

given by the respondents themselves (self-evaluation of effectiveness). The

second is economic indicators of the programs effectiveness constructed on
the basis of information about the respondents job history after participation
in the program.
Data and description of the sample
The empirical foundation of the study is a survey of clients of employment
services in two Russian regions,4 which was conducted in FebruaryMarch
2004. The sample was composed on the basis of administrative data from
2002, which were provided by the regional employment centers. As the general population we took clients of the employment service who registered
there no later than January 2001 and were taken off the register no later than
December 2002. The general population included only those clients of the
employment service who were given unemployed status.
The procedure for constructing the sample was the same in both regions.
The target composition was assigned as follows: 20 percent of the sample
consists of clients of the employment centers who had found a job within
ten days after applying to the center; 20 percentclients of the employment
centers who did not find a job within ten days after applying and participated
in a retraining program; 10 percentthe same clients of the employment
centers who, in addition, participated in public work; 10 percentclients of
the employment centers who, instead of this, participated in various social
adaptation programs; and 40 percentclients of the employment centers
who, not finding a job within ten days after applying, did not participate in
any active employment program (other than vocational guidance).
The target size of the sample was 1,200 people in the first region and 1,000
in the second. According to the results of conducting the survey, the sample
size was 1,277 and 966, respectively.5
The questionnaire contained questions on employment status and stability,
earnings of former clients of the employment center in the year after being
removed from the register of the unemployed (in our case, 2003), and also
on current employment (at the time of the survey), and the respondents
subjective evaluation of the quality and usefulness of the retraining program.
In addition, we asked the respondents about their employment in the period
preceding their registration at the employment center and collected basic
sociodemographic information about them.
Proper selection of the control group requires understanding of the procedures by which participants in the program are chosen. These procedures are
not specified anywhere; the only rule is that registered unemployed people
have the right to priority participation in the program. Nevertheless, there is

Downloaded by [Library Services City University London] at 17:41 17 April 2016

May 2010 9

reason to suppose that there are informal mechanisms for selection to participate in the program. In particular, additional conditions for participation were
imposed in order to improve the job-placement numbers. In some cases, to
participate in the retraining program, the unemployed person had to bring
a letter from a future employer (guaranteeing that they would be hired
after undergoing retraining). However, since the rules for admission to the
program were not formalized, and the content and prevalence of informal
procedures were not studied, we included in the questionnaire a number of
questions enabling us to reveal details of the informal selection procedure
(if there was one).6 Moreover, the respondents were asked questions about
the content of the program itself, how long it lasted, and also about their
occupation, and the kind of jobs they had before and after going through
the program.
Most often, the programs taught the participants how to work on a personal
computer, accounting, automated accounting (the 1C Accounting program),
as well as trades and occupations in the service sector. The next most popular
were the secretary/office manager and manager courses. The average length
of training varies from one month to half a year, depending on the occupation
for which the unemployed person is being trained.
If the respondents occupation before undergoing retraining is compared
with the occupation for which he or she was retrained, it turns out that those
who were trained in accounting, as well as trades and occupations in the
service sector, did improve their skills, for the most part. At the same time,
the overwhelming majority of the participants in retraining learned an occupation that had nothing in common with what they did at their last job (or,
for unemployed people without work experience, with the occupation for
which they received a diploma). For the most part, those who were trained
to be secretaries or office managers had no previous work experience (they
were mostly high school graduates). Those who studied other occupations
were more evenly distributed by types of work history before entering the
program.
Participants, nonparticipants, and control group
The majority of participants in the retraining programs were women: 75
percent in the first region and 86 percent in the second. Interestingly, 80 percent of the programs participants live in cities. Their average age in the first
region was thirty-four and in the second, thirty-two. The educational level of
the participants in retraining was similar in both regions: 44 percent of the
participants had a higher education, 25 percent had a secondary vocational
education, 20 percent had a primary vocational education, and 10 percent

Downloaded by [Library Services City University London] at 17:41 17 April 2016

10 PROBLEMS OF ECONOMIC TRANSITION

had a general secondary education. The proportion of the disabled was 10

percent in the first region and 4 percent in the second. More than 60 percent
of the participants had work experience, the average length of which was
thirteen years.
The characteristics of participants in the retraining programs and those
who did not participate differed. For example, in the first region the average
age of participants in the program was 34.4 years, while the average age of
those who did not participate was 38.7. The proportion of women was higher
among the program participants than among nonparticipants in both regions.
The educational level of participants and nonparticipants was approximately
the same: among the former the proportion of people with a higher education
was slightly higher, which is compensated by the lower proportion of those
who have a general secondary education.
Comparative analysis of work history in the first region showed that the
share of those who did not have work experience was significantly higher
among participants in the retraining programs than among nonparticipants.
The share of long-term unemployed was higher among participants in the
retraining programs in the first region. In the second region, the breakdown
by work history was similar for the two groups.
Thus, the two groups were very close with respect to some parameters, but
significantly different with respect to a number of others. Therefore, in order to
analyze the effectiveness of the retraining programs it was necessary to create
from the group of nonparticipants a control group that would be closer in its
characteristics to the participants than the whole group of nonparticipants was.
Regression analysis had to be used for statistical control of other differences
between the control group and the group of participants when measuring the
programs effectiveness.
The procedure for matching using a propensity score consists in selecting,
for each participant in the retraining program, nonparticipants for whom the
predicted probability of participation in the program was not very different
(i.e., within an assigned interval) from the corresponding probability for the
given individual. The same nonparticipant could be a match for more than one
participant. In this case, such an individual is given a weight in the control
group equal to the number of participants that he or she matches. The number of nonparticipants selected for each participant using this method varied
from nine to twelve. The list of variables taken into account in predicting
the probability of participation in the program and, accordingly, in selecting
nonparticipants for the control group included gender, age (up to thirty, thirty
to forty-five, and older than forty-five), level of education, place of residence
(urban or rural area), situation in the labor market before registering at the
employment center (never worked, had not worked for a long time, laid off

Downloaded by [Library Services City University London] at 17:41 17 April 2016

May 2010 11

because of downsizing or liquidation of the firm, resigned), and state of health

(presence or absence of disability).
As a result, with respect to many characteristics, the control subgroup
that was constructed turned out to be closer to the group of participants in
the program than the group of nonparticipants was. For example, the gender
mix of the control group almost coincides with that of participants in the
program: among the participants in the first region, 25 percent were men;
in the second region, 14 percent; while in the control group the proportions were 28 percent in the first region and 9 percent in the second (the
analogous figures for the groups of nonparticipants were 43 percent and 38
percent for the first and second regions, respectively). With respect to the
age and educational breakdown, as well as the proportion of the disabled,
the control group was also significantly closer to the group of participants.
However, it should be noted that the work history of respondents from the
control group was significantly different from that of participants in the
retraining programs.
On the whole, we can conclude that, as a result of application of the propensity score matching method, a control group was created that was close to the
group of participants in the retraining program with respect to demographic
and certain other characteristics.
Evaluation of the effect of participation in the program
In the course of the interviews, the respondents were asked questions that enabled us to find out what they thought about the effectiveness of the retraining
program. An analysis of their answers allows us to conclude that participants
in retraining in both regions give the quality of the program fairly high marks.
The majority of the respondents thought it was excellent or good, and
less than 5 percent said that it was poor. The overwhelming majority of
participants in the programs noted that the skills and know-how they acquired
helped them find a job.
Using individual data obtained as a result of the survey, we evaluated the
impact of the retraining programs on the probability of job placement (i.e., the
probability of having a job at the time of the interview and the probability of
having had at least one episode of employment after retraining), the length of
episodes of unemployment and employment, and also the amount of earnings.
The results of evaluation of the overall effect are given in Table 1.
For each determination of outcome, we give two evaluations of the programs effect on the participants: without correction, obtained as the simple
difference in the average outcome between the group of participants and
the control group (the difference column), and a more accurate one, with

12 PROBLEMS OF ECONOMIC TRANSITION

Table 1

Downloaded by [Library Services City University London] at 17:41 17 April 2016

Evaluations of the Effect of the Retraining Program (%)

Program
participants
Region 1
Probability of having a job
at the time of the survey
Probability of having had
at least one episode of
employment after being
removed from the register
of the unemployed
Probability of high earnings
(more than 6,000 rubles)
Length of current period of
unemployment, months
Region 2
Probability of having a job at
the time of the survey
Probability of having had
at least one episode of
employment after being
removed from the register of
the unemployed
Probability of high earnings
(more than 6,000 rubles)
Length of current period of
unemployment, months

Control
group

Difference

Effect

68.6

69.7

1.1

0.2

84.1

85.5

1.4

0.6

5.4

4.8

0.6

0.6

16.7

18.5

1.8

1.3

72.0

72.0

0.0

2.0

89.0

87.3

1.7

3.0

6.0

3.7

2.3

1.8

15.4

16.8

1.4

0.2

Note: None of the effects are statistically significant.

correction for the remaining differences in observable variables, obtained

using regression analysis (the effect column). In both cases, we calculated
the standard students t-statistic and tested how much the evaluation of the
effect is statistically different from zero.
The results obtained indicate that retraining programs in Russia do not, on
the whole, have a statistically significant effect (either positive or negative)
on the employment or incomes of their participants in comparison with the
control group. This result is valid for both of the Russian regions.
The overall effect of the programs does, however, vary significantly
among subgroups of participants of different ages, education, and health

Downloaded by [Library Services City University London] at 17:41 17 April 2016

May 2010 13

status (Table 2). The evaluation of group effects of retraining in Russia

indicates that participation in the program by people older than forty-five
increases their chances in the labor market, while participation of young
peopleup to age twenty-fiveworsens their situation. People with general
secondary education also benefit from retraining, while people with primary
vocational education lose from participation in the program in comparison
with the control group. In one of the regions, retraining proved to be extremely useful for the disabled: participants benefit in comparison with the
control group, in terms of both the probability of employment and earnings.
Our analysis did not reveal any gender differences in the effectiveness of
retraining in Russia.
***
There are no general guidelines for making government programs effective.
The same program may prove to be effective in one country or region and
ineffective in another; it may help at one time and be useless at another. Moreover, the impact of government programs may be very different on various
subgroups of participants.
Our analysis indicates that, in their current form, retraining programs in
Russia are, on the one hand, not as effective as the employment service believes, but on the other hand, they are not as ineffective as many opponents
of the program think. For some categories of participants, retraining played
a positive role. Furthermore, retraining programs counteract structural unemployment, but they may be ineffective in the active phase of crisis development. In these conditions, it seems better to provide timely information on
available job openings and financial support for the unemployed.
Data collection and administrative evaluations of government programs
are expensive. However, the benefits obtained from evaluating the programs impact may exceed the costs involved, especially if analysis of the
overall impact is accompanied by evaluation of the effects for subgroups.
The experience of countries that have included evaluation of government
programs in their daily bureaucratic activity indicates that such studies
help politicians to improve the structural quality of the programs, and, in
many cases, to change their direction, thereby helping to increase social
and individual welfare.
It is important that such studies and their results be considered as a tool
for improving the quality of government services provided, rather than as a
source of arguments that the government service responsible for the programs
is incompetent. The inability to form a constructive view of an evaluation of
programs that is done leads to serious resistance to their evaluation and, in the

14 PROBLEMS OF ECONOMIC TRANSITION

Table 2
Evaluations of Group Effects of the Retraining Program

Downloaded by [Library Services City University London] at 17:41 17 April 2016

Effect

Group
Region 1
Age
Up to 30
3045
Older than 45
Gender
Male
Female
Education
General secondary
Primary vocational
Secondary vocational
Higher vocational
State of health
Disabled
Not disabled
Place of residence
City
Rural area
Region 2
Age
Up to 30
3045
Older than 45
Gender
Male
Female

Probability of
having had
at least one
episode of
employment
after being
Probability of removed from Probability of
Length of
having a job
the register high earnings current period
at the time of
of the
(more than
of unemploythe survey
unemployed 6,000 rubles) ment, months

10.3**
0.4
11.1**

1.7
0.1
0.7

1.9
1.0

1.6
0.3

27.5***
7.1
4.1
5.4^
6.8
1.0

5.6***
3.1**
3.1^

4.1
2.5
5.3

1.5
0.4

5.1
0.6

13.4***
0
1.5
3.6

0.1
3.1
3.5*
1.9

12.2*
1.5
0.4
2.3

17.7***
1.2

0.0
0.7

4.5
2.4

1.6
6.5

0.7
0.5

1.2
2.2

3.1
4.6

4.1
6.4
5.5

5.2
1.6
9.2^

2.4
0.2
4.9

0.3
4.1
8.2^

15.4*
4.5

0.9
3.1

3.8
1.5

7.7^
2.0

May 2010 15

Downloaded by [Library Services City University London] at 17:41 17 April 2016

Effect

Group
Education
General secondary
Primary vocational
Secondary vocational
Higher vocational
State of health
Disabled
Not disabled

Probability of
having had
at least one
episode of
employment
after being
Probability of removed from Probability of
Length of
having a job
the register high earnings current period
at the time of
of the
(more than
of unemploythe survey
unemployed 6,000 rubles) ment, months

5.2
1.0
8.9
0.4

3.8
5.6
7.3^
2.6

2.9
1.2
2.3
2.9

8.9
10.1***
2.4
3.6

29.2*
0.8

14.9
2.4

12.4*
1.4

26.6***
1.0

Notes: ***indicates significance at 1 percent; **at 5 percent; *at 10 percent; ^at 15 percent;
in all other cases the effect is statistically insignificant.

final analysis, diminishes social welfare. We hope that our study contributes
to the discussion of the effectiveness of government programs as a whole, as
well as the role and forms of government regulation in the labor market.
Notes
1. J.J. Heckman, H. Ishimura, and P.E. Todd, Matching as an Econometric Estimator:
Evidence from Evaluating a Job Training Programme, Review of Economic Studies,
1997, vol. 64, no. 4, pp. 60554; J. Kluve, H. Lehmann, and C.M. Schmidt, Active
Labor Market Policies in Poland: Human Capital Enhancement, Stigmatization, or
Benefit Churning? Journal of Comparative Economics, 1999, vol. 27, no. 1, pp. 6189;
idem, Disentangling Treatment Effects of Active Labor Market Policies: Evidence from
Matched Samples, William Davidson Working Paper, 2002, no. 447; M. Lechner, An
Evaluation of Public Sector Sponsored Continuous Vocational Training Programs in East
Germany, Journal of Human Resources, 2000, vol. 35, no. 2, pp. 34775.
2. A. Akhmedov, I. Denisova, and M. Kartseva, Active Labor Market Policies in
Russia: Regional Interpretation Determines Effectiveness? Nauchnye trudy RESh i
TsEFIR, 2003, no. 37 (in English); L.I. Nivorozhkina, Professionalnye obuchenie
i perepodgotovka bezrabotnykh: statisticheskoe otsenivanie effekta aktivnykh programm na rynke truda, Voprosy statistiki, 2003, no. 6, pp. 3945; J. Benus, R.C.
Brinza, V. Cuica, I. Denisova, et al., Re-Training Programs in Russia and Romania:
Impact Evaluation Study, CEFIR Policy Papers, 2004.

Downloaded by [Library Services City University London] at 17:41 17 April 2016

16 PROBLEMS OF ECONOMIC TRANSITION

3. The study was done with financial support from the government of Sweden
(Swedish International Development Cooperation Agency [SIDA]). All of the results,
opinions, and conclusions are the authors and do not reflect the views of SIDA.
4. The first region is in the Central federal okrug. and the second is in the Ural
federal okrug. Limited resources and decentralization of the administrative data
needed to construct the sample narrowed the range of regions that could be studied.
At the same time, the data for these two regions are representative for at least part
of the Russian regions.
5. The proportion of those who refused to participate in the survey was 30 percent.
The main reasons for nonparticipation were an incorrect address and inability to find
the respondent at home.
6. As our study showed, a guarantee letter from the employer was required in only
310 percent of the cases.

To order reprints, call 1-800-352-2210; outside the United States, call 717-632-3535.