Journal of

Managerial Self performance appraisal vs

Psychology
11,6
direct-manager appraisal:
A case of congruence
50
Yehuda Baruch
University of East Anglia, Norwich, UK and Visiting Research Fellow at
the London Business School, London, UK
The process of performance appraisal (PA) is of most importance in human
resource management (HRM). In a broad sense, PA systems are used for two
main purposes: as a source for information for management; and as a feedback
instrument for individuals employed by the organization. In the first case, the
applications of the use of PA serve a variety of management functions. These
could be decision-making about promotions, training needs, salaries, etc. Pay-
related PA is a growing facility in many organizations where it provides
information which can help in making relevant decisions[1,2]. However, PA can
be used for enhancing developmental processes of employees or as an
evaluation instrument per se[3,4].
In the second case, where feedback is the main goal, the fundamental
purpose is to provide the employee with information that will improve personal
performance and effectiveness. Recently the second approach has gained more
attention. Providing the employee with feedback is widely recognized as a
crucial activity. Such feedback may encourage and enable self-development,
and thus will be instrumental for the organization as a whole. Organizational
effectiveness is enhanced through improvement in all the constituencies of the
organization, including the people – the employees, beginning with the rank
and file and up to top management. The term “direct manager-subordinate” will
be in use throughout this paper, but of course the subordinate could be a
manager reporting to a higher-level manager, i.e. not necessarily a lower
ranking worker.
While much current development (and research) is concerned with the
feedback given by the manager to the subordinate it is only one-way. Actually,
it emphasizes the traditional tendency of vertical, downward information flow
in organizations. There is a need, however, to expand the flow of information
and to enhance other routes of feedback in the organization. There is a special
need for developing horizontal communication paths and tools for various
purposes, for which feedback may be used. This need has developed hand-in-
hand with current changes in organizational structures such as delayering and

Journal of Managerial Psychology,

Vol. 11 No. 6, 1996, pp. 50-65. The author wishes to thank A. Williams, P. Hind, P. Levy, M. Peiperl, M. Champion, J. Fryatt and
© MCB University Press, 0268-3946 the editor of the Journal for their helpful comments on earlier drafts of the paper.
re-engineering. The old-fashioned, multi-tiered hierarchical organization is out SPA vs DMPA:
of fashion nowadays. a case of
In the last decade, research compared various sources for PA, e.g. evaluation congruence
subordinates – upward appraisal[5-7], peer[8], and multiple sources[9-11].
Tornow[12] indicated, in a special issue of HRM Journal devoted to 360-degree
feedback, that feedback processes should use more information from sources
other than the direct manager. 51
There are quite a few possible sources for evaluation, but the most neglected
source, perhaps, is the employee himself/herself. Very little research has been
carried out relating to this possible source[13]. Most studies which did refer to
self-evaluation indicated positive results (for a more detailed review see [14,15] ),
but then most studies did not indicate high correlations between self
performance appraisal (SPA) and other sources of information[16-18]. The
mismatch between self and other performance appraisal sources has appeared
in past reports, e.g. overrating errors[19]. The reason for this may be method,
methodology, or the nature of self appraisal[20] and concerns regarding
overrating might discourage organizations from the use of SPA. This could be
due to different appraisal methods, lack of objective measures, employee
concerns regarding the link between pay and performance, etc. One
exception[21] found high correlations between some aspects of work-related
performance including SPA. It was, however, a unique sample – faculty
members in a university, and the conditions were too ideal: both the faculty
members and their chairperson were involved in the procedure.
The question which prompts the present study is which characteristics are
likely to lead to higher correlations between self and other forms of PA in
industrial and business settings. In most cases, results of studies which were
directed towards finding a connection between SPA and other sources have not
been highly or overwhelmingly positive, i.e. scholars found only low
correlations between SPA and supervisor appraisal. Some examples were
presented by Harris and Schaubroeck[22] and by Thornton[23]. These studies
indicated that there were quite moderate correlations between SPA and other
sources of appraisal, mainly direct manager PA (DMPA). The correlation
coefficient (r) of 0.2-0.3 was found as a typical case[8]; 0.35 (0.27 for managerial
level and 0.42 for blue-collar employees) in Harris and Schaubroeck[22] meta-
analysis and 0.29 in Mabe and West[24] meta-analysis. Such results might
hinder management from applying and using the SPA system. However, Tsui
and Ohlott[9] suggest that low interrater agreement has led to
recommendations to use a multiple assessment procedure for providing a more
comprehensive PA (see also[25]). They claim that our understanding of the
reasons for low agreement is still quite limited, and have pointed out three
commonly accepted explanations for the low agreement between any two
raters:
(1) selective perception or differences in the performance information
available to different raters;
Journal of (2) variations in performance criteria used by individual raters; and
Managerial (3) unique rating tendency or style, leading to different rating errors of
Psychology different individual raters[9, p. 780] ).
11,6 In cases of such low correlations, it is difficult or practically impossible to point
out which source is more valid and reliable. It has been argued that such low
52 agreement hinders the use of SPA for evaluative purposes or for employees’
development[3]. Thus there is a tendency to leave SPA out of the appraisal
systems. This may lead to misuse or the abandoning of a valuable source of
information, which has a few advantages and some shortcomings. Past
research has failed to explore rigorously the reasons for differences and low
intercorrelations. Ilgen et al.[26], exploring the recent research in the area of PA
accuracy, found four factors affecting the PA process: the setting, the ratees, the
raters and the nature of scales used. They suggested further study to
concentrate on a cognitive variable as a way to improve PA systems. They
pointed out the importance of observation in PA and also that a process of social
judgement is involved in forming an impression. Relating to SPA, it is clear that
observation can be best applied in such a case – the person is observing
him/herself all the time. The second element, social judgement, is also
eliminated in the case of SPA, although self-judgement may be affected too by
general environmental social judgement.
These are, of course, not the sole elements involved in achieving accuracy.
Other elements can be the scales in use – the dimensions (range, construction,
choice, etc.), and their nature – are they based on classification or on
behavioural anchors[27]? The question of scales in general and, more
specifically, the formulation of the questions under study need to be similar.
Otherwise differences in response could be due to the variation in the way one
asks the question.
Greller and Parsons[28] explored the connection between self-evaluation and
feedback. Even though they used only one simple scale (1-5), they indicated a
close association between task feedback and their SPA measure. Does feedback
act as a key factor in providing congruency between perceptions? If so, how can
an open communication improve the match between evaluations? We will
return to these questions later.
Evaluations should ideally be valid, reliable, accurate and generalizable.
How can we measure and define accuracy? Assuming that we have some
sources of appraisal, the best way would be to compare them all to an objective
measure and consider the correlations between them. The problem is that in
most cases, there is no such measure available. If it exists, there is no real need
for other sources. When we have multiple sources of information, we can
examine the intercorrelations – if they are statistically significantly high we
may take it as an indication for congruency[10]. This also means that all the
sources may measure the same aspect (work behaviour and performance in this
case). There are limitations to the levels of intercorrelations among raters.
Weekley and Gier[29] indicated that there might be a ceiling for the level of
accuracy for different raters. They indicated that in what may be the best option SPA vs DMPA:
for fit among raters, intercorrelations were about 0.8. Cleveland and Shore[30] a case of
found interrater reliability of 0.64 in another case where the research congruence
configuration enabled optimum conditions for fit (same observation, qualified
and experienced evaluators, clear instructions, etc.).
There is a need for further research directed to finding conditions, terms,
occasions and contingencies in which SPA can be more valid and reliable. One 53
recent study by Williams and Levy[31] found that SPA was improved when
people had accurate knowledge of the PA system. The present paper suggests
another antecedent, i.e. the former knowledge of the evaluation as a condition to
achieving high correlation between SPA and direct manager appraisal.

Aims and hypotheses

The aims of the present research are to examine the relationship between direct-
manager PA (DMPA) and SPA in a specific setting, to explore possibilities of
reaching a considerable level of congruence between DMPA and SPA, and to
look at the special conditions under which the relationship may be strong.
Another specific aim is to evaluate the variance between various sources of
information. Altogether, the present study will try to highlight how SPA could
serve as a valuable tool in human resource management. The achievement of
high association among various evaluations may provide two practical uses for
HRM: first, combining several evaluations, it may be used to provid more valid
and reliable information; and second, it would be ideal for use in mutual
feedback. Let us consider the research hypotheses and their rational:
H1: Prior knowledge of the direct manager’s PA will enable high association
between SPA and DMPA.
Such a knowledge may increase understanding of criteria used by the managers
and provide a reference point for the employee. This can, however, be due to the
simple adaptation process. Both cases can lead to a situation where PA
feedback will prevent employees from going to extremes and will provide an
anchor for the SPA. Some people will tend to have high self-expectations and
will be severe in their SPA; others will be more lenient with themselves. Coupled
together this can be one reason for the mismatch. Also the lack of knowledge
mentioned before which might reflect the inability to assess performance in
relation to the goals of the organization or function. Knowing the DMPA can
provide such a reference point and bring a balancing of views which may
encourage a further, more fruitful feedback process. Such feedback can confirm
a person’s view of his or her self, or indicate mismatch. Learning the reasons for
such a mismatch can give an appraisee a better understanding of the
requirements and expectations, as well as an opportunity for further
development. Another issue for consideration is the more technical aspect of
formulation – unity of the evaluation scales. In order to achieve consistency in
applying similar criteria, these scales should be identical across the sources:
Journal of H2: PA from various sources will be influenced by different sets of
Managerial antecedents; some are work related and some are not. It is hypothesized
Psychology that the DMPA would be affected more by workplace-related issues
whereas SPA will be connected more to internal and personal
11,6 perceptions.
PA is a judgemental process which is designed to evaluate the performance of a
54 specific person. As already proposed, there are various possible sources for PA,
and the judgement of each of them may be affected by a different set of
variables, including the work place, internal needs of the employee, attitudes
and personal biodata. As for the first hypothesis, having the DMPA as a
reference point which reflects organizational perceptions and expectation may
provide another relevant view which will relate the SPA to these aspects too:
H3: There will be no significant differences in the correlation between SPA
and DMPA among different environments (i.e. country, sector of
activity).
The issues affecting PA in the business world are of general content, and not
directly concerned with the specific external environment. The same ideas are
used all over the world to train and educate HRM professionals (as can be found
in most HRM textbooks). Variety in PA scores emerge from different perceptions
of the sources, not from different cultures (both rater and ratee act in the same
work environment). The hypothesis suggests that the managerial process of PA
will not be subject to cultural differences if managed in a systematic,
professional manner. If supported it means the ideas presented above are
applicable without specific considerations relating to any external environment.

Method
Research instruments and the environment
The research instruments used in this survey consisted of questionnaires which
were used in two different studies (see Appendix for the the relevant questions
which were used).
This research design presents a unique case which enables us to look at the
relationship between DMPA and the SPA, whereby the appraisee receives
regular feedback on performance, and uses the same kind of questions and
scale. The first of the two studies also enables us to compare two different ways
of asking the individual for SPA: the first – general performance score; the
second – self evaluation in comparison to peers’ performance. All the
evaluations were in relation to overall, general performance (one single
dimension measure). Other variables were checked in order to test the
hypotheses. These were biographical data (age, gender, tenure); work-related
data such as salary, present hierarchical position, and perception of the highest
hierarchy level which the respondent believed he/she could reach within the
organization; and personal attitudes towards job, work, and internal needs.
Study No. 1. This study was part of a large-scale research of career planning
and management which took place in 41 high-tech organizations (39 in Israel
and two in the UK). The organizations agreed to distribute questionnaires, SPA vs DMPA:
focused on career development and related issues. The population under study a case of
was characterized as managers and employees of managerial potential (such as congruence
engineers). The sample was stratified, the population separated according to
the hierarchical levels within each organization to ensure that the respondents
represented all the hierarchy levels within the organizations under study. At
each level a random sample was used. Out of a total 1,540 questionnaires, 846 55
were returned – a return rate of 55 per cent, representing all the hierarchy levels
within the target population. Confidentiality was assured through anonymity of
the respondents.
The characteristics of the respondents were as follows: 66 per cent of the
respondents had an academic degree, and almost 90 per cent had a professional
or technical education. This level of education is higher than is usually reported
in the literature. Nevertheless, this sample is representative of all managers in
high-tech organizations (see[32] for a definition of the high technology
organization). Eighty-seven per cent were male, average tenure in the
organization was 11.1 years (SD = 7.4) and average age was 41 (SD = 7.5). The
characteristics of the non-participants were examined and compared with those
of the participants. No demographic differences were found between the
respondents to the questionnaire and the non-respondents.
Study No. 2. The second study was involved with managers from various UK
companies, including some of the UK’s largest leading companies. Most of the
participants took part in a MBA programme, either as students or coaches from
the organizations which sent them to this programme. Out of 225
questionnaires, 114 were returned (51 per cent). The characteristics of the
respondents were: 52 per cent had an academic degree, which indicates high
educational level, similar to the population in the first study; 87 per cent were
males; and the average age was 41 (SD = 8.3). The profile spread was almost
similar to those of the first study.
Analysis. In each case the correlation between the PA of the various sources
was computed. A contingency test was conducted in order to test the third
hypothesis. A regression analysis was directed towards indication of possible
sets of mediators which might have an effect on each source of PA.

Results
The results will be presented separately for each study. In both presentations,
DMPA is the direct manager PA (as reported by the respondent, see question
one in the appendix); SPA1 is the answer to the second question (what should
have been your PA score…); and SPA2 is the answer to the third question
(contribution compared to peers).

Study No. 1
In 442 cases we had the DMPA score: in 754 cases we had the SPA1 score, and
in 829 we had the SPA2 score. The averages (and standard deviations) were:
DMPA: 85.5 (10.6); SPA1: 87.8 (7.8); SPA2: 87.0 (10.4).
Journal of The reason for having fewer responses for the DMPA was either that there
Managerial was no PA system in their organizations or that they did not know how they
Psychology were evaluated. The intercorrelations in this case are presented in Table I.
11,6
DMPA SPA1 SPA2
56 DMPA 1.0 0.73 0.30
SPA1 1.0 0.46
SPA2 1.0
Table I.
Intercorrelations Notes:
between the n = 846
three-PA score Significance: p < 0.01

The results in Table I are far higher than the typical findings which appear in
the literature. In order to interpret these findings and consider the implications
for further development in the SPA area, it would be valuable to look into the
elements of the PA system that enabled such results to emerge. There are
several possible explanations for the results. It is a unique case when the
employees were asked to evaluate themselves, given that they knew the
evaluations of their direct managers. This may cause a bias in the answers,
nevertheless such a bias exists whenever the employee knows his or her
evaluation.
From the correlations in Table II, we see that age has no relation to the
evaluations and has low correlation with SPA2. Education and salary were
found to be positively correlated with all the evaluations. Present position and
the perception of future possible hierarchy advancement were correlated
positively with the DMPA and the SPA2 only. Job and organizational
satisfaction, commitment and the need for achievement all had positive and
significant correlations with the three PA scores, although at a moderate level
(Table III). As for gender differences, men and women had almost the same
averages except for the SPA2. This will be discussed later.
Contingencies. Contingency analysis can contribute to better understanding
of the factors that may have an effect on the general relationships that were

Present Career
position future Salary Age Gender Education

DMPA 0.12* 0.19** 0.22** 0.01 0.01 0.25**

SPA1 0.00 0.07 0.14** 0.04 –0.02 0.21**
SPA2 0.19** 0.20** 0.28** 0.11** –0.17** 0.19**
Table II.
Correlations between Notes:
the three scores and * p < 0.05
other variables ** p < 0.01
found. In order to check possible contingencies, we looked separately at the SPA vs DMPA:
intercorrelations between the three evaluations in the cases of different a case of
educational levels, different gender and different countries. congruence

Job Organization Need for Organization

satisfaction satisfaction achievement commitment 57
DMPA 0.27 0.28 0.25 0.17
SPA1 0.19 0.23 0.36 0.20
Table III.
SPA2 0.12 0.11 0.35 0.17 Correlations between
Note: the three scores and
significance: p < 0.01 personal variables

In the case of high level educated employees, the intercorrelations between the
DMPA and the SPA1 and the SPA2 were 0.73 and 0.23, and between the SPA1
and the SPA2 it was 0.44 (n = 642; p < 0.01). In the case of low level educated
employees (non academics), the intercorrelations were quite similar: 0.72 and
0.30 between DMPA and the SPA1 and SPA2; 0.42 between SPA1 and SPA2
(n = 204; p < 0.01). Thus no meaningful differences were identified. The
conclusion can be that the results hold true across different levels of education.
In the case of male employees, the intercorrelations between the DMPA and
the SPA1 and the SPA2 were 0.71 and 0.23, and between the SPA1 and the SPA2
0.48 (n = 726; p < 0.01). In the case of female employees the intercorrelations
were quite the same: 0.86 and 0.31 between DMPA and the SPA1 and SPA2; 0.34
between SPA1 and SPA2 (n = 111; p < 0.05).
The comparison shows that women’s SPA correlated closer to the DMPA
than in the case of the men’s. It could indicate either that women’s evaluations
are more accurate or that women’s tendency to agree with the direct manager
evaluation is higher than that of men. On the other hand, the correlation
between the two different SPA were lower for females than for males. This may
support the second explanation for the higher correlation between SPA1 and
the DMPA, since the SPA2 is less connected to the DMPA, and may release the
respondent to answer more freely.
Another explanation for the lower correlation of SPA2 in the female case is
that women may have particular difficulties with such self-evaluation
(compared with others), since it is connected with personal self-esteem. In this
specific case, women were working in a male-dominated environment. The
average hierarchy level and salary of women was lower than that of men,
although their educational level was similar. We also found similarity in the first
two scores of the PA, but significant difference in SPA2 scores.
The averages (and standard deviations) for males were: DMPA 85.4 (10.8);
SPA1 88.0 (7.6); SPA2 87.7 (9.7); and for females: DMPA 85.6 (9.6); SPA1 87.5
(8.2); SPA2 82.5 (13.2). The difference in the SPA2 may be explained by lower
levels of competitiveness among women, a tendency towards greater co-
Journal of operation when compared to men, or higher levels of modesty in women. Self-
Managerial esteem was found to have an effect on self-ratings[33] and it could be related to
Psychology gender differences too. From the similarities in DMPA and SP1 we can conclude
that both males and females had the same performance level.
11,6 As for country differences, the results shows somewhat higher correlations
in Israel between the DMPA and the SPA1 (0.71 in Israel vs 0.59 in the UK). It
58 may be due to the fact that in Israel there were more organizations and thus
higher variance in the distribution of course. This may explain the even higher
correlation found in the second study between DMPA and SPA1, where many
organizations were involved.
Regression analysis. The use of a regression comes in order to understand
which factors affect the PA scores in the different cases (see H2). In an ideal
case, the PA scores would be related directly and only to the actual
performance. However, many other factors mediate this relationship. Thus
there was an attempt to explain the three PA scores (DMPA, SPA1, and SPA2)
by the following set of variables: need for achievement, need for control, age,
educational level, tenure (in the organization), and gender. All of these variables
were suggested as possible independent variables in three regression analyses
– where the dependent variable was DMPA in the first, SPA1 in the second and
SPA2 in the third. The aim of these analyses was to test the second hypothesis,
by exploring whether different appraisals have different sets of biases due to
indirect variables, and based on different information sources. Table IV
presents the results of the three regressions.
These results indicate that only educational level appears in all three cases.
The most significant difference is that DMPA was connected to external
variables such as age and tenure whereas in both SPA1 and SPA2, greater
influence was associated to personal characteristics – internal needs, and also
gender in SPA2.

Independent variable Beta T Significant T Multiple R

DMPA
Age 0.21 4.7 0.0000 0.21
Education –0.16 –3.4 0.0009 0.26
Tenure –0.14 –2.4 0.0152 0.28
SPA1
Need for achievement 0.37 10.7 0.0000 0.37
Education 0.18 5.3 0.0000 0.41
SPA2
Need for achievement 0.35 10.5 0.0000 0.35
Education 0.16 4.9 0.0000 0.38
Gender –0.16 –4.9 0.0000 0.41
Table IV. Need for control 0.12 3.4 0.0008 0.43
Regression analyses Tenure 0.08 2.6 0.0101 0.44
It is the goal of any PA system to reflect actual performance. Neither internal SPA vs DMPA:
personal needs and characteristics nor external dimensions such as age or a case of
tenure are supposed to have an effect on the appraisal rating. However, our congruence
results suggest that the DMPA was affected more by age and tenure, perhaps as
they reflect experience, whereas the SPA were more affected by internal needs
(i.e. need for achievement, need for control). It is possible, of course, that there is
a real connection between these variables and the levels of performance. 59
However, the above results suggests that different elements affect different
perceptions which comes from all the sources, and as there is no match among
them, some just represent biases due to irrelevant information. Elements which
intervene could include both “hard” data, considered mainly by DMPA, and
“soft” attitudes and perceptions, having an impact on SPA. Such discrepancies
should be taken into account when trying to explain lower correlations between
various sources of 360-degree PA.

Study No. 2
The aim in carrying out this study was to replicate the PA procedure that was
performed in the first study and to look for similarities in a different
environment. In this case we had only the DMPA and the SPA1. Out of the 114
responses, 76 cases included the DMPA score and 85 cases included the SPA1
score. The averages (and standard deviations) were: DMPA: 76.2 (11.3); SPA1:
79.2 (10.6).
The intercorrelation between the DMPA and the SPA1 was 0.81 – a very high
correlation. This result is even higher than that which was found in the first
study and more than the typical findings to appear in the literature. Most of the
specific circumstances that produced such results in the first study were
repeated, except for the organizational sector of activity – these were not
exclusively high technology in this case, but a mixture of high and mid
technology, finance and service industry.
The correlations between DMPA and SPA1 with age, gender and educational
level were not significant; all were low negative correlations (–0.04 to –0.11),
except salary which was positive but relatively low (0.22 with DMPA, 0.16 with
SPA1). The small sample size meant that statistically significant results were
not available.
In this study only job satisfaction and job involvement were examined as
personal variables; the correlations with DMPA and SPA1 were positive but
moderate. For the DMPA it was 0.11 with job satisfaction and 0.27 ( p < 0.05)
with job involvement, and for the DMPA it was 0.10 with job satisfaction and
0.20 with job involvement. No contingencies or regression models were
performed for this study due to the low number of relevant participants.

Discussions and conclusions

SPA can be a useful practice which may improve the level of internal
understanding and consensus within organizations. Meyer[34] indicated an
increase in satisfaction for both employees and their managers following the
Journal of use of SPA. The process of creating self-appraisal may also be useful as a means
Managerial of increasing employees’ understanding of performance feedback[35].
Psychology In this study we have tried to eliminate possible causes for low congruence
between SPA and DMPA. Only cases where the employees knew their DMPA
11,6 were used for the evaluation, and both appraisers and appraisees used identical
rating scales.
60 The results strongly supported all the three hypotheses: the first hypothesis
was supported in the sense that very close relationships were found between
DMPA and SPA. Compared with other studies which did not indicate such high
correlations, we may assume that the results depend to some extent on the
research design. These results were valid only when the same design was
applied. The hypothesis was supported in both studies. Having the knowledge
of the DMPA seems to reduce variance between it and the SPA, whereas in cases
where the appraised people had no such an information, it might produce
contradicting results (cf. [36] ). The reason could be having a reference point to
start with, i.e. an indication for level of expectations. It can also be associated
with the cognitive dissonance theory – the assessed people may be inclined to
reduce the gap and thus avoid conflict.
H2 was supported too: several regressions which were carried out which
indicated variety in the antecedents of the three sources of PA. DMPA was
found to be affected more by workplace-related issues (i.e. tenure, age) whereas
SPA was connected more to internal and personal issues (personal needs,
gender). Only educational level appeared in all.
As for the third hypothesis, the results were found quite similar for people in
various sectors of activity, and in two western countries within the same
cultural matrix (see[37]). Contingencies analysis indicated that only gender
influenced the correlations, neither education nor country.
These findings emphasize another important aspect for the use of SPA. In
today’s business environment, emphasizing the consensual rather than
disagreements between manager-subordinate is highly desirable. In the
flattened organizations of the future, managers need to encourage autonomy
and self-development and the views of the individual will be sought after.
Applying a SPA system may go some way to fulfil these needs.
The advantages of SPA emerge from various sources: one obstacle which
hinders DMPA scores reliability is limited opportunities for observation (e.g.
time and place). Judge and Ferris[38] have pointed out the importance of
opportunities to observes the subordinate’s job performance in influencing the
evaluation. This problem does not exist in the case of SPA, although such an
appraisal may be concentrating on different job dimensions from those
observed by the direct manager.
SPA, especially if highly correlated with DMPA, may also contribute to the
understanding of the demands and expectations of the organization. It can
enhance the employee’s identification with his or her manager. Above all, by
applying SPA system, the organization manifests genuine interest in the
employees and belief in their ability to provide relevant, valid, and reliable
information. An important factor would be the possible use of such an SPA vs DMPA:
information. a case of
On the other hand there are quite a few possible disadvantages in SPAs. As congruence
self-evaluation it is subjective. The managerial quest we face when considering
SPA is whether, and to what extent, can we expect the employee to relate to
objective criteria and not only to internal perceptions. Thus SPA cannot be
recommended as a source for pay-related PA, since it might restrict the validity 61
and reliability of the appraisal – it is psychologically “charged”[39,40].
Altogether, in order to apply SPA successfully, there is a need for deep belief
and confidence of the management in the ability of the employee to produce
valid and reliable self-judgement.
Mabe and West[24] suggested that aspects such as previous experience in
self-evaluation, instruction accompanying the self-evaluation, and knowledge
that the rating would be compared with some criterion measure, may be
important in determining self-supervisor agreement. The special conditions in
the present study comply with their suggestions, and subsequently produce
such a high agreement.
The main findings of the present study are: very high correlation (0.73 and
0.81 in the two studies) was found between DMPA and SPA. Other moderate
correlations were found between both and attitudes to work and the
organization (satisfaction, commitment). Contingency effect was found mainly
in the gender comparisons, where in one aspect of SPA, women were correlated
higher than men with the DMPA. That may indicate either that women are more
accurate than men or that they tend to agree with their manager more than
men. In the connection between the two SPA measures (direct evaluation and
relative to others’ evaluation), women were found to have lower correlation than
men. Regression analysis indicated that the direct manager PA is affected more
by external elements (age, tenure) and by educational level of the employee,
whereas the SPA is affected more by internal needs (only partially measured in
the studies), education, and in one case, by gender.
The differences between the results of the present study and those presented
in earlier studies relate mainly to the correlations found between SPA and direct
manager PA. The results were significantly higher than have been previously
found. The reasons may be that the employees under study knew in advance
their DMPA, the populations and organizations under study, and the similarity
of the wording in the PA measure. A high level of education as well as seniority
in the organization can increase the validity of SPA (cf. [41]).
Possible applications and conditions for the use of SPA depend on the type of
PA system and its goals. There are two basic goals for PA systems – either to
develop the individual employee through joint processes or to serve as a source
of information for management. In the first case the results indicate the possible
value of having mutual agreement, fewer differences, creating understanding or
indicating problematic areas. For a system that is directed to gather
information for management, the present study supports the idea of expanding
PA systems to make use of more than the traditional DMPA output (see [10,22]).
Journal of When relating to the employee as a source of information on his/her own
Managerial work behaviour and performance, we manifest more beliefs, respect and
Psychology delegation of responsibility – we listen to the employee. Past studies, however,
indicated problems connected with low correlations between the DMPA and
11,6
SPA (e.g. which is more reliable, what is the reason for the incongruence). This
kind of mismatch prevents organizations from using SPA in most cases. These
62 problems had to do with different expectations and perceptions on the meaning
of desired performance, vindictiveness effect, non-realistic evaluations, etc.
Having the DMPA is a form of feedback, which was previously found to be a
helpful factor on various occasions.
The correlations that were found here are overwhelmingly high, especially
when we take into account that no one can expect perfect correlation, which
means full agreement (see for example [29]). The present study shows that in
order to improve the validity and reliability of PA systems, it is not sufficient to
develop better forms of PA by the direct manager. A multiple-source PA system
should also be considered. Such a system can add to the base of knowledge as
well as to the quality of the feedback given to the employee. It can create more
involvement and commitment if one gets feedback not only of one’s work but
also of the way one evaluates it.
In contradiction to other studies, it was found that there could be certain
conditions or cases under which the SPA matched other sources, as in the case
indicated by Williams and Levy[31], in which there was a contingency factor –
perceived system knowledge. The present case suggests additional factors.
These are the knowledge of the direct-manager score and the way the question
is being asked. Another study which investigated the halo effect also indicated
the importance of method and design[20]. The present research indicates the
importance of the design of questions, where similarity can produce a higher
agreement. The simplicity manifested in the present case helps to overcome
barriers where many irrelevant elements might play a role in creating biases in
the PA process[42]. Another factor that may be investigated in further research
is looking at a population of different educational levels.
Some limitations of the present study are due to the data collection
procedure. It would be better to find the DMPA separately and then to compare
it with the SPA. However, such a process was not feasible under the specific
circumstances, and also may not imply that the assessed have the feedback to
which the present results are attributed to. Are the data valid? We may assume
that owing to the anonymity of the respondents there is no reason for them to lie
when referring to their report of the DMPA. On the other hand, a lack of
accurate instructions and dissimilarity of criteria in past studies could be one
reason for the moderate correlations found. Another limitation of the present
study would be that it dealt with a unique population – managers, most of
whom are highly educated. Future research is recommended to replicate the
research under different circumstances, cultures and populations.
References SPA vs DMPA:
1. Heneman, R., “Merit pay research”, in Rowland, K. and Ferris, G. (Eds), Research in a case of
Personnel and Human Resource Management, Vol. 8, JAI Press, Greenwich, CT, 1990.
2. Kessler, I. and Purcell, J., “Performance related pay: objectives and application”, Human
congruence
Resource Management, Vol. 2 No. 3, 1992, pp. 16-33.
3. Campbell, D. and Lee, C., “Self appraisal in performance evaluation: development vs
evaluation”, Academy of Management Review, Vol. 13, 1988, pp. 302-14. 63
4. Farh, J.L., Cannella, A.A. and Bedeian, A.G., “Peer ratings: the impact of purpose on rating
quality and user acceptance”, Group & Organization Studies, Vol. 16 No. 4, 1991,
pp. 367-86.
5. Bernardin, H.J., “Subordinate appraisal: a valuable source of information about managers”,
Journal of Human Resource Management, Vol. 25 No. 3, 1986, pp. 421-39.
6. Grint, K., “What’s wrong with performance appraisals? a critique and a suggestion”,
Human Resource Management Journal, Vol. 3 No. 3, 1993, pp. 61-77.
7. Atwater, L., Roush, P. and Fischthak, A., “The influence of upward feedback on self and
follower ratings of leadership”, Personnel Psychology, Vol. 48 No. 1, 1995, pp. 35-59.
8. Shore, T.H., Shore, L.M. and Thornton, G.C., “Construct validity of self- and peer
evaluations of performance dimensions in an assessment center”, Journal of Applied
Psychology, Vol. 77, 1992, pp. 42-54.
9. Tsui, A. and Ohlott, P., “Multiple assessment of managerial effectiveness: interrater
agreement and consensus in effectiveness models”, Personnel Psychology, Vol. 41, 1988,
pp. 779-803.
10. Baruch, Y. and Harel, G., “Multi-source performance appraisal: an empirical and
methodological note”, Public Administrative Quarterly, Vol. 17 No. 1, 1993, pp. 96-111.
11. Ward, P., “A 360-degree turn for the better”, People Management, Vol. 1 No. 3, 1995,
pp. 20-22.
12. Tornow, W.W., “Perceptions or reality: is multi-perspective measurement a means or an
end?”, Human Resource Management, Vol. 32 Nos 2/3, 1993, pp. 221-30.
13. Basset, G.A. and Meyer, H.H., “Performance appraisal based on self-review”, Personnel
Psychology, Vol. 21, 1968, pp. 421-30.
14. Glueck, W.F., Personnel – a diagnostic approach, Business Publications, Plano, TX, 1978.
15. Love, K.G. and Hughes, F.V., “Relationship of self-assessment rating and written test score:
implications for law enforcement promotional system”, Public Personnel Management,
Vol. 23 No. 1, 1994, pp. 19-30.
16. Heneman, H.G., “Comparisons of self and superior ratings of managerial performance”,
Journal of Applied Psychology, Vol. 59, 1974, pp. 638-42.
17. Klimoski, R.J. and London, M., “Role of the rater in performance appraisal”, Journal of
Applied Psychology, Vol. 59, pp. 445-51.
18. Tsui, A. and O’Reilly, C.A., “Beyond simple demographic effects: the importance of
relational demography in supervisor-subordinate dyads”, Academy of Management
Journal, Vol. 32 No. 2, 1989, pp. 402-23.
19. Nilsen, D. and Campbell, D.P., “Self-observer rating discrepancies: once overrater, always
an overrater?”, Human Resource Management, Vol. 32 Nos 2/3, 1993, pp. 265-82.
20. Somers, M.J. and Birnbaum, D., “Assessing self-appraisal of job performance as an
evaluation device: are the poor results a function of method or methodology?”, Human
Relations, Vol. 44 No. 10, 1991, pp. 1081-91.
21. Farh, J.L., Werbel, J.D. and Bedeia, A.G., “An empirical investigation of self-appraisal-based
performance evaluation”, Personnel Psychology, Vol. 41, 1988, pp. 141-56.
Journal of 22. Harris, M.M. and Schaubroeck, J., “A meta-analysis of self-supervisor, self-peer, and peer-
supervisor ratings”, Personnel Psychology, Vol. 42, 1988, pp. 43-62.
Managerial 23. Thornton, G.C., “Psychometric properties of self-appraisal of job performance”, Personnel
Psychology Psychology, Vol. 33, 1980, pp. 263-71.
11,6 24. Mabe, P.M. and West, S.G., “Validity of self-evaluation of ability: a review and meta-
analysis”, Journal of Applied Psychology, Vol. 67, 1982, pp. 280-96.
25. Latham, G.P. and Wexley, K.N., Improving Performance through Effective Performance
64 Appraisal, Addison-Wesley, Reading, MA, 1981.
26. Ilgen, D.R., Barnes-Farrell, J.L. and McKellin, D.B., “Performance appraisal process
research in the 1980s: what has it contributed to appraisals in use?”, Organizational
Behavior and Human Decision Processes, Vol. 54, 1993, pp. 321-68.
27. Murphy, K.R., “Criterion issues in performance appraisal research: behavioural accuracy
versus classification accuracy”, Organizational Behavior and Human Decision Processes,
Vol. 50, 1991, pp. 45-50.
28. Greller, M.M. and Parsons, C.K., “Feedback and feedback inconsistency as sources of strain
and self-evaluation”, Human Relations, Vol. 45 No. 6, 1992, pp. 601-20.
29. Weekley, J.A. and Gier, J.A., “Ceilings in the reliability and validity of performance ratings:
the case of expert raters”, Academy of Management Journal, Vol. 32 No. 1, 1989, pp. 213-22.
30. Cleveland, J.N. and Shore, L.M., “Self and supervisory perspectives on age and work
attitudes and performance”, Journal of Applied Psychology, Vol. 77 No. 4, 1992, pp. 469-84.
31. Williams, J.R. and Levy, P.E., “The effects of perceived system knowledge on the agreement
between self-ratings and supervisor ratings”, Personnel Psychology, Vol. 45 No. 4, 1992,
pp. 835-47.
32. Baruch, Y., “High technology organization – what it is, what it isn’t”, International Journal
of Technology Management, 1996, forthcoming.
33. Levy, P.E., “Self-appraisal and attributions: a test of a model”, Journal of Management,
Vol. 19 No. 1, 1993, pp. 51-62.
34. Meyer, H.H., “A solution to the performance appraisal feedback enigma”, Academy of
Management Executive, Vol. 5, 1991, pp. 68-76.
35. Teel, K., “Self-appraisal revisited”, Personnel Journal, Vol. 57, 1978, pp. 364-7.
36. Roberson, L., Torkel, S., Korsgaard, A. and Klein, D., “Self-appraisal and perceptions of the
appraisal discussion: a field study”, Journal of Vocational Behaviour, Vol. 14 No. 2, 1993,
pp. 129-42.
37. Hofstede, G., “The interactions between national and organizational value system”, Journal
of Management Studies, Vol. 22, 1983, pp. 347-57.
38. Judge, T.A. and Ferris, G.R., “Social context of performance evaluation decisions”,
Academy of Management Journal, Vol. 36 No. 1, 1993, pp. 80-105.
39. Lawrie, J.W., “Your performance – appraise it yourself”, Personnel, Vol. 66 No. 1, 1989,
pp. 21-3.
40. Cook, M., “Performance appraisal and true performance”, Journal of Managerial
Psychology, Vol. 10 No. 7, 1995, pp. 3-7.
41. Clement, R.W., “Self-appraisal: a solution to evaluating executive performance”, Business,
Vol. 40 No. 3, 1990, pp. 10-14.
42. Wayne, S.J. and Liden, R.C., “Effects of impression management on performance ratings:
the case of expert raters”, Academy of Management Journal, Vol. 32 No. 1, 1989, pp. 213-22.

Appendix
The relevant part of the questionnaire for the purpose of this paper consists of the three following
questions (in the second study only the first two):
 (1) The average of your last two performance appraisal scores is (use routine scheduled SPA vs DMPA:
performance appraisals, general scores):
a case of
on a scale 0-100: [ ] [ ] [ ]
congruence
if the scale is different, please write the score in words, for example “4 out of 7”.
(2) What do you think your performance appraisal score should have been in your own
estimation?
on scale of 0-100: [ ] [ ] [ ] 65
(3) How would you rate your performance and contribution to the organization in
comparison to other employees in the organization with similar positions?
• among the upper 5 per cent
• among the upper 10 per cent
• among the upper 20 per cent
• among the upper 30 per cent
• among the upper 40 per cent
• among the upper 50 per cent
• in the lower 50 per cent
The scale for the third question was transformed into a similar scale as those of the first two,
where the first score was re-named as 95, the second –90 and so on.
Question (3) was not presented in the second study.