IAR coursework

5.1
To check the reliability and validity of the study in the paper, a, correlation and regression
coefficients are used. Firstly, the Cronbachs (1951) alpha coefficient is figured up to estimate the
internal consistency of all the variables measured in the study. The result for the variables is 0.67.
According to Nunnally (1981), measures of reliability between 0.65 and 0.70 will gratify in
majority of the cases. If alpha coefficients below 0.50, the cases should be handled with care. So
the reliability of the study is acceptable. Secondly, to test the validity of the study, two standards
should be met. One is that the independent variables of performance should be assessed
reasonably. In the selected paper, 24 of the possible 28 coupled correlations are around the
correlation among each dimension. So, the measures seem to be relative d to the overall measure.
The other one is that the most changes in the overall rating should be made clear by the eight
items in Table 2. In the case of Brownell and Mclnnes (1986) standard, about 43.1% of the
variance in the entire rating is explained by the eight items. That means the study does not meet
the criteria. As a result, confirmatory factor analysis (Byrne, 2001) is taken to assess the
managerial performance. The best five of original eight items are chosen to form a respecified
model. The new model meets the check of reliability and the first standard of the validity as well.
However, just 41.4% of the changes are explained by the best five items, the level is still low.
Some caution should be taken in the explanation of the results. To deal with some biases, one test
for possible experience effects, one single-factor test and one test for non-response bias are taken.
1. According to the first criteria, to check the reliability and validity of the study in the paper, a,
correlation and regression coefficients are used. To check the reliability of the variable measures,
the study used alpha coefficient to meet Nunnallys (1981) standard. However, for stricter
purposes, alpha coefficients of 0.85 or above may be compulsory (Rosenthal and Rosnow, 1991).
So the result of the reliability was not perfect. Furthermore, according to the second criteria to
check the validity, not only the eight dimensions but also the best five were both not met the
request. So there would be some problems in the selection of the sample. More middle managers
should be invited to take part in the questionnaires.
3. In the beginning of the paper, the expectancy-valence theory was explained clearly and it also
talked about the development of the expectancy model. Unsuitable expectancy model would result
in some questions about the validity in practical use. All the variables were chosen from the
managerial context which is based on the expectancy theory. So they could be better
representative.
4. Eight items were included at the beginning. To meet the standard of construct validity, a new
model include five items was created, but it still met the other criteria which eight items met. The
results of the study were still the same. Finally, the additional tests were very useful, for example,
the participation of the questionnaire was specific anonymity so that non-response bias could be
impossible.