Vous êtes sur la page 1sur 12

Calculating with

Symbols
Release 23.00

Mdling, Austria, November 2003


Copyright by Dr.G.Schuhfried Ges.m.b.H.
Copyright for the Test by Verlag Swets Test Services
Test Authors:
Christoph Schmotzer, Klaus D. Kubinger & Christian Maryschka

Test Label

RIS

Manual Authors:
Thomas Karner and Markus Sommer

Dr. Gernot Schuhfried Gesellschaft m.b.H.


Hyrtlstrae 45, 2340 Mdling, Austria,
phone: +43/2236/42315-0, fax: +43/2236/46597
info@schuhfried.at www.schuhfried.at
FN 104661p Landesgericht Wr. Neustadt, UID Nr. ATU 19273809

RIS
TABLE OF CONTENTS
1. SUMMMARY ......................................................................................... 3
2. DESCRIPTION OF THE TEST ................................................................... 5
2.1. Description of variables ..............................................................................5
2.2. Test forms...................................................................................................5

3. EVALUATION ........................................................................................ 6
3.1. Objectivity ...................................................................................................6
3.2. Reliability ....................................................................................................6
3.3. Validity ........................................................................................................7
3.4. Scaling........................................................................................................7
3.5. Economy.....................................................................................................7
3.6. Falsification.................................................................................................7
3.7. Fairness......................................................................................................7
3.8. Reasonableness .........................................................................................7
3.9. Usefulness ...................................................................................................8

4. NORMS ............................................................................................... 9
5. TEST ADMINISTRATION ....................................................................... 10
5.1. Instruction phase ......................................................................................10
5.2. Test phase................................................................................................10

6. INTERPRETATION OF TEST RESULTS ..................................................... 11


7. REFERENCES .................................................................................... 12

RIS
1. SUMMMARY
Authors:
Schmotzer, C. & Kubinger, K.D. & Maryschka, C.
Application:
Non-verbal assessment of general intelligence based on deductive thinking.
Main areas of application: performance-oriented aptitude diagnostics, industrial and
organizational psychology, consulting regarding school, college and career decisions.
Theoretical background:
The authors followed the factor Reasoning which, according to Thurstone, encompasses
the following abilities: inductive thinking, deductive thinking, and the ability to keep to
restrictions for completing a task (Jger, 1967). The RIS focuses mainly on the latter
aspect.
Administration:
The person taking the test is confronted with a series of simple arithmetic equations using
meaningless shapes instead of numbers. However, the algebraic symbols for these (basic)
calculations remain as usual. The solution is to find the number, which leads to the
mathematical correct solution of the equation if used instead of the symbol. There is no
time limit. It is not possible to omit items or to return to one that has already been
completed.
Test forms:
There is one test form available.
Scoring:
The number of correctly completed items is a measure for the ability of the test respondent.
Reliability:
The test is reliable in the sense of an internal consistency due to Rasch model. The results
of the Rasch analyses could be reproduced with two further independent samples, and now
even includes the assumption of item homogenity (Rost, 1996). The internal consistency
according to Cronbachs Alpha amounts to r=.89.
Validity:
Validity is given with respect to deductive thinking according to Thurstone. The construct
validity results from the correspondence with the Rasch model. This is especially
confirmed by the results of the Martin-Lf test for the examination of item homogeneity.
Correlations with the test WMT are situated at r = 0.71.
Norms:
T-scores and percentile ranks are available of a representative sample of N=236 people
between 16 and 84 years of age divided into two age groups. Furthermore, there are also Tscores and percentile ranks available based on the data of N=165 psychology students.
Reasonableness:
3

RIS
This test will be difficult for people with a performance below average.
Testing time:
Between 20 and 50 minutes.

RIS
2. DESCRIPTION OF THE TEST
A description of the test, instructions for interpretation, details of the paper-pencil-norms
as well as a complete list of references can be found in the test manual of Schmotzer,
Kubinger, and Maryschka (1994).

2.1. Description of variables


Main variables
Number of items solved
Secondary variables
Working time

2.2. Test forms


There is one test form, which consists of 19 items.

RIS
3. EVALUATION
A complete description of test criteria can be found in the manual of Schmotzer, Kubinger
and Maryschka (1994). This manual only provides a summary of these, as well as the
results of a few studies.

3.1. Objectivity
In the computerized tests of the Vienna Test System, test administration, recording and
evaluation of scores takes place automatically and under standardized conditions. This
results in very high measurement accuracy on the one hand, and avoidance of undesired
effects and errors on the other, such as arise e.g. from varying oral instructions or manual
calculation of test results.

3.2. Reliability
Based on the validity of the Rasch model, reliability in the sense of internal consistency
can be assumed.
The results of Rasch analysis for the 17 homogeneous items were duplicated in an analysis
of the representative standardization sample. More detailed information on the description
of the sample can be found in Section 4. The sample was divided according to the median
of the raw score of 9.00, the median of the working time, according to the age median (45
years) and the highest level of schooling completed (compulsory vs. further schooling).
The partition criteria, as well as the empirical and theoretical Chi-Square scores with an
alpha error of 0.01, can be found in the table. If the empirically found Chi-Square scores
are lower than the theoretically expected Chi-Square scores, homogeneity in the sense of
the Rasch model can be assumed. It can be said that at the alpha error-levels selected here,
no significant variation from the Rasch model can be observed for any of the partition
criteria. The test can thus be considered homogeneous with respect to these partition
criteria. A Martin-Lf Test for subdividing the item material according to the complexity
of a calculation (addition and subtraction vs. multiplication and division) also leads to a
non-significant result. Local stochastic independence was tested using the Trasch software
package, whereby the total sample was randomly divided into two subgroups, whose pscores were then aggregated according to the specifications of the program authors. No
deviations were found in the theoretically expected and empirically found relationships
between the individual items of this test (Chi-Square = 3.528; df=4; > 0.4).
Table 1: Empirical and expected Chi-Square values for various partition criteria

Criterion
Median
Age
Sex
Educational level
Working time

Chi-Square (emp.)
29.1664
21.7869
14.1475
23.1394
16.2201

Chi-Square (=0,01)
32.0265
32.0265
32.0265
32.0265
32.0265

Degree of Freedom
16
16
16
16
16

These results show that "Calculating with Symbols" fulfills all three basic assumptions of
the Rasch model. Based on a sample of 179 respondents, unsystematically collected in the
6

RIS
Dr. G. Schuhfried GmbH research laboratory since 1998, the following reliability
calculations were carried out according to classical test theory. For the 17 items, which are
homogeneous in the sense of the Rasch model, there resulted a reliability (CronbachAlpha) of r = 0.87. A reliability analysis of the standardization data (N=147) resulted in a
reliability coefficient (Cronbach-Alpha) of r = 0.89. The test can thus be assessed as
sufficiently reliable based on classical test theory.

3.3. Validity
The authors state that the test exhibits content validity with respect to the factor Reasoning
according to Thurstone. Further validation studies are planned.

3.4. Scaling
Calculating a sum score as an indicator of test performance is justified due to the validity
of the dichotomous Rasch model. All items used in calculation measure the same latent
dimension.

3.5. Economy
The RIS test can be described as economical, in line with the criteria of Lienert and Ratz
(1994), as it requires only a short administration duration, very little material, is easy to
handle and, as it is administered on a computer, can be scored quickly and easily.

3.6. Falsification
Since the RIS is a performance test, a respondent cannot pretend to be better than s/he
really is. It is also not to be expected that a respondent would pretend to be worse than s/he
is.

3.7. Fairness
With reservation as to empirical findings to the contrary, there is nothing which would
indicate that the RIS runs contrary to fairness. It is also not to be expected that persons
without computer experience would be put at a disadvantage. There are no explicit genderspecific differences. Moreover, the results of Rasch analysis with internal and external
partition criteria show that various subsamples, such as for example older and younger
people or men and women, are not systematically disadvantaged on individual items of this
test.

3.8. Reasonableness
The test authors warn that the test is difficult and entails time pressure for persons with
lower levels of performance.

RIS
3.9. Usefulness
The RIS test can be described as useful, since probabilistic-test theoretical approaches are
used to test the ability of inductive and deductive thinking (Reasoning) without allowing
other factors of intelligence to become mixed up in the analysis.

RIS
4. NORMS
Standardized scores were calculated by setting mean percentile ranks PR(x) for every raw
score X according to the formula (Lienert & Raatz, 1994):

PRx = 100

cum fx fx 2
N

cum fx represents the number of respondents who achieved the raw score X or a lower
score, fx is the number of respondents with the raw score X, and N is the scope of the
sample.
The main variable "Score" as well as the secondary variables "Median time of correct
answers" and "Number of correct answers" were standardized. All norms are available as
percentile ranks and T-scores.
Norm sample:
The norms are derived from a sample of the normal population consisting of N=236
people and encompassing 103 (43.6%) men and 133 (64%) women. The age median is 45
years with an inter-quartile range of 16 years. Table 1 gives an overview of the age
distribution of the sample
Table 1

Age distribution of the norm sample


Age group in years
Observed frequency
15 to 20
7
21 to 25
19
26 to 30
26
31 to 35
26
36 to 40
21
41 to 45
21
46 to 50
17
51 to 55
21
56 to 60
19
61 to 65
14
66 to 70
14
71 to 75
12
76 and over
7

Expected frequency
19
18
21
24
20
18
19
20
16
20
15
15
7

Note: Observed frequencies refer to the sample score in the standardization sample. Expected frequencies
refer to data collected based on a census.

There is also an evaluation sample (165) available, which consists of psychology students
of the University of Vienna and is not representative.

RIS
5. TEST ADMINISTRATION
5.1. Instruction phase
In the first part of the instruction phase, it is explained to the respondent that in this test,
s/he will need to try to calculate using symbols instead of numbers. Each number is
replaced by a certain symbol. The respondent is given a simple equation which is to be
solved correctly.
Before the test phase starts, the respondent is once again instructed to work carefully but
also quickly. The respondent is also instructed not to spend too much time on any one
example, but to go on to the next one.
The test phase starts after this.

5.2. Test phase


The 19 items of the test are presented to the respondent consecutively. Answers are marked
using the input device selected. Once an item has been completed, the respondent has to
press "Next" in order to proceed to the next item. It is not possible to go back to an item
which has already been completed. During the test, no feedback messages are provided
concerning the correctness of the answer.

10

RIS
6. INTERPRETATION OF TEST RESULTS
The main variable "Number of items completed" and the corresponding percentile rank or
T-score is used as a measure of inductive thinking (reasoning). The authors warn that at
present the test should only be used as a screening procedure for a rough classifying (pre-)
selection. Test scores under 10 raw score points indicate low ability with respect to the
skill Reasoning; test sscores over 14 indicate ability which is well above average.

11

RIS
7. REFERENCES
Jger, A. O. (1967). Dimensionen der Intelligenz. Gttingen: Hogrefe.
Lienert, G.A. & Raatz, U. (1994). Testaufbau und Testpraxis. Weinheim: Beltz.
Schmotzer, C., Kubinger, K.D. & Maryschka, C. (1994). Rechnen in Symbolen. Frankfurt:
Swets Test Services.

12

Vous aimerez peut-être aussi