See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/10967479

Neural network identification of high-risk suicide patients

Article  in  Medical Informatics and the Internet in Medicine · April 2002

DOI: 10.1080/14639230110119243 · Source: PubMed

CITATIONS READS

7 34

8 authors, including:

Rena Kurs Michael S. Ritsner

Sha'ar Menashe Mental Health Center National Insurance Institute of Israel
42 PUBLICATIONS   1,145 CITATIONS    228 PUBLICATIONS   5,191 CITATIONS   

SEE PROFILE SEE PROFILE

Mendel Shalom
Afeka Tel-Aviv Academic College of Engineering
20 PUBLICATIONS   472 CITATIONS   

SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Cell membranes in schizophrenia View project

Neuroprotective Agents View project

All content following this page was uploaded by Michael S. Ritsner on 22 May 2018.

The user has requested enhancement of the downloaded file.

Eur Psychiatry 2002 ; 17 : 75-81
© 2002 Éditions scientifiques et médicales Elsevier SAS. All rights reserved
S0924933802006314/FLA
ORIGINAL ARTICLE

Validation of the Computerized Suicide Risk Scale – a

backpropagation neural network instrument (CSRS-BP)

I. Modai1*, M. Ritsner1, R. Kurs1, S. Mendel2, A. Ponizovsky2

1
Sha’ar Menashe Mental Health Center, Research Institute for Psychiatric Studies, Bruce Rappaport Faculty of
Medicine, Technion, Haifa, Israel; 2 Rebecca Meirhoff Technical School, Tel Aviv University, Tel Aviv, Israel

(Received 30 March 2000; revised 9 May 2001; accepted 20 June 2001)

Summary – Background. Medically serious suicide attempts have been recognized as the most important predictor of
suicide. The Computerized Suicide Risk Scale based on backpropagation neural networks (CSRS-BP) has been recently
found efficient in the detection of records of patients who performed medically serious suicide attempts (MSSA).
Objectives. To validate the CSRS-BP by: 1) using the CSRS-BP with patients instead of records; 2) comparing the ability
of expert psychiatrists to detect MSSA, using the CSRS checklist; and 3) comparing the results of the Risk Estimator for
Suicide (RES) and the self-rating Suicide Risk Scale (SRS) with the CSRS-BP. Methods. Two hundred fifty psychiatric
inpatients (35 MSSA and 215 non-MSSA) were diagnosed by clinicians using the SCID DSM-IV. Three expert
psychiatrists completed the CSRS checklist, and the RES for each patient, and the patients completed the self-report
SRS assessment scale. The CSRS-BP was run for each patient. Five other expert psychiatrists assessed the CSRS
checklists and estimated the probability of MSSA for each patient. Comparisons of sensitivity and specificity rates
between CSRS-BP, assessment scales and experts were done. Results. Initially, the CSRS-BP, RES, SRS, and experts
performed poorly. Although sensitivity and specificity rates significantly improved (two to four times) after the inclusion
of information regarding the number of previous suicide attempts in the input data set, results still remained insignificant.
Conclusions. The CSRS-BP, which was very successful in the detection of MSSA patient records, failed to detect MSSA
patients in face-to-face interviews. Information regarding previous suicide attempts is an important MSSA predictor, but
remains insufficient for the detection of MSSA in individual patients. The detection rate of the SRS and RES scales was
also poor and could therefore not identify MSSA patients or be used to validate the CSRS-BP. © 2002 Éditions
scientifiques et médicales Elsevier SAS

CSRS / MSSA detection / RES / SRS

INTRODUCTION Intent Scale (SIS) [3], Suicide Death Prediction Scale

In the past few decades, numerous scales for assessing
and predicting suicide have been developed for clinical
and research use. Most widely used are the Scale for
Assessing Suicide Risk (SARS) [30], Los Angeles Sui-
cide Prevention Center Scale (LASPC) [3], Suicide
(SDPS) [3], Index of Potential Suicide (IPS) [33],
Estimator for Suicide Risk (ESR) [25], and Suicide
Risk Scale (SRS) [26]. In a review study [22], the SDPS
and the ESR were considered the most accurate in
predicting individual suicide outcome in a small biased
sample of five clinical cases.

*Correspondence and reprints.

E-mail address: shrmodai@matat.health.gov.il (I. Modai).
76 I. Modai et al.
Psychiatrists assess the suicide risk of patients almost
daily. Experience and intuition of the individual clini-
cian primarily determine how the suicide risk of the
individual patient is evaluated. It has been claimed that
all assessments and predictions of suicide risk ulti-
mately depend on the skill of the clinician [16]. There
are no established or generally accepted guidelines to
help the clinician in this assessment [24]. Goldstein et
al. used stepwise multiple logistic regression in an
attempt to develop a statistical model that would pre-
dict suicide in 1906 patients suffering from affective
disorders. They concluded that it is not possible to
predict suicide even in this high-risk group of inpa-
tients [9].
It has been argued that suicide prediction and assess-
ment tools should always be tempered with clinical
judgment [19, 20, 24]. Motto [24] recommends resolv-
ing conflicts between various instruments or between
instruments and clinical judgment by trusting clinical
judgment. Most scales and assessment procedures are
rigid, static, technical and lack the flexibility or speci-
ficity necessary for suicide prediction of individual per-
sons. On the other hand, clinical judgment may not
always resolve the difficulties associated with suicide
prediction because not every clinician has sufficient
experience and skills necessary to perform this task, and
there are no known definitive risk factors.
Other techniques that may assist in suicide prediction
are self-trained neural networks. Neural networks were
found to be efficient in predicting the diagnosis of acute
myocardial infarction among patients with chest pain
[2], in the differential diagnosis of back pain [21], and
in radiological diagnosis [4]. One such system, the
Computerized Suicide Risk Scale, is based on back-
propagation (CSRS-BP) and trained to detect MSSA
patient files [23]. The input set was comprised of
significant demographic, clinical, history and social
variables. The backpropagation neural network is
trained by comparing the output with the actual target
and retrospectively corrects the weights associated with
each variable. It stabilizes only when the gap between
the output and the target is near zero [8, 31].
The CSRS-BP demonstrated the capacity to dis-
criminate between clinical records of psychiatric
patients with and without a history of medically serious
suicide attempts (MSSA) [23].
MSSA is defined as a self-destructive act causing
serious physical harm that requires medical hospitaliza-
tion [6]. MSSA are caused by violent and highly lethal
methods, such as severe burns, hanging, jumping from
heights and drug overdose. People who had MSSA and
those who completed suicide may be considered to
belong to a similar cohort since the outcome of a MSSA
would most likely have been lethal without urgent
medical intervention. Also, MSSA patients have been
considered at high risk for subsequent suicides com-
pared with those who made less serious attempts [11,
13, 15, 17, 29]. For instance, in Rosen’s [28] cohort of
886 suicide attempters, 6.45% of MSSA category com-
pared with 3.1% of non-MSSA category killed them-
selves during the 5-year follow-up. Other studies show
similar figures for MSSA and those attempters who go
on to complete suicide during an 8–10 year follow-up
[5]. Since detection of MSSA can serve as a target for
both clinicians and neural networks for the definition
of suicide risk [11] it has been hypothesized that an
instrument that could perfectly detect MSSA patients
could stimulate experts to reevaluate their assessment of
suicidal risk for the individual patient.
The objective of the present study was to validate the
CSRS-BP on patients interviewed face-to-face, and
cross-validate it through comparison with the Risk
Estimator for Suicide, self-rating Suicide Risk Scale,
and experts’ predictions based on the CSRS checklist.
METHODS
Three experienced clinicians in face-to-face interview
using the Structured Clinical Interview for DSM-IV
Axis I Disorders, Patient’s Edition [7] screened psychi-
atric patients consecutively admitted to Sha’ar Mena-
she MHC during 1999. Only 250 inpatients fulfilling
DSM-IV criteria for psychiatric disorders [1] and able
to provide written informed consent were enrolled.
The study was approved by the hospital’s Helsinki
Committee. The discriminating variable for this study
was the seriousness of suicide attempts, determined by
the presence or absence of medically serious suicide
attempts across the life span.
The three expert psychiatrists completed the CSRS
checklist and the RES for each patient, and the patients
completed the self-report SRS assessment scale. The
CSRS-BP was run for each patient. Five other expert
psychiatrists assessed the CSRS checklists and estimated
the probability of MSSA for each patient. Comparisons
of sensitivity and specificity rates between CSRS-BP,
assessment scales and experts were done. Three months
later the number of previous suicide attempts were
added to the CSRS checklist and the same five expert
psychiatrists re-assessed the checklists and reevaluated
Eur Psychiatry 2002 ; 17 : 75–81
 Validation of the CSRS-BP
the probability of MSSA. Comparisons of sensitivity
and specificity rates between CSRS-BP and all assess-
ment were repeated with the additional variable of the
number of previous suicide attempts. Patient anonym-
ity was maintained throughout all evaluations.
In addition, objective severity of mental disorder, the
intensity of experienced psychological distress, and level
of perceived social support were evaluated by the Posi-
tive and Negative Syndrome Scale (PANSS) [14], and
by two standardized self-report questionnaires: the Tal-
bieh Brief Distress Inventory (TBDI) [27], and Multi-
dimensional Scale of Perceived Social Support (MSPSS)
[32].
SUBJECTS
The sample was 72.8% male, with the mean age of 40.8
years (SD = 11.8; range = 18–82). The mean length of
education was 9.6 years (SD = 3.2). Of the 250 inpa-
tients, 192 (76.8%) were diagnosed with schizophrenia
or schizoaffective disorder, three (1.2%) with delu-
sional disorders, 13 (5.2%) with mood disorders, and
42 (16.8%) with personality disorders. The total mean
number of previous hospitalizations was 6.5 (SD = 7.4).
The 250 patients reported a total of 346 previous
suicide attempts (PSA). One hundred fifty-four patients
(61.6%) had no history of PSA, while 96 patients
(38.4%) reported from one to 20 PSA. Of the types of
current suicide attempt, 13.2% (n = 33) were by drug
overdose, 7.6% (n = 19) were by hanging, 5.6%
(n = 14) by cutting or stabbing, 4.4% (n = 11) were by
jumping from heights, 2% (n = 4) were by gun, 1.2%
(n = 3) were by burning, and 4% (n = 10) were by other
means.
Instruments
Computerized Scale for Risk of Suicide – back-
propagation neural network
The CSRS-BP, a trained backpropagation neural net-
work program, based on a 43-item CSRS checklist
input data set, was shown to differentiate between
clinical records of MSSA and non-MSSA patients.
Briefly, the CSRS-BP was repeatedly trained and tested
with hospital records of MSSA and non-MSSA patients
until the best configuration was achieved. Optimal
network configuration has been described by the fol-
lowing parameters: architecture: 43-14-2; overall error
level: 0.001; minimum node tolerance: 0.15; learning
rate: 0.0015; momentum coefficient: 0.9, and maxi-
Eur Psychiatry 2002 ; 17 : 75–81
77
mum iterations: 1000. As a result, the CSRS-BP cor-
rectly recognized 93.8% of 49 medical records of the
patients who had a history of MSSA, and 89.8% of 49
records of patients with no history of previous suicide
attempts or thoughts. The total successful recognition
rate was 91.8% (92% true positives and 95.6%, true
negatives). The trained CSRS-BP provides the recogni-
tion of MSSA patient medical records at probability
level of zero to 100%. In the present investigation, the
CSRS-BP was run on each of the 250 patients.
CSRS Checklist
A 43-item CSRS checklist was completed by three
clinicians based on the patient’s answers in a semi-
structured interview. All items of the checklist believed
to be either putative or protective factors of suicidal risk
were derived from previous studies. These included
demographic characteristics: gender, age, ethnicity, reli-
gious affiliation; marital, socioeconomic, and employ-
ment status; years of education, and number of children.
Social characteristics included living alone, number of
social contacts, presence of actual stress factors, and
threat of financial loss. Psychiatric characteristics
included the presence or absence of persecution
thoughts, other delusions, hallucinations, verbal and
motor aggression, resistant depression, alcoholism, drug
abuse or dependence, a history of physical and sexual
abuse, and legal problems, psychopathology in family,
murder in family, recent loss in the family. Age at onset
of mental disorder, the number and duration of lifetime
and current hospitalizations, functioning level, recent
separation from therapist, and compliance to treatment
were also encompassed. Medical history included the
presence or absence of physical illness or disability, and
sleep and weight problems. Specific suicidal items con-
cern the presence or absence of current suicide ideation
and a history of suicide in the patient’s first- and
second-degree relatives.
Self-Reported Suicide Risk Scale (SRS)
The SRS is a well-known self-administered screening
instrument for assessing the risk of suicide attempts,
but not future suicides for a wide variety of patients
[26]. It consists of 24 items worded as questions to be
answered simply as ‘yes’ or ‘no’. The percent of ‘yes’
responses is calculated, with the greater percentage
indicating the higher likelihood of attempted suicide.
In this study, the cutoff point of 8 was utilized to
discriminate between MSSA and non-MSSA patients
as recommended by the authors [26] as an optimum
78 I. Modai et al.
producing both the sensitivity and specificity rates of
approximately 68%.
Risk Estimator for Suicide (RES)
The RES was developed by Motto et al. [25] to estimate
the risk of completed suicide within 2 years in persons
ages 18 to 70 who are known to be at some risk for
suicide. It is an easily administered, 15-item, paper-
and-pencil scale estimating the degree of risk rather
than predicting a specific outcome in individual cases.
The instrument categorizes individuals into five levels
of potential risk for suicide (very low = < 1%,
low = 1–2.5%, moderate = 2.5–5%, high = 5–10%,
and very high = > 10%). We used the cutoff point of 5
to discriminate between MSSA and non-MSSA patients
because that rate was reported by the authors as crite-
rion for ‘high risk’ of suicide [25].
Statistical analysis
The NCSS-2000 PC program [10] was used for all
analyses. Differences between groups on continuous
variables were evaluated with two-tailed t-tests. Values
are given as mean and standard deviation (SD). Differ-
ences in frequency of categorical variables were exam-
ined by the χ2 tests with Yates correction, and two-
sample proportion test where relevant. Discriminant
analysis was performed separately for the data set of
each of the instruments to determine how a set of
variables would discriminate between the MSSA and
non-MSSA groups. All variables in each set were entered
simultaneously into a discriminant analysis to estimate
its sensitivity and specificity rates.
RESULTS
Of the 346 suicide attempts, 35 (10.1%) were defined
as MSSA documented in hospital charts. By this crite-
rion, all patients were divided into two groups: those
who had a history of MSSA (n = 35) and those with no
such history, non-MSSA (n = 215).
Table I shows basic characteristics of the MSSA and
non-MSSA groups. Both groups were comparable for
all variables, except for the number of past suicide
attempts, which significantly prevailed among the
MSSA group compared with the non-MSSA group
(3.88, SD = 3.8 vs. 0.98, SD = 2.8; t = 4.31,
P < 0.001). No significant differences between the two
groups were found on objective severity measures of
acute and chronic psychopathology, such as positive
and negative syndromes, and general psychopathology
(PANSS). Likewise, both groups did not differ signifi-
cantly on subjective ratings of psychological distress
(TBDI) and levels of perceived social support (MSPSS).
Table II shows that all results were lower than chance.
The CSRS-BP did not significantly differ from the RES
in recognizing MSSA patients (sensitivity of 31.4% vs.
14.3%, two sample proportion test, z-value = 1.71,
NS), but the scale was significantly more efficient in
recognizing the non-MSSA patients than the RES
(specificity of 23.7% vs. 5.6%, z = 5.32, P < 0.001).
However, compared with the self-rating SRS, the
CSRS-BP was significantly less sensitive (31.4% vs.
54.3%, z = 1.93, P < 0.05) and considerably less spe-
cific (23.7% vs. 47.9%, z = 5.23, P < 0.001).
Overall, there were not significant differences in sen-
sitivity rates between the CSRS-BP and four of the five
clinical experts; only one expert performed poorer than
the scale did (11.4% vs. 31.4%, z = 2.04, P < 0.05).
Specificity rates of the instrument were significantly
higher than that of three clinical experts, but these were
similar to the remaining experts’ rates (23.7% vs.
22.3%, z = 0.34, NS, and 23.7% vs. 19.5%, z = 0.29,
NS, respectively).
Table III shows that sensitivity and specificity rates
increased (two to four times) for all instruments and
experts, after addition of the information regarding the
number of previous suicide attempts, increasing beyond
chance, except for the fourth and fifth experts. The
differences in sensitivity between performances of the
assessment instrument and three of five experts were
statistically insignificant, but significantly poorer per-
formance was observed for two experts than for the
CSRS-BP. Regarding specificity, the RES and all but
one expert yielded significantly higher rates than the
CSRS-BP.
DISCUSSION
The major findings of the present study are: 1) though
the CSRS-BP was successful in the detection of MSSA
patient files [23], it failed dramatically in the detection
of MSSA patients; and 2) when the number of PSA, a
proven important suicide predictor, was added to the
input set, detection increased to more than chance, but
remained poor.
In general, CSRS-BP did not show any advantages
over other scales, it had less specificity than the RES and
Eur Psychiatry 2002 ; 17 : 75–81
 Validation of the CSRS-BP 79

Table I. Characteristics of MSSA versus non-MSSA patients.

Variable MSSA (n = 35) Non-MSSA (n = 215) Significance test*
Age, yr. 39.1 (9.7) 41.1 (12.1) T = 1.07, NS
Gender:
Male 22 (62.9) 160 (74.4) χ2 = 2.03, NS
Female 13 (37.1) 55 (25.6)
Marital status:
Unmarried 28 (80) 169 (78.6) χ2 = 0.035, NS
Married 7 (20) 46 (21.4)
Children, no. 1.086 (1.96) 1.004 (1.68) T = 0.23, NS
Education, yr. 9.29 (2.94) 9.63 (3.24) T = 0.63, NS
Age at onset, yr. 25.4 (7.8) 24.74 (10.4) T = 0.44, NS
Prior hospitalization, no. 8.49 (11.6) 6.16 (6.42) T = 1.15, NS
Hospitalization length, mo. 33.9 (47.7) 16.7 (5.7) T = 0.14, NS
Past suicide attempt 3.88 (3.8) 0.98 (2.8) T = 4.31, P < 0.001
PANSS:
Positive syndrome 16.1 (5.9) 16.7 (5.7) T = 0.53, NS
Negative syndrome 25.4 (7.7) 24.5 (5.7) T = 0.51, NS
General psychopathology 44.1 (10.4) 41.5 (9.5) T = 1.42, NS
Total score 85.7 (21.8) 82.7 (18.4) T = 0.44, NS
Distress severity (TBDI) 1.30 (1.0) 1.17 (0.8) T = 0.70, NS
Social support (MSPSS) 49.4 (24.0) 52.7 (18.1) T = 0.78 NS
Functioning level (GAF) 52.3 (10.3) 51.2 (11.5) T = 0.56, NS
χ2 tests or t-tests are shown where relevant; number (%) or means (SD) are shown where relevant.

showed no significant differences in sensitivity, even
when the known predictor of prior suicide attempts was
added.
The difference in performance of CSRS-BP detecting
MSSA files in comparison to detection of MSSA
patients can be explained either by the difference
between validation samples or by the differences
between source data for training (files) versus the source
of the data for validation (patients).
Although neural networks were found in some inves-
tigations effective for diagnosis of myocardial infarc-
tions, back pain and radiological pictures [2, 4, 21],
they actually failed to detect the most reliable suicide
predictor, MSSA [12]. This finding further supports
the claim that based on present knowledge [9] and
regardless of methodology, it is not possible to predict
suicide in the individual patient. However, this fact is
unacceptable to the courts and to patients, who con-
tinue to expect psychiatric specialists to accurately pre-
dict suicide.
Special attention should be given to the fact that
generally speaking, expert clinicians were no more suc-
cessful than neural networks or suicidal risk assessment
scales. The failure of SRS and RES can be explained by

Table II. Sensitivity and specificity estimates for medically serious suicide attempt without information regarding prior suicide attempts:
CSRS-BP versus other scales and five clinical experts
Estimates Scale Expert
CSRS-BP SRS RES First Second Third Fourth Fifth
p1 p2 p3 p4 p5 p6 p7
Sensitivity MSSA (n = 35) 11 (31.4) 19 (54.3)* 5 (14.3) 12 (34.3) 6 (17.1) 4 (11.4)* 7 (20.0) 7 (20.0)
Specificity non-MSSA (n = 215) 51 (23.7) 103 (47.9)*** 12 (5.6)*** 42 (19.5) 16 (17.4)*** 48 (22.3) 25 (11.6)*** 6 (12.7)***
CSRS-BP: the Computerized Suicide Risk Scale – backpropagation neural network; SRS: the Suicide Risk Scale; RES, the Estimator for Risk
Suicide. Numbers (percentage) are shown. p1: CSRS-BP vs. SRS, z-value = 1.93 and 5.23; p2- CSRS-BP vs. RES, z-value = 1.71 and 5.32;
p3: CSRS-BP vs. 1st expert, z-value = 0.78 and 0.29; p4: CSRS-BP vs. 2nd expert, z-value = 0.16 and 4.65; p5: CSRS-BP vs. 3rd expert,
z-value = 2.04 and 0.34; p6: CSRS-BP vs. 4th expert, z-value = 1.09 and 3.29; and p7: CSRS-BP vs. 5th expert, z-value = 1.09 and 6.40.
*: P < 0.05; **: P < 0.01; ***: P < 0.001.

Eur Psychiatry 2002 ; 17 : 75–81

80 I. Modai et al.

Table III. Sensitivity and specificity estimates for medically serious suicide attempts after the inclusion of information on past suicide attempts:
CSRS-BP versus other scales and five clinical experts.
Estimates Scale Expert
CSRS-BP SRS RES First Second Third Fourth Fifth
p1 p2 p3 p4 p5 p6 p7
Sensitivity MSSA (n = 35) 23 (65.7) 25 (71.4) 20 (57.2) 19 (54.3) 28 (80.0) 21 (60.0) 11 (31.4)** 3 (8.6)***
Specificity non-MSSA (n = 215) 146/199 (73.4) 155 (72.0) 195 (90.7)*** 180 (83.7)** 168 (78.1) 176 (81.9)* 184 (85.6)** 214 (99.5)***
CSRS-BP: the Computerized Suicide Risk Scale – backpropagation neural network; SRS: the Suicide Risk Scale; RES: the Risk Estimator for
Suicide. Numbers (percentage) are shown. Two-sample proportion test, z-value for sensitivity and specificity are shown. p1: CSRS-BP vs. SRS,
z-value = 0.51 and 0.29; p2: CSRS-BP vs. RES, z-value = 0.73 and 4.62; p3: CSRS-BP vs. 1st expert, z-value = 0.32 and 2.57; p4: CSRS-BP
vs. 2nd expert, z-value = 1.34 and 1.13; p5: CSRS-BP vs. 3rd expert, z-value = 0.49 and 2.08; p6 : CSRS-BP vs. fourth expert, z-value = 2.86
and 3.09; p7: CSRS-BP vs. 5th expert, z-value = 4.94 and 8.00. *: P< 0.05, **:P<0.0 1, *** P < 0.001

the fact that their prediction aim (risk of complete
suicide within 2 years, risk of immediate suicide
attempt) are not focused on MSSA detection. Never-
theless, it remains unclear why the self-rating SRS, a
first-person instrument, was no more accurate than the
other instruments.
Low capacity of expert suicide prediction is most
probably based on insufficient clinical knowledge and
the lack of a definitive valid predictor (which could be
genetic or biological).
The detection of MSSA patients is of primary impor-
tance in assessing suicide risk because the prior MSSA
increases the risk of future complete suicide: nearly
10% of those patients will complete suicide during the
3–10 years following the serious attempt [6]. Identifi-
cation of MSSA patients is of special importance in
dissimulating patients who for various reasons hide
morbid information from the clinician. Therefore, fur-
ther modification of the instrument is needed, either
modification of the variables included in the input set
or adjustment of the mathematical algorithms. Since
fuzzy logic algorithms [18] learn from descending gra-
dients rather than from the all-or-none principle, they
have better potential to fine-tune results and seem to be
preferable for further investigations.
REFERENCES
1 American Psychiatric Association. Diagnostic and Statistical
Manual of Mental Disorders, 4th edition. Washington DC:
APA; 1994.
2 Baxt GW. Application of artificial neural networks to clinical
medicine. Lancet 1995 ; 346 : 1135-8.
3 Beck AT, Resnik LP, Lettieri DJ, Eds. The prediction of suicide.
Philadelphia: Charles Press; 1974.
4 Boone JM, Sigillito VG, Shaber GS. Neural network in radiol-
ogy: an introduction and evaluation in a signal detection task.
Med Phys 1990 ; 17 : 234-41.
5 Cullberg J, Wasserman D, Stefensson CG. Who commits sui-
cide after a suicide attempt? An 8 to 10 year follow up in a
suburban catchment area. Acta Psychiatr Scand 1988 ; 77 :
598-603.
6 Elliott AJ, Pages KP, Russo J, Wilson LG, Roy-Byrne PP. A
profile of medically serious suicide attempts. J Clin Psychiatry
1996 ; 57 : 567-71.
7 First MB, Spitzer RL, Gibbon M, Williams JBW. Structured
Clinical Interview for DSM-IV Axis I Disorders, Patient Edi-
tion (SCID-P), version 2: New York: Stat Psychiatric Institute,
Biometrics Research; 1995.
8 Garson GD. Interpreting neural network connecting weights.
AI Expert April 1991 : 47-51.
9 Goldstein RB, Black DW, Nasrallah A, Winokur G. The pre-
diction of suicide, sensitivity, specificity, and predictive value of
a multivariate model applied to suicide among 1906 patients
with affective disorders. Arch Gen Psychiatry 1991 ; 48 : 418-
22.
10 Hintze JL. NCSS 6.0. Statistical System for Windows. User’s
guide. Number Cruncher Statistical Systems: Kaysville, Utah;
1998.
11 Isometska EL, Lonnqvist JK. Suicide attempts preceding com-
pleted suicide. Br J Psychiatry 1998 ; 173 : 531-5.
12 Kaplan RD, Kottler DB, Frances AJ. Reliability and rationality
in the prediction of suicide. Hosp Community Psychiatry
1982 ; 33 : 212-5.
13 Ivarsson T, Larsson B, Gillberg C. A 2–4 year follow up of
depressive symptoms, suicidal ideation, and suicide attempts
among adolescent psychiatric inpatients. Eur Child Adolesc
Psychiatry 1998 ; 7 : 96-104.
14 Kay SR, Opler LA, Lindenmayer JP. Reliability and validity of
the positive and negative syndrome scale for schizophrenia.
Psychiatry Res 1988 ; 23 : 99-110.
15 Krupinski M, Fischer A, Grohmann R, Engel R, Hollweg M,
Moller HJ. Risk factors for suicides of inpatients with depressive
psychoses. Eur Arch Psychiatry Clin Neurosci 1998 ; 248 :
141-7.
16 Leenaars AA. Clinical evaluation of suicide. Psychiatry Clin
Neurosci 1995 ; 49 (Suppl 1) : s61-8.
17 Leon AC, Friedman RA, Sweeney JA, Brown RP, Mann JJ.
Statistical issues in the identification of risk factors for suicidal
behavior: the application of survival analysis. Psychiatry Res
1990 ; 31 : 99-108.
18 Lin CJ. A fuzzy adaptive learning control network with on-line
structure and parameter learning. Int J Neural Syst 1996 ; 7 :
569-90.

Eur Psychiatry 2002 ; 17 : 75–81

 Validation of the CSRS-BP
19 Litman RE. Predicting and preventing hospital and clinic sui-
cides. In: Maris RW, Berman AL, Maltsberger JT, Yufit RI, Eds.
Assessment and prediction of suicide. New York: Guilford
Press; 1992. p. 448-66.
20 Maltsberger JT. The psychodynamic formulation: aAn aid in
assessing suicide risk. In: Maris RW, Berman AL, Malts-
berger JT, Yufit RI, Eds. Assessment and prediction of suicide.
New York: Guilford Press; 1992. p. 25-49.
21 Mann NH, Brown MD. Artificial intelligence in the diagnosis
of low back pain. Orthop Clin N Am 1991 ; 22 : 303-14.
22 Maris RW, Berman AL, Maltsberger JT. Summary and conclu-
sions: what have we learned about suicide assessment and pre-
diction? In: Maris RW, Berman AL, Maltsberger JT, Yufit RI,
Eds. Assessment and prediction of suicide. New York: Guilford
Press; 1992. p. 640-53.
23 Modai I, Valevski A, Solomish A, Kurs R, Hines IL, Ritsner M,
et al. Neural network detection of files of suicidal patients and
suicidal profiles. Med0 Inform Internet Med 1999 ; 24 : 249-
56.
24 Motto JA. An integrated approach to estimating suicide risk. In:
Maris RW, Berman AL, Maltsberger JT, Yufit RI, Eds. Assess-
ment and prediction of suicide. New York: Guilford Press;
1992. p. 625-39.
25 Motto JA, Heilborn DC, Juster RP. Development of a clinical
instrument to estimate suicide risk. Am J Psychiatry 1985 ; 14 :
680-6.
Eur Psychiatry 2002 ; 17 : 75–81
View publication stats
81
26 Plotchik R, Van Praag HM, Conte HR, Picard S. Correlates of
suicide and violence risk 1: the suicide risk measure. Compr
Psychiatry 1989 ; 30 : 296-302.
27 Ritsner M, Rabinowitz J, Slyuzberg M. The Talbieh Brief
Distress Inventory: a brief instrument to measure psychological
distress among immigrants. Compr Psychiatry 1995 ; 36 :
448-53.
28 Rosen DH. The serious suicide attempt. Five-year follow-up
study of 886 patients. JAMA 1976 ; 235 : 2105-9.
29 Stephens JH, Pascal R, McHugh PR. Suicide in patients hospi-
talized for schizophrenia: 1913–1940. J Nerv Ment Dis 1999 ;
8 : 671-7.
30 Tuckman J, Youngman WF. A scale for assessing suicide risk of
attempted suicides. J Clin Psychol 1968 ; 24 : 17-9.
31 Wasserman PD. Neural computing theory and practice. New
York: Wan Nostard Reinhold, ANZA Research Inclinations;
1989.
32 Zimet GD, Dahlem NW, Zimet SG, Farley GK. The multidi-
mensional scale of perceived social support. J Pers Assess 1988 ;
52 : 30-41.
33 Zung W. Index of potential suicide. In: Beck AT, Resnik LP,
Lettieri DJ, Eds. The Prediction of Suicide. Philadelphia:
Charles Press:; 1974.