Académique Documents
Professionnel Documents
Culture Documents
Discriminant Analysis
18-2
Chapter Outline
1) Overview 2) Basic Concept 3) Relation to Regression and ANOVA 4) Discriminant Analysis Model 5) Statistics Associated with Discriminant Analysis 6) Conducting Discriminant Analysis i. Formulation ii. Estimation iii. Determination of Significance
iv. Interpretation
v. Validation
18-3
Chapter Outline
7) i. ii. Multiple Discriminant Analysis Formulation Estimation
iii. Determination of Significance iv. Interpretation v. 8) 9) Validation Stepwise Discriminant Analysis Internet and Computer Applications
10) Focus on Burke 11) Summary 12) Key Terms and Concepts
18-4
One
Multiple
Multiple
Multiple
Metric Categorical
Metric Metric
Categorical Metric
18-5
Discriminant Analysis
Discriminant analysis is a technique for analyzing data when the criterion or dependent variable is categorical and the predictor or independent variables are interval in nature. The objectives of discriminant analysis are as follows: Development of discriminant functions, or linear combinations of the predictor or independent variables, which will best discriminate between the categories of the criterion or dependent variable (groups). Examination of whether significant differences exist among the groups, in terms of the predictor variables. Determination of which predictor variables contribute to most of the intergroup differences. Classification of cases to one of the groups based on the values of the predictor variables. Evaluation of the accuracy of classification.
18-6
Discriminant Analysis
When the criterion variable has two categories, the technique is known as two-group discriminant analysis. When three or more categories are involved, the technique is referred to as multiple discriminant analysis. The main distinction is that, in the two-group case, it is possible to derive only one discriminant function. In multiple discriminant analysis, more than one function may be computed. In general, with G groups and k predictors, it is possible to estimate up to the smaller of G - 1, or k, discriminant functions. The first function has the highest ratio of between-groups to within-groups sum of squares. The second function, uncorrelated with the first, has the second highest ratio, and so on. However, not all the functions may be statistically significant.
18-7
D b 's X 's
= = =
The coefficients, or weights (b), are estimated so that the groups differ as much as possible on the values of the discriminant function. This occurs when the ratio of between-group sum of squares to within-group sum of squares for the discriminant scores is at a maximum.
18-8
Canonical correlation. Canonical correlation measures the extent of association between the discriminant scores and the groups. It is a measure of association between the single discriminant function and the set of dummy variables that define the group membership. Centroid. The centroid is the mean values for the discriminant scores for a particular group. There are as many centroids as there are groups, as there is one for each group. The means for a group on all the functions are the group centroids. Classification matrix. Sometimes also called confusion or prediction matrix, the classification matrix contains the number of correctly classified and misclassified cases.
18-9
Discriminant function coefficients. The discriminant function coefficients (unstandardized) are the multipliers of variables, when the variables are in the original units of measurement. Discriminant scores. The unstandardized coefficients are multiplied by the values of the variables. These products are summed and added to the constant term to obtain the discriminant scores. Eigenvalue. For each discriminant function, the Eigenvalue is the ratio of between-group to withingroup sums of squares. Large Eigenvalues imply superior functions.
18-10
calculated from a one-way ANOVA, with the grouping variable serving as the categorical independent variable. Each predictor, in turn, serves as the metric dependent variable in the ANOVA. Group means and group standard deviations. These are computed for each predictor for each group. Pooled within-group correlation matrix. The pooled within-group correlation matrix is computed by averaging the separate covariance matrices for all the groups.
18-11
Standardized discriminant function coefficients. The standardized discriminant function coefficients are the discriminant function coefficients and are used as the multipliers when the variables have been standardized to a mean of 0 and a variance of 1. Structure correlations. Also referred to as discriminant loadings, the structure correlations represent the simple correlations between the predictors and the discriminant function. Total correlation matrix. If the cases are treated as if they were from a single sample and the correlations computed, a total correlation matrix is obtained. Wilks' . Sometimes also called the U statistic, Wilks' for each predictor is the ratio of the within-group sum of squares to the total sum of squares. Its value varies between 0 and 1. Large values of (near 1) indicate that group means do not seem to be different. Small values of (near 0) indicate that the group means seem to be different.
18-12
18-13
Identify the objectives, the criterion variable, and the independent variables. The criterion variable must consist of two or more mutually exclusive and collectively exhaustive categories. The predictor variables should be selected based on a theoretical model or previous research, or the experience of the researcher. One part of the sample, called the estimation or analysis sample, is used for estimation of the discriminant function. The other part, called the holdout or validation sample, is reserved for validating the discriminant function. Often the distribution of the number of cases in the analysis and validation samples follows the distribution in the total sample.
18-14
A Sample Problem
Suppose we want to determine the salient characteristics of families that have already visited a vacation resort during the last two years. Data were obtained from a pretest sample of 42 households. Of these, 30 households were included in the analysis sample and the remaining 12 were part of the validation sample.
18-15
The households that visited a resort during the last two years are coded as 1; those that did not, as 2 (VISIT variable as criterion with two categories).
Annual family income (INCOME) Attitude toward travel (TRAVEL) Importance attached to family vacation (VACATION) Household size (HSIZE) Age of the head of household (AGE)
18-16
No.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
50.2 70.3 62.9 48.5 52.7 75.0 46.2 57.0 64.1 68.1 73.4 71.9 56.2 49.3 62.0
5 6 7 7 6 8 5 2 7 7 6 5 1 4 5
8 7 5 5 6 7 3 4 5 6 7 8 8 2 6
3 4 6 5 4 5 3 6 4 5 5 4 6 3 2
43 61 52 36 55 68 62 51 57 45 44 64 54 56 58
M (2) H (3) H (3) L (1) H (3) H (3) M (2) M (2) H (3) H (3) H (3) H (3) M (2) H (3) H (3)
18-17
No.
16 17 18 19 20 21 22 23 24 25 26 27 28 29 30
18-18
No.
1 2 3 4 5 6 7 8 9 10 11 12
1 1 1 1 1 1 2 2 2 2 2 2
50.8 63.6 54.0 45.0 68.0 62.1 35.0 49.6 39.4 37.0 54.5 38.2
4 7 6 5 6 5 4 5 6 2 7 2
7 4 7 4 6 6 3 3 5 6 3 2
3 7 4 3 6 3 4 5 3 5 3 3
45 55 58 60 46 56 54 39 44 51 37 49
M(2) H (3) M(2) M(2) H (3) H (3) L (1) L (1) H (3) L (1) M(2) L (1)
18-19
18-20
Total
Pooled Within-Groups M atrice s Correlation income travel vacation hs iz e age income 1.000 .197 .091 .089 -.014 travel .197 1.000 .084 -.017 -.197 vacation .091 .084 1.000 .070 .017 hs iz e .089 -.017 .070 1.000 -.043 age -.014 -.197 .017 -.043 1.000
Tes ts of Equality of Group M eans Wilks ' Lambda .453 .925 .824 .657 .954 F 33.796 2.277 5.990 14.636 1.338 df 1 1 1 1 1 1 df 2 28 28 28 28 28 Sig. .000 .143 .021 .001 .257
Contd.
18-21
Wilks ' Lam bda Test of Function(s) 1 Wilks ' Lambda .359 Chi-s quare 26.130 df 5 Sig. .000
Standar dize d Canonical Dis crim inant Function Coe fficie nts Function 1 .743 .096 .233 .469 .209
Pooled w ithin-groups c orrelations betw een discriminating variables and standardized canonic al disc riminant f unctions Variables ordered by absolute s iz e of correlation w ithin func tion.
Contd.
18-22
visit 1 2
Contd.
18-23
visit 1 2 Total
Original
Clas sification Function Coe fficie nts
Count %
visit income travel vacation hs iz e age (Cons tant) 1 .678 1.509 .938 3.322 .832 -57.532 2 .459 1.381 .628 2.218 .768 -36.936
visit 1 2 1 2
18-24
18-25
The interpretation of the discriminant weights, or coefficients, is similar to that in multiple regression analysis. Given the multicollinearity in the predictor variables, there is no unambiguous measure of the relative importance of the predictors in discriminating between the groups. With this caveat in mind, we can obtain some idea of the relative importance of the variables by examining the absolute magnitude of the standardized discriminant function coefficients. Some idea of the relative importance of the predictors can also be obtained by examining the structure correlations, also called canonical loadings or discriminant loadings. These simple correlations between each predictor and the discriminant function represent the variance that the predictor shares with the function. Another aid to interpreting discriminant analysis results is to develop a characteristic profile for each group by describing each group in terms of the group means for the predictor variables.
18-26
18-27
1 2 3 Total
1 2 3 Total
Pooled Within-Groups Correlation Matrix INCOME TRAVEL VACATION INCOME TRAVEL VACATION HSIZE AGE 1.00000 0.05120 0.30681 0.38050 -0.20939 1.00000 0.03588 0.00474 -0.34022
1.00000 -0.02512
1.00000
Contd.
18-28
F
38.00 3.634 1.830 1.944 1.804
Function
1* 2*
Eigenvalue
3.8190 0.2469
Cum Canonical After Wilks' % Correlation Function Chi-square df Significance : 0 0.1664 44.831 10 0.00 93.93 0.8902 : 1 0.8020 5.517 4 0.24 100.00 0.4450 :
* marks the two canonical discriminant functions remaining in the analysis. Standardized Canonical Discriminant Function Coefficients INCOME TRAVEL VACATION HSIZE AGE FUNC 1 1.04740 0.33991 -0.14198 -0.16317 0.49474 FUNC 2 -0.42076 0.76851 0.53354 0.12932 0.52447
Contd.
18-29
Unstandardized canonical discriminant function coefficients FUNC 1 FUNC 2 INCOME 0.1542658 -0.6197148E-01 TRAVEL 0.1867977 0.4223430 VACATION -0.6952264E-01 0.2612652 HSIZE -0.1265334 0.1002796 AGE 0.5928055E-01 0.6284206E-01 (constant) -11.09442 -3.791600
Canonical discriminant functions evaluated at group means (group centroids) Group FUNC 1 FUNC 2 1 -2.04100 0.41847 2 -0.40479 -0.65867 3 2.44578 0.24020
Contd.
18-30
Classification results for cases not selected for use in the analysis Predicted Group Membership Actual Group No. of Cases 1 2 3 Group 1 4 3 75.0% 1 25.0% 0 0.0%
Group
Group
2
3
4
4
0 0.0%
3 75.0%
1 25.0%
3 75.0%
18-31
All-Groups Scattergram
Fig. 18.2
Across: Function 1 Down: Function 2 4.0
1
1
1 1 *1
1 1 1
0.0
23 12 * 2 2 1 2 2 2
3 3* 3 3 3
3
-4.0
18-32
Territorial Map
Fig. 18.3
13 13 Across: Function 1 13 Down: Function 2 13 13 * Indicates a 13 group centroid 13 113 112 3 112233 *1 1 1 2 2 2 2 3 3 * 1 1 2 2* 223 233 1122 2233 1122 223 11122 233 11222 223 1122 233 11122 2233 11122 223 1122 11122 233
8.0
4.0
0.0 -4.0
-8.0 -8.0
-6.0
-4.0
-2.0
0.0
2.0
4.0
6.0
8.0
18-33
Stepwise discriminant analysis is analogous to stepwise multiple regression (see Chapter 17) in that the predictors are entered sequentially based on their ability to discriminate between the groups. An F ratio is calculated for each predictor by conducting a univariate analysis of variance in which the groups are treated as the categorical variable and the predictor as the criterion variable. The predictor with the highest F ratio is the first to be selected for inclusion in the discriminant function, if it meets certain significance and tolerance criteria. A second predictor is added based on the highest adjusted or partial F ratio, taking into account the predictor already selected.
18-34
Each predictor selected is tested for retention based on its association with other predictors selected. The process of selection and retention is continued until all predictors meeting the significance criteria for inclusion and retention have been entered in the discriminant function. The selection of the stepwise procedure is based on the optimizing criterion adopted. The Mahalanobis procedure is based on maximizing a generalized measure of the distance between the two closest groups. The order in which the variables were selected also indicates their importance in discriminating between the groups.
18-35
SPSS Windows
The DISCRIMINANT program performs both twogroup and multiple discriminant analysis. To select this procedure using SPSS for Windows click: Analyze>Classify>Discriminant