J. J. Appl. Sci., 2006: Vol. 8, No.

1, 30-42 2006

2 1

1
2

2005/1/ /538
2005 15 2005 31

.67%

. :

:
Yi * = B0 + X 1*i B1 + X 2*i B2 + ... + X *pi B p + i , i = 1,2,3,4,..., n
p n
:
Y = X B +
* *
..........(1 1)
*
X (n1) Y*
((p+1)1) B ((n(p+1)))
(n1)

:
*
Y -1
. X*,s
2 -2
: [1]
Cov (i , j ) = 0 , i j

30
 .[2] 2 -3
(Covariance) X *
-4
(
. Cov X , = 0 :
*
)
*
: X -5
Rank(X*) = p < n
:
. -
. (Multicollinearity) -

.(Orthogonal)

( Least squares method )

. (1-1)
[3]
: (1-1)
Y = X B + ..........(1 2)

[4 1]

.
:
B ts = X T X ( ) 1
X TY ..........(1 3)

(Unbiasedness) :
: B B
() ()
Bais B = E B B = o
()
EB =B :

( The Variance ) :
:
( )
Var BLS = 2 X T X ( ) 1
..........(1 4)
: BiLS

( )
p
Var BiLS = 2 l j1Vij2 ......, i = 1,2,..., p ..........(1 5)
j =1

(XTX) Vj (XTX) lj

(Mean Squared Error) :

: B

31
J. J. Appl. Sci., 2006: Vol. 8, No. 1, 30-42 2006

() (
MSE B = E B B )2

() (
MSE B = E B E B ( )) + E(B E (B))
2 2

= Var (B ) + (Bais (B ))
2
..........(1 6)

(1-6)
:
( )
MSE BLS = Var BLS ( )
=2 XTX ( ) 1
..........(1 7)

(pp) (XTX)
.[5]

: -
T
(X X)

: -
(XTX)
.[6]
.XTX XTX = 0
. X T X =1

0 < X T X <1
: |XTX| .[7]
p
X T X = li
i =1

32
 : (XTX) -
.

.
: [1]
(Condition index) ;

:
l max
CI j = , j = 1,2,..., p
lj
l max
lj
j

(Condition number) ;

:
l max
CN =
l min
100 30 CN [8] (Belsley)
.

-1

.[9]
-2
[10]
.
. -3

Y
.(n(p+1)) A = [Y : X] X
. (p+1) (ATA)
01...p 0,..., p (ATA)
: 0, 1,..., p
0 j
j = j = 0,1,2,..., p
j

33
J. J. Appl. Sci., 2006: Vol. 8, No. 1, 30-42 2006

j
(p1) j

.0j
0 j < 0.1 0j
j<0.05 j
[11]
: [13 12]
B1
q
BLR = M = d 0 j j1 j ..........(1 8)
B p j =0

:
12 12
q n 2
d = 02 j 1 (Y j Y )
j =0 j =1
. q
T
: (A A) B
p
B LS = f j j ..........(1 9)
j =0
:

, = (Y j Y )
p n
f j = 0 j 1 r =0
1
r 02r
j =1

(ATA)
:[12]
p




j = s +1
j1 0 j j

B LR = B LS p ..........(1 10)
j1 02 j
j = s +1

: (1-10)
p

1

( )
j 0j j
j = s +1
E B LR = B + p

j = s +1
1
j 02 j

:
: [12] BiLR

34
 s
2

j ij
1 2
s
( )
Var BiLR = 2 j1 ij2 s
j =0 ..........(1 11)
j =0
j =0
j1 02 j

:[6]
s 1
2
p

2
j 0 j ij
p j 0 j ij
1
p
( ) 2 + j =s +1
p s
= j ij p
1 2 j =0
MSE B LR s ..........(1 12)
i =1 j =0

i =1
1 2
j 0j

i =1
j 0 j
1 2

j =0
j =s +1

.[14]

(Minitab Under Windows)

.

: X 1* , X 2* ,..., X *p
12
Yi * = B0 + X ij* B j + i , i = 1,2,...,30 ..........(1 13)
j =1

(1

( i = 1,2,...,30 , j = 1,2,...,13 ) Zij

(2
X ij*

: [15]

35
J. J. Appl. Sci., 2006: Vol. 8, No. 1, 30-42 2006

(
X ij* = 1 a 2 )
12
Z ij + aZ i13 ..........(1 14)
Zi13
j
i
[15]

:
X *
ij = [(1 a ) 3]Z ij (a 3)Z i13 , j = 1,2,..., m , i = 1, 2,... 30 .......... (1 15 )
X *
ij = Z ij , j = m + 1,...,12
:
.(1-2-8) (i = 1,2,...,30 , j = 1,2,...,12) Zij
. Zi13
: a
( 0.35 , 0.51 , 0.67 , 0.84 , 0.99 )
m

.
:
(1- (1-14) -1
) (1-15) 15)
(1-14) (
.
: -2
SD( X i )
C.V . = * 100
Mean( X i )
(1-15)
: (1-14)

C.V. C.V.
(1-15) (1-14)
488.354 545.449 0.35
238.274 317.427 0.51
182.845 243.321 0.67
166.438 198.406 0.84
165.755 179.874 0.99

(1-14) (1-15)
.(1-14) (1-15)

36
 (3
(2-2-5) X ij*

: (Unit Length Scaling)

X ij* X *j 1 30
X ij =
Sj
, X =
*
j X ij
30 i =1

( )
30
S 2j = X ij* X *j
2
, j = 1,2,...,12
i =1

(4
T
(X X)
Yi .(Minitab)
:[15]
Yi = 1 X i1 + 2 X i 2 + ... + 12 X i12 + i
:
.(XTX) 1,...,12
: N(0,2) i
i = Z i13 Z i13
Zi13
. Yi .
: .., X12 .Y , X1 ,
12
Yi = X ij B j + i ..........(1 16)
j =1

(5
(1-16)
(1-3)
(1-8)
T
.(A A)

(6
(MSE)
.(1-12) (1-7)

(12) (50)
. (30)
(11)
.

37
J. J. Appl. Sci., 2006: Vol. 8, No. 1, 30-42 2006

(3 1) :
(OLS) -1
.(LR)

(0.99) .1
BLS BLR Var(BLS) Var(BLR) LemdaLS LemdaLR
1 0.181623 0.815114 7277.74 0.64716 2.71043 3.32659
2 0.164133 0.521769 2.10 1.91222 1.72647 1.86926
3 -0.284632 -0.900406 7292.42 0.53397 1.56274 1.58525
4 0.201972 0.394017 1.45 1.22742 1.30606 1.35436
5 0.400140 0.318742 2.15 2.00035 1.08429 1.11356
6 -0.303202 0.298736 1.31 1.24662 0.98922 0.99322
7 0.607151 0.214201 2.53 2.43333 0.73268 0.79433
8 -0.125403 0.217944 1.55 1.32132 0.71285 0.72978
9 -0.032273 0.175928 1.82 1.70957 0.60815 0.66599
10 -0.224505 0.111970 1.53 1.52188 0.39343 0.39381
11 -0.037585 0.049172 1.51 1.50887 0.17361 0.17371
12 -0.369052 -0.001186 1.34 1.32903 0.00007 0.00009
13 0.00004

(0.84) .2
BLS BLR Var(BLS) Var(BLR) LemdaLS LemdaLR
1 -0.173402 0.31282 25.1913 0.87396 2.72720 3.36111
2 -0.161064 -0.54773 2.1200 1.92962 1.74869 1.89778
3 0.302819 0.22772 26.1644 0.53848 1.51744 1.52068
4 -0.220251 0.10802 1.4395 1.25426 1.29662 1.33758
5 -0.403884 0.17804 2.2342 1.97469 1.09181 1.11843
6 0.274508 0.05872 1.0334 1.25784 0.99093 0.99669
7 -0.624447 0.15713 2.5872 2.44186 0.73358 0.77764
8 0.119782 0.03527 1.5469 1.32438 0.70700 0.72054
9 -0.008047 0.00386 1.9077 1.71459 0.60822 0.66930
10 0.256424 0.00192 1.5611 1.71459 0.38930 0.38942
11 0.087738 -0.00645 1.5431 1.53919 0.16989 0.16989
12 0.380679 -0.57604 1.3435 1.33971 0.01932 0.02877
13 0.01216

(0.67) .3
BLS BLR Var(BLS) Var(BLR) LemdaLS LemdaLR
1 0.365879 0.330992 10.2597 0.34046 2.52111 3.60971
2 0.326515 0.473446 1.8070 1.31776 1.69893 3.09570
3 -0.924987 -0.962066 10.8438 0.57128 1.67412 2.77589
4 0.420142 0.395023 1.8455 1.22261 1.45535 2.49233
5 0.189818 0.256677 1.3598 1.50953 1.09651 2.04625
6 0.193301 0.194550 1.6074 1.43210 0.89376 1.77456
7 0.155956 0.223696 1.3096 1.34419 0.80782 1.44689
8 0.279612 0.194044 1.8138 0.61506 0.70858 1.04995
9 -0.109377 0.127314 1.3088 0.41532 0.48661 0.84688
10 -0.124856 0.082790 1.5663 0.48462 0.39057 0.53791
11 0.035717 0.054204 1.5496 0.45541 0.21819 0.40539
12 -0.032851 0.000068 1.5193 0.62018 0.04844 0.04971

38
13 0.00661

39
J. J. Appl. Sci., 2006: Vol. 8, No. 1, 30-42 2006

(0.51) .4
BLS BLR Var(BLS) Var(BLR) LemdaLS LemdaLR
1 0.101073 0.101073 3.01471 3.01471 2.60002 3.12108
2 0.373507 0.373507 2.09678 2.09678 1.80076 1.97390
3 -0.192455 -0.192455 3.37694 3.37694 1.46458 1.47990
4 0.194238 0.194238 1.38700 1.38700 1.25813 1.26057
5 0.279609 0.279609 2.35349 2.35349 1.09330 1.10677
6 0.311411 0.311411 1.29958 1.29958 0.99133 1.00037
7 0.177316 0.177316 2.68695 2.68695 0.74841 0.75606
8 0.238649 0.238649 1.55638 1.55638 0.68178 0.69067
9 0.145498 0.145498 2.05168 2.05168 0.63319 0.67869
10 -0.110560 -0.110560 1.72035 1.72035 0.38487 0.39169
11 -0.002885 -0.002885 1.61596 1.61596 0.23086 0.26588
12 -0.027767 -0.027767 1.36080 1.36080 0.11277 0.16604
13 0.10929

(0.35) .5
BLS BLR Var(BLS) Var(BLR) LemdaLS LemdaLR
1 0.073427 0.073427 2.18090 2.18090 2.49772 2.94942
2 0.231904 0.231904 2.03739 2.03739 1.82539 1.99584
3 -0.022035 -0.022035 2.40815 2.40815 1.45727 1.47935
4 0.093959 0.093959 1.35031 1.35031 1.25584 1.25618
5 0.362108 0.362108 2.31823 2.31823 1.08268 1.09222
6 0.289974 0.289974 1.29919 1.29919 0.98542 0.99540
7 0.257341 0.257341 2.66962 2.66962 0.76536 0.76623
8 0.206043 0.206043 1.56267 1.56267 0.66797 0.70283
9 0.250580 0.250580 2.02554 2.02554 0.65401 0.66083
10 -0.075404 -0.075404 1.80623 1.80623 0.38779 0.42695
11 0.062713 0.062713 1.62977 1.62977 0.29542 0.33433
12 -0.009772 -0.009772 1.36843 1.36843 0.12514 0.21706
13 0.12335

: -2

.6
Corr.
MSE(B) 0.35 0.51 0.67 0.84 0.99
OLS 22.6564 24.5206 36.7905 68.9422 14587.4
LR 22.6564 24.5206 23.9573 19.7427 6736.76

-1
( (A) ).
-2
( (B) )
-3
.

40
(0.35) (0.51) -4

j 0.005 & 0 j < 0.1

.
-5
(0.67) (0.84) (0.99)
.(0.35) (0.51)
-6
.
0.99 0.84 -7
.( (B) )
-8
5 4 3 2
.

(2001) .1
.
(1996) .2
.
3. Gunst R.F., and Mason R.L., (1980), Regression Analysis and Its
Application, Marcel Dekker , New York , U.S.A.
(1988) .4
.
5. Dounald F.M., (1978), Multivariate Statistical Methods, 2nd Ed., MC
Graw-Hill Book Company , Tokyo.
(1988) .6
.
7. Mason R.L., (1986), Latent Root Regression: A Biased Regression
Methodology for Use with Collinear Predictor Variables, Commun Statist
theor. Math., 15(9): 2663.
8. Belsey D.A., and Welsch, (1980), Regression Diagnostics, John Wiley
and Sons.
(2002) .9
. ( )
10. Fisher J.C.,and Mason R.L., (1981), The Analysis of Multicollinear Data
in Criminology, John Wiley and Sons.
11. Draper N.R., and Smith H., (1981), Applied Regression Analysis, 2nd Ed.,
John Wiley and Sons, Canada.
(2003) .12
. ( )

41
J. J. Appl. Sci., 2006: Vol. 8, No. 1, 30-42 2006

13. Naik P.A., Hagerty M.R., and Tsai C., (1999), A New Dimension
Reduction Approach for Data-Rich Marketing Environments: Sliced
Invers Regression, Internet.
14. Naylor T.H., Balintfy J.L., Burdick D.S., and Chu K., (1968), Computer
Simulation Techniques, John Wiley & Sons , New York.
15. Dean W. Wichern, and Gilbert A. Churchill, (1978), A Comparison of
Ridge Estimators, Technometrics, 20(3).

(A)

0.35 0.51 0.67 0.84 0.99

19 23 62.5 143.1 38720 2
15.77 19.3 100 130.3 42000 3
18.3 19.37 122.5 191.3 50891 4
9.8 13.77 181 240.6 91500 5
19 28.79 203 353.6 77973.87 6
17.64 32.39 165 583.3 178877.7 7
17.22 35.4 270 542 163489.4 8
15.63 42.1 481 1000 311228.3 9
18.55 46.5 451 965 1008040 10
19.36 45.6 1020 1177.7 1107410 11
20.76 62 353 1642.85 1199850 12

(B)

0.35 0.51 0.67 0.84 0.99

- - 1 1 1 2
- - 2 2 2 3
- - 3 3 3 4
- - 3 4 4 5
- - 4 5 5 6
- - 5 6 6 7
- - 4 7 7 8
- - 5 8 8 9
- - 5 9 9 10
- - 6 10 10 11
- - 7 11 11 12

42
 A Comparison Between Latent Roots and Least Squares
Regression Methods using Simulation Technique
Mowafak M. Al-Kassab and Moustafa I. Al-Heti
Department of Mathematics, College of Science and Art, Al-Beit University, Jordan
Department of Mathematics, College of Education, Al-Anbar University, Iraq

Abstract: In this study the parameters of the multiple linear regression were estimated
by using least square method and latent roots method depending on the data that was
generated by using Mont Carlos way for simulation of twelve independent variables,
by introducing a new equation to evaluate the correlation between these independent
variables. A comparison was made between these two methods to the best of them,
depending on the mean square error criterion. We found that latent root method was
the best as compared with the least squares one when the correlation between these
independent variables is greater than or equal 67%.

43