Académique Documents
Professionnel Documents
Culture Documents
Sandip Kumar
Lahiri
December 2008
Contact
Author
Notify Me
of New Work
(2008)
INTRODUCTION
Conventionally, two approaches namely phenomenological (first principles) and empirical, are
employed for slurry flow modeling. The advantages of
a phenomenological model are: (i) it represents physicochemical phenomenon underlying the process explicitly and (ii) it possesses extrapolation ability. Owing to
the complex nature of many multiphase phase slurry
processes, the underlying physico-chemical phenomenon is seldom fully understood. Also, collection
of the requisite phenomenological information is
costly, time-consuming and tedious, and therefore development and solution of phenomenological process
models poses considerable practical difficulties. These
difficulties necessitate exploration of alternative modeling formalisms.
In the last decade, artificial neural networks
(ANNs) and more recently support vector regression
(SVR) have emerged as two attractive tools for
nonlinear modeling especially in situations where the
development of phenomenological or conventional
regression models becomes impractical or cumbersome. In recent years, support vector regression (SVR)
[1-3] which is a statistical learning theory based machine learning formalism is gaining popularity over
ANN due to its many attractive features and promising empirical performance.
The main difference between conventional ANNs
and support vector machines (SVM) lies in the risk
minimization principle. Conventional ANNs implement the empirical risk minimization (ERM) principle
to minimize the error on the training data, while SVM
adheres to the Structural Risk Minimization (SRM)
principle seeking to set up an upper bound of the generalization error [1].
This study is motivated by a growing popularity
of support vector machines (SVM) for regression
problems. Although the foundation of the SVR paradigm was laid down in mid 1990s, its chemical engineering applications such as fault detection [4, 5] have
emerged only recently.
(1)
where w denotes the weight vector; b refers to a constant known as bias; f(x) denotes a function termed
feature, and w , x represents the dot product in the
feature space, l, such that : x l , w l . The basic
concept of support vector regression is to map
nonlinearly the original data x into a higher dimensional feature space and solve a linear regression
problem in this feature space [1-3].
The regression problem is equivalent to minimize
the following regularized risk function:
R( f ) =
1
1
L ( f ( xi ) yi ) + w
n i =1
2
(2)
where
f ( x ) y , for f ( x) y x
L( f ( x ) y ) =
0,
otherwise
(3)
Equation (3) is also called -insensitive loss function.
This function defines a -tube. If the predicted value is
within the -tube, the loss is zero. If the predicted
value is outside the tube, the loss is equal to the magnitude of the difference between the predicted value
and the radius of the tube. is a precision parameter
representing the radius of the tube located around the
regression function (see Fig. 1); the region enclosed
by the tube is known as -intensive zone.
n
1 2
w + C ( + i* )
2
i =1
(4)
yi w , xi b + i
subject to w , xi + b yi + i*
(5)
*
i , i 0
where the constant C > 0 stands for the penalty degree of the sample with error exceeding . Two positive slack variables i , i* represent the distance from
actual values to the corresponding boundary values of
-tube. The SVR fits f(x) to the data in a manner such
that: (i) the training error is minimized by minimizing
i , i* and, (ii) w2 is minimized to increase the flatness
of f(x) or to penalize over complexity of the fitting
function. A dual problem can then be derived by using
the optimization method to maximize the function:
maximize
1 n
( i* ) j *j
2 i , j =1 i
i =1
i =1
)( x , x )
i
( i + i* ) + yi ( i i* )
subject to
(i i* ) = 0
i =1
*
i , i are
(6)
and 0 i , i* C (7)
where
Lagrange multipliers. Owing to the
specific character of the above-described quadratic
programming problem, only some of the coefficients,
( i* i ) are non-zero and the corresponding input
vectors, xi, are called support vectors (SVs). The SVs
can be thought of as the most informative data points
that compress the information content of the training
set. The coefficients and * have an intuitive interpretation as forces pushing and pulling the regression
estimate f(xi) towards the measurements, yi.
The SVM for function fitting obtained by using
the above-mentioned maximization function is then
given by
n
f ( x ) = ( i i* ) xi , x + b
(8)
i =1
Figure 1 A schematic diagram of support vector regression using -sensitive loss function [4]
data points;
points outside tube; support vectors; fitted
by SVR
f ( x ) = ( i i* ) K xi , x + b
(9)
i =1
2.2
It is well known that SVM generalization performance (estimation accuracy) depends on a good
There are different measures by which SVM performance is assessed, but validation and leave-one-out
error estimates are the most commonly used one. Here
we divide the total available data as training data
(75% of data) and test data (25 % data chosen randomly) on which the SVR performance is assessed.
The statistical analysis of SVR prediction is
based on the following performance criteria:
(1) The average absolute relative error (A) on test
data should be minimized:
A=
2
1 N
ypred,i yexp,i ) / yexp,i A
(
N 1 1
1 N ypred yexp
y
N 1
exp
R=
i =1
Name of kernel
Case 1
linear
Case 2
polynomial
Case 3
Case 4
k = uv
k = (uv + 1)1p ;
p1 is degree of polynomial
( u v )( u v )
2 p2
k =e
;
p1 is width of radial basis function ()
( u v )( u v )
2 p2
k =e
;
p1 is width of exponential radial basis function ()
Case 5
spline
Case 6
B spline
1
1
z = 1 + uv + uv min(u , v) min(u , v)3 ;
2
6
k = prod( z ) ;
z=0 ;
for r = 0 : 2 * ( p1 + 1)
2( p1 +1)
end
k = prod( z ) ;
p1 is degree of B spline
Case
Case 1
Case 2
The stepwise procedure for the GA-based optimization of a SVR model is provided in the flowchart
in Fig. 2.
No
Author
No
Author
11
13
14
Ma (1987) [31]
15
16
17
18
19
10
Figure 2
5.2.1
Collection of data
As mentioned earlier, over the years researchers
Table 4
Pipe diameter
D/m
Particle diameter
dp/m
Liquid density
L/kgm 3
Solids density
S/kgm 3
Liquid viscosity
/mPas
Velocity U
/ms 1
Solids concentration
(volume fraction) f
0.019-0.495
38.3-13000
1000-1250
1370-2844
0.12-4
0.86-4.81
0.014-0.333
xnormal =
2x xmax xmin
xmax xmin
Table 5 (Continued)
Input
Output
dp/dz/Pam 1
D/
cm
dp/
m
2.54
40.1
1.00
2.84
0.85
1.06
0.07
560.00
2.54
40.1
1.00
2.84
0.85
1.13
0.07
600.00
2.54
40.1
1.00
2.84
0.85
1.15
0.07
600.00
1.90
40.1
1.00
2.84
0.85
1.10
0.07
1250.00
1.90
40.1
1.00
2.84
0.85
1.50
0.07
1500.00
1.90
40.1
1.00
2.84
0.85
1.70
0.07
1700.00
5.26
38.3
1.00
2.33
1.00
1.11
0.11
294.10
5.26
38.3
1.00
2.33
1.00
3.01
0.11
1651.30
5.26
38.3
1.00
2.33
1.00
4.81
0.11
3822.90
5.26
38.3
1.00
2.33
1.00
1.33
0.31
542.90
5.26
38.3
1.00
2.33
1.00
3.12
0.31
2352.60
5.26
38.3
1.00
2.33
1.00
4.70
0.31
4727.70
20.85
190.0
1.00
1.37
1.14
2.59
0.33
266.50
20.85
190.0
1.00
1.37
1.14
2.34
0.33
226.30
20.85
190.0
1.00
1.37
1.14
2.01
0.33
177.30
20.85
190.0
1.00
1.37
1.14
1.78
0.33
147.00
20.85
190.0
1.00
1.37
1.14
1.59
0.32
123.40
20.85
190.0
1.00
1.37
1.14
1.37
0.33
99.90
5.15
165.0
1.00
2.65
1.00
1.66
0.07
666.20
5.15
165.0
1.00
2.65
1.00
3.78
0.09
2449.20
5.15
165.0
1.00
2.65
1.00
1.66
0.17
901.30
5.15
165.0
1.00
2.65
1.00
4.17
0.19
3428.90
5.15
165.0
1.00
2.65
1.00
1.66
0.27
1136.40
5.15
165.0
1.00
2.65
1.00
4.33
0.29
4408.10
26.30
165.0
1.00
2.65
1.00
2.90
0.09
261.60
26.30
165.0
1.00
2.65
1.00
3.50
0.09
334.10
26.30
165.0
1.00
2.65
1.00
2.90
0.18
305.70
26.30
165.0
1.00
2.65
1.00
3.50
0.17
382.10
26.30
165.0
1.00
2.65
1.00
2.90
0.26
355.60
26.30
165.0
1.00
2.65
1.00
3.50
0.26
453.60
26.30
165.0
1.00
2.65
1.00
2.90
0.33
414.40
26.30
165.0
1.00
2.65
1.00
3.50
0.32
526.10
49.50
165.0
1.00
2.65
1.00
3.16
0.09
143.00
L/
S/
/
U/
Output
dp/dz/Pam
D/
cm
dp/
m
49.50
165.0
1.00
2.65
1.00
3.76
0.09
186.10
49.50
165.0
1.00
2.65
1.00
3.07
0.17
157.70
49.50
165.0
1.00
2.65
1.00
3.76
0.26
254.70
15.85
190.0
1.00
2.65
1.30
2.50
0.14
475.20
15.85
190.0
1.00
2.65
1.30
2.50
0.29
630.90
15.85
190.0
1.00
2.65
0.12
3.00
0.13
648.90
15.85
190.0
1.00
2.65
0.12
2.90
0.28
866.70
5.07
520.0
1.00
2.65
1.00
1.90
0.09
1175.60
5.07
520.0
1.00
2.65
1.00
2.00
0.21
1763.40
4.00
580.0
1.25
2.27
4.00
2.88
0.16
3926.00
4.00
580.0
1.25
2.27
4.00
2.70
0.14
3580.00
4.00
580.0
1.25
2.27
4.00
2.01
0.10
2217.00
4.00
580.0
1.25
2.27
4.00
1.05
0.06
845.00
26.30 13000.0
1.00
2.65
1.00
3.20
0.04
842.50
26.30 13000.0
1.00
2.65
1.00
4.00
0.04
989.50
training
0.125
0.163
0.978
testing
0.127
0.169
0.977
Figure 4 Experimental vs predicted pressure drop for exponential radial basis function (erbf) kernel
2 p12
for i = 1 to n and j = 1 to m
and n = number of test record, m = number of training record (600 in this case), Tst is a n7 matrix for
testing input (which user want to test for calculating
pressure drop) and should be arrange in a sequence
similar to Table 5, Trn is a 6007 matrix for training
input data which is arranged in a sequence similar to
Table 5, B is a 6001 matrix calculated during training
Table 7
No
Kernel type
Kernel parameter
5043.82
0.50
erbf
-insensitive
0.127
6698.16
0.36
erbf
-insensitive
0.127
7954.35
0.48
erbf
-insensitive
0.127
8380.98
0.54
erbf
-insensitive
0.127
9005.8
0.59
erbf
-insensitive
0.127
Execution time/h
0.127
0.169
0.977
0.132
0.170
0.975
No
Author
A/%
49.51
36.53
93.43
10
50.02
34.50
39.97
26.68
22.31
22.01
10
present work
12.70
CONCLUSIONS
Support Vector Machines regression methodology with a robust parameter tuning procedure has
been described in this work. The method employs a
hybrid SVR- GA approach for minimizing the generalization error. Superior prediction performances were
obtained for the case study of pressure drop and a
comparison with selected correlations in the literature
showed that the developed SVR correlation noticeably
improved prediction of pressure drop over a wide
range of operating conditions, physical properties, and
pipe diameters. The proposed hybrid technique
( SVR-GA) also relieve the non-expert users to choose
the meta-parameters of SVR algorithm for his case
study and find out optimum value of these
meta-parameters on its own. The results indicate that
SVR based technique with the GA based parameters
tuning approach described in this work can yield excellent generalization and can be advantageously employed for a large class of regression problems encountered in process engineering.
NOMENCLATURE
A
, *
,*
REFERENCES
1
2
3
4
5
6
Vapnik, V., Golowich, S., Smola, A., Support vector method for
function approximation regression estimation and signal processing,
Adv. Neural Inform. Proces. Syst., 9, 281-287 (1996).
Vapnik, V., The Nature of Statistical Learning Theory, Springer Verlag, New York (1995).
Vapnik, V., Statistical Learning Theory, John Wiley, New York (1998).
Jack, L.B., Nandi, A.K., Fault detection using support vector machines and artificial neural networks augmented by genetic algorithms, Mech. Sys. Sig. Proc., 16, 372-390 (2002).
Agarwal, M., Jade, A.M., Jayaraman, V.K., Kulkarni, B.D., Supprot
vector machines: A useful tool for process engineering applications,
Chem. Eng. Prog, 23, 57-62 (2003).
Edgar, T.F., Himmelblau, D.M., Optimization of Chemical Processes,
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34