2012 Fifth International Joint Conference on Computational Sciences and Optimization

A algorithm for global optimization problems based on ABC-BFGS

Xiaojing Li Guangxi College for Preschool Education Nanning, 530022, China E-mail: guilixiaoling@163.com Huasheng Zhao,Long Jin Guangxi Climate Center Nanning, 530022, China E-mail: 2006zhaohuasheng@163.com jinlong01@163.com

AbstractTo overcome the problem of premature convergence on Artiifcial Bee Colony (ABC) in optimizing multimodal function, this paper proposed a hybrid algorithm of ABCBFGS, and used a special mutation to make algorithm escape local minima. Four classic multi-valued functions (Rosenbrock, Rastrigin, Griewank, Ackley) were selected as the test functions. The experimental results show that the ABC-BFGS algorithm not only can effectively locate the global optimum, but also have a rather high convergence speed. The ABC-BFGS algorithm is a promising approach for solving global optimization problems. Index Termsglobal optimum; hybrid algorithm; artiifcial bee colony; BFGS method

I. I NTRODUCTION In the practice of science and engineering are often subject to optimal solutions for non-convex, highly non-linear problem. Since the global optimal solution of such issues do not have any representation of mathematical conditions, And the result of optimization method based on mathematical programming is also easily fall into local optimal solution. In recent years, evolutionary computation based on stochastic search methods in global optimization problems in the application to get more and more attention, but such methods are also exist disadvantages, such as precocious, slow convergence, it is difcult to get satisfactory optimization results. Currently, the global optimum solution is still a challenging problem. In order to obtain good results in optimization, it has been made a study of hybrid optimization algorithm carried out by local optimization algorithm combined with global optimization algorithms[ 1, 2]. Hybrid global optimization algorithm using the global search capability of global optimization algorithm and the partial tuning capabilities of local optimization algorithm achieved better results than a single algorithm. Hybrid optimization algorithm was presented in this article uses the Artiifcial Bee Colony combining with BFGS method and its application in solving complex Multipole value minimum value of the function. Experimental results show that mixed algorithm to nd global optimal solution is very well. Articial bee colony algorithm was proposed by Karaboga[ 3] in 2005 of a Bionic optimization algorithms mimic bees swarm intelligent search behavior, it is based on self-organization model of bees and a calculation method based on swarm intelligence. Despite the articial bee colony algorithm and its application are in its early stages, but
978-0-7695-4690-2/12 $26.00 2012 IEEE DOI 10.1109/CSO.2012.197 881

because of the less algorithm control parameters, easy to realize, convenient calculation and so on, it has been more and more attention by scholars. Snivasa et al. [ 4] used ABC algorithm to solve network reconguration of distributed systems problems, and then Karaboga [ 5] proposed an algorithm based on articial swarm method of digital IIR lter design; Singh [ 6] proposed a solution to the problem of articial bees minimum spanning tree algorithm; Ding Haijun et al. [ 7] through improvement of the ABC algorithm parameters proposed an improved of bees algorithm for a combinatorial optimization problem, and used it to solve TSP problem. However, Like many groups of intellectual algorithms, when the ABC algorithm towards global optimal solution, there are a lot of problems, such as slow search speed, early convergence, the decrease in diversity of individuals, even in a locally optimal solutions. In local algorithms, BFGS method has been paid [ 8]much attention for its sound mathematical theoryand the use of inexact linear search of the super-linear convergence and effectiveness of dealing with practical problems. In order to improve the efciency of the search for optimal solutions, this article combined the ABC algorithm with BFGS algorithm to learn from each other. II. H YBRID SWARM ALGORITHM A. Articial Bee Colony Algorithm The ABC algorithm was rstly introduced by Karaboga[ 3]for numerical optimization problems based on the foraging behaviour of honeybee swarm.Further improvements of the ABC algorithm have been carried out by Karaboga and Basturk[ 9 11]. in this model, the foraging bees are classied into three different types: employed bees, onlookers and scouts.A bee which has found a food source to exploit is called an employed bee. Onlookers are those waiting in the hive to receive the information about the food sources from the employed bees and Scouts are the bees which are randomly searching for new food sources around the hive. After exploiting a food source, an employed bee returns to the hive and shares the information about the nectar amount of the food source with other bees by dancing in the dance area of the hive. The duration of the dance is proportional to the protability of the food source. As the quality of a food source enhances, the dancing duration

related to this food source increases, making it more probable for an onlooker to choose this source. After watching several dances, an onlooker bee selects a food source and becomes employed. In a similar way, a scout is called employed when it nds a food source. After completely exploiting a food source, the food source is abandoned and all the employed bees change into onlookers or scouts. Karaboga[ 3]proposed the ABC algorithm inspired by this foraging behaviour of honeybees. In this algorithm a food source position is considered as a candidate solution for the optimization problem and the tness of the solution is represented by the nectar amount of the food source. Similar to the real bee colony, the colony of articial bees is composed of employed bees, Onlookers and scouts. In the ABC algorithm, half of the colony are employed bees while the other half consists of onlookers, also it is assumed that the number of food sources is equal to the number of employed bees. After abandoning a food source, the employed bee of that food source becomes a scout and carries out a random search. Same as other swarm intelligence based algorithms, the ABC algorithm has an iterative process. By assuming the number of food sources as NS which is equal to the number of employed or onlooker bees, and D as the dimension of each solution vector, the main steps of an ABC algorithm can be dened as follows: Step 1: A random population (X1 , , XN S ) is initialized, where Xi = {Xi1 , , XiD }and each solution vector is generated using: xij = xmin j + rand[0, 1](xmax j xmin j ) (1)

determined by: Pi =
SN j =1

f iti f itj

(3)

where is the tness value of the food source Xi . After selecting a food source, the onlooker generates a new food source using Eq.(2). Once the new food source is generated, it will be evaluated and a greedy selection will be applied, same as the ase of employed bees. Step 5: If a candidate solution,represented by a food source cannot be further improved by a predetermined number of trials,the food source is considered abandoned and the employed bee associated with that food source becomes a scout.The scout randomly generates a new food source using: vij = xmin j + rand[0, 1](xmax j xmin j ), j = 1toD (4) The abandoned food source is replaced by the randomly generated food source. In the ABC algorithm, the predetermined number of trials for abandoning a food source is called limit, also in this algorithm atmost one employed bee at each cycle can become a scout. Step 6: If a termination condition is met, the process is stopped and the best food source is reported; otherwise the algorithm returns to step 2. B. BFGS method BFGS method[ 8] was proposed in 1970 by Broyden, Fletcher, Goldfarb and Shanno, it was a quasi-Newton methods, a termination for the second time, the overall nature and superlinear convergence. Its convergence rate faster than the gradient the steepest descent algorithm and the search direction of algorithm is conjugate. BFGS is an efcient local algorithm, BFGS method for solving unconstrained optimization problem f(x), its main steps are as follows: (1) Determine the variable dimension n and the BFGS method convergence precision ( 0), take the initial point X 1 Rn ; (2) Calculate g1 = f (X 1 ),then let X = X 1 and f = f (X 1 ) stop the algorithm if ||g1 || , Otherwise go to step (3); (3) Let k = 1, H1 = In (4)Let Zk = Hk gk (5) Seeking the minimum of k , and then let X k+1 = X k + k Z k where k : min{f (X k + Z k ) = f (X k + k Z k )| 0}; (6) Calculate: gk+1 = f (X k+1 ), and then let X = X k+1 and f = f (X k+1 ) stop the algorithm if ||gk+1 || , Otherwise go to step (7); (7) Let Xk = X k+1 X k , gk = g k+1 g k , and then Calculate: Hk+1 = Hk + [1 + T T T ( gk Hk gk )/(Xk gk )](Xk Xk )/(Xk g k ) T T T (Xk gk Hk + Hk gk Xk )/(Xk gk ); (8) Let k = k + 1, then go to step (4);

where j=1 to D and i=1 to NS; xmax j and xmin j respectively represent the upper and lower bounds for the dimension j. After Initialization of the population, the tness of each food source is evaluated. Step 2: Each employed bee searches the neighbourhood of its current food source to determine a new food source using: vij = xij + ij (xij xkj ) (2)

where k {1, 2, , N S } and j {1, 2, , D} are randomly chosen indexes. It must be noted that k has to be different from i. ij is a random number between[-1,1].Parameter values produced by Eq.(2)which exceed their boundary values are set to their boundary values[12]. Step 3: After generating the new food source, the nectar amount of it will be evaluated and a greedy selection will be performed. If the quality of the new food source is better than the current position, the employed bee leaves its position and moves to the new food source; in other words, If the tness of the new food source is equal or better than that of Xi , the new food source takes the place of Xi in the population and becomes a new member. Step 4: First an onlooker bee selects a food source by evaluating the information received from all of the employed bees. The probability pi of selecting the food source i is

882

C. ABC-BFGS hybrid algorithm The ABC algorithm is a global optimization method. In the optimization of the late ABC algorithmconvergence rate is signicantly slower, even at a standstill, it is difcult to obtain good global optimal solution. To overcome this shortcoming, this paper uses the BFGS method in the late of the ABC algorithm optimization to accelerate the convergence rate of overall convergence. BFGS method belongs to the local algorithms, optimization results are good or bad depending on the initial position of the selection, so this paper uses the ABC algorithm to search good initial for BFGS method. ABCBFGS algorithm is to both learn from each other, to achieve good optimization results. The basic idea of ABC-the BFGS hybrid algorithm is to recycle ABC algorithm and the BFGS method, until meeting scheduled optimization goals. But we found in the experiment, if only the both are mixed simply that it does not guarantee that the ABC algorithms can help the BFGS method to jump local optimization, especially when the ABC algorithm falls into local optimization prematurely. So, this article does those in the ABC-BFGS algorithm as follows: First of all, in the search of the ABC algorithm, if the optimal solution is no improvement after N cycles in a row, and When N is greater than 20 and so far the optimal solution of the corresponding error is smaller than some value (specic issues) or N greater than 100, we will stop the search of the ABC algorithm, and then take the optimal solution found so far of the ABC algorithm as the initial values of the BFGS algorithm to local search. Secondly, during the search of BFGS, set the maximum number of searches, Kt, exceed the maximum number of searches that explains the BFGS method has been into a local optimum, and while the last ABC algorithm results for the BFGS method is not a good initial value. Ending the BFGS search prematurely, then to generate a new initial population of the ABC algorithm that from a larger context of X which was obtained by the last iteration of BFGS method. In addition, if the BFGS method in the search process by the normal end is relatively f (X )and the default adaptation value the size of the eps. and to generate a new population in the smaller range of X .In addition,If ||k || , that the search process of the BFGS method will be ended ,then comparison of f (X )with the default adaptation value the size of the eps. If f (X ) > eps,to generate a new population in a smaller range of X ; then the ABC Search will be carried out in the new populations space; repeat, untilf (X ) < eps. The formula of generating a new population space is: X i = [rand 0.5] X + X i = 1 : SN (5)

the standard ABC algorithm and the BFGS algorithm. The four test functions were: Rosenbrock, Rastrigin, Griewank, Ackley [13]. Table 1 showed the four test functions that the asymmetric initialization rangethe global minimumthe search range [Xmin , Xmax ], the dimension n and the target minimum. The Rosenbrock function:
n1

f1 (x) =
i=1

2 2 [100(xi+1 x2 i ) + (xi 1) ]

(6)

The Rastrigrin function:

n

f 2 ( x) =
i=1

[ x2 i 10cos(2xi ) + 10]

(7)

The Schwefel function:

n

f 3 (x ) =
i=1

x2 i sin(

| xi | )

(8)

The Ackley function:

n

f4 (x) = 20 e

0.2

i=1 n

x2 i

i=1

cos(2xi ) n

(9)

About the ABC algorithm, set the population space SN=50, control parameter limit=100, the maximum iteration count C=2500; the Maximum number of iterations Kt and the optimization targetof BFGS method respectively was 20 and 1e-16; In ABC-BFGS hybrid algorithm, If the BFGS method is terminated in case that the Kt equal the maximum number of iterations, let the adjustment constant = 50 and then generate a new population space. Otherwise, the adjustment constant = 2. Table 2 showed the global minimum of search
TABLE I T HE RELEVANT PARAMETERS OF THE FOUR TEST FUNCTIONS Rosenbrock Initialize range Search range Dimension n Global minimum (15,30) (-2,2) 20 0 Rastrigin (2.12,5.12) (-5,5) 20 0 Schwefel (300,500) (-500,500) 100 -41898.29 Ackley (0,32) (-32,32) 100 0

Where is the adjustment constant that generates the new population size; III. T HE C OMPUTER SIMULATION EXPERIMENT In order to verify the performance of the ABC-BFGS algorithm, this paper selected four classic test functions to test the new algorithm (ABC-BFGS), and the results compared with

results on four test functions of the three kinds of algorithm (ABCBFGS and ABC-BFGS) according to the parameters given in Table 1. It can be seen from Table 2, the BFGS algorithm is often trapped in local optimal solution and difcult to extricate themselves, it is difcult to achieve good results. ABC algorithm is global convergence is relatively good, but the convergence rate of algorithm is slowly in the later search, so it need more number of iterations. ABC-BFGS hybrid algorithm can reliably search the global optimal solution of multimodal function. The four multimodal functions tested in this paper, ABC-BFGS algorithm can 100% search the global optimal solution. Compared with the required iterations of the ABC algorithm search for the global optimal solution, the iterations number of ABC-BFGS hybrid algorithm have greatly reduced, and the number of iterations for each function was not too

883

TABLE II T HE RESULTS OF THE THREE ALGORITHMS Functions Algorithms ABC BFGS ABC- BFGS ABC BFGS ABC- BFGS ABC BFGS ABC- BFGS ABC BFGS ABC- BFGS The minimum of search 0.0144 707.8243 0 0 229.8339 0 -41898.29 -41464.01 -41898.29 5.4072 19.5449 0 The number of iterations 2500 83 ABC:302 + BFGS:21 1301 31 ABC:612+ BFGS:3 1085 18 ABC:721+ BFGS:4 2500 13 ABC:1382+ BFGS:30

[10] Karaboga D and Basturk B. A powerful and efcient algorithm for numerical function optimization:articial bee colony(abc)algorithm,Journal of Global Optimization. vol.39(3), pp.459-471,2007. [11] Karaboga D and Basturk B. On the performance of articial bee colony(abc)algorithm,Applied Soft Computing. vol.8(1), pp.687697,2008. [12] Karaboga D and Akay B. A comparative study of Articial Bee Colony Algorithm, Applied Mathematics and Computation. vol.214(1), pp.108-132, 2009. [13] Janson S,Middendorf M. A Hierarchical Particle Swarm Optimizer and Its Adaptive Variant, IEEE Trans.on Systems Man and Cyberneticss. vol.35(6), pp.1272-1282, 2005.

Rosenbrock

Rastrigin

Schwefel

Ackley

much. This is very important for engineering applications. And the hybrid algorithm not only has good global search capability but also has a faster search speed. ACKNOWLEDGMENT The authors would like to thank the support by the National Natural Science Foundation of China (grant No. 41065002 and No.40965003)and Major Special Funds for Guangxi Natural Science Foundation (No.2011GXNSFE018006). R EFERENCES
[1] LOUDAS C A,AGGARWAL A,CIRIC A R.Global optimum search for non-convex NLP and MINLP problems,Comput.Chem.Engng. vol.13(10), pp.1117-1132, 1989. [2] G.D. Wang, X.Y. Liu,et al. Hybrid Genetic algorithms of Global Optimum for Optimization Problems, Chinese Journal of Theoretical and Applied Mechanics. vol.34(3), pp.469-474, 2002. [3] KARABOGA D.An idea based on honey bee swarm for numerical optimizationTechnical Report-TR06, JKayseri: Erciyes University. Engineeirng Faculty, Computer Engineering Department. 2005. [4] Rao R. S., Narasimham S.V.L., Ramalingaraju M. Aptimization ofdistirbution network conifguration forloss reduction using articial bee colony algorithm, Intenrational Joumal of Electrical Power and Energy Systems Engineering. vol.1(2), pp.709-715, 2008. [5] KARABOGA N. A new design method based on articial bee colony algorithm for digitlal IIR lters, Jounral of the Franklin Institute. vol.346(4), pp.328-348, 2009. [6] Singh A. An atriifcial bee colony algorithm for theleaf-constrained minimum spanning treeproblem, Applied Sotf Computing. vol.9(2), pp.625-631, 2009. [7] J.H. Ding, L.F. Li. Bee Colony Algorithm For TSP Problem And Parameter Improvement, China Science And Technology Information. vol.(3), pp.241-243, 2008. [8] X.Y. Yuan, Y.W. Sun. Optimization Theory and Methods,Science Press. , 1997. [9] Basturk B and Karaboga D. An articial bee colony(abc) algorithm for numeric function optimization, IEEE Swarm Intelligence Symposium 2006. Indianapolis,Indiana,USA,May 2006.

884