See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/241304488

Fitting curves by simulated annealing

Article  in  Computers in Physics · May 1995

DOI: 10.1063/1.168533

CITATIONS READS
33 21

1 author:

Thadeu Penna
Universidade Federal Fluminense
94 PUBLICATIONS   1,592 CITATIONS   

SEE PROFILE

All content following this page was uploaded by Thadeu Penna on 02 December 2016.

The user has requested enhancement of the downloaded file.

 Computers in Physics
Fitting curves by simulated annealing
T. J. P. Penna

Citation: Computers in Physics 9, 341 (1995); doi: 10.1063/1.168533

View online: http://dx.doi.org/10.1063/1.168533
View Table of Contents: http://scitation.aip.org/content/aip/journal/cip/9/3?ver=pdfcov
Published by the AIP Publishing

Articles you may be interested in

Rational quadratic Bézier curve fitting by simulated annealing technique
AIP Conf. Proc. 1522, 844 (2013); 10.1063/1.4801215

A lesson in curve fitting

Phys. Teach. 37, 340 (1999); 10.1119/1.880308

Fits Curves With MINSQ

Comput. Phys. 2, 98 (1988); 10.1063/1.4822785

Curve Fitting
Am. J. Phys. 24, 464 (1956); 10.1119/1.1934274

The Laplace Transform as a Form of Curve Fitting

Am. J. Phys. 17, 507 (1949); 10.1119/1.1989669

Reuse of AIP Publishing content is subject to the terms at: https://publishing.aip.org/authors/rights-and-permissions. Download to IP: 185.34.43.74 On: Fri, 02 Dec
2016 15:20:09
 Fiin~ curves by simulatedannealing
T. J. P. Pennaa)
Instituto de Fisica, Universidade Federal Fluminense, Av. Litorhnea, s/n, 24210-340 Niter& W,
Brazil
(Received 25 October 1994; accepted 13 February I995)
Simulated annealing is shown to be a powerful method to fitting curves. This method is particularly
appropriate to fit curves with discontinuous derivatives where the traditional methods fail. Recent
generalizations of the simulated annealing method enable easy and efficient implementation on
microcomputers. 0 1995 American Institute of Physics.
Fitting observational data (by experiments or numerical
simulations) to a given function is a common task in sci-
ence, sometimes to test a theoretical model or as interpo-
lating purposes, etc. The most popular method consists in
minimizing the squared deviation function
cZ=;:i=l
[yi-y(Xi;a,***UM)]‘,
where yi (i= 1 ***N) are the data, y(x)~y(x,a,***a,) is
the function to be fit, with the M free parameters (cI~***u~).
Let us consider the data without error bars, for the sake of
simplicity. The errors (TV(i= 1 *.*N) can be introduced di-
viding each term on the right-hand side of Eq. (1) by mi.
The specific methods to fitting curves frequently demand
a good estimate of the parameters to be fit.’ In addition,
derivatives of the function have been evaluated. This fact
restricts the application of those methods to fit well-
behaved functions. In this note, we show that a simulated
annealing approach2 can be useful in situations where the
traditional methods fail as, for example, for discontinuous
derivative functions. Moreover, the implementation is
easier and, due to recent developments, the performance is
also good.
The simulated annealing is widely used in the minimi-
zation of nonconvex (multiple minima) functions-as pre-
sented in Eq. (1)-as alternative to gradient descent meth-
ods. As far as we know, it has not been applied in fitting
curves because other methods are more efficient than the
original version of the simulated annealing. Here we
present some improvements in this approach that enable it
to work as an efficient and simple algorithm. The method is
implemented as follows: The variables are allowed to
change (a jump in the configuration space), according to a
visiting distribution that gives the probability of occurrence
of a jump of a given size (we will return to this point in
more detail later). If this jump decreases the value of a cost
function then the changes are always accepted, just like
gradient descent. However, if the jump is in direction of
increasing the cost function, the changes will be accepted
“Electronicmail: glitjpp@vmhpo.uff.br
Reuse of AIP Publishing content is subject to the terms at: https://publishing.aip.org/authors/rights-and-permissions. Download to IP: 185.34.43.74 On: Fri, 02 Dec
2016 15:20:09
according to an acceptance probubilig, depending on an
artificial temperature. This step avoids the trapping in local
minima. Then, the temperature is gradually cooled, accord-
ing to a recipe that depends on the choice of the visiting
distribution, and the process is repeat until the desired pre-
cision is reached.
Geman and Geman showed that, in order to reach a
global minimum, if the visiting distribution is a Gaussian
one, the temperature must decrease logarithmically with
time. This dependence strongly discourages the use of
simulated annealing as modeling data algorithm. Szu and
Hartley4 proposed to use a Cauchy-Lorentz distribution
(semilocal distribution) instead of the Gaussian one (local
distribution). In this case, the temperature decreases like the
inverse of time, therefore the algorithm is more efficient.
Recently, Tsallis and Stariolo’ proposed a new visiting dis-
tribution that recovers both cited before as special cases. A
new parameter 4” E [ 1,3] is introduced, such qU = 1 recovers
the classical simulated annealing (with Gaussian distribu-
tion of jumps, sometimes called Boltztnunn machine),
q,=2 recovers the fast simulated annealing (Cauchy ma-
chine). The probability of trial value for a parameter x
changing from xt to xt+ L is’
g,,(b)
qty-4”’
cx [l+(q,- 1)(AX,)2/T(t)2/‘3-4,‘][ll(q,-1)]+[(D-1)/2],
(2)
where T(t) is the artificial temperature at time t, D is the
dimension of the function to be minimized, and Ax,
=xt+ 1-xt is the size of the jump. For arbitrary D, the
temperature should be cooled as
24u-‘-- 1
Ut)=T(l) (l+t)qL,-l-l’
Therefore the temperature decreases as a (qV- 1) power of
time, consequently in a much more efficient way than the
earlier strategies.
Now, let us focus the discussion on the acceptance
probability as function of temperature. The most frequently
used function is the so-called Metropolis algorithm. Since
the temperature is an artificial parameter the only condition
COMPUTERSIN PHYSICS,VOL. 9, NO. 3, MAY/JUN 1995 341
 Bmcludcwdm h> /* drstnbutmn function *,
Ifdude~ns,th h> prob = 10,
pm I + (qv-l)‘(&ltax’&l1ax)/
tide&x RANDMAX 4294967295 0 pou(T.2 00-v) ).d).
#dcfmc F(X.A.B,C) (X<O) ? \ I ahde(rr>prob).
B*evp(-(X*X)/(2 O*A*A)) B*mp(-
(X$X)/(2 O ’C’C))
t*One pmmeter b, step */ 2.08 -
double da-x( ioO],dat~[ IW].tif(4),fitl[4], for(t=O.,<D.e+) titl[~l-fit[il.
au\.auxI,dc.del~~.D.d,c2.c2I.prob, Rtl~t%DI += deltay.
qa,qv,~,T,TO,tsall~s,~t,
long UII t,k.npts,t, I’ Evaluatmg new error ‘i
k
“nagned long I”, r. eZl=O 0. 3
FILE ‘arq.*arql, for(i=O.~<npts.i++) (
xt = F(~~~~~].fitl~O),fitl~l),~~l~O]), E 2.04 -
mm0 f sZlt= (xt - dat_vlt))*(xt - da,+)),
qn=-5, I* scme useful cn”stmlS ‘/ I
qv=2 /* Gencrahzcd Mctmpobr ‘/
E
r=3721. de - c21*2.
npts=24 tf(du=0) (
D=2, a”Y=(l+(qa-l)‘duT).
d = (D . I )/2 0 + I /(qv-I 0). If @ux>O) awpav@ur,l /(I .qa)),
/’ Data to be Fmcd l / else aux=o,
far(t=O,t<nPLs.,+t) ( if (an> 1) au= I.
datexii) = (double) 1*4.0/npfn - I 7, tsallis=aux;
datJli) = F(datwx[i),2 0.1 0.2 IS) , r+=,rccl,+,rc<,6,.
I rr = (d&j , RANDMAX,
I’ InltutlalGuess ‘/ d (rr<Uallm) (
Iitlol-2 0. c2 = c21.
fitiii=i 01 fit(t%DJ = fitl[t%D].
I’ inr,,al Error */
e2=0 0.
for(t=O.*<npls.it +) ( Clsc (
c2 = CZI;
Step
xt = F(da-x[r],fit[Ol,fitl I].ftt[O]).
c2+- (~l;It_,Ill-xt)*(dat_yIll-xt). fit[t%DI = fitl(t%D],
I )
i0 = lOO*c2.T=TO.
/* fen thousand stem ‘I
/* Anneallnp xhemc l ,
T = TO * (pmv(2 ,qv-1.)-l 0) Figure 2. Time evolution of the variance.
f0r(l=1,1<10000.t++) ( 4pa(ttl,q”-l)-l.O),
/’ *eneraflng random pmps I
accordmg 10 I’ Output *I
the distnbutron funclmn far(,=O.,<D.,++) pnntf(“Parameter %d =
by the rejectton method *I ?G.?h”.l+l,flr[lj): negative values of X, and dispersion C for the positive
pnntq”Error = o/r\n” .rqrf(e2)) .
do t
,‘=16807, /’ qmck and Lny random return 0. ones. The amplitude is given by parameter B. For input
d&x 7 ((double) dRANDMAX-
number generator ‘I data we have chosen A =2.0, B = 1.0, and C =2.15. In this
0 5)‘O 05. test case, we will try to fit a single Gaussian function (fre-
r*-16U07,
II = (double) r/RANDMAX. quently in modeling data we do not know, a priori, the
actual distribution of data, so this example should not be
considered as a biased one). The error landscape is strongly
Figure 1. A code written in C, for fitting a Gaussianfunction. dependent of the distribution of the input data, i.e., whether
we have more data either in the positive or in the negative
section of axis X. In this test, the input data were equally
to be respected is whether the acceptance probability leads spaced in the X axis in the interval [- 1.7, 2.31. In Fig. 2,
to correct equilibrium states. W ithin the framework of a one can see how the variance evolves in time when we
generalized statistical mechanics introduced by Tsallis,7 we started from a low temperature (30 times smaller than ini-
can define a generalized version of the Metropolis
algorithm:’ the new value of the function to be minimized
is accepted with a probability given by
p= m in(l,{l-(l-q,)Cy(x,+~)-y(x,)ll~(t)}”(’-q~)),
1 1
(4)
where qa is a new parameter. W e recover the Metropolis
algorithm in the qa--+l limit. The temperature for the ac-
ceptance probability can be different from the temperature
for the visiting distribution. Here, we made them equal, for
simplicity. Equation (4) also leads to equilibrium states for
all values of qa,5*8 The most important feature of the Eq.
(4)-for our purposes-is that approximate solutions are
found even at higher temperatures (for negative values of
qa) than had we used the Metropolis algorithm. Therefore,

1
starting from a high temperature we do not need good ini-
tial values.
In Fig. 1, we present a complete code written in C
where we try to fit a Gaussian function centered at origin.
The variance and the amplitude are the parameters to be 0.51 ’ ’ ’ I ’ ’ ’ ’ ’
fitted. In order to show this method is useful to fitting -2 -1 0 1 2 3
curves, we present a situation where there are local minima. X
In the fourth line of the code, we define the function to be
fitted (it will be used also to generating the input data): two
combinated Gaussian functions, with dispersion A for Figure 3. Input data (squares)fitted by a Gaussianfunction (solid line).

Reuse of AIP342
Publishing content is subject to the terms at: https://publishing.aip.org/authors/rights-and-permissions. Download to IP: 185.34.43.74 On: Fri, 02 Dec
COMPUTERSIN PHYSICS,VOL.9,NO.3,MAY/JUN 1995
2016 15:20:09
 tial value of the error). The system is initially trapped in a
local minimum (A =3.0, in this case) but, after about 100
iterations, it goes to the global minimum (A=2.1137
-tO.O005). In this we can also see how accurate is the value
of A, checking the fluctuations. In this example the exist-
ence of multiple minima turns evident but the temporary
trapping can be avoided starting from higher temperatures.
In this test, we follow a different approach from the one
suggested by Stariolo and Tsallis:” we have allowed to
change only one parameter by step. In our computer tests,
this approach has been seen as more efficient than the origi-
nal one in finding the global minimum. It is straightforward
to extend this version for more than two parameters, just
changing the value of D. Finally, we show the best fit for
the test case, in Fig. 3.
In summary, we have applied successfully the gener-
alized simulated annealing to fitting curves. The fast cool-
ing scheme and a generalization of the Metropolis algo-
rithm make this method competitive with the traditional
others, with additional advantages as it avoids derivatives
evaluation and good initial values. This method was used’
to fit more complex function as a Levy distribution
3:
P(I,clr,y)=; exp( - yq%os(qZ)dq, (5)
I0
to experimental data from leaky faucets, with y and $ as
free parameters and the integral was evaluated
numerically.’
Reuse of AIP Publishing content is subject to the terms at: https://publishing.aip.org/authors/rights-and-permissions. Download to IP: 185.34.43.74 On: Fri, 02
Dec 2016 15:20:09
View publication stats
ACKNOWLELIGMENTS
Financial support from the Brazilian Agencies CNPq and
FINEP is gratefully acknowledged.
REFERENCES
1. W. H. Press, S. A. Teukolsky, W. T. Vetterling, and B.
P. Flannery, Numerical Recipes in C (Cambridge Uni-
versity Press, New York, 1992).
2. S. Kirkpatrick, C. D. Gellat, and M. P. Vecchi, Science
220, 671 (1983); S. Kirkpatrick, J. Stat. Phys. 34, 975
(1984).
3. S. Geman and D. Geman, IEEE Trans. Patt. Anan.
Mach. Int. PAMI-6, 721 (1984).
3. H. Szu and R. Hartley, Phys. Lett. A 122, 157 (1987).
5. D. A. Stariolo and C. Tsallis, in Annd Review* of
Computational Physics II, edited by D. Stauffer (World
Scientific, Singapore, 1994).
6. N. Metropolis, A. W. Rosenbluth, M. N. Rosenbluth, A.
H. Teller, and E. Teller, J. Chem. Phys. 21, 1087
(1953).
7. C. Tsallis, J. Stat. Phys. 52, 479 (1988).
8. T. J. P. Penna, Phys. Rev. E 51, Rl (1995).
9. T. J. P. Penna, P. M. C. de Oliveira, J. C. Sartorelli, W.
M. Goncalves, and R. D. Pinto, preprint (1994).
COMPUTERSIN PHYSICS,VOL. 9. NO. 3. MAY;JUN 1995 343