Time varying radial basis functions

A. A. Jamshidi1, C. W. Gear1, and I. G. Kevrekidis1,2

1
Department of Chemical and Biological Engineering;
2
Program in Applied and Computational Mathematics
Princeton University, Princeton, NJ
e-mail:{arta,wgear,yannis}@princeton.edu

Abstract
We introduce radial basis functions (RBFs) whose time-varying coecients deter-
mine not only the amplitude and position of each RBF but also their shape. The
intended use of these Time Varying-RBFs (TV-RBFs) is in the local-in-time represen-
tation of low-dimensional approximations of functions that arise in solving spatiotem-
poral evolution problems; in particular, for time-varying spatially localized solutions
with a temporal translation component such as traveling waves, modulated pulses or
soliton-like solutions of evolutionary dierential equations. This paper is restricted to
the one-dimensional spatial case. We also present an algorithm that places the Time
Varying-RBFs (TV-RBFs) over spatiotemporal data that may come from experiments,
from nely discretized PDE simulations, or even from multiscale, particle-based simula-
tions. It rst approximates the function at a single time instant (a temporal snapshot)
as a sum of RBFs using a novel weighted minimization that causes each RBF to pri-
marily approximate one of the localized parts of the function). It then extends that
approximation to TV-RBFs over a sequence of snapshots of the function at dierent
times. We conclude by discussing the potential uses of these TV-RBFs.
keywords Weighted RBF tting, Approximation

1 Introduction
An important motivation for this work comes from our interest in the computation of so-
lutions of evolutionary partial dierential equations that are both localized in space and
have a temporal translation component. Their computer representation in standard nite
element, spectral, or nite dierence methods makes it dicult to determine, for example, if
the solution is a steady or slightly modulated traveling wave. The process described below
can be used to post-process the output of a standard solver, or it could be used in the
equation-free context[24, 25] to coarse-grain the output of a microscopic solver to determine
the macroscopic solution in a multiscale approach.
Figure 1 is an illustration of the type of spatiotemporal data we are interested in rep-
resenting: it shows a solution of the nonlinear Kuramoto-Sivashinsky partial dierential
equation,

ut = uux uxx uxxxx . (1)

Alternate email: arta.jamshidi@gmail.com
Alternate email: cwgear@gmail.com

1
solved
with periodic boundary conditions in a one-dimensional domain of length L =
2/ 4/53.3.

3
2
1
f(x,t)

0
1
2
100
90
80 10
70 5
0
60 5
t 50 10
x

Figure 1: Spatiotemporal patterns of the Kuramoto-Sivashinsky PDE eq (1) (see Section

3.2 for details of this computation).

This equation is well known for producing several dierent nontrivial spatiotemporal
wave patterns as the initial conditions and the size of the periodic box vary. The condi-
tions leading to Figure 1 were chosen so that the solution approaches an attractor that is
a Modulated Traveling Wave [2], and we show the results after a suciently long initial
time (here t = 50 ) for the solution to lie very close to the attractor. The nature of this
attractor can be perceived in the gure: there are two large and one small spatial bumps
(waves) that travel from left to right, but not with constant shapetheir shape is periodi-
cally modulated so that the overall solution appears quasiperiodic. We seek a parsimonious
representation of the solution in space and time, one that takes advantage of the nature of
the solution, and permits the easy calculation of the time derivatives of the waves position
and shape. These time derivatives can then be input to any of a number of standard nu-
merical analysis procedures that analyze the macroscopic properties of the solution, such as
the location of possible steady states or periodic solutions, their stability and bifurcations
([5, 6, 26]). As mentioned above, this can also be helpful in the context of coarse grain-
ing ne-scale, interacting-particle-based-computations for which the macroscopic behavior
exhibits spatiotemporal dynamics of this type.
The spatial representation of solutions of PDEs is often based on judiciously chosen
sets of basis functionssets like nite elements, or eigenfunctions of a linearized part of
the problem, or even empirical basis sets from data analysis (e.g. Principal Components
[15, 21, 4]). In the study of time-dependent problems, the evolution of the solution is then
described by the evolution of the basis function expansion. In most of these cases the
basis functions are independent of time (and often orthogonal in space for computational

2
eciency), so that time evolution is recorded solely in terms of the amplitudes, as in

K
f (x, t) i (t)i (x) (2)
i=1

where {i (x)} is a spatial basis function set. However, in this representation the traveling
nature of a solution may not be transparent.
Since we are interested in solutions with bump-like traveling waves, there is potential for
the use of a Radial Basis Function (RBF) approximation in space in a way that places one
or more RBFs on each major solution bump. RBF approximation in space typically takes
the form

K
f (x) i (||x xi ||) (3)
i=1

where the RBF is a function of the scalar radial distance of x from the center xi at which
the RBF is placed. Since we are interested in evolutionary equations whose solutions have
a traveling natureand do not therefore appear as bumps in timewe do not use RBFs
in both space and time (such RBFs have been considered, for example, in [29]) but let the
parameters characterizing the RBFs vary with time. (We will briey summarize common
RBFs in the next section: in this paper we are going to restrict ourselves to the Gaussian
RBFs (r) = exp(r2 /2 2 ).)
In the typical use of RBFs for function approximation one knows the values of a function
f (x) at a set of points {xi } and wishes to approximate the function over its relevant range.
In numerical analysis this is the interpolation problemextending a function from its known
points to others. Typically in interpolation the RBFs are centered at each known data
point (or a subset thereof). However, since we want to use RBFs to get a parsimonious
approximate representation of f (x), the centers of the RBFs will be chosen to give the most
accurate approximation, and so may not be at any of the data points themselves. The heart
of the method is minimization of

K
f (x) i ( x xi ) 2 dx (4)
xU i=1

over the parameters of the RBFs (amplitudes, centers, and standard deviations) and the
choice of a suciently large K that the resulting error is acceptable.
The example in Figure 1 would reduce to a periodic modulation [2] in a co-traveling
frame. However, other examples may consist of waves traveling at dierent speeds (e.g.,
solitons [40, 9, 8, 7, 35]) or the data shown in Figure 2. (It is synthetic data created for
program test purposes generated from the function

f (x, t) = 1.5e100(x(0.375+0.025t)) + e50(x(1.1250.025t))

2 2
(5)

with x [0, 1.5] and t [1, 16].)

Eq (4) is non-linear in most of its parameters. If it were minimized for a set of f (x, tj )
at slightly dierent tj it would not be surprising if small changes in f (x, tj ) caused abrupt
changes in the solutionmaking the optimization results eectively non-dierentiable in time.
Consider, for example, the data in Figure 2, consisting of two additive Gaussian waves that
totally overlap by time 15. If we optimized eq (4) at time 1 we would hope to obtain a
solution using two RBFs; whereas if we optimized at time 15, one RBF might suce. It
may be interesting to note that this temporal discontinuity due to shifts in optimization

3
 2.5

1.5

f(x,t)
1

0.5

10

1.5
1
t 0 0.5
0
x

Figure 2: A synthetic data set: two waves traveling in opposite directions (see eq (5) for
denition)

minima also arises in the context of receding horizon model predictive control, see for
example [28]. Since we are interested in nding smooth representations of time dependent
solutions we have used Time Varying-RBFs (TV-RBFs): RBFs whose shape, amplitude,
and position can vary with time.
In our TV-RBFs the amplitude, center and all other parameters can vary with time, so
we will write a TV-TBF as (x, p(t)) where the vector p(t) is the set of time-dependent
parameters that determine the RBF properties (in this paper, the amplitude, center, and
width). We are thus interested in approximations of the given data, f (x, t), (typically a
solution of a PDE) of the form

K
f (x, t) (x, pi (t)). (6)
i=1

We assume that we have computed f (x, t) at a set of discrete times {tj } and we will call
the data f (x, tj ) at each time a slice. Our goal is to pick the time-varying coecients of the
RBFs to minimize the cost function
N
K
C({pi (t)}) = f (x, tj ) (x, pi (tj )) 2 dx, (7)
j=1 xU i=1

where N is the number of slices and the {pi (t)} are assumed to be in some nite-dimensional
subset of time-dependent functions. In this paper we will limit ourselves to local linear
polynomials in time, so that six real values are sucient to determine each TV-RBF. While
higher-order polynomials (or other smooth time-dependent functions) could be used, if we
are interested in issues such as the steady state of a traveling wave, we only need the rst
time derivative, while if we were using projective integration described in [10] (although that
paper discussed using higher-order derivatives for higher-order methods) subsequent numeri-
cal experiments have indicated that it is usually better to use multi-step like approaches that
require only rst derivatives, so linear polynomials will usually suce. We also have to de-
termine the number, K, of RBFs to use to achieve the desired accuracy. (In a computational
environment, f (x, tj ) in eq (7) will be known in some nite-dimensional form determined
by the solution technique so the integral will be replaced by an appropriate approximation.
In the experiments reported here we will have point values f (xi , tj ), i = 1, . . . , M and we

4
will replace the integral with a sum. Since the proposed approach is independent of the way
the integral is approximated, we will continue to denote it as an integral.)
We have chosen to use TV-RBFs to avoid the discontinuities likely to arise in slice-
by-slice tting. The result is the function C in eq (7) with many parametersa highly
non-linear function of most of the parameters (it is quadratic in the amplitudes {i })so
its minimization by standard techniques requires a initial guess close to the solution to
avoid other local minima. We propose a procedure to t initially a single slice of data with
time-independent RBFs, determining how many RBFs to use in the process. The initial
guess for the minimization of eq (7) is then taken to be a constant-in-time projection of this
approximation.
The contributions of this paper are: (a) the introduction of a weighted optimization pro-
cedure to identify bumps in spatial data and its comparison with straightforward unweighted
versions (we found that the weighted procedure was an important component in initializing
the t to a single data slice), (b) the procedure for adding additional RBFs to improve the
approximation, and (c) the introduction of TV-RBFs and our protocol for tting them to
spatiotemporal data.
The remainder of the paper is organized as follows: Section 2, after a brief review of
RBFs, will detail the proposed algorithms: rst, a weighted spatial tting scheme for a
single time slice, followed by the introduction of TV-RBFs and our algorithm for placing
them over spatiotemporal data. Section 3 will present some numerical experiments on both
synthetic and PDE-solution data. We will conclude with a brief summary and discussion of
the results in Section 4.

2 Radial Basis Functions

2.1 Brief Overview of RBF Approximation in Space
We assume that we have point-wise values, yi = f (xi ), xi U Rn and yi V Rm and
wish to approximate the mapping
f :U V (8)
from the data. If m > 1 one can simply choose to approximate each component separately,
so here we will assume that m = 1 (for further discussion, see [19]). While we will only
consider the one-dimensional space case n = 1 in this paper, the approach can be extended
in a reasonably straightforward manner to higher space dimensions, as long as the solutions
involved retain a localized-in-space character along with translational motion (traveling) in
space-time. If the patterns involved are isolated hills, then a minor modication to the
RBFs discussed below will suce. If the patterns are greatly elongated in one direction (for
example deep ocean waves, which have a fairly short wavelength of the order of meters in
the direction of motion, yet may extend over several hundred meters in a nearly-orthogonal
direction) then appropriately weighted basis functions can be used.
The standard RBF approximation (using the classical notation) seeks to represent
f (x) as

K
f (x) f(x) = i (x ci ), (9)
i=1

where is an RBF centered at location ci , and i is the weight for ith RBF. In spatial
dimensions greater than 1, the radial distance, x ci , could be calculated after
any
ane transformation of space, which would lead to a norm of the form x = xT W x

5
where W is a positive denite matrix. W is used to stretch an RBF so that it has an
elliptical footprint.
RBFs have been widely used for data approximation. Multiquadrics, (r) = 1 + r2
(r = x ci ), were introduced by [13] for modeling scattered data on topographical sur-
faces. Thin plate splines, (r) = r2 ln r, were introduced by [12] for surface interpolation.
Gaussian radial basis functions, (r) = exp(r2 / 2 ), were proposed by [1] for data tting
and classication (for a comprehensive treatment of this topic and additional new RBFs, see
[18, 20]). There is a signicant literature treating theoretical aspects of RBFs, including uni-
versal approximation theorems, see, e.g., [37, 32, 33, 39]. A large number of algorithms have
been proposed in the literature for computing the model parameters, e.g., [34, 22, 14, 43, 30].
For further references on this topic see, [17, 16]. The research monographs [27, 3, 41] con-
tain references treating additional theory, algorithms and applications. The use of RBFs for
construction of non-linear models has been explored in [1, 42, 38, 36, 37]. One of the main
reasons for the usefulness of RBFs as interpolants in high-dimensional spaces is the fact that
the approximation problem in which the only degrees of freedom are the RBF amplitudes
(in other words, when the shapes and centers are xed), gives rise to linear systems for the
amplitudes {i } which, for positive denite RBFs, are guaranteed to be non-singular (see
e.g. [37]).

2.2 A Methodology to Fit RBFs in Space

We rst deal with the important issue of initializing TV-RBF parameter estimates before
attempting to minimize eq (7). We do this by approximating a single slice in space and
using a constant-in-time version of this as the initial guess. Hence we need a procedure to
systematically determine all the parameters of (traditional) RBFs, including their centers
and widths, as well as the number of RBFs to use, to t a spatial function. The framework
for this task is the minimization of

K
f (x) (x, pi ) 2 dx (10)
xU i=1

where
(x c)2
(x, p) = exp( )
2
and p denotes the set of parameters (weight , center c and width ) to nd the best
approximation using K RBFs.
Since we do not know a priori the number of RBFs to use, we t them sequentially, one
by one, until we have sucient accuracy. The number of RBFs used, K, is called the order
of the t. We dene the Kth-order residual as

K
R (x) = f (x)
K
(x, pi ), (11)
i=1

where the zero-th order residual is the original data f (x). The K-th RBF is selected to
provide the best approximation to the (K 1)st-order residual by minimizing

R(K1) (x) (x, pK ) 2 wK (x, qK )dx, (12)
xU

where wK (x, qK ) is a non-negative weight to be discussed below and qK is its set of param-
eters.

6
 Remember that our ultimate objective is to use RBFs to track pulse-like traveling wave
shapes that we call bumps. If eq (12) were used with wK (x, qK ) = 1, there is a chance that
the minimizer1 would converge to a local minimum that covers several bumps. Figure 3
shows four of the possible outcomes when minimizing eq (12) with K = 1 and w1 (x, q1 ) = 1
for the data shown as the thick line (at t = 11) in Figure 2. The rst two of the outcomes are

1.6 1.6
The initial RBF The initial RBF
1.4 The estimated RBF 1.4 The estimated RBF
The real data values The real data values
1.2 1.2

1 1

0.8 0.8

0.6 0.6

0.4 0.4

0.2 0.2

0 0
0 0.5 1 1.5 0 0.5 1 1.5

(a) c = 0.6, = 0.1 and = 1 - left bump t. (b) c = 0.8, = 0.1 and = 1 - right bump t.

1.6 1.6
The initial RBF The initial RBF
1.4 The estimated RBF 1.4 The estimated RBF
The real data values The real data values
1.2 1.2

1 1

0.8 0.8

0.6 0.6

0.4 0.4

0.2 0.2

0 0
0 0.5 1 1.5 0 0.5 1 1.5

(c) c = 0.8, = 0.3 and = 1 - crossing t. (d) c = 0.8, = 0.01 and = 1 - delta t.

Figure 3: Possible outcomes (both desirable and undesirable) of the RBF tting process.
Original data, initial approximation and the nal t for dierent initial guesses of the
weight, , center, c, and width, of a single RBF t. The data set was generated from
1.5 exp((100(x 0.6)2 ) + exp(50(x 0.9)2 ) on 500 equally-spaced points. Their un-
weighted minimization was done using the data from 252 randomly selected points from
this set to illustrate that the data does not need to be evenly spaced.
1 This minimization, as all others reported in this paper, was performed with the Matlab code fminsearch

based on the Nelder-Mead simplex derivative-free method [31]. Other algorithms, including derivative-
based ones can also be used, and may well be faster. Our objective was not to establish the most ecient
minimization algorithm for this application but to explore the use of TV-RBFs.

7
desirable onesa close t to one of the two bumps.2 The other two outcomes are undesirable
and should either be avoided, or detected, so that a dierent initial guess can be tried. In
the third case, the code has located a local minimum that straddles the two bumps. This
might not be easy to detect automaticallywe will propose a weighting scheme below that
makes this outcome unlikely. The fourth outcome is what we call a delta fit, because is
nearly zero, so the RBF is essentially zero except at one discretization point in space. It can
be seen to be a local minimum, since small perturbations in have no eect while, at this
xed , perturbations in the center and amplitude increase the cost function. Fortunately,
it is very easy to detect this case and restart from a dierent initial guess. There is a
fth outcome (not shown in the gure) which is a zero solution, also easy to detect. (This
arises when the the initial center and are such that the overlap of initial RBF and the
data, f (x), integrates to less that the accuracy considered in the minimization so that small
perturbations in the parameters of the RBF do not cause appreciable change in the error of
the t.)
To avoid the undesirable crossing fit in Figure 3(c) we propose the use of the weight

(x, qw )
wK (x, qw ) = . (13)
xU
(x, qw )dx

with qw = pK , that is, weighting with a scaled form of the RBF itself, which changes as
the iteration to nd the minimum proceeds. The scaling makes the integral of the weight
unity (so that changing c and would not aect the cost function for a constant residual
RK (x)). This weight emphasizes the peak and pays almost no attention to the tails, where
the t might be strongly aected by partially overlapping bumps.
To estimate the eect of this weight on the possible outcomes, we minimized eq (12) for
K = 1 starting from 7,056 dierent initial guesses (21 16 21) with [0 : 0.1 : 2], center
c [0 : 0.1 : 1.5] and width [0.0005 : 0.01 : 0.2005]3 The minimization was run with the
weighting in eq (13) and with w(x) = 1 (no weighting) for the data shown in Figure 3 and
for the same data with independent Gaussian noise with standard deviation 0.2 added to
each data value (this data is shown in Figure 4). Table 1 shows the percentage of results in
each of the ve categories.
The frequency of zero outcomes is increased with weighting because weighting further
decreases the weight of the tails, so that there are more situations where the overlap be-
tween the data f (x) and the tted Gaussian is negligible to the accuracy required in the
minimization.
We used a range of initial guesses to generate Table 1 to illustrate the range of outcomes.
Most of those that lead to the delta and zero outcomes can be avoided by a sensible initial
guess based on the data. In subsequent experiments we started the minimization of eq (12)
with the RBF center at the largest peak in the data, an amplitude equal to that peak, and
the distance to the 20-th nearest neighbor. Table 1 illustrates the value of the weighting
in avoiding frequent crossing minima outcomesthrough the application of the weight
their frequency has dropped from almost two thirds of the cases to less than 1%, and the
percentage of desirable solutions has increased nearly vefold.
In our procedure, RBFs are added one at a time to the current approximation f (K) (x)
2 Since this synthetic data is indeed the sum of two Gaussian bumps, one might expect to be able to get

a good t. Note, however, that for a minimization with a unit weight, both of these two desirable outcomes
are shallow local minima. Many initial guesses will eventually lead to the the third, undesirable, outcome
unless they start very close to the desired solutions.
3 We use Matlab notation to indicate the discrete set of values used.

8
 1.6
1.4
1.2
1
f(x)

0.8
0.6
0.4
0.2
0
0.2
0 0.5 1 1.5
x

Figure 4: The data with added Gaussian noise, standard deviation 0.2.

No Noise Noisy
Type of solution Shown in Fig. Unweighted Weighted Unweighted Weighted
left bump minima 3(a) 0.04% 10.83% 0.16% 10.91%
right bump minima 3(b) 2.98% 15.45% 4.28% 15.16%
crossing minima 3(c) 69.70% 0.60% 63.58% 0.57%
delta minima 3(d) 4.80% 16.67% 27.58% 47.72%
zero solution - 22.48% 56.46% 4.41% 25.64%

Table 1: Percentage distribution of dierent local minimizers computed from dierent initial
guesses when tting the two bump data shown in Figure 3; also with noise as in Figure 4.

starting with f (0) (x) = 0 and using

f (K) (x) = f (K1) (x) + (x, pK ),
where the RBF (x, pK ) is chosen to be a best t to the (K 1)-st residual using eq (12).
We call this a sequential optimization step.
It is clear that, even if the data can be exactly t with a nite sum of RBFs, this
procedure is unlikely to nd that t exactly: the presence of other, overlapping bumps will
cause the t to any single bump to be slightly in error. Consequently, at some point a
simultaneous t of all current RBFs becomes desirable. This is achieved by minimizing

K
f (x) (x, pi ) 2 dx (14)
xU i=1

over all parameters {pi } immediately after the K-th RBF has been sequentially tted using
as an initial guess the parameters of the RBFs determined in the sequential process. In the

9
 0
10

1
10

RMSE
2
10

3
10 weighted cost function
nonweighted cost function

4
10
6 8 10 12 14 16
Number of slices

Figure 5: Error comparison between RBF weighted and non-weighted optimization cost
functions. An RBF was tted to the left bump in each single slice in Figure 2. The errors
are measured based on the true rst left hand side mode.

examples reported later, this was done only after the residual from the sequential t was
suciently small, (even though the simultaneous t could reduce the residual further and
it might have been possible to achieve the desired error with a smaller K).
Another illustration of the value of the weight can be seen in Figure 5. This shows
the error in tting the leftmost (largest) bump using weighted as well as unweighted min-
imization on each of the slices in Figure 2, from t = 1 to t = 16. The initial guess was
as described earliercenter and amplitude of the largest peak and the distance to the
20th-nearest neighbor. The root mean square of the dierences between the tted RBF
and the known shape of the left bump are plotted. Both errors are very small while there
is no overlapas is to be expected since the data is actually the sum of two RBFs but the
unweighted error is an order of magnitude larger than the weighted error until the former
levels o at its maximum at around t = 11 while the weighted error does not reach that
level until t = 13 since it can still locate the partially separate peak until then. Sequential
optimization with the proposed weight, leading to the nal simultaneous optimization step
is our chosen approach to initializing TV-RBF spatiotemporal data tting.

2.3 Time-Varying Radial Basis Functions

The problem with optimizing the t of each successive temporal slice of spatiotempo-
ral data independently (even if the optimization is initialized based on the previous slice
results) is that discontinuities in the time series of the RBF parameters may (and often
will) occur, as we summarized above. We now will circumvent this problem by requiring a
level of (temporal) smoothness in successive slice ts. This is achieved by approximating
the sequence of slices with a radial basis approximation whose coecients are functions of
time restricted to some smooth set, S(t), such as polynomials. In many cases we want to
estimate the rst few time derivatives of the solution (for example, to nd out if the solution
is stationary, or if it is advancing at a constant rate, or even to integrate the macroscopic
properties using projective integration [11]). Hence we assume that the parameters, pj are
low order polynomials. If we use rst order polynomials we can estimate the rst derivatives
(which are sucient for steady state calculations or projective integration). In that case,
each RBF depends on six parametersthe two linear polynomial coecients for each of ,

10
c, and . In principle higher-order polynomials or other smooth bases can also be used. As
indicated in eq (7) we consider approximations of the form

K
g(x, t) (x, pj (t)). (15)
j=1

The set of parameters at our disposal is P (t) = {pj (t) : j = 1, . . . , K} with P (t) S(t).
Our objective is to nd

n
m
K
P (t) = arg min g(xi , tk ) (xi , pj (tk ))2 . (16)
P (t)S(t)
i=1 k=1 j=1

where tk , k = 1, . . . , m are times of the m time slices. If, as in the test reported in the
next section, we restrict ourselves to Gaussian RBFs whose coecients are linear functions
of time, this gives 6K parameters that determine P (t), namely the ij , cij , and ij in

i (t) = i0 + ti1 (17)

ci (t) = ci0 + tci1 (18)
i (t) = i0 + ti1 (19)

for i = 1, . . . , K.
We start by performing a spatial t to a single slice, as described in Section 2.2, which
also determines K. If the parameters are not changing too much from slice to slice, we
can use an initial guess for the minimization eq (16) that keeps their values constant in
time by setting the i0 , ci0 , and i0 to the i , ci , and i found for the single slice and
i1 = ci1 = i1 = 0. If the slice selected for the spatial t is representative of the RBF
structure of all of the slices, and a Linear TV-RBF (LTV-RBF) approximation can give a
good t, then the solution of eq (16) should be able to nd it with such an initial guess.
For example, if the set of slices were as shown in Figure 2 then any of the rst twelve slices
(which have two distinct peaks) should provide a good initial approximation, whereas the
last four (which only have one peak) are likely to provide a poor initial approximation.

3 Numerical Experiments
We now apply the proposed framework to spatiotemporal data from two examples: a syn-
thetic data set, and certain time-dependent solutions of the Kuramoto-Sivashinsky partial
dierential equation. While the long-term objective is to use these approximation methods
in equation-free computations such as projective integration, at this stage in their devel-
opment we concentrate on simply testing their applicability. We will use linear TV-RBFs
(LTV-RBFs). The synthetic data is the sum of two linearly varying Gaussian waves, so a
perfect t to the data is possible using two LTV-RBFs; the rst test really then constitutes
an illustration of the method. The ability of an LTV-RBF approximation to t the solution
of the Kuramoto-Sivashinsky PDE over a time interval provides a rst glimpse of the po-
tential of the method for use in projective integration or coarse-grained bifurcation analysis
as we will discuss.

11
3.1 An Illustration: Synthetic Crossing Waves
In this example we model the synthetic data of two crossing waves shown in Figure 6(a).
This was generated from

(x 0.05t + 0.03)2 (x + 0.05t 0.05)2

f (x, t) = (4.8 0.1t) exp + (1.2 + 0.1t) exp
0.01 0.02
for x (0, 1.5) for t = 1, , 29.
First we performed a space t to slice 5 using the algorithm in Section 2.2. (A slice
was chosen in the interior with respect to time so that we could examine the errors in the
approximation by a linear polynomial both in the forward and backward directions in time.)
This provided a model order of 2, as well as initial guesses for the minimization of eq (7).
This was then performed over slices 5 through 12 (a better initial guess would usually be
found by applying the algorithm in Section 2.2 to a slice in the middle of the group to be
used in eq (7)).
From the results of the latter step we computed data values for all slices (1 through 29)
using the TV-RBF and compared them with the original data. (This is the nature of the
process that would be used in projective integration.) The RMSE (Root Mean Square Error)
plots of the dierences are shown in Figure 6. As expected, slices 5 through 12 (which we
directly tted) give the best results, while, as we extrapolate linearly in time (either forward
in time, slices 13 through 30, or backward in time, slices 4 through 1) the (still small) error
becomes increasingly worse. The residual error is due to termination of the minimizer when
our stopping tolerance was met. We have repeated the experiment with various levels of
additive uniform as well as Gaussian noise, and the results suggest good robustness of the
algorithm. We have also performed numerical experiments with more than two waves and
observed similar results. Neither are shown here.

5
x 10
1.2

6 0.8
RMSE

4 0.6
f(x,t)

2 0.4
1.5
0 0.2
30 1
20 0.5
10 0
0 0 5 10 15 20 25 30
t 0 x Slice number
(a) Original data and output of the TV-RBF (b) RMSE of the linear t projected over the en-
model tire time interval

Figure 6: TV-RBF performance in modeling the data set shown in (a) (see text for details).
Slices 5 through 12 (denoted by blue color in the on-line gures, but appearing as dark
lines in the black and white printed version) have been used for computing the parameters
associated with the TV-RBF model. The RMSE of the (projected) linear t is shown at
each of the 29 slices.

12
3.2 A Test: the Kuramoto-Sivashinsky data set
To illustrate the potential application of this method, we will use the one-dimensional non-
linear Kuramoto-Sivashinsky equation given in eq (1) with periodic boundary conditions.
We have shown a particular solution of this in Figure 1. For the transient in our example,
the spatial domain over which this problem was solved (and thus the edges at which peri-
odic boundary conditions were applied) was [0, 2/ 4/53.3]. A spectral algorithm with 64
Fourier modes was used. The integration was performed with xed time steps of 0.1 using
the algorithm proposed in [23]. 128 spatial points were used, and the Fourier coecients of
the initial conditions were [0.1409; 0.1417; 0.0671; -0.1207; 0.0717; 0.1630; 0; 0; ... 0] where
the size of the vector is 126 (the constant term is zero and the vector contains the real and
imaginary terms in pairs). For this spatial domain size, the trajectory approaches a Mod-
ulated Traveling Wave (MTW) attractor. By t = 50 the eect of the transients from the
initial conditions have essentially disappeared, so the initial 500 time slices corresponding
to startup transient behavior were discarded.
Figure 1 shows the transient solution of eq (1) at unit time intervals from t = 50 to
t = 100 (every 10th slice from the actual output). We extracted two sections from this
solution for testing: the slices for t [50, 55] and the slices for t [60, 65]. Each of these
contains 51 slices since 0.1 is the time increment. Every fth slice of these sections is shown
in Figure 7.
Figure 8 shows the TV-RBF performance on tting/extrapolating the data presented
in Figure 7(a). The fth-order RBF representation of the 20th slice in Figure 8(a) ts
to less than 102 in the normalized RMSE (the RMSE divided by the maximum value of
the function, NRMSE). As indicated in our procedure, this is the result of a sequential t
followed by a simultaneous adjustment to the RBF parameters. The semi-log plot of the
NRMSE as RBFs are added to the model is given in Figure 8(b). This provides the basis
for stopping at a fth order approximation. The linear time projection of the solution to
slices 21 through 51, based on the TV-RBF training on slices 1 20, is compared with
the numerically integrated solution in Figure 8(c). The data and the approximation at the
nal slice (the 51-st that has the worst error) are plotted with thicker lines to make them
more visible. Also, to make that slice more visible, the gure has been rotated so that the
nal slice is in the front. Figure 8(d) shows the semi-log plot of the NRMSE as a function
of time.
The size of these errors indicates that the method is able to produce reduced, yet useful
estimates of the time derivative of the detailed discretized behavior, which can then be
used in higher-level processes (here one step of projective integration is shown) but also
in other processes, such as (coarse) xed point and (coarse) bifurcation computations. We
have also performed a similar analysis of the later part of the KS data set shown in Figure
7 (b). The results of applying our tting method to this data set and the associated time-
projection results are shown in Figure 9. The TV-RBF formulation is thus seen to produce
a parsimonious local model for solutions containing modulated bumps with varying speeds.

4 Summary and Conclusion

We have proposed novel RBFs whose coecients are Time-Varying (TV-RBFs); the coe-
cients of these RBFs are local polynomials in time. The TV-RBFs introduced in this work
are capable of providing parsimonious local nonlinear representations of space-time data,
especially of data in the form of (possibly more than one) travelling/modulating spatially
localized waves. These functions contain no ad hoc parameters that have been tuned.

13
 3 3

2 2

1 1
f(x,t)

f(x,t)
0 0

1 1

2 2
55 65

50 60 10
t 10 5 0 5 10 t 10 5 0 5
x x
(a) Slices between times 50 and 55 (b) Slices between times 60 and 65

Figure 7: The two parts of the original data that fall within the time intervals [50, 55] and
[60, 65] have been selected. These data sets are used for TV-RBF modeling. Each data set
consists of 51 slices. Every 5th slice is plotted for better visibility.

A key idea is to ensure a continuous in time reduced representation of such spatiotem-

poral data; traditional RBF tting of successive temporal slices of such (smooth) data
may well exhibit discontinuities in the reduced representation as an artifact of the tting
process. Such articial discontinuities are eectively resolved with the introduction of the
new basis function sets.
We demonstrated linear-in-time RBFs; the study of the higher order polynomial depen-
dence in time is, in principle, straightforward. While only symmetric RBFs were used in
our study, skewed RBFs recently introduced in [18] could also be employed.
A key motivation is the local data-based estimation of the (explicitly unavailable) right-
hand-sides of the evolution equations for the shape parametrization; the use of such estimates
in equation-free algorithms has been repeatedly demonstrated in the past to accelerate
the dynamic simulation and numerical bifurcation analysis of legacy dynamic simulators
through the use of matrix-free iterative linear algebra techniques. More importantly, if the
TV-RBFs are used to parametrize coarse-grained features of the solutions of atomistic or
individual based simulators (e.g. to parametrize the particle density eld as estimated from
an interacting particle simulation) then coarse projective integration and coarse bifurcation
analysis are enabled. The choice of TV-RBF coecients as the coarse variables may prove
a key enabling factor for such equation-free coarse-grained multiscale computations.
Acknowledgments This work was supported by the US Department of Energy under
Award DE-SC0002097.

References
[1] D. S. Broomhead and D. Lowe. Multivariable Functional Interpolation and Adaptive
Networks. Complex Systems, 2:321355, 1988.

[2] H. S. Brown and I. G. Kevrekidis. Modulated Traveling Waves for the Kuramoto -
Sivashinsky Equation, The Fields Institute Communications Series, Pattern Formation:

14
 0
2.5 10
PDE solution & approximation RBF approx.
2 PDE solution

1.5
1
1 10

log NRMSE
0.5

0
2
0.5 10

1.5
3
2 10
0 20 40 60 80 100 120 0 1 2 3 4 5
x number of sRBFs
(a) The Gaussian RBF t of the tenth slice (b) The semi-log plot of the NRMSE as new basis
(20 slices were used for computing the TV-RBF functions are added to the model
model)

1
10
3
PDE solution & approximation

TVRBF
PDE solution
2 log NRMSE

2
30
40 10 2
5 0 5 10
slic 50 10 20 25 30 35 40 45 50
e x Slice number
(c) The original data and the TV-RBF model (d) The semi-log plot of the NRMSE of the
output for projected slices output of the model for the projected slices 21
through 51

Figure 8: The performance of the TV-RBF algorithm on the data set shown in Figure 7
(a). This data set consists of 51 time slices. The rst 20 time slices have been used for
estimating the TV-RBF parameters and the remaining 31 slices are used for determining
the quality of the projected model. The time slices have been downsampled by a factor of
5 for further visibility, resulting in the 7 slices plotted.

Symmetry Methods and Applications, (ed. J. Chadam, M. Golubitsky, W. Langford and

B. Wetton), 5:4566,1995

[3] M. D. Buhmann. Radial Basis Functions. Cambridge University Press, 2003.

[4] E. Deane, G. E. Karniadakis, I. G. Kevrekidis and S. A. Orszag. Low-Dimensional
Models for Complex Geometry Flows: Application to Grooved Channels and Circular
Cylinders. Physics of Fluids A, 3:23372354, 1991.

[5] E. J. Doedel, H. B. Keller and J. P. Kernevez. Numerical analysis and control of

bifurcation problems I. Bifurcation in Finite Dimensions. Int. J. Bifurcation and Chaos,
1(3):49350 (1991).

15
 3 0
10
RBF approx.
PDE solution & approximation PDE solution

1
10

log NRMSE
1

0
2
10

3
2 10
0 20 40 60 80 100 120 0 1 2 3 4 5
x number of sRBFs
(a) The Gaussian RBF t of the tenth slice (b) The semi-log plot of the NRMSE as new basis
(20 slices were used for computing the TV-RBF functions are added to the model to t the 10th
model) slice

1
10
PDE solution & approximation

3 TVRBF
PDE solution

2
log NRMSE

2
30
40 5 10 2
5 0 10
sli 50 10 20 25 30 35 40 45 50
ce
x Slice number
(c) The TV-RBF model output for projected (d) The semi-log plot of the NRMSE of the
slices output of the model for the projected slices 21
through 51

Figure 9: The performance of the TV-RBF algorithm on the data set shown in Figure 7 (b).
This data set consists of 51 time slices. The rst 20 time slices have been used for learning
the TV-RBF model and the remaining 31 slices are used for determining the quality of the
projected model. For the solution plots, the time slices have been downsampled by a factor
of 5 for further visibility, resulting in 7 slices.

[6] E. J. Doedel, H. B. Keller and J. P. Kernevez. Numerical analysis and control of bi-
furcation problems II. Bifurcation in Finite Dimensions. Int. J. Bifurcation and Chaos,
1(4):745772 (1991).
[7] J. M. English and R. L. Pego. On the solitary wave pulse in a chain of beads. Proc.
Amer. Math. Soc., 133:17631768 (2005).
[8] G. Friesecke, R. L. Pego. Solitary waves on FPU lattices II: Linear implies nonlinear
stability. Nonlinearity, 15:13431359 (2002).
[9] G. Friesecke, R. L. Pego. Solitary waves on FPU lattices I: qualitative properties,
renormalization and continuum limit. Nonlinearity, 12:16011627 (1999).

16
[10] C. W. Gear and I. G. Kevrekidis. Projective Methods for Sti Dierential Equations.
SIAM J. Sci. Comp. 24(4) pp1091-1106 (2003).

[11] C. W. Gear, I. G. Kevrekidis and K. Theodoropoulos. Coarse stability and bifurcation

analysis using timesteppers: a reaction diusion example. Comp. Chem. Engng. 26:941
963 (2002).
[12] R. L. Harder and R. N. Desmarais. Interpolation Using Surface Splines. Journal of
Aircraft, 9(2):189191, 1972.
[13] R. L. Hardy. Multiquadric Equations of Topography and other Irregular Surfaces.
Journal of Geophysical Research, 76(8):19051915, 1997.
[14] C. C. Holmes and B. K. Mallick. Bayesian Radial Basis Functions of Variable Dimen-
sion. Neural Computation, 10(5):12171233, 1998.
[15] P. Holmes, J. L. Lumley and G. Berkooz. Turbulence, Coherent Structures, Dynamical
Systems and Symmetry. Cambridge University Press, Cambridge, 1996
[16] A. A. Jamshidi. Modeling Spatio-Temporal Systems with Skew Radial Basis Functions:
Theory, Algorithms and Applications. Ph.D. dissertation, Department of Mathematics,
Colorado State University, Fort Collins, CO, Summer, 2008.

[17] A. A. Jamshidi and M. J. Kirby. Towards a Black Box Algorithm for Nonlinear Function
Approximation over High-Dimensional Domains. SIAM Journal of Scientific Compu-
tation, 29(3):941963, May 2007.
[18] A. A. Jamshidi and M. J. Kirby. Skew-Radial Basis Function Expansions for Empirical
Modeling. SIAM Journal on Scientific Computation, 31(6):47154743, 2010.
[19] A. A. Jamshidi and M. J. Kirby. Modeling Multivariate Time Series on Manifolds with
Skew Radial Basis Functions. Neural Computation, 23(1):97123, 2011.
[20] A. A. Jamshidi, M. J. Kirby, and D. S. Broomhead. Geometric manifold learning
using dimension reduction and skew radial basis functions. special issue of IEEE Signal
Processing Magazine on Dimensionality Reduction via Subspace and Manifold Learning,
28(2):6976, March 2011.

[21] I. T. Jollie. Principal Component Analysis. Springer Series in Statistics, Springer-

Verlag, New York, 2nd ed. 2002

[22] N. B. Karayiannis and G. W. Mi. Growing Radial Basis Neural Networks: Merg-
ing Supervised and Unsupervised Learning with Network Growth Techniques. IEEE
Transactions on Neural Networks, 8(6):14921506, Nov. 1997.
[23] A. K. Kassam and L. N. Trefethen. Fourth-Order Time-Stepping for Sti PDEs. SIAM
Journal on Scientific Computation, 26(4):12141233, 2005.
[24] I. G. Kevrekidis, C. W. Gear, J. M. Hyman, P. G. Kevrekidis, O. Runborg and
K. Theodoropoulos. Equation-free coarse-grained multiscale computation: enabling
microscopic simulators to perform system-level tasks. Comm. Math. Sciences, 1(4):715
762 (2003)

17
[25] M. E. Kavousanakis, R. Erban, A. G. Boudouvis, C. W. Gear and I. G. Kevrekidis.
Projective and coarse projective integration for problems with continuous symmetries.
J. Comp. Phys., 225:382407 (2007).

[26] Yu. A. Kuznetsov. Elements of Applied Bifurcation Theory, vol. 112, Appl. Math. Sci..
Springer, New York, 2004.
[27] P. V. Lee and S. Haykin. Regularized Radial Basis Function Networks Theory and
Applications. Wiley, New York, 2001.
[28] E. Meadows, M. A. Henson, J. W. Eaton and J. B. Rawlings. Receding hotizon control
and discontinuous state feedback stabilization. Int. J. Control, 62(5):12171229 (1995).
[29] D. E. Myers , S. De Iaco, D. Posa and L. De Cesare. Space-Time Radial Basis Functions.
Computers and Mathematics with Applications, 43:539549, 2002.
[30] I. T. Nabney, A. McLachlan, and D. Lowe. Practical Methods of Tracking Non-
Stationary Time Series Applied to Real World Data. In S. K. Rogers and D. W. Ruck,
editors, SPIE Proceedings Applications and Science of Artificial Neural Networks II,
2760:152163, 1996.
[31] J. A. Nelder and R. Mead. A Simplex Method for Function Minimization. Computer
Journal, 7:308313, 1965.
[32] J. Park and I. W. Sandberg. Universal Approximation Using Radial-Basis-Function
Networks. Neural Computation, 3:246257, 1991.
[33] J. Park and I. W. Sandberg. Approximation and Radial-Basis-Function Networks.
Neural Computation, 5:305316, 1993.
[34] J. Platt. A Resource Allocating Network for Function Interpolation. Neural Computa-
tion, 3:213225, 1991.
[35] D. E. Pelinovsky. Translationally invariant nonlinear Schrdinger lattices. Nonlinearity,
19:2695-2716 (2006).

[36] M. J. D. Powell. Radial Basis Functions for Multivariable Interpolation: A Review. In

J. C. Mason and M. G. Cox, editors, Algorithms for approximation, 143167. Clarendon
Press, Oxford, 1987.
[37] M. J. D. Powell. The Theory of Radial Basis Functions in 1990. In W. Light, editor,
Advances in Numerical Analysis, volume II of Wavelets, Subdivision Algorithms, and
Radial Basis Functions, 105210. Oxford University Press, 1992.

[38] D. E. Rumelhart and J. L. McClelland. Parallel Distributed Processing. MIT Press,

Cambridge, MA, 1986.

[39] R. Schaback and H. Wendland. Characterization and Construction of Radial Basis

Functions. In N. Dyn, D. Leviatan, D. Levin, and A. Pinkus, editors, Multivariate
Approximation and Applications, Eilat Proceedings, 124. Cambridge University Press,
2000.

[40] B. Sandstede. Stability of multiple-pulse solutions. Transactions of the American

Mathematical Society, 350:429472 (1998).

18
[41] H. Wendland. Scattered Data Approximation. Cambridge University Press, 2005.

[42] P. J. Werbos. Beyond Regression: New Tools for Prediction and Analysis in the Be-
havioral Sciences. Ph.D. dissertation, Division of Applied Mathematics, Harvard Uni-
versity, Boston, MA, Aug. 1974.

[43] L. Yingwei, N. Sundararajan, and P. Saratchandran. Performance Evaluation of a

Sequential Minimal Radial Basis Function (RBF) Neural Network Learning Algorithm.
IEEE Transactions on Neural Networks, 9(2):308318, Mar. 1998.

19