2010 International Conference on Computer Application and System Modeling (ICCASM 2010)

A novel learning control method of Macro-Micro robot

Jia Xing Haiyong Chen, Guansheng Xing, Tao Liang

Department of Mathematics and physics School of Control Sciences and Engineering
Tianjin University of Technology and Education Hebei University of Technology
Tianjin, China Tianjin, China
e-mail: xingjiahappy@163.com e-mail: haiyong.chen@hebut.edu.cn

Abstract-A novel control scheme on learning frame is stochastic tracking errors for closed loop systems. Hence, it is
proposed for control of uncertain robot manipulator natural for us to analyze the stochastic distribution of tracking error
performing periodic tasks. In the paper, the uncertainties in from iterative process as a starting point of the required controller
the robot system may follow non-Gaussian distribution or design [7].
Gaussian distribution, but the premise condition is that they Since the probability density function (PDF) of the
are bounded. The nonlinear relation formulation between induced random disturbances in robotic systems may not
controller gain and tracking trajectory error of robot in each obey Gaussian distribution, hence the control methods based
batch has been given. The optimal process of controller gain on stochastic control theory might achieve high performance
has been presented by using the proposed ILC frame. In the [8, 9, 10]. At present, fixed structure PDF controllers such as
end, the convergence condition is analyzed and the
PI and PID have been widely studied because they are the
experimental result is given. It is shown that all the error
most popular controller in industrial applications [11].
signals in the learning control system are bounded and the
Moreover, many industrial systems are controlled in a non­
robot motion trajectory converges to the desired one
continuous way called as "Batch Control". In such control
asymptotically. Experimental results demonstrate the
systems, the idea of Iterative Learning Control (ILC) can be
feasibility and effectiveness of the proposed control method for
repetive robot motion.
implemented to control the output PDF in batch-by-batch
mode. A numerical optimization along with the P-type ILC
Keywords- learning control; robot probability density law has been employed for the output PDF control [12,13].
function Following the recent developments on the modeling and
shape control of the output PDF, in this paper we cast
trajectory tracking control into the stochastic frame. Within
I. INTRODUCTION
this framework a trajectory tracking control strategy is
Despite numerous advanced control schemes have proven to further investigated for robotic system with non-Gaussian
be successful so far [1,2], the classical PID algorithms are stochastic disturbances. Obviously, the approach proposed in
prevalent in current commercial robot systems due to their this study is not only an extension of shape control of the
desirable properties such as robustness, reliability and simplicity[3]. output PDF, but also a novel method to analyze and design
This is especially true under some practical industrial applications, control systems for robotic system subjected to stochastic
where varying environment, imperfect system models, poor
disturbances
operation circumstance usually render the application and
This paper is organized as follows: Section 2 formulates
implementation of advanced control quietly difficult. So, the linear
the problem; Section 3 develops an ILC algorithm
PID controllers are inadequate for precise control of robot
manipulators under these uncertain circumstances. In these cases, formulation and gives the realization of learning control
sometimes the stochastic disturbances or uncertainties of scheme; Section 4 presents a simulation example to illustrate
circumstance will dominate the final performance of robot system the efficiency and feasibility of the proposed approach. The
though the PID-type linear feedback control method is known to last section concludes the paper.
provide reasonably good performance under the small limited
disturbances assumption [4,5]. II. PROBLEM FORMULATION
Our particular interest here is to investigate the influence of When the uncertainties being considered, the dynamic equation
stochastic disturbances or uncertainties of circumstance on the of a general manipulator includes not only the certain part but also
control of robotic manipulators. Regardless of the types of control uncertain part, the dynamics of the n rigid-link manipulators can be
strategies being used, the final control performance is always expressed as
affected by stochastic disturbances or uncertainties of circumstance. H(q)ij+C(q,q)q+FA +G(q)+Tr = T (1)
In addition, there are highly nonlinear characteristics in robotic
manipulator; therefore, an advanced methodology is required to
where the inertial matrix H(q)= Ho(q)+b.H(q) is assumed
handle the uncertain robotic manipulator control systems. to be a bounded and nXn positive definite matrix,
Iterative learning control (ILC) is able to determine a control C(q,q)= Co(q,q)+!1C(q,q) is the nXn Coriolis, centripetal
input iteratively so that it is possible to track a given reference force matrix, and G(q)= Go(q)+!1G(q) is nxl gravity
signal or the output trajectory over a number of fixed time
intervals[6]. Robot control is one of the most common applications vector, q(t)E Rn is the nxl joint variable vector and T(t) is
of ILC. In the stochastic control framework, the random nX 1 the joint control input vector, T r(t) is a unknown disturbance
disturbances induced by various kinds of sources generally lead to vector which is assumed to be bounded, Fv denotes the nXn

978-1-4244-7237-6/10/$26.00 .2010 IEEE V4-511

 2010 International Conference on Computer Application and System Modeling (ICCASM 2010)
diagonal matrix of viscous friction coefficients. The parameter
uncertainties and external disturbances in (I) may be regarded as an
integrated disturbance that can be given by
w = M1(q)+D.C(q,q)q+D.G(q)+Tr
Assume that the disturbance w follows non-Gaussian
distribution. Equation (2) can be given by
Ho(q)ij+Co(q,q)q+FA +Go(q)+w = T
When the robot manipulator is performing a repetitive
operation, the desired robot trajectory and unknown inverse
dynamics inputs can be formulated by using T-periodic
functions as follows:
{qd(t), qd(t), ijd(t), Td(t)} =
{qd(t+T), i]d(t+T), ijd(t+T), Td(t+T)}
Let desired joint trajectory be qd(t) , where the subscript d
represents the given tr�ectories and input. The desired input
vector can be given by
Td = Ho(q)ij+Co(q,q)i]+FA +Go(q)
Then, a learning controller should be designed so as to
realize the trajectory tracking that robot trajectory q(t) track
desired trajectory qd(t) for all t in the defined interval [0,
T]. Joint position error can be given bye = qd-q . Define
variable qr as follows
q,. =i]d+A(qr-q)=qd+Ae
Where A is a positive definite matrix. Taking the control torque T
as a function of the state. Then a learning controller is
introduced, which is a combination of two input components:
a feed forward learning input component and pseudo
dynamic input component.
T(t) = Tfp (t)+Tpd(t)
Where Tfp (t) the feed forward learning is input and Tpd(t)
is pseudo dynamic input. The feedback control input. Tpd(t)
is in charge of stabilizing the closed-loop system and is
computed according to following formulation
Tfp (t) = HO(q)ijr +Co(q,i])i],. +FA +Go(q)
The learning control input Tpd(t) compensates for the
nonlinearity or random component of uncertain robotic
system and can be given by
Tpd(t) = Pk(qr-q)+ik S(qr-q)dt+dk(i]r-q)
Then, the closed-loop system equation is described by
Ho(q)(ijr -ij)+Co(q,q)(qr -q)+
Pk(qr -q)+ ik S(qr -q)dt+dk(i]r -q) = w
Equation ( 9) can be further expressed as
i]r-q = e+Ae ijr-ij = ii+M
By substituting (6) into (S), the error dynamic equation is given by
HO(qd-e)(ii+M)+CO(qd-e,qd-e)(e+Ae)+
dk(e+Ae)+Pk(qr -q)+ ik S(qr -q)dt = w
Denote r = qr-q = e+Ae, Pk = ik = 0 Equation (13) can be
further expressed by
(8)
Obviously, Equation (8) is a second-order linear differential
equation with respect to r. Pk should be modified so as to
compensate the tracking error brought by the nonzero disturbances.
Moreover, if Pk and A are both positive definite matrix, this can
ensure nonlinear dynamic ( 7) to keep stability [17].
(2)
Assume that the w is bounded and its PDF is non­
Gaussian distribution, and the bounded interval of e denoted
as e E [a, b]. The controller design for non-Gaussian
stochastic system should be performed such that the shape
of the PDF of the tracking error is made as close as possible
to the desired PDF. After introducing necessary
mathematical tools, the iterative learning control framework
will be developed further in the next section.
The basic idea in reducing the role of some stochastic
disturbances is to update the PID gains using the adaptive
gain learning rules as well as the learning control inputs
using the feed forward input learning rules. Fig. 2 shows the
adaptive PID learning controller in which PID gains are
tuned on-line using a set of learning rules. By doing so, the
adaptive PID controller keeps the error bound of learning
system within a reasonable region though the some
disturbances existence. Hence, the problem is how to
(3) construct a set of learning rules for on-line tuning of
controller gains to reduce the role of the some disturbances
and guarantee stability of the closed-loop system throughout
the learning process. In the following, a set of feasible
learning rules are derived for controller gain tuning.
III.
ALGORITHM FORMULAnON
The ILC was first proposed for a class of systems where
the closed-loop system operated similarly in a number of
repetitive periods called batches [14]. The key idea of ILC is
that the control input in the k-th batch is determined based
on the control input in the (k-l)-th batch and a correction
term which determines the ILC type. The correction term is
related to the closed-loop performance in the (k-l)-th batch
[IS, 16]. This strategy has the advantage of improving the
closed-loop system performance batch-by-batch.
The aim of using ILC in this work is to adjust the
controller's parameters such that the shape of the PDF of the
(4)
tracking error is made as close as possible to the desired
PDF batch-by-batch [17]. Within each batch, the parameters
of controller are fixed and the process output error data and
(S) the parameters of controller are used to calculate the
parameters of controller in the next batch. Between each
batches, the memorized output error information and other
(6) necessary data from the previous operation are used to tune
the parameters of controller, as is shown by Fig.l. As a
result, the following three-stages of the control algorithm
(7) design can be given by: 1) Control the closed-loop system
based on the initial values of the controller in the first batch;
2) Form the PDF of output error, the performance function
of PDF and stochastic approximation method is used to
calculate parameters of controller; 3) Update the parameters
V4-S12
 2010 International Conference on Computer Application and System Modeling (ICCASM 2010)

of controller and apply them to the process in each bathes, measurement of PDF of the error in the k-th batch can be
then return to stage 2).The iterative tuning rules of whole represented by Yek t- {ek,I' ek,z'"'' ek,N}
control process are applied to tune parameters of controller.
This would be different from exiting ILC where the control Using ek,i and by performing the calculation of the
input is tuned between each batches directly. cumulative distribution function in the k-th batch as

F(ek) = � (The number of {ek'i ek }), \fek,i

� E [a,b] (12)

Then the probability density function in the k-th batch can

be obtained as

(13)

where I'1F and l'1ek are the incremental values for F and ek,
respectively.
Conlroller Parameters update

Figure 1. ILC- based output PDF control. B. Tuning of controller parameter

The controller design is a nonlinear optimization problem that
can be solved using various optimization techniques. However it is
A. The calculation of the PDF
worth noting that performance index in ( 9) changes stochastically
with respect to gain Pk and its optimization IS random
The closed-loop PDF tracking performance to be minimized
throughout the whole control process is defined as
optimization process that can be given by
Jk = r(Yk(y-qd,Pk)-g(y»)zcry PhI =Pk -rk'1J(Pk) (14)
( 9) Where rk is searching step,
(
+ r YYk(Y-qd,Pk>4Y r =
1
LJJ(Pk) _ _[ J(Pk + gJ,)-}(Pk - gJ,»)'i 1, 2,3, '" =

Where, Jk is the closed-loop performance in the k-th batch, 2 gk

Where, gk is step in the k-th batch, Ii is unit coordinated vector,
Yk (e, Pk) denotes the PDF of tracking error in the k-th batch.
l'J(Pk) is the evaluation value of gradient of the function .In order
From ( 9), (10) and (13), it leads to the following nonlinear to enable the random approximation process converge to extreme
function relation point, following conditions should be satisfied
ek,i =fCQk;Pk;Wk,i;qd) (10) condition 1 :rk > 0, gk > 0,
Where, Qk =(qk,pqk,Z,.. ·,qk,N;Uk,pUk,Z, , Uk,N) condition 2 : lim rk =0, lim gk =0,
k�oo k�DO
. . · , N denotes

sample number, ek,i is the i-th error value in the k-th batch,
Wk,i is the i-th noises value in the k-th batch; Pk represents the condition 3 : L rk = 00, (15)
control gain in the k-th batch, qd is the desired value, f k=1
denotes a nonlinear function. According to the probability
theory, the PDF of the ek,i can be given by condition 4 :
k=1 gk
f(!L)Z < 00,

)larl(Qk�:k )1
Yeki =Yw (J-l(Qk;Pk;X;qd)
Where f -1 is the inverse function with respect to w , the
upper and lower bounds of tracking error are a2 and b2,
respectively, x E[a2,b2]. In this case, y is continuous and
first-order differentiable with its variables. From
performance index (9), the gain Pk controls the shape of the
PDF of tracking error at time in the k-th batch. However, in
practice, to control the PDF of the tracking error in each
batch is more important than to control the PDF of the
tracking error at each time in the k-th batch. The PDF of
tracking error in each batch directly reflects the distribution
of output tracking error. For example, if the production
quality depends on the size of tracking error, the quality
distribution can be illustrated by the PDF. Therefore, using
the sampled error data in the k-th batch, ek,i, the
;x;qd
(11) Remark: condition 1 is necessary of solving the minimized
problems, if the aim is to solve the maximized problems r k
<0 is necessary; condition 2 is necessary in order to
accurately solve the minimized problems. If rk didn't
converge to zero the searching process would continue
permanently because of the random characteristics of the
performance index (9); condition 3 ensures that rk reduces
slowly, which is not relative to optimal condition, condition
4 ensures that search process ends quickly in the
neighborhood of the limit.
IV. EXPERIMENTAL RESULTS
The macro manipulator can move widely in the X axis
and Y axis, but is not easy to realize fast and precise motion
because of large mass and volume. To compensate the
tracking error, micro manipulators is attached at the tip of
the Macro part. Both macro part and micro part are rigid,
which is completely different from some space robots that

V4-513
 2010 International Conference on Computer Application and System Modeling (ICCASM 2010)
generally own a flexible macro manipulator and a rigid
micro manipulator.
Macro and Micro translational motion part possess
four degrees of mobility, two degrees of freedom, which are
capable of accurately, smoothly adjusting position of torch
in a large range. The macro part works in the two modes,
namely planning motion and correction motion. The micro
part mainly works in the correction mode. In order to assure
smoothly motion of torch and seam tracking accuracy,
motion correction is carried out by using small steps.
The visual system mainly consists of CCD camera, laser
and master computer which performs the required image­
processing. The vision sensor moves with the end-effector,
as is shown in Fig.2. The vision system can measure the
coordinates of the seam characteristic points reliably based
on the active laser triangulation principle.
Figure 2. the construction of the Macro-micro robot
The vision system is mainly responsible for two tasks.
One is simple teaching before welding; the other is real-time
error detection during the welding. Simple teaching
technique is defined as autonomous acquisition of several
seams coordinates using visual system preceding welding.
Real-time error detection technique is defined as real-time
error detection between the endpoint of welding torch and
seam during welding.
In this section, experiments are provided in order to
verify that the proposed two work modes can be realized by
the macro-micro robot and proposed control methods are
feasible. In the experiment, the macro-micro welding robot
is required to track a planar S curve seam. The curve is easy
to examine the robot's tracking performance. Because both
the seam trajectory tracked by the macro and control
systems of the robot is subjected to various noises, the
accurate trajectory tracking is difficult to implement. So
correction control of the micro manipulator is used for
compensating the seam errors. The bias of the seam can be
detected in real-time, and then the micro manipulator can be
used for correcting the bias because of its quick response
characteristics.
In the experiments with stochastic distribution control
using ILC, the gain of controller is adjusted with the
proposed method in section 3.
V4-S14
l :J �
o 2 4 6 8 10 12 14 16 18 20
iteration
t:�1 S::� �:J
o 2 4 6 8 10 12 14 16 18 20
iteration
Figure 3. the history of controller and performance corresponding
Fig.3 shows that the tuned process of controller gain gradually
convergent. However, it can not convergent to some value, but to a
little interval because of the random effects of the performance,
which is shown by the above figure of the Fig.3.
The history of performance index versus batch is shown
in the below half of Fig.3, which shows the descending
performance index versus the increase of the batch numbers.
This confirms that improvement of the tracking error PDF
using the proposed stochastic distribution control method
based on the ILC scheme.
The aim of iteration control is to drive the PDF of
tracking error to approximate the given PDF as close as
possible, which can be regarded as desired curve, as shown
by the blue curve in the Fig.5. According to the FigA, it is
seen that our aim has been reached apparently, as also can
be seen by the Fig.5, which illustrates the contrast between
initial PDF, final PDF and desired PDF.
As the performance index about PDF of tracking error is
applied, the probability of the tracking error in some interval
can be easily calculated by using integral formulation,
which has been obtained in the Fig.6. The probability or
PDF of tracking is advantage to help us directly know about
distribution and accuracy of tracking error, moreover,
stochastic distribution of the tracking error is not limited to
normal distribution, furthermore stochastic distribution of
the tracking error may follow any distribution.
Fig.6 is also shown that the probability of tracking error
in the interval [-0.5, 0.5] is getting more and more high
when iteration number increases.
v. CONCLUSIONS
Probability density function shape control theory has
been applied to make the PDF of error approach the given
PDF as close as possible. Based on a macro-micro welding
robot, the planar S seam tracking is realized in real time.
Moreover, the robot is able to exploit the advantages of
teaching mode and real-time tracking. To finish the PDF
control of tracking error, an ILC frame and an optimization
method are proposed to adjust the controller parameters
batch-by-batch. The experimental results verify the
effectiveness of the control scheme, and the aim that the PDF
of tracking error follows desired PDF is realized.
 2010 International Conference on Computer Application and System Modeling (ICCASM 2010)

and research starting fund of Hebei University of

Technology for their support to this work.

REFERENCES
[I) M. Liu, "Decentralized control of robot manipulators: nonlinear and
adaptive approaches", IEEE Trans. Automat Control, vol. 44,
pp.357-363,1999.
(2) P. Rocco, "Stability of PID control for industrial robot arms", IEEE
Trans. Robotics Automat voLl2,pp. 606-614,1996.
(3) L Cervantes, 1. A. Ramirez, " On the PID tracking control of robot
manipulators". Systems and Control Letters, volA2,no.l,pp. 37-46,
2001.
(4) J A. Ramirez, L Cervantes, and RKelly, "PID regulation of robot
iteration o .2 manipulators: stability and performance". Systems and Control
e/mm
Letters, voL41,no.2, pp. 73-83, 2000.
Figure 4. the PDF history of the error with respect to iteration (5) A. Pervozvanski, L RFreidovich, "Robust stabilization of robotic
manipulators by PID controllers". Dynamics and Control, vol.9,
no.3,pp. 203-222, 1999.
(6) S. R Oh, Z Bien,and LH. Suh, "An Iterative Learning control
· Inilia ' PD
F 1 method with application for the Robot Manipulator", IEEE J Robotics

2.5 I- - final PDF

- Desired . PDF
(7)
Aut, vol. 4, no.5, pp. 508-514, 1988.
H. Wang, JF. Zhang, H.Yue, "Iterative Learning Control of Output
PDF Shaping in Stochastic Systems", Proc. IEEE Inti. Sym.
intelligent Control, pp. 1225-1230, 2005.
(8) H. Wang, "Minimum Entropy Control of Non-Gaussian Dynamic
Stochastic Systems", IEEE Transaction On Automatic Control,
vol.47,no.2, pp.398-503, 2002.
(9) H. Yue and H. Wang, "Minimum tracking error entropy control for
non Gaussian stochastic systems, "IEEE Transaction On Automatic
Control, vol.48,pp. 118-122, 2003.
(10) Y Yi, L Guo, and H. Wang, "Adaptive Statistic Tracking Control
Based on Two-Step Neural Networks With Time Delays" . IEEE
e/mm Transactions On Neural Networks, vol. 20, no. 3, ppA20-429, 2009.
[II) L. Guo and H. Wang, "Generalized discrete-time PI control of output
Figure 5. the compare between initial PDF, Final PDF and desired PDF PDFs using square root B-spline expansion", Automatica, vol.41,pp.
159-162, 2005.
(12) . P. G. Afshar and H. Wang. "ILC-based Adaptive minimum Entropy
Control for General Stochastic Systems Using Neural Networks",
095
IEEE Conference on Decision and Control, pp.252-257, 2007.
0.9
(13) . J H Zhang and H Wang. "Iterative learning-based minimum tracking
error entropy controller for robotic manipulators with random
085
� communication time delays. lET Control Theory and Applications,
� oe vol. 2,pp. 682-692, 2008
:.g 0.75
"
(14) ,A. Arimoto, S Kawamura and F. Miyazaki. "Bettering operation of
�
� 07 robots by learning", J of robotic systems, voLl,pp.123-140, 1984.
0.65 [IS) .0. H. Owens and J Hatonen. "Iterative Learning Control- An
06
optimization paradigm", Annual Reviews in Control, vol.29, pp. 57-
70, 2005.
065 0������--�1O��12--�14--'�6�1�'
Iteration
(16) .1. X. Xu and RYan. "Iterative Learning Control design without a
priori knowledge of the control direction", Automatica,vol. 40,
Figure 6. the probability of error in the interval [-O.5mm, O.5mm) pp.1S03-1S09, 2004.
(17) P. G. Afshar, H. Yue and H. Wang. "Robust Iterative Learning
Control of Output PDF in Non-Gaussian Stochastic Systems Using
ACKNOWLEDGMENT Youla Parameterization", Proceedings of the 2007 ACC, New York,
USA, pp:576-5SI,2007.
The authors would like to thank the Natural and Science
Fund Of Hebei Province (F2010000160 and F2009000119)

V4-5J5