Haptic Interface for Intuitive Teleoperation of Wheeled and Tracked Vehicles

Ba-Hai Nguyen , Jee-Hwan Ryu

[1] [1] [2]

Hochiminh University of Technical Education, Vietnam Korea University of Technology and Education, Korea

[2]

ABSTRACT In this paper, a novel haptic interface is proposed for general teleoperation of wheeled and tracked vehicle. The new mechanism of the proposed haptic interface shown in Fig. 1 not only allows human operator to easily tele-operate various types of target vehicles including car-like vehicles, mobile robots, tracked vehicles, but also improve the operators perception of target vehicles operating status and its environment by introducing cornering feel in the field of vehicle teleoperation. In addition, the proposed interface enables human operators to give commands to (or drive) target vehicle in a way which is carried out in traditional direct control or driving. Experiments have been done to carefully evaluate and test the performance of the proposed interface. The results have shown that the developed haptic master device is sufficient and suitable for general wheeled and trackedvehicle teleoperation. KEYWORDS: human-vehicle interface, teleoperation, wheeled, tracked, vehicle, mobile robot, human-robot interface, humanrobot interaction INDEX TERMS: K.6.1 [Designing haptic interface, Teleoperation and Control, Vehicular Robotics] 1 INTRODUCTION of its steering wheel-like interface, but also conveniently set the desired speed to the target vehicle using joystick-like accelerating pedals. In addition, operators could percept richer and more comprehensive information of teleoperation such as the feel of the amount of steering angle, vehicle speed, and the feel of cornering effect (a drivers feel when vehicle travels at road corners). As the result, operator can interact and percept in the same manner in direct control using the proposed interface. In other words, with proposed interface, teleoperation or remote control may closer to the normal systems in which the human operators seat in or on the target vehicles and drive them. Accordingly, these feels are the key to help operator in maintaining control of the vehicle following a desired strategy. Beside mentioned advantages, the proposed interface enables number benefits. In particular, several matching possibilities for assigning joint-variables of proposed interface to different target vehicles as well as new schemes of force-feedback reflection. Those advantages have distinguished this research with others in the field of intuitive interfaces for the teleoperation of wheeled and tracked vehicles. After the first phase of this research presented in [1], this second phased of this research focuses on: Deeply study matching strategies to map the proposed interface to various types of target vehicle. Demonstrate the matching capability by conducting several experiments with various types of vehicles.

In teleoperation and control of wheeled and tracked vehicles, a human operates and controls a target vehicle from a distance. In such operations, the human operators can not directly navigate the target vehicle and percept the vehicles status as well as its environment. Therefore, a human-vehicle interface has become a crucial mean which provides robots status and environments information to human operator for accurate and intuitive teleoperation and control. From this point of view, it is worthwhile to examine and study a novel interface for teleoperation of wheeled and tracked vehicles (WTV). In the paper [1], the philosophy of the proposed haptic interface has been presented and the preliminary results of this research have been illustrated that the proposed interface can be mapped to carlike vehicle and mobile robots. The proposed a novel interface for the teleoperation of wheeled and tracked vehicles which is intuitive, effective. And, it can be a low-cost and portable humanvehicle interface. With the proposed interface, operators not only intuitively teleoperate a target vehicle because
[1] [2]

Handles Steering wheel

Base Adjust er

nguyenbahai@hocdelam.org jhryu@kut.ac.kr

Figure 1. The proposed interface

IEEE World Haptics Conference 2011 21-24 June, Istanbul, Turkey 978-1-4577-0298-3/11/$26.00 2011 IEEE

107

2 2.1

VEHICLE TELEOPERATION

Typical teleportation scenarios The teleoperation of wheeled and tracked vehicles is shown in Fig. 2. By giving commands in term of speed, position, or force via haptic interface (master), a user at local side navigates a target vehicle (slave) at remote side to a desired location. And at the same time, he or she percepts the status or information of target vehicle and its environment in term of visual, hapic, audio, and vision feedback. Based on the feedback signals, the performance of teleoperation activities can be improved. Usually, there is a wireless network for exchanging data between the master and slave.

& k ml 0 0 ml r 0 k & 0 = mr mr l 0 0 0 mc 0
where:

(1)

k ml , k mr
and right interface.

are proportional gains. angles of

ml , mr , mc are
axis ,
and

left axis ,

center axis

of the proposed

&r , &l

are angular velocities of right wheel and left

wheel of the mobile robot.

Human Operator Master (Haptic Interface) Communication Channel Slave (Target vehicle) Environment (Road, wind etc.)

Figure 2. Teleportation system of wheeled and tracked vehicles 2.2 Matching issues of vehicle teleoperation One of the most technical difficulties in the teleoperation of wheeled and tracked vehicles is how to develop a general intuitive interface in which users can perform the teleoperation task easily, and the interface can be mapped to various types of target vehicles. It means that users operate the teleoperation system and percept similar feel in direct control. For example, users should see the vision in front of the target vehicle, and feel steering force which applied to his or her hand like a conventional steering wheel. Accordingly, it is benefit to propose a general interface for the teleoperation of wheeled and tracked vehicles. And with a single proposed interface, there must be a certain matching strategy for different types of vehicles.

The proposed interface is matched to the robot with positionvelocity strategy. It should be noticed that the center axis is braked (using a HALL mode of actuator attached to center axis ) in this strategy. where:

k pjx , k pjy
PhanTOM

are proportional gains and axis 2 of the

mp1 , mp 2 , are angles of axis1

X mr : is the translational speed of the mobile robot

mr : is the total rotation of the mobile robot.

When the proposed interface is used. the proposed interface has three degree of freedoms shown in Fig 1.
left axis

right axis

2.3 General matching scheme of vehicle teleoperation In teleoperation of WTV, several researches have proposed different matching strategies which are used to map the master to the slave. They include position-position matching strategy, position-velocity matching strategy, and combined matching strategy. For instance, to map a PhanTOM to a mobile robot, the positions of one or many joints of PhanTOM are used as reference position of mobile robot (position-position matching strategy). Alternatively, the position of joints of the PhanTOM can be also the reference of mobile robots longitudinal velocity and angular velocity (position-velocity matching strategy), or those matching strategy can be combined (combined matching strategy). Lets consider an example of a typical and conventional matching strategy in which the positions of master are used as reference velocities of left and right wheel of a mobile robot. For proposed interface shown in Fig. 3, the relationship between human commands ( ml , and mr movement) and the variables of mobile robot position-velocity matching strategy can be expressed via equation 1.

ml

mc

mr

ml
lm

center axis
Figure 3. The mechanism of proposed interface; (a): overall view, (b) side view 3 VEHICLE TELEOPERATION

In general, the proposed interface can be matched to various types of vehicles including differential-drive based vehicles, and car-like vehicles. In addition, with the proposed interface a novel possibility to match the proposed interface to differential-drive based vehicle. This section proposes a matching strategy which makes driving a mobile robot the same as driving a vehicle. Usually, the rotational velocity of mobile robot is controlled based on a reference position of a masters joint. Conversely, in our proposed matching strategy, the angular position of mobile robot is controlled based on the position of the center axis of the proposed interface. This make driving a differential-drive based

108

vehicle the same as driving a vehicle. The proposed matching strategies are possible with only the steeringwheel based interfaces (proposed interface and driving simulator) since there is no rotational axis in other types of masters (for instance, the PhanTOM, Joysticks, and so on). There are four main strategies to match the proposed interface to target vehicles called joystick-based steering with positionposition mode, and joystick-based steering with position-velocity mode, and steeringwheel-based steering with position-position mode, and steeringwheel-based steering with position-velocity mode. And operators may chose one of these when he or she operate a vehicle from a distance. For joystick-based steering with position-position mode, Fig. Fig. 4 shows the possibilities of matching the position of left axis and right axis of the proposed interface to the angular position

& k ml 0 0 ml r 0 k & 0 = mr mr l 0 0 0 mc 0
where:

(3)

&r , &l

are angular velocities of right wheel and

left wheel of the mobile robot.

and

of mobile robot. In this case, the

center axis is not used. Therefore, this matching strategy is called as a joystick-based steering with position-position mode. This matching strategy can be expressed by:

k ml 0 0 ml r 0 k 0 mr mr = l 0 0 0 0 mc
where:

Figure 5. Joystick-based steering with position-position mode matching strategy

(2)

k ml , k mr
and right interface.

are proportional gains. angles of

ml , mr , mc are
axis ,
and

left axis ,

For steeringwheel-based steering with position-position mode, the strategy shown in Fig. 6 can be used to map the proposed interface to both mobile robot (with differential-drive based steering) and car-like vehicle (single-axis based steering). In this matching strategy, the average angular position of left axis and right axis is mapped to the translational velocity (or position) of mobile robot, the angular position mapped to the angular position of mobile robot. This matching strategy can be expressed by:

center axis

center axis is

of the proposed

r , l

are angular position of right wheel and left

wheel of the mobile robot.

mr + ml 2 0
where:

mc

K mlr = X r K mc mr

(04)

K mlr K mc
,

are proportional gains

X mr
Figure 4. Joystick-based steering with position-position mode matching strategy

: is the translational speed of the mobile robot

mr : is the total rotation of the mobile robot ml , mr , mc are

angles of

For joystick-based steering with position-velocity mode, Fig.ure 5 shows the possibilities of matching the position of left axis and right axis of the proposed interface to the angular velocity

left axis ,

and right interface.

axis ,

and

center axis

of the proposed

&r

and

&l

of mobile robot. In this case, the

center axis is not used but the positions of master treated as mobile reference angular velocities in mobile robot side. Therefore, this matching strategy is called as a joystick-based steering with position-velocity mode. This matching strategy can be expressed by:

are angular velocities of right wheel and left wheel of the mobile robot.

&r , &l

109

Figure 6. New possible matching strategy to make driving mobile robot the same as driving a vehicle: position-position matching for steering control

For steeringwheel-based steering with position-velocity mode, Fig. 7 shows two possibilities of matching the position of center axis of the proposed interface to the angular position

Figure. 8 Map for experiment To compare different types of interfaces using the same matching strategy, the same target vehicle and test condition in the simulation and experiment, we used a PhanTOM Omni to compare with the proposed interface. The force-feedback for two joints of PhanTOM was based on steering angular velocity of target vehicle and target vehicles speed. And, the force-feedback for proposed system was from steering angular velocity of target vehicle and target vehicles speed. The target vehicle in this experiment was a real mobile robot. The result shown Fig. 9 has indicated that the proposed interface could significantly reduce exploration time. This can be easily explained that the proposed interface is closer to traditional steeringwheel.

mr

and angular velocity

mrl

of mobile robot. This matching

strategy can be expressed by:

mr + ml 2 0
where:

mc

K mlr = X r K mc mr

(03)

mr :
robot

is the total angular velocity of the mobile

Figure 7. New possible matching strategy to make driving mobile robot the same as driving a vehicle: position-velocity matching for steering control. This research also finds out that there is improved performance in term of exploration time in the position-position matching for steering control. Section next sex provides detailed experimental results of these matching strategies. 4 EXPERIMENT AND RESULTS Figure. 9: Compare two interfaces: PhanTOM Omni and proposed interface Fig 10 shows the comparison of the proposed interface with a joystick-based interface. Two kinds of joystick were used - the Logitech joystick and the joystick equipped on a gamepad. In this experiment, a virtual mobile robot was used as a target vehicle. The results have shown that the exploration time was significant reduced in the case which the proposed interface was used. Logitech joystick showed better performance than the joystick on a gamepad.

In this research, ten subjects in graduate school of mechanical engineering are asked to participate in the experiment. All subjects have basic engineering background. . Subject ages are from 20 to 28. And they are all in good condition of health, and comfortable to involve the experiments. Each subject has time to be familiar with the task and interfaces before any actual experiments. To complete the experiment, 10 subjects were asked to teleoperate the target vehicle to follow the desired road shown in Fig. 8.

110

Comparision of a joystick-based interface and proposed interface

30 25 Time (s) 20 15 10 5 0 1 2 Subject 3 4 Joystick HuTion

(a)

Fig. 12 presents the comparison of two interfaces: Conventional steeringwheel and proposed interface with cornering feel. This experiment was conducted with a virtual mobile robot. The results have shown that the exploration time was reduced in the case of proposed interface with all subjects. The reduced time in proposed interface was because of the advantage of the cornering feel which was a kind of warning signal sent to the human operator when the robot traveled through a road corner. In addition, the cornering force at single joysticks assisted the driver hands to smoothly rotates the center axis thereby the steering motion was convenient and comfortable for him or her.
Comparision of a conventional steering interface (CSI) and proposed interface with cornering feel (PI-CF)
18 16 Exploration time (s) 14 12 10 8 6 4 2 0 1 2 3 4 Subject 5 6 7 CSI PI-CF

Figure 10. Compare two interfaces; (a): Logitech Joystick and proposed interface; (b): A joystick on a gamepad. Figure 11 illustrates the comparison of the conventional steeringwheel-based interface with the proposed interface without cornering feel. In this experiment, a virtual mobile robot was used as a target vehicle. The conventional steeringwheel-based interface was a Logitech steering simulator with force-feedback. The force feedback in the steering simulator was based on the steering angle. The result has shown that the exploration time was reduced in the case of conventional interface.

Figure 12. Compare two interfaces: Conventional steeringwheel and proposed interface with cornering effect Figure 13 is the number of collisions during teleoperation of a real mobile robot in two cases: PhanTOM Omni and the proposed interface. The result shows that the number of collisions is reduce when the proposed interface is used as the master device.

Figure 11. Compare two interfaces: Conventional steeringwheel and proposed interface without cornering feel This is natural since the conventional interface is intuitive and customary to human operator explained in [1]. However, there is a tradeoff between the proposed interface and the conventional interface. Once the conventional interface was chosen as master device, then there is no way to reproduce cornering effect with a single steering axis. In addition, the cost, power consumption, space occupation of conventional interface may be an issue in some situations. To investigate the advantages of cornering feel in the proposed interface, we have tried experiments in which the cornering feel is displayed in the proposed interface using two single joysticks ( left axis and right axis ) and the steering angle-based force-feedback in conventional steeringwheel-based interface was maintained.

Figure 13. With proposed interface, there are fewer collisions 5 CONCLUSION AND FUTURE WORK

We have demonstrated various matching rules to match the proposed interface to various configurations of target vehicles. These possibilities help system developers freely chose a right matching strategy accordance with operating conditions of the target vehicle which is controlled by the operator. The proposed mechanism of mater device was designed to improve teleoperation performance, and efficiency. The possibility of proposed architecture has been tested in both computer based semi-experiments and real-vehicle based experiments. The results have shown that the developed interface is sufficient and suitable for wheeled and tracked vehicle teleoperation and control. And the proposed interface can be mapped to various target vehicles using different matching rules

111

which are benefit and easy to be applied to different types of slave configurations. Experimental results also have shown that it is possible to display obstacle information to human operator using the proposed interface. In future work, numerous topics should be extended and continued. Study possibility of display other driving feels beside cornering feel. Study mechanical design to reduce the weight of the proposed interface. Develop an embedded controller for the proposed interface. Make plan to manufacture mass production, and exhibition, and commercialization. REFERENCES
[1]

[2]

[3]

[4]

[5]

[6]

[7]

[8]

Ba-Hai Nguyen, Jee-Hwan Ryu, Design of a Master Device for the Teleoperation of Wheeled and Tracked Vehicles. ISBN: 978-1-4244-7453-0, ICCAS2010, Seoul, Korea. Otto J. Rsch, Klaus Schilling, and Hubert Roth. Haptic interfaces for the remote control of mobile robots. Control Engineering Practice, Vol. 10 pp. 13091313, 2002. Terrence Fong, Charles, Thorpe, "Vehicle Teleoperation Interfaces" ISSN:0929-5593, Volume 11 , Issue 1, Pages: 9 18, 2001. T. Fong, C. Thorpe, C. Baur, A. C. Schultz, and L. E. Parker Robot as Partner: Vehicle Teleoperation with Collaborative Control. In Multi-Robot Systems From Swarms to Intelligent Automata. Kluwer Academic Publishers, 2002. Hara, M. Matthey, G. Yamamoto, A. Chapuis, D. Gassert, R. Bleuler, H. Higuchi, T., Development of a 2-DOF electrostatic haptic joystick for MRI/fMRI applications. Robotics and Automation, ICRA '09. IEEE, 2009. Mooncheol Won; Grandpierre, T.; Fleutot, G.; Parent, N. A joystick driving algorithm with a collision stop feature on an electric vehicle (Cycab). Intelligent Vehicle Symposium, IEEE, 2002. J. Yuh, Design and Control of Autonomous Underwater Robots: A Survey. Autonomous Robots, Volume 8, Issue 1, Kluwer Academic Publishers , ISSN:0929-5593, 2000. Sangyoon Lee, Gaurav Sukhatme, Gerard Jounghyun, Kim, Chan-Mo Park, Haptic teleoperation of a mobile robot: a user study. Presence: Teleoperators and Virtual Environments, Pages: 345 - 365, Volume 14 , Issue 3, MIT Press , 2005.

112