Novel Method For

Predicting Dexterous
Individual Finger
Movements By Imaging
Muscle Activity Using A
Wearable Ultrasonic
System

Abstract
2

Recently there have been major advances in the electro-mechanical design

of upper extremity prosthetics. However, the development of control
strategies for such prosthetics has lagged signicantly behind.
Conventional noninvasive myoelectric control strategies rely on the
amplitude of electromyography (EMG) signals from exor and extensor
muscles in the forearm. Surface EMG has limited specicity for deep
contiguous muscles because of cross talk and cannot reliably differentiate
between individual digit and joint motions. We present a novel ultrasound
imaging based control strategy for upper arm prosthetics that can
overcome many of the limitations of myoelectric control. Real time
ultrasound images of the forearm muscles were obtained using a wearable
mechanically scanned singleelement ultrasound system, and analyzed to
create maps of muscle activity based on changes in the ultrasound
echogenicity of the muscle during contraction.
15/08/16

Current system
3

Control strategies for upper extremity prosthetics have lagged

behind their corresponding hardware.

Current non-invasive control technique involves measurement

of amplitude of EMG signals from muscles in the forearm.

Surface EMG has limited specificity for deep contiguous

muscles due to cross-talk

Differentiation between individual digit and joint movements

is challenging.

Strategies to overcome the above limitations are either

invasive, or impractical.
15/08/16

Proposed system
4

New sensing strategy based on ultrasound.

Can provide robust signals from individual functional muscle
compartments
Could lead to the design of more sophisticated upper
extremity prostheses and control systems.
Ultrasound is inexpensive, safe, and provides real time
dynamic imaging capability.
Can avoid many of the limitations of EMG because it can
spatially resolve individual muscles deep inside tissue and
visualize dynamic activity of different functional
compartments.
15/08/16

Inexpensive portable handheld ultrasound systems are

commercially available.

It is now feasible to incorporate real-time imaging into a

wearable prosthetic.

An inexpensive ultrasound imaging probe with a USB

interface is used to track the contraction of forearm muscles
through image processing.

This paper evaluates the generation of proportional control

signals for extension and flexion of separate digits of the hand
at different speeds over different ranges of motion.
15/08/16

Procedure
6

Ten healthy volunteers (five men, five women) were used for
this study.

A 7.5-MHz single-element motor controlled ultrasonic probe

that connects to a PC via USB, was used.

A cuff was designed to hold the probe fixed relative to the

forearm with the ability to control the angle of the probe to
optimize visualization of the muscles of interest.

The probe was placed on the midpoint between the wrist and
the elbow on the anterior of the forearm directly over the
radius, angled at 60 above horizontal.
15/08/16

15/08/16

Image processing
8

The envelope-detected brightness mode (B-mode) ultrasound

image sequences were analyzed offline using MATLAB.

The processing algorithm consisted of two steps:

a training step to determine and store a pattern of muscle
activity during each individual known digit movement
a detection step where the unknown digit movements were
decoded from a test sequence where the operator was
moving different digits.

15/08/16

15/08/16

10

Training Individual Digit

Movements

In this step, the user was asked to move individual digits

(thumb, index, middle, ring, and little fingers) multiple times
on cue to the rhythm of a metronome.

For each sequence, a baseline image frame was subtracted

pixel-wise from each subsequent image frame.

The filtered difference images showed the spatial distribution

of pixel intensity changes corresponding to the forearm
muscle activity corresponding to each digit.

An activity pattern image was created from the maximum and

minimum pixel differences over multiple repetitions for each
digit movement.
15/08/16

Decoding Individual Digit

Movements and Activity
11

The image-processing algorithm calculated a filtered pixel

difference image depicting the ongoing muscle activity.

An activity pattern image was created from the maximum and

minimum pixel differences over a moving time window.

This activity pattern image was correlated with each of the

previously stored known activity pattern images for each
individual digit movement to decode the best match for the
unknown activity.

15/08/16

12

Once the digit movement was decoded, the active pixel

coordinates for that particular digit movements were used to
compute the actual pixel differences.

A time series of these pixel differences was then created to

represent the activity signal for the corresponding digit
movement.

15/08/16

Advantages of new method over

existing method
13

The ability to image the active contractions in functional

muscle compartments is a significant advantage of our method
over EMG.

High signal-to-noise ratio, since the dynamic range of

observable differences on image sequences is significantly
higher than noise.

Continuously varying, proportional, activation signal can be

derived from ultrasound image sequences, rather than mere
classifications of end states.
15/08/16

14

Classification can be accomplished with low latency leading

to lower controller delay.

The ability to track bony interfaces independently of muscle

contractions allows decoupling wrist rotation from other
grasping functions.

15/08/16

Results
15

Based on a sequence of ultrasound images, patterns were

generated that correspond to individual digit movements.
These patterns were then stored as training data.

During a test sequence, the pattern of movement was

correlated with the training data to find the closest match

Some cross talk is present, but the corresponding digit

produces a larger correlation coefficient, which enables the
correct identification of the digit.

15/08/16

16

15/08/16

17

Table I shows a confusion matrix for classifying the five digit

movements using the k-Nearest Neighbor algorithm.

The parameter value of k for the k-NN algorithm was set to 3.

Based on the leave-one-out cross-validation the accuracy was

98.33%.

The procedure developed was found to be suitable for

accurately identifying a given individuals digit movement.

The precision and recall per digit-movement class is reported

in Table II.

15/08/16

18

15/08/16

19

A key advantage of our ultrasound-based method is the ability

to produce proportional signals for different degrees of digit
movement.
Fig. 4 shows a representative example of proportional signals
generated from one volunteer. It shows the change in
ultrasound echogenicity produced when the operator was
asked to flex their thumb at four different levels of flexion.
It can be observed that a different level of activity is observed
on the ultrasound-derived signal. To further quantify this, the
speed of digit movement was measured using a string and
pulley assembly, attached to a linear position encoder.
15/08/16

20

15/08/16

Practical Considerations
21

Safety of continuous ultrasound exposure.

Placement of the ultrasound probe.

Controller delay.

15/08/16

Study limitations
22

This study was performed on able-bodied volunteers, rather

than amputees.

In the preliminary feasibility evaluation, only single digit

movements were classified.

Our ultrasound imaging system is currently limited to a cross

section, thus muscle activation at distant sites cannot be
readily visualized.

15/08/16