Académique Documents
Professionnel Documents
Culture Documents
a r t i c l e
i n f o
Article history:
Received 25 March 2008
Received in revised form
29 September 2008
Accepted 1 October 2008
Keywords:
MATLAB
Timing
Toolbox
Synchronisation
Tapping
Metronome
a b s t r a c t
Investigating movement timing and synchronisation at the sub-second range relies on an experimental setup that has high temporal delity, is able to deliver output cues and can capture corresponding
responses. Modern, multi-tasking operating systems make this increasingly challenging when using standard PC hardware and programming languages. This paper describes a new free suite of tools (available
from http://www.snipurl.com/mattap) for use within the MATLAB programming environment, compatible with Microsoft Windows and a range of data acquisition hardware. The toolbox allows exible
generation of timing cues with high temporal accuracy, the capture and automatic storage of corresponding participant responses and an integrated analysis module for the rapid processing of results. A simple
graphical user interface is used to navigate the toolbox and so can be operated easily by users not familiar
with programming languages. However, it is also fully extensible and customisable, allowing adaptation for individual experiments and facilitating the addition of new modules in future releases. Here we
discuss the relevance of the MatTAP (MATLAB Timing Analysis Package) toolbox to current timing experiments and compare its use to alternative methods. We validate the accuracy of the analysis module
through comparison to manual observation methods and replicate a previous sensorimotor synchronisation experiment to demonstrate the versatility of the toolbox features demanded by such movement
synchronisation paradigms.
2008 Elsevier B.V. All rights reserved.
1. Introduction
Producing rhythmic movements and actions is an everyday task
that we can execute using our own internal pacing signals (e.g.
walking or cycling), or in time with some external event (e.g. dancing with a partner (Gentry and Feron, 2004; Batalha and Macara,
2007) or playing in a musical ensemble (Keller et al., 2007)). Empirical investigations of the timing of rhythmic actions are often
conducted by instructing participants to tap their index nger in
time to an auditory, visual or tactile stimulus produced at regular
intervals (a metronome), typically at rates above 1 Hz. Following
this, the metronome stimulus is often turned off while the participant continues to tap at the same pace for a further period of
time. This procedure, which was rst employed by Stevens (1886),
is referred to as the synchronisation-continuation paradigm. Performance is typically described in terms of the disparity between
the metronome pulse and nger tap response (asynchrony) in
the synchronisation phase (Fig. 1). It is often noted that the tap
precedes the metronome in an anticipatory manner (Vos et al.,
Corresponding author. Tel.: +44 121 414 9517; fax: +44 121 414 4897.
E-mail address: m.t.elliott@bham.ac.uk (M.T. Elliott).
0165-0270/$ see front matter 2008 Elsevier B.V. All rights reserved.
doi:10.1016/j.jneumeth.2008.10.002
1995; Aschersleben, 2002), with the resulting timing error generally in the order of tens of milliseconds (standard notation is
that a negative value indicates an anticipatory tap). Similarly, in
the continuation phase, when the metronome is switched off,
self-paced tapping can be analysed through the measurement of
inter-response intervals (IRI)the interval between the current and
previous tap onset (Wing and Kristofferson, 1973; Billon et al., 1996;
Collyer and Church, 1998; Inui et al., 2001).
Detailed analysis of nger tapping timing accuracy in terms of
mean and variability has given rise to a large literature that has
shaped understanding of movement synchronisation (see Repp,
2005 for a comprehensive review). However, given the small temporal magnitude of these measurements (typically in the tens
of milliseconds range), a considerable empirical challenge is to
produce stimulus events with reliable timing while concurrently
capturing the participants movements with high temporal delity
using sensors appropriate to the movement (e.g. force-transducer,
video-based motion tracking). Meeting this challenge requires an
experimental setup that is able to (i) generate cues with a low
level of temporal lag and uncertainty, (ii) capture responses with
a high sampling accuracy and (iii) allow a true measurement of
the participants timing error by aligning precisely the participants movement with the cue event. Moreover, researchers often
251
Fig. 1. MatTAP captures both the metronome event and participants response to the event. Using this data, asynchronies can be calculated by measuring the temporal
difference between the metronome onsets and the corresponding response onsets. As indicated, responses typically precede the metronome resulting in negative asynchrony
values. Similarly, the inter-response intervals of the responses can be calculated from the time between the current and preceding response.
require more complex setups to the standard synchronisationcontinuation paradigm in which the properties of the metronome
and/or the properties of the produced movement might be varied.
For example, the duration of individual tones has been varied (Vos
et al., 1995), while investigating syncopated synchronisation has
involved using two independently timed auditory tones at different pitches (Keller and Repp, 2005). Further, a number of studies
have involved controlling the metronome rate during a trial by
changing the period in a stepped manner (Thaut et al., 1998), continuously (Schulze et al., 2005) or as a single phase shift (Repp,
2000, 2001, 2002; Elliott et al., 2008). Other studies have investigated multi-effector responses such as bimanual synchronisation
of right and left hands (Drewing and Aschersleben, 2003; Chen et
al., 2005; Doumas and Wing, 2007) or hand and foot (Helmuth and
Ivry, 1996).
Previous implementations for the accurate control of synchronisation experiments have exploited MIDI (Musical Instrument
Digital Interface) systems (e.g. Repp, 2000; Finney, 2001; Schulze
et al., 2005). This system has been used as the MIDI interface uses
dedicated hardware to ensure a constant output data rate that
has high temporal accuracy (latencies of 3 ms and jitter of around
1 ms are typical, Heckroth, 2001) with similar accuracy reported for
input events. In the context of movement synchronisation experiments this enables investigators to produce regular/rhythmic audio
tones whilst simultaneously detecting key strike events instigated
by the participants. The disadvantage of the MIDI standard is that
it is reliant on the use of digital musical instruments to capture
responses (such as a keyboard) imposing constraints on the experimental setup and making the implementation of multi-effector
recordings challenging. Further, there are few research-specic
software tools that exploit the MIDI protocol, hence timing analysis must be performed with separate software using data exported
from the sampling equipment. To date, MIDI-based methods have
remained attractive through their use of hardware whose timing
is independent of the controlling computer workstation. In particular, using an exclusively software-based solution under modern,
multi-tasking operating systems is difcult because the timing process relies on the central computer clock (the CPU) that is shared
with a range of different processes. This often results in temporal
latency and jitter in both the production of a metronome and the
capture of responses that cannot be controlled reliably.
Here we describe a dedicated solution for the control and analysis of synchronisation-continuation experiments. Written in the
MATLAB (The MathWorks, Inc., MA, USA) programming environment, the MatTAP (MATLAB Timing Analysis Package) toolbox
allows sub-millisecond accuracy for the production of metronome
events and the capture of participant responses (from a range of
sensors) through the use of dedicated data acquisition hardware.
The software incorporates a graphical user interface (GUI) to provide users with high-level control in the generation of multiple
timed output cues with a range of different parameters. This allows
exible experimental design without the necessity of writing new
program scripts and hence reduces the likelihood of introducing
errors into a particular paradigm. On the other hand, the toolbox
is fully extensible, allowing new modules to be added by those
familiar with the MATLAB programming language.
1.1. Overview of the toolbox
MatTAP is a MATLAB toolbox written with the goal of simplifying the process of running synchronisation experiments with highly
accurate metronome signals and capturing the responses made by
participants. It uses the MATLAB Data Acquisition Toolbox (The
MathWorks, Inc., MA, USA) to interface dedicated data acquisition
hardware. This removes the signal generation and data capture
tasks away from the PC, avoiding the problems of multi-tasking
operating systems and allowing high temporal precision. A typical
hardware conguration for an experimental setup using MatTAP is
shown in Fig. 2.
MatTAP contains three main modules of operation. First, a
graphical user interface is provided to allow full experimental
control of the metronome events and the capture of participant
movements. Based on the diversity of previous experiments in the
area (see above), the interface allows the user to generate up to two
Fig. 2. A typical hardware conguration for cue generation and response capture
using MatTAP. The toolbox uses data acquisition hardware to achieve a high level of
timing accuracy. (1) Two outputs are generated to drive stimuli. One is an analogue
waveform, typically used to drive an audio speaker, but can also be applied to haptic
or visual devices. The second output is a digital square pulse which can be used to
trigger bespoke stimuli equipment. To increase accuracy further, this pulse is fed
back into the system to compensate for processing delays in the hardware. (2) Up
to two sensors can be utilised to capture responses. Any sensor which produces
an analogue or digital voltage, typically in the range 5 V can be used to record
events. (3) The output signals and corresponding responses are captured and stored
automatically in individual, sequentially numbered .mat format les.
252
Fig. 3. A diagram showing the steps involved from setting up an experiment in MatTAP to running the analysis on the acquired data using the three provided modules. A
User Action is where the user interacts with toolbox; Background Activity is where the toolbox automatically executes a task after a user action.
253
Fig. 4. The user interacts with MatTAP using two GUI screens, File and Metronome. The Metronome page allows the user to fully dene one or two metronomes, either
manually (A) or via a trial control script (B). The File page allows the user to dene a new experiment, lists trials within an experiment (C) and provides a basic plot of the
raw data from a chosen trial (D). The analysis module is also called from the File page (E).
3. Parameter scripts
With the exception of phase shift insertion, MatTAP is able to
import comma separated variable les to automatically set the
parameters described in Table 1 on a trial-by-trial basis. This allows
Table 1
MatTAP allows exible metronome generation to allow a wide range of experimental protocols to be implemented. This table lists the parameters that can be set using the
MatTAP GUI.
Parameter
Description
Sample frequency
Tone frequency
Controls the input and output sampling frequency of the data acquisition card.
The frequency of the tone emitted when using the analogue metronome output. This is typically used to set the pitch of an audio output or vibration
frequency of a tactile stimulus.
The inter-stimulus interval (ISI), i.e. time between the onsets of the metronome tones. For a regular metronome this value is a constant. However,
MatTAP allows variability in the timing of individual metronome pulses. In the latter case this value corresponds to the mean of the distribution.
In the case of a variable metronome, this parameter controls the standard deviation of the normal distribution from which ISIs are randomly selected.
If the metronome is regular (xed) this parameter is ignored.
The duration in milliseconds of each metronome pulse.
The number of metronome pulses to be output in the trial (the synchronisation phase).
The duration (in seconds) of the continuation phase (metronome suppressed). Synchronisation pulses are still emitted during this period, however, no
analogue output is produced. If the continuation phase is set to greater than 0, then a high pitched tone (2 tone frequency) is emitted at the end of
this period to mark the end of the trial.
The temporal offset (in milliseconds) between the rst (M1) and second (M2) metronome if a second metronome is used. 0 would mean the
metronomes have the same phase. A negative offset indicates that the mean ISI of M2 leads M1, whereas a positive offset would produce a mean ISI of
M1 that leads M2.
Insert a phase shift before one of the metronome pulses. This lets the experimenter increase or decrease a single ISI by a xed amount. All
corresponding ISIs return to their standard period such that the metronome is phase shifted.
Mean ISI
S.D. ISI
Tone duration
No. of stimuli
Silence
Offset to M1
254
Fig. 5. MatTAP can calculate the asynchronies between the onset of a participants response and the metronome onset. The owchart above shows how the algorithm aligns
the response onsets such that there is always one response corresponding to a metronome onset.
trials to be randomised or presented in a chosen sequence. Parameter scripts can be produced in simple table format using software
such as Microsoft Excel, with each row of the table representing a
trial and the column relating to the described parameters. Fields can
be manually overwritten using the MatTAP GUI if required, and trials can be freely navigated backwards and forwards once the script
has been loaded.
3.1. Analysis
MatTAP includes an analysis module (Fig. 4; E) to process the
raw data from each trial in order to determine the onsets of (i) the
metronome pulse and (ii) the participants response. Having identied movement onsets, the software calculates the inter-response
intervals (IRI) between successive movements of the participant
and the asynchronies (timing errors) between the participants
responses and the metronome. Below we describe the algorithms
used in the analysis module.
4. Onset detection algorithm
The appropriate algorithm for onset detection can be very signal
specic, depending on the nature of the response and the type of
sensor used to capture it. For this reason MatTAP allows bespoke
algorithms to be developed for onset detection without requiring
changes to other parts of the MatTAP code. In particular, a selfcontained function is used to extract the onsets from the captured
data. This function takes the raw data as an input parameter and
returns a vector of onsets to the main MatTAP program. A default
algorithm is included in the package that provides a means of capturing nger taps from a surface-based sensor. However, MatTAP
has successfully been used with more sophisticated algorithms
designed to extract onsets from motion-captured arm movements
and forces measured by a force-transducer. The default algorithm
classies an onset by rst identifying a point where the current
data value exceeds a specied threshold voltage (default: 0.2 V),
with the previous value having been noted as below this threshold. To avoid false detections, baseline offsets are subtracted and
hysteresis techniques are used to stop effects of noise around the
255
The accuracy of the raw data capture and generation is determined by the sampling rate used and ultimately the DAQ hardware
used. For example, a sample rate of 10,000 samples per second
would provide a temporal accuracy of up to 0.1 ms. Most modern
DAQ hardware devices are capable of maximum sampling rates of
hundreds of kilo-samples per second and hence it can be seen that
potentially MatTAP is many more times accurate than any other
previous synchronisation research software available. Equally, the
use of analogue data acquisition to capture responses enables accurate amplitude measurements as well as temporal ones. Again,
this is dependent on the DAQ hardware used, a 16-bit DAQ with
a voltage range 010 V would provide an amplitude resolution
of around 153 V, while a 12-bit DAQ would have a resolution
of 2.4 mV. Given the appropriate sensor, recording the amplitude
could, for example, be used to measure the level of force applied in
a tap.
To validate the MatTAP system as a whole, a test was set up
to compare the values obtained using the algorithms described
above in comparison to a method that was independent of the
hardware and software used by MatTAP. Using an oscilloscope (Agilent 54621D, Agilent Technologies Inc., CA, USA) simultaneously
with MatTAP and National Instruments USB-6229 data acquisition hardware to capture real experimental data, the metronome
synchronisation pulse and the participants corresponding tap
response were examined by splitting the signal to both the DAQ
(MatTAP) and oscilloscope inputs.
Data from a typical synchronisationcontinuation experiment
were used in which participants were instructed to tap their index
nger in synchrony with an auditory metronome producing beats
at regular 500 ms intervals. Single trials were chosen randomly
from three participants datasets. Using ve consecutive tap-pulse
events from the three trials, the asynchronies were calculated
by eye on the oscilloscope using the built in cursor function to
calculate the difference between tap onset time and metronome
onset time (3 volunteers; time-base: 20 ms/div; cursor resolution: 0.1 ms). The same data were processed using the analysis
module in MatTAP after being captured at a sampling frequency
of 10 kS/s.
The variability of the onset detection by the MatTAP module was contrasted to that of three human observers asked to
classify movement onsets using the oscilloscope. To do this, the
MatTAP asynchrony values were subtracted from the mean asynchrony measurements produced by the three human observers.
The mean and standard deviation of the resulting residuals were
recorded. The same method was then used to calculate the differences between each individuals measurements and the mean of
the measurements recorded by the other two observers. The standard deviation of the differences between MatTAP (0.27 ms) and the
manual measurements (Volunteer 1: 0.29 ms; Volunteer 2: 0.26 ms;
Volunteer 3: 0.27 ms) is comparable to that found between volunteers. There is a slight positive bias in the mean difference (MatTAP:
0.21 ms; Volunteers: 0.08, 0.10, 0.18 ms). This arises as MatTAPs
algorithm denes the onset threshold as 0.2 V above zero whereas
the volunteers were likely to select the onset on the oscilloscope
at a point lower than this value. However, this bias is negligible in
relation to the asynchrony measurements, which are typically two
orders of magnitude larger (Aschersleben, 2002). Furthermore, this
bias is systematic, meaning that relative measures of asynchrony
are unaffected.
256
Table 2
Mean and S.D. of measurements between observers averaged over three trials.
Observers used a cursor method to pick event onsets from plots of data captured
using MatTAP and a National Instruments USB-6229 data acquisition card. The
results were compared to the onsets calculated using the MatTAP analysis module
(shown in italics).
Observer
ISI
IRI
Asynchrony
Mean (ms)
S.D. (ms)
Mean (ms)
S.D. (ms)
Mean (ms)
S.D. (ms)
1
2
3
4
0.00
0.00
0.00
0.00
1.33
2.18
1.03
1.19
0.03
0.08
0.05
0.16
5.61
4.48
4.62
4.79
1.66
0.62
1.14
2.17
4.13
3.69
3.58
3.72
MatTAP
0.00
0.77
0.04
3.44
1.74
2.64
of the phase shift magnitude. The results show, from a methodology aspect, that MatTAP can accurately produce phase shifts down
to very small temporal magnitudes and record the response to the
manipulated stimulus with equal precision.
257
authors thank Dagmar Fraser for his technical input and testing
of the toolbox and Leif Johannson for his suggestion for the asynchrony calculation algorithm.
References
9. Discussion
Research into sensorimotor synchronisation is an active area
of investigation that lacks a widely available software tool which
allows researchers exibility in experimental control and analysis.
MatTAP has been provided as a free toolbox for MATLAB, a widely
used and understood programming tool in the research community. The toolbox has been designed to provide a high level of
exibility in producing a metronome output, uses external data
acquisition hardware to ensure timing accuracy is not impinged
upon by PC multi-tasking operating systems and provides a structured le management system. We have taken the protocols used
in previous studies to dene a set of parameters that has resulted
in a metronome generation tool that allows phase shift insertion
(Repp, 2000, 2001, 2002), adjustable tone duration (Vos et al., 1995),
adjustable tone pitch (Keller and Repp, 2005) and the option to
change the metronome reliability by adding jitter to the intervals (Repp and Penel, 2004). Furthermore, we provide the ability
to record up to two response channels (Drewing and Aschersleben,
2003; Chen et al., 2005; Doumas and Wing, 2007), as well the
option to generate up to two simultaneous metronomes, which
allow presentation in either single or dual modalities (Bertelson
and Aschersleben, 2003; Kato and Konishi, 2006). Finally, we have
demonstrated the validity of the MatTAP setup and in particular
shown that the analysis module provided will produce reliable
analysis of the data to the required sub-millisecond accuracy without adding any signicant measurement noise.
MatTAP currently has a few limitations, that mainly reect the
extent to which the MATLAB Data Acquisition Toolbox can communicate with the acquisition hardware. Notably, real-time response
feedback (Mates and Aschersleben, 2000; Finney and Warren,
2002) is not currently implemented in MatTAP thus, it is not possible to adapt metronome pulses within a trial (Repp and Knoblich,
2007). MatTAP is also dependent on the investigators access to
data acquisition hardware as the software cannot be run using a PC
alone. However, the cost of data acquisition hardware is currently
relatively low in relation to other items of lab equipment, and is
comparable to that of MIDI hardware and software investment. Further, the exibility of MatTAP allows a wide range of sensors to be
used, from simple switches to high accuracy force-transducers. The
nature of MATLAB means that in addition to the supplied functionality, users can change anything from a single value to the whole
analysis module to suit their requirements. It is hoped that this will
allow scalability of the toolbox, with the opportunity to interface to
other appropriate MATLAB toolboxes (e.g. The Psychophysics Toolbox, Brainard, 1997) or to develop further integrated modules in
the future.
Acknowledgments
This work was supported by grants from the Biotechnology
and Biological Sciences Research Council [C520620, EO09565]. The