Auto-align Image Co-registration Supplementary Information

1 Algorithm structures
Algorithm 1 rotational offset
This algorithm is designed to determine the difference in
angle between the same feature in 2 different images, with
respect to the vertical, due to the difference in orientation of
the detection devices used for their acquisition.
If the data in the 2 dimensional image,
, is duplicated,
rotated by an angle, , and scaled by a factor, , we get a
second image:
. The Fourier transform of the
original image is related to the Fourier transform of the
scaled-rotated image by Fourier scale and Fourier rotation
theory. These state: A rotation in the spatial domain will
result in the same rotation the frequency domain; A scale in
the spatial domain will in appear as a reciprocal scale in the
frequency domain (1, 2). These properties form basis theory
in the design of this optimisation type algorithm. Phase
correlation between two images,
, and
, is
defined by:
(1)
Where:
corresponds to the Fourier transform of an
image,
, corresponds to the inverse Fourier transform,
and
denotes the complex conjugate of the Fourier
transform of an image. Fast Fourier Transform (FFT)
algorithms are used to calculate the Fourier transform of
discrete image data. As the sole objective of this algorithm is
to calculate rotation, we are only interested in the value of
the maximum element of
but not its position (phase
correlation is commonly used to calculate the translation
between 2 images). The similarity metric for two images
used in this algorithm is defined by:

between the two images. Choosing two points present in

both images, and calculating the difference in angle of the
lines joining those two points with respect to the horizontal
will give
. A reasonable estimate for human error, , is
also assigned.
(3)
(b) Scale and rotation testing
Every loop of the algorithm there is a scale and rotation test
performed in parallel (Figure S1). The structure of each of
the tests is identical apart from the actual transformation
performed during the test (i.e. scale or rotation). One image
is transformed by some amount, the other is unchanged, and
the correlation between the images is computed. This is
performed for a range of different values using the same
image transformation on the same image and leaving the
other image unaltered. The following description of this
subsection will describe the optimisation of the scale
difference, , between the images. Mathematics for the
rotation test is obtained simply by interchanging, , with, ,
and changing the transformation from a scale to a rotation.
The image being tested is a TIRF image,
, that has
been rotated by the current best estimate of the angle,
,
prior to testing,
. Given an estimate for the scale,
, along with the errors associated with its initial
measurement, , then the search range for, , is defined by
the closed interval:
(4)

Test values are chosen at small intervals,

(5)

(2)
Where:

(a) Initialisation
To determine the rotation accurately we require precise
knowledge of the image pixel scales. Calibrated microscopes
provide a pixel scale (the physical length each pixel
represents) with their images so the ratio of these can be
used as a corrective scale factor. However, the calibration
values will each carry an error and therefore this scale factor
can only be used as an approximation. If, , and,
, are
the pixel calibration values for Confocal and TIRF
microscopes images, respectively, then the ratio gives a first
estimate for the difference in scale for the two images. Each
value will carry some measurement error therefore,
,
carries an error, . Visual assessment of the images
provides a rough estimate for the rotational offset,
,

The correlation between the transformed TIRF image,

, and the un-changed confocal image,
,
is calculated for each, , using Equation 2.
(6)
(7)
The correlation for each, , is used to build dataset,
,
which cubic splines are fitted in order to predict a more
precise value for , should the optimal solution lie in the
region between two of the test values. Taking the argument
of the maximum of the interpolated data gives the optimal
for this test:

(8)

Errors used to define the search range (in Equation 4) are

increased intentionally by
from the initial user set values,
(
in the Java Implementation).
The reason for this is to check if the global result tries to
diverge from result which is known to be within the true
search interval. If the result from Equation 8 is falls outside
the initial errors defined by Equation 4 then
is reassigned with a random value within the initial closed
interval. Alternatively, this could mean that the actual solution
lies outside the given search range and that the initial errors
were too small. The Java implementation of this algorithm
(ImageJ plug-in) performs the scale and rotation tests on
parallel threads to increase the speed of each loop of the
algorithm.

, then scaled using a rough scale estimate,

, from

Equation 3 . Performing normalised cross

correlation on this image and its corresponding confocal
image,
, determines an approximate measure for the
relative translation, (
), between the images (this is
only an approximation as
is only an educated estimate).
A reduced search window, the effective area in,
,
covered by,
, translated by, (
),
is extracted:
. A scale test identical to that in
algorithm
1
is
performed
on
down-sampled
images
, and
over the range defined
by Equation 4, then again at their original resolution over a
reduced range to find the scale difference,
. The final
result is generated by computing NCC again on the complete
confocal
image
and
the
scaled
TIRF
image,
.
Algorithm 3 stage drift correction

(c) Correlation, inspection, optimisation

The values from the previous step,
, and,
, are used
to scale and rotate the images,
and
then the correlation between them is calculated.
(9)

Where: the subscript, , denotes the current iteration;

superscript,
, denotes the comparison of the
transformed images. At every iteration,
, is compared
with the value from the previous iteration,

1,

,(

= 0).

If
, then the values,
, and,
, are
considered to be closer to global solution than the previous
estimates
, and,
, and therefore the estimate values
are updated for the next iteration
, and
. If
, then the correlation of transformed
test images is calculated individually with the confocal image.
The transformation that gave the best correlation with the
confocal image, when applied to the appropriate test image,
had its value saved and was used as the next estimate. The
other was discarded and the estimate from the previous
iteration was used. Limits on the search range that each test
(d) Break conditions
The final solution is generated when both
, and,
,
have converged to some stable result for 3 consecutive
iterations. The definition for convergence here is When all
and
fall within

Motion due to stage drift is detected using normalised crosscorrelation performed on adjacent frames in time from a time
series data set. The assumption used here is that the
morphology of the sample does not change dramatically from
frame to frame i.e.
. Images are
translated with respect to the previous frame in the time
series. If the motion/change in morphology between frames
is significant (significant being a net change in position or
shape greater than the motion due to stage drift) the
algorithm may register images according to the net motion of
the objects in the image.
Algorithm 4 Dual wavelength, single CCD image
alignment
The regions of the images containing data corresponding to
signals from different wavelengths are defined manually and
images are separated. An identical scale test to that used in
algorithm 1 is performed on the two images to determine if
there is a scale difference between the images due to a non
linear chromatic response of the microscopes magnification
optics. Normalised cross-correlation is used to register the
images. The final registered images are cropped to the
largest rectangular size where features in both images lie on
the same coordinates and contain pixels that existed in the
original recorded image i.e. there is no border extensions to
match the image sizes.
Image Padding

,
and are the predefined limits of convergence. The value
of the final result is equal to the average of the 3 consecutive
values that met the convergence conditions.

Images were padded with zeros prior to any image Fourier

transforms/convolutions. When convolving two images, one
image is size n1 by m1, the second is size n2 by m2, then
they are both padded with zeros to size N by M; where, N =
n1 + n2 1 and M = m1 + m2 1. Prior to padding the image
edges were profiled with a decaying exponential to prevent
aliasing artefacts due to discontinuities at the edges of the
images.

Algorithm 2 translation and scale

Fast normalised cross correlation

To determine the difference in translation between the

features in the TIRF and confocal images, an algorithm using
a combination of normalised cross-correlation (NCC) and an
iterative scale correlation test was implemented. The TIRF
image is first rotated by the angle determined by Algorithm

Computing the correlation coefficient for two data sets using

the traditional normalised cross-correlation method is
computationally intensive. All algorithms here use a fast
normalised cross-correlation algorithm defined by Lewis
.J.P., which computes a cross correlation in the Fourier

the

ranges

and

where:

domain normalised with a table of pre-computed tables

containing the integral of the image and image squared over
the search window (3). This is far more efficient than
computing the correlation coefficient.
2 - Robustness against noise
(a) Phantom images
The performance of the algorithms was validated using a set
of 10 different phantom images generated using basic
simulations of the image formation processes of TIRF and
Confocal microscopes imaging through the same objective.
A virtual object was created in a 3D space and convolved
with a 3D point spread function generated using a numerical
integration of the equation A10 in reference (4). The features
in the TIRF phantom images were translated by exactly -20
pixels in both x and y directions. The TIRF images were
rotated by 4, and the Confocal phantom was scaled up by a
factor of 1.1; Both image transformations were performed
using bicubic interpolation. Each test image was affected by
different levels of two different types on noise: Intensity
dependent normally distributed noise to approximate Poisson
noise ( equation 10); Gaussian additive noise to simulate
electrical read noise ( Equation 10). The noise levels were
increased from no noise, to that expected in experimental
conditions, and then to much harsher conditions to find
where the limits of the algorithm lie.
(10)
The signal to noise ratio (SNR) of each phantom image was
calculated using RMS signal and the RMS noise within the
area of the image covered by the cell. The reason for this
being that the surrounding area of the cell before noise
distortion was zero valued, therefore the affects of
on this
region will be zero.
(a) Algorithm 1
The estimate values for rotation were chosen at random
between the ranges: 42 and 1.10.1 for each test of the
algorithm. The values were chosen at random to simulate
variation in users feature selection.
Figure S3 shows the average angle calculated using
algorithm 1 on for the 10 different phantom images. The
results show that the intensity dependent Poisson noise
negatively affects the accuracy of the angle calculation; the
tests with low levels of additive noise (level-0 (red) and level1 (green)) demonstrate this. The effects of the Gaussian
additive noise do not become apparent until levels 4 and 5.
The error bars show that the variance of the results
(
) generally increases with increasing
Poisson noise but does not always affect the accuracy of the
average result. Most of the results here are accurate within
0.1 and no results fall outside 0.2 of the actual result.
The computational efficiency of this algorithm is poor but it is
only required to calculate the rotational offset once per
microscope set up. If implemented using parallel threads
over suitable search range for images approximately
512x512 pixels a result will be calculated in approximately
30s of seconds. To maximise the accuracy of the final result
multiple high contrast, high resolution, and high SNR images
of fixed samples should be used.

(a) Algorithm 2
Before the final translation is calculated the scale difference
is calculated; the accuracy of this determines the accuracy of
the final translation. Figure S4 shows the results for the scale
calculated prior to the final translation (Figure S2). Again, the
variance of the result generally increases with the noise
levels as might be expected. The Intensity dependent
Poisson noise appears to be the main cause of inaccuracies.
The results show that the algorithm is robust against additive
Gaussian noise until they are the main contributor of the total
noise.
Figure S5 shows the results for the final image translation
once the images were scaled. The actual coordinates of
translation of the TIRF image relative to the confocal images
is (20,20); 80 percent of the results fall within a distance of 1
pixel of this result, most of which fall exactly on the correct
result. Calculation of the translation is the most basic part of
the algorithm so any inaccuracies at this stage are almost
certainly due to either inaccurate calculation of the scale or
the rotational offset between images. In practice, the images
the noise conditions of the images used would not be as
harsh as the images used to show the algorithm fail here.
1.
Marks RJ. Handbook of Fourier analysis & its
applications. 1 ed. New York, N.Y. ; Oxford: Oxford
University Press; 2009.
2.
Bracewell RN. Fourier analysis and imaging. New
York: Springer US; 2003.
3.
Lewis J. Fast normalized cross-correlation. 1995:
Citeseer; 1995. p. 120-123.
4.
Webb RH. Confocal optical microscopy. Reports on
Progress in Physics 1996;59:427.

Figure S1: The basic structure of the optimisation algorithm

designed to calculate the rotational offset between TIRF and
Confocal microscopy images.
, and
, are
images from Confocal and TIRF microscopes respectively,
, and
, are the current estimates for the scale
difference and rotational offset between images,
, and
, are the values giving the highest correlation from their
respective tests. Scale is also optimised as precise
knowledge of the scale is required in order to calculate the
rotation precisely.
Figure S2: The basic structure of the algorithm used to
determine the difference in translation between the TIRF and
Confocal images.
Figure S3 A graph showing the average ca cu ated ang e for
different phanto
i ages rotated
and a scale
difference of 1.1 containing different levels of noise. The
Signal to Noise Ratio (SNR) on this graph is that of only the
TIRF images. Each corresponding Confocal image has very
similar noise levels. The error bars here represent the
Standard Error of the Mean (SEM) of the calculated angles.
Each colour series represents a different level of Gaussian
additive noise and has 7 increasing levels of Poisson noise.
Figure S4: The average scale calculated for the 10 phantom
image pairs. The confocal images had a scale factor 1.1
greater than the TIRF images. Error bars represent the
Standard Error of the Mean (SEM). The Signal to Noise
Ratio (SNR) on this graph is that of only the TIRF images.
The corresponding confocal images have very similar noise
levels. Each colour series represents a different level of

Gaussian additive noise and has increasing levels of Poisson

noise.
Figure S5: A plot to show the translation between TIRFconfocal phantom image pairs calculated for different levels
of noise. Prior to testing the TIRF feature was translated by 20 pixels in both x and y directions. The SNR of the images
is indicated by the colour of the data ring. The size of each
ring indicates the number of points already with that value.
Video 1
Video 1 shows movie images; the first is the original data set
which moves due to stage drift, the second shows the same
dataset corrected for stage drift, the third shows the a RGB
merge of the first two images. The RGB merge displays how
much stagedrift occurred over the 45 frames. In the red
plane is the original data and the corrected in the green
plane. The images are of the same cells shown in Figure 3 in
the main article.

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5