S. Bertoluzza, G. Maggi, S.

Tomatis

Problem: individuate anatomical structures in medical images Atlas based segmentation

Image R (to be segmented)
Atlas: image T already segmented Register R and T

R inherits the segmentation of T

Reduce segmentation problem to registration problem

Grey scale images <-> real valued functions R: R -> R+, T: T -> R+ Find mapping : R -> T such that T0 and R are as close as possible Huge number of approaches Huge number of algorithms

See [Brown 92; Zitova,Flusser 03]

Rigid transformations (translations, rotations) Affine transformation Non rigid transformations

Splines, B-splines [Thevenaz, Unser 98] Thin plate splines [Bookstein, 89]

Interpolating wavelets [S.B., G. Maggi 13]

... Parametric transformations

N-dimensional parameter space Parameter -> mapping (Ex: = i ei )

= (x;)
x : spatial coordinate (in R2 ) : parameter (in RN ) : R2 x RN function (represents the selected class

of transformations)

Im: space of images

d: Im x Im -> R functional measuring the discrepancy between the two images : Im -> R defined as (X) = D(X,R) c : RN -> R cost functional c() = d(T0 ,R) = (T0 )

Many different possibilities

Least square error Human Visual System model distance [Mannos, 74] Structural similarity index SSI [Wang et al, 04, Brunet, Vrscay, Wang 11] Besov functional norm Besov norm + divisive renormalization (generalised SSI) [S.B., G. Maggi, 13] Mutual information [Viola, Wells 97; Thevenaz, Unser 98] ...

Find in RN
= arg min c()

Unconstrained optimization problem

Several possible optimization algorithms Choice depends on the characteristics of
Transformation Distance functional

Need rules to
Evaluate images out of their domain of definition

(extrapolation) Compute values of (deformed) images at pixels (interpolation) Compute the values of derivative of images at pixels (numerical differentiation)

Many different possibilities

Huge number of algorithms obtained by combining

Different distance functionals Different transformation classes Different image models Different optimization algorithms

Develop an environment allowing to easily test different combinations

Most optimization algorithms need to compute c Chain-rule

c( ) D (T ; x)T ( ( x))J ( x; )dx

II

III

I. Frchet derivative of the functional

D ( X ; x)

II. Gradient of the image

T ( y )

III. Jacobian (w.r. to the parameter) of the transformation J ( x; )

Image module
Transformation module Distance module

Optimization is dealt with by the Matlab Optimization Toolbox

Input:
The parameter

Output:
The grid G obtained by deforming the reference

grid via the mapping The values assumed by the jacobian (w.r. to the parameter ) of the function at the points of the reference grid

Input:
A sampled image X A grid G of arbitrary points (representing the

center of the deformed pixels)

Output:
An interpolated (or extrapolated) value of X at the

points of G A value of X at the points of G

Input
An arbitrary image X sampled at the points of the

reference grid

Output:
The value of the cost function c(X)

The value at the points of the reference grid of

(the Frchet derivative of the cost functional at X)

To evaluate c()
Call the mapping module to obtain the deformed

grid G Call the image model module to sample T at the grid G -> T = deformed image sampled at the reference grid Call the distance module to evaluate c() = d(T;R).

To evaluate c
Call the mapping module and evaluate J (x; )

at the points of the reference grid Call the image module and evaluate T at the points of the deformed grid Call the distance module and evaluate D (T,x) at the points of the reference grid Evaluate the integral by a chosen quadrature rule

Flexible algorithm allowing to switch one of the ingredient without modifying the others New distance functionals / image models / transformation classes can be incorporated easily
Idea can be extended to the computation of the Hessian

Computational cost: evaluation of the cost functional

64 x 64 128x128 256x256 512x512

LS
SSIM G-SSIM MI(*)

0.2815
0.23438 1.125 2.106

0.35938
0.67188 1.5313 8.0965

0,73438
1.8906 2.9531 31.902

3.0156
6.5625 12.875 134.41

(*) computed on a machine twice as fast

REFERENCE IMAGE

TEMPLATE IMAGE

CT studies, of the same patient by different CT machines

Quality of the results

Unreg. Pearson Overlap 0.245 0.492 LS 0.858 0.992 SSIM 0.872 0.993 G-SSIM 0.926 0.992

M1
M2

0.47
0.253

0.833
0.868

0.846
0.861

0.85
0.933