Analysis of the mitral valves

based on high resolution MR

images
Analyse der Mitralklappen auf Basis hoch aufgelster MR-Bilddaten
Bachelor-Thesis von Li Gong
2010-09-28

GRIS
Medical Computing
Dr. Stefan Wesarg

Analysis of the mitral valves based on high resolution MR images

Analyse der Mitralklappen auf Basis hoch aufgelster MR-Bilddaten
Vorgelegte Bachelor-Thesis von Li Gong
1. Gutachten: Dr. Stefan Wesarg
2. Gutachten: Prof. D.W.Fellner
Tag der Einreichung:

Erklrung zur Bachelor-Thesis

Hiermit versichere ich, die vorliegende Bachelor-Thesis ohne Hilfe Dritter nur mit den angegebenen
Quellen und Hilfsmitteln angefertigt zu haben. Alle Stellen, die aus Quellen entnommen wurden, sind
als solche kenntlich gemacht. Diese Arbeit hat in gleicher oder hnlicher Form noch keiner Prfungsbehrde vorgelegen.
Darmstadt, den

(Li Gong)

Abstract
Todays technology allows physicians and patients with non-invasive methods to look inside the heart in an easy and fast
way. Especially for the heart valves it is practical since they are important for the functionality of the heart and very
difficult to analyze without technical support.
Currently, mitral valve segmentation have been studied thoroughly in the medical image analysis literature mostly using
CT or TEE, but no one have been tried on MR images. This thesis presents a first approach to detect mitral valves on MR
images, it provides a semi-automatic segmentation method that is used on MR high resolved datasets. The developed
algorithm is based on line tracing which requires initial placement of two seed point respectively on the left and right
valve.
From the seed point the best next point is then calculated for the next iteration, assuming that the next point has similar
grayscale value like the current point. To compare the neighboring values, threshold is used in this case, which is developed with k-means, an optimal threshold determination method.
In addition, polar grid is introduced to improve the convergence of the algorithm and to distinguish the opening and
closed mitral valves since the line tracing algorithm is not able to differentiate them. Moreover the algorithm can successfully segment the closed mitral valves.
The algorithm is also visualized by using meshes on the 3D volume which can be modeled and analyzed easily. Additional comparison with manuel segmentation made by an expert are helpful to evaluate the algorithm and put them in
comparison.

Zusammenfassung
Die heutige Technologien ermglichen die rzten und Patienten mit nicht-invasiven Methoden in eine einfachen Art und
Weise im Inneren des Herzens zu schauen. Speziell fr die Herzklappen ist es praktisch, da sie eine wichtig Funktion des
Herzens darstellt, aber sehr schwer zu analysieren sind, zumindest ohne technische Untersttzung.
Derzeit werden Mitralklappe Segmentierung grndlich in der medizinischen Bildanalyse Literatur studiert, meist jedoch
nur mittels CT oder TEE, aber niemand hat auf MR-Bildern versucht. Diese Arbeit stellt einen ersten Ansatz zur Erkennung der Mitralklappen auf MR-Bildern dar, es bietet eine halbautomatische Segmentierungs methode, die auf MR
hochaufgelste Datenstze verwendet wird. Der entwickelte Algorithmus basiert auf Linienverfolgung, die manuelle Platzierung von zwei Saatpunkte erfordert, jeweils auf der linken und rechten Klappen.
Von der Saat Punkt aus wird der beste nchste Punkt fr die nchste Iteration berechnet. Vorraussetzung ist, dass der
nchste Punkt hnliche Graustufen Wert hat wie der aktuelle Punkt. Um die benachbarten Werten zu vergleichen werden
in diesen Fall Schwellwert verwendet, die mit k-means, eine optimale Schwellwert Bestimmungsmethode verwendet.
Darber hinaus werden Polargitter eingefhrt, um die Konvergenz des Algorithmus zu verbessern und die geffnete und
geschlossene Mitralklappe zu unterscheiden, da die Linienverfolgung dies nicht unterscheiden kann. Auf jeden Fall kann
der Algorithmus erfolgreich die geschlossenen Mitralklappen segmentieren. Der Algorithmus wird mit Hilfe von Meshes
auf dem 3D-Volumen visualisiert, dies erleichtert das Modellieren und Analysieren von Mitralklappen. Zustzliche Gegenberstellung mit manuelle Segmentierung von einem Fachmann ist hilfreich, um den Algorithmus zu bewerten und
sie im Vergleich zu stellen.

Contents
1 Introduction

1.1 Task Description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

2 Background
2.1 Medical Background . . . . . . . .
2.1.1 Functionality of the heart .
2.1.2 Cardiovascular system . .
2.1.3 Heart Valves . . . . . . . . .
2.1.4 Cardiac Cycle . . . . . . . .
2.2 Technical Background . . . . . . .
2.2.1 MRT . . . . . . . . . . . . .
2.2.2 Super-resolution Imaging .
2.2.3 Programming Tools . . . .
2.3 Algorithms . . . . . . . . . . . . . .
2.3.1 Sobel Operator . . . . . . .
2.3.2 Segmentation . . . . . . . .

7
8

.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.

3 Related Work

8
8
9
10
10
11
11
11
12
14
14
14
16

3.1 Paper [1]: TRACKING OF THE MITRAL VALVE LEAFLET IN ECHOCARDIOGRAPHY IMAGES by S. Martin,
V. Daanen, J. Troccaz and O. Chavanon . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
3.2 Paper [2]: Patient-Specific Modeling and Quantification of the Aortic and Mitral Valves From 4D Cardiac CT
and TEE by Razvan Ioan Ionasec, Ingmar Voigt, Bogdan Georgescu, Yang Wang, Helene Houle, Fernando
Vega-Higuera, Nassir Navab, and Dorin Comaniciu . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
3.3 Paper [3]: Mitral Annulus Segmentation From 3D Ultrasound Using Graph Cuts by Robert J. Schneider,
Douglas P.Perrin, Nikolay V. Vasilyev, Gerald R. Marx, Petro J. del Nido, and Robert D. Howe . . . . . . . . . . 17
4 Segmentation algorithm
4.1 Preliminary work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.1.1 Build up a new module and integration into Medical Computing Framework
4.1.2 Module interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.2 Segmentation algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.2.1 Image preprocessing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.3 Line tracing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.4 Plane tracing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.5 Polar bounding box . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.6 Segmentation convergence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.6.1 Termination 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.6.2 Termination 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.6.3 Termination 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.6.4 Termination 4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.6.5 Termination 5 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.6.6 Termination 6 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.7 Automatic threshold estimation using k-means clustering . . . . . . . . . . . . . . . .
4.8 visualization of the segmentation result . . . . . . . . . . . . . . . . . . . . . . . . . . .
5 Evaluation

18
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

18
18
18
19
19
20
20
21
22
22
22
23
23
23
23
24
25
26

5.1 Data set . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

5.2 Analysis of the segmentation algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
5.3 comparison of the segmentation algorithm on preprocessed and original data sets . . . . . . . . . . . . . . . . 26

5.4 Analysis of the threshold estimation with k-means . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

5.5 Analysis of valve movement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
6 Conclusion

28

7 Outlook

29

7.1 landmark points . . . . . . . . . . . . . . . . . . . .

7.2 problems . . . . . . . . . . . . . . . . . . . . . . . .
7.2.1 segmentation of the opening mitral valve
7.2.2 data set . . . . . . . . . . . . . . . . . . . . .
7.2.3 dependency of the right seed points . . .
Bibliography

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

29
29
29
29
29
31

1 Introduction
The technical fundamentals of magnetic resonance tomography(MRT such as 1.1) were invented by Bloch and Purcel in
1946, but at the beginning it was only used in the physics and chemical industry. More than 30 years later Lauterbur
and Mansfield constructed an MRT that could be used in medical areas too. This was a huge step for the development of
medical image processing, especially for brain research. Later it was also adapted to other parts of the body.

Figure 1.1: a modern MRT apparat[4]

MR images allow physicians to better evaluate different parts of the body and highlight the presence of certain diseases
that may not be seen adequately with other imaging methods such as x-ray, ultrasound or computer tomography (CT).
One of the advantages of MRT is the high spatiality of the images, which means that it is easier for physicians to
judge images depending on the surroundings of the organ. Also MRT provides good contrast between the different soft
tissues(as can be seen in figure 1.2 and figure 1.3) of the body which makes it useful not only for the brain, but also
for muscles and the heart. Due to the multiplanar capabilities it is possible to record the whole cycles of the heart as a
huge number of 2D image slices and put them together as a series of 3D image volumes which gives the physician the
opportunity to follow cardiac movements, this method is also called cardiac MR imaging. Last but not least, the fact that
MRT has no ionizing radiation makes it a better choice in terms of cancer risks than both x-ray and CT.
Since MRI can provide both anatomical and functional information, it can be used not only to assess cardiac structures,
but also to evaluate cardiac function, cardiac volumes, myocardial viability and perfusion.
The human heart is a muscular organ which provides blood with oxygen and is one of the most important organ in our
body as every part of our body, especially the cells, needs oxygen to survive.
It is like an engine which works days and nights without complaint and without us realizing its importance. But if
something is wrong with the heart, it might be that it is too late to repair it, but in any way it will cause traces of
irreversible embosses.

Figure 1.2: MR Image of the heart(axial plane)[5]

Figure 1.3: CT Image of the heart[6]

The cardiac cycle normally identifiable as the heart beat is one possibility to determine whether a heart is functioning
the normal way. The heart beat causes the blood to flow through our body and hereby supplies all of our organs with
blood. At that point it is time to introduce one of the leading parts of the heart: the heart valves.
The heart valves play a key role in the cardiovascular system as they regulate the blood flow inside the heart chambers
and the blood circulation of the human body. In particular, the aortic and mitral valves execute synchronized rapid
opening and closing movements. The heart valve is a very thin layer and is thus very difficult to see.
At the present state, surgical operations on heart valves, mostly the aortic and mitral valves implantations, are very
common and able to save lives, when detected at an early stage. It is also known that valve operations are the most
expensive and riskiest cardiac interventions, therefore it is important for the physicians to find the abnormality easily. In
most cases echocardiography(Transesophageal echocardiogram as seen in figure 1.4) or CT(and figure 1.5) was used to
detect irregularities in heart valves; however, echocardiography is limited with poor acoustic windows and may be more
operator dependent than other modalities.

Figure 1.4: Image of the heart made with Transesophageal echocardiogram[7]

Figure 1.5: Image of the heart made by CT[8]

Although there are already many segmentation methods of specific heart valves mentioned in various publications,
unfortunately none of these used MR images. That is why it is very interesting to work on it further.

In this thesis the feasibility, capability and quality of the valve segmentation using high resolution MR images will be analyzed. The algorithm used for the segmentation will be a combination of some well-known image processing algorithms
in order to highlight the thin layer of the valve. More details are available in the main part of the thesis.

1.1 Task Description

In this thesis the segmentation of the mitral valves will be implemented through well-known image processing algorithms
on the basis of MR images using super-resolution.
First of all, the valves should be detected through manual initialization while the segmentation algorithm does the rest.
In the next step it should be possible to use the segmentation algorithm on different MR images in each MR image slices.
Using bounding box and landmark points, the algorithm should be able to detect the valves dynamically.
At the end it would be nice to have a visualization including an estimation of the valve movement, so that physicians can
easily detect any abnormality in MR images.

2 Background
2.1 Medical Background

2.1.1 Functionality of the heart

The heart is an organ found in all animals with a circulatory system. The main function of the heart is pumping blood
through our body by repeated, rhythmic contractions.
It weights between 250-350 grams and consists of the different tissues containing muscle, fat and liquid. The heart is as
big as a fist and looks like a rounded cone which is located slightly to the left behind the breastbone(as can be seen in
figure 2.1).

Figure 2.1: location of the heart[9]

The heart is divided into four chambers(figure 2.2), of which the two upper chambers are called the left and right atria
and the two lower ones are called the right and left ventricles. The atria are the receiving chambers in which blood
enters the heart while the ventricles are the discharging chambers because the blood is pushed out of the organ.. The
right ventricle conducts into the lungs to oxygenate the blood. The left ventricle conducts its blood toward the rest of
the body via the aorta which is the bodilyslargest artery. The septum, -thick muscle wall- separates the left and the right
side of the heart. In specialized books, the right atrium and right ventricle combined are known as the right heart and
the left atrium and ventricle as the left heart, respectively.

Figure 2.2: The chambers of the heart[10]

2.1.2 Cardiovascular system

The cardiovascular system -also known as the circulatory system- delivers oxygen and nutrients throughout the body
through a complex network of vessels. Two important and life sustaining vessels are arteries and veins.
Arteries carry blood away from the heart to all parts of the body, veins on the other hand take the oxygen-poor blood
back to the lungs in order to re-oxygen the blood.

Figure 2.3: Pulmonary Circulation: carries oxygen poor blood

away from the heart into the lung(blue) while the
oxygen rich blood returns from the lung to the
heart(red)[11]

Figure 2.4: Systemic Circulation: carries oxygen rich blood

form the heart to the whole body(red) while the
oxygen poor blood returns to the heart in order
to re-oxygen it in the lung(blue)[12]

Pulmonary Circulation
Pulmonary circulation transports oxygen-poor blood from the right ventricle to the lungs where oxygen is added to the
blood. Finally the oxygen rich blood returns to the left atrium.

Systemic Circulation
The systemic circulation provides the blood supply for all organs. It carries oxygen and nutrients to the cells and collects
carbon dioxide and waste products. Systemic circulation distributes oxygenated blood from the left ventricle via the
aorta, through the arteries, to all part of the body. From the tissues, the deoxygenated blood returns through a system of
veins to the right atrium of the heart.

2.1.3 Heart Valves

To avoid disorder of high oxygen blood which comes from the lung and oxygen-poor blood coming from the body, there
are four important mechanisms taking place in the four valves that can be seen in figure 2.5 and 2.6. The valves that
are between the atria and ventricles are called atrioventricular (AV) valves which includes the tricuspid valve, located
between the right atrium and right ventricle and the mitral valve that is between the left atrium and left ventricle.
Each of the outlet valves from the ventricles has three cusps. The valve at the entry to the pulmonary artery is called the
pulmonic valve. The valve at the entry to the aorta is referred to as aortic valve. The valves function like back-pressure
valves which are approximately in the same layer, the so called valve-layer. They are co-located in a connective tissue in
the heart skeleton.

Figure 2.5: the position of the four valves in coronal

perspective[13]

Figure 2.6: the four valves of the heart in the axial

perspective[14]

2.1.4 Cardiac Cycle

The cardiac cycle is the sequence of events that happens when the heart beats. It describes the expansion and contraction
of the heart chambers and correspondingly the opening and closing of valves in the cycle. The cardiac cycle can be
divided into a phase in which the ventricles are contracted and pump blood to the arteries(systole) and a second phase
in which the ventricles are relaxed causing the heart to fill up with blood(diastole).
One cardiac cycle includes one systole and one diastole phase.
For the scope of this thesis it is sufficient to focus on the heart valves role in the cardiac cycle.

Figure 2.7: During the diastole and the systole different valves are opened and closed related on where they are located.
The blue color symbolizes the oxygen poor blood while the red ones emphasized the oxygen rich blood[15]

10

Diastole
During diastole the heart relaxes and fills with blood, causing a relatively low pressure. Oxygen rich blood from the
pulmonary veins flows into the left atrium, while deoxygenated blood flows from the large veins into the right atrium.
The atrioventricular valves open and allow blood to flow from the atria into the ventricles.
The flow of blood through the atrioventricular valves is unidirectional. As volume related pressure increases within the
ventricles about the ten fold causes, it causes the closing of the atrioventricular valves in order to prevent the blood to
flow back into the atria again. Thereby it makes the first part of the heart sound.

Systole
Systole describes the tension and outflow phase of the heart.
At the beginning of the systolic phase, all chambers are filled with blood and the aortic and pulmonary valves -which are
also called the semilunar valves- are open. The pressure in the chambers is very high and therefore both atria contract
to press more blood from the atria into the chambers and the oxygenated blood from the left ventricle is then pumped
through the aortic valve into the aorta which distributes it in the whole body. On the other hand the deoxygenated blood
from the right ventricle is pumped into the pulmonary artery which leads it to the lungs in order to refill the blood with
oxygen. At the same time the chambers are emptied to prepare for the next diastole.
The second part of the heart tone is caused by the closure of the aortic and pulmonary valves at the end of the systole. As
the left ventricle empties, its pressure falls below that of the aorta, and the aortic valve closes. Similarly, as the pressure
in the right ventricle falls below the pressure in the pulmonary artery, the pulmonary valve closes. The second heart
sound has two components: Since the aortic valve closes earlier than the pulmonary valve, they are audibly separated
from each other in the second heart sound.

2.2 Technical Background

2.2.1 MRT
Magnetic resonance tomography (MRT) is a medical imaging technique that uses magnetism, radio waves, and a computer to produce images of body structures. It helps to diagnose, guide and monitor treatment because the images
produced by MRT are very detailed and can detect changes of structures within the body precisely.
Compared to other medical imaging techniques, MRT provides several benefits such as:
No ionizing radiation in comparison to CT or X-Ray
Good soft tissue contrast which makes it especially useful for brain, muscles, heart, and cancer.
Multiplanar capabilities
Volumetric (3D) data sets can be directly acquired
It is quite difficult to understand the whole process of how MRT functions because a lot of physical background information are required. But to provide a simple overview, the general steps of an MR examination will be described briefly:
After the patient is placed on a moveable bed that is inserted into a magnet, a radio wave is sent into the body, just a few
seconds later the radio wave is turned off. A signal is emitted from the body of the patient, which is received and used
for the reconstruction of the image volume which is known as MR images.

2.2.2 Super-resolution Imaging

Super-resolution (SR) are methods that enhance the resolution of the images by using multiple noisy and blurred
low-resolution(LR) images. One necessity is that multiple frames of the same scene/image are available. Most superresolution techniques are based on the same idea: using information from several different images to create one upsized
image. Attention should be payed to the fact that SR methods are not exactly upsize images but also append real details
to the images, therefore the images are clarified instead of blurred.
Super-resolution is often used in medical images, especially in cardiac MR images, because a MR scanner has only a
limited resolution. The problem of cardiac MR images is that the resolution in the perpendicular direction to the slices is

11

lower than within the slices, therefore many important information in that direction may be lost. The solution is to use
two orthogonal sets of image volumes (short axis and long axis) with a different orientation in order to add information
that was lost in the other data set.
Using SR methods does not only help physicians doing better diagnosis but it is also helpful for the segmentation since
they provide a good quality.
All super resolution MR images from this thesis were provided by Sami Ur Rahman who also wrote his Master thesis on
this topic[16].

Figure 2.8: Comparison between super resolution and interpolation[17]

2.2.3 Programming Tools

VTK
The Visualization Toolkit VTK is an open-source C++ software system for 3D computer graphics and visualization of
scientific image data. In addition to visualization it also provides simple image processing It supports a wide variety of
visualization algorithms, for example: vector, tensor, texture, volumetric methods, implicit modeling, polygon reduction,
mesh smoothing, cutting, contouring etc.
It also supports 3D interaction widgets that allow to interact with volumetric data. It is a good visualization option. To
visualize an image, a visualization
pipeline is
required which
transforms information into graphical data. It can be seen
!"#$"%&'()"
#$*+,-$.,/$01
2$2"-$1"
in figure 2.9
E,. =0;N*H0S*6042 :0:.H02. 6/*2;34/5; 0234/5*6042 0264 8/*:,0+*H I*6*

>4N/+.

K*6*(4-G.+6

)0H6./

C4I030.I
K*6*(R-G.+6

C*::./

EF4 -*;0+ 6?:.; 43(4-G.+6; */. 02=4H=.I

Figure 2.9: The source data is read

in and prepared. Filters
are algorithms that modifies the input. Mappers connect the
! #/34,/,567"8/J(15*8.;(@!K(4/
"KL(#/39:,;"4,/,BL(5.;,.;
@#/3<0-=4,/,BL(M
pipeline to the graphics
model.[18]
! #/3>-;0?$/):

ITK

! >4N/+.;J(
! )0H6./;J(
! C*::./;J(

9.2./*6.;OP/.*6.;OA.*I; I*6* 4-G.+6;

Q/4+.;;.;OC4I030.; I*6* 4-G.+6;
>027(43(6,. :0:.H02.#(9.2./*6.; 8/*:,0+*H 4N6:N6L(F/06.; I*6* 30H.;

The Insight Segmentation and Registration Toolkit is another open-source C++ software system used for the development
of image segmentation and image registration programs. ITK is quite similar to VTK in the implementation. Like VTK,
ITK uses a pipeline, too.
!"#$#!%&% '(()*+,-./.0+,(1234/5*607('()*+,8.-0.6(9/*:,0;+,<126./*760=.(>?;6.5.(@9A1>B('(C#>+#(C*66,0*;(D0/;+,2./ '(("

A specialty of ITK is that it can be integrated in VTK when preprocessing of images are necessary or when other segmentation approaches are required.(An example can be seen in figure 2.10)

12

Step 10. Add more ITK

ITK
Reader

ITK
CurvatureFlow
ImageFilter

ITK

VTK

ITK to VTK
ImageFilter

VTK
ImageViewer

VTK
Render
Window
Interactor

Figure 2.10: abstract example of how ITK and VTK work together [19]

Qt-Creator
Qt is a cross-platform and graphical user interface framework implemented in C++. It consists of a class library and
development environment to develop graphical user interfaces (GUI). Qt can be used in several other programming
languages through language bindings.
The class library provides a number of controls with which the user can communicate with the program (e.g. buttons,
combo boxes or sliders). These are also called widgets.

Medical Computing Framework

Medical Computing Framework is developed in the Medical Computing group of TU Darmstadt This application should
help programmers to integrate their implementations in just one application which can be seen in figure 2.11. Every
implementation can be loaded into the application through modules. Modules are plugins that can be dynamically
loaded at runtime. The way these plugins are handled is based on the plug-in system provided by QT. The advantage of
using Medical Computing Framework is that the module that is being created can use the functionality of other existing
modules.

Figure 2.11: The GUI of the Medical Computing Framework; how the modules can be loaded from the application

13

2.3 Algorithms
2.3.1 Sobel Operator
The sobel filter highlights edges in a grayscale image. Edges characterize boundaries and are therefore sometimes
important to be emphasized in image processing. Edges are areas with significant intensity differences, often known as
a leap in intensity from one pixel to the next. Edge detecting means to reduce and filter non-border information while
preserving the important structural properties in an image.
There are several different edge detection filters depending on what one wants to see or to be emphasized in the image.
In this thesis the Sobel operator in combination with a Gauss filter(see remark 1) is the most appropriate filter to highlight
the edges of valves during preprocessing.
Sobel Operator is a directional edge detection filter which belongs to the family of high pass filter. High pass filters
maintain abrupt changes in a grayscale image such as edges or noise but erases the homogeneous part of the image
due to the fact that it suppresses low frequencies in the spatial frequency domain.
Remark 1 (Gauss filter)
Gauss filter belongs to the low pass filter which has the function of blurring/smoothing the images and reduce the noise.
Since the SR-images provided in this thesis contained many noise, it is a good opportunity to reduce them before using the
Sobel filter, that leads to highlight only the edges. Gauss filter uses masks which average the pixel surroundings.

Technically, the Sobel operator is a discrete differentiation operator, more precisely the central difference is used here:
f 0 (x) = f (x + 1) f (x 1). It calculates an approximation of the image gradient at each point. That means areas with
gray value gradients lead to large values whereas on areas of uniform gray value the central difference will be zero.
Remark 2 (Convolution)
Convolution is a way of combining two signals to form a third signal. A filter mask g is applied to the discrete input signal
I (the input image) resulting in the filtered image f . The simplest case of combining the two signals is the linear filter which
replaces each pixel by a linear combination of its neighbors.
XX
f [m, n] = I g =
I[m k, n l]g[k, l]
(2.1)
k

An example is given below shows how the filtered image is generated with the mask g in order to get the f[1,1].

f [m, n]

18

7
9

5
5
4

3
1

-1

-1
-1

g
0
0
0

1
1

It uses a pair of 3x3 symmetric convolution masks(see remark 2), one estimating the gradient in the x-direction (G x ) and
the other estimating the gradient in the y-direction (G y ). A Sobel masks can be seen below( 2.12):
These Sobel masks intensify the edges related to their vertical and horizontal pixel position. The masks can be applied
separately to the input image, to produce separate measurements of the gradient component in each orientation. The
G x mask highlights the vertical edges while the G y mask highlights the horizontal edges. It is also possible to take both
directions in account which means the resulting output detects edges in both directions. The sobel mask additionally
weights edges at the center position higher than that of the surrounding pixels.

2.3.2 Segmentation
Sometimes it is difficult to detect certain structure in an image volumes due to the fact that a volume contains lapped
structures and therefore it is difficult to identify them all. It is thus necessary to simplify and/or change the representation
of an image into something that is more meaningful and easier to analyze.

14

Figure 2.12: A sobel masks for both the x- and the y-direction where the labels highlight the respective weight of the
image[20]

Figure 2.13: The right image shows the result of the Sobel filter of the X-ray image (left).[21]

Segmentation is the way to classify image volumes in different pixel regions either because of the correspondence to
anatomical structures or the regions are homogeneous with respect to some criterion. Segmentation is used in different
areas, it can be used in the medical analyze and planning, for example segmentation of organs for the determination of
physical quantities or the definition of the target region for irradiation of a tumor, but also for the visualization which
means surface extraction and generation of polygon models.
The homogeneity criterion used for segmentation is based on the similarity of pixel intensities. For images containing
noise or blurring, this criterion may be insufficient for successful segmentation. In those cases, segmentation is used
on a preprocessed image volumes for example to intensify the edges(like the Sobel operator see above) will help the
segmentation to be more accurate and efficient.
There are a lot of segmentation algorithms, therefore it is important to know what one wants to see and which criteria
one wants to intensify in the volume. The criteria are: pixel orientated, region orientated, edge orientated, mixed method
or model based method. In this thesis an edge orientated method is developed to detect the valves in a SR-images.
Segmentation can be defined formally:
The image is defined as the union of all regions, where R i is a connected region.

Ri = I

(2.2)

i=1...s

There is no covering between two regions, when they are disjoint with each other.

R i R j = , i, j, i 6= j

(2.3)

The similarity of the pixels leads also to the similarity of the regions.

Is R i , I t R j : Is I t i = j

(2.4)

These formulas are important since any segmentation algorithms work after these criteria.

15

3 Related Work
3.1 Paper [1]: TRACKING OF THE MITRAL VALVE LEAFLET IN ECHOCARDIOGRAPHY IMAGES by S. Martin, V.
Daanen, J. Troccaz and O. Chavanon
S. Martin, V. Daanen, J. Troccaz and O. Chavanon have developed a semi-automatic method for the segmentation and
tracking of the mitral valve leaflet in transesophageal echocardiography(TEE).They used two connected active contours
in order to track the mitral leaflet efficiently. One active contour tracks the cardiac muscle and the other the mitral valve
leaflet. Due to the fact that the mitral valve leaflet is always connected to the cardiac muscle, the connecting point is
labeled as the junction point which can be seen in Figure 3.1.
The segmentation itself is realized in two steps: At first they compute a rough segmentation using curve transformation
and the junction point. The rough segmentation is refined in a second step using snakes((parametric curves fit freely into
the surfaces by using iterative minimization of the sum of internal (shape maintenance) and external (adaptation with
the volume) energy)).
The computation of the mitral curve is necessary at each step and is a transformation of the previous segmentation that
has to be translated
in order to accommodate to the new rough estimate of the junction point. The junction point also
Qm
k+1
plays an important
role in determining the mitral motion.
This method successfully tracks the inner cardiac muscle and the mitral valve leaflets axis.
B UFFER
M USCLE -M ITRAL LEAFLET JUNCTION POINT
M ITRALE LEAFLET ACTIVE C ONTOUR

Qv
k+1
M ITRALE LEAFLET
S EGMENTATION

M USCLE ACTIVE CONTOUR

C)

Figure 3.1: The two contours and their junction point.[22]

MATION

v
Q
k+1

C ONTOUR
3.2 Paper
[2]: Patient-Specific Modeling and Quantification of the Aortic and Mitral Valves From 4D Cardiac CT
R EFINEMENT
and TEE by Razvan Ioan Ionasec, Ingmar Voigt, Bogdan Georgescu, Yang Wang, Helene Houle, Fernando
Vega-Higuera, Nassir Navab, and Dorin Comaniciu

The group around Ionasec has been trying to develop an automatic system for patient-specific modeling and quantification
B)
of the left heart valves (including the mitral and aortic valves)0, which operates on cardiac computed tomography (CT)
and transesophageal echocardiogram (TEE) data. The proceeding is divided into 3 parts: global localization and rigid
motion model, nonrigid landmark motion model, and comprehensive aortic-mitral model.

- b) The two contours - c) Echographic image of the mitral leaflet

ographic
d can be
muscle,

both use
consists
sformamuscle

The global localization of the valves is parameterized through a similarity transformation in the 3D space resulting in a
bounding box limiting the area of the valves.
follows
:
After
that the non-rigid
motion is represented through a model consisting of 18 anatomically defined landmarks(see Fig
3.2.). The anatomical landmark points have to be estimated in every slice in order to track the motion of the valves easily.
The obtained
landmark model guides the
fitting of the full physiological valve model, which
= arg
M
m in [(X X)T S(X
X)+ " W.X +Qc Qt "2 ] is locally refined through
learned boundary detectors.
X
The last step, the comprehensive aortic-mitral model, is realized by using surface meshes of the valves constructed along
(1)
rectangular grids of vertices that can be seen in figure 3.3.

where X is the transformation vector, Qt is the target curve

(the one we want to be close to), Qc is the current curve (i.e.
the initial contour), S is a positive semi-definite weight ma is the mean transformation vector and (" . ") is a
trix, X

16

1640

L/R-Ostium, the two coronary ostia. Besides the well defined

anatomical meaning, the chosen landmarks serve as anchor
points for qualitative and quantitative clinical assessment, are
robustly identifiable by doctors and possess a particular visual
pattern.
TRANSACTIONS
ON MEDICAL
IMAGING,
VOL. 29,
NO. 9, SEPTEMBER 2010
GivenIEEE
the previous
description,
the motion
of each
anatomical
landmark can be parameterized by its corresponding trajectory
over a full cardiac cycle. For a given volume sequence
,
one trajectory is composed by the concatenation of the spatial
coordinates
(2)
where
are spatial coordinates with
and an
equidistant discrete time variable
.
The anatomical landmarks are also used to describe the
global location and rigid motion, defined in Section III-A,
equals to the gravity center of
as follows:
the aortic landmarks, except aortic leaflet tips.
is the
normal vector to the LR-Comm, NL-Comm, RN-Comm plane,
is the unit vector orthogonal to
which points from
to LR-Comm,
is the cross-product of
and
.
is given by the maximal distance
and the aortic landmarks,
between the center
. Analogously to the aortic valve,
along each axes
is computed from the
the barycentric position
mitral landmarks, except mitral leaflet tips.
is the normal
is
vector to the L/R-Trigone, PostAnn MidPoint plane,
and points from
towards the
orthogonal to
are
PostAnn MidPoint. The scale parameters
defined as for the aortic valve, to comprise the entire mitral
anatomy.
C. Comprehensive Aortic-Mitral Model

Fig. 3. Anatomical landmarks of the aorto-mitral complex: (a) aortic and (b)
Figure 3.2:
18 landmark
points
orientate
in model
the valve
Figure 3.3: The right part of the meshes illustrate the mitral
mitral landmarks
in short and long
axis views,to
and (c)
complete landmark
The full geometry of the valves is modeled using surface
(See Fig. Fig.
4 for4.a Isolated
illustration
of thecomponents
landmarks relation
to the comprehensive
surface
with parametric
directions and spatial relations to anatomical landmarks:
(a) aortic
root and
leaflets, mitral as
(c) anterior
motion.[23]
valves
which
are(b)modeled
paraboloids.[24]
meshes constructed along rectangular
grids of vertices.
For each
aortic-mitral model).

and (d) posterior leaflet. Components all together in two different cardiac phases with (e) aortic valve and opened mitral valve closed and (f) vice versa. Aortic L-,
R-, and N-leaflets displayed in green, cyan, and red color, respectively.anatomic structure , the underlying grid is spanned along two

physiologically aligned parametric directions,

and . Each

vertex
has four neighbors, except the edge and corner
B. Nonrigid
Landmark
Motion Model
3.3 Paper
[3]: Mitral
Annulus
Segmentation From 3D
Ultrasound
Using
Graph
Cuts byTherefore,
Robert J. Schneider,
points with
three and
two neighbors,
respectively.
in Fig.and
4 together
with execute
their spatial
relations
to the anatomical
sponding
leaflet
tips. The left/right
trigones and theapostero-anThe aortic
mitral valves
a rapid
opening-closing
vertices
is
represented
by
rectangular
grid
with
Douglas
P.Perrin,
Nikolay
V.
Vasilyev,
Gerald
R.
Marx,
Petro
J.
del
Nido,
and
Robert
D.posterior
Howeleaflets,
landmarks.
nular midpoint further confine the anterior and
movement,
which follows a complex and synchronized motion

The aortic root

connects
the ascending
aorta to
the left venpattern. Normalized
by the
time-dependent
similarity
trans-

triangular faces. The model

respectively.

at a particular time

step
is uniquely defined by vertex collections of the anatomic
tricle
outflow in
tract
and
is
represented
through
tubular
grid
In this paper
theintroduced
authors
are
trying
tothesegment
thea mitral
annulus
with
closed mitral
valves.
The data sets that were used
formation
Section
III-A,
nonrigid
motion
is structures.
the representation
time parameter
D.The
Maintaining
Spatialextends
and Temporal
Consistencyto
[Fig.through
4(a)]. This
is aligned
with
theanatomically-deaortic circumferential
represented
a
model
consisting
of
18
in this paper have been recorded with 3D ultrasound(3DUS)
anddynamics
the algorithm works with a single user-specified point
capture
valve
and ascending
directions
36 points,
20 vertices and
Point correspondence between the models from different carfined landmarks
(see Fig.
3). Three and
aorticincludes
commissure
near theLR-Comm,
center
ofNL-Comm,
the valve.
annulus
location
can belandfounddiac
where
meets
the thicker
1368
faces.
The and
rootThe
is constrained
by
sixthe
anatomical
phasesthe
and thin
acrossleaflet
patients tissue
is required
for building
a statis- heart wall.
RN-Comm,
describe
intermarks,
i.e., tissue
three
commissures
andwhile
threeis
hinges,
with
a fixed
tical shape
model (applied
It is difficult
ob- highlights
To realize
this
alocations
thin
detector(TTD)
used
which
includes
a gauss
filter toin Section
reduceIV-C).
speckle.
The toTTD
connection
of the
aortic
leaflets,
three
hinges,
correspondence
the scale
grid.are
The
three
aortic
leaflets, the L-, R-, tain and maintain a consistent parameterization as(3)
presented in
L-Hinge,
R-Hinge,
and on
N-Hinge,
their
lowest
attachment
thin structures
at a particular
in
the
3DUS.

andtheN-leaflet,
modeled
grid of 11 7 Section III-C in complex 3-D surfaces. However, cutting planes
points to
root. Forareeach
leafletasofparaboloids
the aortic on
anda mitral
andof120
[Fig. 4(b)].free-edge
They are is
stitched
to the root can be applied to intersect surfaces [Fig. 5(b)(d)] and generate
valves, vertices
the center
thefaces
corresponding
marked
on a crown
like attachment
defines
the parametric
[Fig. 5(a)],
which can anatomies
be uniformly
is the number
of represented
and resampled
by the leaflet
tip point:
L/R/N-Tipring,
tips which
for aortic
valves
and where 2-D contours
After the
user
sets
the
initial
point,
the
algorithm
constructs
surface
atmethods.
the
location
ofdefining
the thin
using k-means
direction
at the borders. The
vertex
between the ausing
simple
Hence,
a setleaflets
of physiologare the
numbers
of vertices
for by
a particular
anatomy.
Ant/Post-Tip
(anterior/posterior)
leaflet
tipscorrespondence
for mitral valves.
to compute
the
threshold
for
the
valve
and
the
background.
By
using
a
threshold,
a
corresponding
best-fit
The
six
represented
structures
are
the
aortic
root,
the
three
The two
interconnection
points
of
the
mitral
leaflets
at
their
root and leaflets along the merging curve is symmetric and kept ical-based cutting planes for each model component, surfacesmitral valve
aortic
and the two
mitralsurface
leaflets,
which
are desired
depictedpoint
freebe
edges
areThe
defined
byare
theconstrained
Ant/Post-Comm,
while
fixed.
leaflets
by the corresponding
are consistently
resampled
to establish
the
plane can
computed
which
ismitral
the prerequisite
for
thehinges,
next leaflets
step:
mitral
leaflet
via max-flow.
Withcorrethe estimation
commissures and tip landmarks, where the direction is the as- spondence.
of the valve position
and
orientation,
the
max-flow
algorithm
can
be
used
to
find
a
surface
at
the
location
of the mitral
cending vector from the hinge to the tip.
As mentioned in Section III-C the mitral annulus is a saddle
leaflets. The sink
and
source
of
the
algorithm
are
hereby
located
above
and
below
the
mitral
valve
and
centered
at the
The mitral leaflets separate the LA and LV hemodynami- shaped curve and likewise the free edges are nonplanar too.
and areby
connected
to the
endocardial
wallmin-cut
by the saddle
Thus
a rotation
basedleaflets
resamplingsurface.
method is applied for both
point which iscally
chosen
the user.
The
resulting
defines
the
mitralaxis
valve
shaped mitral annulus. Both are modeled as paraboloids and mitral leaflets [Fig. 5(b) and (c)]. The intersection planes pass
their upper margins define the annulus implicitly. Their grids through the annular midpoints of the opposite leaflet. They are
and the
aligned
with
the circumferential
annulus direction
rotateddetection,
around the normal
of the plane spanned
by the
commisEven though are
there
has
been
a lot of research
in the field
of valve
unfortunately
no one
has
tried it on MRT
orthogonal direction pointing from the annulus toward leaflet sures and the respectively used annular midpoint.
images, but only
on
TEE
or
CT
data
sets.
That
means
that
this
thesis
will
provide
an
first
attempt
to
detect
mitral valves
tips and commissures [Fig. 4(c) and (d)]. The anterior leaflet is
For the aortic root [Fig. 5(d)] a pseudo parallel slice based
in a MRT data
set. This
approach
is quite
similar
to the
mentioned
section
3.2.
distinguish
the mitral valve
and 272
faces while
the paper
pos- method
constructed
from
18 9 vertices
is used.in
Cutting
planes
areTo
equidistantly
distributed
9 vertices
andfrom
368 faces.
terior leafletk-means
is represented
with 24 is
along
the centerline following
and the background,
algorithm
adapted
the last
paper(section
3.3). the direction. To account for
Both leaflets are fixed by the mitral commissures and their corre- the bending of the aortic root, especially between the com-

17

4 Segmentation algorithm
4.1 Preliminary work

4.1.1 Build up a new module and integration into Medical Computing Framework
One requirement of the project was to be able to integrate the application within the Medical Computing Framework(see
2.2.3). That is the reason why the application is written as a module which can be loaded and unloaded into and from
the framework easily.

4.1.2 Module interface

The module interface is set up with 4 views which is from the class QVTKWidget. The 4 views shows the saggital, axial,
coronal and finally the 3D view of the MR dataset. Saggital, axial and coronal views are plane widgets show the different
view directions of the 3D volume. Above the 4 views there is a menu that allows to execute the following actions:
"Load Volume" loads a MR Dataset
"Add Seedpoint" sets a manual seed point. It can be added easily by typing the seed point coordinates into a dialog.
"Start Segmentation" starts the segmentation of the mitral valves by using the line tracing algorithm. The user can
also restart the segmentation with the same seedpoints without using the clean button
"Gauss enable" and "Sobel enable" can be switched on and off depending on whether one want see the original or
the preprocessed dataset
"Clean" resets the whole segmentation and deletes the seedpoints.
a slider that controls the chamber opening, depending on how big the opening in the volume really is.
On the right site there are three widgets one below the other. The first one("file manager") shows the loaded files. The
second widget("file info") shows information about the selected file. The last widget is a listing of seedpoints and their
line and plane thresholds which can be modified by clicking in the corresponding field.

18

Figure 4.1: module interface of the application

4.2 Segmentation algorithm

4.2.1 Image preprocessing

In order to get the optimal data set of the mitral valve for the segmentation, a preprocessing of the image is necessary.
In this case a Gauss filter followed by a Sobel filter is used(see fig. 4.2). The preprocessing algorithms are described in
section 2.3.1.

Gauss filter

Sobel filter

Sobel filter

Figure 4.2: Influence of the filters on the image

19

4.3 Line tracing

The segmentation of the mitral valve is solved by a line tracing, since the valve in the saggital and coronal views look
like a curved line. At first an initial estimation of the position and orientation of the mitral valve relative to the volume
is needed. In the first iteration the position of the valve is estimated by the user, who is asked to add coordinates of two
seed points respectively for the left and right valve.
Starting from the seed point Z1 , which is placed manually on the coronal view where the valve can be clearly seen, in the
following the preceding and succeeding points of the current slice are determined by the line tracing algorithm.
For finding the appropriate successor points Zn + 1 two search beams are used. They are the adjacent points from the
predecessor points, the right search beam contains the above, middle and below points to the right and the left search
beam contains the above, middle and below points to the left respectively(see fig. 4.3). Both directions start at the seed
point, but are being searched independently.

Z'i-3
Z'i+1

Zi-2

Z''i-3
Zi-1

Zi

Z''i+1

Z'''i-3
Z'''i+1

Figure 4.3: Possible succeeding( Zi+1 ) and preceding( Zi3 ) points for the line tracing algorithm

Considering the left to right case, the next point from the seed point is chosen by comparing the grayscale value of
0
00
000
in fig. 4.3). The point with the
the seed point( Zi in fig. 4.3) with those from the right search beam( Zi+1
, Zi+1
and Zi+1
smallest difference in grayscale value from the seed point is chosen for segmentation. Those steps are now being repeated
starting from the newly segmented point. The algorithm stops after a certain number of iterations or occurrence of one
of the termination conditions that will be explained in section 4.6. All chosen points which were segmented are stored
in a point collection. The line tracing from right to left works the same by comparing the grayscale value differences of
0
00
000
the current point( Zi2 in fig. 4.3) with those from the left search beam( Zi3
, Zi3
and Zi3
in fig. 4.3).
The comparison of points is basically based on thresholding which is computationally inexpensive, fast and can easily be
done in real time. The threshold prevents the insertion of points with too big distance from the previous point into the
point collection.

4.4 Plane tracing

Line tracing from the seed points is the initial step of the segmentation, but is limited to the slice where the seed points
were set. To expand the segmentation for the whole volume other slices will also have to be considered.
The problem with line tracing in other slices using the same seed points is the excursiveness of the segmentation result
since the neighboring region might have a similar grayscale value.

20

slicei-1

590 520 555 480 152

1b

weight matrix

590

520

555

480

152

10

550

155

145

158

510

150

500

400

580

520

10

450

570

460

600

440

10

10

10

10

3
3700

310

440

285

10

20

360

25

310

400

30

380

280

3900

490 430 560 405 130

550 155 145 158 510
140 480 120 145 430
150 500 400 580 520

520 150 500 400 510

450 570 460 600 440
450 520 550 450 450

slicei

2b

490

430

560

405

130

140

480

120

145

430

520

150

500

400

450

520

550

450

comparison
point

370

310

440

285

10

20

360

25

310

510

400

30

380

280

390

450

330

400

430

330

330

120

330

4000 4300 3300 3300

Figure 4.4: Plane tracing overview

To solve this problem, a matrix was chosen to represent the grayscale values of the current( 2min fig. 4.4) and previous(
1min fig. 4.4) slice in the volume. First of all a weight matrix is generated from the previous slice is created which has the
min fig. 4.4).
value 10 everywhere, but 1 around the area where the mitral valve was segmented in the previous slice( 1b
At the same time the starting point of the current slice has to be searched, since the seed point no longer exists in this
slice. For the computation of the best point, the point collection containing the segmented points from the line tracing
algorithm will be considered. From the collection the most centric point is taken as the seed point(comparison point) of
that slice( 2m). In the next step the comparison values are subtracted from the matrix. By doing this all values similar
min fig. 4.4). Finally the matrix from 2b will be multiplied with the weight
to the comparison value will be near zero( 2b
m
matrix( 1b ). The resulting matrix has now high values for points that are far away from the previous segmentation
leading to the exclusion of these points during the line tracing. Put differently, only points that have small values will be
considered for segmentation.

4.5 Polar bounding box

So far the line tracing algorithm does not terminate as long as an appropriate next point that falls below the threshold
can be found. To improve this a bounding box with its border specified in the polar coordinate system is introduced.
The motivation for polar coordinates comes from the axial view, in which the mitral valve is roughly a circle with a
defined radius. Polar coordinates bring the benefit that one can change the shape of the radial border in order to fit the
shape of the mitral valve. Also the radius can be set manually, to give the algorithm a clue of the state in the heart cycle.
An additional polar bounding box has been used to define the mitral valve opening, helping the line tracing algorithm to
stop at a certain radius away from the center of the opening. This radius however can be modified individually depending
on the heart cycle using a slider in the GUI. The center of this polar bounding box is estimated from the average of the
two seed points specified by the user.

21

Figure 4.5: The polar grid(yellow) is shown as an overlay on the axial view and the polar bounding box(red) that fits the
mitral valves shape

4.6 Segmentation convergence

To achieve a more precise segmentation, the following break conditions have been used:

4.6.1 Termination 1
The first break condition stops the algorithm when the grayscale value of the current point exceeds the threshold.
minDifference the smallest difference from the current point to its neighboring points
lineThreshold the threshold calculated by the k-means algorithm
if minDifference < lineThreshold then
return
end if

4.6.2 Termination 2
The second break condition removes duplicate points from the collection, this achieves a better mesh representation.
point current segmented point
allsegmentedPoints a collection with all segmented points
if point allsegmentedPoints then
return
end if

22

4.6.3 Termination 3
The third break condition checks whether the current point is inside the chamber bounding box specifying the opening
of the mitral valves. If so the point will not be taken into consideration.
point current segmented point
chamberOpening percentage of valve opening
chamberBoundingbox set of points in the chamber polar bounding box
if chamberOpening > 0 point chamberBoundingbox then
return
end if

4.6.4 Termination 4
If the mitral valves overlap in the horizontal direction of the saggital view, the segmentation stops for the current slice.
point current segmented point
segmentedPointsOtherValve set of points in the segmentation of other valve
{the leftmost x-coordinate of the other valves segmented points}
leftmostX min(extractX(segmentedPointsOtherValve))
{the rightmost x-coordinate of the other valves segmented points}
rightmostX max(extractX(segmentedPointsOtherValve))
if point x leftmostX point x rightmostX then
return
end if

4.6.5 Termination 5
This convergence criteria checks if the current point is inside the polar bounding box, if not the segmentation stops for
the current slice.
point current segmented point
pointsBoundingBox set of points in the polar bounding box
if point 6 pointsBoundingBox then
return
end if

4.6.6 Termination 6
Terminates the line tracing algorithm if the principal direction of the line segmentation changes more than a certain
threshold.

23

buffer buffer which contains the last 10 directions of the search beam
principalDirection
loop
{in the segmentation loop}
...
direction change in direction from the last to current point(can be -1, 0, 1 for up, middle, down)
push(buffer, direction)
if full(buffer) principalDirection > 1 then
principalDirection average(buffer)
end if
...
point current segmented point
if full(buffer) |average(buffer) principalDirection| > threshold then
return
end if
end loop

4.7 Automatic threshold estimation using k-means clustering

The threshold is set manually so far. Using clustering it can be computed not only automatically but also minimizes the
error in segmentation. K-means determines the threshold as the gray-level value that separates the two clusters(mitral
valves and background)with a minimal error. The two clusters represent the grayscale value distributions that belong to
the mitral valves and the background, respectively.
To optimize the result not the whole slice is considered, but only a small cutout(from now on referred to as f cut out (x, y))
showing the surrounding of the seed point and therefore including the mitral valve is taken into account. For the
estimation of the line tracing threshold a slice in the coronal direction is used(see fig. 4.6), whereas the computation of
the plane threshold uses a slice in the saggital direction(see fig. 4.7).
Saggital view

Coronal view

Figure 4.6: the cutout of the area around the seed point in
the coronal view

Figure 4.7: the cutout of the area around the seed point in
the saggital view

At step t = 0 the valve centroid(the center point of the "valve cluster") is initialized with the grayscale value of the seed
point(see eq. 4.1) and the background centroid is initialized by taking all grayscale values in f cut out divided by number
of all pixels in the cutout matrix(see eq. 4.2).

0M V = f cut out (seed x , seed y )

P
0B

(i, j)cut out

f cut out (i, j)

# of pixels

(4.1)

(4.2)

24

At first the threshold between background and mitral valve is computed by taking the average of the two means(see eq.
4.3).
t
t + M
V
(4.3)
T (t+1) = B
2
With the change of the threshold the assignment of grayscale values to the clusters have changed as well. At step t , the
t
centroids Bt and M
V are adapted to the new threshold(see eq. 4.4 and 4.5).
P
(i, j)mi t r al v al v e f cut out (i, j)
t
M
(4.4)
V =
# of object_pixels

P
Bt =

(i, j)backg r ound

f cut out (i, j)

(4.5)

# of background_pixels

Since the most points have a grayscale value in either mitral valve or background range those values have the biggest
impact on the clusters mean value. Consequently the centroids will move towards those two maxima(see fig. 4.8).
grayscale
frequency

mitral valve

background

grayscale
frequency

65536

mitral valve

threshold

Figure 4.8: adjustment of the centroids(see eq. 4.4 and 4.5)

background

65536

threshold

Figure 4.9: adjustment of the threshold (see eq. 4.3)

In each step the adjustment of the threshold (see eq. 4.3) and the centroids(see eq. 4.4 and 4.5) are repeated alternately
until the threshold does not change anymore: T (t+1) = T (t)

4.8 visualization of the segmentation result

The segmentation results are showed as two meshes in the 3D view, where each mesh represents one valve. The segmented points were hereby triangulated using the Delaunay algorithm to create the mesh. The meshes are further
optimized through smoothing and triangle decimation.

Figure 4.10: mesh representation of the segmented mitral valves

25

5 Evaluation
The greatest advantage of the method presented in this thesis is the simplicity it provides in the segmentation. By
using automatic threshold estimation the tracing can be done automatically. A disadvantage of the procedure is that the
algorithm compares only the neighboring grayscale values for the assignment of the next point, and therefore only finds
a local instead of a global optima. Consequently, it is difficult for the line tracing algorithm to decide for the right point
when the surrounding points have similar grayscale values.
In this section, the performance of the segmentation algorithm will be analyzed and evaluated.

5.1 Data set

The image data for this study include 12 MRT volumes and were improved through super-resolution. They were collected
from one patient during a whole cardiac cycle. Each volume has a dimension of 125x125x125 and a color depth of 16
bits(ranging from 0 to 65535).

5.2 Analysis of the segmentation algorithm

The segmentation algorithm is applied on the whole dataset to compare the results of closed and opened mitral valves.
Prominent datasets are the data sets named superResolved0, superResolved10 and superResolved11. The first dataset is
important because it is taken in the systole where the mitral valves are totally closed. The dataset 10 and 11 are taken in
the diastole and are therefore a perfect example for opening valves.
The segmentation results are compared with a manual segmentation provided by Dr. Stefan Wesarg.

5.3 comparison of the segmentation algorithm on preprocessed and original data sets
It is interesting to see how the segmentation performs on the original data sets because the gauss filter smoothed the
edges???????????

5.4 Analysis of the threshold estimation with k-means

Below three exemplary iterations of the k-means algorithm are shown(fig. 5.1). In the graph one can see how the optimal
threshold is computed depending on the mean of the valve and the background. The centroids(red circle) of the valve
and the background are moving up to the peak due to the fact that the grayscale value of the peak is the highest and
therefore affects the choice of the next value. After the three iterations the centroids are located at the peak of each
cluster. The threshold perfectly separates the two clusters(dashed line).

Remark 3 (Histogram)
A histogram is a graphical representation, showing a visual impression of the distribution of data, in this case the distribution
of grayscale values of the current slice. Histograms help to analyze the grayscale values of a slice in order to see which value
belongs to the valve and which do not. It is created by assigning the grayscale value of every pixel to a bin which count the
occurrences of different grayscale values.

5.5 Analysis of valve movement

The evaluation of the mitral valve movement during the whole cardiac cycle is visualized through single linkage and
(best point) average linkage connectivity. Single linkage is a method to measure the distance between two clusters, in

26

First Iteration

Second Iteration

150

Third Iteration

150

135

150

135

135

background

valve

background

valve
120

120

105

105

105

90

90

90

75

75

75

60

60

60

45

45

45

30

30

30

15

15

15

150

300

450

600

750

900

1050

1200

1350

1500

150

300

450

600

750

900

background

valve

120

1050

1200

1350

1500

150

300

450

600

750

900

1050

1200

1350

1500

Figure 5.1: The graphs show the first 3 iterations of the k-means algorithm(from left to right). The red circles symbolized
the centroids and the dashed line depicts the threshold

this case the clusters are the two valves. For single linkage not the whole cluster is being considered but only the two
points from each cluster which are nearest to each other(see eq. 5.2).

s
d(x, y) =

n
X
(x i yi )2

(5.1)

i=1

Figure 5.2: Euclidean Distance

dkl =

min

iC M V , jCB

d(X i , X j )

(5.2)

Figure 5.3: Single Linkage Computation

As an improvement not only the distance of the two closest points is taken, but instead the average distance of the nearest
10 point pairs are calculated, representing the distance between the two valves(see eq. 5.4).

(
mindiff(C1 , C2 , pos) :=

{arg miniC1 , jC2 d(X i , X j )},

if pos = 1,

{arg miniC1 , jC2 (i, j)6mindiff(C1 ,C2 ,pos1) d(X i , X j )} mindiff(C1 , C2 , pos 1), else.

(5.3)

Figure 5.4: computation of the mindiff from two clusters

dkl =

10 (i, j)mindiff(C

d(X i , X j )

(5.4)

M V ,CB ,10)

Figure 5.5: modification of the single linkage Computation

27

6 Conclusion
To recap: The goal of this thesis is to develop a semi-automatic segmentation algorithm on mitral valves using high
resolution MR images. The segmentation algorithm includes line tracing method by comparison of neighboring points
using an automatic threshold which also can be changes individually. Besides it presents a physiologically-based model
by using meshes and also a short evaluation of the valve movement during one cardiac cycle.
The segmentation results should be considered as a first step, but does not justify a final verdict about the possible uses
in the clinical practice.
In summary it should be noted that this is the first approach to segment the mitral valves in MR images. But MR images
have low resolution although they are high resolved and preprocessed, thats the reason why the results are not very
satisfactory. The validation study demonstrates that there are much more effort needed when trying to segment the
mitral valves that are usually done with CT or TEE. The fact that the segmentation were not successful on opening valves
only means that future research and additional options need to be evaluated and more time has to be invested to make
them better than what was possible in this thesis.

28

7 Outlook
In the programming and evaluation presented in this thesis, one realize that there are a lot of barriers which are unfortunately impossible to solve in the time provided for the bachelor thesis.

7.1 landmark points

landmark points can help to achieve the segmentation more precisely by showing important points including the chamber
opening points. It allows to understand where and how the valves move. The landmark points also serve as anchor points
for qualitative and quantitative clinical assessment, are robustly identifiable by doctors and possess a particular visual
pattern.

7.2 problems
7.2.1 segmentation of the opening mitral valve
It was very difficult to segment the opening mitral valve because the line tracing algorithm which looks at the neighboring
points are not able to distinguish the difference of opened and closed valve since they have the same grayscale value.
One possibility is the valve are surrounded by a border which has a bigger grayscale value as the mitral valve itself. It
can be easily distinguish when the mitral valve is closed( 7.1), but soon as the mitral valve is opened, it is difficult to
determine where the border ends and when the algorithm should stop, because the grayscale value of the valve is quite
the same as the grayscale value of the subjacent area( 7.2).
500

100

Figure 7.1: extract from the closed mitral valve where the
bright border below and above the valve can be
clearly seen

Figure 7.2: It can be seen that the grayscale value of subjacent area is quite the same like mitral valve. Also
the border isnt consistent

7.2.2 data set

The dataset has a low resolution therefore it is difficult even for human to detect the mitral valve clearly.In addition, the
datasets are gauss filtered which leads to edge smoothing and thats the reason why the data sets are all the more poor
and also the mitral valve are sometimes not exactly identifiable. It is a drawback because on the one hand the gauss
filter is necessary to remove the noises after sobel filter and on the other hand the edges are smoothed which lead to
inaccuracy of the segmentation. The poor resolution also complicate the tracing of the mitral valve, especially the opened
valves.

7.2.3 dependency of the right seed points

Unfortunately the choice of the seed point is every important and necessary to be on two areas respectively left and right
of the mitral valve. It is also important that the average point of the seed points should be ungefhr in the middle of the

29

valve which is the center of the polar grid needed as a convergence criteria. Using only one seed point is possible but
works only on closed valve due to the fact that the line tracing are not able to jump to another point if the mitral valve is
opened.

30

Bibliography
[1] S Martin, V Daanen, J Troccaz, and O Chavanon. Tracking of the mitral valve leaflet in echocardiography images.
Biomedical Imaging: Nano to Macro, 2006. 3rd IEEE International Symposium on, pages 181184, 2006.
[2] Razvan Ioan Ionasec, Ingmar Voigt, Bogdan Georgescu, Yang Wang, Helene Houle, Fernando Vega-Higuera, Nassir
Navab, and Dorin Comaniciu. Patient-specific modeling and quantification of the aortic and mitral valves from 4-d
cardiac ct and tee. IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. 29, NO. 9, SEPTEMBER 2010, pages 116,
Aug 2010.
[3] Robert J Schneider, Douglas P Perrin, Nikolay V Vasilyev, Gerald R Marx, Pedro J del Nido, Robert D Howe, Senior
Member, and IEEE. Mitral annulus segmentation from 3d ultrasound using graph cuts. IEEE TRANSACTIONS ON
MEDICAL IMAGING, VOL. 29, NO. 9, SEPTEMBER 2010, pages 112, Aug 2010.
[4] image mrt, 2011. http://www.radnuk-am-ring.de/html/kernspintomographie.html.
[5] image heart mrt, 2011. http://de.academic.ru/dic.nsf/dewiki/610755.
[6] image heart ct, 2011. http://anatomy.med.umich.edu/radiology/thorax/heart_ct_d.html.
[7] image heart tee, 2011.
http://upennanesthesiology.typepad.com/upenn_anesthesiology/transesophagela_echocardiography/.
[8] image ct valve, 2011.
http://www.flickr.com/photos/nuclearcardiology/sets/72157602024917842/detail/.
[9] image heart location, 2011. http://bodybrowser.googlelabs.com/body.html#ui=0,0&opa=s:0,m:0.06,sk:
0.39,c:0.39,o:0,ci:1,n:0&sel=&lab=&nav=1.62,77.68,238.25.
[10] Brendan Phibbs. Human heart, the: A basic guide to heart disease, 2nd edition. page 2, Jan 2007.
[11] image pulmonary, 2011. http://www.daviddarling.info/encyclopedia/C/circulatory_system.html.
[12] image systemic, 2011. http://www.daviddarling.info/encyclopedia/C/circulatory_system.html.
[13] image heart valve, 2011. http://healthguide.howstuffworks.com/heart-valve-surgery-picture.htm.
[14] image heart valve2, 2011. http://healthguide.howstuffworks.com/heart-valve-surgery-picture.htm.
[15] image heart valve2, 2011. http://www.texasheartinstitute.org/HIC/anatomy/systole.cfm.
[16] Sami Ur Rahmen. Super-resolution reconstruction of cardiac mr images, 2009.
http://www.gris.informatik.tu-darmstadt.de/~srahman/masterThesisSuperresolution.pdf.
[17] image superresolution, 2011. http://www.gris.tu-darmstadt.de/research/medcomp/projects/srr.en.htm.
[18] Matthias Kirschner. Vtk_tutorial_2, 2010.
http://www.gris.tu-darmstadt.de/teaching/courses/ss10/medvis/exercises/VTK_Tutorial_1.pdf.
[19] Insight Software Consortium Luis Ibanez, William Schroeder. Getting started with itk + vtk, 2010.
http://www.itk.org/CourseWare/Training/GettingStarted-II.pdf.
[20] sobel maske, 2011. http://www.pages.drexel.edu/~weg22/edge.html.
[21] E. Pelikan R. Repges T. Lehmann, W. Oberschelp. Bildverarbeitung fr die medizin, 1997.
[22] J. Troccaz O. Chavanon S. Martin, V. Daanen. Tracking of the mitral valve leaflet in echocardiography images,
2006.

31

[23] Bogdan Georgescu Yang Wang Helene Houle Fernando Vega-Higuera Nassir Navab Razvan Ioan Ionasec,
Ingmar Voigt and Dorin Comaniciu. Patient-specific modeling and quantification of the aortic and mitral valves
from 4d cardiac ct and tee, 2010.
[24] Bogdan Georgescu Yang Wang Helene Houle Fernando Vega-Higuera Nassir Navab Razvan Ioan Ionasec,
Ingmar Voigt and Dorin Comaniciu. Patient-specific modeling and quantification of the aortic and mitral valves
from 4d cardiac ct and tee, 2010.
[25] Roger Boyle Milan Sonka, Vaclav Hlavac. Image Processing, Analysis, and Machine Vision, chapter Thresholding,
page 181. Cengage Learning, 2008.

32