Automatic Ground Truth Generation For Image Sequences

International
Journal of Electronics and JOURNAL

Communication Engineering
& Technology (IJECET),
ISSN 0976
INTERNATIONAL
OF ELECTRONICS
AND
6464(Print), ISSN 0976 6472(Online), Volume 5, Issue 9, September (2014), pp. 24-34 IAEME
COMMUNICATION ENGINEERING & TECHNOLOGY (IJECET)
ISSN 0976 6464(Print)

ISSN 0976 6472(Online)
Volume 5, Issue 9, September (2014), pp. 24-34
IAEME: http://www.iaeme.com/IJECET.asp
Journal Impact Factor (2014): 7.2836 (Calculated by GISI)
www.jifactor.com
IJECET
IAEME
AUTOMATIC GROUND TRUTH GENERATION FOR IMAGE SEQUENCES

Shivank Bhagotra
Electronics & Communication Engineering, BKBIET, Pilani, Rajasthan, India
ABSTRACT
This paper presents an approach for generating efficient Ground Truth for image sequences
related to Traffic Signals and Direction Signs by making suitable annotations around the area of
interest and saving corresponding information in the form of XML files. This feature can be readily
used in the cars to detect the speed limit signals and automatically optimize their speed in case of
driverless cars. The Graphic User Interface (GUI) has been designed in GUIDE Matlab and the
Image Database has been provided by the MIPS-MIAM laboratory, Ensisa, Mulhouse, France.
The entire project deals with making a driverless car wherein the real time images as
perceived by the camera placed in front of the car are processedand annotations are formed around
Direction signs and Traffic Speed Limit Signals. The specifications such as Content,
Representativeness, Occlusion, Image Identifier, and Uncertainty get stored in the form of XML
files. This stored data is processed and subsequent changescan be made in the speed, direction of the
car.
Keywords: Annotation, Content, GUIDE, GUI, Ground Truth, XML Files.
1. INTRODUCTION
The annotation tool has been adapted from the software developed at University of Bonn,
Germany, as a general annotation tool. The objective of the project deals with the transformation of
the Tool from the Annotation Tool to the Traffic Annotation Tool which requires deep
understanding of the Annotation tool environment and the ways callbacks are behaving to make
changes in the tool. Objectives to be achieved are:
i.
ii.
iii.
iv.
Addition of static textbox Content and Popup menu associated with it.
Writing the Callback for the popup menu content to make it work like other popup menus.
Changes in the popup menu Source.
Changes in the popup menu Class Name.
24
International Journal of Electronics and Communication Engineering & Technology (IJECET), ISSN 0976
2. ANNOTATION TOOL
Image Annotations generally refer to highlighting features, details, or points of
interest within an image. Annotation Tool is a Matlab based image Annotation Tool developed at
University of Bonn for the Etrims image database and this particular Annotation Tool has been
modified to make it suitable to annotate images from the database (Traffic Signs) related to our
project.
We are working upon the Annotation tool version 1.7 as the latest version of the Annotation
tool which is 2.4 had some bugs indeed, the annotations are not being saved properly in the XML
files and the generated files had some errors as all class names were being shown up in the file.
The Annotation Tool originally employed LabelMe Matlab toolbox developed at the Massachusetts
Institute of Technology (MIT) for the LabelMe image database.Here is an example of Annotation
Tool interface where an image from the Etrims Image database is being annotated:
Fig 1: Screenshot of the Annotation Tool

2.1. SETUP
i. Download the Matlab toolbox LabelMe and add it to the Matlab search path, e.g.
C:\LabelMeToolbox
ii. Download the annotation tool and add it to the Matlab search path, e.g.
C:\AnnotationTool
iii. Create home folders for your image data and annotations e.g.
C:\AnnotationTool\home-images
C:\AnnotationTool\home-annotations
iv.
An annotation tool comes along with three text files two of which should contain home folder
paths. Write the image data home directory and the annotations home directory in the
corresponding text files:
C:\ AnnotationTool\home-images.txt
C:\AnnotationTool\home-annotations.txt
v.
Now we create directories with your image datasets, e.g.:
C:\AnnotationTool\home-images\image-dataset1
25
C:\AnnotationTool\home-images\image-dataset2
vi.
And copy your image data in this e.g.
C:\AnnotationTool\home-images\image-dataset1\my-image.jpg
It is obligatory to create the image dataset subdirectory as this folder name will be stored with the
annotation.In Matlab, we can now run AnnotationTool.m after the first image is labelled, the
Annotation is stored in an XML file of the same name as the labelled image. The XML file with
annotation is saved in the annotations home directory of the same name as the image dataset
directory of the labelled image, e.g.
C:\AnnotationTool\home-annotations\image-dataset1/my-image.xml
3. IMAGE ANNOTATION
3.1. LABELLING IMAGE DATA
Looking at the Data panel in the upper right corner of the (Fig 2).Opening an image dataset
folder by pressing the button Open Image Folder.Then selecting an image in the listbox underneath.
We may now specify the Image sourcein the Current image panel. In case the popup menu does not
offer a relevant option, we may specify an alternative source of the image by choosing the option
other.
After pressing the annotate button on the New Annotation panel the tool will switch to
annotation mode where only image labelling using the mouse is allowed. Annotate by pressing the
left mouse button and clicking in the image area. Pressing the right mouse button will finish the
object labelling and will close the polygon (Fig. 2). If we press the Annotate button now, the
previous label will be erased and we may label the object again. Pressing the right mouse button
without having labelled anything will just cancel the annotation mode. The zoomfeature will ease the
annotation of smaller objects.
Fig 2: Annotating an image object from our database

3.2. OBJECT ANNOTATION
Having labelled an image we may now specify its class, degree of its occlusion,
representativeness and Content (modification) (Fig 3). It is possible to specify an alternative object
class name by choosing the option otherin the Object class pop up menu. The Annotation tool
26
comes along with three text files, one of which, called object-classes.txt contains a list of object class
names. We can simply extend or change the classes by simply editing the text file.
The next value to be specified is the annotation uncertainty: it is an estimate, in pixels, of
how precise the labelling is. A set of green and red concentric circles is drawn in the middle of the
new object label. A diameter of the red circle in pixels corresponds to the value chosen in the
Annotation Uncertainty popup menu.
And the last value to specify is the Content which has been added by me in the panel, where
the user can enter the value of the speed sign when it is being annotated and the same will appear in
the information about the annotation in the XML file generated. Having specified all the required
information we may add the annotation to the Current annotationslistbox by pressing the Add Object
button (Fig 4). A login that should specify the labeller and the place of annotation is required when
adding the first object. This information is added to the annotation of every labelled object.
(a)
(b)
(c)
Fig 3: (a) Specifying the Image Source, (B) Class of the Image Object, (C) Degree of Occlusion and
Content
After adding the new annotation to the pool of the Current Annotationwe have the possibility
of deleting it by pressing the Delete Last button.It is only the last added annotation that the labeller is
allowed to erase. Pressing the Save Annotationbutton will save all current annotations to a file.
(a)
(b)
Fig 4: (a) Adding an annotated image object to current annotations, (b) A login specifying the
labeller and the place of annotation is required when adding the first object
27
4. ANNOTATION TOOL TRANSFORMATION
When I added an additional Content Popup menu, as it would be useful for the tool to
have to annotate images from our database which consisted of Speed Limit Signals,
Guide was not generating a corresponding Callback in the Annotation Tool for the
Content Popup menu even after being saved with the same filename. Then after searching
over the internet, the problem was solved as there was a change to be made in the settings
of the Guide.
Fig 5: Changes to be made for Callback

After adding the desired uicontrol to the Guide environment, we need to go to tools>GUI
options (Fig 5). After this, we need to select the radio button for GenerateFIG-file and M-file
(Fig 6).
Fig 6: Selecting Radio button for both files

After the following changes are made, corresponding callbacks for the added uicontrols will
be generated in the .m file of the tool. Now changes can be made in the Callback related to the
uicontrol to make them function accordingly.
The abling and disabling of the Content popup menu should be the same like the other popup
menus from the tool, meaning it should get activated when annotation is over as the right
28
mouse button is clicked. This was initially not working as there were errors in the Matlab
command window when the code was being run. It was due to static number of properties to
be activated or deactivated fixed to 20 which was prohibiting any addition done to the code
(Fig 7).
Fig 7
Then we set a global variable defines as Handles.NUMCONTROLS and set its value to 21
and then we were able to make popup menu Content enabled like other popup menus (Fig 8).
Fig 8
Therefore, in order to add any uicontrol to the Annotation Tool code we need to increase the
value for the Handles.NUMCONTROLS which is a global variable we have defined at the starting
of the code and make corresponding changes in the code.
Following the above changes, corresponding changes have been made in the subsequent code
and Callback for the content to make it work like the other popup menus and get the value of
Content in the XML file which is generated after the annotation is saved (Fig 9).
Fig 9
29
5. TRAFFIC ANNOTATION TOOL

Several changes have been made in the Annotation Tool to make it Traffic Annotation Tool.
Changes in the popup menu SOURCE.
Change in the source of the image can be made by opening the Property Inspector of
the popup menu Sourceand modifying the String section.
(a)
(b)
Fig 10: (a) Annotation Tool (b) Traffic Annotation Tool
As we can see modification has been done in the popup menu Source according to our
project (Fig 10).
Changes in the popup menu Class name.
The Annotation Tool comes along with three text files, one of which, called objectclasses.txt contains a list of object class names. We can make changes to the set of classes by
simply editing the text file.
Fig 11: Command given in the code to read the text file and load it into the popup menu
As we can see modification has been done in the popup menu Class Name according to our
project (Fig 12).
(a)
(b)
30
Adding an additional Static Textbox Content and popup menu associated with it.
Static textbox Content and corresponding popup menu can be added to the interface
when editable guide window for Annotation Tool is opened by the guide(AnnotationTool)
command in the command window (Fig 13).
(a)
(b)

Writing the Callback for the popup menu Content to make it work like other popup
menus.
Several additions to the code have to be done for adding the Content popup menu and
making it work like the other popup menus. This section has been explained in the later part of
the report (See Fig 9).
Therefore, all the changes have been made successfully and the Annotation Tool is
working correctly. The XML files generated for the annotations have the additional tag called
Content which specifies the value of the speed limit sign in addition to the Class name,
Occlusion, Representativeness and Annotation Uncertainty.
6. RESULTS
By the end of the project we have following Ground truth which is generated for an
annotation when it is saved and can be retrieved from the XML file generated.
Image identifier
We can determine that from the XML file where it is stored as the filename.
Position of the top left corner of the elements
We can determine that by having a look at the XML file generated as it contains the coordinates
the points plotted.
Resolution(in pixels)
Image size (Pixel Dimension) as is in the axis where the image is loaded:
Width: 640 pixels
Height: 480 pixels
Therefore, the resolution of the annotation made can be determined by the axis and the
coordinates we have in the XML file.
Type of Traffic Sign
We can determine the type of traffic sign we are annotating as it is loaded in the Class names
Popup Menu and that similar will be reflected in the XML file whether it is a speed limit sign,
danger sign, no parking sign, etc.
31
Annotation Specifications
We have been able to determine Occlusion, Representativeness, Annotation Uncertainty and Content
for a particular traffic signal.
Creation of an XML File.
Command used = writexml.It transforms a struct or struct array into an xml file.
Filename = name of xml file. V= struct variable containing fields of type char or numeric.
Writexml (filename, xml) will generate an .xml file.
All the results obtained can be seen in this example of an annotated image (3257.jpg) from
our database.
Fig 14: Image annotation by the Traffic Annotation Tool

All the ground truth generated for a particular image by the Traffic Annotation Tool can be
seen over the screen once the annotation is saved (Fig 14). The corresponding XML generated is for
the above image and all the results obtained are well depicted in the XML file and are highlighted.
<annotation>
<filename>3257.jpg</filename>
<folder>image-dataset2</folder>
<sourceImage>ENSISA_UHA</sourceImage>
<rectified>0</rectified>
<viewType>terrestrial</viewType>
<scale>n/a</scale>
<imageWidthHeight>640 480</imageWidthHeight>
<transformationMatrix>n/a</transformationMatrix>
<object>
<name>speed limit signals</name>
<objectID>73552514801738</objectID>
<occlusion>60</occlusion>
<representativeness>40</representativeness>
32
<uncertainty>n/a</uncertainty>
<Content>50</Content>
<deleted>0</deleted>
<verified>0</verified>
<date>18-Oct-2013</date>
<sourceAnnotation>Shivank@UHA</sourceAnnotation>
<polygon>
<pt>
<x>577.1029</x># position of the top left corner
<y>223.9076</y>
</pt>
<pt>
<x>577.8747</x>
<y>264.8098</y>
</pt>
<pt>
<x>537.7443</x>
<y>264.8098</y>
</pt>
<pt>
<x>537.7443</x>
<y>224.6794</y>
</pt>
<pt>
<x>578.6464</x>
<y>223.1359</y>
</pt>
</polygon>
<objectParts>n/a</objectParts>
</object>
</annotation>
7. CONCLUSION
In this paper, Transformation of the Annotation Tool to the Traffic Annotation Tool has
been discussed. The steps to make the transition have been discussed in detail citing all problems
encountered while transforming the code. The corresponding XML file generated contains all the
desired ground truth as our objective in the beginning of the project. Further changes and adoptions
can be made in the code for the Annotation Tool to redesign it according to need of the project. This
work can be further enhanced by designing an interface to incorporate the ground truth generated to
efficiently modify the speed of the car.
8. ACKNOWLEDGEMENT
It gives me immense pleasure to express my deepest sense of gratitude and sincere thanks to
my highly respected and esteemed Mentors Prof. Jean-Philippe Lauffenburger and Jeremie Daniel
(Ecole Nationale Suprieure dingnieurs Sud Alsace (ENSISA), MIPS-MIAM Laboratory) for their
valuable guidance, encouragement and help for completing this work.
33
REFERENCES
[1]
[2]
[3]
[4]
[5]
[6]
[7]
Annotation Tool Documentation:

[http://www.ipb.uni-bonn.de/html_pages_software/annotation
bonn.de/html_pages_software/annotation-tool/
@techreport{annotation-tool
tool-2007,
Author = "Kor{\v
v c}, F. and Schneider, D.",
Title = "Annotation Tool",
Number = "TR-IGG-P-2007--01",
Month = "June",
Year = "2007",
Institute = "Dept. of Photogrammetry, University of Bonn",
URL = http://www.ipb.unibonn.de/html_pages_software/annotation-tool/publ/Korc
http://www.ipb.unibonn.de/html_pages_software/annotation tool/publ/Korc-TR-IGGP-2007-01.pdf
Bryan C. Russell, Antonio Torralba, Kevin P. Murphy, William T. freeman MIT, USA
LabelMe: a database and webbased tool for Image Annotation for International Journal
J
of
Computer Vision, 2008.
For further information on Etrims image database:
[http://www.ipb.uni-bonn.de/projects/etrims_db/]
bonn.de/projects/etrims_db/]
Matlab tutorials can be viewed here:
[http://www.mathworks.com/products/matlab/videos.html]
om/products/matlab/videos.html]
Detailed tutorial on Guide for beginners can be viewed on Math Works Website:
[http://www.mathworks.com/videos/creating
[http://www.mathworks.com/videos/creating-a-gui-with-guide-68979.html]
Abbass.A.Khorsheed, Hussein Chible, Giorgio Giacinto and
nd Ammar J. Fattah, Automatic
De-Noising for
or Image Annotation using Latent Semantic Analysis, International Journal of
Electronics and Communication Engineering &Technology (IJECET), Volume 5, Issue 1,
2014, pp. 113 - 118, ISSN Print: 09760976 6464, ISSN Online: 0976 6472.
BIOGRAPHY
Shivank Bhagotra was born in Jammu, India in 1993. He is currently pursuing
his Bachelors in Electronics and Communication Engineering from B.K. Birla
Institute of Engineering and Technology, Pilani. He has been a Research Intern
at MIPS-MIAM
MIAM Laboratories, ENSISA, Mulhouse,
Mulhouse, France. He has been
associated with IEEE for the past 3 years and currently acting as the IEEE
Mentor BKBIET Student Branch.
34

Automatic Ground Truth Generation For Image Sequences

Transféré par

Informations du document

Titre original

Copyright

Formats disponibles

Partager ce document

Partager ou intégrer le document

Options de partage

Avez-vous trouvé ce document utile ?

Ce contenu est-il inapproprié ?

Droits d'auteur :

Formats disponibles

Automatic Ground Truth Generation For Image Sequences

Transféré par

Droits d'auteur :

Formats disponibles

International

Journal of Electronics and JOURNAL

COMMUNICATION ENGINEERING & TECHNOLOGY (IJECET)

ISSN 0976 6464(Print)

AUTOMATIC GROUND TRUTH GENERATION FOR IMAGE SEQUENCES

Fig 1: Screenshot of the Annotation Tool

Fig 2: Annotating an image object from our database

4. ANNOTATION TOOL TRANSFORMATION

Fig 5: Changes to be made for Callback

Fig 6: Selecting Radio button for both files

5. TRAFFIC ANNOTATION TOOL

Fig 13: (a) Annotation Tool (b) Traffic Annotation Tool

Fig 14: Image annotation by the Traffic Annotation Tool

Annotation Tool Documentation:

Vous aimerez peut-être aussi