REAL TIME BRAILLE RECOGNITION WITH SONIC FEEDBACK

by Charanya C Kalpana S Nithya R

A Report submitted to

Intel India Research Challenge Contest (2006 07)

Guide Dr.K.S.Easwarakumar Department of Computer Science Anna University

Intel Mentor Ashok Jagannathan

ACKNOWLEDGEMENTS
We express our sincere gratitude to our guide Dr.K.S.Easwarakumar, Professor, Department of Computer Science and Engineering, College of Engineering, Guindy, Anna University, Chennai for his invaluable support, guidance and encouragement for the successful completion of this project. His knowledge, attitude, commitment and spirit have inspired and enlightened us.

We extend our thanks to Ashok Jagannathan, our mentor at Intel for spending his valuable time in giving innovative and useful suggestions to improve the project.

We convey our sincere thanks to St. Louis School for Blind and Deaf, Adyar and The Christian Foundation for Blind, Pallavaram for providing us the Braille samples and cooperating with us during the field research.

And last, but not the least, we wish to thank our parents and family members for bearing with us throughout the project period .

ABSTRACT

Braille scripts are special scripts used by visually challenged people for reading and writing. Real Time Braille Recognition with Sonic Feedback is an application that aims at providing an effective and easy way for reading and writing Braille documents. In this application, a variety of functionalities like Braille Input Interface (for writing Braille documents), Braille character recognition (for converting hand punched Braille documents to English), e-Braille processor (to convert the documents of Braille in the form of Simbraille to text), speech to Braille converter are provided. Complete sonic interaction is provided by means of speech recognition for getting input and responses from the users and sonic feedback for guiding the users through the application. A simple Integrated Handheld Device is designed that eliminates the need for a separate keyboard and mouse. A semi automatic Braille punching device is also designed that makes Braille output effective and meaningful.

LIST OF FIGURES
FIGURE NO 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 TITLE Text to Speech Conversion Engine Speech Recognition Engine Hand Punched Braille Document e-Braille Document Integrated Handheld Device(IHD) Semi Automatic Braille Punching Device(SBPD) Schematic - Script Talk Overview of SAPI Half Character Braille detection Overall Design Hand Punched Braille Processing e-Braille Processing Braille Input Interface Speech to Braille Circuit diagram for semi automatic Braille punching device Schematic for semi automatic Braille punching device Installation UI Braille Character Recognition UI Braille Input Interface UI Speech to Braille conversion UI eBraille Processing Test case 1 Test case 2 Test case 3 Test case 4 Output Performance Analysis 1 Performance Analysis 2 PAGE NO 10 11 11 12 12 12 13 14 14 17 18 19 19 20 20 21 21 22 22 23 23 35 35 36 36 37 38 39

LIST OF ABBREVIATIONS
IHD SBPD TTS SAPI BCR OCR SDK DLL UI API Integrated Handheld Device Semi Automatic Braille Punching Device Text To Speech Speech Application Programming Interface Braille Character Recognition Optical Character Recognition Software Development Kit Dynamic Link Library User Interface Application Programming Interface

CHAPTER 1 INTRODUCTION
1.1 Introduction to Braille Visually challenged people in this world make use of Braille scripts to read and write documents. Braille Scripts are a collection of Braille characters. Standard Braille is an approach to creating documents which could be read through touch. This is accomplished through the concept of a Braille cell consisting of raised dots on thick sheet of paper. The protrusion of the dot is achieved through a process of embossing. A cell consists of six dots arranged in the form of a rectangular grid of two dots horizontally and three dots vertically. With six dots arranged this way, one can obtain sixty three different patterns of dots. A visually Handicapped person is taught Braille by training him or her in discerning the cells by touch, accomplished through his or her fingertips.

1.2 Need for Sonic Interaction Visually challenged users can effectively use the computer for easy reading and writing of Braille documents. Sonic interaction is used to provide a complete and step by step instruction while working with the software. Two features of Sonic Interaction provided here are

1. Sonic Feedback 2. Speech Recognition 1.2.1 Sonic Feedback The Sonic Feedback feature in the software is implemented as a text to speech conversion process wherein a step by step feedback to the users is given. The Microsoft TTS Engine is used here for this purpose. Sonic Feedback is provided in a simple and elegant manner so that the users can make their choice easily.

Fig 1.1 Text to Speech Conversion Engine

1.2.2

Speech Recognition Using the Microsoft Speech Recognition Engine, an application that accepts

the voice input from the users and converts that to English and consequently to Braille for future reference is developed. A high sensitive microphone is used through which the user speaks. An initial training of voice needs to be carried out before using the Speech recognition application. Speech Recognition is introduced here to ensure complete sonic interaction while reading and writing Braille documents. It follows the Speech Application Programming Interface that has the following architecture

Fig 1.2 Speech Recognition Engine

1.3

Braille Character Recognition The Braille documents come in two forms. Hand Punched Documents and

e Braille documents. Methodologies for converting both the hand punched and e Braille documents to English and Speech are provided by this software. 1.3.1 Hand Punched Braille Documents Braille documents are normally hand punched on thick sheets of paper. Those Braille documents can be recognized and converted to English after some preliminary image processing techniques.

Fig 1.3 Hand Punched Braille Document

1.3.2

e-Braille Documents e-Braille documents are the Braille documents that have a Simbraille font

followed. They are generally the printed Braille documents that are available on the web. Such documents can also be recognized and converted to English.

Fig 1.4 e-Braille Document

1.4

Integrated Handheld device(IHD) Visually challenged users may find using the keyboard and mouse difficult.

To ensure ease of use and to provide a totally mouse free software, an integrated handheld device that contains a set of 12 keys is used for the basic operations and data entry. Attached to it are the microphone and headphone facilities that come together as a single unit for easy operation.

Fig 1.5 IHD

1.5

Semi Automatic Braille Punching Device(SBPD) The voice input of the users as said earlier would be converted to English

and subsequently to Braille. To make the Braille output effective and meaningful, a semi automatic Braille punching device is designed that reads the Braille output and punches the output on the paper.

Fig 1.6 SBPD

It works on the basic functionality of a stepper motor and rotates to a certain angle on reading a particular value of input from the file. There needs to be a manual punching at that point of time that creates dots on the paper surface. The paper can be moved through a slit for subsequent character punches. 1.6 Script Talk In todays totally automated and computational world, a software that makes reading and writing of Braille documents more effective and easy is needed. The software is intended to be audio interactive. To avoid the difficulty in the usage of the mouse, speech recognition is also provided. An Integrated Handheld device is developed to make Braille input entry easier. Keeping in mind the requirements of the visually challenged users, our software is aimed at performing the following

Braille input to be entered using the Integrated Handheld device and subsequent conversion to text and then to speech.

Scanned Braille documents to be stored and converted to text and speech on requirement.

Since we aim at providing speech recognition, a speech to text and subsequent text to Braille conversion are also included.

A totally sonic interactive user interface that provides a step by step tutorial for users.

Providing a Simple semi automatic Braille punching device that helps in producing Braille output of the corresponding speech entries.

So entire mouse free sonic interactive software is developed and deployed in a desktop version.

Script Talk Speech Engine

Input Interface

Document Processing

Generate Braille Val Convert to text file Gamma Correction

Image doc

Text to Speech

Speech to Text

Levels

Pixelate

Apply Filter

Image

Filter preview and destination image

Fig 1.7 Schematic - Script Talk

1.7

Overview of SAPI The SAPI provides a high-level interface between an application and speech engines. SAPI implements all the low-level details needed to control and manage the real-time operations of various speech engines. The two basic types of SAPI engines are text-to-speech (TTS) systems and speech recognizers. TTS systems synthesize text strings and files into spoken audio using synthetic voices. Speech recognizers convert human spoken audio into readable text strings and files.

Fig 1.8 Overview of SAPI

CHAPTER 2 LITERATURE SURVEY

This chapter describes the existing works that have taken place in the field of braille character recognition and image processing and approaches that have been followed to improve the performance of braille recognition.

2.1 Prototype for Braille Recognition Lisa Wong et al [1] have proposed a software prototype solution to optically recognize embossed Braille documents using a probabilistic neural network. Braille is a tactile format of written communication for sight-impaired people worldwide. This paper proposes a software solution prototype to optically recognize single sided embossed Braille documents using a simple image processing algorithm and probabilistic neural network. The output is a Braille text le formatted to preserve the layout of the original document which can be sent to an electronic embosser for reproduction. The working of it is like reading the scanned Braille documents and converting them to half characters. The half characters are then converted to braille character codes using neural networks and later to formatted braille document.

Fig 2.1 Half Character Braille detection

The changes that have been proposed to this technique are, instead of using neural networks to recognize the Braille half characters that consumes a lot of time for

training, simple image processing techniques have been adopted that convert the scanned Braille documents to binarized documents recognizable by Braille character recognition. Additionally converting the formatted braille to English has been carried out. Because of this technique, the following advantages are obtained

Time for training neural networks is saved. Error rate is reduced Accuracy improved Speed improved due to character by character processing of the Braille document.

2.2 Analysis of Scanned Braille Document

R.T. Ritchings, et al [4] have proposed a paper on analysis of scanned Braille documents. This paper discusses the significant need for a system to recognize Braille documents in order to preserve them and make them available to visually impaired people. Past approaches used expensive purpose-built and complicated hardware arrangements. This paper describes an approach to a prototype system that uses a commercially available flat-bed scanner to acquire a grey-scale image of a Braille document from which the characters are recognized and encoded for reproduction or further processing. The system is flexible in that it is compact, easy to set up and use and in that it also handles single sided Braille documents. In addition, this solution is cost-effective since the hardware can also be shared with other applications.

Modifications proposed

The error rate that the above paper talks about is because of the absence of dots in some characters in one column of the Braille cell. This

error is handled by our implementation wherein we merge the binarized Braille document with boxes that cover the boundary of the Braille cells. As a result of this we make the Braille character recognition easier and also ensure that the error rate is much reduced.

2.3 Optical Character Recognition Gary Huang et al [7]have proposed a Cryptogram Decoding for Optical Character Recognition wherein they talk about collecting clusters of similar characters Optical character recognition (OCR) systems for machine-printed documents typically require large numbers of font styles and character models to work well. When given a document printed in an unseen font, the performances of those systems degrade even in the absence of noise. In this paper, they perform OCR in an unsupervised fashion without using any character models by using a cryptogram decoding algorithm. The results are presented on real OCR data. Optical character recognition is the task of converting images of text into their editable textual representations. Most OCR systems for machine print text need large collections of font styles and canonical character representations, whereby the recognition process involves template matching for the input character images. Such systems are font dependent and suer in accuracy when given documents printed in novel font styles. They have examined groups of similar characters in the document and assigned labels to the clusters of characters.

Braille Character Recognition We have proposed a new scheme called Braille Character Recognition wherein we simplify the recognition process by dividing the Braille cells into 6 equal divisions and recognize the presence or absence of dots in the 6 regions. A binarized image is taken as input and Braille recognition as explained above is done to convert it to sequence of base 8 values.

2.4 Handheld devices for Control Brad A. Myers et al [3] have proposed handheld devices for control of applications. In this paper they have discussed about the handheld devices that will frequently be in use with the present and future technologies, for interactive communication. Many environments, including offices, meeting rooms, automobiles and classrooms already contain many computers and computerized appliances, and the smart homes of the future will have ubiquitous embedded computation. They have spoken about using handheld devices to improve the user interfaces of everything else in the users environment, rather than being just be another complex gadget that must be learned. Discussions about providing a user interface that spans multiple devices that are in use at the same time have been made. Integrated Handheld Device This discussion was a motivation for us to come up with an integrated hand held device for the visually challenged users so that they can use it handy without the need for any special keyboard or mouse. The integrated device holds a keyboard subset with a microphone and earphone attached to it. This device can be used to do all operations with the software. Thus a totally mouse free sonic interaction is achieved by this device.

CHAPTER 3 DESIGN
This chapter discusses the design methodology adopted to construct the hardware and the software components of the project. First the overall methodology adopted is explained followed by the detailed design of the hardware and software components. 3.1 OVERALL DESIGN Braille input using Integrated Handheld device

Braille Input Module

Image Processing + Optical Character Recognition

English Text

English Speech

Audio Input

English Braille Document

Braille Document Fig 3.1 Overall Design

3.2 DETAILED DESIGN The application involves both software and hardware design as discussed below.

3.2.1 Software Design

The software design includes four modules namely Hand Punched Braille processing e-Braille processing Braille Input Interface

Speech to Braille Conversion

3.2.1.1 Hand Punched Braille Processing The Scanned Braille Document is initially subjected to Gamma correction to enhance the brightness of image. This image is then pixilated to increase the area of coverage of the dots .The process of pixelating is such that the width of each pixel block is kept low and the height is increased to merge the vertical dots to create a Braille column representation This is different from the half character representation as suggested in the paper This intermediate image is then subjected to Binarization with threshold to increase the clarity and represent the column merge in black. This is the ultimate image to be recognized by the Image analysis and Recognition module The Image is then analyzed to look for black pixel around and then position the windows to extract a single column and represent it with 1 and 0 matrixes. This matrix is sensed with stored knowledge base for Braille dots occurrence position and Converted to codes between 0 and 7 representing the binary output. Subsequent codes are translated to English alphabets. Gamma Correction Pixelate Intermediate Image (noised removed) Pixilated Image

Braille Scanned Document Image analyzed to cover each Braille Character (Window)

Edge Detection Binarize Matrix Representation Binary Codes Resize English Text Edge Detected Image

Fig 3.2 Hand Punched Braille Processing

3.2.1.2 e-Braille Processing Braille documents are also available as e braille documents that come in that Simbraille font. Such documents are processed as following

Line count Simbraille Documents Number of dots in each row and num of lines calculated Each dot analyzed

Based on threshold English Text Binary Codes 0 or 1 assigned

Fig 3.3 e-Braille Processing Design

3.2.1.3 Braille Input Interface To Interop.Speech.dll. Braille input using Integrated Hand held device Mapping the entered dots to text Text output stored in a file be implemented using the Microsoft Speech SDK and

Audio Output

Speech SDK and Interop.Speech .dll

Fig 3.4 Braille Input Interface

3.2.3 Speech Text - Braille The input in this case is an audio input which would be converted to English text and subsequently to Braille documents. So the input here is obtained using the Microphone. Microsoft Speech Recognition software is used for recognition purpose. It is converted to text and later to Braille document. Since conversion to text is done, formatted audio output can also be obtained on demand.

Speech input from the users

Speech Recognition using MS Speech Recognition Engine

Domain Specific Mapping

Conversion to Braille files (to be sent to Semi automatic braille punching device) Fig 3.5 Speech to Braille

Conversion to text file

3.2.2 Hardware Design The design for the simple semi automatic braille punching device is shown below. It operates with a stepper motor as base. The circuit diagram for a stepper motor operated braille punching device is shown below

Fig 3.6 Circuit diagram for semi automatic Braille punching device The design of the semi automatic braille punching device is shown below

Braille output from s/w

File Read

Data fed to the stepper motor controller

Manual Punch and paper slider

Buzzer after correct character at the reference point

Angle of Rotation for the read character controlled in program

Braille output in paper

Next character read

Fig 3.7 Schematic for semi automatic Braille punching device

3.2.3

Interface Design

Our application as said earlier is completely sonic interactive and mouse free. So the user interface design gains much importance. Here are a few snap shots that show the user interface designed. Braille Character Recognition

Fig 3.8 Snapshot of Braille Character Recognition UI

Braille Input Interface

Fig 3.9 Snapshot of Braille Input Interface UI

Speech to Braille Converter

Fig 3.10 Snapshot of Speech to Braille conversion UI

E Braille Processing

Fig 21 Snapshot of e-Braille Processing UI

CHAPTER 4 IMPLEMENTATION
This chapter discusses the detailed implementation of the hardware and software components of the project. First the software implementation is explained in detail followed by hardware implementation. Tools and Software used Language - VC#, VB Platform - .Net Supporting SDK- SAPI

VC# for software implementation and VB for hardware implementation

Software Tools

1. Microsoft Speech Recognition Used in the software for enabling speech recognition. This is used to make the visually challenged use the software easily and respond to the sonic feedback received.

Hardware Devices

1. Integrated handheld device For giving the Braille input and speech recognition. 2. Semi Automatic braille punching device for printing the braille documents

4.1 Software Implementation This section will give the overview of the modules implemented and code snippets showing the main functionality

4.1.1

Braille Input Interface Braille input as said earlier gets the input from the visually

challenged user by means of the integrated handheld device and converts it to corresponding text file and audio output.

We have followed a mapping for detecting the entered braille input using the IHD.

The mapping is as follows.

Input
8 , 9 , 5, 6, 2, 3 4 * / +

Action
Braille dots Backspace Right Left Up Next

This module implements the mapping of braille characters to English based on the entered input. The pseudo code is given below

Procedure braille_input_processing begin activate Braille input subsystem of keyboard with NUMLOCK accept input using a braille keyboard subsystem scan the entire document to check if that is end of document or new line switch ( entered key ) begin case valid braille input for each braille character in the input begin if dots are highlighted corresponding column is converted to Braille base 8 code by calling generate_braille_val module

every two consecutive Braille base 8 codes are combined and converted to English character SpVoice class instance converts the recognized character to speech The audio out put is played with Speak module of SpVoice class end end for

case backspace dots corresponding to previous character are cleared off case Up arrow control transferred to upper line Braille dots case down arrow control transferred to lower line Braille dots case Next control transferred to next Braille dot case English option generate _ Braille _val for entire document Convert to Text file for Braille base 8 code Case Speech option generate _ Braille _val for entire document Convert to Text file for Braille base 8 code Convert to speech using Speak method of SpVoice Case New page Generate_braille_val for entire document Convert to Text file for Braille base 8 code Store it in a main string Clear off all the Braille dots on interface Transfer control to first Braille dot End switch

end 4.1.2 Speech to Braille The Speech to braille part of the software is implemented using Microsofts speech recognition engine. The code snippets for accepting speech input and comparing with dictionary is shown below.

1. Initializing the Speech recognition class objects and adding the words to the grammar is shown below

objRecoContext = new SpeechLib.SpSharedRecoContext(); objRecoContext.Hypothesis+=new ISpeechRecoContextEvents_HypothesisEventHandler(Hypo_Event); objRecoContext.Recognition+=new _ISpeechRecoContextEvents_RecognitionEventHandler(Reco_Event); grammar = objRecoContext.CreateGrammar(0); menuRule=grammar.Rule.Add("MenuCommands",SpeechRuleAttributes.SRA TopLevel|SpeechRuleAttributes.SRADynamic,1); object PropValue = "";

grammar.Rules.Commit(); grammar.CmdSetRuleState("MenuCommands", SpeechRuleState.SGDSActive); }

4.1.3 Text to Speech This is another feature of the Microsoft Speech Application Programming Interface that is useful in obtaining speech feedback from the system. It converts the given text to speech and plays back to the users. The code snippet for this module is shown below

The initializations are

private SpeechLib.SpSharedRecoContext objRecoContext = null;

private SpeechLib.ISpeechRecoGrammar grammar = null; private SpeechLib.ISpeechGrammarRule menuRule=null;

The code for converting text to speech is

String t1="English Version"; SpVoiceClass voice1 = new SpVoiceClass(); voice1.Speak(t1, SpeechVoiceSpeakFlags.SVSFDefault);

The SpVoiceClass is the class available in SAPI for converting the text to speech. The usage is shown above.

4.1.4.Image Processing Image processing is carried out as a modified approach in this application to improve the time of response. The following techniques were carried out

4.1.4.1 Gamma Correction Gamma Correction is carried out to increase the brightness of the given image. In this brightness refers to increasing the intensity of the projections that indicate the braille dots. The gamma correction module is shown below

The factor by which we increase the value of each of the entries in the color table is Value = (value/ max intensity) ^ (1/gamma) *max intensity
void processfilter(Bitmap img) { Gamma correction done to increase the brightness of the image Calculate the height and width of the image For the entire height and width of the image, Assign the gamma transformation values to the color table entries Move to the next value End for }

Implementing this module increases the brightness of the image.

4.1.4.2

Pixelation

Pixelation is done to increase the pixel size to a certain extent suitable for further processing and Braille character recognition. The code snippet for pixilation is shown below.
void processfilter(Bitmap img) { // Grayscale image int[] tmp = new int[len];// create a tmp array for holding len num of pixels for ( int y1 = 0, y2 = 0; y1 < height; )// for the entire height of the img { for ( i = 0; ( i < pixelHeight ) && ( y1 < height ); i++, y1++ )//for each pixel's height and img height { // for each pixel for ( x = 0; x < width; x++, src ++ )// for the entire img width { tmp[(int) ( x / pixelWidth )] += (int) *src;// } src += offset;//to move to next row }

// get average values t1 = i * pixelWidth; t2 = i * rem; for ( j = 0; j < len - 1; j++ ) tmp[j] /= t1;//setting the pixel size finally tmp[j] /= t2; // save average value to destination image } }

4.1.4.3 Levels

Levels is one of the image processing techniques that helps to increase the contrast of the braille document. This is required as a preliminary step for binarization. The code snippet for levels is shown below.
void processfilter(Bitmap img) { byte * ptr = (byte *) data.Scan0.ToPointer( ); if ( fmt == PixelFormat.Format8bppIndexed ) { // grayscale image for ( int y = 0; y < height; y++ ) { for ( int x = 0; x < width; x++, ptr ++ ) { // gray *ptr = mapGreen[*ptr]; } ptr += offset; } } }

4.1.4.4 Binarization

Binarization is the process wherein the gray scale image is converted to a binary or black and white image for further recognition. It is implemented as a simple calculation by just comparing with a threshold value and setting the pixel values to 255 or 0 based on the threshold.
void processfilter(Bitmap img) { if (srcFmt == PixelFormat.Format8bppIndexed) { // graysclae binarization for (int y = 0; y < height; y++) { for (int x = 0; x < width; x++, src ++, dst ++) {

*dst = (byte)(((*src >= min) && (*src <= max))?255:0 } src += srcOffset; dst += dstOffset; } } }

4.1.4.5 Edge Detection

The next step of image processing is edge detection where each pixel value is checked for its adjacent diagonal pixel values and a particular pixels value adjusted accordingly. It finally leaves the braille document with dots or protrusions highlighted and the rest of the image left darkened.

The code snippet is shown below.

void processfilter(Bitmap img) { for (int y = 1; y < heightM1; y++) { for (int x = 1; x < widthM1; x++, src ++, dst ++) { max = 0; d = (int) src[-1] - src[1]; if (d < 0) d = -d; if (d > max) max = d; *dst = (byte) max; } src += offset + 1; dst += offset + 1; } } }

4.1.4.6 Invert

The resulting image from edge detection makes the dots lighter and hence they are white. So we now invert the image and change the dots to black color and leave the background image white.

The code snippet for this is shown below.

void processfilter(Bitmap img) {

for ( int y = 0; y < height; y++ ) { for ( int x = 0; x < lineSize; x++, ptr ++ ) { // ivert each pixel *ptr = (byte)( 255 - *ptr ); } ptr += offset; } }

4.1.5

Braille Character Recognition

Braille Character recognition deals with recognition of the scanned braille documents. There are two types of braille documents that are looked at here.

1. Hand Punched Braille documents 2. Simbraille documents

4.1.5.1 Hand Punched Braille Documents

The hand punched braille documents are subjected to image processing techniques and then recognized. The binarized image is segmented into 6 divisions each acting as a boundary for the individual dots in the braille characters. They are recognized based on the threshold value of the number of pixels filled and converted to braille base 8 codes and then to English. The scanned documents are merged with boxes so that each braille character comes inside a particular boundary. The code snippet shows the recognition of the braille documents
void map_ann_input_matrix(Bitmap img) { int interval_width,interval_height; for(j=4;j<matrix_height-5;j+=(matrix_height-9)/3) { for(i=4;i<matrix_width-4;i+=(matrix_width-8)/2) { u=0; interval_height=(matrix_height-9)/3; interval_width=(matrix_width-8)/2; for(k=j;k<(j+(matrix_height-9)/3);k++) { for(l=i;l<(i+(matrix_width-8)/2);l++) { if(ann_input_value[l,k]==1) u++; } } if(u>=5) textBox1.Text+="1"; else textBox1.Text+="0"; } } }

The threshold values as shown above will help in setting the values to a 0 or 1.

4.1.5.2 e-Braille processing

Another type of braille document is the Simbraille document that is available as an e document. Such documents can also be processed using the software. The logic behind this processing is simple. We first count the number of dots in each line of the braille document. Based on the count we run the loop through the entire document length and find the size of the black pixel. If it is beyond a threshold value, it is set to the value of 1 otherwise 0.

The code snippet for this is shown below.

void map_ann_input_matrix(Bitmap img) { count=richTextBox1.TextLength; count=count-sum; co[current_line]=count; sum=sum+count; current_line++; image_start_pixel_y=line_top[current_line]; image_start_pixel_x=0; prev_right=20; output_string=output_string + "\n"; character_present=true; if(ann_input_value[matrix_width/2,matrix_height/2]==1) braille_val[p++]=1; else braille_val[p++]=0; }

4.2

Hardware Implementation

4.2.1 Simple Printer prototype design The design of the simple semi automatic braille punching device is already shown in the previous section. Now we will look at the code that controls the stepper motor connected to the systems parallel port.

Command1_click actions
strInFile = "C:\sample.txt" InFileID = FreeFile Open strInFile For Input As #InFileID If (Sheet1.Cells(17, 1) = Sheet1.Cells(17, 9)) Then Sheet1.Cells(17, 3) = p End If

Do While countit > 0 myTimer = Sheet1.Cells(4, 3)

Out 888, 9 Do While myTimer > 0 myTimer = myTimer - 1 Loop myTimer = Sheet1.Cells(4, 3) Out 888, 8 Do While myTimer > 0 myTimer = myTimer - 1 Loop Out 888, 12 myTimer = Sheet1.Cells(4, 3) Do While myTimer > 0 myTimer = myTimer - 1 Loop Out 888, 4 myTimer = Sheet1.Cells(4, 3) Do While myTimer > 0 myTimer = myTimer - 1 Loop

countit = countit - 1 Loop Out 888, 0 i = i + 1 Loop End Sub

This code controls the stepper motor rotation and stops at a reference point after every character is punched. A buzzer is present to indicate the completion of every character printed. This is only a semi automatic punching device and it can be developed further.

CHAPTER 5 RESULTS 5.1 TEST CASES

5.1.1 Hand punched Braille Document Processing

Sample Input 1

Fig 5.1 Sample braille sheet 1

Sample Input 2

Fig 5.2 Sample braille sheet 2

Test Case

Expected Output

Obtained Output

Sample Braille sheet 1

my name is arun gandhi

xy name is arun gandh

Sample Braille sheet 2

I like it ate my name

I like jt ats fame

Refer the performance analysis section for variations in the obtained output.

5.1.2

e-Braille Processing

Sample Input

Fig 5.3- e-Braille sample 1

Expected output I take immense pleasure in welcoming all

Output I take immense pleasure in welcoming all

5.1.3

Braille Input Interface

Fig 5.4-Braille input interface sample input snapshot

Expected Output

Cat is a pet

Output Cat is a pet

5.2 Performance Analysis

Let us examine each of the facilities in the software for their performance and record the performance parameters.

5.2.1

Hand Punched Braille Processing

Performance parameters

The following three performance parameters influence the braille recognition in this case. They are

1. Reflections and shadows created by protrusions and depressions 2. Position and sizing of the dots in the braille documents 3. Threshold value

Reflections and protrusions

The presence of light and dark regions that are not parts of Braille dots may lead to wrongly identified dots if they happen to fulfill the conditions set for true dots

Position and sizing of dots

If a valid dot lies outside the expected bounds of the Braille character it will not be assumed as part of that character. Hence, the character will not be correctly recognized. The fact that Braille characters do not have between them distinct differences in shape does not enhance recognition .If there is one or more dots at valid positions then the corresponding character will be recognized. It is difficult to assess whether there is an extra dot or whether a dot is missing. Hence, on a single character basis, it is not possible to assess the correctness of the recognition result. Threshold Value

Fixing up the threshold values correctly for the recognition is not possible as the Braille dots may not be accurately placed as mentioned earlier. So the threshold value variations will be reflected in the results.

Taking these performance parameters into consideration we find that our software gives the following performance.

On an average for around 24 characters, our software produces 2 errors. The previous results indicate an error of about 4 characters for every 24 characters. So there is a 50% improvement in the performance compared the earlier implementations. The improvement can be attributed to the following

1. Drawing boxes around the Braille characters to ensure that there is a boundary for every character and hence the 2nd performance parameter is effectively dealt with here. 2. The threshold values have been set after a lot of trials to ensure good results for a maximum number of documents

The results are depicted below

performance analysis of hand punched braille documents

% of correct recognitions 120 100 80 60 40 20 0 1 2 3 4 5 5 6 7 8 9 10 sample number grid technique boxed braille

Fig 5.5 Performance analysis for hand punched braille documents

5.2.2

e-Braille Processing

In this case the performance parameter is the time taken to recognize the entire simbraille document. The following graph shows the relationship between the number of lines in the simbraille document and the corresponding time taken to recognize the document.
performance analysis of simbraille documents
140 120 100 number of 80 characters 60 40 20 0

123 96 72 35 10 20 7 25 2 14 13 60 chars

time in sec

Fig 5.6 Performance analysis 2

5.2.3

Speech to Braille

Performance parameters for speech input would be the following 1. Sensitivity of the microphone 2. Training of voice 3. Clarity in speech

CHAPTER 6 CONCLUSION AND FUTURE WORK

In this project, work has been carried out to improve the performance of the some existing braille recognition systems. Experiments include scanning of single sided braille documents, Braille input by the users by means of IHD, and design/prototype of a semi automatic braille punching device. It should be noted, however, that this project concentrates on the application of the single sided braille documents, which pose good challenges as mentioned earlier. As a whole, the approach described here shows the feasibility of a cost-effective, fast and easy to use Braille reading system. It does not require expensive custom made and complicated hardware.

Further development of such braille recognition systems could focus on

1. Scanning of double sided Braille documents. 2. Automating the developed braille punching device 3. Conversion of hand written documents to braille

REFERENCES
[1] Wong, L. Abdulla, W. Hussmann, S. ,A software algorithm prototype for

optical recognition of embossed Braille, In Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004. pages 586- 589 Vol.2,Aug. 2004

[2] Soumyajit Dey, Susmit Biswas, Arijit Mukhopadhyay, Anupam Basu An Approach to Architectural Enhancement for Embedded Speech Applications In Proceedings of the 19th International Conference on VLSI Design pages 371 - 376 , 2006

[3] Brad A. Myers, Jeffrey Nichols, Jacob O. Wobbrock, Kevin Litwack, Michael Higgins, Joe Hughes, (HCI 2003) Handheld Devices for Control Human-Computer Interaction Consortium

[4] R.T. Ritchings, A. Antonacopoulos and D. Drakopoulos, Systems Engineering Group, Department of Computation (1995), Analysis of scanned braille documents In the book Document Analysis Systems, pages 413-421

[5] J. Dubus., M. Benjelloun., V.Devlaminck, F. Wauquier, and P. Altmayer. Image processing techniques to perform an autonomous system to translate relief braille into black-ink, called: Lectobraille. In Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, pages 1584 1585, 1988

[6] I. Murray and T. Dias. A portable device for optically recognizing braille - part i: hardware development. In The Seventh Australian and New Zealand Intelligent Information Systems Conference 2001, pages 129134, 2001.

[7] G. Franois and P. Calders, The Reproduction of Braille Originals by Means of

Optical Pattern Recognition, Proceedings of the 5th International Workshop on Computer Braille Production, Heverlee, 1985, pp. 119122.