Académique Documents
Professionnel Documents
Culture Documents
Usha Mittal Institute of Technology S.N.D.T Womens University Mumbai 400049 2011-2012
CERTIFICATE
This is to certify that Ms. Ketaki Subhash Sawant has successfully completed her final seminar report on Digit recognition using neural networks satisfactorily in partial fulfillment for the award of degree of Bachelors of Technology in Electronics and Communication under S.N.D.T. University, Mumbai during the academic year 2011-2012.
Guide
__________________
Examiner
_________________
(Mrs.Kiran Dange)
ABSTRACT
Digits are used in different types of data like vehicle number plate, numeric data, written data, meter reading etc. Digit recognition plays an important role in many user authentication applications in the modern world. Recently, a lot of work was done by depending on the computer; In order to let the processing time to be reduced and to provide more results that are accurate, for example, depending on different types of data, such as characters and digits and the numbers are used frequently in normal life operation. In order to automate systems that deal with numbers such as postal code, banking account numbers and numbers on car plates various approaches in image processing and pattern recognition have been developed by scientists and engineers to solve this problem. That is because it has an importance in several fields and it may probably be used in checks in banks or for recognizing numbers in cars plates, or many other application
CONTENTS
AbstractI List of figures.II CHAPTER 1 1.1 Introduction..06 CHAPTER 2 2.1 Literature Survey.07 2.2 Methodology....09 CHAPTER 3 3.1 The Feed Forward Neural Network....10 CHAPTER 4
LIST OF FIGURES
Fig 1.1 Examples of different types of four Fig. 1.2 The scenario of number recognition with Artificial Neural Network Which contains input and hiding layer and output with the number (4) for examination the Network. Fig 3.1 Fully Connected, Feed Forward Neural Network
CHAPTER 1: INTRODUCTION
Recently, a lot of work was done by depending on the computer; In order to let the processing time to be reduced and to provide more results that are accurate, for example, depending on different types of data, such as characters and digits and the numbers are used frequently in normal life operation. In order to automate systems that deal with numbers such as postal code, banking account numbers and numbers on car plates. Digit recognition has been extremely found and studied. Various approaches in image processing and pattern recognition have been developed by scientists and engineers to solve this problem .That is because it has an importance in several fields and it may probably be used in checks in banks or for recognizing numbers in cars plates, or many other application. In this study, system for recognized of digits is built, which may benefit various fields, the system concerning on isolated digits, the input is considered to be an image of specific size and format, the image is processed and then recognized to result of an edited digits. An Artificial Neural Network (ANN), often just called a "neural network" (NN), is a mathematical model or computational model based on biological neural networks. It consists of an interconnected group of artificial neurons and processes information using a connectionist approach to computation. In most cases an ANN is an adaptive system that changes its structure based on external or internal information that flows through the network during the learning phase. In more practical terms neural networks are non linear statistical data modeling tools. They can be used to model complex relationships between inputs and outputs or to find patterns in data
There are many fields concern with numbers, for example, checks in banks or recognizing numbers in car plates, the subject of digit recognition appears. A system for recognizing isolated digits may be as an approach for dealing with such application. In other words, to let the computer understand the Arabic numbers that is written manually by users and views them according to the computer process. Here, we present a way to recognize isolated Arabic digits exist in different applications. For example, different users have their own handwriting styles where here the main challenge falls to let computer system understand these different handwriting styles and recognize them as standard writing. Figure 1 shows some examples of different user-handwritten
Fig. 1.2 the scenario of number recognition with Artificial Neural Network Which contains input and hiding layer and output with the number (4) for examination the Network.
2.2 METHODOLOGY
There are four steps to build the isolated digits recognition system. These steps are presented below Image acquisition: We will acquire an image to our system as an input .this image should have a specific format, for example, bmp format and with a determined size such as 3020 pixels. This image can be acquired through the scanner or, digital camera or other digital input devices Preprocessing: After acquiring the image, it will be processed through sequence of preprocessing steps to be ready for the next step. Noise removal: reducing noise in an image. For on-line there is no noise to eliminate so no need for the noise removal. In off-line mode, the noise may come from the writing style or from the optical device captures the image Normalization-scaling: standardize the font size within the image. This problem appears clearly in handwritten text, because the font size is not restricted when using handwriting. Thinning and skeletonization: Representing the shape of the object in a relatively smaller number of pixels .Thinning algorithms can be parallel or sequential. Parallel is applied on all pixels simultaneously. Sequential examine pixels and transform them depending on the preceding processed results. Segmentation: Since the data are isolated, no need for segmentation. With regards to the isolated digits, to apply vertical segmentation on the image containing more than digit will isolate each digit alone. Normalization scaling and translation: Handwriting produces variability in size of written digits. This leads to the need of scaling the digits size within the image to a standard size, as this may lead to better recognition accuracy. We tried to normalize the size of digit within the image and also translate it to a specific position.
Input Layer
Hidden layer
Output Layer
Fig 3.1 Fully Connected, Feed Forward Neural Network The individual nodes, or perceptrons, are representative of the neuron. The input to the node is the input to the neural network or, if the node is on a hidden layer or the output layer, the output from a previous layer. The node is the key to the training of the neural network. The back propagation algorithm propagates the changes to the weights through the neural network by changing the weights of one individual node at a time. With each iteration, the difference between the neural networks output and the desired response is calculated. In the case of a single output, the output of the entire neural network is the output of one individual node whose
10
inputs are the outputs of nodes on the previous layer. By breaking the neural network down to the nodes, the training process becomes manageable. The back propagation algorithm is an LMS-like algorithm for updating the weights. During the training process, the inputs enter the neural network and get summed into the first layer of nodes. The outputs from the first layer of nodes get summed into the second layer of nodes. This process continues until the output comes from the neural network. The output is compared to the desired output, and the error is calculated. The error is used to adjust the weights backwards through the neural network. The weight adjustment equation has one shortcoming; the weights on a particular node cannot be the same as another node on the same layer because the weights will be adjusted the same for each node that has identical weights. If all of the neural network's weights were initially set at zero, the weights would be adjusted the same on each layer. Mathematically, it would be equivalent to having a single node per layer. This is why the weights of the neural network need to be randomly initialized when there is a multi-layer neural network configuration. The other reason for randomly initializing the weights is to properly search the weight space, which is not a quadratic function, as is the linear perceptron. The randomly initialized weights make it very difficult to estimate the initial performance of the control system. Thus Neural Network is a network of non-linear system that may be characterized according to a particular network topology. Where, this topology is determined by the characteristics of the neurons and the learning methodology. The most popular architecture Of Neural Networks used in Arabic digits recognition takes a network with three layers. These are: Input layer, hidden layer and output Layer. The number of nodes in the input layer differs according to the feature vectors dimensionality of the Segment image size. In the hidden layer the number of nodes governs the variance of samples which can be accurately and correctly recognized by this Network.
11
12
13
14
Fig. 1 Total number of rows in each form is 10 and numbers of column in each form are also 10. We can have 100 samples from one person as a specimen sample to analyze.We got nearly 100 forms filled by different users A then scanned takes hundred images.. in next step we cut these
15
images and store each type of image in a different array so that we have 10 samples of each digit image type from on author or person. So total we have so have 100 people fill similar form in Order to have a total of 10*100= 1000 samples for each digit from 1 to 10. So we do the same treatment to each scanned form and then by doing image slicing on the scanned image of a form and made a new image which contained 10 * 100 from different people. And each slice is stored
6.1 Standardization
We convert these image of 1000 slices of 3 into a standard resolution let say 16 *16 pixel. [2]. This will give us a standard size and resolution of the image s o that we are able to apply neural network because the identity of the image through neural network is directly proportional to the resolution of the image . in other words if we use different resolution for these images then we will not be able to compare or standardize as the input neurons will vary with the variation in resolution.
16
6.3 Determining the Horizontal Histogram Vertical Histogram, Left diagonal Histogram and right Diagonal Histogram
Horizontal histogram helps to filter out the variations in the left and right in the shift in horizontal positioning of the digit hand written in the form. Its size is an array of 16 elements
Vertically histogram helps to filter out the variations in they-axis where the shift in vertical positioning of the digit hand written in the form. i.e Its size is an array of 16 elements Diagonal histogram helps to filter out the variations in the angular x-axis and y-axis movements of the digit hand writtenin the form. i.e
17
RIGHT DIAGONAL HISTOGRAM Its size is an array of 31 elements Horizontal histogram helps to filter out the variations in the left and right in the shift in horizontal positioning of the digit hand written in the form. i.e The working nature of these histograms can be ascertained through the following equations and figure. For example horizontal histogram can be represented through equation:(HH+VH+LDH+RDH )= GH HH (Horizontal Histogram ) VH (Vertical Histogram ) RDH (Right diagonal Histogram (Image) LDH (Left diagonal Histogram (Image) GH ( Global Histogram) The complete image signature of a single digit let say ( 3 ) can be represented as (HH+VH+LDH+RDH)= (Global Histogram) Its size is represented by an array of 94 elements
18
We make a database to store each image of data for all the digits. We make a table let say Digitdata which has 95 columns (a1-a94). In column 1 we store the digit ID lets say any digit from( 1 to 10 )So we read all the 1000 images previously sliced and stored one by one and store the global histogram In these column for digits 0 to 9
Fig. 4 Sample Digit At the end we obtain a huge data base of global histograms for each of the digits in the sample pool.
19
IV. MAKING A NEURAL NETWORK The Neural network that we use for this analysis is a 3 layer neural network where 94 input neurons in the input layer: 15 hidden neurons in the hidden layer 10 output neurons: output layer which Correspond to digits 0-9
Input Neurons Input required for 94 input neurons is read from the p4 elements of the global histogram. Output Neurons 10 output neurons suggest the corresponding detection of digits from (1 to 10)
Each neuron stores the following information fields. Layer: Layer means to which layer does each neuron belong to, whether it is input/output or hidden neurons. Index: means what is the index number of the element in the corresponding layer Input data: The input data fed to it. Output data: The data it outputs. Expected output: The desired output expected by us Weights: Value that facilitates the neural network required result Bias: The value to facilitate to calculate the desired output and a already Error: Expected output output data
20
21
This means that for each digit we will have the text files stored in the following manner: Digit 0 = WEIGHT and Bias hidden layer ( 2 files) Digit 0 = WEIGHT and Bias output layer ( 2 files) The same procedure will be used to train neural network for all the 10 digits whose data is available in the INPUTARRAY 10,000 records
22
VI. RECOGNITION
So once the neural network has been trained for all ten digits, now it is possible to identify the meaning of any hand written digit with the help of the trained neural network. So now whenever a handwritten digit will be given as sample input the system will calculate its global histogram and then feed the global histogram to the neural network. This time the neural network will take the bias and weight from the already stored text files and use that for detecting the neuron firing sequence. So in the output array it will automatically give the digit whose corresponding match value is detected. And in case none of the neurons is fired that means that it is a new digit which is not available in the trained neural network files which are also called knowledge base library. Trained data is placed in these text files
VII. CONCLUSION Neural Networks have really created a new vision in the computer and industrial applications. Previously mat lab was used for such simulations but in such implementations one does not have full control, nor the ability to understand that what is happening behind the application. However with more understanding of neural networks, now we have more control over its applications and now we can easily implement such intelligence to identify objects into machines and computers in order to cater our needs in the industrial applications. This
23
application developed as an in house effort by Comsats ITC Center to develop an evaluation system which can be used for grading exams, teacher evaluation and performance assessment of employees. For future work we plan to use the same technique to identify signatures for processing cheques in banking Industry and secondly to develop a face recognition system for HRM Department for student attendance system based on Computer Vision. REFERENCES [1] W.K. Pratt, Digital Image Processing, Wiley, 1978.
[2] Digital image processing (Rafeel C.Gonzalez,Richard E. Woods). [3] Fundamental of Electronic Image Processing (Arthur R. Weeks, Jr. ) [4] M. Minsky, S. Papert, Perceptrons, MIT Press, Cambridge [5] Dejm Gorgevik, Dusan Cakmakov, Vladimir RadevskiHandwritten Digit Recognition Using Statistical and Rule-Based Decision Fusion [6] Cheng-Lin Liu, Kazuki Nakashima, Hiroshi Sako, Hiromichi Fujisawa, ] Handwritten Digit Recognition Using State-of-the-Art Techniques 2002 IEEE [7] Gang Liu, Di Wu, Hong-Gang Zhang, Jun Guo , A new Feature Extraction Method Based on Fourier Transform in Handwriting Digits Recognition, IEEE 2002 [8] Dejan Gorgevik, Dusan Cakmakov , Vladimir Radevski, Handwritten digit recognition by combining support vector machines using rule based ITI 2001
Proceedings of the World Congress on Engineering 2007 Vol I WCE 2007, July 2 - 4, 2007, London, U.K. ISBN:978-988-98671-5-7 WCE 2007
24
CONCLUSION
We can conclude that there is a significant importance of use of isolated digits recognition for different applications. This recognition starts with acquiring the image to be preprocessed through a number of steps. As an important point, classification and recognition have to be done to gain a numeral text.. We will further see how a multilayer neural network will be used for the recognition phase; a feed forward back propagation algorithm will be applied for training the network and finally change them into numeral text. In a final conclusion, neural network seems to be better than other techniques used for recognition
25
26
27
REFERENCES
1. Neural Networks - http://www.csjournals.com/IJCSC/PDF1-2/30..pdf 2. Printed and Handwritten Digits Recognition Using Neural Networks http://webs.uvigo.es/gpi-rv/ficheros/pub/papers/icsp98_1.pdf 3. IEEE journal on -Handwritten digit recognition using combination of neural network classifiers - http://ieeexplore.ieee.org/xpl/freeabs_all.jsp?arnumber=666880 4. IEEE journal on Neural network models and their application to handwritten digit recognition http://ieeexplore.ieee.org/search/freesrchabstract.jsp?tp=&arnumber=366030 5. IEEE research paper on Back-Propagation of a neural network for word spottinghttp://ieeexplore.ieee.org/search/freesrchabstract.jsp?tp=&arnumber=226046 6. DIGIT RECOGNITION NEURAL NETWORK 7. http://www.sff.net/people/dave_slusher/network.htp
28