A

Seminar Report On

N EURAL NETWORK

Submitted By
Suraj Maurya - 111P004
Sanjeev Vishawakarma - 111P019
Sandeep Warang - 111P006

Under the guidance of

Prof. Dinesh B. Deore

Department of Computer Engineering

Rizvi College of Engineering
New Rizvi Educational Complex, Off-Carter Road,
Bandra(w), Mumbai - 400050

Affiliated to
University of Mumbai
 Rizvi College of Engineering
Department of Computer Engineering
New Rizvi Educational Complex, Off-Carter Road,
Bandra(w), Mumbai - 400050

CERTIFICATE
This is certify that

Suraj Maurya
Sanjeev Vishwakarma
Sandeep Warang
of Third Year Computer Engineering have completed the seminar work entitled Seminar Topic Ti-
tle under my supervision at Rizvi College of Engineering, Mumbai under the University of Mumbai.

Prof. Dinesh B. Deore Prof. Dinesh B. Deore

Project Guide HOD, Computer Department

Internal Examiner External Examiner

Date:
 Acknowledgements

I am profoundly grateful to Prof. Dinesh B. Deore for his expert guidance and continuous encourage-
ment throughout to see that this report rights its target since its commencement to its completion.

I would like to express deepest appreciation towards Dr. Varsha Shah, Principal RCOE, Mumbai and
Prof. Dinesh B. Deore HOD Computer Department whose invaluable guidance supported me in com-
pleting this report.

At last I must express my sincere heartfelt gratitude to all the staff members of Computer Engineering
Department who helped me directly or indirectly during this course of work.

Suraj Maurya
Sanjeev Vishwakarma
Sandeep Warang
 ABSTRACT

This report presents an emergence of an Artificial Neural Network (ANN) as a tool for analysis of differ-
ent parameters of a system. An Artificial Neural Network (ANN) is an information-processing paradigm
that is inspired by the way biological nervous systems such as brain, process information. ANN consists
of multiple layers of simple processing elements called as neurons. The neuron performs two functions,
namely, collection of inputs & generation of an output. Use of ANN provides overview of the theory,
learning rules, and applications of the most important neural network models, definitions and style of
Computation. The mathematical model of network throws the light on the concept of inputs, weights,
summing function, activation function and outputs. Then ANN helps to decide the type of learning for
adjustment of weights with change in parameters. Finally the analysis of a system is completed by ANN
implementation & ANN training and prediction quality.

Keywords : Biological Inspiration, ANN Methodology, ANN Implementation and Prediction.

 I NDEX

1 Introduction 1
1.1 Evoluation of Data Mining . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.2 Data mining functionality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.3 Data mining techniques . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

2 Neural Network 3
2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
2.2 Neuron in brain . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
2.3 Characteristic of ANN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
2.4 Activation Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

3 Perceptron 8
3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
3.2 Perceptron Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
3.3 Classes of learning algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
3.4 Learning algorithms for neural networks . . . . . . . . . . . . . . . . . . . . . . . . . . 10
3.4.1 Supervised Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
3.4.2 Unsupervised Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
3.4.3 Reinforcement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
3.5 Perceptron Training . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

4 Feedforward Neural network 12

4.1 Defination . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
4.2 One-hidden-layer Neural network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

5 back-propagation 14
5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
5.2 Back-propagation network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
5.2.1 ANN Development & Implementation . . . . . . . . . . . . . . . . . . . . . . 14
5.2.2 ANN Training & Prediction quality . . . . . . . . . . . . . . . . . . . . . . . . 15

6 Conclusion and Future Scope 16

6.1 Recent advances and future applications of NNs include: . . . . . . . . . . . . . . . . . 16
6.2 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

References 18

APPENDICES 18
A Project Hosting 19
List of Figures

1.1 General Scheme of Genetic Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

2.1 Neural Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

2.2 characteristic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
2.3 Threshold . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
2.4 Sigmoid . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
2.5 Gaussian . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

3.1 Neural Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

3.2 Three cluster and a classifier network . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
3.3 Classess of larning algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
Chapter 1 Introduction

Chapter 1

Introduction

Data mining is the semi-automatic discovery of patterns, associations ,changes, anomalies, and statisti-
cally significant structures and events in data. Traditional data analysis is assumption driven in the sense
that a hypothesis is formed and validated against the data. Data mining, in contrast, is data driven in
the sense that patterns are automatically extracted from data. The goal of this tutorial is to provide an
introduction to data mining techniques. The focus will be on methods appropriate for mining massive
datasets using techniques from scalable and high performance computing. The techniques covered in-
clude association rules, sequence mining, decision tree classification, and clustering. Some aspects of
pre-processing and post processing are also covered. The problem of Predicting contact maps for protein
sequences is used as a detailed case study.

Figure 1.1: General Scheme of Genetic Algorithm

Data mining is process of identify patterns and establish relationships Data Mining defined as
The nontrivial extraction of implicit, previously unknown, and potentially useful information from data.
Data mining is the process of analyzing large amount of data stored is a data warhorse for useful informa-
tion which makes use of artificial intelligence techniques ,neural network ,and advance statistical tools
(such as cluster analysis) to reveal trends, patterns and relationship, which otherwise may be undetected.
[3]

Rizvi College of Engineering, Bandra, Mumbai. 1

Chapter 1 Introduction

1.1 Evoluation of Data Mining

Since the 1960s, database and information technology has been evolving systematically from primitive
le pro-cessing systems to sophisticated and powerful databases systems. The research and development
in database systems since the 1970s has led to the development of relational database systems , data
modelling tools, and indexing and data organization techniques. In addition, users gained convenient and
edible data access through query languages, query processing, and user interfaces. E- Clientmethods
for on-line transaction processing (OLTP), where a query is viewed as a read-only transaction, have
contributed substantially to the evolution and wide acceptance of relational technology as a major tool
for e-client storage, retrieval, and management of large amounts of data.

1.2 Data mining functionality

Association
Looking for patterns where one event is connected to another events
Sequence or path analysis
Looking for patterns where one event leads to another events
Forecasting
Discovering patterns in the data that can lead to reasonable predictions about the future
Multidimensional concept description
Generalize, summarize and contrast data characteristics e.g. dry vs. wet region
Cluster analysis
Class label is unknown : group data to form new classes eg cluster houses to fing distribution
patterns
Maximizing intra class similarity and minimizing interclass similarity
Trend and evaluation analysis
Trend and deviation eg regression analysis
Sequential pattern mining eg digital camera tends to large memory
Periodicity analysis
Similarity based analysis
Statistical analysis

1.3 Data mining techniques

Techniques based on statistical concept
Point estimation
Bayes theorem
Hypothesis testing
Regression and correlation
Similarity measures
Neural network

Rizvi College of Engineering, Bandra, Mumbai. 2

Chapter 2 Neural Network

Chapter 2

Neural Network

2.1 Introduction
first wave of interest in neural networks (also known as connectionist models or parallel Distributed
processing emerged after the introduction of simplified neurons by McCulloch and Pitts in 1943 (Mc-
Culloch & Pitts 1943) .These neurons were presented as models of biological neurons and as conceptual
components for circuits that could perform computational tasks. When Minsky and Papert published
their book Perceptrons in 1969 (Minsky and Papert 1969) in which they showed the deficiencies of per-
ceptron models most neural network funding was redirected and researchers left the field.Only a few
researchers continued their efforts, most notably Teuvo Kohonen, Stephen Grossberg, James Anderson,
and Kunihiko Fukushima.The interest in neural networks re-emerged only after some important theo-
retical results were attained in the early eighties (most notably the discovery of error back-propagation)
and new hardware developments increased the processing capacities. This renewed interest is reflected
in the number of scientists, the amounts of funding, the number of large conferences, and the number
of journals associated with neural networks, Nowadays most universities have a neural networks group,
within their psychology, physics, computer science, or biology departments. Artificial neural networks
can be most adequately characterized as computational models with particular properties such as the
ability to adapt or learn, to generalize, or to cluster or organize data, and which operation is based on
parallel processing. However, many of the above-mentioned properties can be attributed to existing
(non-neural) models, the intriguing question is to which extent the neural approach proves to be better
suited for certain applications than existing models. To date an equivocal answer to this question is not
found. Often parallels with biological systems are described. However, there is still so little known
(even at the lowest cell level) about biological systems, that the models we are using for our artificial
neural systems seem to introduce an oversimplification of the biological models.
Data mining is process of identify patterns and establish relationships Data Mining defined as
The nontrivial extraction of implicit, previously unknown, and potentially useful information from data.
Data mining is the process of analyzing large amount of data stored is a data warhorse for useful informa-
tion which makes use of artificial intelligence techniques ,neural network ,and advance statistical tools
(such as cluster analysis) to reveal trends, patterns and relationship, which otherwise may be undetected.
[3]

2.2 Neuron in brain

Although heterogeneous, at a low level the brain is composed of neurons
1. A neuron receives input from other neurons (generally thousands) from its synapses
2. Inputs are approximately summed

Rizvi College of Engineering, Bandra, Mumbai. 3

Chapter 2 Neural Network

Figure 2.1: Neural Network

3. When the input exceeds a threshold the neuron sends an electrical spike that travels that travels
from the body, down the axon, to the next neuron(s)
Brains learn
1. Altering strength between neurons
2. Creating/deleting connections

1. Hebbs Postulate (Hebbian Learning)

2. When an axon of cell A is near enough to excite a cell B and repeatedly or persistently takes
part in firing it, some growth process or metabolic change takes place in one or both cells such
that As efficiency, as one of the cells firing B, is increased.

Long Term Potentiation (LTP)

1. Cellular basis for learning and memory
2. LTP is the long-lasting strengthening of the connection between two nerve cells in response to
stimulation
3. Discovered in many regions of the cortex

A neuron ( neurone or nerve cell) is an electrically excitable cell that processes and transmits information
through electrical and chemical signals. These signals between neurons occur via synapses, specialized
connections with other cells. Neurons can connect to each other to form neural networks. Neurons are
the core components of the nervous system, which includes the brain, and spinal cord of the central
nervous system (CNS), and the ganglia of the peripheral nervous system (PNS). Specialized types of
neurons include: sensory neurons which respond to touch, sound, light and all other stimuli affecting
the cells of the sensory organs, that then send signals to the spinal cord and brain; motor neurons that re-
ceive signals from the brain and spinal cord, to cause muscle contractions, and affect glandular outputs,
and interneurons which connect neurons to other neurons within the same region of the brain or spinal
cord, in neural networks. A typical neuron possesses a cell body (soma), dendrites, and an axon. The
term neurite is used to describe either a dendrite or an axon, particularly in its undifferentiated stage.
Dendrites are thin structures that arise from the cell body, often extending for hundreds of micrometres
and branching multiple times, giving rise to a complex dendritic tree. An axon is a special cellular
extension that arises from the cell body at a site called the axon hillock and travels for a distance, as
far as 1 meter in humans or even more in other species. The cell body of a neuron frequently gives

Rizvi College of Engineering, Bandra, Mumbai. 4

Chapter 2 Neural Network

rise to multiple dendrites, but never to more than one axon, although the axon may branch hundreds
of times before it terminates. At the majority of synapses, signals are sent from the axon of one neu-
ron to a dendrite of another. There are, however, many exceptions to these rules: neurons that lack
dendrites, neurons that have no axon, synapses that connect an axon to another axon or a dendrite to
another dendrite, etc. All neurons are electrically excitable, maintaining voltage gradients across their
membranes by means of metabolically driven ion pumps, which combine with ion channels embedded
in the membrane to generate intracellular-versus-extracellular concentration differences of ions such as
sodium, potassium, chloride, and calcium. Changes in the cross-membrane voltage can alter the func-
tion of voltage-dependent ion channels. If the voltage changes by a large enough amount, an all-or-none
electrochemical pulse called anaction potential is generated, which travels rapidly along the cells axon,
and activates synaptic connections with other cells when it arrives. Neurons do not undergo cell division.
In most cases, neurons are generated by special types of stem cells. A type of glial cell, called astrocytes
(named for being somewhat star-shaped), have also been observed to turn into neurons by virtue of the
stem cell characteristic pluripotency. In humans, neurogenesis largely ceases during adulthoodbut in two
brain areas, the hippocampus and olfactory bulb, there is strong evidence for generation of substantial
numbers of new neurons.[
Since the 1960s, database and information technology has been evolving systematically from prim-
itive le pro-cessing systems to sophisticated and powerful databases systems. The research and devel-
opment in database systems since the 1970s has led to the development of relational database systems
, data modelling tools, and indexing and data organization techniques. In addition, users gained conve-
nient and edible data access through query languages, query processing, and user interfaces. E- Client-
methods for on-line transaction processing (OLTP), where a query is viewed as a read-only transaction,
have contributed substantially to the evolution and wide acceptance of relational technology as a major
tool for e-client storage, retrieval, and management of large amounts of data.

2.3 Characteristic of ANN

Basically Computers are good in calculations that basically takes inputs process then and after that gives
the result on the basis of calculations which are done at particular Algorithm which are programmed in
the softwares but ANN improve their own rules, the more decisions they make, the better decisions may
become.The Characteristics are basically those which should be present in intelligent System like robots
and other Artificial Intelligence Based Applications. There are six characteristics of Artificial Neural
Network which are basic and important for this technology which are showed with the help of diagram:-

2.4 Activation Function

Activation Functions are basically the transfer function which is output from a artificial neuron and
it send signals to the other artificial neuron. There are four form of Activation Functions Threshold,
Piecewise Linear, Sigmoid and Gaussian all are different from each other . In Below figures you can see
the Activation function with its demonstration

Rizvi College of Engineering, Bandra, Mumbai. 5

Chapter 2 Neural Network

Figure 2.2: characteristic

Figure 2.3: Threshold

Rizvi College of Engineering, Bandra, Mumbai. 6

Chapter 2 Neural Network

Figure 2.4: Sigmoid

Figure 2.5: Gaussian

Rizvi College of Engineering, Bandra, Mumbai. 7

Chapter 3 Perceptron

Chapter 3

Perceptron

3.1 Introduction
Perceptrons redirects here. For the book of that title, see Perceptrons (book).
In machine learning, the perceptron is an algorithm for supervised classification of an input into one
of several possible non-binary outputs. It is a type of linear classifier, i.e. a classification algorithm that
makes its predictions based on a linear predictor function combining a set of weights with the feature
vector. The algorithm allows for online learning, in that it processes elements in the training set one at a
time.
The perceptron algorithm was invented in 1957 at the Cornell Aeronautical Laboratory by Frank
Rosenblatt.

3.2 Perceptron Learning

Learning algorithms for neural networks A learning algorithm is an adaptive method by which a net-
work of computing units self-organizes to implement the desired behavior. This is done in some learning
algorithms by presenting some examples of the desired input/output mapping to the network. A correc-
tion step is executed iteratively until the network learns to produce the desired response. The learning
algorithm is a closed loop of presentation of examples and of corrections to the network parameters, as
shown in Figure

Figure 3.1: Neural Network

Rizvi College of Engineering, Bandra, Mumbai. 8

Chapter 3 Perceptron

Data mining is process of identify patterns and establish relationships Data Mining defined as
The nontrivial extraction of implicit, previously unknown, and potentially useful information from data.
Data mining is the process of analyzing large amount of data stored is a data warhorse for useful informa-
tion which makes use of artificial intelligence techniques ,neural network ,and advance statistical tools
(such as cluster analysis) to reveal trends, patterns and relationship, which otherwise may be undetected.
[3]
In some simple cases the weights for the computing units can be found through a sequential test of
stochastically generated numerical combinations. However, such algorithms which look blindly for a
solution do not qualify as learning. A learning algorithm must adapt the network parameters according
to previous experience until a solution is found, if it exists.

3.3 Classes of learning algorithms

Learning algorithms can be divided into supervised and unsupervised methods. Supervised learning
denotes a method in which some input vectors are collected and presented to the network. The output
computed by the network is observed and the deviation from the expected answer is measured.
The weights are corrected according to the magnitude of the error in the way defined by the learning
algorithm. This kind of learning is also called learning with a teacher, since a control process knows
the correct answer for the set of selected input vectors. Unsupervised learning is used when, for a given
input, the exact numerical output a network should produce is unknown. Assume, for example, that
some points in two-dimensional space are to be classified into three clusters. For this task we can use
a classifier network with three output lines, one for each class (Figure). Each of the three computing
units at the output must specialize by firing only for inputs corresponding to elements of each cluster. If
one unit fires, the others must keep silent. In this case we do not know a priori which unit is going to
specialize on which cluster. Generally we do not even know how many well-defined clusters are present.
Since no teacher is available, the network must organize itself in order to be able to associate clusters
with units.

Figure 3.2: Three cluster and a classifier network

Supervised learning is further divided into methods which use reinforcement or error correction. Re-
inforcement learning is used when after each presentation of an input-output example we only know
whether the network produces the desired result or not. The weights are updated based on this informa-
tion (that is, the Boolean values true or false) so that only the input

Rizvi College of Engineering, Bandra, Mumbai. 9

Chapter 3 Perceptron

3.4 Learning algorithms for neural networks

vector can be used for weight correction. In learning with error correction, the magnitude of the error,
together with the input vector, determines the magnitude of the corrections to the weights, and in many
cases we try to eliminate the error in a single correction step.

Figure 3.3: Classess of larning algorithm

The perceptron learning algorithm is an example of supervised learning with reinforcement. Some
of its variants use supervised learning with error correction (corrective learning).

3.4.1 Supervised Learning

Supervised learning or Associative learning in which the network is trained by providing it with input
and matching output patterns. These input-output pairs can be provided by an external teacher, or by
the system which contains the neural network (self-supervised). Example: An archaeologist discovers a
human skeleton and has to determine whether it belonged to man or woman. In doing this, the archaeolo-
gist is guided by many past examples of male and female skeletons. Examination of these past examples
(called the training set) allows the archaeologist to learn about the distinctions between male and female
skeletons. This learning process is an example of supervised learning, and the result of learning process
can be applied to determine whether the newly discovered skeleton belongs to man or woman.

3.4.2 Unsupervised Learning

Unsupervised learning or Self-organization in which an (output) unit is trained to respond to clusters of
pattern within the input. In this paradigm the system is supposed to discover statistically salient features
of the input population. Unlike the supervised learning paradigm, there is no a priori set of categories
into which the patterns are to be classified; rather the system must develop its own representation of
the input stimuli. Example: In a different situation, the archaeologist has to determine whether a set
of skeleton fragments belong to the same dinosaur species or need to be differentiated into different
species. For this task, no previous data may be available to clearly identify the species for each skeleton
fragment. The archaeologist has to determine whether the skeletons (that can be reconstructed from
the fragments) are sufficiently similar to belong to the same species, or if the differences between these
skeletons are large enough to warrant grouping them into different species. This is an unsupervised
learning process, which involves estimating the magnitudes of differences between the skeletons. One
archaeologist may believe the skeletons belong to different species, while another may disagree, and
there is no absolute criterion to determine who is correct.

Rizvi College of Engineering, Bandra, Mumbai. 10

Chapter 3 Perceptron

3.4.3 Reinforcement
Reinforcement Learning is type of learning may be considered as an intermediate form of the above
two types of learning. Here the learning machine does some action on the environment and gets a
feedback response from the environment. The learning system grades its action good (rewarding) or bad
(punishable) based on the environmental response and accordingly adjusts its parameters. Generally,
parameter adjustment is continued until an equilibrium state occurs, following which there will be no
more changes in its parameters. The self organizing neural learning may be categorized under this type
of learning.

3.5 Perceptron Training

Rizvi College of Engineering, Bandra, Mumbai. 11

Chapter 4 Feedforward Neural network

Chapter 4

Feedforward Neural network

4.1 Defination
Feedforward neural networks (FF networks) are the most popular and most widely used models in many
practical applications. They are known by many different names, such as multi-layer perceptrons.

4.2 One-hidden-layer Neural network

Below figure illustrates a one-hidden-layer FF network with inputs x1...x2 and output y Each arrow in
the figure symbolizes a parameter in the network. The network is divided into layers. The input layer
consists of just the inputs to the network. Then follows a hidden layer, which consists of any number of
neurons, or hidden units placed in parallel. Each neuron performs a weighted summation of the inputs,
which then passes a nonlinear activation function sigma also called the neuron function.

Mathematically the functionality of a hidden neuron is described by

Rizvi College of Engineering, Bandra, Mumbai. 12

Chapter 4 Feedforward Neural network

The network output is formed by another weighted summation of the outputs of the neurons in the
hidden layer. This summation on the output is called the output layer. In Figure 2.5 there is only one
output in the output layer since it is a single-output problem. Generally, the number of output neurons
equals the number of outputs of the approximation problem. The neurons in the hidden layer of the
network in Figure 2.5 are similar in structure to those of the perceptron, with the exception that their
activation functions can be any differential function. The output of this network is given by

where n is the number of inputs and nh is the number of neurons in the hidden layer.

Rizvi College of Engineering, Bandra, Mumbai. 13

Chapter 5 back-propagation

Chapter 5

back-propagation

5.1 Introduction
The back propagation algorithm (Rumelhart and McClelland, 1986) is used in layered feed-forward
ANNs. This means that the artificial neurons are organized in layers, and send their signals forward, and
then the errors are propagated backwards. The network receives inputs by neurons in the input layer,
and the output of the network is given by the neurons on an output layer. There may be one or more
intermediate hidden layers.

5.2 Back-propagation network

.
The back propagation algorithm uses supervised learning, which means that we provide the algorithm
with examples of the inputs and outputs we want the network to compute, and then the error (difference
between actual and expected results) is calculated. The idea of the back propagation algorithm is to
reduce this error, until the ANN learns the training data. The training begins with random weights,
and the goal is to adjust them so that the error will be minimal. Back propagation network has gained
importance due to the shortcomings of other available networks. The network is a multi layer network
(multi layer perception) that contains at least one hidden layer in addition to input and output layers.
Number of hidden layers & numbers of neurons in each hidden layer is to be fixed based on application,
the complexity of the problem and the number of inputs and outputs. Use of non-linear log-sigmoid
transfer function enables the network to simulate non-linearity in practical systems.Due to this numerous
advantages, back propagation network is chosen for present work. Implementation of back propagation
model consists of two phases. First phase is known as training while the second phase is called Testing.
Training, in back propagation is based on gradient decent rule that tends to adjust weights and reduce
system error in the network. Input layer has neurons equal in number to that of the inputs. Similarly,
output layer neurons are same in the number as number of outputs. Number of hidden layer neurons is
deciding by trial and error method using the experimental data.

5.2.1 ANN Development & Implementation

In this work, both ANN implementation & training is developed, using the neural network toolbox of
Mat Lab. Different ANNs are build rather than using one large ANN including all the output vari-
ables. This strategy allowed for better adjustment of the ANN for each specific problem, including the
optimization of the architecture for each output.

Rizvi College of Engineering, Bandra, Mumbai. 14

Chapter 5 back-propagation

5.2.2 ANN Training & Prediction quality

One of the most relevant aspects of a neural network is its ability to generalize, that is, to predict cases
that are not included in the training set. One of the problems that occur during neural network training
is called over fitting. The error on the training set is driven to a very small value, but when new data is
presented to the network, the error is large. The network has memorized the training examples, but it
has not learned to generalize to new situations. One method for improving network generalization is to
use a network that is just large enough to provide an adequate fit. The larger network you use the more
complex functions the network can create. There are two other methods for improving generalization
that are implemented in Mat Lab Neural Network Toolbox software: regularization & early stopping.
The typical performance function used for training feed forward neural networks is the mean sum of
squares of the network errors.
The weights are corrected according to the magnitude of the error in the way defined by the learning
algorithm. This kind of learning is also called learning with a teacher, since a control process knows
the correct answer for the set of selected input vectors. Unsupervised learning is used when, for a given
input, the exact numerical output a network should produce is unknown. Assume, for example, that
some points in two-dimensional space are to be classified into three clusters. For this task we can use
a classifier network with three output lines, one for each class (Figure). Each of the three computing
units at the output must specialize by firing only for inputs corresponding to elements of each cluster. If
one unit fires, the others must keep silent. In this case we do not know a priori which unit is going to
specialize on which cluster. Generally we do not even know how many well-defined clusters are present.
Since no teacher is available, the network must organize itself in order to be able to associate clusters
with units.

It is possible to improve generalization, if you modify the performance function by adding a term
that consists of the mean of the sum of the squares of the network weights & biases,
msereg = mse +(1-)msw,
Where is the performance ratio, &

Rizvi College of Engineering, Bandra, Mumbai. 15

Chapter 6 Conclusion and Future Scope

Chapter 6

Conclusion and Future Scope

6.1 Recent advances and future applications of NNs include:

Integration of fuzzy logic into neural networks
Fuzzy logic is a type of logic that recognizes more than simple true and false values, hence better
simulating the real world. For example, the statement today is sunny might be 100% true if there are no
clouds, 80% true if there are a few clouds, 50% true if its hazy, and 0% true if rains all day. Hence, it
takes into account concepts like -usually, somewhat, and sometimes. Fuzzy logic and neural networks
have been integrated for uses as diverse as automotive engineering, applicant screening for jobs, the
control of a crane, and the monitoring of glaucoma.
Pulsed neural networks
Most practical applications of artificial neural networks are based on a computational model in-
volving the propagation of continuous variables from one processing unit to the next. In recent years,
data from neurobiological experiments have made it increasingly clear that biological neural networks,
which communicate through pulses, use the timing of the pulses to transmit information and perform
computation. This realization has stimulated significant research on pulsed neural networks, including
theoretical analyses and model development, neurobiological modeling, and hardware implementation.
Hardware specialized for neural networks
Some networks have been hardcoded into chips or analog devices ? this technology will become
more useful as the networks we use become more complex. The primary benefit of directly encoding
neural networks onto chips or specialized analog devices is SPEED! NN hardware currently runs in a
few niche areas, such as those areas where very high performance is required (e.g. high energy physics)
and in embedded applications of simple, hardwired networks (e.g. voice recognition).
Many NNs today use less than 100 neurons and only need occasional training. In these situations,
software simulation is usually found sufficient
When NN algorithms develop to the point where useful things can be done with 1000s of neurons
and 10000s of synapses, high performance NN hardware will become essential for practical operation.
Improvement of existing technologies
All current NN technologies will most likely be vastly improved upon in the future. Everything
from handwriting and speech recognition to stock market prediction will become more sophisticated as
researchers develop better training methods and network architectures.
NNs might, in the future, allow:
robots that can see, feel, and predict the world around them
improved stock prediction
common usage of self-driving cars
composition of music
handwritten documents to be automatically transformed into formatted word processing documents

Rizvi College of Engineering, Bandra, Mumbai. 16

Chapter 6 Conclusion and Future Scope

trends found in the human genome to aid in the understanding of the data compiled by the Human
Genome Project
self-diagnosis of medical problems using neural networks.

6.2 Conclusion
As the ANN is an emerging technology it can be used for data analysis in applications such as pattern
recognition, prediction, system identification & control. From above theories it can be seen that ANN is
a radial basis function back propagation network. The network is capable of predicting the parameters
by experimental system. The network has parallel structure and fast learning capacity. The collected
experimental data such as speed, load, & values of pressure distribution etc. are also employed as
training and testing data for an artificial neural network.
The neural network is a feed forward three layered network. Quick propagation algorithm is used
to update the weight of the network during the training. The ANN has a superior performance to fol-
low the desired results of the system and is employed to analyze such systems parameters in practical
applications.

Rizvi College of Engineering, Bandra, Mumbai. 17

 References

References

[1] Introductin To Artificial Neural Network; A.D.Dongare,R.R Kharde,Amit D.Kachare, International

Journal of Engineering and Innovative Technology (IJEIT), 1,July 2012
[2] A Comprehensive Study of Arificial Neural Network; Vidushi Sharma,Sachin Rai,Anurag Dev, In-
ternational Journal of Advanced Research in Computer Science and Software Engineering, 10,Oct
2012
[3] Genetic reinforcement learning for neural networks,Dominic, S., Das, R., Whitley, D., Ander-
son, C. (July 1991). IJCNN-91-Seattle International Joint Conference on Neural Networks. IJCNN-
91-Seattle International Joint Conference on Neural Networks. Seattle, Washington, USA: IEEE.
doi:10.1109/IJCNN.1991.155315. ISBN 0-7803-0164-1. Retrieved 29 July 2012.
[4] http://en.wikipedia.org/wiki/Artificial neural network

Rizvi College of Engineering, Bandra, Mumbai. 18

 Project Hosting

Appendix A

Project Hosting

The report is shared at Academia.edu. The complete report about the seminar is uploaded here for future
reference.

Report Link : http://www.academia.edu/attachments/6516122/download_file

QR CODE:

Rizvi College of Engineering, Bandra, Mumbai. 19