A Report

on

Simulated Study of Signal Classifiers using Artificial Neural Network

At

DRDO (Defense R&D Organisation)

By:
Aman Gupta
B.Tech (Electronics and Communication Engineering)

Report submitted in Partial fulfillment for the degree of

BACHELOR OF TECHNOLOGY

Ambedkar Institute of Advanced Communication Technologies and Research

Geeta Colony, New Delhi- 110031
2017-2018
 ACKNOWLEDGEMENTS

I wish to express my sincere gratitude to Dr. A.K. Razdan, for providing me an opportunity to
do my internship and project work in LASTEC lab, DRDO on Artificial Intelligence and
Machine Learning.

I sincerely thank Mr. Saurabh Inamdar for his guidance and encouragement in carrying out
this project work. I also wish to express my gratitude to the officials and other star members
of Defence Research and Development Organization who rendered their help during the
period of my project work.

I also would like to thank the Head of Department, Electronics and Communication, of Indira
Gandhi Delhi Technical University for providing me the opportunity to embark on this
project.
 LASTEC LAB

Laser Science & Technology Centre (LASTEC), Delhi has its origin as Defence Science Laboratory
(DSL) established as a nucleus laboratory of DRDO in 1952. In the beginning DSL operated from
National Physical Laboratory (NPL) building. Later in 1960, it was shifted to Metcalfe House. In 1982,
the DSL moved to its new technical building in Metcalfe House complex and was renamed as Defence
Science Centre (DSC). In 1999, in view of the R&D thrust shifting to development of lasers and
optoelectronics systems & related technologies, the laboratory was rechristened as Laser Science &
Technology Center (LASTEC). With time, many of DSL activities were given to newly formed,
specialised DRDO laboratories. DSL has served as a precursor for as many as 15 present DRDO labs,
which include DRDL, SSPL, INMAS, FRL, ISSA, DESIDOC, CFEES, SAG and ITM.
 TABLE OF CONTENTS

1. Artificial neural networks in AI - an introduction

2. Problems faced during classification
3. Machine Learning
4. Machine Learning Techniques
5. MATLAB Learning
6. Problem
1.Electrocardiogram
2.Trigonometric functions
7. Conclusion
 Artificial neural networks

Artificial Neural Networks (ANNs) are adaptive nonlinear information processing systems which
combine numerous processing units with self-adapting, self-organizing and real-time learning
characteristics, among others. Extensive research on ANNs has made remarkable developments, since
its inception in the 1980s, and ANNs have been widely applied in the field of artificial intelligence.
Despite the progress in research and development in this field, many problems have been encountered
as well, for instance, the selection of the structure and the parameters of the networks, the selection of
the learning samples, the selection of the initial values, and the convergence of the learning algorithms.
It is known that the performance of neural networks is sensitive to the number of neurons: too few
neurons can result in poor approximation, while too many neurons may contribute to overfitting
problems. Obviously, there seems to be an inverse correlation between achieving a better network
performance and simplifying the network topology used for optimizing the design and the parameters
of ANNs.

Techniques such as fuzzy technology, neural networks, and evolutionary algorithms have resulted in a
stage of rapid growth for computational intelligence. With the advent of AI, these methods have already
achieved huge progress and exhibit a high degree of convergence. They can reinforce each other by
their mutual complementation; thus gaining more powerful abilities of representing and solving
practical problems. ANNs are, albeit currently theoretical, results of applying biological principles to
scientific research. In recent years, more and more researchers have tried to employ ANNs in the hope
that by combining the advantages of inherent methods, they can create a more eAcient method. This is
mainly manifested in using EAs to optimize network design, pre-process the network input data, and in
the assembly of ANNs.

The advantages of ANNs are mainly represented by the network architecture and the algorithms.
Currently, the research on neural networks is mainly focused on these two aspects. An ANN consists
of a set of processing units which also called neurons, and they are connected to each other. Each neuron
can be described as an oriented graph, or as a transfer function. A neuron is usually a multi-input and
single-output nonlinear element. The architecture of a neural network is determined by all the
connections of the network and the transfer functions of the neurons.

The learning process in ANNs, also called ‘training’, is usually implemented by examples. The learning
process is achieved by iteratively adjusting the connection weights. This learning process of ANNs can
be divided into three types: supervised, unsupervised and reinforcement learning.
Supervised learning is based on the direct comparison between the actual output and the expected
output. The optimization algorithms are based on a gradient-based descent, because of which they can
be used to iteratively adjust the connection weights and thereby minimize errors. Reinforcement
learning is a special case of supervised learning; it is only based on whether the actual output is correct.
Unsupervised learning is only based on the correlation of the input data.

The essence of a learning algorithm is knowing that the performance of neural networks is sensitive to
the number of neurons. Too few neurons can result in poor approximation, while too many neurons
may contribute to overfitting problems. Obviously, it is contradictory between achieving a better
network performance and simplifying the network topology. This provides new ideas and methods for
problem solving approach in terms of its special natural evolution rules and the superiority of population
optimizing search.

ANNs have the advantage of good global searching capability and learning the approximate optimal
solution without the gradient information of the error functions. New algorithms provide ideas and
methods for problem solving. In recent years, more and more researchers working on ANNs hope that
by combining all these advantages, they can create a more eAcient method by optimizing the network
design, pre-processing of the network input data and the assembly of ANNs.
 Problems and issues faced during classification

The major challenges in classification is the selection of proper attributes and parameters based on the
datasets that would be classified, and also after a huge number of calculations and selection of
parameters the accuracy in classification is not achieved, hence we are proceeding with the supervised
learning techniques like support vector machine, decision tree algorithm and clustering techniques, so
that we could provide a novel system that can classify the data more properly and accurately.

Inductive machine learning is the process of learning a set of rules from instances (examples in a
training set), or more generally speaking, creating a classifier that can be used to generalize from new
instances.

The first step in this process is collection of the dataset, wherein an expert picks and chooses the fields
(attributes, features) that are the most informative. If this is not feasible, then the alternative method is
that of “brute-force,” which means measuring everything available in the hope that the right
(informative and relevant) features can be isolated. However, a dataset collected by the “brute-force”
method is not directly suitable for induction. It contains, in most cases, noise and missing feature values,
and therefore requires a subsequent step of significant pre-processing.

The second step is the data preparation and data preprocessing. Depending on the circumstances,
researchers have a number of methods to choose from to handle missing data using contemporary
techniques for outlier (noise) detection.

Instance selection in these datasets is an optimization problem that attempts to maintain the mining
quality while minimizing the sample size. It reduces data and enables a data mining algorithm to
function and work effectively with very large datasets. Thus, instance selection is not only used to
handle noise but also to cope with the infeasibility of learning from very large datasets. There are several
procedures for sampling instances from a large dataset.

Feature subset selection is the process of identifying and removing as many irrelevant and redundant
features as possible. This reduces the dimensionality of the data and enables data mining algorithms to
operate faster and more effectively. The fact that many features depend on one another often unduly
influences the accuracy of supervised ML classification models.
Feature construction/transformation addresses this problem constructing new features from the basic
feature set. These newly-generated features may lead to the creation of more concise and accurate
classifiers. In addition, the discovery of meaningful features contributes to better comprehensibility of
the produced classifier, and a better understanding of the learned concept.

Algorithm selection:

The choice of which specific learning algorithm we should use is a critical step. Once preliminary
testing is judged to be satisfactory, the classifier (mapping from unlabeled instances to classes) is
available for routine use. The classifier’s evaluation is most often based on prediction accuracy (the
percentage of correct prediction divided by the total number of predictions).

There are at least three techniques which are used to calculate a classifier’s accuracy. One technique is
to split the training set by using two-thirds for training and the other third for estimating performance.
In another technique, known as cross-validation, the training set is divided into mutually exclusive and
equal-sized subsets and for each subset the classifier is trained on the union of all the other subsets. The
average of the error rate of each subset is therefore an estimate of the error rate of the classifier.

The leave-one-out validation is a special case of cross validation. All test subsets consist of a single
instance. This type of validation is, of course, more expensive computationally, but useful when the
most accurate estimate of a classifier’s error rate is required. If the error rate evaluation is unsatisfactory,
we must return to a previous stage of the supervised ML process. A variety of factors must be examined:
perhaps relevant features for the problem are not being used, a larger training set is needed, the
dimensionality of the problem is too high, the selected algorithm is inappropriate or parameter tuning
is needed. Another problem could be that the dataset is imbalanced. A common method for comparing
supervised ML algorithms is to perform statistical comparisons of the accuracies of trained classifiers
on specific datasets. If we have sufficient supply of data, we can sample a number of training sets of
size N, run the two learning algorithms on each of them, and estimate the difference in accuracy for
each pair of classifiers on a large test set. The average of these differences is an estimate of the expected
difference in generalization error across all possible training sets of size N, and their variance is an
estimate of the variance of the classifier in the total set.
Our next step is to perform paired t-test to check the null hypothesis that the mean difference between
the classifiers is zero. This test can produce two types of errors:

1. Type I error is the probability that the test rejects the null hypothesis incorrectly (i.e. it finds a
“significant” difference although there is none).

2. Type II error is the probability that the null hypothesis is not rejected, when there actually is a
difference.

The test’s Type I error will be close to the chosen significance level. In practice, however, we often have
only one dataset of size N and all estimates must be obtained from this sole dataset. Difference training
sets are obtained by subsampling, and the instances not sampled for training are used for testing.
Unfortunately this violates the independence assumption necessary for proper significance testing. The
consequence of this is that Type I errors exceed the significance level. This is problematic because it is
important for the researcher to be able to control Type I errors and know the probability of incorrectly
rejecting the null hypothesis. Several heuristic versions of the t-test have been developed to alleviate
this problem. Ideally, we would like the test’s outcome to be independent of the particular partitioning
resulting from the randomization process, because this would make it much easier to replicate
experimental results published in the literature. However, in practice there is always certain sensitivity
to the partitioning used. To measure replicability we need to repeat the same test several times on the
same data with different random partitionings — usually ten repetitions— and count how often the
outcome is the same. Supervised classification is one of the tasks most frequently carried out by so-
called Intelligent Systems. Thus, a large number of techniques have been developed based on Artificial
Intelligence (Logical/Symbolic techniques), Perceptron-based techniques and Statistics (Bayesian
Networks, Instance-based techniques).
 Machine Learning

Machine learning is the ability of machines to learn; this is the process of a machine is building up using
certain algorithms through which it can take its own decisions and provide the result to the user.
Basically it is considered the subfield of artificial intelligence. Today machine learning is used for
complex data classification and decision making. In simple terms it is the development of algorithms
that enables the system to learn, and to make necessary decisions. It has strong ties to mathematical
optimization that delivers methods, theory and application domain to the field and, it is employed in a
range of computing tasks where designing and programming explicit algorithms is infeasible. Certain
examples of the applications of ML are in spam filtering, optical character recognition (OCR), search
engines and computer vision. Machine learning methods and tasks are broadly divided into three
categories as follows.

• Supervised learning
• Unsupervised learning
• Reinforcement learning

A. Supervised learning: Here, the system is provided with a sample inputs and it is mapped with the
output. In this type of learning, each example is a pair consisting of an input object (basically a vector)
and a desired output value (supervisory signal). A supervised learning algorithm analyses and studies
the training data and produces an inferred function that can be used for mapping new examples. The
optimal scenario will allow the algorithm to correctly determine the class labels for unseen instances.
It is required by the learning algorithm to generalize from the training data to unseen situations in a
“reasonable” way. Approaches for supervised learning include support vector machines, decision trees,
etc.

B. Unsupervised learning: In this type of learning the system is provided with some sample inputs but
there is no any output present. Since there is no desired output over here categorization is done so that
the algorithm differentiates correctly between the data sets. It is a task of defining a function to describe
hidden structure from unlabelled data. Since samples or training sets given to the learner are unlabelled,
there is no error to reward signal to evaluate a potential solution. In this way unsupervised learning
differs from supervised learning and reinforcement learning. It is closely related to the problem of
density estimation and statistics.
C. Reinforcement learning: This is a sub domain of machine learning inspired by behaviourist
psychology, dealing with how software agents ought to take actions in an environment so as to
maximize some notion of cumulative reward. It is studied and used in many theories like game theory,
control theory, operations research, information theory, swarm intelligence, statics and genetic
algorithms. This paper is majorly focusing on studding difference algorithms of machine learning which
would help system to accurately classify the data and would enable it for decision making in complex
situations. The two main algorithms to study are decision tree algorithm and support vector machines.

There are several applications for Machine Learning (ML), the most significant of which is data mining.
People are often prone to making mistakes during analyses or, possibly, when trying to establish
relationships between multiple features. This makes it difficult for them to find solutions to certain
problems. Machine learning can often be successfully applied to these problems, improving the
efficiency of systems and the designs of machines. Every instance in any dataset used by machine
learning algorithms is represented using the same set of features. The features may be continuous,
categorical or binary. If instances are given with known labels (the corresponding correct outputs) then
the learning is called supervised, in contrast to unsupervised learning, where instances are unlabeled.
By applying these unsupervised (clustering) algorithms, researchers hope to discover unknown, but
useful, classes of items. Another kind of machine learning is reinforcement learning. The training
information provided to the learning system by the environment (external trainer) is in the form of a
scalar reinforcement signal that constitutes a measure of how well the system operates. The learner is
not told which actions to take, but rather must discover which actions yield the best reward, by trying
each action in turn. Numerous ML applications involve tasks that can be set up as supervised.
 Machine learning techniques

1. Pattern classification

Pattern recognition is the action to take raw data and activity on data category (Michalski, Bratko, &
Kubat, 1998). The methods of supervised and unsupervised learning can be used to solve difference
pattern recognition problems (Theodoridis & Koutroumbas, 2006, 2006). In supervised learning, it is
based on using the training data to create a function, in which each of the training data contains a pair
of the input vector and output (i.e. the class label). The learning (training) task is to compute the
approximate distance between the input–output examples to create a classifier (model). When the model
is created, it can classify unknown examples into a learned class labels.

2. Single classifiers

The intrusion detection problem can be approached by using one single machine learning algorithm. In
literature, machine learning techniques (e.g. k-nearest neighbor, support vector machines, artificial
neural network, decision trees, self-organizing maps, etc.) have been used to solve these problems.

2.1. K-nearest neighbor (k-NN) is one of the most simple and traditional nonparametric technique to
classify samples (Bishop, 1995; Manocha & Girolami, 2007). It computes the approximate distances
between difference points on the input vectors, and then assigns the unlabeled point to the class of its
K-nearest neighbors. In the process of create k-NN classifier, k is an important parameter and difference
k values will cause difference performances. If k is considerably huge, the neighbors which used for
prediction will make large classification time and influence the accuracy of prediction. k-NN is called
instance based learning, and it is difference from the inductive learning approach (Mitchell, 1997).
Thus, it does not contain the model training stage, but only searches the examples of input vectors and
classifies new instances. Therefore, k-NN ‘‘on-line” trains the examples and finds out k-nearest
neighbor of the new instance.

2.2. Support vector machines (SVM) were first proposed by Vapnik (1998). SVM first maps the input
vector into a higher dimensional feature space and then obtain the optimal separating hyper-plane in
the higher dimensional feature space. Moreover, a decision boundary, i.e. the separating hyper-plane,
is determined by support vectors rather than the whole training samples and thus is extremely robust to
outliers. In particular, an SVM classifier is designed for binary classification. That is, to separate a set
of training vectors which belong to two difference classes. Note that the support vectors are the
training samples close to a decision boundary. The SVM also provides a user specified parameter called
penalty factor. It allows users to make a tradeoR between the number of misclassified samples and the
width of a decision boundary.

3. Artificial neural networks The neural network is information processing units which to mimic the
neurons of human brain (Haykin, 1999). Multilayer perceptron (MLP) is the widely used neural network
architecture in many pattern recognition problems. A MLP network consists of an input layer including
a set of sensory nodes as input nodes, one or more hidden layers of computation nodes, and an output
layer of computation nodes. Each interconnection has associated with it a scalar weight which is
adjusted during the training phase. In addition, the backpropagation learning algorithm is usually used
to train a MLP, which are also called as backpropagation neural networks. First of all, random weights
are given at the beginning of training. Then, the algorithm performs weights tuning to define whatever
hidden unit representation is most effective at minimizing the error of misclassification.

4. Self-organizing maps (SOM) (Kohonen, 1982) is trained by an unsupervised competitive learning

algorithm, a process of self organization. The aim of SOM is to reduce the dimension of data
visualization. That is, SOM projects and clusters high-dimensional input vectors onto a low-
dimensional visualized map, usually 2 for visualization. It usually consists of an input layer and the
Kohonen layer which is designed as two-dimensional arrangement of neurons that maps n dimensional
input to two dimensions. Kohonen’s SOM associates each of the input vectors to a representative output.
The network finds the node closest to each training case and moves the winning node, which is the
closest neuron (i.e. the neuron with minimum distance) to the training case. That is, SOM maps similar
input vectors onto the same or similar output units on such a two-dimensional map. Therefore, output
units will self-organize to an ordered map and those output units with similar weights are also placed
nearby after training.

5. Decision trees classify a sample through a sequence of decisions, in which the current decision helps
to make the subsequent decision. Such a sequence of decisions is represented in a tree structure. The
classification of a sample proceeds from the root node to a suitable end leaf node, where each end leaf
node represents a classification category. The attributes of the samples are assigned to each node, and
the value of each branch is corresponding to the attributes (Mitchell, 1997). A well-known program for
constructing decision trees is CART (Classification and Regressing Tree) (Breiman, Friedman, Olshen,
& Stone, 1984). A decision tree with a range of discrete (symbolic) class labels is
called a classification tree, whereas a decision tree with a range of continuous (numeric) values is called
a regression tree.

6. Naïve bayes networks There are many cases where we know the statistical dependencies or the causal
relationships between system variables. However, it might be difficult to precisely express the
probabilistic relationships among these variables. In other words, the prior knowledge about the system
is simply that some variable might influence others. To exploit this structural relationship or casual
dependencies between the random variables of a problem, one can use a probabilistic graph model
called Naïve Baysian Networks (NB). The model provides an answer to questions like ‘‘What is the
probability that it is a certain type of attack, given some observed system events?” by using conditional
probability formula. The structure of a NB is typically represented by a directed acyclic graph (DAG),
where each node represents one of system variables and each link encodes the influence of one node
upon another (Pearl, 1988). Thus, if there is a link from node A to node B, A directly influences B.

7. Genetic algorithms (GA) use the computer to implement the natural selection and evolution (Koza,
1992). This concept comes from the ‘‘adaptive survival in natural organisms”. The algorithm C.-F. Tsai
et al. / Expert Systems with Applications 36 (2009) 11994–12000 11995 starts by randomly generating
a large population of candidate programs. Some type of fitness measure to evaluate the performance of
each individual in a population is used. A large number of iterations is then performed that low
performing programs are replaced by genetic recombinations of high performing programs. That is, a
program with a low fitness measure is deleted and does not survive for the next computer iteration.

8. Fuzzy logic (or fuzzy set theory) is based on the concept of the fuzzy phenomenon to occur
frequently in real world. Fuzzy set theory considers the set membership values for reasoning and the
values range between 0 and 1. That is, in fuzzy logic the degree of truth of a statement can range
between 0 and 1 and it is not constrained to the two truth values (i.e. true, false). For examples, ‘‘rain”
is a commonly natural phenomenon, and it may have very fierce change. Raining may be able to convert
the circumstances from slight to violent (Zimmermann, 2001).

9. Hybrid classifiers In the development of an IDS, the ultimate goal is to achieve the best possible
accuracy for the task at hand. This objective naturally leads to the design of hybrid approaches for the
problem to be solved. The idea behind a hybrid classifier is to combine several machine learning
techniques so that the system performance can be significantly improved. More specifically, a hybrid
approach typically consists of two functional components. The first one takes raw data as input and
generates intermediate results. The second one will then take the intermediate results as the input and
produce the final results (Jang, Sun, & Mizutani, 1996). In particular, hybrid classifiers can be based
on cascading difference classifiers, such as neuro-fuzzy techniques. On the other hand, hybrid classifiers
can use some clustering-based approach to preprocess the input samples in order to eliminate
unrepresentative training examples from each class. Then, the clustering results are used as training
examples for classifier design. Therefore, the first level of hybrid classifiers can be based on either
supervised or unsupervised learning techniques. Finally, hybrid classifiers can also be based on the
integration of two difference techniques in which the first one aims at optimizing the learning
performance (i.e. parameter tuning) of the second model for prediction.

10. Ensemble classifiers Ensemble classifiers were proposed to improve the classification performance
of a single classifier (Kittler, Hatef, Duin, & Matas, 1998). The term ‘‘ensemble” refers to the
combination of multiple weak learning algorithms or weak learners. The weak learners are trained on
difference training samples so that the overall performance can be effectively improved. Among the
strategies for combining weak learners, the ‘‘majority vote” is arguably the most commonly used one
in the literature. Other combination methods, such as boosting and bagging, are based on training data
resampling and then taking a majority vote of the resulting weak learners.
 MATLAB Learning

Neural Network Toolbox provides functions and apps for modeling complex nonlinear systems that are
not easily modeled with a closed-form equation. Neural Network Toolbox supports supervised learning
with feedforward, radial basis, and dynamic networks. It also supports unsupervised learning with self-
organizing maps and competitive layers. With the toolbox you can design, train, visualize, and simulate
neural networks. You can use Neural Network Toolbox for applications such as data fitting, pattern
recognition, clustering, time-series prediction, and dynamic system modeling and control.
To speed up training and handle large data sets, you can distribute computations and data across
multicore processors, GPUs, and computer clusters using Parallel Computing Toolbox™.

Key features:

1. Supervised networks, including multilayer, radial basis, learning vector quantization (LVQ),
time-delay, nonlinear autoregressive (NARX), and layer-recurrent
2. Unsupervised networks, including self-organizing maps and competitive layers
3. Apps for data-fitting, pattern recognition, and clustering
4. Parallel computing and GPU support for accelerating training (using Parallel Computing
Toolbox)
5. Preprocessing and postprocessing for improving the efficiency of network training and
assessing network performance
6. Modular network representation for managing and visualizing networks of arbitrary size
7. Simulink blocks for building and evaluating neural networks and for control systems
applications

Neural networks are composed of simple elements operating in parallel. These elements are inspired by
biological nervous systems. As in nature, the connections between elements largely determine the
network function. You can train a neural network to perform a particular function by adjusting the
values of the connections (weights) between elements.
Typically, neural networks are adjusted, or trained, so that a particular input leads to a specific target
output. The next figure illustrates such a situation. Here, the network is adjusted, based on a comparison
of the output and the target, until the network output matches the target. Typically, many such
input/target pairs are needed to train a network.
Neural networks have been trained to perform complex functions in various fields, including pattern
recognition, identification, classification, speech, vision, and control systems.
Neural networks can also be trained to solve problems that are difficult for conventional computers or
human beings. The toolbox emphasizes the use of neural network paradigms that build up to—or are
themselves used in— engineering, financial, and other practical applications.
The following topics explain how to use graphical tools for training neural networks to solve problems
in function fitting, pattern recognition, clustering, and time series. Using these tools can give you an
excellent introduction to the use of the Neural Network Toolbox™ software:
Fit Data with a Neural Network
Classify Patterns with a Neural Network
Cluster Data with a Self-Organizing Map
Neural Network Time Series Prediction and Modeling

Workflow for Neural Network Design:

The work flow for the neural network design process has seven primary steps. Referenced topics discuss
the basic ideas behind steps 2, 3, and 5.
1. Collect data
2. Create the network — Create Neural Network Object
3. Configure the network — Configure Neural Network Inputs and Outputs
4. Initialize the weights and biases
5. Train the network — Neural Network Training Concepts
6. Validate the network
7. Use the network
Data collection in step 1 generally occurs outside the framework of Neural Network Toolbox™
software, but it is discussed in general terms in Multilayer Neural Networks and Backpropagation
Training. Details of the other steps and discussions of steps 4, 6, and 7, are discussed in topics specific
to the type of network.
The Neural Network Toolbox software uses the network object to store all of the information that
defines a neural network. This topic describes the basic components of a neural network and shows
how they are created and stored in the network object.
After a neural network has been created, it needs to be configured and then trained. Configuration
involves arranging the network so that it is compatible with the problem you want to solve, as defined
by sample data. After the network has been configured, the adjustable network parameters (called
weights and biases) need to be tuned, so that the network performance is optimized. This tuning process
is referred to as training the network. Configuration and training require that the network be provided
with example data. This topic shows how to format the data for presentation to the network. It also
explains network configuration and the two forms of network training: incremental training and batch
training.

Four Levels of Neural Network Design:

There are four difference levels at which the Neural Network Toolbox™ software can be used. The first
level is represented by the GUIs that are described in Getting Started with Neural Network Toolbox.
These provide a quick way to access the power of the toolbox for many problems of function fitting,
pattern recognition, clustering and time series analysis.
The second level of toolbox use is through basic command-line operations. The command-line
functions use simple argument lists with intelligent default settings for function parameters. (You can
override all of the default settings, for increased functionality.) This topic, and the ones that follow,
concentrate on command-line operations.
The GUIs described in Getting Started can automatically generate MATLAB code files with the
command-line implementation of the GUI operations. This provides a nice introduction to the use of
the command-line functionality.
A third level of toolbox use is customization of the toolbox. This advanced capability allows you to
create your own custom neural networks, while still having access to the full functionality of the
toolbox.
The fourth level of toolbox usage is the ability to modify any of the code files contained in the toolbox.
Every computational component is written in MATLAB code and is fully accessible.
The first level of toolbox use (through the GUIs) is described in Getting Started which also introduces
command-line operations. The following topics will discuss the command-line operations in more
detail.

Multiple Layers of Neurons:

A network can have several layers. Each layer has a weight matrix W, a bias vector b, and an output
vector a. To distinguish between the weight matrices, output vectors, etc., for each of these layers in the
figures, the number of the layer is appended as a superscript to the variable of interest. You can see the
use of this layer notation in the three-layer network shown next, and in the equations at the bottom of
the figure.
The network shown above has R1 inputs, S1 neurons in the first layer, S2 neurons in the second layer,
etc. It is common for difference layers to have difference numbers of neurons. A constant input 1 is fed
to the bias for each neuron.
Note that the outputs of each intermediate layer are the inputs to the following layer. Thus layer 2 can
be analyzed as a one-layer network with S1 inputs, S2 neurons, and an S2 × S1weight matrix W2. The
input to layer 2 is a1; the output is a2. Now that all the vectors and matrices of layer 2 have been
identified, it can be treated as a single-layer network on its own. This approach can be taken with any
layer of the network.
The layers of a multilayer network play difference roles. A layer that produces the network output is
called an output layer. All other layers are called hidden layers. The three-layer network shown earlier
has one output layer (layer 3) and two hidden layers (layer 1 and layer 2). Some authors refer to the
inputs as a fourth layer. This toolbox does not use that designation.
The architecture of a multilayer network with a single input vector can be specified with the notation R
− S1 − S2 −...− SM, where the number of elements of the input vector and the number of neurons in
each layer are specified.
The same three-layer network can also be drawn using abbreviated notation.

Multiple-layer networks are quite powerful. For instance, a network of two layers, where the first layer
is sigmoid and the second layer is linear, can be trained to approximate any function (with a finite
number of discontinuities) arbitrarily well. This kind of two-layer network is used extensively in
Multilayer Neural Networks and Backpropagation Training.
Here it is assumed that the output of the third layer, a3, is the network output of interest, and this
output is labeled as y. This notation is used to specify the output of multilayer networks.

Custom Neural Network Helper Functions:

The toolbox allows you to create and use your own custom functions. This gives you a great deal of
control over the algorithms used to initialize, simulate, and train your networks.
Be aware, however, that custom functions may need updating to remain compatible with future
versions of the software. Backward compatibility of custom functions cannot be guaranteed.
Template functions are available for you to copy, rename and customize, to create your own versions
of these kinds of functions. You can see the list of all template functions by typing the following:
help nncustom

Each template is a simple version of a difference type of function that you can use with your own
custom networks.
For instance, make a copy of the file tansig.m with the new name mytransfer.m. Start editing the new
file by changing the function name a the top from tansig to mytransfer.
You can now edit each of the sections of code that make up a transfer function, using the help
comments in each of those sections to guide you.
Once you are done, store the new function in your working folder, and assign the name of your
transfer function to the transferFcn property of any layer of any network object to put it to use.
 Problems

Electrocardiogram (ECG) feature extraction

The designed ANN model contained approaches the neural network adaptive potential approach and
Cascade feedforward backpropagation method is used as an optimization method. The
Electrocardiogram (ECG) dynamic and nonlinear signal characteristic requires an accurate and precise
detection and recognition system.
Electrocardiogram (ECG) represents the electrical activity of the heart. When the ECG is abnormal, it
is called arrhythmia. ECG is one of the most commonly used tests to diagnose the heart problem.
Detection and treatment of arrhythmias have become one of the cardiac care unit’s major functions. Few
of the arrhythmias are Ventricular Premature Beats, a systole, Couplet, Bigeminy, Fusion beats.
Artificial Neural Networks (ANN’s) have often been proposed as tools for realizing classifiers that are
able to deal even with nonlinear discrimination between classes and to accept incomplete or ambiguous
input patterns.The interpretation may vary by physician to physician.
Hence this work is all about the automation and consistency in the analysis of the ECG signals so that
they must be diagnosed and interpreted accurately irrespective of the physicians. Any disorder of heart
rate or rhythm, or change in the morphological pattern, is an indication of an arrhythmia, which could
be detected by analysis of the recorded ECG waveform. A typical cycle of an ECG is shown in Figure.
Physicians first locate such fiduciary points as Q points, R points, and S points in the ECG from which
they locate the P-complexes, QRS-waves, T-complexes, and U-waves in the ECG. These waves and
complexes are defined in Figure. Physicians then interpret the shapes of those waves and complexes.
They calculate parameters to determine whether the ECG shows signs of cardiac disease or not. The
parameters are the height and the interval of each wave, such as RR interval, PP interval, QT interval,
and ST segment.
However, the MIT-BHI database doesn’t have a label as to what type or arrhythmia the sample has due
to which we couldn’t develop a neural network for the type of arrhythmia for types of samples.
The feature extraction procedure involved R peak detection using fourier transform and further
calculating the heart rate using the frequency or the time between each consecutive peaks.
Trigonometric/Circular function Classifier using Artificial Neural Network

Using the Neural Network toolbox in MATLAB, we were able to successfully train the Network to
classify all types of real world Trigonometric signals using the generated samples.
The mean square error is shown in the performance plot of the toolbox. The number of hidden neurons
can be changed according to the performance curve which demonstrates the number of epochs and the
best validation performance. The lesser the mean square error is, the better the result of the network is.
Shown is the difference number of hidden neurons and their best validation performance:

Hidden Neurons Best Validation Epochs Number of iterations

Performance to train the network

2 0.0015027 87 93

5 0.0067844 60 66

7 0.00026659 98 104

10 0.0012988 54 60

15 0.0022097 67 73

As seen in the above table, when the number of hidden neurons are 10, the best validation
performance graph and the confusion plot is:
CONCLUSION

Machine learning is the ability of machines to learn, this is the process of a machine is building up using
certain algorithms through which it can take its own decisions and provide the result to the user.
Basically it is considered the subfield of artificial intelligence. Today machine learning is used for
complex data classification and decision making. The network is trained by feeding it sample features
and the corresponding training vectors. Then the network is trained using Artificial Neural Network
toolbox in MATLAB.
ECG (Electrocardiogram) is used for the diagnosis of heart disease. Various supervised and
unsupervised artificial neural network models using the feature extraction methods. Such a method was
designed by the team using threshold and filtering methods. These features were used to calculate the
heart rate of the samples.
Using the MATLAB ANN toolbox, a neural network was developed using the trigonometric function
samples