Data Mining and

Neural Networks

Danny Leung
CS157B, Spring 2006
Professor Sin-Min Lee
Artificial Intelligence
for Data Mining
Neural networks are useful for data mining and
decision-support applications.

People are good at generalizing from experience.

Computers excel at following explicit instructions

over and over.

Neural networks bridge this gap by modeling, on

a computer, the neural behavior of human
brains.

2
Neural Network
Characteristics

Neural networks are useful for pattern

recognition or data classification,
through a learning process.

Neural networks simulate biological

systems, where learning involves
adjustments to the synaptic
connections between neurons
3
Anatomy of a Neural
Network
Neural Networks map a
set of input-nodes to a
set of output-nodes

Number of
inputs/outputs is variable

The Network itself is

composed of an arbitrary
number of nodes with an
arbitrary topology

4
Biological Background
A neuron: many-inputs / one-output unit

Output can be excited or not excited

Incoming signals from other neurons

determine if the neuron shall excite ("fire")

Output subject to attenuation in the

synapses, which are junction parts of the
neuron

5
Basics of a Node
A node is an
element which
performs a
function

y = fH((wixi) +
Connection
Wb)
Node

6
A Simple Preceptron
Binary logic
application

fH(x) [linear
threshold]

Wi = random(-1,1)

Y = u(W0X0 + W1X1
+ Wb)
7
Preceptron Training
Its a single-unit network

Adjust weights based on a how well the current

weights match an objective

Perceptron Learning Rule

Wi = * (D-Y).Ii

= Learning Rate
D = Desired Output

8
Neural Network
Learning
From experience: examples / training
data

Strength of connection between the

neurons is stored as a weight-value for
the specific connection

Learning the solution to a problem =

changing the connection weights
9
Neural Network
Learning
Continuous Learning Process

Evaluate output

Adapt weights

Take new inputs

Learning causes stable state of the

weights
10
Learning Performance
Supervised
Need to be trained ahead of time with lots of data

Unsupervised networks adapt to the input

Applications in Clustering and reducing dimensionality
Learning may be very slow
No help from the outside
No training data, no information available on the
desired output
Learning by doing
Used to pick out structure in the input:
Clustering
Compression

11
Topologies Back-
Propogated Networks
Inputs are put
through a
Hidden Layer
before the output
layer

All nodes
connected
between layers
12
BP Network
Supervised Training
Desired output of the training examples

Error = difference between actual & desired output

Change weight relative to error size

Calculate output layer error , then propagate back to

previous layer

Hidden weights updated

Improved performance
13
Neural Network
Topology
Characteristics
Set of inputs

Set of hidden nodes

Set of outputs

Increasing nodes makes network more

difficult to train
14
Applications of Neural
Networks
Prediction weather, stocks, disease

Classification financial risk assessment, image

processing

Data association Text Recognition (OCR)

Data conceptualization Customer purchasing

habits

Filtering Normalizing telephone signals (static)

15
Overview
Advantages
Adapt to unknown situations
Robustness: fault tolerance due to network
redundancy
Autonomous learning and generalization

Disadvantages
Not exact
Large complexity of the network structure
16
Referenced Work
Intro to Neural Networks - Computer Vision Applications and Training
Techniques. Doug Gray. www.soe.ucsc.edu/~taoswap/
GroupMeeting/NN_Doug_2004_12_1.ppt

Introduction to Artificial Neural Networks. Nicolas Galoppo von

Borries. www.cs.unc.edu/~nico/courses/ comp290-58/nn-
presentation/ann-intro.ppt

17