Neuron Model and

Network Architecture

sSingle-Input Neuron
Inp
uts

\_____J \_______L

a = f(wp +
b)

Transfer Functions

Function

-b/w

-1

a=
hardlim
(n)

a = hardlim
(wp + b)
Single-Input

hardlim
Neuron

Hard Limit Transfer

a

l +b

-b/w
0

+1

Transfer Functions

/
a=
purelin
(n)

Linear Transfer
Function

a = purelin
*+1

(wp + b)
Single-Input

purelin

-1

Neuron
a

*+
1
n

-b/w

0
-1

a = logsig(n)
P

a = logsig (wp + b)
Log-Sigmoid Transfer Function SingleInput lo si Neur
gg

Multiple-Input Neuron

Inputs Multiple-Input Neuron Input Multiple-Input Neuron

t s\
Rp^[WK,

1x
1

1x R ^

1
\_____J \_______1

a = f(Wp + b)

Abreviated Notatio

a = f(Wp + b)

1x
1

nLayer

of Neurons

Inputs Layer of S Neurons

nL -------- aL

a = f(Wp+b)

W=

1, 1 w1, 2 w1, R w 2, 1 w 2, 2

2, R

Input Layer of S Neurons

Abbreviated Notation
TT i+H-V.
SxR
^

S x1

!"SXT*

p1

P2

b=

a=

=
p

a = f(Wp+b)

a
Sx

S, 1 wS, 2 wS, R

Multilayer Network

Second Layer
Inputs First Layer

------------------------------- -------------------------------------------------

"A r

a 1 = f i (W i p+b 1 )

Third Layer
a 2 , w 3 i ii------------1 n 3 i

a 2 = f (W 2 a i + b 2 )
2

a 3 = f 3 (W 3 f 2 (W 2 f 1 (W i p+b i) +b 2) +b 3)

a 3 = f (W 3 a 2 +b 3 )
3

Abreviated Notation

Hidden Layers
Output Layer
Third Layer
Input

First Layer

Second Layer

re.
;S

S1 x R 1

->Er
R

s>

n
1

S1 x 1

f
S1

S1 x 1

S1 x 1

S2 x 1

Jv

S2 x S1

+0^
2

S x1

f2

3
2
S>
S2 xS1 x S 1

S3 x 11

[jES
JK*
V

S2

S x1

a 1 = f 1 (W 1 p+b 1 )
a 3 = f 3 (W 3 a 2 +b 3 )

a 2 = f 2 (W^ 1 + b 2 )

a 3 = f 3 (W 3 f 2 (W 2 f 1 (W 1 p + b 1 ) + b 2 ) + b 3

Recurrent Network
Sym. Sat. Linear Layer
+ 1)

Initial
Condition
f------------ \
s

SxB

a (t + 1)

J+h^ ,-rf^

S
S

x1

a (0) = p a (t + 1) = satlin (Wa (t)+b)

a ( 1 ) = s a t l i n s ( Wa ( 0 ) + b )
= s a t l i n s ( W p + b ) a (2) =

x1

a (t )

x1

satlins( Wa( 1) + b)