Académique Documents
Professionnel Documents
Culture Documents
ty
,w
ab
or
.m
ha
kr
yr
ea
de
rs
.in
fo
Return to Website
Associative Memory
Soft Computing
Associative Memory (AM), topics : Description, content terms - encoding address-
ability, working, classes of AM : auto and hetero, AM related or memorization, retrieval or recollection, errors and noise, performance measure - memory capacity and contentaddressability. Associative memory models : network architectures linear associator, Hopfield model and bi-directional model (BAM). Auto-associative memory (auto-correlators) : how to store patterns ? how to retrieve patterns? recognition of noisy patterns. Bi-directional hetero-associative memory (hetero-correlators) : BAM operations retrieve the nearest pair, addition and deletion of pattern pairs, energy function for BAM - working of Kosko's BAM, incorrect recall of pattern, multiple training encoding strategy augmentation matrix, generalized correlation matrix and algorithm.
ea
de
rs
.in
fo
,w
.m
Associative Memory
Soft Computing
Topics
(Lectures 21, 22, 23, 24 4 hours) Slides 03-12
ha
kr
ab
or
ty
yr
Content addressability; Working of AM; AM Classes : auto and hetero; AM related terms - encoding or memorization, retrieval or recollection, errors and noise; Performance measure - memory capacity and content-addressability.
2. Associative Memory Models
13-20
Network architectures -
BAM operations - retrieve the nearest pair, Addition and deletion of pattern pairs; Energy function for BAM - working of Kosko's augmentation matrix, generalized correlation matrix and algorithm .
5. References
02
BAM,
42
yr
ea
de
rs
.in
fo
or
ty
ha
kr
ab
,w
Associative Memory
w C
.m
memory :
auto-associative
and
to
implement
these
associative
memory
de
rs
.in
fo
or
ty
,w
.m
1. Associative Memory An associative memory is a content-addressable structure that maps a set of input patterns to a set of output patterns. structure refers to a memory organization where the by its content as opposed to an explicit address and hetero-associative.
An auto-associative memory
yr
ea
SC - AM description
kr
ab
ha
from the input pattern not only in content but possibly also in type and format.
04
de
rs
.in
fo
.m
1.1 Description of Associative Memory An associative memory is a content-addressable structure that allows,
or
ty
,w
yr
ea
SC - AM description
ha
kr
the recall of data, based on the degree of similarity between the input pattern and the patterns stored in memory.
ab
Alex Graham Bell Thomas Edison Christopher Columbus Albert Einstein Crhistpher Columbos Charles Darwin Blaise Pascal Marco Polo Neil Armstrong Sigmund Freud Christopher Columbus
Fig.
Note : An associative memory is accessed by its content, opposed to an explicit address in the traditional computer memory system. The memory allows the recall of information based on partial knowledge of its contents.
[Continued in next slide]
05
de
rs
.in
fo
ea
.m
yr ty ,w w
kr
ab
such that when one is encountered, the other can be recalled. The associative memory are of two types : auto-associative memory and hetero-associative memory.
ha
or
Auto-associative memory
y[3], . . . . . y[M],
representing features extracted from the patterns. The auto-associative memory will output a pattern vector y(m) incomplete version of y(m).
Hetero-associative memory
Here the memory function is more general. Consider, we number of key-response pairs {c(M), y(M)}.
have
Linear
associater
is and
the
simplest
artificial
neural
associative
memory.
Hopfield model Bidirectional Associative Memory (BAM)
are the other popular ANN models used as associative memories. These
06
models
follow
different
neural
network
architectures
to
memorize information.
de
rs
.in
fo
.m
ab
or
ty
,w
yr
Example
An associative memory is a storehouse of associated patterns which are encoded in some form.
When the storehouse is triggered or excited with a pattern, then
ha
kr
ea
SC - AM description
a stored pattern. Fig below illustrates the working of an associated memory. The associated pattern pairs
Input Pattern
( , ), ( , +), (7 , 4).
Recalled Pattern The association is represented
When the memory is triggered with an input pattern say then the associated pattern is retrieved automatically.
07
de
rs
.in
fo
.m
1.3 Associative Memory - Classes As stated before, there are two classes of associative memory:
auto-associative and hetero-associative memory.
ha
kr
ab
or
ty
,w
yr
ea
SC - AM description
An
A hetero-associative memory, also known as hetero-associative correlator, is used to retrieve pattern in general, different from the input pattern not only in content but possibly also different in type and format. Examples
Hetero-associative memory
Auto-associative memory
de
rs
.in
fo
.m
ea
SC - AM description
or
ty
Here explained : Encoding or memorization, Retrieval or recollection, Errors and Noise, Memory capacity and Content-addressability.
ha
kr
ab
,w C
Encoding or memorization
Building an associative memory means, constructing a connection weight matrix W such that when an input pattern is presented, and
the stored pattern associated with the input pattern is retrieved. This process of constructing the connection weight matrix is called
encoding.
the weight values of the correlation matrix Wk are computed as (wij)k = (xi)k (yj)k (xi)k (yj)k for represents the i th component of pattern Xk , and represents the j th component of pattern Yk i = 1, 2, . . . , m and j = 1, 2, . . . , n.
W
is accomplished by
k=1
Wk
where
de
rs
.in
fo
.m
Retrieval or recollection
After memorization, the process of retrieving a stored pattern, given an input pattern, is called decoding. Given an input pattern X, the decoding or recollection is accomplished by: first compute
input j =
ha
kr
ab
or
ty
,w
yr
ea
SC - AM description
using
j=1
xi w i j
where input j is weighted sum of the input or activation value of node j , for j = 1, 2, ..., n. then determine the units output using a bipolar output function:
Yj = +1 if input j j -1 other wise
where
10
de
rs
.in
fo
.m
ha
kr
ab
or
ty
,w
yr
ea
SC - AM description
retrieve the stored pattern that is closest to actual input pattern. The presence of noise or errors results only in a mere decrease rather than total degradation in the performance of the network. are robust and fault tolerant because and
elements
performing
highly
parallel
de
rs
.in
fo
.m
Performance Measures
The of memory associative capacity memory and content-addressability for correct are the measures two performance retrieval. These
kr
ab
or
ty
,w
yr
ea
SC - AM description
ha
refers
to
the
maximum
number
of
associated
is
to
retrieve
the
correct stored pattern. If input patterns are mutually orthogonal - perfect retrieval is possible. If the stored input patterns are not mutually orthogonal - non-perfect retrieval can happen due to crosstalk among the patterns.
12
de
rs
.in
fo
or
ty
,w
.m
2. Associative Memory Models An associative memory is a system which stores mappings of specific input representations to specific output representations.
yr
ha
kr
ab
Most
ea
SC - AM models
encountered, the other can be reliably recalled. associative memory implementations are realized as connectionist
networks. The simplest associative memory model is Linear associator, which is a feed-forward type of network. It has very low memory capacity and therefore not much used. The popular models are
Memory (BAM) model. Hopfield Model
de
rs
.in
fo
.m
2.1 Associative Memory Models The simplest and among the first studied associative memory models is Linear associator. It is a feed-forward type of network where output is produced in a single feed-forward computation. It used as an auto-associator as well as a hetero-associator, but the it can be
ha
kr
ab
or
ty
,w
yr
ea
SC - AM models
possesses a very low memory capacity and therefore not much used. The popular associative memory models are
Hopfield Model
and
John Hopfield in 1982. It is an ensemble of simple processing units that have a fairly complex collective computational abilities and behavior. time until The Hopfield model computes its output recursively in the system becomes stable. Hopfield networks are
is
similar to and
but BAM
the model
connections can
are
bi-directional
allows forward and backward flow of information between perform both auto-associative
and hetero-associative recall of stored information. The network architecture the next few slides.
14
of
these
three models
are
described
in
de
rs
.in
fo
.m
2.2 Network Architectures of AM Models The neural associative memory models architectures to memorize follow different neural network
or
ty
,w
yr
ea
SC - AM models
kr
ab
ha
The
Linear
associator
model,
network,
consists, two layers of processing units, one serving as the input layer while the other as the output layer.
The
each unit is connected to every other unit in the network other than itself.
The
is similar to
that of linear associator but the connections are bidirectional. In this section, the neural network architectures of these models and the construction of the corresponding connection weight matrix W of
de
rs
.in
fo
.m
ha
kr
ab
or
ty
,w
yr
ea
SC - AM models
inputs is calculated in each neuron node. The network architecture of the linear associator is as shown below.
weights wij neurons
x1
w12
y1 y2
outputs
x2
inputs
Xn
wnm
Fig. Linear associator model
Ym
all n input units are connected to all m output units via connection
weight matrix of
W = [wij]n x m
th
output unit.
the
connection
weight
matrix
stores
the
different
associated
pattern
building
weight matrix W
[Continued in next slide]
16
then the stored pattern associated with the input pattern is retrieved.
de
rs
.in
fo
ea
ty
,w
.m
yr
kr
ab
is called encoding.
or
matrix Wk for an associated pattern pair (Xk, Yk) are computed as: (wij)k = (xi)k (yj)k (xi)k is the i
th
ha
and
is accomplished
k=1
by summing those individual correlation matrices Wk, ie, W = where is the constant of proportionality, for set to 1/p to store p different associated pattern pairs.
Wk
normalizing, usually
the
process
of
retrieving
stored
pattern,
is
called
decoding;
input j stands for the weighted sum of the input or activation value of
j=1
xi w i j where
node j , for j = 1, 2, . . , n. and xi is the i th component of pattern Xk , and then determine the units Output using a bipolar output function:
Yj = +1 if input j j -1 other wise
where
Note: The output units behave like linear threshold units; that compute a weighted sum of the input and produces a -1 or +1 depending whether the weighted sum is below or above a certain threshold value.
Performance :
incomplete version of some previously encoded pattern. When such corrupt input pattern is presented, the network will retrieve the stored pattern that is closest to actual input pattern. Therefore, the linear associator is robust and fault tolerant. The presence of noise or error results in a mere decrease rather than total degradation in the performance of the network.
17
de
rs
.in
fo
.m
kr
ab
or
ty
,w
yr
ea
SC - AM models
ha
alternate view
1
I1
2
I2
3
I3
neurons
I4 inputs I
V1
V3 V2 outputs V
V4
unit is connected to every other unit in the network but not to itself.
connection
number wij. The collection of all such numbers are represented by the weight matrix W which is square and symmetric, ie, w for i, j = 1, 2, . . . . . , m.
each
ij
=wji
i=1
xi w i j + I j
for j = 1, 2, . . ., m.
X k Yk
where XK = YK
de
rs
.in
fo
ea
.m
yr
,w
is accomplished by
k=1
ty
kr
ab
Wk
where
or
is the constant of proportionality, for normalizing, usually set to 1/p the Hopfield an auto-associative memory model, it is the patterns
ha
the process of retrieving a stored pattern, is called decoding; given an input pattern X, the decoding or retrieving
j
computing, first the net Input as input for j = 1, 2, ..., n. and xi is the i
+1 if input j j -1 other wise
th
stands for the weighted sum of the input or activation value of node then determine the units Output using a bipolar output function:
Yj =
j=1
is
accomplished
by
xi w i j where input j
j,
where
Note: The output units behave like linear threshold units; weighted sum is below or above a certain threshold value.
Decoding in the Hopfield model is achieved by a collective and recursive relaxation search for a stored pattern given an initial stimulus pattern. Given an input pattern X, decoding is accomplished by computing the net input to the units and determining the output of those units using the output function to produce the pattern X'. The pattern X' is then fed back to the units as an input pattern to produce the pattern X''. The pattern X'' is again fed back to the units to produce the pattern X'''. The process is repeated until the network stabilizes on a stored pattern where further computations do not change the output of the units. In the next section, the working of an auto-correlator : patterns, recall a pattern from the stored patterns recognize a noisy pattern are explained.
19
de
rs
.in
fo
.m
kr
ab
or
ty
,w
yr
ea
SC - AM models
hetero-associations
ha
bidirectional
memory model is similar to that of the linear associator but the connections are bidirectional; i.e.,
wij = wji ,
for i = 1, 2, . . . , n
neurons
and
j = 1, 2, . . . , m. neurons
weights wij
x1
w12
y1 y2
outputs
x2
inputs
Xn
wnm
Ym
X k Yk .
construction of the connection weight matrix W, to store p pattern pairs simultaneously, is accomplished
Wk
de
rs
.in
fo
,w
.m
yr
ea
SC - AM
auto correlator
or
ha
kr
explained. It is an auto-associative memory model which means patterns, In this is section, the working of an auto-associative memory (auto-correlator) illustrated using some examples.
Working of an auto-correlator :
how to store the patterns, how to retrieve / recall a pattern from the stored patterns, how to recognize a noisy pattern
21
ab
and
de
rs
.in
fo
.m
yr
ea
SC - AM
auto correlator
ha
kr
ab
or
ty
A1 , A2, A3
to be stored as
,w
and
V is
U1V3 U2V3 U3V3 U4V3
V1 V2 V3
A1 , A2, A3
bipolar
=
i
1 -1 1 -1 -1 1 -1 1 1 -1 1 -1 -1 1 -1 1
j
T
=
i
1 1 1 -1 1 1 1 -1 1 1 1 -1 -1 -1 -1 1
j
T
=
i
1 1 1 -1 1 1 1 -1 1 1 1 -1 -1 -1 -1 1
is
j
3 1 3 -3 1 3 1 -1 3 1 3 -3 -3 -1 -3 3
i=1
de
rs
.in
fo
.m
yr
ea
SC - AM
auto correlator
or
ty
of
the
three
,w
ha
kr
ab
i=1
aj
new
= (ai t i j , aj
old
for j = 1 , 2 , . . . , p = ai t i j and
= ai = ai = ai = ai
= 10 6 = 10 = -1
1 1 1 -1
Therefore
aj
a1 a2 a3 a4
new
= (ai t i j = (10 , 1) = (6 ,
1)
, aj
) for j = 1 , 2 , . . . , p
is ( , )
The values of is the vector pattern ( 1, 1 , 1 , -1 ) which is A2 . This is how to retrieve or recall a pattern from the stored patterns. Similarly, retrieval of vector pattern A3 as
( a1
23 new
, a2
new
, a3
new
a 4 , ) = ( -1, -1 , -1 , 1 ) = A3
new
de
rs
.in
fo
.m
yr
ea
SC - AM
auto correlator
or
ty
A'
= ( 1,
1 ,
1 , 1 )
,w
kr
ab
of one among the stored patterns. the proximity of the noisy vector using Hamming distance measure.
note that the Hamming distance (HD) of a vector X from Y,
ha
X = (x1 , x2 , . . . , xn) HD (x , y) =
i=1
| (xi - yi ) |
The HDs of A' from each of the stored patterns A1 , A2, A3 are
HD (A' , A1) = |(x1 - y1 )|, = 4 HD (A' , A2) HD (A' , A3) = 2 = 6 |(x2 - y2)|, |(x3 - y3 )|, |(x4 - y4 )| |(1 - (-1) )|, |(1 - 1)|
Therefore the vector A' is closest to A2 and so resembles it. In other words the vector A' is a noisy version of vector A2. Computation of recall equation using vector A' yields :
i=
= ai = ai = ai = ai
+ + + +
+ + + +
+ + + +
= 4 = 4 = 4 = -4
1 1 1 -1
Therefore
aj
a1 a2 a3 a4
new
, aj
) for j = 1 , 2 , . . . , p
is ( , )
is A2 .
Note : In presence of noise or in case of partial representation of vectors, results in the refinement of the pattern or removal of noise to retrieve the closest matching stored pattern.
de
rs
.in
fo
,w
or
ty
.m
4. Bidirectional Hetero-associative Memory The Hopfield one-layer unidirectional auto-associators have been discussed in previous section. Kosko (1987) extended this network to two-layer bidirectional structure called Bidirectional Associative Memory (BAM) which can achieve hetero-association. The important performance attributes of the BAM is its ability to recall stored pairs particularly in the presence of noise. Definition : If the associated pattern pairs (X, Y) are different model recalls a pattern Y given a pattern X or vice-versa, termed as hetero-associative memory. This section illustrates the bidirectional associative memory : and if the then it is
yr
ha
kr
ab
Operations (retrieval, addition and deletion) , Energy Function (Kosko's correlation matrix, incorrect recall of pattern), Multiple training encoding strategy (Wang's generalized correlation matrix).
25
ea
SC - Bidirectional hetero AM
de
rs
.in
fo
.m
4.1 Bidirectional Associative Memory (BAM) Operations BAM is a two-layer nonlinear neural network. Denote one layer as field A as field B with elements Bi. The basic coding procedure of the discrete BAM is as follows. Consider N training pairs { (A1 , B1) , (A2 , B2), . . , (Ai , Bi), . . (AN , BN) } where Ai = (ai1 , ai2 , . . . , ain) and Bi = (bi1 , bi2 , . . . , bip) and aij , bij are either in ON or OFF state.
in binary mode ,
or
ty
,w
yr
ea
SC - Bidirectional hetero AM
ha
kr
ab
with
elements Ai
and
the
other layer
and
N
in bipolar mode,
= -1 M0 =
where Xi = (xi1 , xi2 , . . . , xin) and Yi = (yi1 , yi2 , . . . , yip) and xij(yij) is the bipolar form of aij(bij) The energy function E for the pair ( , ) and correlation matrix M is
E= -M
T
i=1
[ Xi ] [ Yi ]
With this background, the decoding processes, means to retrieve nearest pattern pairs, the pattern pairs are illustrated in the next few slides.
26
the operations
deletion of
de
rs
.in
fo
.m
ab
or
ty
,w
yr
ea
SC - Bidirectional hetero AM
ha
kr
) )
(F) = G = g1 , g2 , . . . . , gr , F = ( f1 , f2 , . . . . , fr )
M is correlation matrix 1 0 (binary) gi = -1 (bipolar) previous g i , fi = 0 , fi < 0 if fi >0
Kosko
has
proved
that
this
process
will
converge
for
any
correlation matrix M.
27
de
rs
.in
fo
.m
yr
ea
SC - Bidirectional hetero AM
kr
ab
or
ty
and a set
,w
or
ha
an existing pair (Xj , Yj) can be deleted from the memory model.
+ X1 Y1 + . . . . + Xn Yn
+ X'
Y'
from the correlation matrix M , them the new correlation matrix Mnew is given by
Mnew = M
( Xj
Yj )
Note :
The addition
functioning of
the system
exhibiting
and forgetfulness.
28
de
rs
.in
fo
.m
ha
kr
ab
or
ty
,w
yr
ea
SC - Bidirectional hetero AM
adding
new
patterns
must
not
destroy
the
previously
stored patterns. The stability of a BAM can be proved by identifying the energy function E with each state (A , B) .
For auto-associative memory : the energy function is
E(A)
= - AM A
E(A, B) = - AM B
it corresponds
to Hopfield auto-associative function. We wish to retrieve the nearest of (Ai , Bi) pair, when any ( , ) pair is presented as initial condition to BAM. The neurons change when it their states until a bidirectional stable state (Af , Bf) is reached. Kosko has shown that such stable state is reached for any matrix M corresponds to local minimum of the energy function. decoding lowers the energy
T
Each cycle of
E
T
if
the
energy
be recalled, even though one starts with = Ai. Thus Kosko's encoding method does not ensure that the stored pairs are at a local minimum.
29
de
rs
.in
fo
.m
ha
kr
ab
or
ty
,w
yr
ea
SC - Bidirectional hetero AM
( -1 -1
( -1
1 -3
M =
X1 Y1 +
X2 Y2 +
X3 Y3
-1 -1
1 -1 1 3 3 1
-1 -1 -1 -3 -1 1 1
3 -1
1 -1
Suppose we start with = X3, and we hope to retrieve the associated pair
Y3 .
(f , f) = (X3 , Y3 )
de
rs
.in
fo
.m
ha
kr
ab
or
ty
,w
yr
ea
SC - Bidirectional hetero AM
( -1 1 -1 1 -1 -1 1 0 1
+
3 -3 -1 -1 3 3 -3 -1 -1
X2 Y2 +
1 -1 -3 1 1 1 -1 -3 -3 1 -1 1 -3 1 1 -1 1 1 -1 1 -1 -1 -1 -1 1 -1 -1
X3 Y3
-1 1 -1 -1 -1 -1 1 -1 -1 1 -1 1 -3 1 1 -1 1 1 1 -1 -3 1 1 1 -1 -3 -3 -1 1 3 -1 -1 -1 1 3 3
de
rs
.in
fo
ea
SC - Bidirectional hetero AM
.m ty ,w
yr
ha
kr
ab
or
Start with = X2, and hope to retrieve the associated pair Y2 . The calculations for the retrieval of Y2 yield : M = ( ( M) = ( (' M ) = ( ' M = ( (' M) = (
= '
T
5 1
-19 -13 -5 -1 -1 5 1 -1 5 1
1 1
1 1
-5 -13 13 -1 -1 5 1 1 5 1
) ) ) =
' M T = ( -11 11
-1 5 1 1
-11 -11 11 -1 1 1 -1 1 1
1 -1
)
)
-19 -13 -5 -1 -1 -1
-5 -13 13 -1 1
Here " = ' . Hence the cycle terminates with F = ' = F = ' =
( ( -1 1 1 -1 1 -1 1 -1 -1 1 -1 1 1 1 -1 1 1 ) =
X2
-1
Y2
But ' is not Y2 . Thus the vector pair (X2 , Y2) is not recalled correctly by Kosko's decoding process.
( Continued in next slide )
32
de
rs
.in
fo
ea
SC - Bidirectional hetero AM
.m ty ,w w
yr
ha
kr
for the coordinates of pair (X2 , Y2) , the energy E2 = - X2 M Y2 = -71 for the coordinates of pair (F , F) , the energy EF = - F M F = -75 However, the coordinates of pair is "one Hamming distance" ' Y2
= ( 1 (X2 , Y2)
T
ab
or
-1 -1 -1
= - 73
which is lower than E2 confirming the hypothesis that (X2 , Y2) is not at its local minimum of E. Note : The correlation matrix M used by Kosko does not that pair Pi can be recalled of the energy surface.
33
guarantee
the energy of a training pair is at its local minimum. Therefore , a if and only if this pair is at a local minimum
de
rs
.in
fo
.m
ea
SC - Bidirectional hetero AM
or
ty
Kosko extended
the unidirectional
,w
ab
Xi
ha
kr
Yi computed from the pattern pairs. The system proceeds to nearest pair given any pair ( , ), with the help of recall
retrieve
the
equations. However, Kosko's encoding method does not ensure that the stored pairs are at local minimum and hence, results in incorrect recall.
Wang and other's, introduced multiple training encoding strategy ensures the correct
which
enhancement / generalization of Kosko's encoding strategy. The Wang's generalized correlation matrix is M = qi as pair weight for
T Xi
Xi
Yi
where qi
is viewed
Yi
as
It denotes the
minimum number of times for using a pattern pair (Xi , Yi) for training to guarantee recall of that pair. To recover a pair (Ai , Bi) using multiple training of order q, let us
Yi
The augmentation implies adding (q - 1) more pairs located at (Ai , Bi) to the existing correlation matrix. As a result the energy E' can reduced to an arbitrarily low value by a suitable choice of q. This also ensures that the energy at (Ai , Bi) does not exceed at points which are one Hamming distance away from this location. The new value of the energy function E evaluated at (Ai , Bi) then becomes
E' (Ai , Bi) = Ai M Bi
T
(q 1) Ai Xi
Yi Bi
The
next
few
slides
explains
Multiple training encoding strategy for the recall of three pattern pairs
(X1 , Y1 ) , (X1 , Y1 ) , (X1 , Y1 ) using one and same augmentation matrix M . Also an algorithm to summarize the complete
process of multiple
de
rs
.in
fo
,w
.m
ha
kr
ab
or
ty
yr
ea
SC - Bidirectional hetero AM
( -1 1 -1 1 -1 -1 1 0 1
Y2 =
1 -1 -1 -1 -1 -1 -1 -1 1
+ 2 X2 Y2 +
2 -2 -4 0 2 2 -2 -4 -4 2 -2 0 -4 2 2 -2 0 0 0 0 -2 -2 0 0 0 -2 -2 0 0 -2 -2 0 0 0 -2 -2
X3 Y3
2 -2 0 -4 2 2 -2 0 0 2 -2 -4 0 2 2 -2 -4 -4 -2 2 4 0 -2 -2 2 4 4
de
rs
.in
fo
ea
SC - Bidirectional hetero AM
.m
yr
ty
,w
ha
kr
ab
is correctly recalled as shown below. M = ( 14 -28 -22 -14 -8 ( M) = ( (' M T) = ( (' M) = ( Here
1 -1 -1 -1 -1 -8 -14 -22 22 -1 -1 -1 1 ) ) ) =
or
1 -1
)
)
-8 -14 -22 23 1 -1 1
X2
-1
= Y2 correlation
Thus,
(X2
Y2 )
is correctly
recalled,
using
augmented
matrix M .
de
rs
.in
fo
ea
SC - Bidirectional hetero AM
.m ty ,w w kr ab or
yr
Note : The previous slide showed that the pattern pair (X2 , Y2 ) is correctly recalled, using augmented correlation matrix
M = X1 Y1 + 2 X2 Y2 + X3 Y3
T T T
ha
but
then
the
same
matrix
can not
recall
correctly
the
other
Let = X1 and to retrieve the associated pair Y1 the calculation shows M = ( ( M) = ( (' M T) = ( (' M) = (
-6 -1 24 22 1 1 6 1 4 1 4 1 6 1 22 -22 1 -1 ) ) =
-1 1
-1
-1
)
)
' M = ( -14 28 22 14
-1 1 1 1
14 22 -22 1 -1
Here " = ' . Hence the cycle terminates with F = ' = F = ' =
( ( 1 -1 -1 1 -1 1 -1 1 1 1 1 1
-1 1
-1 1
-1 -1
X1
Y1
Thus, the pattern pair (X1 , Y1 ) is not correctly recalled, using augmented correlation matrix M. To tackle this problem, the augmented
37
correlation
matrix
needs to be further
de
rs
.in
fo
ea
SC - Bidirectional hetero AM
.m
yr
ty
,w
Y1 )
cannot be recalled
ha
kr
ab
under the same augmentation matrix M that is able to recall (X2 , Y2). However, this problem can be solved by multiple training of (X1
,
or
Y 1)
Now observe in the next slide that all three pairs can be correctly recalled.
( Continued in next slide )
38
de
rs
.in
fo
ea
SC - Bidirectional hetero AM
.m
yr
,w
(X1 , Y1 )
Y1 = ( 1 1 1 -1 -1 -1 -1 1 -1 )
ab
or
1 -1 -1 1 1 1 -1 -1 -1 )
ty
ha
kr
(X2 , Y2 )
Y2 = ( 1 -1 -1 -1 -1 -1 -1 -1 1 )
( -1 1 1 1 -1 -1 1 1 1 )
(X3 , Y3 )
Y3 = ( -1 1 -1 1 -1 -1 1 0 1 )
1 -1 1 -1 1 1 -1 1 1 )
de
rs
.in
fo
ea
SC - Bidirectional hetero AM
.m ty ,w w
yr
Thus, the multiple training encoding strategy ensures the correct recall of a pair for a suitable augmentation of
N
kr
ab
M . The generalization of
the
or
correlation matrix, for the correct recall of all training pairs, is written as
M =
ha
i=1
qi Xi
Yi
real numbers.
This modified
correlation matrix
Using one and same augmentation matrix M, it is possible to recall all the training pattern pairs .
40
de
rs
.in
fo
.m
yr
ea
SC - Bidirectional hetero AM
ha
kr
ab
or
ty
,w
where
( (
1, X 1 , Y
Step 1 Initialize correlation matrix M to null matrix Step 2 Compute the correlation matrix M as
For i 1 to N
i ) ( X i ) M M [ qi Transpose ( X
end
(symbols
Step 4 Compute
A_M where
A_M
M A
to
A_M
' to get B
ie
' B
( A_M )
' B
de
rs
.in
fo
.m
5. References : Textbooks
yr ty
1. "Neural
ea
SC AM References
ha
kr
Network, Fuzzy Logic, and Genetic Algorithms - Synthesis and Applications", by S. Rajasekaran and G.A. Vijayalaksmi Pai, (2005), Prentice Hall, Chapter 4, page 87-116. and Sanjay Ranka, (1996), MIT Press, Chapter 6, page 217-263.
ab
or
,w C
2. "Elements of Artificial Neural Networks", by Kishan Mehrotra, Chilukuri K. Mohan 3. "Fundamentals of Neural Networks: Architecture, Algorithms and Applications", by
Laurene V. Fausett, (1993), Prentice Hall, Chapter 3, page 101-152.
5. "An Introduction to Neural Networks", by James A. Anderson, (1997), MIT Press, 6. Related documents from open source, mainly internet. An exhaustive list is
being prepared for inclusion at a later date.
42