ECE EEE F418 Channel Coding Feb 21, Mar 2, 4, 7,9 2017

ECE / EEE F418
Modern Communication Technologies

Topic # 4: Channel Coding:
T1. DIGITAL COMMUNICATIONS Fundamentals and

Applications: Bernard Sklar and Pabitra Kumar Ray;
Pearson Education 2009, 2/e. Chapters 6,7
R1. Communication Systems Engineering, John G Proakis

and Masoud Salehi, Pearson Education, 2002, Second Ed.
Chapter 9
Feb 21, Mar 2, 4, 7 2017
1 ELECTRICAL ELECTRONICS COMMUNICATION INSTRUMENTATION
Channel Coding
What is Channel coding ?
Why Channel Coding ?
Where is it done ?
How is it done ?

Channel Coding
(n,k) code has k message bits and is expected to be mapped on to n coded bits.
All 2k k-bit message words are called k-tuples For each of the 2k
message k-tuples, the
encoding process
assigns one member of
All 2n n-bit blocks are called n-tuples
the 2n n-tuples
Which one is better and in what sense?

Messages and Liner Block codes
The mapping can be accomplished via a lookup table.
Q1. Since 2n >> 2k. Then, which 2k out of 2n to be used ?
Q2. For large (n, k) is it possible to have lookup table implementation ?.
Ex: for ( 127, 92) code, There will be 292 = 5 X 1027 code words.
We need techniques for generating the code words dynamically from the
message words.

Messages and Liner Block codes
The set of all 2n n-tuples, Vn , form an n-dimensional vector space over

binary field.
Code words are a subset S of the vector space Vn satisfies the

following Rules:
Rule 1: The all-zeros vector is in S
Rule 2: The sum of any two vectors in S, is also a vector in S. ( Closure

Property)

Vector Spaces and Liner Block codes
Ex: Consider a 24 tuples vector space
Does the subset form the subspace ?
Apply Rules 1 & 2 to verify.
Rule 1: The all-zeros vector is in S
Rule 2: The sum of any two vectors in S, is also a vector in S. ( Closure

Property)

Liner Block codes
The set of all 2k n-tuples is called a Linear Block Code, IFF, it is a subspace
of the vector space Vn of all n tuples
How to obtain the subspace (Linear Block code)?
Since, the code words that form a linear block code is a k-dimensional
subspace of the n-dimensional space,
It is possible to find a set of n-tuples, fewer than 2k, that can generate all the
2k code words of the subspace ( from Rule 2: The sum of any two vectors in
S, is also a vector in S: Closure Property )

Liner Block codes
2n n-tuples, Vn , form an n-
dimensional vector space. 2n n-tuples, Vn
2k n-tuples subspace,
Linear Block Code
< 2k span
The generating set of
vectors (fewer than 2k )
are said to span the
subspace.
2k k-tuples
The Smallest linearly
independent set that span the subspace,
subspace is called the basis. Linear Block
Code
The number of vectors in the
basis is called the dimension
of the subspace.
Liner Block codes
Any basis set of k linearly independent n-tuples, say

V1, V2, ..,Vk ( each of them is a (1 X n) row vector )
can be used to generate the required 2k code words of the linear block code.
How? : Because, each of the 2k code vectors is a linear combination of

V1, V2, ..,Vk
We can use the k message bits as coefficients for linear combination !!
Let the message bits be: m1, m2, , mk and

V1, V2, ..,Vk be the (1 X n) row vectors
Then one of the ( 1 X n) Code Word can be U = m1 V1 + m2 V2 + , mk Vk

Liner Block codes
Ex: (6,3) Code, Let
V1 = [ 1 1 0 1 0 0 ] Are these vectors linearly independent ?
V2 = [ 0 1 1 0 1 0 ] (That is , Does any linear combination of two

vectors give the third vector?)
V3 = [ 1 0 1 0 0 1 ]
For the message (1 1 0), the code vector is
U = m1 V1 + m2 V2 + , mk Vk
U7 = [ 1 0 1 1 1 0 ]
For the message (0 1 1), the code vector is U4 = [ 1 1 0 0 1 1 ]

Liner Block codes
Let us define a (k X n) generator matrix,

consisting of the basis vectors V1, V2, ,Vk as rows
Let the message vector (1 X k) be: m = m1, m2, , mk
The code vector, corresponding to a

message vector can then be obtained as

Liner Block codes
Ex: (6,3) Code
Let the generator matrix be
The 2k, k-bit Messages are ( 000, 001, 010, 011, 100,101,110,111)
For the message (1 1 0), the message vector (1 X k) : m7 = [ 1 1 0 ]
U7 = m7 G = [ 1 1 0] =[1 0 1 1 1 0]

Liner Block codes
For all The 23, 3-bit Messages
The encoder needs to store only the k linearly independent vectors and not
2k code words !!
Liner Block codes Important Points
Minimum Distance of a code: Each Error detection capability : dmin-1 = 2

code word differs from another code
word in at least d min components.
Error correction capability : =1
floor [(dmin-1)/2]
Code can be used for simultaneous

correction of a errors and detection of
b errors, where b a, provided the
minimum distance
dmin = 3
dmin a + b + 1
a = 1 and b = 1 satisfy the above.

Weight of a Code: Number of non
zero components in the vector. Hence, this code can detect 1 error
Minimum Weight = 3 and correct 1 error simultaneously
Hamming & Plotkin Bounds
The Error detection & Correction capabilities of a code are dictated by the
minimum distance between the codes. ( dmin)
How is ( dmin) related to (n, k) ? ( Plotkin Bound)
Plotkin bound dictates bound on ( dmin)
In the previous (6,3) code, The bound dictates as
dmin (6 X 4) /(8 1) = 3. 42 dmin = 3

Hamming & Plotkin Bounds
How are the number of correctable errors t related to (n,k) ?
( Hamming Bound) Log2 { 1 + nC1 + nC2 + ... + nCt } n - k
In the previous (6,3) code, for 1 error correction
Log2 { 1 + 6C1} = 2.8 is (6 3)
for 2 error corrections Log2 { 1 + 6C1 + 6C2 } = 4.5 not (6 3)
While designing a code, Hamming & Plotkin bounds have to be met.

Systematic Liner Block codes
A desirable property for a linear block code is to possess a systematic

structure in the code words.

Since, And message vector m = m1, m2, , mk
To have the code word as,
The generator matrix takes the form,

pij = 0 or 1
P is a k X (n-k) Parity Matrix
and Ik is a k X k Identity matrix

= [ m1, m2, , mk ]
The parity bits come out to

be summations of
message bits in different
combinations.
20ELECTRICAL ELECTRONICS COMMUNICATION INSTRUMENTATION

Ex: (7,4)
Are the rows of G Linearly Independent ?
What is the code vector for message bits m=[ 1 1 01]
U= mG = [ 1 1 01] = [0 0 0 1 1 0 1]

(7,4) Systematic Liner Block codes - Encoder
Ex: (7,4)
U = [ m1 m2 m3 m4 ] G
u1 = m1 + m3 + m4 u2 = m1 + m2 + m3 u3 = m2 + m3 + m4
u4 = m1, u5 = m2, u6 = m3, u7 = m4
p2
U = [m1 + m3 + m4 m1 + m2 + m3 m2 + m3 + m4 m1 m2 m3 m4 ]
p1 p3
(7,4) Systematic Liner Block codes - Encoder
p2
U = [m1 + m3 + m4 m1 + m2 + m3 m2 + m3 + m4 m1 m2 m3 m4 ]
p1 p3
p1 p2 p3

Systematic Liner Block codes - Encoder

Parity-Check Matrix
For every (k X n) generator matrix of the form
There is another useful matrix H, associated

with every linear block code., such that
GHT = 0
H is with an (n k) X n
matrix, with n k linearly 0 is a k X (n-k) matrix with all zero entries.
independent rows
That is, any vector in the row space of G is orthogonal to the rows of H..
And any vector that is orthogonal to the rows of H is in the row space of G

Parity Check Matrix
The (n-k) X n Parity Check Matrix H takes the form
GHT
ELECTRICAL ELECTRONICS COMMUNICATION INSTRUMENTATION

Parity Checking
Since U = m G And U HT = 0 U H T = m G HT = 0
That is, for the true code vector U
And for a code vector with corrupted bits, Ur HT 0
i.e., Ur HT will have some non zero entries in some locations
This is the Parity check process and H is called the parity check matrix.

Parity Check Matrix
Ex: (7,4)

Parity Check Matrix
Ex: (7,4)
GHT = ??

Systematic Liner Block codes Syndrome Testing
If U = u1, u2, u3, ., un is the transmitted code vector ( one of the 2k tuples)
And r = r1, r2, r3, ., rn is a received vector ( one of the 2n tuples), then
Where e = e1, e2, e3, ., en is a channel introduced error vector
Then the Syndrome of r is defined as

Systematic Liner Block codes Syndrome Testing
Since
And since GHT = 0 It results in to
That is, If there are no channel induced errors, then Syndrome S = 0
If r contains detectable errors, the S has some non zero entries.
If r contains correctable errors, then S has some non zero entries that can
earmark the particular error pattern.

(6,3) code Syndrome Testing
Example: (6,3) code.
Let the message vector be m7 = [ 1 1 0 ]
The corresponding code word is
U7 = m7 G = [ 1 1 0] =[1 0 1 1 1 0]
If there is an error e = 0 0 0 0 0 1, the corresponding received word is
r = e + U7 = [ 1 0 1 1 1 1 ]
Calculate the Syndrome S for the received vector r

r=[1 0 1 1 1 1]
= [ 1 0 1]
e =000001 S = [ 1 0 1]
Calculating the Syndrome S for the received vector r is same as the one
calculated for error vector e
For each error vector with single error, there is a corresponding syndrome S
(6,3) code Syndrome for All single error patterns
Syndrome S Corresponding to all Error Patterns is
= X =
Error pattern Syndrome
Syndrome testing not only helps the error detection but also
correction as the syndrome can be mapped to hidden error pattern.
Concept of standard array and error detection:

Systematic Liner Block codes Standard Array
1. Form a Standard array for all possible error patterns. Ex: For the (6,3)
example, with Single error correction capability, the standard array is
All code vectors
received code vector

All single bit error patterns

(6,3) code Error Correction
2. Compute the Syndrome S Corresponding to all Error Patterns & Store

(Example: (6,3) code)
= X =
Error pattern Syndrome
3. Calculate the Syndrome S for the received vector r

Example: (6,3) code.
Let m7 = [ 1 1 0 ]
If there is an error e = 1 0 0 0 0 0,
code word is
r = e + U7 = [ 0 0 1 1 1 0 ]
U7 = m7 G
Syndrome S for the

received vector r
= [ 1 1 0]
=[1 0 1 1 1 0] S = [ 1 0 0]
4. For the syndrome of received vector in step 3, locate the error

pattern in the error matrix ( see the step 2).
5. Go to the standard array

6. Locate the error pattern (e = 1 0 0 0 0 0) in the standard array
7. Locate the received vector in standard array, in the row corresponding to the
error pattern
8. The true bit pattern ( error corrected) is the top most entry in the column.
For a simple case, one can do

Systematic Liner Block codes Decoder for (6,3)
Syndrome Computation

Error Mapping

Error Corrected Output

C
Systematic Liner Block codes Error Correction
Standard Array
All code vectors
bit error patterns
All received code vectors

C

Cyclic Codes

What are Cyclic Codes ?
Cyclic codes are a sub set of linear block codes with a special property :
If U = (u0, u1, u2 , un-1)
Is a code word in the subspace S, then U (1) obtained by cyclic rotation of U
That is : U (1) = (un-1 , u0, u1, un-2)
Is also a code word in the same subspace S.
Example 1: Consider the words 000 110 101 011
Are these linear block codes? Do they satisfy the 2 properties ?
Are these Cyclic ? That is, by cyclic rotation of one word do we get
another code word belonging to the same set ?
What are Cyclic Codes ?
Example 2: How about these words ? 000 010 101 111
Are these linear block codes? Do they satisfy the 2 properties ?
Are these Cyclic ? That is, by cyclic rotation of one word do we get
another code word belonging to the same set ?
All linear block codes need not be cyclic codes
In general, the cyclic code U (i) = (un-i , un-i+1 , un-1 , u0, u1, un-i-1)
is a code word in the subspace S, obtained by cyclic shift of U by i times.

Why Cyclic Codes ?
1. Encoding is easily implementable using feedback shift registers
2. Syndrome calculations are easily implementable using

feedback shift registers
3. Underlying algebraic structure of these codes lends itself to

efficient decoding methods

Polynomial Representation of Cyclic Codes
If U = (u0, u1, u2 , un-1) U (1) : 1-cyclic rotated version of U is
U (1) = (un-1 , u0, u1 ,,un-2 )
The polynomial representation for the code word is
U (X) = u0 + u1 X + u2 X2 + ..un-1 Xn-1
Consider X U (X) = u0 X + u1 X2 + .. un-2 Xn-1 + un-1 Xn

X U (X) = u0 X + u1 X2 + .. un-2 Xn-1 + un-1 Xn
Suppose we Add & Subtract un-1 . (Equivalent to adding un-1 twice !! )
X U (X) = un-1 + u0 X + u1 X2 + un-2 Xn-1 + un-1 Xn + un-1
U (1) (X) : Polynomial corresponding to

1-cyclic rotated version of U
X U (X) = U (1) (X) + un-1 Xn + un-1
X U (X) = U (1) (X) + un-1 (Xn + 1)

U (1) (X) = un-1 + u0 X + u1 X2 + un-2 Xn-1
U (1) (X) is polynomial whose degree is (n-1) and is not divisible by (Xn + 1)
Can we state: when we divide X U (X) with (Xn + 1),
we get un-1 as quotient and U (1) (X) is the remainder,
OR, U (1) (X) = X U (X) Modulo (Xn +1) Paves a way to obtain cyclic
shifted codes !!
In general, for i-th cyclic rotation,
U (i) (X) = Xi U (X) Modulo (Xn +1)

Example
Let U = 1 1 0 1. Express the code word in polynomial form and obtain the 3 cyclic
shifted code word, using polynomial division method.
U (X) = u0 + u1 X + u2 X2 + ..un-1 Xn-1
In general, for i-th cyclic rotation, U (i) (X) = Xi U (X) Modulo (Xn +1)

Example
In general, For polynomial division, it is convenient to re order the polynomial from

highest degree to the lowest.
Re order the remainder from

lowest degree to the highest.
verify by cyclic rotation of
The corresponding code word is U (3) = 1 0 1 1 U = 1 1 0 1 by 3 times

Generator Polynomial for Cyclic Codes
For an (n, k) linear block code,
Where G is the generator matrix, m the message vector and U is the code word.
Similarly, the generator polynomial g(X) for an (n, k) cyclic code is given by
g (X) = g0 + g1 X + g2 X2 + ..gn-k Xn-k g0 and gn-k must be equal to 1
The message bits are represented

m (X) = m0 + m1 X + m2 X2 + ..mk-1Xk-1
by a message polynomial m (X)
The code word polynomial U (X) is U(X) = m(X) g(X)
Note that polynomial U(X) is an (n-1) th degree polynomial

U(X) = m(X) g(X) g (X) = g0 + g1 X + g2 X2 + ..gn-k Xn-k
U(X)
= m(X)
g(X)
=> U is a valid code word only when g(X) divides U(X) without a remainder !!
Decoding: At the receiver, if the remainder of r(X) / g(X) zero, valid code is
received.
and the quotient is the message !!
Property: for g(X) to be generator polynomial for the (n,k) code, it

should be a factor of (Xn + 1)
Property: for g(X) to be generator polynomial for the (n,k) code, it

should be a factor of (Xn + 1)
For Example:
Factors are
One may use
And generate a code (7,4) Cyclic Code ( Since n-k = 3 )

And generate a code
OR, One may use (7,3) : (n-k) = 4
In general, if g(X) is polynomial of degree (n k), and is a factor of (Xn + 1),

then g(X) uniquely generates an (n, k) code.
Example: Obtain the generator polynomial for a (7,4) cyclic code:
n = 7, k = 4, n-k = 3 From the Properties of generator

polynomial :
1. g(X) should be a 3rd degree Polynomial g(X) = ( ? X3 + ? X2 + ? X + ?1)
2. The coefficient of X3 and X0 should be 1 g(X) = ( X3 + ? X2 + ? X + 1)
3. g(X) should divide (X7 + 1) without a remainder. How do we

factor (X7 + 1) with above conditions ?
Can we start with an assumption that g(X) = ( X3 + X2 + X + 1) Is this a

factor of (X7 + 1)
Check by dividing (X7 + 1) with ( X3 + X2 + X + 1) It is not a factor of (X7 + 1)

Next Can we try g(X) = ( X3 + X2 + 1), without losing the characteristics

of g(X) ?
Check by dividing (X7 + 1) with ( X3 + X2 + 1)
Yes It is a factor of (X7 + 1) with quotient (X4 + X3 + X2 + 1)
Thus g(X) = ( X3 + X2 + 1), is valid generator polynomial.
Can we factor (X4 + X3 + X2 + 1) further, to see if we can have another

possible g(X) , without losing the characteristics of g(X) ?
(X4 + X3 + X2 + 1) = (X + 1) ( X3 + X + 1)
Thus we can have another g1(X) = ( X3 + X + 1) as generator polynomial.

g (X) = ( X3 + X2 + 1) g1(X) = ( X3 + X + 1)
Use g(X) to obtain Encoded word For message bits 1 1 1 1 :
For bits 1 1 1 1; m (X) = 1 + X + X2 + X3
U(X) = m(X) g(X) = (1 + X + X2 + X3 ) ( X3 + X2 + 1)
= ( 1+ X + X3 + X6) U=1101001
Use g1(X) to obtain Encoded word For message bits 1 1 1 1 :
U1 = 1 0 0 1 0 1 1

Systematic Cyclic Codes
We know that, in a Systematic Code, the

message bits are a part of the code words
Let the message polynomial m (X) = m0 + m1 X + m2 X2 + ..mk-1Xk-1
Let the parity polynomial p (X) = p0 + p1 X + p2 X2 + ..mn-k-1Xn-k-1
To make the message bits to appear the end of the code word, the message
polynomial should start with terms like m0 Xn-k
This is achieved by multiplying m (X) with Xn-k
Xn-k m (X) = m0 Xn-k + m1 Xn-k -1+ ..mk-1Xn-1
The code word polynomial then is U (X) = p(X) + Xn-k m (X)

We know that U is a valid code word only when g(X) divides U(X) without a
remainder !!
U(X) = q(X) g(X) Where q(X) is a quotient polynomial
Also, U(X) = p(X) + Xn-k m (X) Hence, q(X) g(X) = p(X) + Xn-k m (X)
Adding p(X) on both sides
q(X) g(X) + p(X) = Xn-k m (X) p(X) / g(X) + q(X) = Xn-k m (X) / g(X)
q(X) is quotient and p(X) is the remainder, when Xn-k m (X) is divided with g(X)
Parity polynomial is obtained as a reminder by dividing Xn-k m (X) with g(X)
Which in turn implies that p(X) = Xn-k m (X) Modulo g(X)

While Decoding:
Since U(X) = q(X) g(X), At the receiver, if the remainder of r(X) / g(X) zero,
valid code is received.
Also, Since the code word polynomial is U(X) = p(X) + Xn-k m (X)
the message bits are the right most k-bits of the received vector..

Example: Using g(X) = ( X3 + X + 1), obtain a systematic cyclic code

word, from a code word set (7,4), for message bits 1 0 1 1,
n = 7, k = 4, n - k = 3
For bits 1 0 1 1; m (X) = 1 + X2 + X3
Xn-k m (X) = X3 m (X) = X3 + X5 + X6
p(X) = Xn-k m (X) Modulo g(X)
p(X) = (X3 + X5 + X6 )Modulo (1 + X + X3 )

p(X) = (X3 + X5 + X6 )Modulo (1 + X + X3 )
Divide (X3 + X5 + X6 ) with (1 + X + X3 )
(1 + X2 + X3 + X4 ) is the quotient and Reminder = 1
Hence, p(X) = 1 U(X) = p(X) + Xn-k m (X)
U(X) = 1 + X3 + X5 + X6 U=1001011

BCH Codes
BCH Codes are a sub class g(X)

of cyclic codes
BCH Codes can be designed

for correction of t errors
For a given k, the number of

message bits, and t, the number
of errors to be corrected,
There exist BCH Code with
n = 2m-1 ; m = 3,4,5
The coefficients of g(X) are in the octal number format
The generator polynomials g (X) = g0 + g1 X + g2 X2 + ..gn-k Xn-k
g(X) are given in tabular form
g0 and gn-k must be equal to 1
BCH Codes
The coefficients of g(X) are

in the octal number format
g(X)
g (X) = g0 + g1 X + g2 X2 + ..gn-k Xn-k
For (7,4) code g (X) in Octal is 13
001011 Highest degree of

polynomial is n k =3
g0 and gn-k must be equal to 1
0 0 1 0 1 1
g0 g1 g2 g3
g (X) = 1 + X2 +X3
Shift Register Structures for Polynomial Division
Self Study: Sections 6.7.4, 6.7.5, 6.7.6 of Sklar,

How to form an (n , k) code ?
Invoke the Plotkin bound

Let us say we want a code to have an
error correcting capability of 2 errors. t = 2
Error correction capability dictates
t = floor [(dmin-1)/2] Then, dmin = 2t + 1 = 5 Does n = 7 satisfy the Plotkin bound?
NO: RHS = 4.66 & dmin= 5

Since 2 error to be corrected,
Minimum number of data bits k =2
Minimum value of n, to satisfy
So the code is expected to be (n,2) code Plotkin bound is = 8
What is the minimum value of n ?
Invoke the Hamming bound Thus, we need to chose the code
as (8,2)
Log2 { 1 + nC1 + nC2 + ... + nCt } n - k
Get the Smallest value of Of Course this code
n=7
n that meets this bound is not BW efficient !!
Parity Codes and Tx , Rx Operations
Message Bits Code Bits

Parity
Modulation
Message Data Coding Symbol Rate Rs
Rate Rb Coded Data Rate Rcb
or Code data rate
Coded Symbol
or bit Energy Es
Message Bits Code Bits
Parity De De Modulation
Coding and detection
Message bit error
probability = PB Bit error probability = p = Pb
Parity Decoder works on a block of n bits to identify whether the block is in error or not.
If the block is in error, then the block is dropped. Error detection.

If the error in the block is not detected, then the block is used to decode the message.
How to do the parity checking in the systematic liner block codes ?

Parity-Check Matrix
Problem
(f) Obtain the error pattern corresponding to the syndrome calculated

in (C) and obtain the correct code word corresponding to the received
vector
Solution
a) mT=
[ 0000 0001 0010 0011 0100 0101 0110 0111 1000 1001 1010 1011 1100 1101 1110 1111]
b)
= P I

Solution
Weight of a Code:
Syndrome for Number of non zero
C) r=1101101 components in the vector. D)
Error detection
Minimum Weight of a
capability :
Code = dmin
dmin-1 = 2
E)
S=[1101101]
Error correction
capability :
floor [(dmin-1)/2]
=1
F) Solve
yourself
Its not a valid codes as the
Syndrome is not 0 dmin = 3
Convolutional Codes
For an (n, k) linear block code,
For an (n, k) linear systematic block code,
For an (n, k) Cyclic code, U(X) = m(X) g(X)
For an (n, k) Systematic Cyclic code, U(X) = p(X) + Xn-k m (X)
In all the above, for a given sequence of message bits, the code
word depends on the current block message bits.
In convolutional codes, for a given sequence of message bits, the code
word not only depends on the current block message bits but also on the
previous / next block of message bits.
That is, Convolutional codes have Memory:
Convolutional Codes
A block of k-message bits are shifted at a time.
After each shift, n outputs are collected and used as coded bits.
k/n = code rate

K (or L) = Constrained length: Number of stages
Convolutional Codes
Mostly used Convolutional encoders treat single

message bit (k = 1)as a block and shift.

Convolutional Codes

Convolutional Codes
For the convolutional Encoder, to characterize the encoding function G(m),

that is to compute the output sequence U for the given input sequence m,
there are several methods: Appropriate tools for these methods are
2. Connection Vectors or Polynomials
3. The State Diagram
4. The Tree Diagram
5. The Trellis Diagram

Example of an Encoder
connection vector
Constrained length of the encoder
= number of shift register stages :
K=3
Number of message bits shifted at

a time :k =1
Number of code bits produced on

each shift : n= 2
Code Rate = k / n =

Impulse Response of an Encoder
Impulse Response: Output code As time progresses, the sequence in

values when a single 1 is passed which the bits enter the shift register and
through the shift register. the output code bits are as follows:
EQUIVALENT Input
Sequence: 1 0 0
output Sequence: 11 10 11
Called as Impulse Response:

Note the effect of memory on the
convolutional code
Output sequence from an
Encoder for a given message sequence
What is the output sequence for a

message bit sequence of {1 0 1} ?
Impulse response of this encoder is

Polynomial Representation
A convolutional code may be defined by a set of n generating polynomials

for each input bit.
g1(X) = 1 + X + X2
U (X) = m(X) g1(X)
interlaced with
m(X) g2(X)
g2(X) = 1 + X2

Polynomial Representation
g1(X) = 1 + X + X2
for input message { 1 0 1} m(X) = 1 + X2

g2(X) = 1 + X2
U1(X) = m(X) g1(X) = (1 + X2 ) (1 + X + X2 ) = (1 + X + X3 + X4 ) = (1 + X + 0.X2 + X3 + X4 )
U2(X) = m(X) g2(X) = (1 + X2 ) (1 + X2 ) = (1 + X4 ) = (1 + 0.X + 0.X2 + 0.X3 + X4 )
U (X) = (1, 1) + (1,0) X + (0, 0) X2 + (1, 0) X3 + (1, 1) X4 U = 11 10 00 10 11

Example
Connection Vectors ? Output for an input of { 1 0 1} ?
10 01 11
g1 = 1 0 1 00 00 00
10 01 11
---------------------------------
10 01 01 01 11 Output
Polynomials ? g1(X) = 1 + X2 g2(X) = X + X2

g2 = 0 1 1
Output for an input of { 1 0 1} ?

Impulse Response ?
100 10 m(X) = 1 + X2
m(X) g1(X) = 1 + 0X + 0X2 + 0X3+ 1X4

010 01 10 01 11
m(X) g2(X) = 0 + 1X + 1X2 + 1X3+ 1X4
001 11
Output is (1,0) (0,1) (0,1) (0,1) (1,1)
Encoder Output - State Diagram Representation
Current state is the contents of the

right most K-1 registers
Next state is the contents of the left

0 0 0 most K-1 registers
The possible states of the encoder are
00 10 11 01
The state transition takes place for The output depends on the current input and
every shift of input message bit into the current state of the encoder: i. e. the
the shift register. contents of the right most K-1 registers.

If the state is 00
and if input is 0 :
0 0 0
Output word = 00
The next state will be 00
The output depends on the current input State diagram depicts the these state
and the state of the encoder ( i. e. the transitions and the corresponding
contents of the shift register). outputs of the encoder, for every
possible input bit
The possible states are 0/00

represented as blocks
a = 00
1/11
0/11
1/00
b = 10 c = 01
0/10
Current Input New output State
state bit state Connection 0/01
1/01
0,0 0 0,0 0,0 Solid Line d = 11
0,0 1 1,0 1,1 Dotted Line
1,0 0 0,1 1,0 Solid Line 1/10
1,0 1 1,1 0,1 Dotted Line

Encoded words for given message bits
0/00
a = 00
1/11
0/11
1/00
b = 10 c = 01
0/10
1/01 0/01
d = 11
1/10
State diagram gives only the states and associated encoded words in
static condition.
How to obtain the encoded bits for message bits that progress in time ?

Tree Diagram
0/00
a = 00
1/11
0/11
1/00
b = 10 c = 01
0/10
1/01 0/01
d = 11
1/10
How to obtain the encoded bits for

this input message bits ?
t1 t2 t3 t4 t5
Trellis Diagram
Input is 0 00
a a
Input is 1
11
11
b b
10
00
01
c c
01
d d
10
tn tn+1

Trellis Diagram
00 00
a a a a
11 11
11 11
b b b b
10 10
00 00
01 01
c c c c
01 01
d d d d
10 10
tn-1 tn tn+1

Trellis Diagram
00 00 00 00 00
a 00
11 11 11 11
11
b 10 10 10 10
10
01 01
01
01 11 11
11
c 01
00 00 00
01 01 01
d 11
10 10 10
t1 t2 t3 t4 t5 t6

Trellis Diagram
Input is 1 Tracing the Path for an input

Input is 0
1 1 0 1 1
Encoded bits 11 01 01 00 01
00 00 00 00 00
00
11 11 11 11
11
10 10 10 10
10
01 01 01
01 11 11
11
01 00 00 00
01 01 01
11
10 10 10
t1 t2 t3 t4 t5 t6
93
Distance Prop. of Conv. Codes
The free distance, dfree, is the minimum Hamming distance between any two code
sequences.
Convolutional codes are linear. Therefore, the Hamming distance between any pair of
code sequences corresponds to the Hamming distance between the all-zero code
sequence and some nonzero code sequence.
The nonzero sequence of minimum Hamming weight diverges from the all-zero path
at some point and remerges with the all-zero path at some later point.

Repetitive Portion of Trellis : Hamming Weight
00 0
a a
Input is 0
11 2
Input is 1
2
11
b b
10 0
00
01
1
c c
1
01
1
d d
10 1

Distance Prop. of Conv. Codes
0 00 0 00 0 00 0 00 0 00
00
11
2 2 2 11 2 2 11
11 11 11
11 11
10 2 2
1 1 00 00
0 00 0
1
10 10 2 10
1 10
01 0
1 01 1 01 1 01
1
1 01 1 1 01
01
01 1 1
11 t
10 10 1 10 6
t1 t2 t3 t4 t5
The nonzero sequence of minimum Hamming weight diverges from the all-zero path
at some point and remerges with the all-zero path at some later point.
the minimum difference = df = Path metric of that trace with minimum value
Error Correcting Capability

df = Path metric of that trace with minimum value = 5; t = (5-1) / 2 = 2;
Distance Prop. of Conv. Codes (contd)
convolutional code doesn't use blocks, processes a continuous bit stream,
the value of t applies to a quantity of errors located relatively near to each other.
That is, multiple groups of t errors can usually be fixed when they are relatively
far apart.
Free distance can be interpreted as the minimal length of an erroneous "burst" at

the output of a convolutional decoder.

Decoding
Message Bits 1 1 0 1 1
Transmitted Code bits 11 01 01 00 01
Received code bits 11 01 01 10 01
How to get back the message bits

from the received code bits ?

Decoding
Step1. Compute the Hamming distance of the received code word with each of the
branch code words (branch metric) in the trellis.
Rx 11 01 01 10 01
00 2 00 1 00 1 00 1 00 1
00
0 1 1 11 11 1
11 11 11 1
1 1 1
10 10 10 10
1 1 10
2 01 1
01 01
2 2
01 0 11
11 11
0
01 0 2 0
00 00 00
0
2
0
01 01 01
2 0
11
10 10 10 2
t1 t2 t3 t4 t5 t6
Decoding
Rx 11 01 01 10 01
00 2 00 1 00 1 00 1 00 1
00
0 1 1 11 11 1
11 11 11 1
1 1 1
10 10 10 10
1 1 10
2 01 1
01 01
2 2
01 0 11
11 11
0
01 0 2 0
00 00 00
0
2
0
01 01 01
2 0
11
10 10 10 2
t1 t2 t3 t4 t5 t6

Decoding
3. Compute the accumulated path metric G , at each branch, from the branch metric.
Rx 11 01 01 10 01
00 2
00 Ga = 2
11
0
10 Gb = 0
01
11
t1 t2
Decoding
Rx 11 01 01 10 01
00 2 00 1 00 1 00 1 00 1
00
0 1 1 11 11 1
11 11 11 1
1 1 1
10 10 10 10
1 1 10
2 01 1
01 01
2 2
01 0 11
11 11
0
01 0 2 0
00 00 00
0
2
0
01 01 01
2 0
11
10 10 10 2
t1 t2 t3 t4 t5 t6

Decoding
Compute the accumulated path metric G , at each branch, at next stage.

Rx 11 01 01 10 01
00 2 00 1
00 Ga = 3
11
11 1
0
10 Gb = 3
10
2
01
01 Gc = 2
0
11 Gd = 0
t1 t2 t3
Decoding
Rx 11 01 01 10 01
00 2 00 1 00 1 00 1 00 1
00
0 1 1 11 11 1
11 11 11 1
1 1 1
10 10 10 10
1 1 10
2 01 1
01 01
2 2
01 0 11
11 11
0
01 0 2 0
00 00 00
0
2
0
01 01 01
2 0
11
10 10 10 2
t1 t2 t3 t4 t5 t6

Decoding
Add the third stage and Compute the accumulated path metric G .
Eliminate the path/paths that lead to higher accumulated path metric.
Rx 11 01 01 10 01
00 2 00 1 00 1
00 Ga= 4 , 3 Ga= 3
11
11 1
11 1
0
10 1 Gb= 4 , 3 Gb= 3
10
10
11 2
2 1
01 00 0
01 Gc= 5 , 0 Gc= 0
01
0 0
01
10 2 Gd= 2
11 Ga= 3 , 2
t1 t2 t3 t4
Decoding
Decoded Bits 1
Rx 11 01 01 10 01
00 Ga= 3
11
0
10 1 Gb= 3
11
10
2 1
11
01
01 Gc= 0
0
0
01
10 2
11 Ga= 2
t1 t2 t3 t4
Decoding
Decoded Bits 1 Eliminate the path/paths that lead to higher accumulated path metric.
Rx 11 01 01 10 01
Ga= 1
00 1
00 Ga= 4,1
11
11 1
0 1
1 Gb= 1
10 Gb= 4,1
11 11
10 1 10 1
2
11 0
01 00 Gc= 3
01
01 Gc= 3,4
0 2
0 2
01
01
Gc= 2
10 2 0
11 10 Gd= 5,2
t1 t2 t3 t4 t5 t6
Decoding
Decoded Bits 1 Eliminate the path/paths that lead to higher accumulated path metric.
Rx 11 01 01 10 01
Ga= 2
00 1
00
11
11 Ga= 2
1
0
10 1 1
Gb= 2
00 1 G = 2,4
11 b
00 2
11 10
01 00 1
01
01
0 0 Gc= 2
0
01 Gc= 2,3
0
01
Gd= 1
10 2 10 0 10 2
11
Gd= 1,4
t1 t2 t3 t4 t5
108 ELECTRICAL ELECTRONICS COMMUNICATION t
INSTRUMENTATION
Decoding
Trace the survived path and obtain the decoded bits

Decoded Bits 1 1 0 1 1
Rx 11 01 01 10 01
Encoded bits 11 01 01 00 01
00
0
11
10
00
1
01
01
01 0
0
01
0
Gd= 1
11
t1 t2 t3 t4 t5 t6
The Viterbi Algorithm
Walk through the trellis and compute the Hamming distance between received
code word and the branch word in the trellis ( Branch metric) .
Compute the accumulated path metric G , at each branch, from the branch
metric.
At each level, consider the two paths entering the same node and are identical
from this node onwards.
From these two paths, the one that has smallest G at this stage, will still be so at
any time in the future. This path is retained, and the other path is discarded.
Proceeding this way, at each stage one path will be saved for each node. These
paths are called the survivors. The decoded sequence is guaranteed to be one of
these survivors.
Carry out this process until the received sequence is considered completely.
Choose the survivor with the smallest metric.
Implementation of Viterbi Decoder

Implementation of Viterbi Decoder
d aa d ca d ab d cb

Convolutional Encoding & Decoding
Soft & Hard Decoding
Soft Decoding : Involves representing the demodulator outputs into some

quantized level, correspondingly 3 or 4 or ..n bits.
Hard Decoding : Involves representing the demodulator outputs into just 1
bit either 1 or 0
2 level to 8 level quantization provides 2 dB improvement in Eb / N0
Coding Gain

Best Known Convolutional codes

Best Known Convolutional codes

ECE EEE F418 Channel Coding Feb 21, Mar 2, 4, 7,9 2017

Transféré par

Informations du document

Description originale:

Copyright

Formats disponibles

Partager ce document

Partager ou intégrer le document

Options de partage

Avez-vous trouvé ce document utile ?

Ce contenu est-il inapproprié ?

Droits d'auteur :

Formats disponibles

ECE EEE F418 Channel Coding Feb 21, Mar 2, 4, 7,9 2017

Transféré par

Droits d'auteur :

Formats disponibles

ECE / EEE F418

Modern Communication Technologies

T1. DIGITAL COMMUNICATIONS Fundamentals and

R1. Communication Systems Engineering, John G Proakis

What is Channel coding ?

Why Channel Coding ?

2 ELECTRICAL ELECTRONICS COMMUNICATION INSTRUMENTATION

Which one is better and in what sense?

The mapping can be accomplished via a lookup table.

Q1. Since 2n >> 2k. Then, which 2k out of 2n to be used ?

Q2. For large (n, k) is it possible to have lookup table implementation ?.

4 ELECTRICAL ELECTRONICS COMMUNICATION INSTRUMENTATION

The set of all 2n n-tuples, Vn , form an n-dimensional vector space over

Code words are a subset S of the vector space Vn satisfies the

Rule 1: The all-zeros vector is in S

Rule 2: The sum of any two vectors in S, is also a vector in S. ( Closure

5 ELECTRICAL ELECTRONICS COMMUNICATION INSTRUMENTATION

Ex: Consider a 24 tuples vector space

Does the subset form the subspace ?

Apply Rules 1 & 2 to verify.

Rule 1: The all-zeros vector is in S

Rule 2: The sum of any two vectors in S, is also a vector in S. ( Closure

6 ELECTRICAL ELECTRONICS COMMUNICATION INSTRUMENTATION

How to obtain the subspace (Linear Block code)?

7 ELECTRICAL ELECTRONICS COMMUNICATION INSTRUMENTATION

Any basis set of k linearly independent n-tuples, say

How? : Because, each of the 2k code vectors is a linear combination of

We can use the k message bits as coefficients for linear combination !!

Let the message bits be: m1, m2, , mk and

Then one of the ( 1 X n) Code Word can be U = m1 V1 + m2 V2 + , mk Vk

9 ELECTRICAL ELECTRONICS COMMUNICATION INSTRUMENTATION

Ex: (6,3) Code, Let

V1 = [ 1 1 0 1 0 0 ] Are these vectors linearly independent ?

V2 = [ 0 1 1 0 1 0 ] (That is , Does any linear combination of two

For the message (1 1 0), the code vector is

For the message (0 1 1), the code vector is U4 = [ 1 1 0 0 1 1 ]

Let us define a (k X n) generator matrix,

Let the message vector (1 X k) be: m = m1, m2, , mk

The code vector, corresponding to a

11 ELECTRICAL ELECTRONICS COMMUNICATION INSTRUMENTATION

Ex: (6,3) Code

Let the generator matrix be

For the message (1 1 0), the message vector (1 X k) : m7 = [ 1 1 0 ]

12 ELECTRICAL ELECTRONICS COMMUNICATION INSTRUMENTATION

For all The 23, 3-bit Messages

Minimum Distance of a code: Each Error detection capability : dmin-1 = 2

Code can be used for simultaneous

a = 1 and b = 1 satisfy the above.

How is ( dmin) related to (n, k) ? ( Plotkin Bound)

Plotkin bound dictates bound on ( dmin)

In the previous (6,3) code, The bound dictates as

dmin (6 X 4) /(8 1) = 3. 42 dmin = 3

15 ELECTRICAL ELECTRONICS COMMUNICATION INSTRUMENTATION

How are the number of correctable errors t related to (n,k) ?

( Hamming Bound) Log2 { 1 + nC1 + nC2 + ... + nCt } n - k

In the previous (6,3) code, for 1 error correction

Log2 { 1 + 6C1} = 2.8 is (6 3)

for 2 error corrections Log2 { 1 + 6C1 + 6C2 } = 4.5 not (6 3)

While designing a code, Hamming & Plotkin bounds have to be met.

16 ELECTRICAL ELECTRONICS COMMUNICATION INSTRUMENTATION

A desirable property for a linear block code is to possess a systematic

17 ELECTRICAL ELECTRONICS COMMUNICATION INSTRUMENTATION

Since, And message vector m = m1, m2, , mk