LOW DENSITY PARITY CHECK CODES

PRESENTATION OVERVIEW
Digital Communication System
Why Coding?
Shannons Coding Theorems
Error Correction Codes
LDPC Codes
Summery
DIGITAL COMMUNICATION SYSTEM
Information
Source
Source
Encoder
Channel
Encoder
Modulator
Channel
Demodu-
lator
Channel
Decoder
Source
Decoder
Data Sink
r
b
r
c

r
s

JPEG, MPEG,
etc.
RS code, Turbo
code, LDPC
QPSK, QAM,
BPSK, etc.
CHANNEL CODING
Channel encoding : The application of redundant symbols
to correct data errors.
Modulation : Conversion of symbols to a waveform for
transmission.
Demodulation : Conversion of the waveform back to
symbols, usually one at a time.
Decoding: Using the redundant symbols to correct errors.
WHAT IS CODING?
Coding is the conversion of information to another form for some
purpose.
Source Coding : The purpose is lowering the redundancy in the
information. (e.g. ZIP, JPEG, MPEG2)
Channel Coding : The purpose is to defeat channel noise.

WHY CHANNEL CODING?
Trade-off between Bandwidth, Energy and Complexity.
Coding provides the means of patterning signals so as to reduce
their energy or bandwidth consumption for a given error
performance.

CHANNELS
The Binary Symmetric Channel(BSC)
The Binary Erasure Channel (BEC)
HOW TO EVALUATE CODE PERFORMANCE?
Need to consider Code Rate (R), SNR (Eb/No), and Bit Error Rate
(BER).
Coding Gain is the saving in Eb/No required to achieve a given BER
when coding is used vs. that with no coding.
Generally the lower the code rate, the higher the coding gain.
Better Codes provides better coding gains.
Better Codes usually are more complicated and have higher
complexity.

SHANNONS CODING THEOREMS
If C is a code with rate R>C, then the probability of error
in decoding this code is bounded away from 0. (In other
words, at any rate R>C, reliable communication is not
possible.)
It Tells the maximum rate at which information can
be transmitted over a communications channel of a
specified bandwidth in the presence of noise.

SHANNONS CODING THEOREMS
STATEMENT OF THE THEOREM
Where,
C is the channel capacity in bits per second.
B is the bandwidth of the channel in hertz.
S is the average received signal power over the bandwidth
N is the average noise or interference power over the bandwidth
S/N is the signal-to-noise ratio (SNR) or the carrier-to-noise ratio (CNR) of
the communication signal.
COMMON ERROR CORRECTION CODES
Convolutional Codes
Block Codes (e.g. Reed-Solomon Code)
Trellis-Coded-Modulation (TCM)
Concatenated Codes
Low density Parity Check Codes


THE ERROR-CONTROL PARADIGM
Noisy channels give rise to data errors: transmission or storage systems
Need powerful error-control coding (ECC) schemes: like linear or non-linear
Linear EC Codes: Generated through simple generator or parity-check matrix



Binary information vector (length k)
Code vector (word): (length n)
Key property: Minimum distance of the code, , smallest separation between two
codewords.
Rate of the code R= k/n
1 0 0 0 1 0 1
1 1 1 0 1 0 0
0 1 0 0 1 1 0
0 1 1 1 0 1 0
0 0 1 0 1 1 1
1 0 1 1 0 0 1
0 0 0 1 0 1 1
G H
(
(
(
(
(
= =
(
(
(
(

(

min
d
1 2 3 4
( , , , ) u u u u u =
, 0
T
x uG Hx = =
Binary linear codes:
HISTORY OF LDPC CODES

Low Density Parity Check Code
A class of Linear Block Codes
Invented by Robert Gallager in his 1960 MIT Ph. D. dissertation
Being ignored for long time due to
Requirement of high complexity computation
Introduction of Reed-Solomon codes and Turbo Codes
The concatenated RS and convolutional codes were considered perfectly suitable
for error control coding
Rediscovered by MacKay(1996) and Richardson/Urbanke(1998).

FEATURES OF LDPC CODES

Approaching Shannon capacity
For example, 0.3 dB from Shannon limit
Irregular LDPC code with code length 1 million. (Richardson:1999)
An closer design from (Chung:2001), 0.04 dB away from capacity
(block size of 10
7
bits, 1000 iterations)
Good block error correcting performance.
Decoding complexity is reduced.
Suitable for parallel implementation

FEATURES OF LDPC CODES

BER

LOW-DENSITY PARITY-CHECK CODES
Regular LDPC codes
- All rows and columns contain the same number of ones

Irregular LDPC codes
- The reverse of the statement of regular LDPC codes

Regular LDPC codes are the primary concern in this presentation


LINEAR BLOCK CODES
A Linear Code can be described by a generator matrix G or a parity
check matrix H.
A (N,K) block encoder accepts K-bit input and produces N-bit
codeword
c= xG, and cH
T
= 0
where c = codeword, x = information
G = Generator matrix, H = parity check matrix
PROPERTIES OF LDPC CODES

LDPC codes are defined by a sparse parity-check matrix
Parity Check Matrix (H) for decoding is sparse
Very few 1's in each row and column.
Expected large minimum distance.
Regular LDPC codes
H: m x n where (n-m) information bits are encoded into n codewords
H contains exactly W
c
1's per column and exactly W
r
= W
c
(n/m) 1's per row, where W
c

<< m.
The above definition implies that W
r
<< n.
W
c
3 is necessary for good codes.
If the number of 1's per column or row is not constant, the code is an irregular LDPC
code.
Usually irregular LDPC codes outperforms regular LDPC codes.

A SIMPLE LDPC CODE

Parity Check Matrix
W
c
= 3
n = 10 and m = 5
W
r
= 3 * (10/5) = 6

G can be found by Gaussian
elimination.

H

SIMPLE REGULAR LDPC CODES
We can represent a LDPC code in two ways
Tanner graph
Matrix
REPRESENTING REGULAR LDPC CODES
notation and its length N
: the number of ones in a column
: the number of ones in a row

If the dimension of H is ,

: the number of 1s in H

Rate R =

) , (
c v
d d
N M
v c
Nd Md =
N M d d
c v
/ 1 / 1 =
v
d
c
d
BIPARTITE GRAPH REPRESENTATION
N variable nodes
- circular shape
- each connected with check nodes

M check nodes
- rectangular shape
- each connected with variable nodes

Edge e = {v,c}
N i v
i
, , 1 , =
v
d
c
d
M j c
j
, , 1 , =
ENCODING OF LDPC CODES

General encoding of systematic linear block codes


Issues with LDPC codes
The size of G is very large.
G is not generally sparse.
Example: A (10000, 5000) LDPC code.
P is 5000 x5000.
If we assume that the density of 1's in P is 0.5
There are 12.5x10
6
1's in P
12.5x10
6
addition operations are required to encode one code word.
An alternative approach to simplified encoding is to design the LDPC code via algebraic
or geometric methods
Such structured codes can be encoded with shift registers
DECODING OF LDPC CODES
General decoding of linear block codes
Only if c is a valid code word, we have
c H
T
= 0
For binary symmetric channel (BSC), the received code word is c added with
an error vector e
The decoder needs to find out e and flip the corresponding bits
The decoding algorithm is usually based on linear algebra
Graph-based Algorithms
Sum-product algorithm for general graph-based codes
MAP algorithm for trellis graph-based codes
Message passing algorithm for bipartite graph-based codes
We have code bits x = {x1,x2,x3.xn}
We have observation bits y={y1,y2,y3yn}
We have the probability between x and y
f(x,y)=f(x1,x2,x3.xn,y1,y2,y3yn)
=f(y1,y2,y3yn|x1,x2,x3..xn)f(x1,x2,x3.xn)
For discrete memor yless channel
f(x,y)= f(yi|xi)f(x

=1
)
X1 X2 X3 X4 X5 X6 X7
Y1 Y2 Y3 Y4 Y5 Y6 Y7
H3 H2 H1
1
1
1
1
1
1
1
1
1
1
1
0 0
0
0
0 0
0
0
0
H1
X3 X2 X5 X4 X7 X6 X1
1
H2
H3
H =
1
1
1
1
1
1
1
1
1
1
1
H1
X3 X2 X5 X4 X7 X6 X1
1
H2
H3
H =
X1
Y1
1 2 3
X2
Y2
4 5 6
X3
Y3
7 8 9
XN
YN
n-2 n-1
n
H1 H2 H3 Hm
dV = Weight of the variable Nodes.
dC = Weight of the check Nodes.
i(xi)dv-1(xi) is incoming message bits from yi.
Xi is identical to yi(xi)
out(xi)=product of all incoming messages
out(xi)=f(yi|xi) i(xi)
1
=1


Xi


2(Xi)
Yi
OUT(Xi)
Yi(Xi)
Observations
dv-1
i(xi)dv-1(xi) is incoming message bits.
Xi is variable code.
out(xout)= i(xi)(1, 2, 3, . . 1. . )
1
=1

1


X1
X2
Xdv


XOUT
OUT(XOUT)

(X1+x2+x3+x4+..xdc-1+xout)= 0
I(Xi,x2Xout) = 1, check is satisfied
0, otherwise
MESSAGE PASSING SCHEDULES
Iterative Decoding Procedure
1. What order we have to star the procedure.
2. What order we have to pass the message.
3. How do we know when to stop.
Initially we get yi and Lc
Execute the sum product algorithm on each variable and pass messages to
all attachment checks.
Exectute the sum product algorithm each parity check, pass msgs to all
attachement variables
Goto Start until stop
Stop if
1. Valid Code word is found
2. Maximum number of Iterations reached.
SUM PRODUCT ALGORITHM AT VARIABLE NODE
Xi


L2
Yi
LOUT
Yi(Xi)
Observations
dv-1
Check
Lout = Lc +
1
=1

SUM PRODUCT ALGORITHM AT CHECK NODE
L2
LOUT dc-1
Lout = 2 tanh
-1
(tanh(

2
)
1
=1

HOW DOES STANDARD MESSAGE PASSING ALGORITHM WORK?
Variable nodes
.
..
. . . . . . . . .
.
check nodes
.
error bits
?
HOW DOES STANDARD MESSAGE PASSING ALGORITHM WORK?
variable nodes
.
..
. . . . . . . . .
.
check nodes
.
Lout = 2 tanh
-1
(tanh(

2
)
1
=1

ADVANTAGES OF LDPC BLOCK CODES
These are suited for implementations that make heavy use of
parallelism Consequently, error-correcting codes with very long
code lengths are feasible.
No tail bits are required for block coding providing additional bits for
data transmission.
LDPC has excellent BER performance under AWGN.
LDPC is extremely useful for large values of data bits N.

SUMMARY
Low-density-parity-check codes have been studied a lot in the last
years and huge progresses have been made in the understanding
and ability to design iterative coding systems.
The iterative decoding approach is already used in turbo codes but
the structure of LDPC codes give even better results.
In many cases they allow a higher code rate and also a lower error
floor rate.
The main disadvantages are that encoders are somehow more
complex and that the code length has to be rather long to yield good
results.
RESEARCH OPPORTUNITY
LDPC Code Design
Efficient Implementation on FPGA
LDPC Application for next generation communication systems
(Wireless, OFDM, ADSL).

REFERENCES
http://www.csee.wvu.edu/wcrl/ldpc.htm
R. G. Gallager, Low-Density Parity Check Codes. Cambridge, MA: MIT Press, 1963.
LDPC Codes: An Introduction by Amin Shokrollahi (Digital Fountain, Inc.)
T. Richardson and R. Urbanke, The capacity of low-density parity-check codes under message-
passing decoding, IEEE Trans. Inform. Theory, Feb. 2001.

THANK YOU!