The Role of Specialization in LDPC Codes

Jeremy Thorpe Pizza Meeting Talk 2/ 2/!"

Talk #$er$ie%
LDPC Codes Message Passing Decoding &nalysis of Message Passing Decoding 'Density ($ol)tion* &ppro+imations to Density ($ol)tion Design of LDPC Codes )sing D,(,

The Channel Coding Strategy

(ncoder chooses the mth code%ord in code-ook C and transmits it across the channel Decoder o-ser$es the channel o)tp)t y and generates m. -ased on the kno%ledge of the code-ook C and the channel statistics,
m 1!0 /k

xC X n

(ncoder Channel Decoder

m2 1!0 /k y Y n

Linear Codes
& linear code C 'o$er a finite field* can -e defined in terms of either a generator matri+ or parity3check matri+, 4enerator matri+ 4 'k5n*

C = 1mG/

Parity3check matri+ 6 'n3k5n*

C = 1c 7 cH2 = 0/

LDPC Codes
LDPC Codes 33 linear codes defined in terms of 6, 6 has a small a$erage n)m-er of non3zero elements per ro%
! ! ! ! ! ! ! ! ! ! ! H= ! ! ! ! ! ! ! ! ! ! ! ! ! ! !

4raph Representation of LDPC Codes

6 is represented -y a -ipartite graph, There is an edge from $ to c if and only if7
Variable nodes

H 'v0 c * !
& code%ord is an assignment of $2s s,t,7
...

...

x
v8c

= !c

Check nodes

Reg)lar '90:* LDPC codes

($ery $aria-le node Variable nodes has degree 90 e$ery check node has degree :, (nsem-le is defined -y matching left edge ;st)-s; %ith right edge ;st)-s $ia a random Check nodes perm)tation
... ...

Message3Passing Decoding of LDPC Codes

Message Passing 'or <elief Propagation* decoding is a lo%3comple+ity algorithm %hich appro+imately ans%ers the =)estion >%hat is the most likely x gi$en y?@ MP rec)rsi$ely defines messages m$0c'i* and mc0$'i* from each node $aria-le node v to each adAacent check node c0 for iteration iB!0 0,,,

T%o Types of Messages,,,

Liklihood Ratio
x 0 y =
p' y 8 x = * p ' y 8 x = !*

Pro-a-ility Difference
x 0 y = p' x = 8 y * p' x = ! 8 y *

Cor y 0,,,yn independent conditionally on +7

x 0 y n = x 0 yi
i

Cor + 0,,,+n independent7

xi 0 y = xi 0 y
i i

,,,Related -y the <iliniear Transform

Definition7
x B' x* = +x
p' y 8 x = * * p ' y 8 x = !* p' y 8 x = !* p ' y 8 x = * = p ' y 8 x = !* + p' y 8 x = * p ' y 8 x = !* p ' y 8 x = * = 2 p' y * 2 p' x = ! 8 y * p' y * 2 p' x = 8 y * p' y * = 2 p' y * = p' x = ! 8 y * p' x = 8 y * = x0 y B ' x 0 y * = B '

Properties7
B ' B' x** = x B ' x 0 y * = x 0 y B' x 0 y * = x 0 y

Message Domains
robability !i""erence
' x = ! 8 y* ' x = 8 y*

B '*

Likelihood Ratio
' y 8 x = !* 'y 8 x = *

Log rob. !i""erence

B2 '*

Log Likelihood Ratio

Daria-le to Check Messages

#n any iteration i0 the message from $ to c is7
'e 8 x = * 'e 8 x = !*
... v c ...

Et is comp)ted like7
mv 0c
'i *

= xv 0 yv B 'mc 20v
c 28v c

'i *

Check to Daria-le Messages

#n any iteration0 the message from c to $ is7 ' v = ! 8 e* ' v = 8 e * Et is comp)ted like7
mc 0v
'i *

v c ... ...

&ss)mption7 Encoming messages are indep,

v 28c v

B'm

'i *

v 20 c

Decision R)le
&fter s)fficiently many iterations0 ret)rn the likelihood ratio7
!0 if x 0 y B 'mc 0v ' i * * < ! v v F= c 8v x other%ise 0

Theorem a-o)t MP &lgorithm

Ef the algorithm stops after r iterations0 then the algorithm ret)rns the ma+im)m a posteriori pro-a-ility estimate of xv gi$en y %ithin radi)s r of v, 6o%e$er0 the $aria-les %ithin a radi)s r of v m)st -e dependent only -y the e=)ations %ithin radi)s r of v0 r

... ...

...

&nalysis of Message Passing Decoding 'Density ($ol)tion*

in Density ($ol)tion %e keep track of message densities0 rather than the densities themsel$es, &t each iteration0 %e a$erage o$er all of the edges %hich are connected -y a perm)tation, Ge ass)me that the all3zeros code%ord %as transmitted '%hich re=)ires that the channel -e symmetric*,

D,(, Hpdate R)le

The )pdate r)le for Density ($ol)tion is defined in the additive domain of each type of node, Ghereas in <,P0 %e add 'log* messages7
mc 0v
'i *

En D,(0 %e con$ol$e message densities7

! 'mc 0v * =
'i * v 28c v

v 28c v

B2 'm

'i *

v 20c

*
'i *

I B2 ' !'m

v 20c

**

Camiliar (+ample7
Ef one die has density f)nction gi$en -y7
# ( $ ' & %

The density f)nction for the s)m of t%o dice is gi$en -y the con$ol)tion7

( $ ' & % ) * , #+ ###(

D,(, Threshold
Ci+ing the channel message densities0 the message densities %ill either ;con$erge; to min)s infinity0 or they %on2t, Cor the ga)ssian channel0 the smallest 'infim)m* SJR for %hich the densities con$erge is called the density e$ol)tion threshold,

Reg)lar '90:* LDPC codes

($ery $aria-le node has degree 90 e$ery check node has degree :, <est rate /2 code is '"0K*0 %ith threshold ,!L d<, This code had -een in$ented -y LK2 -y Ro-ert 4allager,

D,(, Sim)lation of '"0K* codes

Set SJR to , 2 d< ',!" a-o$e threshold* Gatch fraction of ;erroneo)s messages; from check to $aria-le, 'note that this fraction does not characterize the distri-)tion f)lly*

Empro$ement $s, c)rrent error fraction for Reg)lar '"0K*

Empro$ement per iteration is plotted against c)rrent error fraction, Jote there is a single -ottleneck %hich took most of the decoding iterations,

Erreg)lar '90 :* LDPC codes

a fraction 9i of $aria-le Variable nodes nodes ha$e degree i, :i -( of check nodes ha$e .' degree i, -$ (dges are connected -y a single random perm)tation, .m -n Jodes ha$e -ecome Check nodes specialized,
... ...

D,(, Sim)lation of Erreg)lar Codes 'Ma+im)m degree !*

Set SJR to !,M2 d< 'N,!" a-o$e threshold* Gatch fraction of erroneo)s check to $aria-le messages, This Code %as designed -y Richardson et, al,

Comparison of Reg)lar and Erreg)lar codes

Jotice that the Erreg)lar graph is m)ch flatter, Jote7 Capacity achie$ing LDPC codes for the eras)re channel %ere designed -y making this line e+actly flat,

M)lti3edge3type constr)ction
(dges of a partic)lar ;color; are connected thro)gh a perm)tation, (dges -ecome specialized, (ach edge type has a different message distri-)tion each iteration,

D,(, of M(T codes,

Cor M)lti3edge3type codes0 Density e$ol)tion tracks the density of each type of message separately, Comparison %as made to real decoding0 ne+t slide 'co)rtesy of Oen &ndre%s*,

M(T D,(, $s, decoder sim)lation

Reg)lar0 Erreg)lar0 M(T comparison

M)lti3edge3type LDPC codes impro$e grad)ally thro)gh most of the decoding, M(T codes ha$e a threshold -elo% the more ;comple+; irreg)lar codes,

Design of Codes )sing D,(,

Density e$ol)tion pro$ides a moderately fast %ay to e$al)ate single3 and m)lti3 edge type degree distri-)tions 'hypothesis3testing*, M)ch faster appro+imations to Density ($ol)tion can easily -e p)t into an o)ter loop %hich performs f)nction minimization, Semi3&nalytic techni=)es e+ist as %ell,

Re$ie%
Eterati$e Message Passing can -e )sed to decode LDPC codes, Density ($ol)tion can -e )sed to predict the performance of MP for infinitely large codes, More sophisticated classes of codes can -e designed to close the gap to the Shannon limit,

<eyond M(T Codes 'f)t)re %ork*

Traditional LDPC codes are designed in t%o stages7 design of the degree distri-)tion and design of the specific graph, Hsing $ery fast and acc)rate appro+imations to density e$ol)tion0 %e can e$al)ate the effect of placing or remo$ing a single edge in the graph, Hsing an e$ol)tionary algorithm like Sim)lated &nnealing0 %e can optimize the graph directly0 changing the degree profile as needed,