Parallel Concatenated Codes

– “Turbo Codes”

Ingredients
Parallel Code Concatenation
Iterative Decoder (“Turbo Decoder”)

Advanced Channel Coding – p.1

Motivation

Product codes
Encoder: Arrangement of info symbols in matrix,
horizontal and vertical encoding
Decoder: Iteration between vertical and horizontal
decoding

Generalization of these concepts

Parallel Concatenated codes

Encoder: Encoding of original and of interleaved
(permuted) info word
Decoder: Iteration between constituent decoders using
soft-values

Advanced Channel Coding – p.2

Ingredients

Recursive systematic convolutional (RSC) encoders

Soft-in soft-out decoders

Interleavers

Advanced Channel Coding – p.3

Convolutional Encoder

Example: (7, 5)8 convolutional encoder

xt,1

ut xt,1 xt,2
D D
xt,2

Definition via generator polynomials:

“7” → g1 (D) = 1 + D + D2
“5” → g2 (D) = 1 + D2

Memory length m = 2
Encoding rate R• = 1/2

▽Advanced Channel Coding – p.4

Convolutional Encoder

Example: (7, 5)8 convolutional encoder

xt,1

ut st,1 st,2 xt,1 xt,2

xt,2

Definition via generator polynomials:

“7” → g1 (D) = 1 + D + D2
“5” → g2 (D) = 1 + D2

Memory length m = 2
Encoding rate R• = 1/2
Encoder state st = [st,1 st,2 ]
Advanced Channel Coding – p.4
Convolutional Encoder

Trellis Diagram
00 00 00 00 00 00
00
11 11 11 11
01 01 01 01 01
01
10 10 10 10
11 11 11 11 11 11
10
00 00 00 00
10 10 10 10 10
11
01 01 01 01

ut = 0: solid line; ut = 1: dashed line

Encoder starts in the zero state
Error event: detour from the all-zero path
(zero-state at the beginning and the end but not in between)
Free distance dfree = 5:
minimum weight of any single error event

Advanced Channel Coding – p.5

Convolutional Encoder

Encoder starts and ends in the zero-state (“trellis termination”)

00 00 00 00 00 00
00
11 11 11 11
01 01 01 01
01
10 10 10
11 11 11 11
10
00 00
10 10 10
11
01 01

Information word length K=4

Trellis length T =K+m=6
Codeword length N = (K + m)/R• = 12
Code rate R = R• K+m
K
= 1/2 · 4/6 = 4/12
Minimum distance dmin = dfree

Advanced Channel Coding – p.6

Convolutional Encoder

Formulation of Encoding via Polynomials

Information-word polynomial:
u(D) = u1 + u2 D + u3 D2 + u4 D3

Generator polynomials:
g1 (D) = 1 + D + D2
g2 (D) = 1 + D2

Code-word polynomials:
x1 (D) = u(D) g1 (D) = x1,1 + x2,1 D2 + · · · + x6,1 D5
x2 (D) = u(D) g2 (D) = x1,2 + x2,2 D2 + · · · + x6,2 D5

Advanced Channel Coding – p.7

RSC Encoder

Recursive Systematic Convolutional Encoder

Example: (1, 5/7)8 convolutional encoder
xt,1

ut st,1 st,2

xt,2

Definition via generator functions:

“1” → g1 (D) = 1
1 + D2
“5/7” → g2 (D) =
1 + D + D2

Advanced Channel Coding – p.8

RSC Encoder

Properties of an RSC Encoder

Generates the same code as a non-recursive encoder

Information words of weight 1 generate large (infinite)
codeword weights
Finite codeword weights require information weights of at
least 2

Effective free distance df,eff :

minimum codeword weight for information weight 2

Advanced Channel Coding – p.9

APP decoding

APP decoding
For each info symbol, the decoder computes the
a-posteriori probabilities:

X N
Y −1
Pr(Uk = 0|y) = α· p(yn |xn )
x∈C: n=0
uk =0

X N
Y −1
Pr(Uk = 1|y) = α· p(yn |xn )
x∈C: n=0
uk =1

BCJR algorithm
Bahl, Cocke, Jelinek, and Raviv found an efficient way to
compute the APPs on the code trellis (similar to the Viterbi
algorithm)
Advanced Channel Coding – p.10
LogAPP decoding

LogAPP decoding
For each info symbol, the decoder computes the
a-posteriori L-value:

Pr(Uk = 0|y)
L(Uk |y) = ln
Pr(Uk = 1|y)

The logarithmic implementation of the BCJR algorithm has

numerical advantages and operates on L-values and
log-probabilities.
A sub-optimum variant of LogAPP decoding is MaxLogAPP
decoding.

Remark: Using the approximation of boxplus is MaxLogAPP

decoding of single-parity-check codes.

Advanced Channel Coding – p.11

Generalized Decoding Model

lU,p
z lU,ch
Ch1 L lU,e
Decoder
lX,p
u x y lX,ch
Encoder Ch2 L lX,e

lUk ,ch = L(Uk |zk ) lUk ,p = L(Uk |z, y) = L(Uk |lU,ch , lX,ch )
lUk ,e = L(Uk |z \k , y) = L(Uk |lU,ch,\k , lX,ch )
lXn ,ch = L(Xn |yn ) lXn ,p = L(Xn |z, y) = L(Xn |lU,ch , lX,ch )
lXn ,e = L(Xn |z, y \n ) = L(Xn |lU,ch , lX,ch,\n )

Advanced Channel Coding – p.12

Generalized Decoding Model

Relation to “Conventional” Decoding Model

Extension of encoder by a systematic branch

(an RSC encoder is extended by an additional(!)
systematic branch)

xt,0
xt,1

ut st,1 st,2

xt,2

“Conventional” decoding of virtual R• = 1/3 encoder

Notice: A new algorithm is not necessary!

Advanced Channel Coding – p.13

Puncturing

Puncturing may be used to increase the code/encoder rate

Operation at the encoder side:

puncturing (removing) of some of the symbols

Operation at the decoder side:

(i) insertion of L-value zero (“don’t know nothing”) at the
positions of the punctured symbols
(ii) use of the decoder for the original code/encoder

Advantage: the code rate can be increased

while the same devices for encoding and de-
coding are used

Advanced Channel Coding – p.14

Interleavers

An interleaver permutes the order of a sequence of

symbols. The corresponding de-interleaver performs the
reverse operation.
Some authors prefer the term permuter. (Which is actually a
better name.)
Formulation using a function π(.):
original sequence u = [u0 , u1 , u2 , u3 ]
permuted sequence v = [uπ(0) , uπ(1) , uπ(2) , uπ(2) ]
Formulation using a matrix P :
 
0 1 0 0
0 0 0 1
[v0 , v1 , v2 , v3 ] = [u0 , u1 , u2 , u3 ] · 
0

0 1 0
1 0 0 0

Advanced Channel Coding – p.15

Types of Interleavers

Block Interleaver
Symbols are written into a matrix column-wise and read out
row-wise

Random Interleaver
The permutation pattern is chosen randomly
(but only once then it is fixed, of course)

S-rand Interleaver
The permutation pattern is chosen randomly with a
“spreading” constraint given by the parameter s:

|π(i) − π(k)| > s for all i, k with |i − k| < s

Advanced Channel Coding – p.16

Parallel Concatenated Codes

Question:
Given two LogAPP-decoder modules ...
What can you do with them?

▽Advanced Channel Coding – p.17

Parallel Concatenated Codes

Question:
Given two LogAPP-decoder modules ...
What can you do with them?

Answer:
Invent turbo codes.

▽Advanced Channel Coding – p.17

Parallel Concatenated Codes

Question:
Given two LogAPP-decoder modules ...
What can you do with them?

Answer:
Invent turbo codes.

Next Question:
What encoder constructions are reasonable?

Advanced Channel Coding – p.17

PC Codes - Encoder

u = u(1) x(1)
ENC1
P x
/
u(2) x(2) S
π ENC2

Typical Settings

Encoder 1: rate-1/2 RSC convolutional encoder

Encoder 2: rate-1 RSC convolutional encoder
(without systematic output)
Parallel-to-serial device includes puncturing
to increase rate from 1/3 to 1/2

Advanced Channel Coding – p.18

PC Codes - Decoder

l(1) = l û

(1)
lch a(1)
DEC1
lch S e(1)
/
P
(2)
lch a(2)
π
DEC2
π −1
e(2)
Typical Settings

Serial-to-parallel device includes insertion of

L-values being zero for punctured symbols
Decoder 1 and Decoder 2 are LogAPP decoders

▽Advanced Channel Coding – p.19

PC Codes - Decoder

l(1) = l û

(1)
lch a(1)
DEC1
lch S e(1)
/
P
(2)
lch a(2)
π
DEC2
π −1
e(2)
L-values for Decoder 1 (for Decoder 2 analogous)
(1) (1) (1) (1) (1)
ln,ch = L(Xn(1) |yn(1) ) ek = L(Uk | lch , a\k )

(1) (1) (1)

lk = L(Uk | lch , a(1) )

Advanced Channel Coding – p.19

PC Codes - Decoder

Some Remarks

The constituent decoders perform local decoding,

i.e., they know only their code constraints.

The LogAPP decoder 1 computes correct L-values only if all

input L-values are conditionally independent, i.e., if
(1)
N K
(1) (1) (1) (1)
Y Y
p(lch , a(1) | x(1) , u(1) ) = p(ln,ch |x(1)
n )· p(ak |uk )
n=1 k=1

(analogous for decoder 2)

As the independence is fulfilled only for the first few

iterations, the decoder outputs are only approximate
L-values.

Advanced Channel Coding – p.20

PC Codes - Design Issues

Constituent encoders

Interleaver (spreading properties)

Puncturing pattern

Constituent decoders (LogAPP, MaxLogAPP, ...)

Number of iterations / stopping criteria

Advanced Channel Coding – p.21

The Original Turbo Encoder

Berrou, Glavieux, Thitimajshima, 1993.

Advanced Channel Coding – p.22
The Original Turbo Decoder

Berrou, Glavieux, Thitimajshima, 1993.

Advanced Channel Coding – p.23

Performance of the First Turbo-Code

Berrou, Glavieux, Thitimajshima, 1993.

Advanced Channel Coding – p.24