IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. 57, NO.

9, SEPTEMBER 2009

2505

Transactions Letters
Improving the Distance Properties of Turbo Codes Using a Third Component Code: 3D Turbo Codes
Claude Berrou, Fellow, IEEE, Alexandre Graell i Amat, Member, IEEE, Youssouf Ould-Cheikh-Mouhamedou, Member, IEEE, and Yannick Saouter, Member, IEEE

AbstractThanks to the probabilistic message passing performed between its component decoders, a turbo decoder is able to provide strong error correction close to the theoretical limit. However, the minimum Hamming distance (min ) of a turbo code may not be sufciently large to ensure large asymptotic gains at very low error rates (the so-called attening effect). Increasing the min of a turbo code may involve using component encoders with a large number of states, devising more sophisticated internal permutations, or increasing the number of component encoders. This paper addresses the latter option and proposes a modied turbo code in which a fraction of the parity bits are encoded by a rate-1, third encoder. The result is a noticeably increased min , which improves turbo decoder performance at low error rates. Performance comparisons with turbo codes and serially concatenated convolutional codes are given. Index TermsChannel coding, iterative decoding, parallel concatenation, serial concatenation, turbo codes.

I. I NTRODUCTION

URBO codes (TCs) are today mainly used in Automatic ReQuest (ARQ) systems, which do not usually require very low error rates. Target Frame Error Rates (FER) from 102 to 105 are typical for this kind of communication systems. However, in future communication system generations, lower FER, down to 108 , may be necessary to open the way to real-time and more demanding applications, such as TV broadcasting or videoconferencing. The minimum Hamming distance (min ) of a TC may not be sufciently large to offer such error correction at the required signal to noise ratio. For the current commercial applications of TCs (3G, DVB-RCS, WiMax...), commonly based on 8-state component encoders, there are several ways to increase min and thereby improving the performance at very low error rates. For instance, one
Paper approved by T. M. Duman, the Editor for Coding Theory and Applications of the IEEE Communications Society. Manuscript received October 10, 2007; revised August 13, 2008. C. Berrou, A. Graell i Amat, and Y. Saouter are with Institut TELECOM, TELECOM Bretagne, UMR CNRS Lab-STICC, Universite Europeenne de Bretagne, CS 83819 29238 Brest Cedex 3, France (e-mail: e-mail: {claude.berrou, alexandre.graell, yannick.saouter}@telecom-bretagne.eu). Y. Ould-Cheikh-Mouhamedou is with the Prince Sultan Advanced Technologies Research Institute (PSATRI), King Saud University, Riyadh, Saudi Arabia (e-mail: ycheikh@ksu.edu.sa). The material in this paper was presented in part at the IEEE Information Theory Workshop (ITW), Lake Tahoe, CA, US, Sept. 2007. A. Graell i Amat was supported by a Marie Curie Intra-European Fellowship within the 6th European Community Framework Programme. Y. OuldCheikh-Mouhamedou was supported by Region Bretagne. Digital Object Identier 10.1109/TCOMM.2009.09.070521

might use stronger component codes, e.g. 16-state codes, at the price of doubling the decoding complexity. Devising more appropriate internal permutations [1, 2] is an appealing alternative to improve min , since it does not incur any complexity increase. Unfortunately, designing such powerful permutations is not an easy task, and there are limits to the min and multiplicity values, and thus to the performance improvements, that can be achieved. Another way to improve min which has been widely explored in the literature, is to concatenate the component encoders in series rather than in parallel [3]. Serially concatenated codes (SCCs) yield higher minimum distances compared than parallel concatenation, but shows a penalty in convergence threshold, which might be unacceptable for several applications. On the other hand, irregular SCCs have been shown to achieve near-capacity convergence thresholds, but at expense of a poorer error oor [4]. Mixed structures, like those proposed in [57], are also possible, combining the features of the two concatenations. Finally, multiple concatenation using an increased number of component encoders, can be used to eliminate low-weight codewords and so improve the distance properties of the code. This paper addresses the latter alternative to improve the minimum distance of TCs by proposing a three-dimension TC (3D-TC). The 3D-TC we describe here is inspired by the contributions in [6] and [7] and calls for both parallel and serial concatenation in an original approach: the proposed 3DTC is simply derived from the classical TC by adding a partial rate-1 third dimension. A rate-1 post-encoder is concatenated at the output of the parent turbo code, encoding only a fraction of the parity bits. The 3D-TC is a very versatile code and provides very low error rates for a wide range of block lengths and coding rates. It signicantly improves performance in the so-called attening region with respect to the 8-state classical TCs, at the expense of very small increase (less than 10%) in complexity. It compares favorably to more complex codes, such as 16-state TCs. II. T HE E NCODING S TRUCTURE A block diagram of the proposed 3D-TC ( ) is depicted in Fig. 1. The information data sequence u of length bits is encoded by a TC consisting of the parallel concatenation of two codes, and . The corresponding code sequences are denoted by y and y , respectively. We call this turbo code the parent TC. In this paper, we consider the double-binary

c 2009 IEEE 0090-6778/09$25.00

2506
u u 1- yuch patch yup P/S ' ylp Postencoder w
p u n c t u r i n g

IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. 57, NO. 9, SEPTEMBER 2009

Cu

yuch,p

yu

ylch,p

Cl

yl

1-

ylch

a parallel to serial (P/S) multiplexer which takes alternately the parity bits y and y to be post-encoded and groups them into a single block of bits, a permutation denoted by which permutes the parity bits before feeding them to the post-encoder, a rate-1 post-encoder, working on a fraction of the parity bits of each component encoder.

Fig. 1.

Block Diagram of the 3D-TC.

TC of the DVB-RCS standard [8] as the parent TC. Therefore, and are rate-2, 8-state, convolutional codes (CCs) with generator polynomials 15 (recursivity), 13 (redundancy), and 7 (second input). A fraction (0 1) of the parity bits y = {y , y } stemming from and are post-encoded by a rate-1, third encoder . In the sequel, we shall refer to as the permeability rate. The bits to be post-encoded are chosen on a regular basis. For instance, if = 1/4 the permeability pattern is [1000] for both the upper and the lower encoders, i.e., every fourth bit in y and y is post-encoded. The input sequence of the post-encoder is made of alternate y and y (surviving) parity bits. The number of parity bits which are fed to the post-encoder is given by: = . (1)

The proposed structure combines the features of parallel and serially concatenated codes. Increasing turns the code into more serial, while the case = 0 corresponds to the standard parallel TC. Parameter can be tuned to tradeoff between convergence and minimum distance. The proposed encoding principle can be extended in a straightforward manner to any parent TC. III. T HE C HOICE OF THE P OST-E NCODER Given the parent TC and the interleaving laws for and , the performance of the 3D-TC depends on the post-encoder and the permeability rate , which must be properly optimized. The choice of the post-encoder has to meet the following requirements: 1) Its decoder must be simple, adding little complexity to the classical turbo decoder, while being able to handle soft-input and soft-output information, 2) in order to prevent the decoder suffering from any side effects, while searching for very low error rates, the post-code has to be a homogeneous block code, 3) at the rst iteration (without any redundant input information), the pre-decoder associated with the rate-1 postencoder must not exhibit too much error amplication, to prevent from a high loss in convergence. Memory 1 and memory 2 CCs satisfy requirement (1). On the other hand requirement (2) is compatible with the use of Circular CCs having memory 2 (the memory 1 accumulate code cannot be made circular using standard circular encoding, and has to be discarded). Circular convolutional codes (also called tail-biting codes), are such that any state of the encoder register is allowed as the initial state and that the encoding always starts and ends in the same state. This makes the convolutional code a perfect block code and prevents it from any side effects. Moreover, no rate loss is induced by terminating the code. We consider the 4-state CC with polynomials (4/5) (in octal form) for the post-encoder, instead of the classical (4/7) code. At the rst step of the iterative process, the decoder of the latter will (roughly) triple the number of errors of its input. On the other hand, the decoder of the (4/5) code only doubles the number of errors at the rst step. Therefore, from the convergence point of view, it will be preferable to the (4/7) code. Unfortunately, the (4/5) code cannot be made circular directly. However, circularity can be achieved using a statemapping encoding [9, 10]. For short block sizes ( < 1000 bits) and for medium sizes (1000 < < 5000) with low rates, the linear post-encoder (4/5) was chosen. However, for large blocks (5000 > ) and for medium sizes with high rates, the 3D-TC with postencoder (4/5) shows a attening around FER= 106 , due

The fraction 1 of parity bits which is not post-encoded is sent directly to the channel or punctured to achieve the ch ch and y the desired code rate. In Fig. 1 we denote by y sub-codewords of y and y , respectively, directly sent to ch,p ch,p and y their punctured versions. the channel, and by y p p Similarly, we denote by y and y the sub-codewords of y and y which are post-encoded. The output of the post-encoder is denoted by w. The coded sequence y of length bits is ch,p ch,p and y . The overall obtained by multiplexing u, w, y code rate is given by: = 1 , 1 + + (1 ) (2)

ch where (0 1) is the fraction of surviving bits in y and ch y after puncturing. Note that given the highest achievable 1 code rate is max = 1+ . Higher code rates may be achieved by puncturing systematic bits. On the other hand, the lowest code rate is given by the rate of the double-binary TC, i.e., 1/2. However, the proposed 3D-TC is intended for a wide range of code rates, from high ones to low ones, down to = 1/4. For a global coding rate lower than 1/2, the component double-binary encoders in Fig. 1 need to generate extra parity bits. In particular, three parity bits are needed for the lowest rate, = 1/4. For such cases, we devised an appropriate 8-state component code with three binary ouputs.

In this paper, very simple regular or quasi-regular puncturing patterns are applied. For example, if rate-1/2 is sought for and = 1/4, then, according to (2) = 1/3, and the ch ch and y . puncturing pattern [100] will be applied to y The material added to a standard turbo encoder, which we shall refer to as patch because it is placed just behind a preexisting turbo encoder, is composed of:

C. BERROU et al.: IMPROVING THE DISTANCE PROPERTIES OF TURBO CODES USING A THIRD COMPONENT CODE: 3D TURBO CODES

2507

to poor distance properties. In these cases, we used a nonlinear encoder, with recursion polynomial gr = 1 + 2 (i.e., the same recursion as the linear encoder) and feedforward polynomial gf = 1 + ( 2 ). The trellis of the proposed non-linear encoder is simply obtained from the trellis of the encoder (4/5) by swapping two branch metrics. The nonlinear encoder contributes to improve the distance properties of the 3D-TC, with favorable consequences in the cases cited above. IV. T HE C HOICE OF THE P ERMEABILITY R ATE The choice of is a matter of trade-off between the convergence loss and the required min . Convergence designates the zone of the error rate versus signal to noise ratio /0 curve where the error rate begins to decrease noticeably. Choosing a large value of will turn the code into more serial, hence leading to a higher min . However, performance will be penalized from the convergence point of view. This results from the decoder associated with the post-encoder, which does not benet from any redundant information at the rst iteration and therefore multiplies the errors during the rst processing. A. Convergence Properties Let us assume for instance that the post-encoder is the (4/5) CC. At the rst iteration the associated decoder (the predecoder), without any extra information, roughly doubles the errors at its input. From (1), the fraction of the codeword bits that are post-encoded is: = . (3) The fraction of the data processed by the component decoder of each code ( = , ) that is processed by the pre-decoder is: . (4) = 1 + Then, if is the probability of error at the channel output, the average probability of error at each decoder intrinsic input is: = = 2 + (1 ) = (1 + ) . Using (4) we get =

respectively. The convergence loss with respect to the parallel turbo code ( = 0) is very small for = 1/8 and = 1/4, and increases with the value of . For = 1 the loss is very signicant. B. Union Bound to the Error Probability An upper bound on the frame error probability over memoryless binary-input channels with coherent detection of code can be obtained as (7) ( /0 , )

is the output weight enumerating function (OWEF) where of , i.e., the number of code sequences of weight , and () represents the pair-wise error probability. Denote by 1 , 2 and 3 the number of quaternary symbols 10, 01 and 11, respectively, at the input of the double-binary TC. Denote p p also by and the binary weight of vectors y and y , respectively, and its sum by , = + . Similarly, we denote ch,p ch,p by and the binary weight of vectors y and y , respectively, and its sum by , = + . Following [12] the OWEF of the 3D-TC with double-binary components can be written (using random permutations) as
TC , , ( ) =

(8)

where TC , is the number of codewords of the outer doublebinary TC with output weights and , respectively, and , is the input-output weight enumerating function (IOWEF) of the post-encoder, i.e., the number of codewords of weight generated by an input weight . TC , can be expressed as TC ,+
1, 2, 3, , 2, 1, 3, , ( ) /2 { 1, 2, 3, , } 1, 2, 3 (9) ( ) /2 where is the multinomial coefcient. 1, 2, 3 In Fig. 2 we plot the union bound to the error probability for the 3D-TC with = 1/2 for several values of on the AWGN channel for a block length of 1504 bits. Lower error oors are obtained for increasing values of . A slight lower oor with respect to the to the parent TC ( = 0) is observed for = 1/8. The gain is already signicant for = 1/2, while for higher s very signicant low oors are observed. However, as shown above, this gain is obtained at a expense of a signicant convergence loss. In this paper, = 1/4 and = 1/8 (for very long blocks) are considered, since they represent a good trade-off between convergence and error oor performance.

(5)

1 + (1 + ) 1 +

) , (6)

i.e., the probability of each decoder intrinsic input ( error at ) 1+(1+) . Therefore, the addition of is risen by a factor 1+ the post-encoder induces a convergence loss, no matter the value of . This observation is in agreement with the results in [11], where it is shown by using extrinsic information transfer (EXIT) charts analysis that a threefold concatenation cannot achieve any performance gain in terms of convergence with respect to a twofold concatenation unless the decoders can be chosen freely. We used EXIT charts to estimate the convergence threshold of the 3D-TC for several values of . The convergence threshold is 0.55 dB, 0.63 dB, 0.68 dB, 0.86 dB, and 1.32 dB for = 0, = 1/8, = 1/4, = 1/2 and = 1,

V. P ERMUTATIONS AND AND D ISTANCE P ROPERTIES The proposed 3D-TC is characterized by two permutations, denoted by and , respectively. A rst permutation ,

2508
0

IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. 57, NO. 9, SEPTEMBER 2009

0

10 10 10 10 10

10
=0 =1/8 =1/4 =1/2

-1 -2 -3 -4 -5 -6 -7 -8 -9

k=188 bytes

10 10 10

-1 -2 -3 -4 -5 -6 -7 -8 -9

3D-TC DVB-RCS TC 16-state DB TC SCCC Theoretical Limit

FER

10 10 10 10 10 10

FER

10 10 10 10 10

R=4/5 R=1/2 R=1/4

-10

10

10

0.5

1.5

2.5

3.5

4.5

Eb/N0 (dB)

Eb/N0 (dB)

Fig. 2. Union bound on the frame error probability of the 3D-TC (with random permutations) for several values of . = 1504 bits.

Fig. 3. FER performance of the 3D-TC with = 1/4 for = 188 bytes, = 1/4, 1/2 and 4/5. 8 iterations.
10
0

internal of the double binary TC at the basis of the proposed 3D-TC, deals with messages of = /2 quaternary symbols. Here, is built from the combination of the Almost Regular Permutation (ARP) [1] and an intrasymbol permutation, as suggested in [13]. The intrasymbol permutation permits to eliminate a large proportion of composite return to zero (RTZ) sequences for both codes. A second permutation is used to spread = parity bits at the output of the double binary TC before post-encoding. We assume to be the simplest one, dened by the congruence relation = ( ) = 0 + 0 (mod ) , where 0 is the starting index, and 0 is an integer relatively prime with . For each block length, the parameters of permutations and have been carefully chosen to guarantee a large min , even for high rates. For 57-bytes packets, the use of 3D-TC resulted in an increase in min by at least 15% and 20% for rate 4/5 and 1/2, respectively, compared to the standardized DVB-RCS TCs. For 188-byte packets, the increase in min is signicant (i.e., more than 40%) for rate 1/2. It is also worth mentioning that the achieved min values have reasonably low multiplicities. VI. D ECODING THE 3D T URBO C ODE The decoding of the 3D-TC calls for the classical turbo procedure in the logarithmic domain. The decoder consists of three SISO decoders: two 8-state SISO decoders matched 1 and 1 , respectively, and a to and , denoted by 4-state SISO decoder (the pre-decoder) to decode the post1 1 . As for standard TCs, and 1 exchange encoder, extrinsic information on the systematic symbols of the re1 ceived codeword. Also, they must provide with extrinsic information on the post-encoded parity bits. In turn, the pre1 and 1 with extrinsic information on decoder feeds these parity bits. A decoding iteration consists of a single 1 1 , and 1 , in this order. No differences activation of in terms of performance were observed for other activation orders. However, the activation order might have an impact on convergence speed. 1 and 1 are quaternary 8-state decoders Because processing = /2 couples of bits and the pre-decoder is a binary 4-state decoder processing = data, the relative

k=57 bytes

10 10 10

-1 -2 -3 -4 -5 -6 -7 -8 -9

3D-TC DVB-RCS TC 16-state DB TC SCCC Theoretical Limit R=1/2 R=4/5

FER

10 10 10 10 10 10

R=1/4

1.5

2.5

3.5

4.5

5.5

Eb/N0 (dB)

Fig. 4. FER performance of the 3D-TC with = 1/4 for = 57 bytes, = 1/4, 1/2 and 4/5. 8 iterations.

computational complexity added by the latter is very small. For instance, with = 1/4 (the largest value considered in this paper), the additional complexity is roughly 6% in terms of number of trellis transitions per decoded bit. To this, however, some extra-functions must be added to the classical turbo decoder, the main one being the calculation of the extrinsic information on parity bits to be fed to the pre-decoder. Overall, the additional complexity per iteration compared to the classical turbo decoder is less than 10% for = 1/4. In this paper, we consider the simple Max-Log-MAP decoding algorithm, which does not require the knowledge of the channel noise variance. VII. S IMULATION R ESULTS The performance of the 3D-TC was assessed by means of simulation. In Figs. 3 and 4, we report frame error rate (FER) results for block sizes 188 and 57 bytes, respectively, and coding rates 1/4, 1/2 and 4/5. In both gures = 1/4 and a maximum of 8 iterations were assumed. The proposed code shows excellent performance for both short and medium block sizes. In particular, for information block size 188 bytes (see Fig. 3) only 0.8 dB loss is observed with respect to the Gallagers random coding bound at FER 107 for all code rates. For comparison purpose, the

C. BERROU et al.: IMPROVING THE DISTANCE PROPERTIES OF TURBO CODES USING A THIRD COMPONENT CODE: 3D TURBO CODES
0

2509

10 10 10 10

-1 -2 -3 -4 -5 -6

FER

3D-TC 3GPP2 8-state TC SCCC

VIII. C ONCLUSIONS In this paper, we presented a modied turbo code combining the features of parallel and serial concatenation in order to obtain increased Hamming minimum distances with respect to classical turbo codes. The simulation results corroborate the interest of this approach. Frame error rates down to 107 are obtained near the theoretical limits without the use of any outer block code, such as BCH or Reed-Solomon codes. This characteristic makes this new code, called 3D turbo code (3D-TC), very versatile from the standpoint of block size and coding rate. Finally, the internal permutations of the 3D-TC are based on very simple models enabling large degrees of parallelism, if needed. R EFERENCES

BER,FER

10 10 10 10 10 10

BER

-7 -8 -9

k=12288 bits, R=1/2 0.5 1 1.5 2 2.5 3 3.5 4

Eb/N0 (dB)

Fig. 5. BER and FER performance of the 3D-TC with = 1/8 for = 12288 bits, = 1/2. 8 iterations.

performance of the original DVB-RCS TC is also reported for rates 1/2 and 4/5. For rate 1/2 the 3D-TC shows a small convergence loss with respect to the DVB-RCS TC. On the other hand, the error oor is signicantly lowered. The largest gain is obtained for 188 bytes and = 1/2 (about 1.4 dB at FER = 107 ). For rate 4/5, the convergence loss is reduced while a signicant improvement for low error rates is also observed. We can also notice that the performance obtained with optimized permutations is clearly better than the one predicted by the union bound based on random permutations (Fig. 2). We also report in Figs. 3 and 4 the performance of the 16state double binary TC described in [13] and the performance of the SCCC proposed in [14] for rates 1/2 and 4/5. The performance of the proposed 3D-TC is comparable to that of the more complex 16-state TC. For a block length of 188 bytes, the 3D-TC loses 0.1 dB in convergence with respect to the 16-state double-binary TC. However, the proposed code outperforms the 16-state TC in the error oor. A similar behavior is observed in Fig. 4 for a block length of 57 bytes. On the other hand the 3D-TC shows a convergence gain of 0.1 dB and 0.3 dB with respect to the SCCC for = 1/2 and = 4/5, respectively, and = 188 bytes. However, the code in [14] is much simpler, since it is built from 4-state CCs. In Fig. 5, performance comparison is given with respect to the 8-state TC adopted in the 3GPP2 standard and the SCCC in [14]. An information block length of 12288 bits and 8 iterations are assumed for the three codes. Very similar performance are observed in the waterfall region with respect to the 3GPP2. However, the proposed 3D-TC signicantly improves the 3GPP2 code in the oor. On the other hand, the 3D-TC shows slightly better convergence than the SCCC.

[1] C. Berrou, Y. Saouter, C. Douillard, S. Keroudan, and M. Jzquel, Designing good permutations for turbo codes: towards a single model," in Proc. IEEE Int. Conf. Commun. (ICC04), June 2004, pp. 341-345. [2] S. Crozier and P. Guinand, Distance upper bounds and true minimum distance results for turbo-codes designed with DRP interleavers," in Proc. 3 Int. Symp. Turbo Codes, Sept. 2003, pp. 169-172. [3] S. Benedetto, D. Divsalar, G. Montorsi, and F. Pollara, Serial concatenation of interleaved codes: performance analysis, design and iterative decoding," IEEE Trans. Inform. Theory, vol. 44, pp. 909-926, May 2005. [4] M. Tuchler and J. Hagenauer, Exit charts of irregular codes," in Conf. Inform. Sciences Syt., pp. 748-753. [5] D. Divsalar and F. Pollara, Hybrid concatenated codes and iterative decoding," in Proc. IEEE Intl. Symp. Inform. Theory (ISIT97), July 1997, p. 10. [6] J. Li, K. R. Narayanan, and C. N. Georghiades, Product accumulate codes: a class of codes with near-capacity performance and low decoding complexity," IEEE Trans. Inform. Theory, vol. 50, no. 1, pp. 31-46, Jan. 2004. [7] H. Gonzalez, C. Berrou, and S. Keroudan, Serial/parallel turbo codes for low error rates," in Proc. IEEE Military Commun. Conf. (MILCOM05), vol. 1, 2004, pp. 346-349. [8] DVB, Interaction channel for satellite distribution systems, 2000, eTSI EN 301 790, vol. 1.2.2. [9] C. Berrou, A. Graell i Amat, Y. Ould-Cheikh-Mouhamedou, and Y. Saouter, Adding a rate-1 third dimension to turbo codes," in Proc. IEEE Inform. Theory. Work. (ITW07), 2007, pp. 156-161. [10] O. Y. Takeshita, O. M. Collins, P. C. Massey, and D. J. Costello, Jr., On the frame-error rate of concatenated turbo codes. IEEE Trans. Commun., vol. 49, pp. 602608, Apr. 2001. [11] M. Tuchler, Convergence prediction for iterative decoding of threefold concatenated systems," in Proc. IEEE Glob. Conf. Commun. (GLOBECOM02), vol. 10, Nov 2002. [12] S. Benedetto and G. Montorsi, Unveiling turbo codes: some results on parallel concatenated coding schemes," IEEE Trans. Inform. Theory, vol. 42, pp. 409-429, Mar. 1996. [13] C. Douillard and C. Berrou, Turbo codes with rate-/( + 1) constituent convolutional codes," IEEE Trans. Commun., vol. 53, no. 10, pp. 1630-1638, Oct. 2005. [14] A. Graell i Amat, G. Montorsi, and F. Vatta, Design and performance analysis of a new class of rate compatible serial concatenated convolutional codes," submitted to IEEE Trans. Commun. Available: arxiv:cs.IT/0510035, Oct. 2007.