Thesis

OPTIMAL CODE RATES FOR CONSTRAINED SYSTEMS WITH UNCONSTRAINED POSITIONS: AN APPROACH TO COMBINING ERROR CORRECTION CODES WITH
MODULATION CODES FOR DIGITAL STORAGE SYSTEMS
A DISSERTATION SUBMITTED TO THE DEPARTMENT OF ELECTRICAL ENGINEERING AND THE COMMITTEE ON GRADUATE STUDIES OF STANFORD UNIVERSITY IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF PHILOSOPHY
Tze-Lei Poo June 2005
c Copyright by Tze-Lei Poo 2005 All Rights Reserved
ii
I certify that I have read this dissertation and that, in my opinion, it is fully adequate in scope and quality as a dissertation for the degree of Doctor of Philosophy.
Brian H. Marcus Principal Adviser
John T. Gill III
John M. Cio
Approved for the University Committee on Graduate Studies.
iii
iv
Abstract
Error-control codes (ECCs) and constrained codes are both widely used in digital storage systems, which can viewed as noisy channels with modulation-constrained inputs. For practical reasons, independent error correction and modulation encoders are typically used. Serially cascading the ECC and modulation encoders in that order often results in high error propagation. An improved scheme that mitigates this eect and simultaneously permits high rates is reverse concatenation, whereby a systematic ECC encoder is used to generate only the parity bits of a modulated data stream, which are then modulated and appended to the main modulated data stream. Reverse concatenation can be used to facilitate soft decoding but such procedures tend to be complicated and introduce inaccuracies. A recent variant of reverse concatenation, constrained systems with unconstrained positions, designs the modulation encoder to produce constrained sequences such that certain bit positions are unconstrained, in the sense that these positions can take on either bit value without violating the constraint. These positions are used for the insertion of ECC parity bits. This scheme facilitates soft iterative decoding, since the ECC decoder has direct access to all soft information from the channel decoder. Although it has been implemented in some commercial disk drives, usage has been limited by the lack of a systematic approach to code design and an understanding of the achievable rate regions. To this end, we present a new general framework for analyzing this scheme. We construct a graph containing all information on permissible sets of unconstrained positions, without requiring a pre-specied period or set of unconstrained positions. v
Properties of this graph are established. We also investigate the maximum insertion rate (maximum density of unconstrained positions permitted by the constraint), which corresponds to the highest redundancy permissible for any ECC that can be used with the constraint. It is shown that this quantity is rational and computable. We formally dene the tradeo function, which represents the frontier of the achievable code rate region over the range of permissible insertion rates. The tradeo function is shown to be monotonic, continuous, and concave for nite-type constraints. We also show that periodic patterns of unconstrained positions achieve the optimal code rate for a given insertion rate. A result on timesharing is given, which allows for weighted combinations of congurations and makes it possible to exactly meet the rate of any ECC in the permissible range. Exact tradeo functions for certain MTR and RLL constraints are determined. For a general constraint, the tradeo function remains dicult to compute exactly, but we present several ecient algorithms for computing upper and lower bounds. In particular, we construct an interesting upper bound on the MTR constraints based on the analysis of time-varying MTR constraints on the constrained positions.
vi
Acknowledgements
I would like to thank my advisor Brian Marcus for his invaluable guidance, encouragement and help in so many ways; my collaborators, John Fan and Panu Chaichanovong, with whom it has been a pleasure working with and bouncing ideas o; John Gill and John Cio for serving on my thesis committee. I have beneted greatly from their courses in error correction coding, information theory, digital communications, and advanced coding techniques at Stanford University. I am grateful to John Gill for all his advice and help as my academic advisor at Stanford University. I also would like to thank Leonard Tyler for serving as chair for my examination committee. I have received nancial support from the Department of Electrical Engineering at Stanford University, as well as from the Faculty of Science, the University of British Columbia. I am immensely grateful to my parents and sisters for their unending love and support, and to Joseph Blitzstein, who has been my family away from home, and without whose help and encouragement this thesis would be very far from completion.
vii
Contents
Abstract Acknowledgements 1 Introduction 1.1 1.2 1.3 A Typical Magnetic Storage System . . . . . . . . . . . . . . . . . . . Constrained Coding for Recording Channels . . . . . . . . . . . . . . Code Concatenation . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.3.1 1.3.2 1.4 1.4.1 1.4.2 1.5 Standard Concatenation . . . . . . . . . . . . . . . . . . . . . Reverse Concatenation . . . . . . . . . . . . . . . . . . . . . . Soft Demodulation Codes . . . . . . . . . . . . . . . . . . . . Bit Insertion . . . . . . . . . . . . . . . . . . . . . . . . . . . . v vii 1 2 5 8 8 9 11 11 13 21 26 26 29 32 34 36 39 39 42
Soft Decoding in Concatenated Schemes . . . . . . . . . . . . . . . .
Summary of Contributions . . . . . . . . . . . . . . . . . . . . . . . .
2 Constrained Systems with Unconstrained Positions 2.1 2.2 2.3 2.4 2.5 Optimizing for Asymptotic Rates . . . . . . . . . . . . . . . . . . . . Constrained Coding and Notation . . . . . The Graph Presentation G . . . . . . . . . 2.3.1 Size of G . . . . . . . . . . . . . . . Properties of G . . . . . . . . . . . . . . . 2.5.1 2.5.2 Dening the Tradeo Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Optimal Code Rates over G . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Periodic Tradeo Functions . . . . . . . . . . . . . . . . . . . viii
2.5.3
Relation to Previous Denitions . . . . . . . . . . . . . . . . .
44 45 46 48 50 50 52 57 58 62 66 66 69 69 75 77 81 83 83 86 87 88 92 94 97
3 Maximum Insertion Rates 3.1 3.2 3.3 Rationality of the Maximum Insertion Rate . . . . . . . . . . . . . . Numerically Computing the Maximum Insertion Rate . . . . . . . . . Closed Form Solutions for RLL and MTR Codes . . . . . . . . . . . . 3.3.1 3.3.2 3.4 3.4.1 3.4.2 RLL(d, k) Constraints . . . . . . . . . . . . . . . . . . . . . . MTR Constraints . . . . . . . . . . . . . . . . . . . . . . . . . Maximal Subgraph Induced by Max-Insertion Rate Cycles . . f = f (p) = f (o) at the Maximum Insertion Rate . . . . . . . .
Tradeo Functions at the Maximum Insertion Rate . . . . . . . . . .
4 Properties of the Tradeo Function 4.1 4.2 Monotonicity and Left-Continuity of f . . . . . . . . . . . . . . . . . Finite-Type Constraints . . . . . . . . . . . . . . . . . . . . . . . . . 4.2.1 4.2.2 4.3 4.4 4.5 Concavity and Continuity . . . . . . . . . . . . . . . . . . . . Periodic Congurations Achieve Optimal Code Rates . . . . . Timesharing Implications for G with Multiple Components . . . . . .
Relating f to f (o) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . NRZ and NRZI Tradeo Functions . . . . . . . . . . . . . . . . . . . 4.5.1 RLL(0, k) and MTR(j) Systems . . . . . . . . . . . . . . . . .
5 Constraints with a Known Tradeo Function 5.1 5.2 5.3 5.4 Extensions of bit-stung . . . . . . . . . . . . . . . . . . . . . . . . . Tradeo Function for MTR(2, 2) . . . . . . . . . . . . . . . . . . . . . Tradeo Function for RLL(d, 2d + 1) . . . . . . . . . . . . . . . . . . Optimal Code Rates for MTR(j) at Low Insertion Rates . . . . . . . 5.4.1 5.4.2 5.4.3 Topping up Small Gaps . . . . . . . . . . . . . . . . . . . . .
Reduction of Large Gaps . . . . . . . . . . . . . . . . . . . . . 100 Unifying the Arguments on Large and Small Gaps . . . . . . . 103 ix
6 Bounds for the Tradeo Function 6.1 6.1.1 6.1.2 6.1.3 6.1.4 6.2 6.2.1 6.2.2 6.3 6.3.1 6.3.2 6.3.3
106
Lower Bounds . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106 Greedy Lower Bound . . . . . . . . . . . . . . . . . . . . . . . 107 Dynamic Programming Lower Bound . . . . . . . . . . . . . . 108 Standard Bit-stung Lower Bound . . . . . . . . . . . . . . . 109 Irregular Bit-stung Lower Bound . . . . . . . . . . . . . . . 110 Straightline Upper Bound . . . . . . . . . . . . . . . . . . . . 111 Dynamic Programming Approximate Upper Bound . . . . . . 112 Bounds for MTR(j) constraints . . . . . . . . . . . . . . . . . 113 Bounds for MTR(j, k) constraints . . . . . . . . . . . . . . . . 115 Bounds for RLL(d, k) with k > 2d + 1 . . . . . . . . . . . . . 116 119
General Upper Bounds . . . . . . . . . . . . . . . . . . . . . . . . . . 111
Performance of some Upper and Lower Bounds . . . . . . . . . . . . 113
7 Time-Varying MTR Constraints 7.1 7.2 7.3 Tight vectors for TMTR(m) 7.3.1 7.3.2 7.4 7.5 7.4.1 7.5.1
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120 . . . . . . . . . . . . . . . . . . . . . . 121 Graph Presentations . . . . . . . . . . . . . . . . . . . . . . . . . . . 122 Shannon cover when m = (M p + 1, M p + 2, . . . , M ) . . . 125 Rome Graphs and Characteristic Equations . . . . . . . . . . 126
Bounds on the Capacity of TMTR Constraints . . . . . . . . . . . . . 131 Complete Linear Ordering of C(m) up to period 4 . . . . . . . 132 A New Upper Bound for fMTR(j) . . . . . . . . . . . . . . . . 136 141 Relations to MTR Constraints with Unconstrained Positions . . . . . 134
8 Future Work 8.1 8.2
Open Problems on the Tradeo Function . . . . . . . . . . . . . . . . 141 Tradeo Functions for n-dimensional Constraints . . . . . . . . . . . 145 148
A Long Proofs for Chapter 5
A.1 Generalized Fibonacci Relations for MTR codes . . . . . . . . . . . . 149 A.1.1 Pre and Post-Multiplication by A . . . . . . . . . . . . . . . . 151 x
A.1.2 Pre and Post-multiplying by B . . . . . . . . . . . . . . . . . 152 A.2 Proofs of some Matrix Inequalities . . . . . . . . . . . . . . . . . . . . 152 B Long Proofs for Chapter 7 Bibliography 177 189
xi
List of Tables
2.1 3.1 3.2 3.3 7.1 Sizes of G for some constraints . . . . . . . . . . . . . . . . . . . . . . Maximum insertion rates for some RLL(d, k) constraints . . . . . . . Adding is better when (3.4) holds . . . . . . . . . . . . . . . . . . Maximum insertion rates for some MTR(j, k) constraints . . . . . . . 35 52 56 58
Tight forms of m for TMTR for p = 2, 3, 4 . . . . . . . . . . . . . . . 122
xii
List of Figures
1.1 1.2 1.3 1.4 1.5 1.6 1.7 1.8 1.9 Block Diagram of a Magnetic Storage System. . . . . . . . . . . . . . Runlength Limited RLL(d, k) constraint. . . . . . . . . . . . . . . . . MTR(j) constraint. . . . . . . . . . . . . . . . . . . . . . . . . . . . . MTR(j, k) constraint. . . . . . . . . . . . . . . . . . . . . . . . . . . . A nite-state encoder for RLL(1, 2) with input tagging. . . . . . . . . Standard concatenation scheme. . . . . . . . . . . . . . . . . . . . . . Reverse concatenation scheme . . . . . . . . . . . . . . . . . . . . . . Soft Decoding in Reverse Concatenation. . . . . . . . . . . . . . . . . Encoding diagram of Example 1. . . . . . . . . . . . . . . . . . . . . 3 5 6 7 7 8 9 12 14 15 16 18
1.10 Schematic diagram of bit insertion. . . . . . . . . . . . . . . . . . . . 1.11 A schematic picture of standard bit-stung. . . . . . . . . . . . . . . 1.12 Gradient comparison of simple bit comparison (point Q) and standard bit-stung (point P). . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.14 Rate regions of simple bit insertion, standard bit-stung h() and reverse concatenation with a systematic modulation encoder C2 for MTR(4). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.1 2.2 2.3 2.4 2.5 The Shannon cover G and G for the constraint that forbids 00 and 11. The Shannon cover G and G for the RLL(0, 1) constraint. . . . . . . The Shannon cover G and G for the RLL(1, 3) constraint. . . . . . . The Shannon cover G and G of MTR(j) . . . . . . . . . . . . . . . . The Shannon cover G and G for the 2-charge constraint . . . . . . . . xiii
1.13 Rate regions of simple bit insertion and standard bit-stung for MTR(4). 19
20 33 33 34 35 42
2.6
An example showing that f () need not be zero, even when S is nitetype and primitive . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43 55 70 77 79 87
3.1 4.1 4.2 4.3 5.1 5.2
Choices of appending symbols to a word with state (a, b). . . . . . . . Subgraphs of G that can be reached from H for Example 29. . . . . . An example of f dened over f1 , f2 and f3 . . . . . . . . . . . . . . . An example of g(f1 , f2 ). . . . . . . . . . . . . . . . . . . . . . . . . . Tradeo Functions for MTR(1) and MTR(2). . . . . . . . . . . . . . The Shannon cover G and the only non-trivial component G (of G) for MTR(2, 2). The subgraph of G with solid edges is H, which is isomorphic to G. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
90 91 93 93 96 99
5.3 5.4 5.5 5.6 5.7 5.8
Tradeo functions for RLL(0, 1) and MTR(2, 2). . . . . . . . . . . . . The irreducible component G of G for RLL(d, 2d + 1). . . . . . . . . Tradeo function for RLL(d, 2d + 1). . . . . . . . . . . . . . . . . . . Tradeo Function for MTR(3) for 1/4. . . . . . . . . . . . . . . . Illustrating Lemma 56 and Proposition 57 step-by-step for the case of Illustrating Lemma 53 for the case of three Bs (shaded in gray). . . .
three Bs (shaded in gray) . . . . . . . . . . . . . . . . . . . . . . . . 102 6.1 6.2 6.3 6.4 Standard bit-stung lower bound (solid), dynamic programming lower bound (dashed) and approximate upper bound (dotted) of f for MTR(4)113 Standard bit-stung lower bound (solid) versus irregular bit-stung lower bound (dashed) of f for MTR(4) for insertion rates in [0.56, 0.58] 114 Standard Bit-Stung lower bound (solid), greedy lower bound (dashed) and straightline upper bound (dotted) of f for MTR(2, 3) . . . . . . . 115 Bit-stung lower bound (dashed), dynamic programming lower bound (solid) and approximate upper bound (dotted) of the tradeo function for the MTR(2, 3) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116 6.5 Irregular bit-stung lower bound (solid), dynamic programming lower bound (dashed) and approximate upper bound (dotted) of the tradeo function for the MTR(4, 4) . . . . . . . . . . . . . . . . . . . . . . . . 117 xiv
6.6 6.7 7.1 7.2 7.3 7.4 7.5 7.6 7.7 7.8
Some Bounds for RLL(1, 4) . . . . . . . . . . . . . . . . . . . . . . . 118 Best known bit-stung lower bounds for RLL(1, 7) and RLL(1, 8) . . 118 Standard trellis diagram for the period 2 TMTR(j = 2, 3) constraint. 120
Shannon cover for TMTR(1, 2, . . . , p). . . . . . . . . . . . . . . . . . . 130
Shannon cover for TMTR(0, 1, . . . , p 1). . . . . . . . . . . . . . . . 130
Shannon cover of TMTR(2, 3, 4). . . . . . . . . . . . . . . . . . . . . 132 Graph presentation of H2 for MTR(2). . . . . . . . . . . . . . . . . . 139 New TMTR upper bound for MTR(2) in comparison with other bounds.139 New TMTR upper bound for MTR(3) in comparison with other bounds.140 New TMTR upper bound for MTR(4) in comparison with other bounds.140
xv
xvi
Chapter 1 Introduction
The application of soft iterative decoding techniques in digital storage systems is currently an area of active research. Turbo codes [9] and low density parity check (LDPC) [45] codes are two types of codes which make use of probabilistic (soft) decoding and which have demonstrated signicant improvements in performance on magnetic recording channels [68, 21, 4]. One of the goals in this area is to integrate modulation codes with ECCs eectively so that soft information can be passed through the modulation code to reach the ECC decoder, thereby achieving higher coding gains. Many papers [4, 24, 49, 69, 26] focus on the development of soft-in-soft-out (SISO) decoders for use with specic runlength limited (RLL) constrained codes. This typically employs some form of equalization on the combined trellis of the RLL code and the precoded channel, or performs soft demodulation on the joint trellis of certain RLL codes and turbo codes. For most codes, this trellis approach generates a prohibitively large number of states that grows rapidly with the complexity of the codes used, limiting the types of modulation constraints (and constraint lengths of the turbo codes) that can be used. In practice, it is desirable to use simplied alternatives with low complexity and increased exibility in the choices of codes used, if the corresponding rate loss incurred can be minimized. In this dissertation, we investigate one such alternative. We focus on the design of constrained codes that facilitate soft iterative decoding but are not restricted to a 1
CHAPTER 1. INTRODUCTION
specic constraint or ECC. In particular, the design involves selecting certain positions in the constrained code that can be ipped arbitrarily without violating the constraint, and using those positions for the insertion of ECC parity bits. This approach is a generalization of Fan and Cios simple bit insertion technique [24], and has the advantage that soft information on the entire received word can be used directly by the ECC decoder (without the need for prior demodulation). We demonstrate in this work that this generalized scheme admits higher code rates than previously believed, thus making it a more attractive choice in soft demodulation. This chapter is organized as follows. Section 1.1 introduces magnetic storage and some current trends. Section 1.2 gives a short background in constrained coding. Section 1.3 describes some code concatenation techniques for combining an error correction code with a constrained code, while Section 1.4 discusses current methods that are used to facilitate soft decoding in concatenated coding schemes.Finally, in Section 1.5 we summarize our original contributions and give an overview of the remaining chapters in this dissertation.
1.1
A Typical Magnetic Storage System
The standard framework for the signal processing and coding in a digital storage system is depicted in Fig. 1.1. The system can be divided into channel, modulation and coding.
Channel. The magnetic recording channel introduces intersymbol interference
(ISI), in which the received samples are aected by the spreading of neighboring pulses in addition to media or thermal noise. In digital communications, the channel is often presented as a linear ISI channel followed by additive white Gaussian noise. In practical systems, this ISI is often equalized to some target response, in a technique known as partial response maximum likelihood (PRML) [16]. The equalized samples are then decoded using a channel decoder based on the Viterbi algorithm.
1.1. A TYPICAL MAGNETIC STORAGE SYSTEM
user u bits {0, 1}
ECC encoder
v bits {0, 1}
Constraint encoder
w bits {0, 1}
Binary signal mapper
x symbols {1, +1}
Channel u decoded bits {0, 1} ECC decoder v bits {0, 1} Constraint decoder w bits {0, 1} Channel decoder (Viterbi) signals y
Figure 1.1: Block Diagram of a Magnetic Storage System. Modulation. For magnetic recording channels, the input to the channel is restricted to binary signals due to the magnetic media having two levels of polarization. Additional constraints are usually imposed on the binary sequences to enable timing recovery, DC-balance, spectral shaping and the elimination of certain error events. Such restrictions are known as modulation constraints, and the process by which a user data sequence is encoded losslessly into a sequence that satises the constraint is known as constrained coding. A short background on constrained coding is provided in the next section, along with some examples of constrained codes. Modulation encoders and decoders are usually implemented via look-up tables or by dedicated circuitry. Coding. The ECC introduces redundancy into the sequences in such a way that duced by a noisy channel. For linear ECC codes, it is possible to put the ECC into systematic form, in which the parity bits are appended to the user bits. Traditionally, the ECC in magnetic storage is a byte-based Reed Solomon code that is cross-interleaved [36] so as to correct multiple bursts across sectors (that are 512 bytes in size) in addition to random symbol errors. Typically, hard decision outputs (0 or 1) are produced.
the receiver can decode the original transmitted sequence despite errors intro-
More recently, there have been breakthroughs in the development of ECCs such as turbo codes and LDPC codes that make use of soft information for near-optimal decoding performance. To use such ECCs in magnetic storage, the channel decoder has to be modied to produce soft information on the transmitted bits from the received samples. The Bahl-Cocke-Jelinek-Raviv (BCJR) algorithm [5] gives an ecient method for computing these probabilities for the decoded bits. There are a large number of papers on the application of turbo codes to magnetic recording, demonstrating between 3 to 6 dB of coding gain under various conditions; see [67, 68, 62, 4, 26, 44]. The application of LDPC codes to magnetic storage has also received much consideration and was shown to perform almost as well as turbo coded systems while requiring less complexity [25, 64, 21]. The lack of convincing evidence that LDPC codes do not have an error oor above 1015 still necessitates the conventional use of an outermost Reed Solomon ECCs to guard against residual errors. Unfortunately, the error statistics of LDPC codes in the presence of a large number of errors within a single block may easily exceed the capacity of the outer Reed Solomon ECC [64, 42], thereby limiting the coding gains and resulting in unacceptably high block failure rates. This has led some researchers to turn to the use of turbo product codes [60, 42] in a serial concatenated structure, which yields similar performance to LDPC codes but have a harmonious compatibility with the outer Reed Solomon ECC. VLSI Architectures of iterative decoding techniques (using serial and parallel turbo codes) for magnetic recording have also been discussed in [70]. Note that even though constrained coding and error-control coding share many similarities, they are typically designed and implemented separately in magnetic storage, since the use of existing encoders with ecient implementations is encouraged. Also, it is crucial to use constrained codes and ECCs that have high rate and low complexity. The high rate is essential because the binary-input constraint already reduces the overall capacity available. Any reduction in disk capacity due to coding redundancy needs to be compensated for by an increase in the recording density (the number of bits per unit area of a disk). However, increasing the density exacerbates the ISI problem, which results in degraded performance. For high recording density
1.2. CONSTRAINED CODING FOR RECORDING CHANNELS
1 0 0 d 1
0 d+1 0 0 k 1 1
Figure 1.2: Runlength Limited RLL(d, k) constraint. disk drives, it is then essential that the coding redundancy be kept to a minimum. This will be an important consideration in the construction of soft modulation codes using the generalized bit insertion technique. The reader is referred to [8] for discussion of other channel design issues such as timing recovery, channel identication, equalization, and quantization.
1.2
Constrained Coding for Recording Channels
In constrained coding, user data is encoded into sequences that satisfy some modulation constraint. The constrained code often improves the performance of a communications or recording channel in various magnetic and optical data storage systems. One of the most widely used constraints is the runlength limited RLL(d, k) constraint on binary sequences, where runlengths of 0s are bounded below by d and bounded above by k. Instances of this constraint such as the RLL(1, 7) constraint have been applied to Compact Disc storage. Sequences satisfying the RLL(d, k) constraint correspond to sequences of edge labels obtained from traversing paths in the graph shown in Fig. 1.2. This graph presents the RLL(d, k) constraint in the Non-Return to Zero Inverse (NRZI) domain, where a 1 corresponds to a transition from one magnetic (or optical) polarity to another, while a 0 corresponds to no transition. Thus, the parameter d in RLL(d, k) species the minimum separation between transitions (that can be used to reduce interference), while the parameter k controls the maximum separation (that can be used to force transitions for timing-recovery). Another scheme to express binary data in recording channels is the Non-Return to Zero (NRZ), where two distinct symbols, usually + and , represent the two
1 0
2 0
j-1 1 0
j 0
Figure 1.3: MTR(j) constraint. polarities. Simple precoding can be used to convert a binary sequence from NRZI representation to NRZ. This is described later in Section 4.5. Of particular importance for todays high-density disk drives are constraints that enhance the performance of more complex detection methods, such as PRML detection, mentioned earlier. An important example is the Maximum Transition Run MTR(j) constraint [51], which limits each runlength of 1s to be at most j (see Fig. 1.3). This constraint was originally introduced to limit the number of times the recording head has to switch polarity so that the medium will not be over-stressed. A variant of the MTR(j) constraint is the MTR(j, k) constraint (see Fig. 1.4), where the additional k parameter restricts the number of consecutive 0s to be at most k for timing-recovery. The j parameter on the 1s has the added functionality of increasing the minimum distance between distinct codewords and therefore providing some error correction coding gain. Typical values of j and k are 4 and 15 respectively. Other important constraints include the asymmetrical runlength limited constraints [35] used in optical recording, the charge constraints used in recording channels, the (0, G/I) constraints for timing recovery and reduction of path memory in PRML, and the time-varying MTR (TMTR) constraints, which are a class of distance-enhancing constrained codes (see [50], [38]). TMTR constraints are discussed in Chapter 7. We will formally dene a constrained system in Chapter 2, but for now, it suces to think of a constrained system S as being the set of all sequences that can be obtained by reading the labels of some xed nite-state labeled directed graph. Typically,
1.2. CONSTRAINED CODING FOR RECORDING CHANNELS
1 1 0
2 1
3 1
0 0
1 0 0 3 1 2 1 0 1
Figure 1.4: MTR(j, k) constraint. encoding requires imposing the constraint on words of nite length. Let M (S, n) denote the set of words of length n satisfying S. The capacity of S, as rst dened by Shannon [63], is cap(S) = lim log |M (S, n)| . n n
Given a constraint S, its nite-state graph is usually a starting point for constructing an encoder, which is a nite-state machine. Such an encoder is called a nite-state encoder. Based on the current input symbol and its current internal state, the nite-state encoder produces an output codeword such that the concatenation of consecutive codewords does not violate the constraint. An example of a nite-state encoder with input tagging is shown in Fig. 1.5, where each edge is assigned input tag/output label. A rate p : q nite-state encoder for S accepts p bits of user data to generate a q-bit codeword. This code rate p/q cannot exceed cap(S).
0/001
1/010 0 0/101 1
1/010
Figure 1.5: A nite-state encoder for RLL(1, 2) with input tagging. A sliding-block code is a coder where the demodulation is based in a deterministic way on a sliding window of blocks, i.e., the decoder decides the input symbol based on
ECC Encoder
v Constraint w Encoder
Channel
Channel Decoder
w Constraint v Decoder
ECC Decoder
Figure 1.6: Standard concatenation scheme. a local window comprised of the current block, m previous blocks, and a upcoming blocks. The corresponding encoder is called (m, a)-sliding-block decodable and the window length is given by m + a + 1. When the window is of length 1, the encoder is called block decodable and the resulting code is commonly referred to as a block code [33, 30, 31, 32]. The well-known state-splitting algorithm introduced by Adler, Coppersmith and Hassner [1] can be used to construct a nite-state encoder for a sliding block modulation code at any rate p : q that is less than cap(S). Good references on the construction and theory of block and sliding block encoders for constrained codes are [47, 12].
1.3
Code Concatenation
Two primary schemes for the integration of the constrained code and the error-control code are surveyed here, namely standard concatenation and reverse concatenation.
1.3.1
Standard Concatenation
The standard concatenation scheme, shown in Fig. 1.6, consists of the serial concatenation of the constrained code as the inner code and the error-control code as the outer code. The channel decoder is a SISO decoder which operates on the received samples and produces accurate reliability information about the bits, which can be used as prior information for the ECC decoder. On the encoding side, the user sequence u is rst encoded by the ECC encoder to produce a codeword sequence v, before being mapped by the constrained encoder to w. If kECC /NECC and kC /NC are the rates of the ECC and the constrained code
1.3. CODE CONCATENATION
Encoder C1
w(m) Channel
parity w(p) v ECC Encoder C2 Encoder
w(m) Hard ECC Channel Decoder v (p) Decoder w Decoder C2
Decoder C1
Figure 1.7: Reverse concatenation scheme .
respectively, then the overall rate of standard concatenation is given by
kECC kC . NECC NC
The main problem with standard concatenation for hard-decision decoding is errorpropagation, which arises when a small number of errors in the received word becomes magnied by demodulation. The impact of error-propagation depends on the details of the ECC decoder. As noted in [22], a way to reduce error propagation is to use short block codes for the constrained code; in particular, choose kC to equal the ECC symbol size B (or a multiple of B). However, such a design restriction often results in low rate codes that do not come close to the capacity of the modulation constraint.
1.3.2
Reverse Concatenation
In reverse concatenation, the order of the constrained code and ECC code are switched. This is depicted in Fig. 1.7. This scheme was analyzed in [34] and [22] as a method of preventing error-propagation in magnetic recording systems, but goes back to Bliss [10] and Mansuripur [46]. It is also related to specic schemes presented in [6, 29, 41, 53, 54] that combine (d, k) run-length constraints with error-control codes for detecting and correcting bit errors or bit shifts. Reverse concatenation involves a main constrained code C1 and an auxiliary constrained code C2 , which have rates RC1 = kC1 /NC1 and RC2 = kC2 /NC2 , respectively. It also assumes the use of a systematic ECC of rate RECC = kECC /NECC . The encoding rst maps the user word u to the main constrained codeword w(m) . This
10
constrained codeword is then encoded by the systematic ECC encoder to produce parity bits v, which are further encoded by a second constrained code to yield w(p) . channel. The overall rate is given by RRevCon = The binary word w = w(m) w(p) is then mapped into 1s before transmission on the 1
1 RC1
1+
1 RC2
1 RECC
kC1 = nC1
nC2 1+ kC2
nECC 1 kECC
There are no restrictions on the design of the main constrained code C1 since the demodulation of this code takes place after its decoding by the ECC, so that it does not suer from error-propagation. This allows the use of long block lengths in the constrained code C1 where the rate can come arbitrarily close to the capacity of the constraint. The auxiliary constrained code C2 , however, is implemented using block codes or sliding-window block codes with short block length, as in the case of the constrained code in standard concatenation. The hard-decision decoding procedure for reverse concatenation is given as follows: Suppose the channel decoder (e.g., a Viterbi decoder) produces a received sequence that is a possibly erroneous copy of the transmitted word w. This sequence can be divided into a message portion w(m) and a parity portion w(p) , based on the correspondence to C1 and C2 , respectively. The parity portion w(p) is rst demodulated by the code C2 , which has limited error-propagation by design, to obtain the word v . On the other hand, the message portion w(m) can go directly to the ECC decoder without any need for demodulation. The ECC decoder performs decoding on the version of the constrained message bits, and demodulation of C1 yields a replica u of the original user bits u. In short, reverse concatenation has the advantage of simultaneously limiting error propagation from demodulation and achieving high rates for the main constrained code C1 . It is also most eective when RECC is high (i.e., when the parity portion constitutes a small part of the whole codeword). We will see in the next section word {w(m) , v }. If the ECC decoding is successful, then the output is an error-free
1.4. SOFT DECODING IN CONCATENATED SCHEMES
11
that there is another advantage of using reverse concatenation when soft decoding is required. Reverse concatenation gives an eective method of meeting a desired modulation constraint using a near-capacity constrained code.
1.4
Soft Decoding in Concatenated Schemes
Many channels provide soft information, which is typically a real-valued metric on each bit indicating its reliability, e.g., a probability or a log-likelihood ratio (LLR). For many situations, the ability to obtain this soft information on the channel output allows the ECC decoder to improve its performance.
1.4.1
Soft Demodulation Codes
With standard concatenation, the demodulation step hinders the soft information from reaching the turbo or LDPC decoding in the sense that given bit-wise probabilities for the channel decoder output w, it is not straightforward to obtain bit-wise probabilities for the demodulated word v in Fig. 1.6. A way to overcome this di culty is to use soft demodulation codes that are specically designed to facilitate the ecient passing of soft information. One approach is to use a systematic modulation encoder; see [24, 21]. The basic idea is to map the input bits to a xed (but not necessarily contiguous) set of positions in the encoded codeword and ll in the other bits as a deterministic function of the input bits so that the modulation constraint is met. The demodulation uses a suboptimal procedure that extracts only the probabilities corresponding to the systematic input bits. Expressions for the extrinsic probabilities obtained from soft decoding of a systematic modulation code are given in [21]. Note that this approach typically yields very low rates, which makes it less attractive for standard concatenation. Higher rates can be obtained if reverse concatenation is used, since only the auxiliary code C2 is required to be soft demodulated. Clearly there are other designs of soft demodulation codes that can be used, but it is generally an art to construct ones of high-rate and low complexity. In the next section, we shall
12
Encoder C1
w(m) Channel Soft Channel
w(m) Soft ECC Decoder
Decoder C1
parity w(p) v ECC Encoder C2 Encoder
v (p) Decoder w Decoder C2
Figure 1.8: Soft Decoding in Reverse Concatenation.
examine another approach called bit insertion, whose generalized version is the main focus in this dissertation. Soft-decision decoding for reverse concatenation is depicted in Fig. 1.8. As can be seen from the gure, it is possible to directly use the soft information for the message portion w(m) , since error correction is performed before its demodulation. For the auxiliary code C2 that encodes the parity portion, a SISO decoder, which uses some or all of the following: channel constraint, modulation encoder, and modulation constraint, is applied. Examples using RLL(d, k) constraints can be found in [24, 21, 26, 69]. A SISO decoder can also be used to demodulate the main constrained code C1 . When incorporated into a soft iterative scheme with the soft ECC decoder, additional coding gain can be obtained [24]. This is because the SISO constraint decoder helps the ECC decoder to avoid words that violate the the constraint by reecting this knowledge in the updated message bit probabilities or LLRs. Thus, the general scheme of soft iterative decoding can be described as follows. The channel decoder outputs a LLR for each modulated bit, which serves as prior information for the ECC decoder. Meanwhile, the parity portion is demodulated according to C2 , and the LLRs are input as prior information to the ECC decoder. Performing one iteration of SISO ECC decoding produces extrinsic LLRs, which exclude the prior LLRs that were input to the ECC decoder. These extrinsic LLRs are then summed at the channel decoder to obtain the prior LLRs for input to the SISO decoder for C1 . The SISO
13
decoder for C1 then updates the soft information and produces further extrinsic LLRs that are then summed and passed back into the soft ECC decoder. By iterating back and forth between the ECC and the constraint, it is possible to obtain additional coding gain by using the redundancy that is inherent in the constraint. Theoretical limits of this additional coding gain are discussed in [23]. Soft iterative joint decoding of the constraint and the ECC is also suitable for standard concatenation schemes. This typically involves decoding on the Cartesian product trellis of the combined RLL and turbo codes [2, 3], or simply passing the soft outputs from a SISO RLL decoder to the turbo decoder without feeding back soft information to the channel decoder [49]. Several papers have shown that standard concatenation involving only a single convolutional code of moderate constraint length yields similar, and in some cases better performance than reverse concatenation turbo coded structures for PRML [20, 65, 48, 26]. However, the improved performance often comes at a price of increased complexity as the convolutional code components of a turbo code require maximum a posteriori probability (MAP) decoding. Such trellises usually require many states and are costly to implement, making it desirable to consider other low complexity alternatives, such as bit insertion.
1.4.2
Bit Insertion
The bit insertion scheme is a variation of reverse concatenation where the main constrained code C1 is designed so that ECC parity bits can be inserted into certain predetermined locations that are designated as unconstrained. In other words, those locations in the modulation code can take on bit values of 1 or 0 arbitrarily without violating the constraint. This simplies the reverse concatenation structure since the auxiliary constrained encoder C2 is no longer needed. An example that illustrates the idea is given below. Example 1 ([66]) Consider a block code for MTR(2) with 2 codewords: {10101, 01101}.
It is clear that if the third or fth bit of either codeword is ipped, the constraint is still satised. Hence, we can map a user bit to {100, 010} and then encode these words by a systematic rate-3/5 ECC to obtain an overall rate of 1/5; see Figure 1.9. The
14
message 100 user bit 0 Constrained 1 Encoder 010 100 010 Systematic ECC Encoder 11 parity 00 = 10101 01000
Figure 1.9: Encoding diagram of Example 1. period N is 5 and the insertion rate is 2/5. The beauty of this scheme lies in the simplicity of the decoding process. At the receiver, the ECC is rst decoded with no loss of soft information since the entire sequence received from the channel decoder can be directly used by the ECC decoder. This is followed by the decoding of the modulation code with no error propagation. This method also facilitates the joint iterative decoding of the constraint and the ECC. Simple Bit Insertion Fan and Cio [24, 21] introduced a simple bit-insertion scheme which starts with a message sequence that satises a tighter MTR(j a) constraint for the code C1 . A parity bit is then inserted in between sets of L bits to obtain a sequence which satises To maximize the number of parity bits, the value of L is chosen to be ja+1 . A a Clearly, for a given value of L, the insertion rate is
1 , L+1
the target MTR(j) constraint. The number L is required to satisfy ja+ ja+1 j. L schematic diagram of simple bit insertion is pictured in Fig. 1.10.
since L modulated bits
alternate with one inserted parity bit. The insertion rate places a restriction on the rate of the ECC since the number of parity bits from the ECC cannot exceed the number of inserted bits, so RECC
L . L+1
The overall rate of this insertion scheme
by the capacity cap(MTR(j a)), it is desirable to chose small values of a, such as
is given by cap(MTR(j a)) RECC . Since the overall modulation rate is determined
15
Message C1 Message
ECC
Parity
Figure 1.10: Schematic diagram of bit insertion. a = 1, for higher rates.
Constrained Systems with Unconstrained Positions The simple bit insertion scheme was then extended by Wijngaarden and Immink [66] to allow more than one parity bit insertion after every set of L message bits. Additionally, they also considered nonuniform parity bit insertion, i.e., allowing L to vary. The insertion rates of such congurations are no longer restricted to the form 1/L for some L. They showed that the resulting code rates could be higher than those obtained from Fan and Cios scheme. They also observed that the patterns that achieve higher code rates are highly dependent on the structure of the constraint and on the length of the repeating pattern of parity insertions. A periodic conguration of positions such that all position indices i (mod N ) U correspond to values that can be ipped freely and independently without violating the constraint. This is also called a periodic (U, N ) conguration. The code in Example 1 is a periodic ({2, 4}, 5) conguration. Wijngaarden and Immink named this extended constrained systems with unconstrained positions and provided a recursive enumerative technique for computing the number of codewords in their ad-hoc low complexity combinatorial constructions of block codes for some RLL(0, k) and (0, G/I) constraints. or periodic pattern is dened by specifying a period N and a set U {0, 1, . . . , N 1}
16
This scheme was later explored for the construction of nite-state codes for completely general constraints by Campello, Marcus, New, and Wilson [11]. They gave the construction of a trellis graph GU,N corresponding to a periodic (U, N ) conguration, where the nite-state graph of a constraint was concatenated (in an appropriate sense) N times (for N phases) and transitions were allowed to pass from phase to phase. In the phases indexed by U , certain edges are restricted. This graph enables the straightforward computation of the code rate of any (U, N ) conguration since the code rate is the logarithm of the spectral radius of the adjacency matrix of GU,N . An important question raised in [11] is that of the asymptotic optimal code rates for MTR(j) constraints. This was answered explicitly for MTR(1) and MTR(2), while a lower bound for MTR(j) with j 3 was established using a variant of bit bit after every L = ja+1 message bits to achieve an insertion rate of 1/(L + 1) as in a j a + 1 message bits as pictured in Fig. 1.11. Thus, the insertion rate achieved is a/(j a + 1) while the resulting code rate is cap(MTR(j a)) 1
a ja+1
insertion, which we refer to as standard bit-stung. Instead of stung a single parity simple bit insertion, standard bit-stung inserts a contiguous parity bits after every . Clearly,
if a = 1 (corresponding to L = j), standard bit-stung is identical to simple bit insertion, while for a > 1 and j a + 1 divisible by a, standard bit-stung achieves periodic patterns.
a a
the same insertion rate and code rate as simple bit insertion despite having dierent
j+1
j+1
Figure 1.11: A schematic picture of standard bit-stung. To solve the problem of extrapolating existing code designs to meet dierent ECC rates (i.e., ECC power), time-sharing was proposed in [11]. Time-sharing refers to the combination of existing codes in proportionate amounts to obtain any arbitrary insertion rate in the allowable range and simultaneously achieve a code rate that is proportionately higher. Using time-sharing, RECC can always be made exactly equal
17
to 1 , where is the insertion rate.
This idea was illustrated for the MTR(j) constraint, where they constructed a
code whose insertion rate is not a multiple of 1/(j + 1) by using a weighted average of two bit-stung schemes. For a/(j + 1) < < (a + 1)/(j + 1), take a weighted average of a-bit-stung-MTR(j) and (a + 1)-bit-stung-MTR(j) as follows: subdivide an interval of some large length N into two subintervals, and do a-bit-stung in the rst subinterval and (a + 1)-bit-stung in the second subinterval. The subintervals should have lengths (1 )N and N , where =
a j+1 1 j+1
The asymptotic code rate of such a block code is given by (1 ) ja+1 ja cap(MTR(j a)) + cap(MTR(j a + 1)). j+1 j+1
The area below the piecewise-linear curve h() connecting the following j + 1 points (0, cap(MTR(j)))
j 1 ( j+1 , j+1 cap(MTR(j 1))) j1 2 ( j+1 , j+1 cap(MTR(j 2)))
j 1 ( j+1 , j+1 cap(MTR(0))),
is thus an achievable rate region of this scheme for MTR(j) [11, Proposition 1]. In fact, h() is optimal for MTR(1) and MTR(2) as shown in [11, Theorem 3]. While it is not known whether h() is optimal for MTR(3), it was shown in [11] that h() is not optimal for MTR(4). It can be seen later in Chapter 6 that h() is not optimal for MTR(j), with j 4 using the method of irregular bit-stung.
18
cap(MTR(j)) cap(MTR(j a)) a)) 2 Q P 1
L cap(MTR(j L+1 j+1a cap(MTR(j j+1
a))
0
1 L+1 a j+1
Figure 1.12: Gradient comparison of simple bit comparison (point Q) and standard bit-stung (point P). Rate Comparisons The way simple bit insertion scheme attempts to meet dierent ECC rates is to vary L. For a xed L however, timesharing is performed between segments with no insertions and segments with 1-in-L bit insertions. Thus, for a given a (thereby xing L), we plot in Fig. 1.12, the points of operation of simple bit insertion (point Q) and standard bit-stung (point P). It is easy to check that point Q always lies under the straight line connecting (0, cap(MTR(j))) and P. We verify this by computing the gradient 1 of the line from Q to P, which yields 1 = cap(MTR(j a)). On the other hand, the slope due to standard bit-stung is given by 2 = j+1 (cap(MTR(j)) cap(MTR(j a))) + 1 , a
insertion scheme is contained within the framework of standard bit-stung.
which is clearly steeper than 1 . Thus, the time-shared version of the simple bit The explicit rate regions of simple bit insertion and standard bit-stung h() are plotted in Fig. 1.13 for MTR(4). As remarked earlier, h() is not optimal for MTR(4). As for simple bit insertion, observe the zig-zag frontier of the rate region. This is the actual rate region specied by the scheme. If time-sharing between codes
19
with varying L were permitted in simple bit insertion, the new rate region would be its convex hull. Even so, the rate region of simple bit insertion is contained strictly within that of standard bit-stung. 0.9752
0.7574
+ + +
h() for MTR(4) Simple bit insertion
1 5
2 5
3 5
4 5
Figure 1.13: Rate regions of simple bit insertion and standard bit-stung for MTR(4).
Using a Systematic Modulation Encoder As a simple method for facilitating soft decoding, Fan [21] argued for the use of a reverse concatenation scheme that implements a high-rate ( cap(MTR(j))) code C1 as the main constrained code and a systematic modulation code C2 with rate j/(j +1) as the auxiliary code. Its overall code rate is given by RRevCon = 1
1 cap(MTR(j))
1+
j+1 j
1 RECC
. 1
(1.1)
We point out that RRevCon is always below the lower bound h() as this reverse concatenation scheme is a special instance of standard bit-stung. This can be seen as follows. A user data stream u is encoded via C1 to a modulated sequence w
20
satisfying MTR(j) and is subsequently encoded via a systematic ECC to produce a parity stream v. The operation of C2 encodes v into a stream x by inserting a 1 after every j bits of v. Thus the resulting encoded stream is the concatenated streams wx. Here, w is the result of 0-bit-stung in MTR(j), while x can be viewed as the sequence obtained from j-bit-stung, i.e., starting with the all 1s sequence r (which is the only element of MTR(0)) and inserting j bits from v at a time between is = |v|/(|w| + each consecutive 1 in s. Note that RECC = |w|/(|w| + |v|), while the insertion rate Eqn. (1.1) reduces to
j+1 |v|). j
It is straightforward to show by direct computation that
RRevCon = cap(MTR(j)) 1
j+1 j
which demonstrates the linear dependence of RRevCon on insertion rate . The rate region of RRevCon is plotted in Fig. 1.14 for MTR(4) in comparison with that of simple bit-insertion and standard bit-stung. 0.9752
0.7574
+ + +
h() for MTR(4) Simple bit insertion Rev. Concatenation with Systematic Mod. Encoder
1 5
2 5
3 5
4 5
Figure 1.14: Rate regions of simple bit insertion, standard bit-stung h() and reverse concatenation with a systematic modulation encoder C2 for MTR(4).
1.5. SUMMARY OF CONTRIBUTIONS
21
Other Codes based on Bit Insertion Other recent work based on the bit insertion scheme includes that of Cideciyan and Eleftheriou [13, 14] from IBM Research. They cascaded a modulation encoder with a parity-check encoder followed by a precoder to derive an ecient design for combined MTR/parity-check codes that largely preserves the properties of the MTR code. Their code design methodology is based on the matrix transformation of a set of desired parity-check equations at the input of a partial-response channel into a set of paritycheck equations at the input of the precoder. In particular, they constructed a dualparity code with rate 96/104 that is based on a sixfold concatenation of a special MTR(3, 8) code [15], followed by a parity check encoder. To derive the parity-check equations, the nonlinear MTR code is rst used in conjunction with a 1/(1 D) precoder and the detector error events are analyzed. It turns out that the four error events that are the most likely are of the forms +, +, + +, and
+00+. All of these four error events can be detected with two parity bits that check either all-even or all-odd positions in the 104-bit codeword. This requirement is transformed accordingly into a parity check matrix with two rows at the precoder input. The newly congured parity check encoder then inserts 2 bits at carefully selected bit positions (using the bit insertion technique) to reduce both the encoding delay and the degradation of the j constraint prole, and to satisfy both parity check equations. Simulation results using this code over a Lorentzian recording channel with a 16-state detector followed by a SISO parity check decoder demonstrated that this code signicantly outperformed a single-parity code for channels dominated by electronics noise, with coding gains of around 1 to 2 dB. In fact, this particular rate-96/104 dual-parity code has been implemented extensively in commercial disk drives.
1.5
Summary of Contributions
The main topic of this dissertation is the analysis and design of constrained codes that facilitate soft iterative decoding by allowing certain positions in the code to take
22
on any value freely without violating the constraint, and using those positions for the insertion of ECC parity bits. Ecient designs are patterns of unconstrained positions that meet the redundancy of the ECC exactly with no wastage every unconstrained position is lled in by a parity bit. Good designs are those that admit a high code rate close to the optimal. We seek to achieve both ecient and good designs here, and address the optimization problem for nding a pattern that achieves (or comes close to achieving) the optimal code rate for a given density of unconstrained positions (i.e., insertion rate). The importance of studying the asymptotic code rates of this scheme is thus twofold. First, it supports the design of nite-state sliding block constrained codes with long block lengths that achieve rates close to the optimal. Second, the notion of timesharing can be applied to codes of dierent rational insertion rates i to produce a code with an insertion rate that is close to any arbitrary insertion rate in the allowable range, thereby making it possible to meet the rate of any ECC. The original contributions in the various chapters are summarized below. Most of Chapters 2-4 and 6 are found in our papers [56, 55], while Chapter 7 is found in [58, 59], and Chapter 5 is found in [57]. Gave a more precise understanding of how earlier approaches compare previously discussed in Section 1.4.2. Introduced a general graph construction G for analyzing constrained systems with unconstrained positions. We extend any given binary con straint S to a new set of sequences S over the extended alphabet {0, 1, }, where represents an unconstrained position. We prove in Section 2.3 that S is a con-
with constrained systems with unconstrained positions. This has been
strained system presented by an explicitly given graph presentation G. This graph G is unique for a constraint S in that it is independent of blocklength or any prespecication of unconstrained positions, unlike trellis-based constructions as in [11]. Established several properties of G. We show in Section 2.4 that the con struction of G yields a deterministic graph that has several interesting features. In particular, we show for nite-type constraints that G has only one nontrivial
23
irreducible component, which eases the analysis of the tradeo between insertion rate and code rate for such constraints. Provided a general denition of the tradeo function f based on G and
compared it to previous denitions. We give a formal denition of the tradeo
function f in Section 2.5 using the sequences of labels obtained by traversing paths in G. This represents the frontier of the achievable code rate region over the range of insertion rates permissible for the constraint. We compare our denition of the tradeo function to several alternative notions of tradeo functions, namely the tradeo function f (p) based on periodic patterns of unconstrained positions and the denition f (o) given in [11], based on sequences that achieve the target insertion rate exactly. This is given in Sections 3.4, 4.2.2 and 4.4. Determined the rationality and computability of the maximum insertion
rate of a constraint. The maximum insertion rate of a constraint corresponds to the lowest rate (or highest redundancy) permissible for any ECC that is to be used with the scheme, and is thus a rst check of compatibility for putting together an existing ECC and modulation encoder. In Chapter 3, we show that
the maximum insertion rate is given by the largest density of labels of any simple cycle in G; in particular, it is rational. We also reduce the problem of computing the maximum insertion rate to that of nding the maximum-cost-to-time ratio in computer science, which can be done using a fast polynomial time algorithm of Karp [39]. Explicit closed form expressions of the maximum insertion rates for RLL(d, k) and MTR(j, k) constraints are obtained in Section 3.3. Established continuity and concavity of the tradeo function for irre-
ducible, nite-type constraints, which are dened in Section 2.2. We obtain several properties of the tradeo function, including monotonicity and left continuity in Section 4.1. These are combined with structural properties of G to
prove continuity and concavity of the tradeo functions for irreducible, nite-type constraints in Section 4.2.
24
Showed that periodic congurations achieve optimal code rates for nitetype constraints. In Section 4.2.2, we show that for nite-type constraints, periodic congurations achieve the optimal code rate for a given rational insertion rate. This gives formal justication for the intuitive use of periodic patterns of unconstrained positions, and in some cases simplies the analysis of the tradeo function. Implemented timesharing between multiple components in G. In Sec tion 4.3, we show that timesharing between components of G can be achieved, depending on the connectivity of the graph. For example if G has two components G1 and G2 where G2 can be reached from G1 , then the tradeo function of G is the least concave majorant of the tradeo functions of G1 and G2 . This corresponds to a weighted average of the codes obtained from the labels of cycles in G1 and in G2 (i.e., using the code from G1 a certain fraction of the time and that of G2 the rest of the time). This can be used to prove concavity and continuity of the tradeo function of irreducible constraints. Showed that the NRZI tradeo function of RLL(0, k) and MTR(j) are greater than their NRZ counterparts. This is proved in Section 4.5.
Determined explicit descriptions of certain tradeo functions. Prior to constraints. In Sections 5.2 and 5.3, we add MTR(2, 2) and RLL(d, 2d + 1) for all d 1 to the list of known cases. We also present some results on optimal periodic
our work, explicit descriptions of the tradeo function were known for only two
congurations for the MTR(j) constraint at low insertion rates in Section 5.4. In particular, we prove that the tradeo function for MTR(3) in the low insertion rate region is a straight line that can be achieved by timesharing of bit-stung schemes. We also show a similar result for MTR(j) with j 7, under the assumption that no two consecutive unconstrained positions are allowed. In the process of proving the latter, we show that the conguration that optimizes the code rate is one that spreads the unconstrained positions apart exactly by j (except for one large interval of constrained positions at the end of the block).
25
Developed upper and lower bounds for tradeo functions. Upper and
lower bounds on the tradeo function for an arbitrary constraint are outlined in Chapter 6. Examples and plots of the bounds are given for some runlength constraints and the bounds are compared.
Analyzed time-varying MTR constraints and related them to MTR con-
strained systems with unconstrained positions. In Chapter 7, we analyze the mathematical structure of time-varying MTR constraints and present new results on their minimum graph presentations, general analytic upper bounds on their capacity, and provide a complete linear ordering of their capacities for repeating patterns up to length 4. A connection to MTR constrained systems with unconstrained positions is drawn, and this is used to derive an upper bound on the tradeo function for MTR constraints.
Chapter 2 Constrained Systems with Unconstrained Positions

In this chapter, we consider generalized bit insertion, known as constrained systems with unconstrained positions, as introduced in Section 1.4.2. For this scheme, we present a new graph construction that is independent of blocklength or any prespecication of unconstrained positions. It turns out that the properties and structure of this new graph are integral to a new systematic approach to understanding system tradeos between optimal code rates and ECC power. We begin by formulating the problem of nding the achievable rate region of the constrained systems with unconstrained positions scheme as an optimization problem in Section 2.1. This is followed by a guide to notation and technical denitions commonly used in constrained coding in Section 2.2. Section 2.3 presents the graph construction that will serve as the main tool in analyzing this scheme. Section 2.4 explores various properties of the graph construction. Section 2.5 relates the optimal code rates to the sequences of labels obtained by traversing paths in this graph.
2.1
Optimizing for Asymptotic Rates
The problem of determining the asymptotic optimal code rates for a given constraint using the scheme of constrained systems with unconstrained positions is a dicult 26
2.1. OPTIMIZING FOR ASYMPTOTIC RATES
27
one. As pointed out by S. Boyd, the problem can be formulated as an optimization problem over an integer lattice. Given a constraint S, a period N , and a xed number of unconstrained positions k (such that the insertion rate is xed at k/N ), let x be a vector representing the conguration, with xi = 1 if the i-th position is unconstrained and xi = 0 otherwise. The objective function we wish to maximize is the code rate f (S, x), which is a function of S and x. The optimization problem is then: maximize subject to
i=0
f (S, x)
N 1
xi = k xi {0, 1}
There are several things to note about this optimization problem: With the xi s restricted to 0 or 1, this is an instance of integer programming.
problem [17, pg 335] in computer science, where a thief robbing a store tries to take k out of N items as he can carry only k of them but wants to maximize the value of the k items he takes. The dierence is that the code rate we are attempting to maximize here is typically implicit and nonlinear in the xi s, as it is highly dependent on the structure of the underlying constraint. One way to compute the objective function is to use the approach in [11, page 22]
In fact, this problem is similar in concept to a variant of the 0 1 Knapsack
to rst construct the trellis presentation GU,N of a periodic (U, N ) conguration, nd its adjacency matrix AGU,N , and evaluate the logarithm of the spectral radius of AGU,N to obtain the code rate. For specic constraints like MTR(j), it is possible to analyze GU,N to obtain two
and B corresponds to the adjacency matrix of a phase i U . Then taking the Mi = B if i U and Mi = A if i U yields the code rate. / logarithm of the spectral radius of the matrix product M = M0 M1 MN 1 with
matrices A and B, where A corresponds to the adjacency matrix of a phase i U /
28CHAPTER 2. CONSTRAINED SYSTEMS WITH UNCONSTRAINED POSITIONS
Before our work in this dissertation and in our paper [56], the objective func-
tion f (S, x) was not known to be concave, except for the explicit descriptions for MTR(1) and MTR(2) in [11]. Even though we now know that f (S, x) is concave,
convex optimization techniques cannot be easily adapted to solve this problem because of the restrictions imposed by the integral feasible set. A possibility is to the code rate using the method in [11] since that requires looking at fractional portions of an unconstrained position. To nd the asymptotic code rates for a xed insertion rate , we require an outer maximization loop over all N as follows. max max
N 1 x
relax the 0 1 conditions on xi to 0 xi 1, but it is not clear how to compute
f (S, x, N )
N 1
subject to
i=0
xi = N xi {0, 1}
This is a much harder problem to solve. If the approach of [11] is applied, this entails the construction of GU,N for all unconstrained sets U and all possible periods N such that |U |/N = . This leads us to consider the following questions: 1. Is there a graph presentation that does not depend on the unconstrained set and period, but only on the given constraint (i.e., one per constraint)? 2. If there is, can it be used as a tool in systematic code design, i.e., give an unconstrained set U with some guaranteed code rate (without having to resort to exhaustive search or random guessing to nd U )? 3. More importantly, if there is such a presentation, what is the achievable rate region for the constraint as a function of the insertion rate? We give an armative answer to the rst question by giving details of its construction and features in the remaining sections in this chapter. Partial answers to
2.2. CONSTRAINED CODING AND NOTATION
29
the second and third questions are given in subsequent chapters, but rst we need some notation and technical concepts from the theory of constrained codes.
2.2
Constrained Coding and Notation
In this section, we introduce some notation and technical concepts on constrained coding that will be needed for the rest of the chapters. For a more thorough treatment, the reader may consult [47]. A labeled graph G = (V, E, L) consists of a nite set of states V = VG , a nite set of edges E = EG , where each edge e has an initial state (e) and a terminal state (e) both in V , and an edge labeling L = LG : E , where is a nite alphabet. We notation u v to denote an edge or transition from state u to state v with label a. that (ei+1 ) = (ei ) for i = 1, 2, . . . , l 1. The state sequence of a path e1 e2 . . . el
a
allow loops (e) = (e) and multiple edges. For convenience, we sometimes use the A path of length l in a graph G is a nite sequence of edges e1 e2 . . . el such
is the sequence (e1 )(e2 ) . . . (el ) (el ). A cycle in G is a path e1 e2 . . . el where state sequence consists of distinct states (except the rst and last states). Two labeled graphs can be viewed as the same if there is a labeled graph isomorphism from one to the other, i.e., a bijection from states to states and a bijection from edges to edges which preserves initial states, terminal states and labels. For a graph G, the adjacency matrix A = AG is the |VG | |VG | matrix whose
(e1 ) = (el ). A cycle of length one is a loop. A simple cycle is a cycle in which the
entries are indexed by the states of G with Au,v being the number of edges from u to v in G. A constrained system or constraint S = S(G) is the set of nite sequences obtained
by reading the edge labels of a path in a labeled graph G. Such a graph is called a presentation of the constraint. An element of S is called a constrained sequence or a codeword. We will use to denote the empty word, which is in S via an empty path. A homing word for a state v in a labeled graph G is a word w such that all paths in G that generate w terminate at v. We use wz to denote the concatenation of two words w and z, wn to denote the concatenation of n copies of w, |w| to denote the
length of w, and wi to denote the ith symbol of w. A graph G is irreducible if any state in G can be reached from any other state. A reducible graph can be viewed as a graph consisting of many irreducible subgraphs, called the irreducible components with possible transitions or transitional edges from component to component. We say that an irreducible component G1 can be reached from an irreducible component G2 if there is a path from a state in G2 to a state in G1 . An irreducible component is trivial if it has only one state and has no incoming edge or has no outgoing edge (i.e., it is a sink or a source state). A graph is primitive if there exists a positive integer p such that for every ordered pair (u, v) of states, there is a path of length p from u to v. A constrained system is said to be irreducible (primitive) if it has an irreducible (primitive) presentation. A labeled graph is deterministic if at each state, all outgoing edges carry distinct labels. A labeled graph has nite memory if there is an integer m such that all paths of length m with the same labeling terminate at the same state. The smallest such m is called the memory of the graph. A constrained system S is nite-type or has nite memory if there is an integer m such that, for any symbol a and any word sux of x of length m. The smallest m for which this holds is called the memory of S. It is well-known that a constrained system S has nite memory if and only if there is a presentation of S with nite memory. Moreover, the memory of S is the smallest memory of any presentation of S with nite memory. Note that the MTR(j), RLL(d, k), and MTR(j, k) constraints are all primitive and nite-type. The follower set of u in a labeled graph G, denoted by F(u) = FG (u), is dened x S of length at least m, we have xa S if and only if ya S, where y is the
to be the set of all nite words that can be generated starting from u in G. The
graph G is reduced if for any two distinct states u and v in G, FG (u) = FG (v). For an irreducible constraint, there is a unique minimal (in terms of the number of states) deterministic presentation, called the Shannon cover. It is the unique presentation that is irreducible, deterministic, and reduced. Every state of the Shannon cover has a homing word.
2.2. CONSTRAINED CODING AND NOTATION
31
We also dene the follower set of a nite word x (rather than a state) as F(x) = FS (x) = {y S : xy S}. We allow x to be the empty word , in which case the follower set is all of S. Note that where the union is over the terminal states u of all paths with label x. Since a constrained system is presented by a nite-state graph, it has nitely many follower sets. F(x) is empty if x is not in S. If G is a presentation of S, then FS (x) =
u
FG (u),
The follower sets can be used to construct a special presentation G, called the follower set graph, of a constrained system S. The states of G are the follower sets of all words in S and the transitions are given by F(x) F(xa), where a and xa S. Note in particular that the follower set graph is detera
ministic. Note also that whenever a word x is the label of a path in the follower set turns out to be the subgraph of the follower set graph, determined by follower sets of homing words. graph ending at state F(y), then F(y) F(x). The Shannon cover, dened above,
As mentioned in Section 1.2, the capacity of a constrained system S, denoted by cap(S) (or capS ), is dened as cap(S) = lim log M (S, n) , n
where M (S, n) is the number of words of length n in S. Let G be a deterministic presentation of S. The capacity can be computed by cap(S) = log (AG ), where (A) denotes the spectral radius of A, which is dened as the maximum of the absolute values of the eigenvalues of A.
2.3
The Graph Presentation G
One of the main contributions in this work is the introduction of a constrained system that can be presented by a graph whose edge labels include the use of an unconstrained symbol denoted by . This symbol represents an unconstrained position and can be thought of as a wildcard symbol that is allowed to take on any symbol in the alphabet of the constraint. Primarily for notational simplicity, in this work we focus most results can be easily extended to non-binary constraints. our attention to binary constraints (i.e., constraints over the alphabet {0, 1}), but Let A2 and A3 denote the alphabets {0, 1} and {0, 1, } respectively. Given an
word). For a given word w A , dene the set of all llings of w as 3
alphabet A, dene A to be the set of all nite words over A (including the empty
(w) = {x A : |x| = |w|, xi = wi if wi {0, 1}, and xi {0, 1} if wi = 2
}.
That is, x is a lling of w if x is obtained from w by substituting each occurrence of example, if w = 0 1 let (w) be given by (w) = {x AN : xi = wi if wi = 0 or 1, xi {0, 1} if wi = 2 Let S be a binary constrained system. Dene S = {w A : (w) S}. 3 in w is replaced independently by 0 or 1, a word in S is obtained. We will construct a graph G using S, and show that S is the constrained system presented by G. Dene the graph G from S as follows. States: All intersections of the follower sets of words in S. Thus, S is the collection of words w such that if each occurrence of }. in w by 0 or 1. Note that |(w)| = 2b(w) , where b(w) is the number of in w. For then (w) = {0010, 0011, 0110, 0111}. For convenience, we
extend the denition of to include right-innite words: for w a right-innite word,
2.3. THE GRAPH PRESENTATION G
33
Transitions: The transitions are given by

k k
i=1 k
F(xi ) F(xi ) F(xi )

1
i=1 k
F(xi 0) F(xi 1)
if xi 0 S for all 1 i k if xi 1 S for all 1 i k
i=1 k
i=1 k
i=1
i=1
F(xi 0) F(xi 1) if xi 0 and xi 1 S for all 1 i k
We need to check that G is well dened. Suppose that F(x) = F(y), then for any b {0, 1}, F(xb) = F(yb). Moreover, xb S if and only if yb S. Thus, G is well dened. Fig. 2.12.3 show some examples of G for some simple constraints. An important observation from the gures is that G may not be irreducible, even if the original binary constraint presentation G is. F() {} F(0)
0 1 G
1 F(1)
0 0 1 G
Figure 2.1: The Shannon cover G and G for the constraint that forbids 00 and 11.
0 1 G
F()
0 1 G
F(0)
Figure 2.2: The Shannon cover G and G for the RLL(0, 1) constraint. This leads to the main theorem in this section.
F() 1 0 1 0 1 1 0 F(1)
F(1) F(10) F(1) F(100) 0 0 0 0 1 F(10) 1 1 G 0 F(100) 0
{}
F(1000)
Figure 2.3: The Shannon cover G and G for the RLL(1, 3) constraint. Theorem 2 S is the constrained system presented by G. Proof. Let be a path in G with label w. Let k F(xi ) be the initial state of . i=1 By the denition of transitions of G, xi y S for all y (w). It follows that every subword of xi y is also in S. Hence y S for all y (w), which implies that w S.
Conversely, let w be a word of length l in S. Then there is a path from the state F() = S in G with label w. Specically,
l 1 2 3 F() u1 u2 ul1 ul ,
wl1
where ui =
x(w1 ...wi )
F(x).
2.3.1
Size of G
In general, G can be exponentially large (in the number of states); but if the constraint S has enough structure, G may have a reasonable size. For the three important classes of constraints that are mostly commonly used in digital storage systems, namely the MTR(j), MTR(j, k) and RLL(d, k) constraints, it turns out that the size of G ranges from linear to quadratic, as summarized in
2.3. THE GRAPH PRESENTATION G
35
Table 2.1. Size MTR(j) MTR(j, k) RLL(d, k) j+1 (j + 1)(k + 1) dk + k + 2d + 1 d2 Table 2.1: Sizes of G for some constraints For the MTR(j) constraint, the number of states in its G presentation is exactly j + 1. This can be seen from inspection of the graph below in Fig. 2.4.
0 1 1 1 2 0 0
j-1 1 j 0 0
F() 1 F(1) 1 F(11) 0 0
F(1j1 ) 1 F(1j ) 0 0
Figure 2.4: The Shannon cover G and G of MTR(j) For the MTR(j, k) constraint, the number of states in its G presentation is exactly Then F(x) is determined by the last run of 0 or 1 in x, e.g., F(011000) = F(000).
n i=1
(j + 1)(k + 1). This can be seen from the following argument. Let x MTR(j, k).
Moreover, if a b, then F(0b ) F(0a ) and F(1b ) F(1a ). Thus, for any collection of words xi S, 1 i n,
F(xi ) = F(0a ) F(1b ) for some 0 a k and 0 b j. This implies that the number of states of G is at most (j + 1)(k + 1).
To see that the number of states is exactly (j + 1)(k + 1), it suces to check that F(0a ) F(1b ) = F(0c ) F(1d ) if a = c or b = d. Without loss of generality, assume that 0 a < c k. Then 0ka is in F(0a ) F(1b ) but not in F(0c ) F(1d ), so the intersections of follower sets are distinct. Lastly, for the RLL(d, k) constraint, the number of states in its G presentation is exactly dk + k + 2d + 1 d2 . A sketch of the proof of the upper bound is given below. then F(10a ) F(10c ) F(10b ). With these observations, any intersection U of First it is easy to check that F(0a ) F(0b ) = F(0max{a,b} ). Note that if a b c,
follower sets can be written as the intersection of at most three sets of the forms F(0a ), F(10b ), F(10c ), for some a, b, c. Without loss of generality, assume that b c. Case 2: U = F(10b ) F(10c ), b < c. If d b, then U = F(10c ) which is in Case Case 1: U = F(10b ). There are k + 1 possible choices of U in this case.
1. If d > b, it can be shown that there are d(k d + 1) possible intersections. then U = F(10a ) F(10b ) which is Case 2. the same argument as in Case 3.
Case 3: U = F(0a ) F(10b ). If a b, then U = F(10b ) which is Case 1. If a > b,
Case 4: U = F(0a ) F(10b ) F(10c ). This case also falls into Cases 1 and 2 by
d possible choices of U in this case.
Case 5: U = F(0a ). If a d, then U = F(10a ) which is in Case 1. Thus there are
Combining Cases 1, 2, and 5, the number of states of G for RLL(d, k) is at most distinct). k + 1 + d(k d + 1) + d = dk + k + 2d + 1 d2 (since these states are not necessarily
2.4
Properties of G
In this section, several structural properties of G are established. Proposition 3 The graph G is deterministic and reduced. If u and v are states of G and u v, then FG (u) FG (v). Proof. The graph G is deterministic by denition. Let u be a state of G. Then FG (u) = {w A : (w) u}. 3 Hence no two states of G have the same follower set, and FG (u) FG (v) if u v. Proposition 4 If S has memory m, then G and S have memory m. Proof. Let 1 and 2 be paths of length m in G with the same label w. Let
k1 i=1
terminal states of the two paths are
F(xi ) and
k2 j=1
F(yj ) be the initial states of 1 and 2 respectively. Then the

z(w) k1 i=1
F(xi z) and
z(w)
k2 j=1
F(yj z).
2.4. PROPERTIES OF G
37
Suppose that S has memory m. Then for any words x and y of length at least m F(xi z) = F(yj z) for all xi , yj , z. Thus 1 and 2 have the same terminal state, which is z(w) F(z). This proves that G has memory at most m. This in turn implies that S has memory at most m. Next we show that G has memory exactly m. Since S has memory m, there exist x, y S of length at least m with the same sux of length m 1 such that F(x) = F(y). In G, there are two paths from F() to F(x) and F(y) with label in S, F(x) = F(y) if x and y have the same sux of length m. It follows that
x and y respectively. The last portions of these paths have the same label, namely
the sux of length m 1 of x and y, but they do not terminate at the same state. Therefore if G has nite memory, its memory is at least m. Since we have shown that G has memory at most m, we conclude that G must have memory m. To see that S has memory exactly m, it suces to show that FS (x) = FS (y). From the denition of S, a binary word xz S if and only if xz S. Since F(x) = F(y), it follows that FS (x) = FS (y). Notice in Fig. 2.12.4 that in each case, the G presentations contains the Shannon cover of the original constraint as a subgraph. This feature holds in general. Proposition 5 Let S be an irreducible constrained system and G be the Shannon cover for S. Let G be constructed from S as described above. Then there is a unique subgraph H of G that is isomorphic to G. Proof. Since G is the Shannon cover for S, there is a homing word for every state of G. Let u be a state of G and h be a corresponding homing word. It is clear that FS (h) = FG (u). This implies that for any homing words h and h of the same FS (h) = FS (h ), then u = u because the Shannon cover G is reduced. state, FS (h) = FS (h ). Conversely, if h and h are homing words of u and u , and Let u1 , . . . , un be the states of G. For 1 i n, let hi be a homing word of ui in
G. Since G is reduced, F(hi ) = FG (ui ) are distinct. Thus F(hi ) are distinct states in G. We will show that there is a subgraph H of G with VH = {F(hi ) : 1 i n} that is isomorphic to G.
Suppose that there is an edge from ui to uj in G with label b A2 . Then hi b S and hence there is a transition from F(hi ) to F(hi b) with label b in G. Since hi b is a This denes the transitions of H.
homing word of uj , F(hi b) = F(hj ). Thus this transition indeed terminates at F(hj ). Conversely, suppose that there is an edge in G from F(hi ) to F(hj ) with label
state uj . It follows that there is an edge in G from ui to uj with label b.
b A2 . Then F(hi b) = F(hj ) and thus hi b and hj are homing words of the same To show that H is a unique such subgraph of G, it suces to show that each hi is a homing word of G. First, there is a path from F() to F(hi ) with label hi . Suppose there is another path starting from is
k j=1 k j=1
F(zj hi ). Since every word zj hi is a homing word of ui in G, F(zj hi ) = F(hi ) and so the terminal state is indeed F(hi ). It follows that hi is a homing word in G.
F(zj ) with label hi . Then its terminal state
Throughout, we will let H denote the unique subgraph of G isomorphic to the Shannon cover G. Dene G to be the irreducible component of G that contains H. The following proposition shows the importance of this component when the constraint is nite-type.
Proposition 6 If S is nite-type, then G is the only non-trivial irreducible compo nent of G.
Proof. Let m be the memory of S. By Proposition 4, S and G have memory m. Let G1 be a non-trivial irreducible component of G. There is a cycle in G1 of length at least m with some label w. Let u be the initial state of . Then any Conversely, there is an irreducible component G2 that can be reached from H such that S(G1 ) S(G2 ). So, there is a path starting in H with label w. But G has x (w) is a homing word for a state in H and x F(u), so H is accessible from G1 .
memory m, so terminates at u. Therefore, G1 is reachable from H, which shows that G1 is the irreducible component containing H. It turns out that Proposition 6 does not hold for non-nite-type constraints.
2.5. OPTIMAL CODE RATES OVER G
39
2.5
Optimal Code Rates over G
The graph G contains all information on the sets of unconstrained positions admis sible for the constraint S. In particular, traversing a path of length L in G and reading its label l() induces a (U, N ) conguration, where N = L and U is the index set indicating the locations of the boxes in l(). Using the method in [11], it is straightforward to compute the code rate for this (U, N ) conguration, and the insertion rate is |U |/N . Clearly, the (insertion rate, code rate) pair obtained lies in the achievable rate region of the constraint. However, we are interested to nd the optimal achievable rate region for any given constraint, or equivalently, the frontier of the achievable rate region, which we call the tradeo function. A precise denition is given in the following.
2.5.1
Dening the Tradeo Function
Let q be a positive integer, and I {0, . . . , q 1}. As before, S denotes a binary constrained system. Dene M (q, I) to be the number of words w S such that: |w| = q, wi = if and only if i I. in w; we say that w satises I.
Hence the set I represents the positions of
insertion rate to be
{0, 1, . . . , q 1} and limq |Iq |/q = . We dene the tradeo function for S at log M (q, Iq ) . q
For 0 1, dene I() to be the set of all sequences (Iq ) such that Iq
f () = fS () =
sup lim sup

(Iq )I() q
(2.1)
Note that f (0) = cap(S) as seen by taking each Iq to be empty. With this notation, dene the maximum insertion rate to be = (S) = sup .
f ()>0
Note that the supremum in the denition of can in fact be taken over f () 0 (see the second half of the proof of Theorem 14 in Chapter 3). as shown by the following proposition. Proposition 7 Let 0 1. There is a sequence (Iq ) I() such that f () = lim sup
q
The outer supremum in the denition of the tradeo function is in fact a maximum,
log M (q, Iq ) . q
Proof. We use a diagonalization argument. From the denition of f , there exist q1 < q2 < and Iqn {1, . . . , qn } such that |Iqn | qn log M qn , Iqn qn
(n) (n) (n)
1 , n 1 . n
(n)
f ()
0 n, and Iq otherwise. It is easy to see that (Iq ) I(). Moreover, (n)
0 Fix Iq I() arbitrarily. For each q N, dene Iq to be Iqn if q = qn for some
lim sup
q
log M (qn , Iqn ) log M (q, Iq ) lim sup f (). q qn n
Hence (Iq ) achieves the supremum in the denition of f (). The following example shows why the lim sup is used in the denition of f : the limit may not exist, even for reasonable choices of (Iq ). Example 8 Let S be the RLL(0, 1) constraint. Its Shannon cover G and G are shown in Fig. 2.2. We will give an example of (Iq ) I(1/3) such that limq (log M (q, Iq ))/q by a binary word x, where xj = 1 if j J. does not exist. To simplify the construction of Iq , we represent a a set J {1, 2, . . . , q} We construct a right-innite sequence x recursively as follows. Let y = 010010
k 3k. We dene Iq = {i {1, . . . , q} : xi = 1}. It is clear that (Iq ) I(1/3).
and z = 100010. Set k 2 and x yz. Then recursively compute x xy k z k and
41
Let A= 1 1 1 0 , B= 0 1 0 0 .
The matrix A is the part of the adjacency matrix AG of G that corresponds to edges with label 0 and 1, while the matrix B is the part of AG that corresponds to edges with label . Let C(q) =
q i=1
checked that M (q, Iq ) = c11 + c12 , where c11 and c12 are the (1, 1) and (1, 2) entries of C(q) respectively. The basic building blocks that correspond to y k and z k are (ABAABA)k = 1 0 1 0 , (BAAABA)k = 2k 0 0
n
Ci , where Ci = A if i Iq and Ci = B if i Iq . It can be /
It follows that for any n N, if q = 12 3n , then M (q, Iq ) = 23 = M (2q, I2q ). Thus, These are two subsequences of (log M (q, Iq ))/q that converge to dierent values and hence the limit does not exist. The following two examples show that f () need not be zero. Example 9 The Shannon cover G and G of the 2-charge constraint is shown in Fig. 2.5. The capacity for this constraint is 0.5, and so f (0) = 0.5. However, there is no word in S that has more than two . Therefore f () = when > 0. Example 10 Let G be the graph in Fig. 2.6 without the dashed edges and let S = S(G). The constraint S is primitive since G is irreducible and aperiodic. The graph G has memory 7 and hence S is nite-type. The irreducible component G of G that contains H is obtained by adding the dashed edges. It can be shown that = 1/8 and f () = 1/8 using Theorem 14 in Chapter 3. This example shows that f () may be strictly positive. Later, in Chapter 3, it will be shown that such examples are very common among nite-type constraints. For example, for any RLL(d, k) constraint with k < 2d + 1, = 0 but f () is nonzero. for q = 12 3n , (log M (q, Iq ))/q = 1/12. For q = 24 3n , (log M (q, Iq ))/q = 1/24.
1 1 0 F(0) F(11) 1 1 0 0 F(1) 1 1 0 0 F(1) F(00) F(0) F()
F(11) 1 0 {}
F(0) F(1) 1 0
0 F(00)
Figure 2.5: The Shannon cover G and G for the 2-charge constraint
2.5.2
Periodic Tradeo Functions

, the
In the spirit of [11], which examined constructions with periodic insertions of
notion of periodicity can be dened in our new notation as follows. A set I N is
if
i I if and only if i i (mod N ) for some i U . A sequence (Iq ) I() is periodic
said to be periodic if there exists a period N N and U {0, . . . , N 1} such that
Iq Iq+1 for all q N, i.e., the sequence of sets is increasing; I=

q=1 Iq
is a periodic set.
In this case the sequence (Iq ) is determined by I. for periodic unconstrained positions is dened to be f (p) () = sup
(Iq )Ip ()
Let Ip () = {(Iq ) I() : (Iq ) is periodic}. The maximum achievable code rate log M (q, Iq ) . q
lim sup
q
43
0 1 0 1 1 0 0 0 0 1 1 1 1 1 0 1 0
0 1 1 0
Figure 2.6: An example showing that f () need not be zero, even when S is nite-type and primitive We call this the periodic tradeo function. Note that f (p) is dened only on the rationals 0 1, and f (p) () f (). In contrast to Example 8, if the sequence (Iq ) is periodic, the lim sup in the
denitions of f and f (p) can always be replaced by the limit, as is shown in the next proposition. Proposition 11 Let (Iq ) be a periodic sequence in I() with period p for some [0, ], such that M (q, Iq ) > 0 for all q. Then lim log M (q, Iq ) log M (kp, Ikp ) = inf . k q kp
In particular, this limit exists. Proof. Write aq = log M (q, Iq ), and consider the sequence (aq /q). Let bk = akp . We rst show that the limit of the subsequence (bk /k) = (akp /(kp)) exists. Note that for any positive integers k and q, M (kp + q, Ikp+q ) M (kp, Ikp )M (q, Iq ) because of the periodicity condition: for any codeword w of length kp + q, write w = w1 w2 where w1 has length kp and w2 has length q; then w1 is a codeword satisfying Ikp and w2 is a codeword satisfying Iq . Applying the above with q = jp and taking logs, we have
the subadditivity condition bj+k bj + bk . So by Feketes Subadditivity Lemma [43, Lemma 4.1.7], the limit of bk /k exists (and equals inf bk /k). Using the fact that M (kp, Ikp ) 2q M (kp + q, Ikp+q ), we have 2q M (kp, Ikp ) M (kp + q, Ikp+q ) M (kN, Ikp )M (q, Iq ). For any n, we have n = kp + q for some k and some q {0, , p 1}, and then kp akp q an akp log q + kp + q kp kp + q n kp kp + q Thus, by the Sandwich Theorem, the limit of
an n
exists and equals inf bk /k.
2.5.3
Relation to Previous Denitions
Let f (o) denote the denition of code rate in [11], which is repeated below in our notation for clarity of presentation: f (o) () = lim sup max
q
q
log |M (q, Iq )| . |Iq | q =
Note that this denition only makes sense for rational . It will be shown in later chapters that the following inequality among the various denitions of the tradeo functions holds: f (p) () f (o) () f (). Furthermore, studying paths in G yields a stronger statement: f (o) () = f () on the rationals 0 for irreducible constraints, and f () = f (p) () for nite-type constraints.
Chapter 3 Maximum Insertion Rates

Recall from the previous chapter that the maximum insertion rate for a given constraint S corresponds to the largest density of unconstrained positions in sequences in S that admits a nonnegative optimum code rate. Since the unconstrained positions are reserved for the insertion of ECC parity bits, the maximum insertion rate thus systematic ECC that can be used with the constraint S. Thus, one can use the maximum insertion rate to decide whether an o-the-shelf ECC is compatible with S in this particular scheme. Certainly, other performance factors will contribute to the nal decision of the ECC, but the maximum insertion rate still serves as a certicate explaining why the scheme is inadmissible for the constraint beyond a certain ECC rate or ECC power. Section 3.1 relates the maximum insertion rate of a given constraint S to the maximum ratio of number of to length of any path label in its G presentation. Section 3.2 describes a numerical solution to computing the maximum insertion rate for any constraint by drawing a connection between nding the maximum insertion rate in G to the well-known problem of nding the maximum cost-to-time ratio for circuit design and scheduling in computer science. Section 3.3 establishes closed form solutions for the maximum insertion rates of RLL(d, k) and MTR(j, k) constraints. Lastly, Section 3.4 investigates the tradeo function at the maximum insertion rate. 45
equals 1 RECC , where RECC is the rate of the lowest rate (or highest redundancy)
46
CHAPTER 3. MAXIMUM INSERTION RATES
3.1
Rationality of the Maximum Insertion Rate

in the label of to its length. We rst
Using the framework established in the previous chapter, consider a path in G and dene () to be the ratio of the number of show that is maximized on a simple cycle. Proposition 12 For any cycle in G, there exists a simple cycle such that () ().
Proof. Any cycle can be written as 1 1 2 , where 1 is the rst simple cycle (from left to right). Extract out 1 , which results in a new path 1 2 . Repeating this process until no simple cycles remain, we obtain a collection of simple cycles labels equals the number of 1 , 2 , . . . , k whose total length equals || and whose total number of in their in s label. Then () is the weighted average of
(i ), 1 i k. Hence there is a simple cycle i such that (i ) (). We dene =

all cycles in G
max
().
A cycle that achieves is called a max-insertion-rate cycle. Since a nite graph has only nitely many simple cycles, Proposition 12 implies that exists for any constraint and that only simple max-insertion-rate cycles need be considered in computing . Before presenting the main theorem in this section, a lemma on path decomposition is presented. Lemma 13 Any path in a nite graph can be decomposed as 1 e1 2 e2 . . . em1 m em , where the j are cycles (possibly of length 0), the ej are edges, and m is at most the number of distinct states visited by . Proof. Perform induction on the number of distinct states visited by as follows. Let ui be the initial state of the edge i so ui+1 is the terminal state of i . The lemma is trivial if only one state is visited, in which case must consist entirely of loops at that state. Assume the hypothesis is true when the number of distinct states visited is less than k (where k 2), and assume visits k distinct states. We then
3.1. RATIONALITY OF THE MAXIMUM INSERTION RATE
47
take 1 to be the subpath 1 . . . k1 1 where k1 is the time of the last visit to u1 , i.e., k1 = max{j : uj = u1 } (if k1 = 1, take the path to be empty). Write as 1 e1 . By the inductive hypothesis, can be decomposed as 2 e2 3 e3 . . . em1 m em with m k, which completes the proof. Our terminology makes it sound almost tautological that the maximum insertion rate of a constraint equals the density of boxes in the label of a max-insertion-rate cycle. We are now in a position to prove that this is indeed the case.
Theorem 14 The maximum insertion rate of a constraint S is equal to of its corresponding G presentation. Proof. Let be a simple max-insertion-rate cycle in G, so () = . To show that (), rst assume > () and f () 0. Then there exist q and Iq {1, . . . , q} such that |Iq | > q() + |VG |, and M (q, Iq ) 1. Hence there exists a path with label w that satises Iq . Applying Lemma 13 to the path , the number of
m
(3.1)
in w is at most
|VG | +
i=1
|i |(i ) |VG | + q(),
which contradicts (3.1). Hence f () = when > () and so (). w, and N = ||. For any > 0, there is an integer such that () < Let i1 be the rst position of k 1 k < = (). N N
Next, we show that (). Let w be the label of , k be the number of
in
consider the word (w )n . By viewing this word as having period N , replace the rst
in each period by 0 and 1 to obtain 2n words of length nN that satisfy InN . It
, and i mod N = i1 }. It is clear that (Iq ) I((k 1)/(N )). For an integer n,
in w. Take Iq = {i {0, 1, . . . , q 1} : wi
mod N
48
follows that M (q, Iq ) 2q/N when q is a multiple of N . Thus
k 1 N
1 > 0. N
Hence (k 1)/(N ) > () . Since this is true for any > 0, (). A direct consequence of Theorem 14 is the following corollary. Corollary 15 The maximum insertion rate of any constraint is rational.
3.2
Numerically Computing the Maximum Insertion Rate
Proposition 12 and Theorem 14 imply that the maximum insertion rate of a con straint can be determined from a simple max-insertion-rate cycle in its G presenta tion. Clearly, if G is irreducible and relatively small, one can nd a max-insertion-rate cycle by complete enumeration of all simple cycles in G. However, this approach is not usually practical as the number of simple cycles in G can be exponentially large. The problem of computation then boils down to nding a numerically ecient algorithm that looks for any max-insertion-rate cycle in G. First, observe that when a weight of 1 is assigned to the edges labeled in G and a weight of 0 is assigned to all the other edges, the quantity equals the maximum cycle mean. Thus, the problem of nding the maximum insertion rate is equivalent to that of nding the maximum cycle mean (also known as the maximum cost-to-time ratio) in computer science, see [17, pg. 548]. A fast polynomial-time solution is given by Karps algorithm [39], whose empirical running time is (nm) for an irreducible graph with n vertices and m edges. There are other algorithms that solve the same problem, see [18] for a good summary. Here, we will focus on Karps solution. Karps algorithm works in the following way. Let G = (V, E) be an irreducible directed graph with nonnegative weights assigned to its edges, given by a matrix W . The algorithm begins by rst selecting an arbitrary source vertex s from V . Then for
3.2. NUMERICALLY COMPUTING THE MAXIMUM INSERTION RATE
49
of a path of length k from s to v; if no such path exists, then Dk (v) = . Karps algorithm then employs the following recurrence for Dk (v): Dk (v) = max [Dk1 (u) + W (u, v)] ,
(u,v)E
every v V , and every nonnegative integer k, dene Dk (v) as the maximum weight
k = 1, 2, n
with the initial condition D0 (s) = 0 and D0 (v) = , v = s. These quantities are easily computed and can be stored in a table of size (n + 2) n such that the (k, n)-th entry contains Dk (v) for k = 0, 1, , n and v V .
Karps Algorithm Inputs: G, VG , weight matrix W , adjacency matrix AG n |VG | Dk (v) for all k = 0, 1, . . . , n and all v VG
for k = 1 : n
D0 (s) 0 where s is any vertex in VG for each v VG
for each predecessor vertex u such that A(u, v) = 1 Dk (v) max{Dk (v), Dk1 (u) + W (u, v)}
for each v VG
for k = 1 : n 1
Dn+1 (v)
Output:
max{, Dn+1 (v)}
Dn+1 (v) min{Dn+1 (v), (Dn (v) Dk (v))/(n k)}
The key to computing the maximum cycle mean lies in the following theorem. Theorem 16 (Karps Theorem, [39]) The maximum cycle mean of G is given by = max min
vV
Dn (v) Dk (v) 0kn nk
(3.2)
50
A version of Karps Algorithm from [19] is included above for completeness. Note that if G has multiple non-trivial irreducible components, the maximum insertion rate of each component can be determined separately by Karps algorithm in polynomial time, and the largest of these then yields the maximum insertion rate for the constraint.
3.3
Closed Form Solutions for RLL and MTR Codes
In this section, closed form solutions for the maximum insertion rates of RLL and MTR constraints are presented. The proof technique used is to directly look for a word in S that can be freely concatenated and has the maximum ratio of to its length. This approach is equivalent to nding a max-insertion-rate cycle by the following lemma. Lemma 17 Let S be the constrained system presented by a graph G. Let w be a word for some k. in S such that wm S for all m N. Then there exists a cycle in G with label wk
as a concatenation of n subpaths of the same length: = 1 2 . . . n . Let u0 be the j1 , j2 {0, 1, . . . , n} such that j1 < j2 and uj1 = uj2 . Then j1 +1 . . . j2 is a cycle with label wj2 j1 . initial state of and let ui , 1 i n, be the terminal states of i . There must be
Proof. Let n = |VG |. Let be a path in G with label wn . This path can be written
3.3.1
RLL(d, k) Constraints
Theorem 18 The maximum insertion rate for RLL(d, k) is given by kd d+1 . k + 1 (d + 1) d+1
RLL(d,k) =
(3.3)
3.3. CLOSED FORM SOLUTIONS FOR RLL AND MTR CODES
51
Proof. We will nd a word w in S that can be freely concatenated and has the maximum to length ratio. We rst show that every run of 0 in w has length d. To see this, note that if there is a run of 0 with length greater than d, then this run can be replaced by a run of length d without violating the constraint and without reducing the insertion rate. Thus it suces to assume that w is of the form a1 0d a2 0d . . . a 0d where ai is a string comprised of only 1 and 1 or . If d 1, a string of 1 and of length
greater than one will violate the constraint; thus each ai must be a single symbol of . If d = 0, w can be written as a word of length : a1 . . . a , where ai {1, }. From the expression for w, it is clear that the maximum insertion rate is at most 1/(d + 1). Now, suppose that there are at least two 1s in w. If there are at most rst 1 symbol. The resulting insertion rate does not decrease because the ratio of to length of the inserted word is 1/(d + 1), which exceeds the maximum possible insertion rate. Thus any portion of w between two consecutive 1s is a string of the m = (k d)/(d + 1). Therefore, w can be assumed to have the form (1(0d )m 0d )n , whose insertion rate is m = (m + 1)(d + 1) kd d+1 k d + 1 (d + 1) d+1 kd d+1 . k + 1 (d + 1) d+1 form 1(0d )m 0d 1, where m is the largest integer such that (d + 1)m + d k, i.e., k d 1 symbols between two consecutive 1s, insert the string 0d after the
Some values of the maximum insertion rate for RLL(d, k) are given in Table 3.1. Clearly for k < 2d + 1, the numerator in Eqn. (3.3) is 0, so the tradeo function for RLL(d, k) with k < 2d + 1 is a single point operating at (0, cap(RLL(d, k))). In other
52
d 1 2 3 4 5 6 7 8 9 10 11
k=d 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
d+1 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
d+2 0.2500 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
d+3 0.2500 0.1667 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
d+4 0.3333 0.1667 0.1250 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
d+5 0.3333 0.1667 0.1250 0.1000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
d+6 0.3750 0.2222 0.1250 0.1000 0.0833 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
d+7 0.3750 0.2222 0.1250 0.1000 0.0833 0.0714 0.0000 0.0000 0.0000 0.0000 0.0000
0.5000 0.3333 0.2500 0.2000 0.1667 0.1429 0.1250 0.1111 0.1000 0.0909 0.0833
Table 3.1: Maximum insertion rates for some RLL(d, k) constraints words, the scheme of constrained positions with unconstrained positions is simply inadmissible for RLL(d, k) with k < 2d + 1. can be seen from Table 3.1. It is thus not surprising that earlier papers considered RLL(d, k) constraints as poor choices for applying this scheme of unconstrained poof unconstrained positions does admit higher code rates than previously thought. sitions. However, in later chapters, it will be shown that for k 2d + 1, this scheme For k 2d + 1, the insertion rate for RLL(d, k) is nonzero but generally low, as
3.3.2
MTR Constraints
j j+1
For the MTR(j) constraints, the maximum insertion rate is shown to be
in [11]
using a packing argument. A simpler way to see this is to examine the single non trivial irreducible component of its G presentation shown in Figure 2.4 in Chapter 2: inspect the simple cycles of G starting at state 0 and apply Theorem 14. The following theorem presents the maximum insertion rate of the MTR(j, k) constraint. Theorem 19 Let S be the MTR(j, k) constraint. If gcd(j + 1, k + 1) = 1, S = 1 1 1 . j+1 k+1
53
If gcd(j + 1, k + 1) = 1, let m be the smallest positive integer such that m(j + 1) mod (k + 1). Then
k mod (k + 1), and let n be the smallest positive integer such that n(j + 1) 1 S = where L0 = 1 L1 n 1 , n(j + 1) 1 k + 1 1 m(j + 1) + 1 = 1 . j + 1 m(j + 1)(k + 1) L1 if m > n,
max{L0 , L1 } if m < n,
Proof. Let w be a word that can be freely concatenated. We rst show that w can be assumed to start with 10. Let c be the smallest integer such that 1 by c to obtain another word with the same ratio of starts with 10. Starting with the prex 10, we will show that an optimal strategy to construct a word that has the maximum ratio of is to keep appending until the constraint is almost violated. Given a word w with prex 10, dene a = |w| max i,
wi =0 c
0 is a
subword of ww. Then we can cyclically shift the positions of all 0 in w to the left . This word contains 10 as a subword, and we can cyclically shift the positions of all 0 and 1 so that the word
b = |w| max i.
wi =1
For example, if w = 10
0 , then a = 1 and b = 5. This pair (a, b) keeps track of
the last 0 and 1 in w. We can view (a, b) as the state of w. In the above example, the history of (a, b) is given by (0, 1) (1, 2) (2, 3) (0, 4) (1, 5). If S is the MTR(j, k), then w S only if a j and b k. Moreover, it species
0
what symbol can be appended to w. In other words, FS (w) can be determined from
(a, b). It is easy to check that if (a, b) is the state of a word w, (c, d) is the state of a
54
word w , a c, and b d, then FS (w ) FS (w). with prex 10. Let (a, b) and (c, d) be the states of Let w and w be words in S w and w respectively. We wish to have a way to compare which word is better as a prex. If z is a word in A3 , dene B(z) to be the number of
in z. If both of
the following are true, we prefer prex w over w to construct a word with maximum to its length.
ratio of
(a) B(w) B(w ) |w| |w | 0, (b) (a, b) (c, d) componentwise. To see this, rst note that if w z is a word that can be freely concatenated, then so B(w z) |w z| and |w| |w |, is wz; this is because FS (w ) FS (w) by (b), and w and w have prex 10. Since B(w z) B(w z) + |w| |w | |w z| |w z| + |w| |w | B(w z) + B(w) B(w ) |wz| B(wz) , |wz| where the second inequality follows from (a). Hence wz has a better ratio of w z. Let w S with state (a, b). Suppose that a = j, Then we can append b = k, (a, b) = (j 1, k 1). (3.4)
than
will show that it is optimal to append
to w and still be able to continue, i.e., FS (w ) = {}. We
when (3.4) holds. Fig. 3.1 shows some
choices, denoted by AR, of appending symbols to a word w with state (a, b). From the discussion above, it is clear that G is better than A. Table 3.2 summarizes all choices and shows that it is optimal to add whenever (3.4) is true. when (3.4) holds; a With this strategy at hand, we begin with w = 10 and add
0 is added when a = j while a 1 is added when b = k. Symbols are continually added
55
0 (0, b + 1) 1
(0, b + 2) A B (1, 0) (1, b + 2) C (a + 2, 0) D 0 E (0, 1) (a + 2, 1) F 0 (0, b + 2) (a + 2, 0) G H
0 1
(0, 1) I (2, 0) J (2, 1) K (0, 2) L (1, 0) M (1, 2) N
0 1 (a, b) 1 (a + 1, 0)
0 1
1 0
(1, 0) (0, 1)
O P
0 1
(0, 1) Q (1, 0) R
(a + 1, b + 1)
Figure 3.1: Choices of appending symbols to a word with state (a, b).
in this way until the word ends with 10 again. However, there are several cases to consider if (a, b) = (j 1, k 1). Case 1: gcd(j + 1, k + 1) = 1. In this case, we will show that the state (j 1, k 1)
will never occur if we start with state (0, 1). By using the above strategy, there are exactly k symbols between two consecutive 1 and exactly j symbols between two mod (j + 1) and x + 1 = (k 1) mod (k + 1). Equivalently, there are integers y and that gcd(j + 1, k + 1) = 1, so (j 1, k 1) does not occur. When the state (0, 1) consecutive 0. If (j 1, k 1) occurs, there is an integer x such that x = (j 1)
z such that x = y(j + 1) 2 = z(k + 1) 3. But this contradicts the assumption
occurs again, the word length (not including the last 10) is lcm(j + 1, k + 1) and there are (lcm(j + 1, k + 1))/(j + 1) and (lcm(j + 1, k + 1))/(k + 1) numbers of 0 and 1
56
choices A,C B D,F E I,K J L M,N
better alternative G O H P P R Q O
remark if b k 2 if if if if if a j 2 b = k 1 (and so a j 2) b = k 1 (and so a j 2) a = j 1 (and so b k 2) a = j 1 (and so b k 2)
Table 3.2: Adding
is better when (3.4) holds
respectively. Thus the maximum insertion rate is lcm(j + 1, k + 1) lcm(j + 1, k + 1) + 1 1 j+1 k+1 1 =1 . lcm(j + 1, k + 1) j+1 k+1 Case 2: gcd(j + 1, k + 1) = 1. Without loss of generality, we assume that the state sequence s0 = (0, 1) s1 s|w| = (0, 1) does not contain the same state twice except the initial and terminal states (by the same idea as Proposition 12). Since m is the smallest integer such that m(j + 1) = k to continue adding symbols after this state:
1 0
mod (k + 1), sm(j+1)2 = (j 1, k 1). Let p = m(j + 1) 2. There are two choices
(i) sp sp+1 = (j, 0) sp+2 = (0, 1). Now we return to the state (0, 1) again and nish constructing w. We have |w| = p + 2 = m(j + 1); the number of 0 in w rate in this case is L1 = 1
0 1
is m; the number of 1 in w is (m(j + 1) + 1)/(k + 1). Thus the maximum insertion 1 m(j + 1) + 1 . j + 1 m(j + 1)(k + 1)
(ii) sp sp+1 = (0, k) sp+2 = (1, 0). From state (1, 0), we wish to determine whether we will see state (0, 1) or (j 1, k 1) rst. If we reach (0, 1) rst, then (0, 1) = st+p+2 for some t where t + 1 = 0 mod (j + 1) and t = 1 mod (k + 1).
3.4. TRADEOFF FUNCTIONS AT THE MAXIMUM INSERTION RATE
57
Hence t + p + 2 = 0 mod (k + 1). Moreover, there is an integer x such that This implies that x + m is an integer such that (x + m)(j + 1) = 1 mod (k + 1). Since n is the smallest such integer, if m > n then x + m > k + 1. If we reach (j 1, k 1) rst, then (j 1, k 1) = sy+p+2 for some y where mod (k + 1). Moreover, there is an integer z such that y = z(j + 1) 3 and thus y + p + 2 = z(j + 1) 3 + m(j + 1) = (z + m)(j + 1) 3. This implies that If m > n, then x > z and hence t > y, which means that we will reach state twice. Hence we must choose (i) when m > n. If m < n, then x + m = n < k + 1 and hence t < y. We will reach (0, 1) rst and w is completed. In this case, |w| = t+p+2 = n(j +1)1; the number of 0 is n; the number of 1 is (n(j +1)1)/(k+1). Thus the maximum insertion rate is L0 = 1 n 1 . n(j + 1) 1 k + 1 (j 1, k1) before (0, 1). This contradicts the assumption that no state is repeated t = x(j + 1) 1 and thus t + p + 2 = x(j + 1) 1 + m(j + 1) = (x + m)(j + 1) 1.
y + 1 = j 1 mod (j + 1) and y = k 1 mod (k + 1). Hence y + p + 2 = k 2
(z + m)(j + 1) = 0 mod (k + 1). It follows that z + m = k + 1.
The maximum insertion rates for some MTR(j, k) constraints are tabulated in Table 3.3 for easy reference and can be seen to be typically large (> 0.5 for j, k 3).
3.4
Tradeo Functions at the Maximum Insertion Rate
In this section, tradeo functions at the maximum insertion rate are considered. The main result of this section is the relationship f () = f (p) () = f (o) ().
58
j 1 2 3 4 5 6 7 8 9 10 11
k=j 0.0000 0.3333 0.5000 0.6000 0.6667 0.7143 0.7500 0.7778 0.8000 0.8182 0.8333
j+1 0.0000 0.3750 0.5333 0.6250 0.6857 0.7292 0.7619 0.7875 0.8081 0.8250 0.8392
j+2 0.2500 0.4444 0.5833 0.6500 0.7083 0.7429 0.7750 0.7963 0.8167 0.8312 0.8452
j+3 0.2500 0.5000 0.6000 0.6667 0.7222 0.7551 0.7818 0.8056 0.8222 0.8367 0.8500
j+4 0.3333 0.5000 0.6250 0.6857 0.7333 0.7636 0.7917 0.8111 0.8286 0.8417 0.8542
j+5 0.3333 0.5333 0.6296 0.7000 0.7407 0.7714 0.7969 0.8163 0.8333 0.8462 0.8571
j+6 0.3750 0.5556 0.6500 0.7045 0.7500 0.7792 0.8036 0.8222 0.8375 0.8497 0.8611
j+7 0.3750 0.5556 0.6563 0.7143 0.7538 0.7857 0.8077 0.8254 0.8403 0.8531 0.8636
j+8 0.4000 0.5714 0.6667 0.7200 0.7619 0.7889 0.8125 0.8296 0.8444 0.8561 0.8667
0.5000 0.6667 0.7500 0.8000 0.8333 0.8571 0.8750 0.8889 0.9000 0.9091 0.9167
Table 3.3: Maximum insertion rates for some MTR(j, k) constraints This result will be used in the next chapter to prove important properties of the tradeo function. To consider the tradeo function in a more general setting, dene the tradeo function for any labeled graph G with alphabet A3 by simply letting M (q, I) in (2.1) The tradeo function for G is denoted by fG . Similarly, we dene G = supfG ()>0 . Then the tradeo function f is equal to fG . Similar extensions can be applied to f (p) and f (o) . to be the number of words w S(G) such that |w| = q and wi = if and only if i I.
3.4.1
Maximal Subgraph Induced by Max-Insertion Rate Cycles
Throughout this section, let T denote the subgraph of G consisting of all edges that belong to a max-insertion-rate cycle. Lemma 20 A cycle in G is contained in T if and only if it is a max-insertion-rate cycle. Proof. Let be a max-insertion-rate cycle. Then is contained in T by denition.
59
For the other direction, suppose that is a cycle of length n in T . For each edge i , there is a path i such that i i is a max-insertion-rate cycle. Note that
1 . . . n n . . . 1 is a cycle in T , and so is n . . . 1 . Then its insertion rate is n (1 . . . n n . . . 1 ) n
=
i=1
i (i i )
n i=1 i
=
i=1
i = ,
However, = (1 . . . n n . . . 1 ) = ()+(1)(n . . . 1 ), for 0 < < 1. Note that if either () < or (n . . . 1 ) < , then this would imply (1 . . . n n . . . 1 ) < , which is a contradiction. So () = (n . . . 1 ) = .
where i = |i i |/|1 . . . n n . . . 1 | and
= 1.
The next lemma proves that if there is a path in G whose insertion rate is within of the maximum insertion rate, then most of the edges in must be in T . Lemma 21 For any > 0, there exists > 0 such that if is a path in G with |() | < and || > 1/, then Number of edges in that are not in T < . || Proof. By the same path decomposition argument as in the proof of Proposition 12, a collection of simple cycles 1 , . . . , k and a simple path can be found such that
k
|| =
i=1
|i | + ||, |i | || + () . || ||
|| |VG |,
k
() =
i=1
(i )
(3.5)
(i.e., these cycles j are contained in T ). Let = maxj J (j ). / Fix > 0. Pick = ( ) . 2|VG |
Let J {1, . . . , k} be the set of indices such that j J if and only if (j ) =
60
From (3.5), () =
iJ
|i | + ||
iJ /
(i )
|| |i | + () || ||
Let =
iJ /
iJ
i + ||
iJ /
|V | i + G . || ||
in not contained in T . By the assumptions on in the statement of the lemma, < () < (1 ) + + So, which is equivalent to < . This completes the proof. ( ) . 2
|i | + || /||. Note that is an upper bound on the fraction of edges
( ) ( ) < ( ) + , 2 2
For each pair of states (u, v) of a graph G, dene MG (q, Iq , u, v) to be the number of words of length q satisfying Iq and generated by a path in G starting at u and ending at v. The period of a graph G is greatest common divisor of the lengths of all cycles in G. Lemma 22 Let T1 , . . . , Tn be the irreducible components of T . Let ti be the period of Ti . Let t be the least common multiple of {t1 , . . . , tn }. For any > 0, there exists an integer L with the following property. Suppose that > L and is a multiple of t.
Then there are a state u VT and a set I {0, . . . , 1} such that |I |/ = and log MT (, I ) log MT (, I , u, u) <+ . I {0,...,1}
max
Proof. It is clear that T has no transitional edges, i.e., T is the union of its irreducible components. Since ti is the period of the component Ti , by [43, Proposition 4.5.6] VTi can be partitioned into ti period classes: D(i, 1), . . . , D(i, ti ). (Two states
61
are in the same period class if they are connected by a path whose length is a multiple of the period of the graph. This is easily seen to be an equivalence relation.) For any integer that is a multiple of t, any path of length has initial and terminal states in the same period class. Thus, for any I {0, . . . , 1},
n ti
MT (, I ) =
i=1 j=1 (i,j)
MT
(i,j)
(, I ),
(3.6)
where MT
(, I ) is the number of words of length , satisfying I and generated by

(i,j)
a path in T beginning and ending in period class D(i, j). Let (i , j ) maximize MT (, I ). Restricting our attention to words starting and ending in the period class D(i , j ), it follows from (3.6) that MT (, I ) sMT where s =
n i=1 ti . (i ,j )
(, I ),
(3.7)
Let A be the adjacency matrix of T , with the rows and columns grouped according to period classes. From [43, pp.127128], At is block diagonal. Each block is primitive and corresponds to one of the period classes. It follows that for each period class D(i, j), there exists an integer k(i, j), a multiple of t, such that for any states u and multiples of t. Let k = max1in max1jti k(i, j). v in D(i, j), there is a path from u to v of length k , for all k k(i, j) that are Suppose that > k. Let q = k. Dene Iq = I {0, . . . , q 1}. Then, MT
(i ,j )
(, I ) 2k MT
(i ,j )
(q, Iq ).
(3.8)
Next, observe that there are states u and v in D(i , j ) such that MT (q, Iq ) MT (q, Iq , u, v). , j )|2 |D(i
(i ,j )
(3.9)
|I |/ = . Then MT (q, Iq , u, v) MT (, I , u, u) trivially holds. On the other hand,
Suppose that MT (q, Iq , u, v) = 0. We pick any I {0, 1, . . . , 1} such that
62
suppose that MT (q, Iq , u, v) 1. Let z be the label of a path from u to v of length q such that z satises Iq . Let z be the label of a path from v to u of length k. Let
w = zz . Let I = {i {0, 1, . . . , 1} : wi = }. By Lemma 20, |I |/ = since
w is the label of a cycle in T . Furthermore, the 1-1 correspondence z zz gives

MT (q, Iq , u, v) MT (, I , u, u).
(3.10)
From (3.7)(3.10),
MT (, I ) s2k |D(i , j )|2 MT (, I , u, u) s2k |VT |2 MT (, I , u, u).
Choosing L such that L > k and log s2k |VT |2 < , L the lemma follows.
3.4.2
f = f (p) = f (o) at the Maximum Insertion Rate
Using the above lemmas, we show in the next proposition that all three denitions of the tradeo function agree with one another at the maximum insertion rate on the subgraph T . We then use the information about the subgraph T to study the overall tradeo function at . Proposition 23 fT () = fT () = fT (). Proof. It suces to show that fT () fT () and fT () fT () since the other
(p) (o) (p) (o)
direction is immediate.
Lemma 22. Let q be a multiple of t such that q > L and fT () < + log MT (q, Iq ) . q
Let (Im ) I() achieve fT (). Fix > 0. Let t and L be as described in
63
From Lemma 22, there exist a state u VT and a set Iq {0, 1, . . . , q 1} such that |Iq |/q = and log MT (q, Iq , u, u) . q mod q) Iq }. It is clear that
fT () < + +
(Jl ) Ip (). Now, for any n N,
For each l N, let Jl = {i {0, . . . , l 1} : (i
log MT (q, Iq , u, u) fT () 2 + nq log MT (nq, Jnq ) 2 + nq (p)
Letting n yields fT () 2 + fT () and fT () 2 + fT (). Since is arbitrary, it follows that fT () fT () and fT () fT ().
(p) (o)
(o)
The following proposition solidies the intuition that the tradeo function of G
equals that of T at since any contribution to the code rate must come from the max-insertion-rate cycles. Proposition 24 fG () = fT (). Proof. It suces to show fT () fG (), since the other direction is immediate.
I(n) {0, . . . , (n) 1} such that |I(n) |/(n) = , (n + 1) > (n), and log MT ((n), I(n) ) log MT ((n), I) <+ . max I{1,...,(n)} (n) (n)
Let > 0. By Lemma 22, for each n N, there exist an integer (n) and a set
(3.11)
Let (Im ) I() achieve fG (). Let q(n) be a multiple of (n) such that log MG (q(n), Iq(n) ) > fG () , q(n) |Iq(n) | < n , q(n) 1 . q(n) > n
For each n N, replace in Lemma 21 with /(n) and choose an appropriate n .
(3.12) (3.13)
64
Let n be a set of paths in G of length q(n) with labels that satisfy Iq(n) such in n has at most q(n)/(n) edges that are not in T . Subdivide any path in q(n)/(n) non-overlapping subpaths that are not entirely in T . that |n | = MG (q(n), Iq(n) ). By Lemma 21, (3.12) and (3.13) imply that any path n into q(n)/(n) non-overlapping subpaths of length (n). Then there are at most Let s(n) = q(n)/(n). We can partition n into s(n) + 1 classes: C0 , . . . , Cs(n) ,
q(n)/(n) j
where Cj is the set of paths with exactly j subpaths not entirely contained in T . We can further partition each class Cj into each subclass of Cj is at most 2(n) Since
m k=0 m k j 2(n) MT ((n), I(n) )
q(n) j (n)
subclasses corresponding to the
positions of subpaths that are not entirely in T . From (3.11), the number of paths in
= 2m , q(n)/(n) j 2 (n) .
q(n)
Thus,
s(n)
MG (q(n), Iq(n) )
j=0
2 (n) 2(n)
q(n)
q(n)
2(n) MT ((n), I(n) ) s(n)
q(n) j (n)
(s(n) + 1) 2 (n) 2(n)

q(n)
2(n) MT ((n), I(n) )
q(n) (n)
(q(n) + 1) 2 (n) 2q(n) 2q(n) MT ((n), I(n) ) (n) log MT ((n), I(n) ) log MG (q(n), Iq(n) ) log(q(n) + 1) 1 + + 2 + q(n) q(n) (n) (n) log MT ((n), I(n) ) log MG (q(n), Iq(n) ) lim sup 0 + 0 + 2 + lim sup q(n) (n) n n fG () 2 + fT ().
q(n)
Since is arbitrary, it follows that fG () fT (). Combining the results of Propositions 23 and 24, the following corollary is obtained. Corollary 25 fG () = fG () = fG ().
(p) (o)
65
denitions of f (p) and f (o) , fG () fG () and fG () fG ().
(o) (p) (o) (p) Proof. Since T is a subgraph of G, fG () fT () and fG () fT (). By the (p) (o)
This shows that periodic insertions are crucial to achieving the tradeo function at , and that an admissible periodic conguration of unconstrained positions can be obtained directly from reading the label of a max-insertion rate cycle in G. It is natural to examine periodic insertions rather than arbitrary non-periodic sequences (Iq ), but it is not obvious that it suces to consider only periodic insertions. In the next chapter, we will see that periodic insertions constitute a practical way of achieving the tradeo function at insertion rates less than for all nite-type constraints.
Chapter 4 Properties of the Tradeo Function

This chapter discusses properties of the tradeo function which we dened at the end of Chapter 2. Section 4.1 outlines basic properties, including monotonicity and left-continuity. Section 4.2 details properties of the tradeo function for nite-type constraints, which includes the MTR(j), MTR(j, k) and RLL(d, k) systems. These are constraints that induce a G presentation with a unique non-trivial irreducible component which we call G . From studying the paths and cycles in just one component, we prove concavity and continuity, and show additionally that the tradeo function on the component can be achieved by periodic congurations. This analysis is then extended to two components in Section 4.3, which has direct implications for the tradeo function of an arbitrary constraint with multiple components in its G presentations. In Section 4.4, the connection between our denition of the tradeo function and that of [11] is explored. Finally, Section 4.5 discusses the relationship between NRZI and NRZ tradeo functions for specic instances.
4.1
Monotonicity and Left-Continuity of f
As before, let S denote a constrained system. Let f = fS be its tradeo function and let = S be the maximum insertion rate for S. The next two propositions establish that f is decreasing on [0, 1] and left-continuous on [0, ].
66
4.1. MONOTONICITY AND LEFT-CONTINUITY OF F
67
1,
Proposition 26 The tradeo function f is decreasing on [0, 1]. In fact, for 0 < f ( ) f () . (4.1)
min{( )q, |Iq |} entries of Iq . Then |Jq | lim = min q q
Proof. Let 0 < 1 and (Iq ) I(). Let Jq be the set of the smallest ( )q |Iq | , lim lim q q q q
= min{ , } = .
(4.2)
in Jq of every word of length q that satises Iq with 0 and 1, we obtain

M (q, Iq ) 2|Jq | M (q, Iq ) log M (q, Iq ) |Jq | log M (q, Iq ) lim sup lim + lim sup q q q q q q log M (q, Iq ) . = + lim sup q q
Let Iq = Iq \ Jq . It follows from (4.2) that (Iq ) I( ). By lling in
at positions
It follows that f ( ) + f (). Thus, f is decreasing on [0, 1]. The following proposition gives a continuity property of the tradeo function. A stronger result is proven in the next section.
Proposition 27 The tradeo function f is left-continuous on [0, ].
as n .
Proof. Fix [0, ] and let n with n < . We will show that f (n ) f ()
68
CHAPTER 4. PROPERTIES OF THE TRADEOFF FUNCTION
Since f (n ) = sup(Iq )I(n ) lim supq
log M (q,Iq ) , q
n qn N and a set Iqn {0, 1, , qn 1} such that
for each n N, we can choose
n log M (qn , Iqn ) 1 f (n ) , qn n n |Iqn | 1 , n qn n qn+1 > qn .
Dene a sequence (Iq ) by
0 We now use a diagonalization argument. Let (Iq ) be an arbitrary sequence in I().
0 We have |Iq |/q as q and by the triangle inequality, n |Iqn | qn
I n , if q = qn for some n; qn Iq = I 0 , otherwise.

q
n |Iqn | n + |n | qn 1 + |n | 0, n
so (Iq ) I(). For the code rate, we have f () lim sup

q n log M (qn , Iqn ) log M (q, Iq ) lim sup lim sup f (n ). q qn n n
By Proposition 26, f is decreasing, so f (n ) f () for all n. It follows that lim sup f (n ) lim inf f (n ) f ().
n n
Thus, f (n ) f ().
4.2. FINITE-TYPE CONSTRAINTS
69
4.2
Finite-Type Constraints
Recall from Chapter 1 that any nite-type constrained system S has a presentation with nite memory. Examples include the MTR and RLL systems that have been used as recurring examples throughout this work. From Proposition 5, the Shannon cover for a constrained system S is isomorphic to a subgraph H of G. Recall also that G is the irreducible component of G that contains H. The following corollary shows that if S is nite-type, it suces to use only G to compute the tradeo function. This result is immediate from Proposition 6. Corollary 28 Let S be a nite-type constrained system. Then f () = fG () for all [0, 1]. The following example shows that Corollary 28 may not hold for a constraint that is not nite-type. Example 29 Let S be the constrained system presented by the graph H in Fig. 4.1. It is easy to check that H is the Shannon cover for S. The subgraph of G that can be reached from H is given in two separate components for ease of viewing. Since H has a cycle at A, C with label 00, S is not nite-type. From the gure, if we start from a state in H and follow an edge with label , we cannot return to H. Thus the irreducible component of G containing H is H itself. Since H does not have any edge with label the component with states A B and C D. This component can generate a word , fH () = for all > 0. Consider
not hold for a non-nite-type constraint.
0x0 , where x can be 0 or 1. This word can be freely concatenated and still satises S. Therefore f (1/4) 1/4 > = fH (1/4). This implies that Corollary 28 may
4.2.1
Concavity and Continuity
Next, an important feature of the tradeo function for nite-type constraints is presented, which describes the shape of the tradeo curve. But rst, we require the following lemma about periodic insertions on an irreducible graph G over A3 , namely,
70
0 BD 1 0 A 0 E 1 B H 1 0 0 D A 1 0 C C AE 1 0 0 AC 1 C D B 0 0 1 AB E
Figure 4.1: Subgraphs of G that can be reached from H for Example 29. we show that for any insertion rate , there is a periodic insertion rate with insertion rate arbitrarily close to and code rate arbitrarily close to f (). Lemma 30 Let G be an irreducible graph over A3 . Let < and let > 0. There
(p)
is a rational number with | | < and fG () > fG () .
Proof. For each pair of states (u, v) in VG , let MG (q, Iq , u, v) denote the number of words of length q satisfying Iq and generated by a path in G beginning at u and ending at v. 1} such that
Fix > 0. By the denition of fG (), there exist a q0 N and Iq0 {0, 1, . . . , q0
log MG (q0 , Iq0 ) fG () , q0 3 |Iq0 | , q0 2
71
and q0 1 , q0 + |VG | 3 2 log |VG | . q0 3

Let ( , ) be a pair of states such that MG (q0 , Iq0 , , )
MG (q0 ,Iq0 ) . 2 |VG |
Since
a path exists since G is irreducible. Let c = |x| and let k be the number of
satisfying Iq0 . Let x be the label of a path from to such that |x| |VG |. Such
, MG (q0 , Iq0 ) 1. Thus there is a path from to with some label
in x.
(x) is . Then let (Iq ) be the periodic sequence given by Iq = I{0, 1, . . . , q1}. q=1
Now dene I N by putting i I i the i-th position of the right-innite sequence
Let (w1 , . . . , wn ) be an n-tuple, where wi is the label of a path beginning at and

ending at and wi satises Iq0 . We map (w1 , . . . , wn ) to the sequence w1 xw2 x wn x
which is generated by the cycle

w x
1 2
n .
are at least MG (q0 , Iq0 )/|VG |2
The mapping (w1 , . . . , wn ) w1 xw2 x . . . wn x is one-to-one. Thus for any n, there

n
codewords of length n(q0 + c) satisfying In(q0 +c) .
The new insertion rate induced by (Iq ) is thus

|Iq0 | |Iq0 | + k k q0 = + q0 + c q0 q0 + c q0 + c + ,
since k/(q0 + c) c/(q0 + c) |VG |/(q0 + |VG |) /3. Also,

|Iq0 | + k q0 + c
2 2 .
k + 3 q0 + c 1 2 1
72
Thus, | | . The code rate is bounded as follows: fG () lim sup

n (p)
log MG (n(q0 + c), In(q0 +c) ) n(q0 + c) log

MG (q0 ,Iq0 ) 2 |VG |
lim sup
n
n(q0 + c) log MG (q0 , Iq0 ) 2 log |VG | q0 = q0 q0 + c q0 + c 1 fG () 3 3 3 fG () .
Using this lemma, we will show that fG is concave in the next proposition. The key idea of the proof is that one can stitch together two periodic congurations to obtain a target intermediate insertion rate and code rate.
continuous over [0, G ].
Proposition 31 Let G be an irreducible graph over A3 . Then fG is concave and
Proposition 27, it suces to take to be rational, say with = a/b in lowest terms n 1 + (1 n )2 converges to 1 + (1 )2 from the left).
2 and (0, 1), in order to show f (1 + (1 )2 ) f (1 ) + (1 )f (2 ). By
Proof. First, we will show concavity of fG . Fix insertion rates 1 , 2 with 0 1 <
(since then the result holds for irrational by taking a sequence n of rationals where For each n N, recursively construct a length qn and index set Iqn as follows.
(1) (2)
By Lemma 30 and Proposition 11 (and Corollary 25 if 2 = ), there exist rational insertion rates 1,n , 2,n and periodic (Iq ) I(1,n ), (Iq ) I(2,n ) with |1,n 1 | 1 , n 1 , |2,n 2 | n
73
and log MG (q, Iq ) 1 f (1 ) , q n 1 log MG (q, Iq ) f (2 ) q n for each q which is a multiple of the periods of (Iq ) and (Iq ). Let Ni be the period of (Iq ) for i = 1, 2 and N = lcm(N1 , N2 ). We combine (Iq ) and (Iq ) into a new conguration as follows. Let i , i be states maximizing MG (q, Iq , i , i ) for i = 1, 2 and x a shortestlength path from 1 to 2 . Let l be the length of and k be the number of boxes in the label of . (Note that k and l are bounded by the diameter of the graph.) Let q0 = 1 and, having constructed q0 , . . . , qn1 , recursively dene qn = cbN + l where c>
ln bN (i) (i) (1) (2) (1) (2) (2) (1)
is an integer chosen suciently large so that qn > qn1 . Note that the (Iq )
(i)
are chosen separately for each n, and the quantities c, k, l, N also depend on n; we suppress the n from the notation to reduce clutter. Let J() {0, 1, . . . , l1} be the index set indicating the positions of the
(1)
symbol
(2)
in the label of . Now dene the index set Iqn {0, 1, . . . , qn 1} by concatenating the congurations (Iq ) for length caN , followed by J() for length l, and then (Iq ) for length c(b a)N . More precisely, for i {0, 1, . . . , qn 1}, let i I (1) , caN i i caN J(), i caN l I (2) c(ba)N , for 1 i caN ; for caN + 1 i caN + l; for caN + l + 1 i cbN + l. For q {q1 , q2 , . . . }, dene Iq /
i Iqn
This completes the denition of (qn , Iqn ). q {q1 , q2 , . . . }. Note that /
{0, 1, . . . , q 1} arbitrarily such that |Iq |/q 1 + (1 )2 as q with cbN 1 1 n+1 cbN + l
74
since 0 l/(cbN ) < 1/n. To see that the insertion rate of (Iq ) is as desired, i.e., (Iq ) I(1 + (1 )2 ), observe that |Iqn | caN 1,n + c(b a)N 2,n + k = qn cbN + l = k cbN caN 1,n + c(b a)N 2,n + cbN + l cbN cbN + l = cbN k (1,n + (1 )2,n ) + cbN + l qn 1 + (1 )2 as n . For the code rate, we have log(MG (caN, IcaN )MG (c(b a)N, Ic(ba)N )/|VG |4 ) log MG (qn , Iqn ) qn qn
(1) log(MG (c(b a)N, Ic(ba)N )) c(b a)N 4 log |VG | log(MG (caN, IcaN )) caN + = caN qn c(b a)N qn qn (2) (1) (2)
(f (1 )
1 caN cbN 1 c(b a)N cbN 4 log |VG | ) + (fG (2 ) ) n cbN cbN + l n cbN cbN + l qn 1 1 1 1 4 log |VG | )(1 ) + (f (2 ) )(1 )(1 ) . n n+1 n n+1 qn
(fG (1 ) Thus,
f (1 + (1 )2 ) lim sup
n
log MG (qn , Iqn ) fG (1 ) + (1 )fG (2 ). qn
Thus fG is concave. Concavity of fG implies continuity of fG on (0, ). Since fG is nonincreasing and concave, it is continuous at 0. By Proposition 27, f is left continuous at . Hence, fG is concave and continuous on [0, ]. By letting G = G in the above proposition for the case of nite-type constraints, we obtain the following theorem.
75
Theorem 32 If S is a nite-type constrained system, then f () is concave and continuous on the domain [0, ]. It turns out that concavity is very useful for deriving improved lower bounds for constraints whose tradeo functions cannot be described exactly. This is discussed in detail in Chapter 6. Note that even though fS for a nite-type constraint S is concave and continuous, fS is generally not dierentiable. For example, the MTR(2) tradeo function depicted in Fig. 5.1 was proved in [11] to be piecewise linear with a single kink, and is hence not dierentiable.
4.2.2
Periodic Congurations Achieve Optimal Code Rates
In all of the previous work on constrained systems with unconstrained positions, namely [22, 66, 11], the constraints of interest were the RLL(d, k) and MTR(j) systems; the code design was primarily based on periodic congurations. In particular, the work of [11] showed that for the special case of MTR(2), time-sharing between two periodic congurations with dierent (insertion rate, code rate) pairs was the key in achieving the tradeo function. This motivated the denition of the periodic tradeo function in Section 2.5.2. Using our new approach outlined in Chapter 2 for analyzing constrained systems with unconstrained positions, especially Proposition 6 and Corollary 28, it is natural to raise the question of whether periodic congurations can achieve the tradeo function at all rational insertion rates. To address this, we rst show that the periodic tradeo function is nonincreasing for all constraints. Proposition 33 The periodic tradeo function f (p) () is nonincreasing on rationals 0 . N and periodic set U {0, 1, . . . , N 1}. Choosing a common denominator, write Proof. Let 1 < 2 be rational numbers in [0, ]. Pick (Iq ) Ip (2 ) with period
1 = a/c, 2 = b/c. Without loss of generality, we can assume N is divisible by c
(since any multiple of N is also a period for (Iq )). Then let U = U \ J where J is any
76
for the conguration given by (Iq ). Let (Iq ) be the periodic sequence determined by N and U . Then (Iq ) Ip (1 ) and M (Iq , q) M (Iq , q). So f (p) (1 ) f (p) (2 ),
xed subset of U of size b a. This corresponds to lling in b a boxes in each period
proving that f (p) is nonincreasing.
In the following theorem, we show that for nite-type constraints, periodic inserperiodic insertions in certain (nite-type) constrained block codes [22], [66], and [11]. tions can achieve f () for rational 0 . This further justies the usage of
Theorem 34 Let S be an irreducible nite-type constrained system. Then f (p) () = f () on rationals 0 . Proof. By Corollary 25, f (p) () = f (), so it suces to prove the result for rational 0 < . Fix a rational < . Fix 0 > 0. We will show that f (p) () f () 0 . By
0 /2 whenever | | < . Pick to be a rational number in (, + min{, }). Applying Lemma 30 with = min{ , 0 /2}, and in place of , we obtain f (p) () > f ( ) 0 > f () 0 , 2
Theorem 33, f () is continuous at , so there exists > 0 such that |f () f ( )| <
where . Since f (p) is nonincreasing by Proposition 33, we have f (p) () f (p) () > f () 0 . This holds for all 0 > 0, so f (p) () f (). It follows from the above f (p) can be extended uniquely to a continuous function on the entire interval [0, ].
Corollary 35 The tradeo function f for an irreducible nite-type constrained system is the uniformly continuous extension of f (p) to the domain [0, ].
4.3. TIMESHARING IMPLICATIONS FOR G WITH MULTIPLE COMPONENTS77
4.3
Timesharing Implications for G with Multiple Components
An arbitrary constraint may give rise to a G presentation with multiple non-trivial components. In such cases, the topological structure of G plays a major role in determining the tradeo function. Suppose that G has n components G1 , . . . , Gn . Then by Theorem 33, the tradeo function fi = fGi for component Gi is concave and continuous on the domain [0, Gi ]. Let f be the pointwise maximum of f1 , . . . , fn , i.e., f () = max{f1 , . . . , fn }(). region. An example illustrating this idea is given in Fig. 4.2. Note that f is not Clearly, f fG , so the area under the graph of f constitutes an achievable rate
f1 f (f1 , f2 , f3 )
f2 f3
Figure 4.2: An example of f dened over f1 , f2 and f3 . necessarily concave. If G contains only isolated components, i.e., components that cannot be reached from another, then f = f . However, the components of G are
G
typically connected by transient paths, so we expect fG to exceed f . A useful lemma that will assist in clarifying this is presented next. Consider a more general setting where G is a labeled graph with alphabet A3 , and let G1 and G2 be irreducible components of G such that G2 can be reached from G1 . For any insertion
rates 1 [0, 1 ] and 2 [0, 2 ], fG (1 ) fG1 (1 ) and fG (2 ) fG2 (2 ). Now
78
suppose 1 < < 2 . Concatenate long sequences in G1 with insertion rate 1 and long sequences in G2 with insertion rate 2 in such a way that the resulting sequences will have insertion rate . This is a time-shared scheme between the sequences obtained from G1 and G2 . Dene g(f1 , f2 ) to be the least concave majorant of f1 and f2 , i.e., the smallest concave function which upper bounds each fi , i = 1, 2. Then it turns out that the tradeo function fG is at least g(f1 , f2 ). Lemma 36 Let G be a labeled graph with alphabet A3 . Let G1 and G2 be irreducible components of G such that G2 can be reached from G1 . Then g(f1 , f2 )() fG (). Proof. Just as we write fi to denote fGi , we will apply the same subscript abbreviation to Mi (q, Iq ), i = 1, 2. [0, 1], 1 [0, 1 ], 2 [0, 2 ]. This is obvious if = 0 or = 1 (by considering paths we can assume , 1 , and 2 are rational (otherwise, let n , 1,n , 2,n be sequences of of both sides in f (n 1,n + (1 n )2,n ) n f1 (1,n ) + (1 n )f2 (2,n )).
(i)
It suces to show that f (1 + (1 )2 ) f1 (1 ) + (1 )f2 (2 ) for all
within only one component), so assume (0, 1). By left continuity of f, f1 , f2 ,
rational numbers approaching , 1 , 2 from the left respectively, and take the limit So x rational , 1 , 2 as above, and let = 1 + (1 )2 . By the Theorem 34
achieves fi (i ), and moreover, the limit exists:

(i)
applied to the component Gi (for i {1, 2}), there is a periodic (Iq ) I(i ) which
log Mi (q, Iq ) lim = fi (i ). q q Write = a/b. For each n N, consider codewords generated by paths of the
(1) (2)
form 1 2 , where 1 is a path in G1 of length na satisfying Ina , 2 is a path in G2 of length n(b a) satisfying In(ba) , and is a transition path from G1 to G2 .
Fix the terminal state of 1 to be u (chosen to maximize the number of possible
1 ) and the initial state of 2 to be v (chosen to maximize the number of possible 2 ). Let be a xed path from u to v, say with d boxes and of length c. Let J be
4.3. TIMESHARING IMPLICATIONS FOR G WITH MULTIPLE COMPONENTS79
the set of box positions for . Then take Iq = Ina (J + na) (In(ba) + na + c) for q = nb + c and Iq = Iq
(0)
(1)
(2)
a state in G2 . We then have log
Note that (Iq ) I(). Let c0 be the maximum distance between a state in G1 and
M1 (na,Ina )M2 (n(ba),In(ba) ) |V (G1 )||V (G2 )|
(1) (2)
otherwise, where (Iq ) is any xed sequence in I().
(0)
f () lim sup
n
nb + c0
ba a f2 (2 ), = f1 (1 ) + b b
which completes the proof. Thus, for G with only two components G1 and G2 such that there is a transition from one into the other, and there exists some insertion rates 1 and 2 where f1 (1 ) > f2 (1 ) and f1 (2 ) < f2 (2 ), then f < g(f1 , f2 ) on some interval, as illustrated below in Fig. 4.3.
g(f1 , f2 ) f1
f2 Figure 4.3: An example of g(f1 , f2 ). In fact, Lemma 36 can be easily extended using the same proof idea to a chain of components G1 , . . . , Gk such that Gi+1 is reachable from Gi . Dene g(f1 , . . . , fn ) to be the least concave majorant of f1 , . . . , fn , i.e., the smallest concave function which upper bounds each fi . Lemma 37 Let G be a labeled graph with alphabet A3 . Let G1 , . . . , Gk be irreducible
80
components of G such that Gi+1 is reachable from Gi . Then g(f1 , . . . , fk )() fG (). Note that g(f1 , . . . , fk ) = g(f ), (4.3)
where f is the pointwise maximum of f1 , . . . , fk . Irreducible constraints may give rise to G presentations with multiple components. In joint work with P. Chaichanavong, the following theorem and corollary on the tradeo function of an irreducible constraint are established. Theorem 38 (Theorem 40, [56]) Let S be an irreducible constrained system. Let G be the irreducible component of G that contains H. Let G1 , . . . , Gk = G be the irreducible components of G that can be reached from H. Let G = Gk , . . . , Gm be the irreducible components of G that can reach H. Then f () = g(fG1 , . . . , fGk )() = g(fGk , . . . , fGm )(). Corollary 39 (Corollary 41, [56]) The tradeo function f for an irreducible constraint S is concave. Moreover, f is continuous on [0, ], where is the maximum insertion rate for S. Theorem 38 identies a subgraph G of G from which the optimal achievable rate region of an irreducible constraint (under this scheme) can be computed. This allows any algorithm requiring a construction of G to be made more ecient in time and space complexity. Despite the above characterization, we remark that that we still do not know of a general analytic or numerical method for computing the exact tradeo function from G. The missing piece of the puzzle is a method for computing the exact tradeo function for a single component of G. If such an algorithm can be found, then Eqn. 4.3 points to a more practical way of deriving the tradeo function for the constraint: just apply the algorithm to each component Gi of G directly to derive fi , take the pointwise maximum of the fi , and take the least concave majorant.
4.4. RELATING F TO F (O)
81
Nevertheless, in the next chapter, we will discuss the cases for which the tradeo function have been explicitly described, including the RLL(d, 2d + 1) constraint. For other constraints, we have a variety of techniques for computing upper and lower bounds. These are discussed in Chapter 6. More open problems will be discussed in the concluding chapter.
4.4
Relating f to f (o)
In the previous two sections, we explored properties of the periodic tradeo function f (p) . Interestingly, the proof of the Timesharing Lemma (Lemma 36) demonstrates precisely why f (p) is suboptimal for arbitrary constraints timesharing using periodic congurations between components generally does not result in periodic congurations due to the transition paths from one component to another. Thus, f (p) f in general. This section continues that investigation into the connections between the dierent tradeo denitions by examining properties of the function f (o) . First, we show that f (o) is nonincreasing on rational . The proof is similar to that of Proposition 33 for f (p) . Proposition 40 The tradeo function f (o) () is nonincreasing on rationals 0 Proof. Let 1 < 2 be rational numbers in [0, ]. In lowest terms, write 1 = a/b and 2 = c/d, where a, b are co-prime, and c, d are co-prime. Let qn be an increasing sequence, Iqn {1, ..., qn } with |Iqn |/qn = 2 . We can rewrite 1 as rn /sn where sn = bqn /b, so that qn sn qn + b. Now, rn = a for all suciently large n since ad < cb by 1 < 2 .
|Iqn |d cb Let Iqn
is a subset of Iqn of size |Iqn | rn . This corresponds to lling in |Iqn | rn boxes
= Iqn \ Jn , where Jn
ad |I | cb qn
+ a |Iqn |
for the conguration given by (Iqn ). Then extend the sequence to length sn by a
path of length sn qn with no boxes. This forms a new sequence (Isn ) such that M (Isn , sn ) M (Iqn , qn ). Since sn and qn dier by at most b, f (o) (1 ) f (o) (2 ),
showing that f (o) is nonincreasing.
82
The next theorem reconciles our denition of the tradeo function f with that of [11]. Theorem 41 Let S be an irreducible constrained system. Then f () = f (o) () on rationals 0 . of f (o) (), there exists a subsequence of pairs (q, Iq ) such that where |Iq |/q = . Using this subsequence, we can form a single f () lim sup
q log M (q, Iq ) = f (o) (). q
Proof. We rst show that f () f (o) () for rational in [0, ]. From the denition
log M (Iq ,q) f (o) (), q (Iq ) I() such that
To show the other direction, rst note that f (o) () = f () by Corollary 25, so it suces to prove f (o) () f () for rational 0 < . Fix < . Let = k/N in lowest terms, so k and N are co-prime. Fix > 0. By
Corollary 39, f () is continuous at , so there exists > 0 such that |f () f ()| < whenever | | < . Fix a rational (, min{ + , }). Let (Iq ) I() achieve f (). For any sequence q of length q satisfying Iq , we
construct a new sequence q that has insertion rate exactly (by lling in extra boxes and extending the length). The new sequence will induce a code rate equal to f (). For all suciently large q, For each q, rewrite = k/N = rq /sq , where sq = N q/N , so that q sq q + N . q N
rq = k
q kq +k = |Iq | + k N |Iq | |Iq |, since q/|Iq | / < 1. Thus we can ll in |Iq | rq boxes within q and extend the
Denote the resulting set of box positions by Isq . The sequence q obtained by this procedure has insertion rate exactly , length sq , and satises Isq .
length of q by a small amount sq q, using only a path with no boxes in its label.
4.5. NRZ AND NRZI TRADEOFF FUNCTIONS
83
Now,
M (q, Iq ) M (sq , Isq )
log M (sq , Isq ) sq log M (q, Iq ) q sq q log M (sq , Isq ) log M (q, Iq ) lim sup 1 f (o) (). f () = lim sup q sq q q
This implies that f (o) () f () for all > 0. Thus, f (o) () f (). Thus, the relationship between f , f (o) and f (p) is given by f (p) () f (o) () f ().
4.5
NRZ and NRZI Tradeo Functions
In recording channels, binary data is often represented in two dierent ways: NRZI and NRZ as mentioned in Section 1.2. Let x = x1 . . . xn be a binary sequence in NRZI. The corresponding NRZ sequence y is given by yi = yi1 xi . Thus, given a sequence in one representation, there are two corresponding sequences in the other representation, depending on the value of y0 . For example, 001011 in NRZI corresponds to 001101 and 110010 in NRZ, while 001101 in NRZ corresponds to 001011 and 101011 in NRZI. Suppose that S1 is a constrained system in NRZI and S2 is the NRZ-precoded version of S1 . It can be shown that cap(S1 ) = cap(S2 ) [47].
4.5.1
RLL(0, k) and MTR(j) Systems
Recall that the denitions of RLL and MTR constraints in Chapter 1 use the NRZI representation. As mentioned in Chapter 1, the RLL(0, j) constraints are equivalent to the MTR(j) constraints with the labels reversed (0 1, 1 0). The main result presented here is that the NRZI tradeo functions of RLL(0, k) (and thus MTR(j)) are greater than their NRZ counterparts.
84
Proposition 42 Let S1 be the RLL(0, k) constraint . Let S2 be the NRZ-precoded version of S1 . Then for 0 1, fS2 () fS1 (). Proof. Letting q N and I {0, . . . , q 1}, dene Mi (q, I) to be the number of words in Si of length q satisfying I. Let [0, 1]. By Proposition 7, there is a sequence (Iq ) I() such that log M2 (q, Iq ) = fS2 (). q
lim sup
q
Let Wq be the set of words in S2 of length q satisfying Iq . For each q, we construct a set Yq of words of length q from Wq+1 as follows.
1. If 1 Iq+1 , replace the rst position of every word in Wq+1 with 1 to obtain Wq+1 . Otherwise, let Wq+1 = Wq+1 .
2. For each w Wq+1 , dene (w) to be the word of length q as follows.
Case 1: S1 is the RLL(0, k) constraint. If wi+1 = j is the largest index such that j i and wj = Case 2: S1 is the MTR(j) constraint. If wi+1 = (w)i = .
. If wi+1 =
, let (w)i = wi+1 wj where , let (w)i = .
mod 2) where j is the largest index such that j i and wj =

3. Let Yq = {(w) | w Wq+1 }.
, let (w)i = wi+1 wj (i j . If wi+1 =
, let
Observe the following properties of Yq :

Yq S1 and all y Yq satisfy Iq = {i 1 : i Iq+1 \ {0}}. The number |Iq | of unconstrained positions of Yq satises |Iq+1 | 1 |Iq | |Iq+1 |. Wq+1 .
This is because we neglect at most one unconstrained position when constructing
4.5. NRZ AND NRZI TRADEOFF FUNCTIONS
85
|Yq | |Wq+1 |/2. This follows from the fact that, given y Yq , there are at most
two words, w and w , in Wq+1 such that (w) = (w ) = y. These words are the
ipped versions of each other.

From the second property, (Iq ) I(). Hence, log log M1 (q, Iq ) log |Yq | fS1 () lim sup lim sup lim sup q q q q q |Wq+1 | 2
= fS2 ().
Chapter 5 Constraints with a Known Tradeo Function

In the previous chapter, we established many interesting properties of the tradeo function f , aided by the graph construction G. However, as noted at the end of Sec tion 4.3, computing f exactly from G remains largely unsolved. In the next chapter, we will describe some general lower and upper bounds for the tradeo function of any constraint. However, in this chapter, we will focus on constraints for which the tradeo function can be determined precisely and report results on our investigation into optimal congurations for the class of MTR(j) constraints. Before our work on constrained systems with unconstrained positions, explicit descriptions of the tradeo functions were known for 2 constraints, namely MTR(1) and MTR(2), as given in [11]. One of our contributions is to add MTR(2, 2) and RLL(d, 2d + 1) for all d 1 to the list of known cases. This section is structured as follows. We start by exploring bit-stung schemes
and its extensions in Section 5.1, as bit-stung schemes were shown to achieve the tradeo function for MTR(1) and MTR(2). Then in Sections 5.2 and 5.3, we describe the tradeo functions of the MTR(2, 2) and RLL(d, 2d + 1) constraints. Finally, in Section 5.4, new results on optimal periodic congurations for the MTR(j) constraint at low insertion rates are presented. We also prove that the tradeo function for low insertion rates for MTR(3) is a straight line achieved by standard bit-stung. 86
5.1. EXTENSIONS OF BIT-STUFFING
87
5.1
Extensions of bit-stung
We mentioned in Section 1.4.2 that a piecewise-linear lower bound h() on the tradeo function of MTR(j) was derived from the timesharing of standard bit-stung schemes. The curve h() was shown to be concave and to equal the tradeo function for MTR(1) and MTR(2) in [11]. We plot the tradeo functions for MTR(1) and MTR(2) in Fig. 5.1. Note that the MTR(2) tradeo function is not a straight line as there is a kink at insertion rate 1/3.
0.8791 0.6942
fMTR(1)
fMTR(2)
1 2
2 3
Figure 5.1: Tradeo Functions for MTR(1) and MTR(2). The b-bit-stung scheme for MTR(j) can be extended in several ways. In particular, the string of b ones that are stued do not have to be stued contiguously. In fact, any two periodic congurations with period j + 1 and unconstrained sets introduces additional exibility in the choice of unconstrained positions for MTR(j) operating at insertion rates b/(j + 1), for 1 b j. Proposition 43 For S = MTR(j), let U {0, . . . , N 1} be a set of unconstrained positions with |U | = b. We denote the complement of U by U c . Let x be a sequence of period N over A3 with xi = if i mod N U . Denote the restriction of x to the U1 {0, . . . , j} and U2 {0, . . . , j} achieve the same code rate as shown below. This
88
CHAPTER 5. CONSTRAINTS WITH A KNOWN TRADEOFF FUNCTION
positions in a set W by x|W . For N = j + 1, the following holds. x S = x|U c satises MTR(j b). The above implication is false for N > j + 1. Conversely, it is true for all N that: x|U c MTR(j b) = x S. (5.2) (5.1)
Proof. We prove the contrapositive of statement (5.1) for N = j + 1. Suppose x|U c x|U results in x having a binary lling of j + 1 ones, thereby violating the MTR(j) constraint. For N > j + 1, we show that statement (5.1) is false by a counterexample. The sequence x = 1 1 0 0
jb+1 Uc N j1 Uc
does not satisfy MTR(j b). Then x|U c contains j b + 1 ones. Stung b
into
b U
is in S for MTR(j), but clearly, x|U c violates MTR(j b). Statement (5.2) is true because each
that is inserted increases the length of the
longest string of ones in the binary lling of x by at most one. We can also extend the bit-stung scheme to any nite-type constraint with memory m, where b-bit-stung would result in the periodic conguration with block length reected in the restriction that the insertion rate b/(m + 1) be strictly less than the maximum insertion rate S . The asymptotic code rates achieved and the corresponding lower bound are discussed in the next chapter. N = m + 1 and unconstrained set U = {b, . . . , m b}. The limit on the size of b is
5.2
Tradeo Function for MTR(2, 2)
The RLL(0, 1) constraint (see Fig. 2.2) is equivalent to the MTR(j = 1) constraint. Thus, the tradeo function of RLL(0, 1) is the same as that of MTR(1).
5.2. TRADEOFF FUNCTION FOR MTR(2, 2)
89
The MTR(2, 2) constraint is the NRZ version of the RLL(0, 1) constraint. At zero insertion rate, it shares the same capacity 0.6942 as RLL(0, 1). From Theorem 19, the maximum insertion rate for MTR(2, 2) is 1/3. Clearly, any point on the straight line connecting the points (0, 0.6942) and (1/3, 0) can be achieved by weighted bit-stung schemes, and the line presents a lower bound to the tradeo function for MTR(2, 2). We show in the next proposition that this line is indeed the tradeo function. Proposition 44 Let S denote the MTR(2, 2) constraint. Then fS is given by the line through the points (0, 0.6942) and (1/3, 0). Proof. The essence of this proof is to use a matrix inequality to imply an eigenvalue inequality: namely, if X and Y are nonnegative matrices and X entry-by-entry inequality, then (X) (Y ).
(p)
Y , where
denotes
Since MTR(2, 2) is nite-type, it suces by Corollary 28 to use only G to compute

(p)
fS . However, by Theorem 34, fS () achieves fS (), so it suces to show that fS () is upper bounded by the straight line. From G in Fig. 5.2, let A denote the part of AG corresponding to the edges with binary labels and B denote the part of AG corresponding to the edges with box labels, i.e., 0 1 1 0 0 0
and
A=
B=
0 0 1 0 0 0 1 0 0 1 0 0 , 1 0 0 0 0 0 0 0 0 1 0 0 0 1 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 1 . 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
90
u 0 y
0 1 0 1 G
v 1 x F(1) F(00) 1
F(0) 0 F(11) 0
0 1 1 G
F(00) 1 F(1)
0 F(0) F(11)
Figure 5.2: The Shannon cover G and the only non-trivial component G (of G) for MTR(2, 2). The subgraph of G with solid edges is H, which is isomorphic to G.
By inspecting G in Fig. 5.2, it is clear that an unconstrained position with label is always followed by two constrained positions. This can only happen in the follower set transitions given below: F(1) F(0) F(11) F(00) F(1), F(0) F(1) F(00) F(11) F(0). Given a length N , a rational insertion rate [0, 1/3], and a specication of
1 0 0 1
unconstrained positions U {0, . . . , N 1} with |U |/N = , dene the matrix representing the periodic (U, N ) conguration by M = M0 M1 MN 1 ,
where Mi = B if i U and Mi = A if i U , /
with the indices taken mod N . Each entry of M represents the number of llings in S satisfying U , with length exactly N , that begin and end with a restricted set of prexes and suxes respectively. The restrictions in the follower set transitions deduced above imply that any when evaluated has only two nonzero entries (each of value 1), in the (1, 1) and (3, 3) any n 1. It is also straightforward to verify the matrix inequality AC A. M0 M1 MN 1 can be written in terms of A and another matrix C = BA2 , which
entries. Since the nonzero entries are situated along the main diagonal, C n = C for
5.2. TRADEOFF FUNCTION FOR MTR(2, 2)
91
Thus, for any N and M = An1 C m1 An2 C m2 . . . Anl C mk , where ni , mi , l, k are nonnegative integers such that bound M repeatedly: M = An1 C m1 An2 C m2 . . . Anl C mk , An1 +n2 C m2 . . . Anl C mk , A = A
l i=1
ni + 3
k i=1
mi = N , we can use AC
A to upper
l i=1 l i=1
ni ni
C mk , C
k i=1
mi
where the last inequality uses the relation C n = C for any n 1.
The expression on the right corresponds to another periodic conguration sharing
the same length N but with a dierent set U of unconstrained positions, and higher code rate (due to the eigenvalue inequality). However, the new (U , N ) conguration can be viewed as using a weighted average of the 0-bit stung and the one-in-three bit stung schemes, which have code rates 0.6942 and 0 respectively. This implies that the tradeo function fS must be a straight line connecting those two points.
0.6942
fRLL(0,1) fMTR(2,2)
1 3 1 2
Figure 5.3: Tradeo functions for RLL(0, 1) and MTR(2, 2). The tradeo functions for RLL(0, 1) and MTR(2, 2) are given in Fig. 5.5.
92
5.3
Tradeo Function for RLL(d, 2d + 1)
In this section, we consider runlength constraints of the form S = RLL(d, 2d + 1). From Theorem 18, the maximum insertion rate is S = 1/(2d + 2). Clearly, an achievable rate region is given by the area under the straight line connecting the points (0, capS ) and (1/(2d + 2), 0). It is shown in the next proposition that this line is indeed fS . Proposition 45 Let S be the RLL(d, 2d + 1) constraint. Then fS is the line connecting (0, capS ) and (1/(2d + 2), 0). Proof. As the proof is similar to that of Proposition 44, only a proof sketch is provided. Since S is nite-type, there is some periodic conguration induced by the label of some cycle in G that achieves fS . The G presentation of RLL(d, 2d + 1) is shown in Fig. 5.4. From the gure, it is clear that an unconstrained position with label can only happen in the follower set transition given below:
0 0
is always followed by d constrained positions to reach state F(102d+1 ). This

0
F(10d ) F(1) F(10d+1 ) F(10d1 ) F(102d ) F(102d+1 ). Note that even though there are box edges emanating from F(10s ) for s = d+1, d+
2, . . . , 2d in the full unconstrained presentation, all the these follower set transitions terminate in {} as shown below F(10s ) F(1) F(10s+1 ) F(102ds ) F(102d+1 ) {}. Thus, given a length N , a rational insertion rate [0, 1/(2d + 2)], and a set
0 0
taken mod N .
M = M0 M1 . . . MN 1 where Mi = B if i U and Mi = A if i U with the indices / Since each B is followed by 2d + 1 As, M can be viewed as a product of As
of unconstrained positions U {0, . . . , N 1} with |U |/N = , dene the matrix
and Cs, where C = BA2d+1 . It is easy to check that the matrix C has only one nonzero entry of value 1 in the (d + 1, d + 1)-th entry. In particular, C represents a
5.3. TRADEOFF FUNCTION FOR RLL(D, 2D + 1)
93
periodic (U, N ) conguration with insertion rate 1/(2d + 2) and asymptotic code rate log2 ((C))/(2d + 2) (which when evaluated equals 0). Use the matrix relations AC veried) to upper bound any M by a matrix M = AN (2d+2)|U | C |U | that spans the same number of time steps as M . The resulting matrix M represents a time-sharing scheme between the congurations induced by C and A.
F(10) F(10d+2 ) 0 F(1) F(10d+1 ) F(1) 0 F(10) 0 F(10d ) 0 F(10d+1 ) 1 1 0
A and C n = C for any n 1 (which can be easily
F(10d1 ) F(102d ) 0 0 0 F(102d+1 ) 1
Figure 5.4: The irreducible component G of G for RLL(d, 2d + 1).
The tradeo function for RLL(d, 2d + 1) is depicted in Fig. 5.5.
capS (d)
fRLL(d,2d+1)
1 2d+2
Figure 5.5: Tradeo function for RLL(d, 2d + 1).
94
5.4
Optimal Code Rates for MTR(j) at Low Insertion Rates
In this section, we describe results on the optimal congurations and code rates obtained for low insertion rates for MTR(j). From extensive computer simulations on the code rates of MTR(j) constraints in the low insertion rate region, we observed that high code rates are usually obtained from periodic congurations that spread the unconstrained positions apart by j spaces; this is in contrast to low code rates obtained from periodic congurations that cluster the unconstrained positions. Throughout this section, let S = MTR(j). Recall that the G presentation for S is given in Fig. 2.4. Let A denote the part of the (j + 1) (j + 1) adjacency AG corresponding to the edges with box labels, i.e., A= and 1 1 0 0 , 0 1 0 0 0 0 . 0 0 matrix AG corresponding to the edges with binary labels and B denote the part of
1 0 1 0 0 0 . . 0 0 ... 0 0 . 1 0 0 1 0 0 0 0
(5.3)
Clearly, B is nilpotent with B j+1 =0. Let C denote the product BAj . U {0, . . . , N 1} such that = |U |/N , the periodic conguration can be represented Given a length N , an insertion rate [0, 1], and a set of unconstrained positions
0 0 1 0 0 0 B = 0 0 0 ... 0 0 0 0 0 0 0 1 0 0 0 0 0 0
0 1 0
(5.4)
i U . Viewing M as a sequence of As and Bs, we can decompose the sequence into /
by a matrix product M = M0 M1 MN 1 , where Mi = B if i U and Mi = A if
5.4. OPTIMAL CODE RATES FOR MTR(J) AT LOW INSERTION RATES
95
runs of As (representing constrained positions) and Bs (representing unconstrained positions). Note that if M = 0, each runlength of Bs is at most j since B j+1 = 0. We are interested in the eect that the spacings between two unconstrained positions have on the code rates achieved. We refer to the spacing or the size of the spacing as a gap or separation, and use them interchangeably. The two main results of this section are presented next. The rst result is a theorem showing that the tradeo function for MTR(3) is achieved by the bit-stung lower bound h() (and is thus linear) for insertion rates close to 0. Theorem 46 Let j = 3. Let g1 , . . . , gk 0 be gaps and G = k 1 < , k+G 3(j + 1) then (C k AGkj ) (BAg1 BAgk ).
k i=1
gi . If
Note that the matrix C represents the 1 : j + 1 bit-stung scheme and is thus a periodic conguration that yields the second corner point in the standard bitstung lower bound h(). Therefore, the matrix product C k AGkj can be viewed as a time-sharing scheme using 0-bit-stung and 1 : j + 1 bit-stung. Using this and concavity arguments, the tradeo function for MTR(3) can be extended to the insertion rate region [0, 1/(j + 1)] as is stated in the following corollary. Corollary 47 The graph of the tradeo function fMTR(3) () for 0 1/4 is
the straight line that connects (0, cap(MTR(3))) to (1/4, (3/4)cap(MTR(2))). So, picted in Fig. 5.6. In fact, Theorem 46 is conjectured to hold for all j, that is, Conjecture 48 The tradeo function for MTR(j) for insertion rate 0 a straight line given by the standard bit-stung lower bound h().
1 j+1
fMTR(3) () agrees with the standard bit-stung lower bound in this interval, as de-
is
96
0.9468 fMTR(3) 0.6594
1 4
Figure 5.6: Tradeo Function for MTR(3) for 1/4. Recall that Conjecture 48 is true for MTR(1) and MTR(2). We present a partial result pertaining to Conjecture 48 in the next theorem. Theorem 49 Let j 7. Let g1 , . . . , gk be strictly positive gaps and G = k 1 < , k+G 3(j + 1) then (C k AGkj ) (BAg1 BAgk ). Theorem 49 states that for MTR(j) with j 7, a matrix M = BAg1 BAgk with
k i=1
gi . If
no two consecutive Bs can be mapped into another matrix M = C k AGkj with the same number of Bs and the same length, where the spectral radius (or equivalently code rate) of M is at least that of M . Ideally, we would like to remove the assumption of disallowing two consecutive Bs so that Conjecture 48 can be established for all j. Empirically, consecutive Bs are avoided whenever possible when operating in the low insertion rate region, as they result in low code rates. Thus in practice, Theorem 49 is a useful guide to optimal congurations for MTR constraints with unconstrained positions for j up to 7. In the remaining sections, we describe some analysis of the optimal periodic congurations with regard to the gap spacings.
97
5.4.1
Topping up Small Gaps
It turns out to be extremely dicult to show analytically that small gaps (< j) result in suboptimal code rates, despite heuristic and empirical evidence. This is not only because the matrix inequalities we have to deal with contain entries which are multivariate in at least 3 quantities (including the size), but also because the matrix entries involve changes in the length of recursion, as is shown in Appendix A. We present some partial results below. The basic proof technique is to use a matrix inequality to imply an eigenvalue inequality, i.e., if X and Y are nonnegative matrices and X Y , then (X) (Y ). The following lemma deals with topping up one small gap up to j, and is proved using generalized Fibonacci relations in Appendix A. Lemma 50 (Inductive Step) Let g be a gap with 0 g < j. Then the following Ajg BAg+1 BAj+1 Ajg BAg BAj+2 . (5.5)
inequality holds:
In fact, for any integer m 0, the above inequality can be extended to Ajg BAg+1 C m BAj+1 Ajg BAg C m BAj+2 . (5.6)
The next two lemmas show that consecutive small gaps gi can be systematically topped up to size j, under the condition that gi j gi+1 . Lemma 51 Let g be a gap 0 g < j, and let m be an integer 0. Then the following inequality holds:
Ajg C m+1 BAj+1
Ajg BAg C m BA2jg+1 .
(5.7)
98
Proof. Let l = j g. Apply inequality (5.6) of Lemma 50 inductively to obtain Ajg BAg C m BA2jg+1 Ajg BAg+1 C m BA2jg Ajg BAg+l C m BA2jgl1 = Ajg C m+1 BAj+1 .
Lemma 52 Let g1 , . . . , gk1 be small gaps which satisfy gi jgi+1 for i = 1, . . . , k 2. Then the following inequality holds: Ajg1 C k1 BAj+1
k1
Ajg1 BAg1 BAg2 BAgk1 BA(k+1)jG+1 ,
(5.8)
where G =
i=1
gi .
Proof. It follows directly from Lemma 51 that Ajg1 BAg1 BAg2 BAgk1 BA(k+1)jG+1 Ajg BAgl BAg2 BAgk2 CBAkj+1 Ajg1 BAg1 BAg2 C k3 BA3jg1 g2 +1 Ajg1 BAg1 C k2 BA2jg1 +1 Ajg1 C k1 BAj+1 .
k2 i=1 gi
Ideally, we would like to show that any stretch of small gaps (< j) can be topped up to size j simultaneously no matter how long the stretch is. Instead, we prove a weaker version of that for j 7, where we assume the sequences of As and Bs do not contain two consecutive Bs anywhere.
Lemma 53 (Weak Small Separation Lemma) Assume j 7. For gaps 0
99
g1 j and 0 < g2 , g3 , . . . , gk < j, the following inequality holds: Ajgk BAgk BAg2 BAj C m BA2j+2 Ajgk BAgk BAg2 BAg1 C m BA3jg1 +2 . (5.9)
Applying Inequality (5.9) repeatedly then yields Ajgk C k+m BA2j+2 where G =
k i=1
Ajgk BAgk BAg1 C m BA(k+2)jG+2 ,
(5.10)
gi .
The proof is very long and technical and is thus included in Appendix A. Note that Inequality (5.9) is true when the gaps gi are not 0, whereas we expect Inequality (5.10) to hold even when there are gaps gi of size 0. The idea of the Weak Small Separation Lemma is illustrated in Fig. 5.7 for the case of three Bs (shaded in gray). The three Bs are not consecutive, so thus satisfy the assumptions of Lemma 53. Applying the lemma once tops up the rightmost gap that is sandwiched between 2 Bs to size j, with the excess As taken from the long stretch on the right. Applying Lemma 53 a second time tops up the rightmost gap that is sandwiched between 2 Bs with size < j. Again, this is done by siphoning As from the long stretch on the rightmost end.
Figure 5.7: Illustrating Lemma 53 for the case of three Bs (shaded in gray). We can prove another version of Lemma 53 where we restrict to MTR(3) but allow gaps of size 0.
100 CHAPTER 5. CONSTRAINTS WITH A KNOWN TRADEOFF FUNCTION
Lemma 54 (Small Separation Lemma for MTR(3)) Let j = 3. Let g1 , . . . , gk1 be small gaps and let G = Ajgk1 C k1 BAj+1
k1 i=1
gi . Then (5.11)
Ajgk1 BAgk1 BAgk2 BAg1 BAkjG+1 .
The proof is given in Appendix A, using Lemma 53 and four additional matrix inequalities.
5.4.2
Reduction of Large Gaps
When the period is a multiple of (j + 1), the average gap size is exactly j, so the resulting conguration with uniformly spaced unconstrained positions corresponds to the 1 : j + 1 bit-stung scheme with matrix representation C. If |U |/N < 1/(j + 1), then the average gap size is > j. This leads to the question of whether at insertion rates < 1/(j + 1), congurations with unconstrained positions spaced uniformly apart by the average gap size yield code rates that are close to optimal. We show below that this is not the case: for a xed period and insertion rate < 1/(j +1), it is possible to reduce all the large gaps (except one) to size exactly j, yielding improvements in code rates. In the next lemma, we show that a large gap g ( j) can be reduced to j by siphoning the excess As (representing the constrained positions) to the rightmost end of the sequence. However, the result can be stated more generally by combining it with Lemma 50. Lemma 55 Let g be a gap g 0, and m be an integer 0. Then Aj C m+1 BAj+g+1 Aj BAg C m BA2j+1 . (5.12)
Eqn. (5.6) in Lemma 50. When d = 0, equality holds in Eqn. (5.12). Thus, it remains
Proof. Write g = j + d, where d is an integer j. For d < 0, this is covered by
5.4. OPTIMAL CODE RATES FOR MTR(J) AT LOW INSERTION RATES 101
to prove Eqn. (5.12) by induction on d > 0. First, we apply the Cayley-Hamilton Theorem to the MTR(j) constraint to obtain Aj+1 = I + A + + Aj1 + Aj . (5.13)
Using Eqn. (5.13), we can rewrite Ag as Ad1 Aj+1 = Aj+d1 +Aj+d2 + +Ad +Ad1 . d + 1. Expanding Aj+d+1 in Aj BAj+d+1 C m BA2j+1 , we obtain
Now assume the inductive hypothesis is true for d 0 and show that it is true for
Aj B(Aj+d + Aj+d1 + + Ad )C m BA2j+1

j+d
=
x=d
Aj BAx C m BA2j+1
where each term in the sum is upper bounded by Aj C m+1 BAj+x+1 by the inductive hypothesis.Thus, Aj BAj+d+1 C m BA2j+1
j+d
Aj BAj C m BAj+x+1 = AC This completes the induction. A stronger result than Lemma 55 holds for j 7 below: be an integer 0. Then Lemma 56 (Large Separation Lemma) Let j 7. Let g be a gap j, and m
x=d j m+1
BA2j+d+2 .
BAj C m BAj+g
BAg C m BA2j .
(5.14)
The technical proof is included in Appendix A. Lemma 56 can be used to reduce a run of large gaps progressively from right to left, provided that there is an isolated large gap of size at least 3j to the right, as shown in the next proposition.
Proposition 57 Let j 7. Let g1 , . . . , gk be large gaps such that g1 3j and gi j for i = 2, , k, and let Gk =
k i=1
gi . Then
C k AGk kj
BAgk BAgk1 BAg2 BAg1 .
(5.15)
Proof. We perform an induction on the number of separations k. This is trivially true for k = 1. For k = 2, C 2 AG2 2j BAg2 BAg1 is true by Lemma 56, since we can let the exponent of C in Eqn. (5.14) be zero, and post-multiply both sides by Ag1 2j . Now assume that Inequality (5.15) is true for k = n, and show that it is also true for n + 1. BAgn+1 BAgn BAg2 BAg1 (by Lemma 56)
BAgn+1 C n AGn nj
BAj C n AGn nj+(gn+1 j) C n+1 AGn+1 (n+1)j .
Thus, for a sequence with all spacings exceeding j, we can apply the Large Separation Lemma inductively from right (except the rightmost gap) to left, as illustrated in Fig. 5.8.
Figure 5.8: Illustrating Lemma 56 and Proposition 57 step-by-step for the case of three Bs (shaded in gray)
5.4.3
Unifying the Arguments on Large and Small Gaps
The proofs of Theorems 46 and 49 are similar, and involve applying the lemmas on large and small gaps alternately to a matrix M0 = BAg1 BAgk with k gaps of length N , and whose insertion rate k/N is < 1/(3(j + 1)). It will be shown in stages M and (M ) = (Mi1 ) and Mq = C k AGkj . Each that there is a sequence of matrices M0 , . . . , Mq such that for i = 1, . . . , q, there is a matrix M such that Mi Mi1 will be a product of a sequence of As and Bs and M will be a product of a cyclically permuted sequence of As and Bs. Proof of Theorem 49:. As before, we say that a gap gi is large if gi j and small
the unit circle, with numbers increasing to the right moving clockwise. Subdivide odd (even) s and for all i Is , gi is large (small). Note that r must be even due to
otherwise. View the interval {1, . . . , k} as a sequence of uniformly spaced points on
the circle into maximal, consecutive disjoint subintervals I1 , . . . , Ir such that for all working on the circle, so every subinterval contains an interval of opposite parity on both the right and the left. The average value of the gi s is at least 3j because of the assumption on the one, choose the rightmost one) such that gl 3j, then applying Proposition 57 to the gi s for i = p, . . . , l yields new gi = j for i = p, . . . , l 1, gl 3j +
lp i=p
insertion rate. For odd s, if Is = {p, . . . , q} contains an index l (if there is more than gi (l p)j,
and gi < 3j for i = l + 1, . . . , q. The resulting matrix product in this interval is thus (BAj )lp BAgl BAgl+1 BAgq .
Note that in the process, we have changed neither the sum nor the number of gi s, so the average gi remains the same. Decompose the intervals into adjacent pairs, Is Is+1 (where s is even and the gi
iIs Is+1
subscripts are taken mod r). A weighted average over all s in {1, . . . , r} of
|Is Is+1 |
gives the average gi and hence 3j, so we can choose a pair Is Is+1 with indices Is = {u, . . . , v}, satisfying
w
Is+1 = {v + 1, . . . , w},
i=u
gi 3j(w u + 1).
(5.16)
for i = l + 1, . . . , w.
As noted above, the interval Is+1 contains an index l such that gl 3j. and gi < 3j
We now check that the Weak Small Separation Lemma (Lemma 53) is applicable to Is by showing that gl has sucient As for topping up the small gaps in Is . Let H =
v i=u
gi . The number of As needed to be siphoned from gl to top up the
small gaps in Is is (v u + 1)j H. Thus, we need to show gl ((v u + 1)j H) 0. By Inequality (5.16),
w w
(5.17)
H 3j(w u + 1) Using this, we have
i=v+1
gi = 3j(w u + 1) gl (l v 1)j
gi .
i=l+1
gl ((v u + 1)j H) gl (v u + 1)j + 3j(w u + 1) gl (l v 1)j

w
gi
i=l+1
= 3j(w u + 1) (l u)j
gi
i=l+1
3j(w u) (l u)j 3j(w l) = 2j(l u) 0,
where the second-to-last inequality is due to gi < 3j for i = l +1, . . . , w, while the last inequality is because l is to the right of u by at least one. This proves Inequality (5.17). Applying the Weak Small Separation Lemma to Is Is+1 thus results in replacing BAgu B gl BAgl+1 BAgw1 BAgw by C lu BAgl BAgl+1 BAgw , where gl is the new value of gl after topping up the small gaps in Is . gaps. If gl 3j, Proposition 57 can be applied to Is+1 Is Is+1 to drain the excess process replaces BAgt BAgu1 C lu BAgl BAgl+1 BAgw with C lt BAgl BAgl+1 BAgw . If gl < 3j, we can pick another pair of intervals Is Is +1 (with s is even, and s = s) of the sum of all gaps does not change. The procedure of alternately applying Proposition 57 and the Weak Small Separation Lemma can be then performed repeatedly until there are no remaining gaps gi < j. The proof of Theorem 46 is very similar except that the Small Separation Lemma for MTR(3), (Lemma 54) is used in place of the Weak Small Separation Lemma (Lemma 53). In fact, the ideas of this proof are applicable to any j provided that the Small Separation Lemma and the Large Separation Lemma can be established for MTR(j). It is an open problem to extend these lemmas to an arbitrary j. where there is a gap in Is +1 with size 3j. Such a gap always exists because the size

Next we merge Is Is+1 with Is1 = {t, . . . , u 1}, which also consists of large
As from each large gap in Is1 into gl . Denote the new value of gl by gl . The merging
Chapter 6 Bounds for the Tradeo Function

As seen in the previous chapter, the tradeo function is generally dicult to characterize explicitly, even for the well-known RLL(d, k) and MTR(j) constraints. Thus, it is useful to look for good upper and lower bounds applicable to arbitrary constraints. A good lower bound can be used in place of the actual tradeo function as a blueprint for code design, while a good upper bound can impose limits on code performance. Section 6.1 focuses on the various techniques we investigated in obtaining lower bounds to the tradeo function. This is followed by Section 6.2 on general upper bounds. Performance of these bounds for some MTR(j), MTR(j, k) and RLL(d, k) constraints are illustrated in Section 6.3. In the next chapter, we will describe an interesting upper bound on the tradeo function of MTR(j) obtained from the study of time-varying MTR constraints.
6.1
Lower Bounds
Lower bounds for a given constraint S are obtained by timesharing between admissible (U, N ) congurations for S, where U is some unconstrained set of period N . Some low complexity strategies for nding these admissible (U, N ) congurations are explored in this section. We remark that in the case of irreducible constraints, given any two lower bounds f1 and f2 for f , their convex combination g(f1 , f2 ) is also a lower bound for f . This technique is used to construct an improved lower bound. 106
6.1. LOWER BOUNDS
107
The notation used in this chapter is as follows. Let G denote a deterministic, irreducible graph over A3 and fG denote its tradeo function. Decompose the adjacency matrix AG = A + B where A corresponds to the edges with label 0 and 1 in G and B corresponds to the edges with label . Generally, AB = BA.
6.1.1
Greedy Lower Bound
First, we use Proposition 11 to derive a lower bound based on the greedy strategy [17, Chapter 17]; we call this the greedy lower bound. Recall that a greedy strategy is one that selects the locally optimal choice at any time instant, without any regard for global optimality. For a given number c and integer N , the following algorithm returns an insertion rate such that fG () c. Greedy lower bound Inputs: c, N , A, B CA for 1 k N 1 b0
if (((C))1/k c and (CBAN k1 )1/N c) then C CB else C CA bb+1
Output: b/N .
The algorithm starts with C = A and iteratively multiplies C by B until the code rate is less than c. At the end, C is a product
N i=1
Ci where Ci is either A or B.
N i=1
The algorithm ensures that (log (C))/N c. By Proposition 11, c is a lower bound for fG (b/N ), where b is the number of Bs in the product Ci . It turns out that the greedy strategy is suboptimal and performs worse than most other lower bounds, as will be seen in Section 6.3.
108
CHAPTER 6. BOUNDS FOR THE TRADEOFF FUNCTION
6.1.2
Dynamic Programming Lower Bound
The next technique is based on the method of dynamic programming [17, Chapter 16]. Recall that dynamic programming recursively breaks a problem down into subproblems of a similar structure in such a way that the solutions to the subproblems can be optimally extended to solve the problem at the next level. We rst describe the dynamic programming idea applied to maximizing the spectral radius of a product of non-commutating nonnegative matrices A and B. This is given as an exercise in [7]. Fix a nonnegative square matrix D of the same size as A and B. Dene N (D) = maxZi {A,B} (ZN ZN 1 Z1 D). Then 1 (D) = N (D) =
Zi {A,B}
max ((AD), (BD)) , max (N 1 (AD), N 1 (BD)) .
Zi {A,B}
Dynamic programming lower bound Inputs: N , A, B C0 A C1 B
for 2 N
C0 = C0 A
for 1 b 1 E Cb A D Cb1 B
if ((D) (E))
then Cb D else Cb E
C C1 B
Outputs: (log (Cj ))/N for all 0 j N
Cj Cj for all 0 j
6.1. LOWER BOUNDS
109
For our purpose, take D to be the identity matrix. However, the above description precludes the constraint on insertion rate, which corresponds to the ratio of the number of Bs to the length N . In the algorithm presented, the optimal partial solutions stored are the matrices Cj for all 0 j N , which are the matrix products maximize the spectral radius. Hence (log (Cj ))/N is a lower bound for fG (j/N ). insertion rates but drops o sharply at mid to high insertion rates. consisting of N j copies of A and j copies of B (i.e., with insertion rate j/N ) that Empirically, examples in Section 6.3 show that this bound performs well at low
6.1.3
Standard Bit-stung Lower Bound
As discussed previously, standard bit-stung achieves the tradeo function for the MTR(1) and MTR(2), constraints and gives rise to a piecewise linear lower bound for MTR(j) with j 3. With the generalization of the standard bit-stung scheme from MTR(j) to an
arbitrary nite-type constraint S with memory m, we can obtain a similar lower bound hS , which we call the (standard) bit-stung lower bound. For insertion rate , the bound hS () is given by the piecewise-linear curve that connects the following points: (0, log (A))
1 m ( m+1 , m+1 log (Am B)) 2 m1 ( m+1 , m+1 log (Am1 B 2 ))
b ( m+1 , mb+1 log (Am+1b B b ) m+1
(S , 0), where b is the largest integer such that

b m+1
separately from methods listed in Chapter 3. The last point (S , 0) is added since standard bit-stung generally does not yield the maximum insertion rate, i.e.,
b m+1
S . Note that S can be found =
110
S . If a max-insertion-rate cycle is found while computing the maximum insertion rate, the 0 code rate can be replaced by f (S ). Typically, 0 is used for low complexity (e.g., if nding a max-insertion rate cycle takes too long).
6.1.4
Irregular Bit-stung Lower Bound
In the previous lower bounds, the approach was to perform some combinatorial selection of the unconstrained positions either by some deterministic method or through problem subdivision. Here, we attempt to use the G presentation directly in the search for periodic congurations that yield high code rates. We rst describe the approach for nite-type constraints but remark that the method can be easily ex tended to any constraint with multiple components in its G presentation. As before, let G denote the single non-trivial irreducible component of G when the constraint is nite-type. The steps are listed below. 1 Find the set of simple cycles in G . This can be done by a backtracking algorithm returns cycles starting with the smallest numbered state in its state sequence. 100, the algorithm runs very fast. from [61]. Typically, the states in G are labeled from 1 to |VG |, and the search
Theoretically, the cycle search is exponential in running time but for small |VG | < 2 For each cycle , nd the periodic (U, N ) conguration induced by each label of . (There can be more than one label sharing the same state sequence.) For example, if a label of is 01 0 11, then the U = 2, 4 and N = 7. 3 Compute the code rate of each (U, N ) conguration for each cycle in . This can with Mi = A if i U and Mi = B if i U . For each insertion rate encountered, / be computed in constant time by taking log2 (M )/N , where M = M0 M1 MN 1
store the highest code rate obtained as well as its (U, N ) conguration.
4 Take the least concave majorant of the (insertion rate, code rate) pairs. 5 Fix an n 2. For i = 1, , n, concatenate any i cycles and repeat steps 3-4. If the chosen cycles are disjoint, randomly complete the cycles into one long cycle.
6.2. GENERAL UPPER BOUNDS
111
The resulting congurations often yield irregular positioning of the unconstrained positions, so we name this approach irregular bit-stung. This technique achieves at least the rate region of standard bit-stung, and in fact typically achieves more since it returns many improved insertion rate and code rate pairs. Empirical results indicate improvements in the achievable rate region, even for a small number of cycle concatenations. obtained in Section 6.3 for MTR(j) (with j 4), MTR(j, k) and RLL(d, k) constraints
6.2
General Upper Bounds
There are few known upper bounds for the tradeo function for an arbitrary constraint. However, for a given constraint such as MTR(j), the structure of the constraint may give rise to a better upper bound than the techniques listed here. This is discussed in the next chapter. Note that if f1 and f2 are upper bounds for f , then their pointwise minimum h(f1 , f2 ) is an improved upper bound.
6.2.1
Straightline Upper Bound
The following proposition gives a simple but crude upper bound on f . Proposition 58 For 0 , f () cap(S) . Proof. Let = 0 and f ( ) = cap(S) in (4.1). used in conjunction with the constraint in this scheme, Eqn. (6.1) becomes f () cap(S) + RECC 1, where the expression on the right is highly reminiscent of the rate of average intersection, which was shown to equal cap(S)+cap()1 in joint work with J. Fan [23] (where Interestingly, rewriting as 1 RECC , where RECC is the rate of the ECC to be (6.1)
112
this implies that the tradeo function f for any constraint is always less than the rate of average intersection. Thus the scheme of constrained systems with unconstrained positions pays a price in reduced code rate (eectively achieving a lower rate than the average of all schemes involving an intersection of an ECC and the constrained code) in exchange for low complexity in decoding.
cap() is the capacity of the channel with unrestricted inputs). Since RECC cap(),
6.2.2
Dynamic Programming Approximate Upper Bound
Technically, the curve generated in this section is not an upper bound, but rather an approximation. The objective function in this dynamic programming algorithm is to maximize the componentwise row sums instead of the spectral radius of a product of As and Bs. Dynamic programming approximate upper bound Inputs: N , A, B for 1 N x0 Ax0 x0 all-one vector of dimension |VG |
for 1 b 1 z Axb x Bx1 y Bxb1
xb max{y, z} componentwise
xj xj for all 0 j log 1k|VG | xj (k) Outputs: for all 0 j N N (where xj (k) is the k-th entry of vector xj ) At the end of the algorithm, wi =
k
|I| = j, an upper bound on the number of words w = w1 . . . wN S(G) satisfying N is nite, but it converges to a true upper bound as N .
xj (k) represents, for all I {1, ..., N } such that
if and only if i I. Note that this is an approximation to an upper bound as
6.3. PERFORMANCE OF SOME UPPER AND LOWER BOUNDS
113
6.3
Performance of some Upper and Lower Bounds
We now present some examples showing the performance of the various bounds discussed in the rst two sections.
6.3.1
Bounds for MTR(j) constraints
The MTR(4) constraint has capacity 0.9752 and maximum insertion rate 4/5. In Fig. 6.1, the dotted line is the dynamic programming approximate upper bound, while the dashed line is the dynamic programming lower bound with N = 1000, while the solid line is the standard bit-stung given in [11].
0.8 0.6 0.4 0.2 0 0 0.2 0.4 0.6
standard bit-stung Approx. DPUB DPLB
0.8
Figure 6.1: Standard bit-stung lower bound (solid), dynamic programming lower bound (dashed) and approximate upper bound (dotted) of f for MTR(4) The improvements of irregular bit-stung over standard bit-stung mostly congregates in the interval between 0.5 and 0.6. A zoomed-in plot over insertion rates in the interval [0.56, 0.58] that shows the improvements is depicted in Fig. (6.2). In particular, the search replicated the counterexample in [11] with insertion rate 0.5 and code rate 0.4031 that demonstrated the lack of optimality of standard bitstung for MTR(4). Even though the search was only conducted for concatena tions of up to 4 cycles in G (which was very fast), we obtained several points above
114
the standard bit-stung lower bound. For example, (0.5263, 0.3702) with a (U, N ) conguration given by N = 19, U = {1, 2, 3, 6, 7, 11, 12, 15, 16, 17}; (0.5714, 0.3137) {1, 2, 3, 6, 7, 8, 11, 12, 15, 16, 17} and N = 19. with U = {1, 2, 3, 6, 7, 10, 11, 12} N = 14; and (0.5789, 0.3043) induced by U =
0.3279 irregular bit-stung 0.3229 standard bit-stung
0.3179
0.3129
0.3079
0.3029 0.5600
0.5650
0.5700
0.5750
0.5800
Figure 6.2: Standard bit-stung lower bound (solid) versus irregular bit-stung lower bound (dashed) of f for MTR(4) for insertion rates in [0.56, 0.58]
rened points above the corresponding standard bit-stung bound. For example, for MTR(5), the points (0.4, 0.5554) and (0.6, 0.3170) are above the standard bitand (U = {1, 2, 3}, N = 5) respectively. stung lower bound, which are induced by the congurations (U = {1, 2}, N = 5)
For j 5, even the search over the set of simple cycles in G of MTR(j) yield
115
6.3.2
Bounds for MTR(j, k) constraints
The MTR(2, 3) constraint has capacity 0.7947 and maximum insertion rate 0.3750 by Theorem 19. Taking N = 1000, the dashed line in Fig. 6.3 shows the greedy lower bound of f for MTR(2, 3). The dotted line is the upper bound in Proposition 58. The solid line is the standard bit-stung lower bound. Taking N = 1000 again, the dashed line in Fig. 6.4 shows the dynamic programming lower bound (DPLB) while the approximate upper bound is given by the dotted line. The solid line is due to irregular bit-stung, where the search was conducted over concatenations of up to 5 cycles in G.
0.8 0.6 0.4 0.2 0 0 0.2 0.4 0.6
Straightline Upper Bound Greedy Standard Bit-Stung
0.8
Figure 6.3: Standard Bit-Stung lower bound (solid), greedy lower bound (dashed) and straightline upper bound (dotted) of f for MTR(2, 3)
The MTR(4, 4) constraint has capacity 0.9468 and maximum insertion rate 0.6 by Theorem 19. Taking N = 1000, the dashed line in Fig. 6.5 shows the DPLB and the dotted line shows the approximate upper bound. The solid line is the irregular bit-stung lower bound, which has an additional corner point at (0.5000, 0.2500) over the standard bit-stung bound.
116
0.8 0.6 0.4 0.2 0 0 0.2 0.4 0.6
Irregular Bit-stung LB Approx. DPUB DPLB
0.8
Figure 6.4: Bit-stung lower bound (dashed), dynamic programming lower bound (solid) and approximate upper bound (dotted) of the tradeo function for the MTR(2, 3)
6.3.3
Bounds for RLL(d, k) with k > 2d + 1
As the greedy lower bound rarely outperforms the DPLB or the standard bit-stung lower bound, we will omit it here. It will be seen in the examples below that the DPLB drops o rapidly at insertion rates close to the maximum. We only consider RLL(d, k) constraints with k > 2d + 1 here since (i) for k < 2d + 1, the tradeo function is a single point at (0, capRLL(d,k) ) (see Chapter 3); (ii) the tradeo function when k = 2d + 1 is given in Chapter 5. Irregular bit-stung improves upon the achievable rate region given by standard bit-stung at a slight cost in computational time. In fact, for RLL(d, 2d + 2) constraints, irregular bit-stung yields at least two points strictly above the bit-stung lower bound at insertion rates near . For instance, by examining concatenations of up to 3 simple cycles, we obtained the periodic congurations and 2d+1 2d+2 2d+1 2d+2 , 2d+1 , which yield the following respectively pairs of (inser-
tion rate, code rate): (2/(4d + 5), (log (BA2d+1 BA2d+2 ))/(4d + 5)) and (3/(6d + 7), (log (BA2d+1 BA2d+2 BA2d+1 )))/(6d + 7)). The RLL(1, 4) constraint has capacity 0.6174 and maximum insertion rate 0.25.
117
0.8 0.6 0.4 0.2 0 0 0.2 0.4 0.6
Standard Bit-stung LB Approx. DPUB DPLB
0.8
Figure 6.5: Irregular bit-stung lower bound (solid), dynamic programming lower bound (dashed) and approximate upper bound (dotted) of the tradeo function for the MTR(4, 4)
In Fig. 6.6, the irregular bit-stung bound is the solid line obtained by searching over concatenations of up to 5 cycles in G. The standard bit-stung lower bound is the dashed line, while the DPLB (with N = 1000) is the dot-dashed line that drops rapidly to 0 at low insertion rates. The dotted line is the straightline upper bound, which is drawn for comparison. Clearly, the irregular bit-stung bound dominates the lower bounds. We also remark that in [11], an achievable rate point of (0.1667, 0.116) was computed for RLL(1, 4), using an ad-hoc periodic conguration of rate of 0.2372 is achieved, which is at least twice that of the point in [11]. Let m be an odd integer > 1. By Theorem 18, RLL(1, 2 + m) and RLL(1, 3 + m) have the same maximum insertion rate. We have seen that performing iterative cycle search in G (via the irregular bit-stung approach) often produced an improved lower bound, even when the cycles are taken from the set of simple cycles with no concatenations. For the RLL(1, 2 + m) constraints, we observed from simulations that no improvement was obtained even by searching over concatenations of up to 6 simple cycles; that is, it appears that the optimal achievable rate region is governed 5 . Note that from the irregular bit-stung lower bound that at insertion rate 0.1667, a code
118
0.6 0.4 0.2 0 0 0.2
Irregular bit-stung LB Straightline UB Standard bit-stung LB DPLB
0.4
Figure 6.6: Some Bounds for RLL(1, 4) by congurations induced by the set of simple cycles. Examples of this are the RLL(1, 3) and RLL(1, 4) constraints. The tradeo function for RLL(1, 3) is obtained from congurations induced by the set of simple cycles in G, but for RLL(1, 4), we need to consider concatenations of 2 or 3 simple cycles before we can nd a point above the standard bit-stung bound. In Fig. 6.7, we plot the best known bitstung lower bounds for RLL(1, 7) and RLL(1, 8) for contrast. We conjecture that for RLL(1, k) constraints with odd k > 3, the tradeo function is the piecewise linear curve generated by the (insertion rate, code rate) pairs of periodic congurations that are induced from the set of simple cycles in G. RLL(1, 7) has capacity 0.6793 while RLL(1, 8) has capacity 0.6853. Both have maximum insertion rate 0.375. Bit-stung LB for RLL(1, 7) Bit-stung LB for RLL(1, 8)
0.6 0.4 0.2 0 0 0.2
0.4
Figure 6.7: Best known bit-stung lower bounds for RLL(1, 7) and RLL(1, 8)
Chapter 7 Time-Varying MTR Constraints
In this chapter, we present work on time-varying MTR (TMTR) constrained systems and establish their connections to MTR constrained systems with unconstrained positions. In particular, at the end of the chapter, we present an upper bound for the tradeo function of MTR(j) derived from TMTR constraints. The resulting upper bound is shown to improve upon the straightline upper bound at high insertion rates, particularly for large j. Section 7.1 presents a short introduction to TMTR constraints, parameterized by a nonnegative vector m. This is followed by Section 7.2, where we dene a notion of tightness on the vector m to derive a canonical way of classifying TMTR(m) constraints. Minimal graph presentations of TMTR constraints are discussed in Section 7.3. In Section 7.4, we present new upper bounds on the capacity of TMTR constraints and show that there is a complete linear chain of tight TMTR(m) for vectors m up to length 4 ordered by capacity. Lastly in Section 7.5, we show that for MTR constrained systems with unconstrained positions, the set of sequences restricted to the constrained positions yields a TMTR constraint. We also derive a new upper bound on the tradeo function of MTR constraints. 119
120
CHAPTER 7. TIME-VARYING MTR CONSTRAINTS
7.1
Introduction
TMTR constraints are a generalization of MTR constraints in which the allowable runlengths of 1s are phase-dependent. They have been recently shown by several papers [10, 27, 50, 38] to eliminate certain error events and to provide high coding gain for the E2 PR4 and E3 PR4 channels. An example is given below, followed by the precise denition. Example 59 ( [10], [27] and [50]) The TMTR(2, 3) constraint allows a run of 2 (3) ones to end at odd (even) time periods. Its standard trellis diagram shown in Figure 7.1, with solid edges having label 1 and dotted edges having label 0. The Shannon capacity is 0.9163. This constraint has been used to construct a rate 8/9 block code that achieves 2.2dB coding gain on the E2 PR4 channel in additive-white-Gaussian noise. phase 0 0 1 2 phase 1 0 1 2 3 Figure 7.1: Standard trellis diagram for the period 2 TMTR(j = 2, 3) constraint. phase 0 0 1 2
mk = mi where i {0, . . . , p1} and k i (mod p). From [10, 27, 50], the TMTR(m)
Let m be the nonnegative integer vector (m0 , . . . , mp1 ). For all k Z, put
constraint is dened as the set of sequences such that at time (or position) i, the
maximum allowable runlength of 1s ending at i is mi . For this to be a constrained system (see [47]), all shifts of valid sequences have to be included. Let i denote the cyclic right shift operation on the vector m by i positions. Then TMTR(m) TMTR( i (m)) is a constrained system. p1 i=0
7.2. TIGHT VECTORS FOR TMTR(M)
121
We can also consider the set TMTRs (m) of nite sequences with mi the maximum allowable runlength of 1s starting at time i. Sequences of a xed length in TMTR(m) and TMTRs (m) are related as follows. For a word w of length L, let = wLi1 for wi 0 i L 1. If L is a set of words of length L, let L = { : w L }. Then w there is a permutation m = L mod p ( such that TMTR(m ) = TMTR (m) . m)
L s L
For the rest of this chapter, we will only deal with TMTR(m); dual results hold for TMTRs (m).
7.2
Tight vectors for TMTR(m)
It is possible to have TMTR(m) = TMTR(m ) for m = m . An example is m = (2, 3, 4) and m = (2, 3, 5). To provide a canonical choice of m, we introduce the notion of tightness. Let (m) be the collection of all vectors m of length p such that TMTR(m ) = TMTR(m). Let m be the componentwise minimum of (m), and call m the tight form of m. We show below that TMTR(m ) = TMTR(m), so in studying TMTR constraints, we can restrict to tight forms without loss of generality. Proposition 60 For any m, we have m (m). a word in TMTR(m) and x an arbitrary position i. By denition of m , there is runlength of 1s starting at position i in w is at most mi = m . So w TMTR(m ). i Hence, TMTR(m ) = TMTR(m). some m (m) such that mi = m . Then w is also contained in TMTR(m ), so the i Proof. Clearly, TMTR(m ) TMTR(m). For the reverse containment, let w be
We present a necessary and sucient condition for a vector to be the tight form in the next proposition. Proposition 61 A vector m (m) is the tight form of m if and only if mi+1 mi + 1, for all i {0, . . . , p 1}. (7.1)
122
p 2 3 4
Tight forms (s, s), (s, s + 1) (s, s, s), (s, s + 1, s + 2), (s, s + 1, s + 1), (s, s, s + 1) (s, s, s, s), (s, s, s, s + 1), (s, s, s + 1, s + 1), (s, s + 1, s, s + 1), (s, s, s + 1, s + 2), (s, s + 1, s + 1, s + 1), (s, s + 1, s + 1, s + 2), (s, s + 1, s + 2, s + 2), (s, s + 1, s + 2, s + 3), (s, s + 1, s + 2, s + 1) Table 7.1: Tight forms of m for TMTR for p = 2, 3, 4
Proof. Suppose for contradiction that m > m + 1 for some xed i. Dene m i1 i by mi1 = m + 1 and mk = mk for 0 k p 1, k = i 1. It is easy to check i that TMTR(m ) = TMTR(m ), contradicting tightness of m . For suciency, let m satisfy (7.1). Suppose m is not tight. Then there exists a vector m (m) and some
component i0 such that mi0 < mi0 . Let w = 0i0 1mi0 0, so that at index i0 (mod p), (7.1). However, w TMTR(m ) since mi0 < mi0 , which contradicts m (m). /
the runlength of 1s starting at i0 is exactly mi0 . Then w TMTR(m) by condition
Therefore, m is tight.
The tight forms of m for p = 2, 3, 4 are listed in Table 7.2. Using Proposition 61, we can compute the tight form of a vector m by proceeding iteratively from left to right, replacing mi+1 by mi + 1 if mi+1 > mi + 1. When mp1 is reached, we wrap around to m0 , and continue until a vector has been reached satisfying Eqn. (7.1) for all i. Since no component of m ever increases or becomes negative in any iteration and TMTR(m) remains constant, this algorithm must terminate and output m . Example 62 Let m = (3, 0, 10, 3, 4, 2). Then reduce m to its tight form by: (3, 0, 10, 3, 4, 2) (3, 0, 1, 3, 4, 2) (3, 0, 1, 2, 4, 2) (3, 0, 1, 2, 3, 2) = m .
7.3
Graph Presentations
A graph presentation of TMTR(m) is constructed by concatenating in an appropriate the transitions pass from phase i to phase i + 1. This is known as the standard trellis sense the standard presentations of MTR(mi ) for each i = 0, . . . , p 1, and letting
7.3. GRAPH PRESENTATIONS
123
presentation GT , and has the following description: States: ml + 1 states in l-th phase, for 0 l p 1, {(l, i) : 0 l p 1, 0 i ml }. Here i represents the number of preceding consecutive 1s. Transitions: Beginning at phase l, (l, i) (l, i)
1 0
((l + 1) mod p, i + 1) if i < ml+1 ((l + 1) mod p, 0)
The Shannon cover for TMTR(m) is obtained by applying the state-merging algorithm, which nds and merges equivalent states. If m is not tight, the number of states and transitions in this construction can be decreased by rst tightening m. We say that m has minimal length if there does not exist m of shorter length than m such that TMTR(m) = TMTR(m ). For an equivalence class C in phase l of GT , let x = min{y : (l, y) C}; we choose (l, x) as the representative of C. We
call such points the division points of phase l. For m tight with minimal length, the following properties of GT are obtained below. 1. Any two states with dierent phases are not equivalent. 2. Two states (l, x) and (l, y) in the same phase l (with x < y) are not equivalent if and only if there exists an integer i 1 such that x ml+i i < y. 3. The division points for phase l are given by the set of states (l, x) with x (z)+ = z if z 0 and 0 if z < 0). The equivalence class of a division point (l, y) {(ml+i + 1 i kp)+ : 0 i p 1, 0 k max(m)/p} (where the notation
point or the state (l, ml ).
consists of the states {(l, y), (l, y + 1), . . . , (l, z 1)} where (l, z) is another division
This leads to the following description of the Shannon cover.
124
Proposition 63 Let m be tight with minimal length. Then the states of the Shannon cover of TMTR(m) are the division points of all phases in GT as specied above. Proof. We prove the three properties of GT . (1) Assume (for contradiction) that the two states (l, x) and (l , x ) with dierent phases 0 l < l p 1 are equivalent. Then the follower sets F(l, x) = F(l , x ). So and F(l + 1, 0) = TMTR(ml +1 , ml +2 , . . . , mp1 , m0 , . . . , ml ). So this implies TMTR F(l + 1, 0) = F(l + 1, 0). But F(l + 1, 0) = TMTR(ml+1 , ml+2 , . . . , mp1 , m0 , . . . , ml )
(ml+1 , ml+2 , . . . , mp1 , m0 , . . . , ml ) equals TMTR(ml +1 , ml +2 , . . . , mp1 , m0 , . . . , ml ). The shifted vectors (ml+1 , ml+2 , . . . , ml ) and (ml +1 , ml +2 , . . . , ml ) are tight since m is tight by assumption. Then l+1 (m) = l +1 (m), which implies that k (m) = m m0 mk1 m0 mk1 , so TMTR(m) = TMTR(m0 , . . . , mk1 ), contradicting (2) Assume that (l, x) and (l, y) are not equivalent. Then F(l, y) is a proper subset for k = l l, 1 k p 1. This in turn implies that m0 mp1 m0 mp1 =
the assumption that m was chosen to be of minimal length.
w can be taken to be of the form 1i . Then x + i ml+i (since 1i F(l, x)) and y + i > ml+i (since 1i F(l, y)). This implies that x ml+i i < y. / Conversely, suppose there is an integer i 1 such that x ml+i i < y. Then
of F(l, x). Let w be a word in F(l, x) which is not contained in F(l, y). The word
w = 1i can be generated from state (l, x) but not from state (l, y). This implies that F(l, x) = F(l, y), and so the states (l, x) and (l, y) are not equivalent. (3) For any phase l, we can enumerate the set {ml+i i [0, ml ] : i Z+ } as
(2), (l, x) is equivalent to (l, vi+1 ). Thus, the division points for phase l are given by (l, 0) is always a division point). Note that if m is tight, then ml+i i ml . Writing division points in the claimed form. the set of states (l, x) for x in {ml+i i + 1 [0, ml ] : i Z+ } {0} (since the state
{v1 , v2 , . . . , vk }, with v1 < v2 < < vk . Note that if vi < x vi+1 , then by part
i = i + kp, where 0 i p 1 and 0 k max(m)/p, we can write the set of
Using part (1), it is clear that the division points in each of the p phases are distinct. Thus, they constitute the states of the Shannon cover of TMTR(m) (if merged into those representative classes).
125
7.3.1
Shannon cover when m = (M p + 1, M p + 2, . . . , M )
with equality. This includes the TMTR(2, 3) constraint mentioned in the Introduction. Construct the graph G(M, p) with exactly M + 1 states as follows. States: The states are labeled 0, 1, . . . , M . Transitions: The transitions are given by s s
1 0
Let m = (M p + 1, M p + 2, . . . , M ), i.e., m satises the tightness condition (7.1)
s + 1 for s = 0, 1, . . . , M 1, (s + 1) mod p for s = 0, 1, . . . , M .
Note that G(M, p) is irreducible and deterministic. Moreover, it is reduced since for any state s, the follower set FG (s) contains the word 1M s but not 1M s+1 , so the follower sets of the states in G are distinct. We present a combinatorial proof that G is the Shannon cover of TMTR(m). Proposition 64 The graph G(M, p) is the Shannon cover of TMTR(M p + 1, M p + 2, . . . , M ). Proof. We rst show that every word w that can be generated from G is contained in TMTR(m). Since G is irreducible, WLOG it suces to consider paths starting at state 0. Let s0 , s1 , s2 , . . . be the sequence of states in the path generating w: 0 = s0
w1
s1
w2
s2
w3
(mod p) since if wi+1 = 1, then si+1 = si + 1 and if wi+1 = 0, then si+1 = (si + 1) mod p. Therefore, for all i {0, 1, 2, . . .}, si i (mod p).
By construction, w cannot have a run of more than M 1s. Note that si+1 si + 1
(7.2)
Let ri be the runlength of 1s ending at position i in w (take ri = 0 if wi = 0). Then ri si M , since for any state si in G, and path in G terminating at si
126
such that the label of is a run of 1s, the length of is at most si . namely There is a unique element of {M p + 1, . . . , M } that is congruent to i (mod p), M p + 1 + ((i M 1) mod p) . (7.3)
yields ri M p + 1 + ((i M 1) mod p), which in turn implies that w satises TMTR(m).
Equations 7.2 and 7.3 together imply si M p+1+((i M 1) mod p), which
i such that at position M , the TMTR value is M . It suces to show that any word in T can be generated by a path in G starting at state 0. To do this, let w be a right innite word which cannot be generated by a path in G starting at state 0, and show that w violates T . at 0. Let s0 , . . . , si be the sequence of states for the path w1 . . . wi starting at 0. Let i 0 be minimal such that w1 . . . wi+1 is not the label of a path in G starting Note that wi+1 = 1 and si = M since otherwise the path could be extended further. For the case p = M + 1, it follows immediately that w is not in T since wi+1 = 1 but p < M + 1. Then wi = 1 since the state M does not have an incoming 0 edge. the TMTR value at i + 1 is M p + 1 = 0 (since M = si i (mod p)). So assume So choose j with 0 j < i maximal such that wij , wij+1 , . . . , wi are all 1s. Then
To prove the converse, let T = TMTR( i (M p + 1, . . . , M )), choosing the shift
either sij1 = 0 or sij1 has an incoming 0 edge, so in either case sij1 p1. But then the run of 1s ending at wi has length j+1 = si sij1 M (p1) = M p+1, so the run of 1s ending at wi+1 has length at least M p + 2. But the TMTR value
at i + 1 is M p + 1 since M = si i (mod p). Thus, w is not in T .
7.3.2
Rome Graphs and Characteristic Equations
A labeled variable-length graph (VLG) is a labeled graph in which each edge e has length l(e) Z+ and is labeled by a word L(e) (in some alphabet) of length l(e). A Rome set R of a labeled graph G is a set of states in G such that every cycle in G passes through a state in R. Trivially, every graph has a Rome set, but we are
interested in a Rome set of minimal size. If G has a minimal Rome set with a single
127
element r, then we call G a Rome graph and r a Rome state. Let S be a constrained system presented by an irreducible graph G with a Rome Let denote the set of simple paths of rst return to R in G, namely paths whose sequence of states s0 , . . . , sl satises s0 , sl R, and si R for i = 0, l. Then for each / , with initial state , terminal state and label L(), there is an edge in K cycles in G. set R. Then there is a VLG K presenting S with vertex set R and edge set as follows.
from to with label L(). If |R| = 1, the edge labels of K are the labels of simple
Proposition 65 Let m be a nonnegative vector of minimal length p. Let d denote the minimum number of division points in any phase of the standard trellis presentation of TMTR(m). Then the size of the minimal Rome set for the Shannon cover of TMTR(m) is at most min (p, d). Proof. It is clear that the set of division points in any phase within the trellis presentation constitutes a Rome set. Thus, we can select the smallest set which has size d. Also note that the set containing the top most state in each phase, i.e. {(l, 0) : 0 l p 1}, also forms a Rome set since all cycles in the standard trellis eventually has to return to a state in that set. Thus, the size of the minimal Rome set for the Shannon cover of TMTR(m) is at most min (p, d). A trivial example where the Shannon cover of TMTR(m) is a Rome graph, is when mk = 0 for some 0 k p 1. Then there is only one state in phase k in the trellis presentation GT for TMTR(m), so all paths of length at least p in GT must
pass through (k, 0). Therefore all cycles in the Shannon cover of TMTR(m) contains (k, 0). Note that the Shannon cover is also a Rome graph whenever the size of the minimal Rome set is 1. The edge labels of a VLG can be viewed as distinct symbols (of a new alphabet) of varying time duration. Shannon [63, Theorem 1] proved the following well-known result. Theorem 66 ([63]) Let bs be the duration of the s-th symbol from state i to state ij j. Then the channel capacity is equal to log z, where z is the largest real root of the determinantal equation:
128
z bij ij | = 0,
(7.4)
where ij = 1 if i = j and 0 otherwise. Thus, we can nd the characteristic equation of any constrained system by applying Theorem 66 to its VLG. When the VLG is a Rome graph with period p, and nk is the number of simple cycles of length kp for 1 k q (for some q), Eqn. (7.4) reduces to 1 =
q k=1
nk z kp .
state. Let (q, r) denote the quotient and remainder respectively of M + 1 divided by p. There are 2p 1 simple cycles of length kp for k = 1, . . . , q 1, and 2p 2p1r of G(M, p) is simple cycles of length qp starting at state (p 1). Thus, the characteristic equation
q1
Proposition 67 The graph G(M, p) is a Rome graph and state (p 1) is a Rome
1 = (2p 1)
k=1
z kp + (2p 2p1r )z qp .
(7.5)
and all its incoming and outgoing edges. Let V1 = {0, . . . , p2} and V2 = {p, . . . , M }.
Proof. Consider the subgraph H of G(M, p) obtained by deleting the state p 1
Note that the states in V2 are inaccessible in G from the states in V1 , so there is no
cycle in H containing states from both V1 and V2 . There is also no cycle in H whose states are all in V1 , since if there is an edge from x to y, with x, y V1 , then x < y. a Rome state. Similarly, there is also no cycle in H whose states are all in V2 . Therefore, (p 1) is If M 2p 2, the state (p 1) is the only Rome state since we can nd two cycles
1 1 1 1 0 1 1 1 1 0
(p 1) 0 and (p 1) p (p + 1) (2p 2) (p 1).

0
in G(M, p) whose only common state is (p 1), namely, 0 1 2 To prove the form of the characteristic equation, call the edges in G(M, p) of the
simple cycle in G(M, p) has exactly one long edge. For simplicity, count by assuming 2p1t simple cycles starting with a long edge, where t = (s + 1) mod p. The length that the long edge is used rst. Starting at state s, with s p 1, there are exactly
form s (s + 1) mod p, where (s + 1) mod p = s + 1, long edges. Note that each
129
of all these cycles is 1 + (s t) = s + 1 ((s + 1) mod p) , which is clearly a multiple of p. {kp 1, kp, . . . , (k + 1)p 2}, and there are exactly p such states. If M (k + {kp 1, kp, . . . , (k + 1)p 2} is
(k+1)p2
It is easy to check that the set of s satisfying s + 1 ((s + 1) mod p) = kp is
1)p 2, then the total number of simple cycles of length kp starting at a state in
p1
2
s=kp1
p1(s+1) mod p
=
i=0
2i = 2p 1,
which is independent of k. Clearly, the largest k for which this holds is q 1, where q is the quotient of M + 1 divided by p. The set of s satisfying s + 1 ((s + 1) mod p) = qp is {qp 1, qp, . . . , (q + 1)p 2}.
1)p 2} is given by a partial sum

M
of length qp starting at a state in s + 1 ((s + 1) mod p) = qp is {qp 1, qp, . . . , (q +

p1
Since qp 1 M (q + 1)p 2, this means that the total number of simple cycles
2
s=qp1
p1(s+1) mod p
=
i=p1r
2i = 2p 2p1r ,
where r is the remainder of M + 1 divided by p. Example 68 Let m = (m, m + 1). From Eqn. (7.5), the characteristic equation of TMTR(m, m + 1) can be simplied to
m+2 1 p
z m+2 3
k=1
z m+2kp 2 = 0
m+2 p
for m even,
(7.6)
z m+2 3
z m+2kp = 0
k=0
for m odd.
(7.7)
130
is shown in Fig. 7.2, and has exactly p states according to Proposition 64. Clearly, all states are Rome states. There are exactly 2p1 simple cycles of length exactly p starting at state 0, so the characteristic equation is z p 2p1 = 0, and the largest positive real root is 2(p1)/p .
0 0 1 1 0 1 2 . . . p-2 0 1 p-1
Example 69 Let m = (0, 1, . . . , p 1). The Shannon cover of TMTR(0, 1, . . . , p 1)
(7.8)
Figure 7.2: Shannon cover for TMTR(0, 1, . . . , p 1). Example 70 Let m = (1, 2, . . . , p). The Shannon cover of TMTR(1, 2, . . . , p) is shown in Fig. 7.3, and has exactly p + 1 states by Proposition 64. By inspection, states 1 through (p 1) are all Rome states. There are exactly 2p1 + 2p2 simple cycles of length p starting at state 1, so the characteristic equation is z p+1 (2p1 + 2p2 )z = 0, and the largest positive real root is (2p1 + 2p2 )1/p .
0 1 1 0 1 2 0 1 3 . . . p-2 0 1 p-1 0 1 0 p 0
(7.9)
Figure 7.3: Shannon cover for TMTR(1, 2, . . . , p).
7.4. BOUNDS ON THE CAPACITY OF TMTR CONSTRAINTS
131
For the TMTR(m) constraints with non-Rome Shannon covers, such as, TMTR(2, 2, 3, 3), we can use their standard trellis presentations to compute Eqn. (7.4) by selecting a Rome set R consisting of the p states {(l, 0) : 0 l p 1}. The entry bs in ij at state i and terminating in state j, where i, j R, and whose path label is s. Theorem 66 would then correspond to the length of a path of rst return originating
7.4
Bounds on the Capacity of TMTR Constraints
To simplify notation, let C(m) denote the capacity of the TMTR(m) constraint. We present new upper bounds on C(m) in the next three theorems, whose proofs are given in Appendix B. The rst shows that modifying m by adding 1 to the minimum entry and subtracting 1 from the maximum entry does not decrease capacity. Theorem 71 Let m be a vector with max(m) min(m) + 2. Let i1 < i2 be a pair of positions such that mi1 = min(m), mi2 = max(m), and min(m) < mi < max(m) for i1 < i < i2 (performing a cyclic shift of m if necessary). Let m be obtained from m by putting mi1 = mi1 + 1 and mi2 = mi2 1, and mi = mi otherwise. Then C(m) C(m ). Let m = based on m. Theorem 72 C(m) capMTR (m). For TMTR vectors of the form (M p + 1, M p + 2, . . . , M ), a strict upper bound
p1 i=0
mi /p. Applying Theorem 71 repeatedly to m yields an upper bound
on the capacity can be obtained, which is sharper than the bound of Theorem 72 when m is not an integer. Theorem 73 Let m = (M p + 1, M p + 2, . . . , M ) (with p 2, m = (0, 1)), so
m = M (p 1)/2. Then C(m) < log2 (m), where (m) is the largest positive real for MTR(m) if m Z+ ).
root of the equation z m+2 2z m+1 + 1 = 0 (the latter being the characteristic equation
132
Example 74 Let m = (2, 3, 4, 5), whose capacity C(m) = 0.9518. Theorem 71 gives C(m) C(3, 3, 4, 4) = 0.9615, and Theorem 72 implies C(m) capMTR (4) = 0.9752, while Theorem 73 yields C(m) < 0.9639.
Example 75 Consider the TMTR(2, 3, 4) constraint, whose Shannon cover is given by Proposition 64 and pictured in Fig. 7.4. Its exact capacity is 0.9358. Now, m is exactly 3. By Theorem 73, C(2, 3, 4) < capMTR(3) = 0.9468. 0 0 0 1 0 1 1 2 0 1 4 0 1
Figure 7.4: Shannon cover of TMTR(2, 3, 4).
7.4.1
Complete Linear Ordering of C(m) up to period 4
Using the VLG technique and Theorem 66 from the Section 7.3.2, we can prove the following capacity relations. Proposition 76 For any positive integer t, 1. (a) C(t, t + 1) = C(t, t, t + 1, t + 1) for t 2 (mod 4). (b) C(t, t + 1) > C(t, t, t + 1, t + 1) for t 0 (mod 4). (c) C(t, t + 1) < C(t, t, t + 1, t + 1) for t 1, 3 (mod 4). 2. C(t, t + 1) < C(t, t, t + 1, t + 2) for t 0 (mod 4). 3. C(t, t + 1, t + 2, t + 1) = C(t, t + 1, t + 1, t + 2). 4. C(t, t + 1, t + 2, t + 2) > capMTR (t + 1), except when t + 1 0 (mod 4). 5. capMTR (t + 1) < C(t, t + 1, t + 2, t + 3).
7.4. BOUNDS ON THE CAPACITY OF TMTR CONSTRAINTS
133
6. C(t, t + 1, t + 2, t + 3) < C(t + 1, t + 1, t + 1, t + 2). The proof of Proposition 76 is given in Appendix B. Using Proposition 76 and Theorem 71, we show for tight m of lengths p = 2, 3, and 4 that there is a complete linear ordering of the TMTR(m) constraints by capacity. Let t be a positive integer. For p = 2, C(t, t) < C(t, t + 1) < C(t + 1, t + 1). For p = 3, C(t, t, t) < C(t, t, t + 1) < C(t, t + 1, t + 1) < C(t, t + 1, t + 2) < C(t + 1, t + 1, t + 1), where the last strict inequality is due to Theorem 73. For p = 4, C(t, t, t, t) < <
(b) < (c)
C(t, t, t, t + 1) C(t, t, t + 1, t + 1) C(t, t, t + 1, t + 2) C(t, t + 1, t + 1, t + 1), C(t, t + 1, t + 1, t + 2) (d) C(t, t + 1, t + 2, t + 1) = C(t, t + 1, t + 2, t + 2) (e) C(t + 1, t + 1, t + 1, t + 1) C(t, t + 1, t + 2, t + 3) C(t + 1, t + 1, t + 1, t + 2),
(a)
C(t, t + 1, t, t + 1)
< <
(f ) < (g) <
where steps (a), (b), (d), (e), (f ) and (g) are due to Proposition 76 parts 1, 2, 3, 4, 5, and 6 respectively while step (c) is due to Theorem 71. (The symbol () indicates that the relation between the two quantities could be =, >, or < (> or <),
134
depending on the value of t mod p. In lines (b) and (f ), it should be made clear that the quantity on the right of the inequality is strictly greater than all of the quantities above it.)
7.5
Relations to MTR Constraints with Unconstrained Positions
Let S be a constrained system, and consider a periodic (U, N ) conguration. Let SU,N denote the set of all sequences w S such that wi = for all i mod N U . Let SU,N (k) denote the set of all sequences in SU,N of length q. Let S denote the The code rate R for a given (U, N ) conguration is dened as R = R(SU,N ) = lim sup
q
restriction of SU,N to the constrained positions U c .
log |SU,N (q)| . q
For the rest of this section, let S = MTR(j). View U as a set of points on a circle of N points. Addition of numbers will proceed clockwise (to the right). For a point x on the circle, let n(x) = |U [x (j s(x)), x]| where s is the number of consecutive 1s immediately to the right of x. Note that when both x and x + 1 are in U c , then s = 0, and n(x) = |U [x j, x]| j. When x is in U c and x + 1 in U , then n(x) = |U [x (j s), x]| j s. Let g denote
positions on the circle in (U c mod N ).
the natural 1-1 correspondence from constrained positions indexed (mod N |U |) to
Theorem 77 S is a TMTR(m) constrained system with tight parameters mi = j s(g(i)) n(g(i)) for i = 0, , N |U | 1.
7.5. RELATIONS TO MTR CONSTRAINTS WITH UNCONSTRAINED POSITIONS135
position on the circle, i.e., g(i) = x. We will show that the longest run of 1s (in the constrained positions) ending at x has length j s(x) n(x). containing n(x) + s(x) unconstrained positions and j + 1 (n(x) + s(x)) constrained If all of the j + 1 (n(x) + s(x)) constrained positions in I were lled in with 1s, Let I denote the interval [x (j s(x)), x + s(x)]. Then I is a block of length j + 1
Proof. For a position i mod N |U | for the constraint, let x be the corresponding
positions.
then I would be a block of length j + 1 with only unconstrained positions and 1s, which violates the constraint. So at least one constrained position in I must be a 0. This shows that the longest run of 1s ending at x has length at most j (n(x)+s(x)). On the other hand, if the rst constrained position in I is a 0 and the rest of the constrained positions in I are 1s, then I satises the constraint. Moreover, since the next position after I is constrained (by denition of s(x)), it is possible to have a run of j s(x) n(x) 1s ending at x, by making the position immediately after I a 0. 1}, let x = f (i) and y = f (i + 1). Let Ix denote the interval [x (j s(x)), x + s(x)] To show that m is tight, consider the following. For any i, i+1 {0, . . . , N |U |
and share some overlap Ixy . Let B(I) denote the number of unconstrained positions in an interval I. Then the dierence in the number of unconstrained positions in Ix and Iy \ Ixy , i.e., and Iy is equal to the dierence in the number of unconstrained positions in Ix \ Ixy
and Iy denote the interval [y (j s(y)), x + s(y)]. Both Ix and Iy are of length j + 1
B(Ix ) B(Iy ) = B ([x (j s(x)), y (j s(y)) 1]) B ([x + s(x) + 1, y + s(y)]) |[x (j s(x)), y (j s(y)) 1]| s(y) y x + s(y) s(x) s(y) y x (y x 1) 1, since s(x) = y x 1. This shows that mi+1 mi + 1 for all i {0, . . . , N |U | 1}.
136
Example 78 Let S = MTR(3). Choose insertion rate x = 4/9, period N = 9, and an unconstrained set U = {1, 2, 3, 6}. Then by Theorem 77, S = TMTR(0, 0, 1, 2, 2). Theorem 77 also gives a relation between the rate of the SU,N system and the capacity of the corresponding TMTR constraints. Corollary 79 R(SU,N ) = (1 rem 77.
|U | )capTMTR (m), N
where m is obtained from Theo-
7.5.1
A New Upper Bound for fMTR(j)
We will use the matrices A and B as dened in Eqns. (5.3) and (5.4) respectively. set of unconstrained positions U {0, . . . , N 1} such that = |U |/N , let M = As before in Chapters 5 and 6, given a length N , an insertion rate [0, 1], and a
representing M as a sequence of As and Bs, As represent constrained positions and Bs represent unconstrained positions. If M = 0, each runlength of Bs is at most j since B j+1 = 0.
M0 M1 MN 1 , where Mi = B if i U and Mi = A if i U . Viewing the product /
Dene the variable length graph H = Hj where VH has size 2j1 + 1 and consists of all j-tuples over the alphabet {A, B} ending in B, plus the state Aj . For every labeled Y from a vertex v to one labeled by the sux vY of length j. In particular, there is an edge labeled Aj from every state ending in B to the state Aj and a selfloop labeled A at state Aj . Clearly, H is deterministic. Note that each edge in H yields a denite TMTR vector, since we can determine the TMTR values from its edge label and starting state label using Theorem 77. Thus for any two connected simple cycles 1 , 2 in H with TMTR vectors m1 and m2 respectively, the TMTR vector for the combined cycle 1 2 is m1 m2 . Since any cycle can be decomposed into a collection of simple cycles, this special property of H imposes a linear relationship between TMTR values on a cycle and those on simple cycles, despite there not being any corresponding connection between the spectral radii of the matrix products of the cycle labels. vertex v and each Y = Ar B s with 1 r j and 0 s j, endow H with an edge
The insertion rate of a path in H corresponds to the proportion of Bs in its path label to the length of its path label. Let be the set of simple cycles in H. Let be a cycle in H, which when decomposed into a collection of simple cycles, has nt copies of the t-th simple cycle t , where t = 1, . . . , k, and k is a positive integer ||. Let it = i(t ), ct = c(t ), lt = l(t ), and t = (t ) denote the insertion rate, number of constrained positions, length of cycle label, and mean of the TMTR values of t respectively. The length of the label of is L = constrained positions in is C = Q=
k t=1 k t=1 k t=1
nt lt , and the number of
nt ct . The insertion rate of is I = I() =
1 C/L. Let Q = Q() denote the average of the TMTR values induced by . Then n t c t t /
k t=1
for t = 1, . . . , k. Then Q becomes
nj cj max1tk t . Dene gt = g(t ) = (1 it )t = (ct t )/lt

k t=1
Q= The factor
k t=1
nt lt gt 1 . L 1I
nt lt gt /L is a convex combination of g1 , . . . , gk and is upper bounded
(pointwise) by the convex hull h of {(i(), g()) : }, so Q() h(I()) . 1 I() (7.10)
Since u(a) is increasing with a (see [11, Appendix]), applying u to (7.10) yields u (Q()) u h(I()) 1 I() .
For a R+ , dene u(a) to be log of the spectral radius of z a+2 2z a+1 + 1 = 0.
(7.11)
Taking the supremum of both sides of inequality (7.11) over all cycles in H and combining it with Corollary 79 and Theorem 72 leads to the following new upper bound. Proposition 80 For any insertion rate [0, 1], fMTR(j) () (1 ) u h() 1 .
138
Proof. By Theorem 34, the tradeo function f at insertion rate can be achieved by a periodic (U, N ) conguration, which in turn corresponds to a cycle in H. The cycle induces a sequence of TMTR values which we can express as a vector m. Using Corollary 79, we have f (x) = fSU,N (x) = (1 x)capTMTRe (m) (1 x) sup
{H:I()=x}
u (M ()) ,
(1 x)u
h(x) 1x
where the third line is due to Theorem 72, and the fourth line stems from taking the ceiling on both sides in Eqn. (7.11) before applying the function u and taking the supremum of both sides over all cycles in H with the same insertion rate. Example 81 For MTR(2), H2 consists of 3 states AB, BB, AA, and 4 edge labels {AB, ABB, AAB, AABB}. The graph H2 is shown in Fig. 7.5, and the upper bound is shown in Fig. 7.6 in comparison with the straightline upper bound [56, Prop. 48], the programming approximate upper bound. We present corresponding plots for MTR(3) and MTR(4) in Fig. 7.7 and 7.8 respectively. Note that this new TMTR upper bound is an improvement over the straightline upper bound mainly in the high insertion rate region (which increases as j increases). We can take the pointwise minimum of the two upper bounds to further obtain an improved upper bound. standard bit-stung lower bound (which equals fMTR(2) in this case) and the dynamic
AAB AB AB AAB AB AB, AAB AA A
ABBAABB AA AA
BB ABB AABB
ABB, AABB
Figure 7.5: Graph presentation of H2 for MTR(2).
0.8 0.6 0.4 0.2 0 0 0.2 0.4 0.6
Straightline UB TMTR UB Approx. DPUB, (N = 1000) Bit-stung LB
0.8
Figure 7.6: New TMTR upper bound for MTR(2) in comparison with other bounds.
140
0.8 0.6 0.4 0.2 0 0 0.2 0.4
Bit-stung LB Approx. DPUB DPLB TMTR UB
0.6 0.8
0.8 0.6 0.4 0.2 0 0 0.2 0.4 0.6
Straightline UB TMTR UB Approx. DPUB, N = 1000 Bit-stung LB
0.8
Chapter 8 Future Work

This work has centered around a new framework for analyzing constrained systems with unconstrained positions, and in particular, the characterization of their optimal code rates and achievable rate regions. It is hoped that our results will contribute to a deeper understanding of this scheme and encourage future development of its usage with soft iterative decoding techniques in the storage industry. We close with some interesting topics and questions that stem from our work.
8.1
Open Problems on the Tradeo Function
There is always room for improved bounds and exact solutions of the tradeo function for specic constraints, as we have seen in Chapters 5-7. We list some specic conjectures below. Tradeo functions for RLL(1, k) constrained systems: Empirical results with odd k > 3 could be the piecewise linear curve whose (insertion rate, code rate) corner points are given by periodic congurations induced from the set of simple cycles in G. This remains to be explored. Tradeo functions for MTR(j) constrained systems in low insertion rate regions: As mentioned in Section 5.4, it is conjectured that the tradeo function for MTR(j) for insertion rates < 1/(j + 1) equals the straight line (derived from 141
from Section 6.3.3 suggest that the tradeo function for RLL(1, k) constraints
142
CHAPTER 8. FUTURE WORK
j 1 bit-stung) connecting the points (0, capMTR(j) ) and ( j+1 , j+1 capMTR(j1) ). We
attempted to show this analytically by considering matrix (and hence eigenvalue) inequalities of products of certain nonnegative integer companion matrices which represent periodic (U, N ) congurations. In particular, our line of attack was to prove that certain periodic congurations were better (in terms of code rate) by studying the implicit inuence of gap sizes on the code rates. Even though we resolved the issue for large gaps for any j, we were only able to provide a partial solution for the case of small gaps. Thus, future work would involve extending Lemmas 53 and 54 to any arbitrary j. The core arguments unifying the results on small and large gaps in the proof of Theorem 46 would still apply. We remark that an immediate corollary of this conjecture would be that the to [0, ] would yield an upper bound on fMTR(j) (). This upper bound would be tighter than the straightline upper bound in Section 6.2.1. An improved upper bound on the capacity of TMTR constraints that straight line extension of the tradeo function of fMTR(j) () for [0, 1/(j + 1)]
renes the upper bound on the tradeo function for MTR(j) based on
TMTR constraints: Numerical results indicate that Theorem 73 is true for a larger class of TMTR constraints. We conjecture that for any tight vector m with minimal length p 2 and m 2, capTMTR (m) < log2 , (8.1)
then Eqn. (8.1) reads as capTMTR (m) < capMTR(m) . The latter cannot be proven by concavity arguments such as Jensens inequality. This is because applying Jensens inequality to the expression on the right using the strict concavity of the capacity of MTR constraints gives a quantity smaller than the corresponding TMTR capacity. Even though this conjecture is interesting in its own right as a mathematical problem, we are interested in its functional role in rening the upper bound given in Proposition 80.
where is the largest root of the equation xm+2 2xm+1 + 1 = 0. If m is an integer,
8.1. OPEN PROBLEMS ON THE TRADEOFF FUNCTION
143
There are also several important open questions on the tradeo function not tied to any specic constraint, including: Is there an ecient algorithm that computes the tradeo function from Section 4.3. One of our main contributions in Chapter 4 has been to reduce the problem of nding the tradeo function of a given constraint to that of nding the tradeo function from an irreducible graph over the alphabet A3 . Even though an irreducible constraint may induce a multiple component G presentation, a practical method for nding the tradeo function of the constraint is to derive the tradeo function fi of each component Gi , take the pointwise maximum of the fi s, and then take the least concave majorant. Note that the pointwise maximum of the fi s constitutes a lower bound on the tradeo function of any constraint, while the least concave majorant on the pointwise maximum would constitute an upper bound. Is the tradeo function for any nite-type constraint piece-wise linear? an irreducible graph over the alphabet A3 ?. This problem was raised in
We emphasized the importance of time-sharing throughout this dissertation in achieving high code rates and meeting any ECC rate in the allowable range. For constraints including MTR(1), MTR(2), and RLL(d, 2d + 1), the straight line
segments on the tradeo function are the results of timesharing between extreme points. In all of the lower bounds described in Chapter 6, time-sharing was an integral component. The size of the achievable rate regions for these various bounds often depended on how close to optimal the extreme points were. In our iterative scheme of irregular bit-stung, we sought to rene these extreme points via cycle search in G and thus improve the achievable rate region. This approach did yield signicant improvements for MTR(j, k) and RLL(d, k) constraints, though the improvements became marginal after performing concatenations of up to 5 simple cycles. At the same time, we know from Theorem 34 that there is some periodic (U, N ) conguration that achieves the tradeo function. This raises two questions: (i) Are the periods of the (U, N ) congurations that achieve the tradeo function small? (ii) Are there a nite number of extreme points? An armative answer to
144
the second question would imply the necessity of applying timesharing, and that the tradeo function is piece-wise linear. Tradeo Functions for NRZ representations of MTR and RLL con-
straints. Several of the codes using bit-stung which have been implemented
in practice are designed using the NRZ representation instead of the NRZI representation that we used in this work. An example is the code designed by Cideciyan and Eleftheriou from IBM research that we mentioned in Section 1.4.2. We remark that the same framework of analysis we developed in Chapter 2 would still apply to constraints in NRZ representation. We have, in fact, shown in Section 4.5.1, that the NRZI tradeo function of RLL(0, k) is greater than its NRZ counterpart. However, the general relationship between the NRZ and NRZI tradeo functions for an arbitrary constraint is not known. Performance when combined with LDPC or turbo codes over partial rein Chapter 6 will be useful in code design for practical systems. This is because the extreme points dening the lower bounds are known achievable congurations whose weighted usage in appropriate amounts would achieve any ECC rate and achieve an extrapolated code rate within reasonable limits. Most of the current implementations use extremely simple bit-stung in short block codes that typically yield low rates, as mentioned in Chapter 1. It would be interesting to see the performance of long block length codes designed from the results of our work for magnetic or optical systems, as the complexity of such codes typically lies in the design and not in the implementation (of the encoder or decoder). Once the design is xed (i.e., the set of unconstrained positions, period, etc.), there is exibility for using the scheme in conjunction with any LDPC, turbo or block code, as long as the ECC encoders are of the specied rate. Future work include simulating and evaluating the performances of such codes over the Lorentzian recording channel or some partial response channel in PRML for independent hard decoding of the ECC and the modulation constraint, soft independent decoding of the ECC and the constraint, and joint soft iterative decoding of the ECC and the constraint.
sponse channels. The lower bounds on the tradeo functions that we described
8.2. TRADEOFF FUNCTIONS FOR N -DIMENSIONAL CONSTRAINTS
145
8.2
Tradeo Functions for n-dimensional Constraints
Another interesting future direction is to explore the tradeo functions for n-dimensional constrained systems. As before, let S denote a constrained system. For any n-tuple binary array whose entries are indexed by n-tuples of integers of position integers m = (m1 , . . . , mn ), let be an n-dimensional m1 m2 mn
i = (i1 , . . . , in ) {0, 1, . . . , m1 1} {0, 1, . . . , mn 1}. The n-dimensional capacity of S [43, 40, 37, 52] is dened as CS =
(S,n) (n)
log2 Nm1 ,m2 ,...,mn , m1 ,m2 ,...,mn m1 m2 mn lim
(S,n)
where Nm1 ,m2 ,...,mn is the number of sequences in S on . In general, for an arbitrary constraint S, little is known about its n-dimensional capacity [43]. However, when S = RLL(d, k), the n-dimensional capacity exists, which can be shown using sub-additivity arguments from [37] and [40]. In particular, the n-dimensional capacity that are known in the literature are listed below: 1. In [43], it was observed that for any integer n 1 CRLL(d,d) = 0. 2. In [37, Theorem 1], it was noted that for every n 2, d 1 and k > d CRLL(d,d+1) = 0. 3. In [37, Theorem 2], it was shown that for every n 2, d 1 and k > d
n (n) (n)
CS is decreasing in n for a xed d and k, i.e., CS CS . Some examples of
(n)
(1)
(2)
lim CRLL(d,k) = 0 k 2d.
(n)
146
4. It is proven in [52] that

n
lim CRLL(d,) =
(n)
1 . d+1
My collaborator P. Chaichanavong pointed out that it seemed more than a coincidence that the n-dimensional capacities of these constraints equaled their maximum insertion rates, which we had explicitly described by Theorem 18. {0, 1, . . . , q1}n and limq |Iq |/q = . We dene the n-dimensional tradeo function log M (q, Iq ) , q For 0 1, dene I() to be the set of all sequences (Iq ) where Iq
for S at insertion rate to be

(n)
fS () =
sup lim sup

(Iq )I() q
positions by 0 or 1 independently yields an array which satises the n-dimensional constraint of S. Dene the n-dimensional maximum insertion rate to be S = sup .
n fS ()>0
alphabet A3 , with
where M (q, Iq ) is now the number of n-dimensional arrays in {0, . . . , q 1}n over the positions indexed by Iq , such that any replacement of the
(n)
The proof of the following proposition showing that the maximum insertion rate is invariant across dimensions is due to joint eorts with P. Chaichanavong and J. Gill. Theorem 82 For any n 1, S = S . Proof. We drop the S subscripts in for notational clarity in this proof. First we show that (n) . Let w be the label of a max-insertion-rate cycle in G and let from w as follows. N = |w|. We construct an n-dimensional array over the alphabet A3 of size N n
(n)
[i1 , . . . , in ]
w((i1 + + in )
mod p),
is a cyclic-shift of w. Hence can be freely concatenated without violating the
for all (i1 , . . . , in ) {0, . . . , N 1}n . It is clear that each one-dimensional row of
8.2. TRADEOFF FUNCTIONS FOR N -DIMENSIONAL CONSTRAINTS
147
constraint and the number of
rows of length m1 . By the proof of Theorem 14, the number of most m1 + |VG |. Thus the ratio of ratio of
that satises the constraint. This array consists of m2 mn one-dimensional in each row is at in is at most + |VG |/m1 . We conclude that
Next we show that (n) . Let be an n-dimensional array of size m1 mn
in is N n . It follows that (n) .
for any > 0, there is no suciently large pattern satisfying the constraint and the The following questions are avenues for future investigation: Is true for S = RLL(d, k) that
n
is at least + . Hence (n) .
lim CS = S ?
(n)
If so, is it true for an arbitrary constraint S? Presumably, this requires proving that CS fS + S , for any n 1, and therefore requires examining the properties of the n-dimensional tradeo functions fS .
(n) (n) (n) (n)
Will fS
with n 2 still be concave and continuous for nite-type constraints? How do we dene a hypergraph corresponding to G in n dimensions?
In particular, how do we extend the scheme of constrained systems with uncon-
strained positions to 2 dimensions? This will be particularly useful for optical storage systems. Will the 2-dimensional bit-stung schemes given in [28] achieve the tradeo function for some 2-dimensional constraints?
Appendix A Long Proofs for Chapter 5

(j + 1) matrices dened by Eqns. (5.3) and (5.4). Let C = BAj . We consider only the MTR(j) constraint in this Appendix. Let A and B be (j + 1) Let M denote a periodic (U, N ) conguration, where M = M0 M1 . . . MN 1 where permutation of M results in the same spectral radius. Mi = B if i U and Mi = A otherwise. The following lemma shows that any cyclic
Lemma 83 Let U and V be n n matrices. Then U V and V U have the same characteristic polynomial, and hence the same eigenvalues (with multiplicity). Proof. Note that for any n 1, Tr ((U V )n ) = Tr (U (V U U V )) which in turn
equals Tr ((V U U V )U ) = Tr ((V U )n ).
the characteristic polynomial of U V . Note that f (x) = xn e1 xn1 + e2 xn2 + (1)n en , where ei =
k1 <<ki
Now let 1 , . . . , n be the eigenvalues of U V and f (x) = (x 1 ) (x n ) be k1 ki is the ith elementary symmetric function
in 1 , . . . , n . Similarly, the characteristic polynomial of V U is determined by the elementary symmetric functions in the eigenvalues of V U . Note that Tr((U V )i ) =
n k=1
i is the ith power sum symmetric function in k
1 , . . . , n , which we denote by pi . The elementary symmetric functions e1 , . . . , en are determined by the power sum symmetric functions p1 , . . . , pn (or vice versa) by and ei = 0 for i > n. These equations can be solved recursively for the ei in terms 148 Newtons Formulas: for each i {1, . . . , n},
i k1 pk eik k=1 (1)
= iei (taking e0 = 1
A.1. GENERALIZED FIBONACCI RELATIONS FOR MTR CODES
149
of the pi . So the characteristic polynomial of U V is determined by the values of Tr((U V )i ), and similarly for V U . Then, since Tr((U V )i ) = Tr((V U )i ), the characteristic polynomials (and hence the eigenvalues) of U V and V U are identical. The above lemma is an integral tool in the manipulation of periodic congurations to attain optimal rates. The following section includes several results on inequalities of products of As and Bs, which are generalizations of the relations given in the Appendix of [11]. For notation, let AT denote the transpose of A.
A.1
Generalized Fibonacci Relations for MTR codes
Write A = A(j) , B = B(j) and C = C(j) for MTR(j). For each row r of A(j) , denote the i-th entry by fr,i , where the rst subscript denotes the row number (from the top) and the second subscript denotes the time instant. Thus, the element in the rst column at row r is represented by fr,1 .
f2,1 f2,0 f2,1 . = . .. . . . . . fj,0 fj,1 fj,1 fj+1,1 fj+1,0 fj+1,1 f1,1 f1,0 f1,1 .. . .. fl,1j f2,1j 1 1 . = . . . . . 1 1 1 0 . 0 1 .. . .. .. . . . . 0 0 0 0 0 0
A(j)
. . . . fj,1j fj+1,1j
.. . . 1 0
The matrix A also corresponds to the transfer function of a shift register circuit whose operation on each row corresponds to summing up the entries in each column. In other words, raising A to some power k results in the recursive generation of a (j + 1)-step Fibonacci sequence (one for each row) using fr,1 , fr,0 , , fr,(j1) as the (j + 1) initial conditions, i.e.,
f2,k = . . . fj+1,k f1,k f1,k1 f2,k1 . . . fj+1,k1 fl,k(j1) f2,k(j1) . . . fj+1,k(j1) fl,kj f2,kj . . . fj+1,kj ,
Ak (j)
.. .
k1
where fr,k is given by fr,k =

i=kj1
fr,i .
(A.1)
150
APPENDIX A. LONG PROOFS FOR CHAPTER 5
m the top right hand corner of the product C(j) B(j) , i.e.,
It is easily checked that Amj for any integer m 0 is a j j submatrix nested in (j1)
m C(j) B =
0 Amj (j1) 0 0
(A.2)
In the next lemma, we prove some generalized Fibonacci sum equalities. Lemma 84 Let fr,t denote the (r, t) entry of At (j1) . Then for rows r = 1, . . . , j and any integer t 1, the following equalities hold: fr,t+1 = 2fr,t fr,tj . fr+1,t+1 = f1,t + fr,t ,
t1
(A.3) (A.4) (A.5)
fr,t =
k=tj+r1
f1,k ,
Proof. To obtain Eqn. (A.3), rst use Eqn. (A.1) on fr,t+1 . Thus
t t1
fr,t+1 =
k=t+1j
fr,k = fr,t +
k=t1j
fr,k fr,tj = 2fr,t fr,tj .
Eqn. (A.4) is the direct consequence of pre-multiplying [f1,t , f2,t , . . . , fj,t ]T by A. This is discussed in detail in Section A.1.1. Eqn. (A.4). The base case when r = j is true fj,t = f1,t1 , by the fact that the recursion in the last row is a delayed version (by one time step) of the recursion in the rst row due to the initial conditions. Now assume Eqn. (A.5) is true for r = n, fn1,t from Eqn. (A.1), we have fn1,t = Eqn. (A.4) into
t1 k=tj+(n1)1 t2 k=tj1
We prove Eqn. (A.5) by induction on the rows r = j, j 1, . . . , 2 assuming
and show that it is true for r = n 1. Starting with the recursive denition of
t1 k=tj
fn1,k , which can be expanded using which equals
(f1,k + fn,k ) = f1,t1 +
inductive hypothesis for n, we obtain fn1,t =
t2 k=tj1 fn,k . Then, f1,t1 + t2 k=tj+n2 f1,k ,
applying the
f1,k , thus proving the hypothesis.
A.1. GENERALIZED FIBONACCI RELATIONS FOR MTR CODES
151
A.1.1
Pre and Post-Multiplication by A
For pre-multiplication of M by A, we make use of the row vector representation of M = [r1 , r2 , , rn ]T . Thus, AM = [r1 + r2 , r1 + r3 , , r1 + rn1 , r1 ]T . 2g1 r1 + 2g2 r2 + + 2rg1 + rg + rg+1 It is sometimes useful to consider pre-multiplication by Ag for g j, namely, 2g1 r1 + 2g2 r2 + + 2rg1 + rg + rg+2 . . . g1 g2 2 r1 + 2 r2 + + 2rg1 + rg + rj+1 g g1 g2 , A M = 2 r1 + 2 r2 + + 2rg1 + rg g1 g2 (2 1) r1 + 2 r2 + + 2rg1 + rg . . . (2g1 2g3 ) r + (2g2 2g4 ) r + + 2r 1 2 g1 + rg 2g2 r1 + 2g3 r2 + + 2rg2 + rg1 + rg Then M A yields M A = [cn+1 , c1 , c2 , , cn1 ] where cn+1 =
n i=1
(A.6)
For post-multiplication, express M in terms of column vectors, i.e., M = [c1 c2 cn ]. ci .
Lemma 85 Suppose M contains a row, r = [a1 , a2 , a3 , , aj , aj+1 ], where ai d=

j i=2
0 for all i = 1, , j + 1, and in particular aj+1 > 0. Let d denote the row sum, i.e., ai + a1 + aj+1 . If 2d aj+1 , then rAj 0.
and post-multiplying r by A2 yields rA2 = d(2) , d, a1 , a2 , , aj1 . Note that the rst entry d(2) = d+a1
j1 i=2 j i=2 ai (3)
Proof. Suppose 2d aj+1 , then the product rA becomes rA = [d, a1 , a2 , , aj ], = 2daj+1 0 by our assumption.
d(3) = d(2) + d + a1
Post-multiplying r by A3 yields rA3 = d , d(2) , d, a1 , a2 , , aj2 , where ai d(2) + d(2) 0.
Clearly, further post-multiplication by A will result in positive entries. For ink1 i=2
stance, the rst entry of rAk for 3 k j becomes d(k) =

jk+1 i=2
d(i) + d + a1 +
For k j, rA
ai
k1 i=2 k
d(i) + d(2) 0. Thus rAk
0 for k = j.
0 since rAk is already nonnegative.
152
A.1.2
Pre and Post-multiplying by B
Let M denote an arbitrary nonnegative n n matrix with integer entries, say M = [rT , rT , , rT ]T , where ri denotes the i-th row of M . Pre-multiplying M by B yields 1 2 n row at the bottom. an upward row shift: BM = [rT , rT , , rT , 0T ]T . Note the addition of the all 0s 2 3 n1 To see the eect of post-multiplication by B, express M in terms of column vectors, M being cyclically shifted to the right, and the zeros column vector being shifted in from the left: M B = [0, c1 , , cn1 ]. i.e, M = [c1 , c2 , , cn ]. Post-multiplying M by B results in the column vectors of
A.2
Proofs of some Matrix Inequalities
In the following proofs, fr,t denotes the (r, t) entry of At (j1) , and corresponds to the t-th j-step Fibonacci number for row r based on MTR(j 1). Proof of Lemma 50. showing Ajg BAg (AC m B C m BA)Aj+1 For 0 g j 1, we will rst prove inequality (5.6) by 0. Inequality (5.5) then follows when
m = 0. From Eqn. (A.2), the matrix product C m BA is obtained as

f1,mj+1 f2,mj+1 . . . f j1,mj+1 fj,mj+1 0 0 0 f1,mj f2,mj f1,mj1 f2,mj1 . .. f1,mjj+1 f2,mjj+1 . . . , fj1,mjj+1 fj,mjj+1 0
. . . . . . . . . 0 fj1,mj fj1,mj1 0 fj,mj fj,mj1 0 0 0
while AC m B is
0 f1,mj + f2,mj f1,mj1 + f2,mj1 0 f 1,mj + f3,mj f1,mj1 + f3,mj1 . . . . 0 . . 0 f1,mj + fj,mj f1,mj1 + fj,mj1 f1,mj f1,mj1 0 0 f1,mj f1,mj1
f1,mjj + f2,mjj f1,mjj + f3,mjj . .. . . . . f1,mjj + fj,mjj f1,mjj f1,mjj
A.2. PROOFS OF SOME MATRIX INEQUALITIES
153
Using Eqn. (A.4) to simplify the terms, the dierence AC m B C m BA becomes

f1,mj+1 f1,mj+1 0 f f2,mj+1 0 2,mj+1 . . . . . . . . . fj1,mj+1 fj1,mj+1 0 fj,mj+1 0 fj,mj+1 0 fj,mj+1 fj,mj .. 0 0 .
. . . . 0 0 fj,mjj+1
(A.7)
It is easy to check that the product BAg (AC m B C m BA) Aj+1 yields
. . . . . . fjg,mj+1+g 0 g fj,mj+j+2 f1,mj+i fj,mj+j+1 i=1 fjg+1,mj+1+g 0 . . . . . . fj,mj+1+g 0 0 0 f2,mj+1+g f3,mj+1+g 0 0 . . .. .. . . . 0 fj,mj+2 , 0 . . . 0 0 0 0
which we denote by M . Only the rst column in M contains negative entries. Any further pre-multiplication by Ajg performs only row operations so the last j columns of M will remain nonnegative. Thus, it suces to check that the rst column of Ajg M is nonnegative.
T Let vg denote the rst column of M , and let vg denote its transpose. Then T vj1 = j1
fj,(m+1)j+2
i=1
f1,mj+i , f2,(m+1)j , . . . , fj,(m+1)j , 0 , which can be rej1
duced to f1,(m+1)j , f2,(m+1)j , , fj,(m+1)j , 0 since the term fj,(m+1)j+2

j j1
f1,mj+i
i=1
equals
i=1
f1,mj+i
f1,mj+i = f1,(m+1)j .
i=1
Working through the recursion, it is straightforward to check that the following
154
relation holds for any 0 g < j:

Avg = vg+1 + (f1,mj+g+1 f2,mj+g+1 ) 1, . . . , 1, f2,mj+g+1 f1,mj+g+1 f2,mj+g+1
T
In the worst case, g = 0, so pre-multiplication by Aj is necessary:

Aj v0 = Aj1 v1 + Aj1 (f1,mj+1 f2,mj+1 ) 1, . . . , 1, = Aj2 v2 + Aj2 (f1,mj+2 f2,mj+2 ) 1, . . . , 1, + Aj2 (f1,mj+1 f2,mj+1 ) 2, , 2, 1 = f2,mj+j1 ,1 f1,mj+j1 f2,mj+j1 f2,mj+1 f1,mj+1 f2,mj+1 f2,mj+2 f1,mj+2 f2,mj+2
T T
f2,mj+1 ,1 f1,mj+1 f2,mj+1
= vj + (f1,mj+j1 f2,mj+j1 ) 2, , 2, 1 + (f1,mj+j2 f2,mj+j2 ) 4, , 4, 3 +
f2,mj+j2 , 3, 2 f1,mj+j2 f2,mj+j2
+ (f1,mj+1 f2,mj+1 ) 2j1 , (2j1 1)

j1
f2,mj+1 , 2j1 2, , 2j 2j1 f1,mj+1 f2,mj+1 ei+1
vj +
i=1
(f1,mj+i f2,mj+i ) 2ji 1

j1 i l=1
f2,mj+i f1,mj+i f2,mj+i
= f1,mj e1 +
i=1
f1,mj+l + 2ji (f1,mj+i f2,mj+i ) ei+1 + f1,(m+1)j ej+1
0,
where ei is the standard basis vector with a 1 in the i-th coordinate. We will rst prove Lemma 56 before Lemmas 53 and 54 as Lemma 53 uses the same matrices as Lemma 56, while Lemma 54 depends on Lemma 53 Proof of Lemma 56. dierence of BAj C m BAj+g and BAg C m BA2j as BAj (C m BAd Ad C m B)A2j . We 0. Assume j 7. For g = j + d, where d 1, we write the
will show that BAj (C m BAd Ad C m B)A2j
155
For the case 1 d j, the matrix product C m BAd is f1,mj+d f1,mj+d1 f 2,mj+d f1,mj+d1 . . . . . . fj1,mj+d fj1,mj+d1 fj,mj+d fj,mj+d1 0 0 f1,mj+1 f2,mj+1 . . . fj1,mj+1 fj,mj+1 0 ... 0 0 . . . 0 0 0 f1,mj f1,mj . . . fj1,mj fj,mj 0 f1,mjj+d+1 f2,mjj+d+1 . . . , fj1,mjj+d+1 fj,mjj+d+1 0 ...
(A.8)
and the matrix product Ad C m B is 0 0 0 0 0 0 0 0 0 0 f1,mj+d f2,mj+d . . . fjd,mj+d fj+1d,mj+d fj+1d,mj+d fj+2d,mj+d . . . fj1,mj+d fj,mj+d f1,mj+d1 f2,mj+d1 . . . fjd,mj+d1 fj+1d,mj+d1 fj+1d,mj+d1 fj+2d,mj+d1 . . . fj1,mj+d1 fj,mj+d1 ... ... f1,mj+dj+1 f2,mj+dj+1 . . . fjd,mj+dj+1 fj+1d,mj+dj+1 . fj+1d,mj+dj+1 fj+2d,mj+dj+1 . . . fj1,mj+dj+1 fj,mj+dj+1
(A.9)
Let D = Ad C m B C m BAd for 0 < d j and D is of the form D= T 0 , Y Z where T is a (j d + 1) (d + 1) matrix given by f2,mj+d f2,mj+d1 f2,mj+d f2,mj+1 f2,mj+2 f2,mj+1 . . . . ... . . . . . . . . f fjd,mj+d1 fg1 ,mj+d fjd,mj+1 fjd,mj+2 fjd,mj+1 jd,mj+d fjd+1,mj+d fjd+1,mj+d1 fjd+1,mj+d fjd+1,mj+1 fjd+1,mj+2 fjd+1,mj+1 f1,mj+d f1,mj+d1 f1,mj+d f1,mj+1 f1,mj+2 f1,mj+1 ,
156
0 is a (j d + 1) (j d) zero matrix, Y is a d (d + 1) matrix given as fjd+3,mj+d f1,mj+d1 . . . . Y = . . f f1,mj+d1 j,mj+d 0 f1,mj+d1 and Z is a d (j d) matrix given as f1,mjd1 . . Z= . f 1,mj2 f1,mj1 f1,mjd f1,(m1)j+1 . fjd+2,mj+d f1,mj+d1 f1,mj+1 fjd+1,mj+1 ,
f1,mj+1
f1,mj+1
f1,mj+1 fjd+2,mj+1 . . . fj1,mj+1 fj,mj+1
f1,(m1)j+2 . . .
f1,(m1)j+d1 f1,(m1)j+d
Pre-multiplying D by BAj performs row operations on each column of D. Let M = BAj D. Denote the i-th column of M as M (:, i). Then the rst column is M (:, 1)T = f1,(m+1)j+d , f2,(m+1)j+d , . . . , fj,(m+1)j+d , 0 . We use Eqn. (A.1) and Lemma 84 to reduce the expressions obtain for the subsequent columns of M . For columns i = 2, . . . , d, let t = mj + d i + 2. Then f1,t1 2d1 f1,t+jd f1,t 2d1 f1,t+jd . . . f1,t+jd1 2d1 f1,t+jd . M (:, i) = (2d1 1)f1,t+jd d1 (2 2)f1,t+jd . . . 2d2 f1,t+jd 0
157
For d 3, the (d + 1)-th column M (:, d + 1) is 2 f1,(m+1)jd+1 f2,(m+1)j 2d1 f1,(m+1)jd+1 f3,(m+1)j . . . d1 2 f1,(m+1)jd+1 fjd+2,(m+1)j d1 2d1 f1,(m+1)jd+1 i=2 f1,(m+1)jd+i M (:, d + 1) = d1 (2d1 2)f1,(m+1)jd+1 i=2 f1,(m+1)jd+i . . . d1 (2d1 2d3 )f1,(m+1)jd+1 i=2 f1,(m+1)jd+i 2d2 f1,(m+1)jd+1 f1,(m+1)j1 0 For columns i = d + 2, . . . , j + 1, let t = mj + d i + d + 2. Then 2 f1,t+d2l . . . d2 l l=0 2 f1,t+d2l d3 d2 M (:, i) = (2 1)f1,t l=0 2l f1,t+d3l . . . d3 l=0 2l f1,t+d3l f1,t+d2 0
d2 l=0 l
d1
In the case where d = 1, it reduces to the negative of Eqn. (A.7). Pre-multiplication by BAj and post-multiplication by Aj yields 0 0 . . j m m j BA (C BA AC B) A = . 0 0 0 0 0 . . . 0 0 0 ... f1,(m+1)j+1 f2,(m+1)j+1 . . . , fj1,(m+1)j+1 fj,(m+1)j+1 0
158
which is a non-negative matrix. When d = 2, the matrix product BAj (C m BA2 A2 C m B) Aj is of the form:
j1
f1,mj+i i=0 j1 f1,mj+i i=1 . . . f1,(m+1)j1 0 0
f1,(m+1)j1 f1,mj+1 f1,(m+1)j+d f1,(m+1)j1 f1,mj+1 f2,(m+1)j+d . . . . ... . . . . . . f1,(m+1)j1 f1,mj+1 fj1,(m+1)j+d f1,(m+1)j1 f1,mj+1 fj,(m+1)j+d 0 0 0
Clearly, this satises the conditions of Lemma (85), so further post-multiplication by Aj results in a nonnegative matrix. Now for notational simplicity, let ai = f1,(m+1)ji+1 for i = 2, , j, and let bl
(d)
fl,(m+1)j+d for l = 1, , j.
in terms of ai s and bi s.
j1
For d = 3, it is straightforward to express the matrix BAj C m BAd Ad C m B Aj

j
ai ai a2 + aj + 2 i=3 i=2 j1 j2 aj1 + 2 ai ai a2 + i=3 i=2 . . . . . . 2a2 + a3 a2 + a3 a2 a2 0 a2 0 0
a2 2a3 aj1 2aj b1 (3) a2 2a3 aj1 2aj b2 . . . ... . . . . . . . (3) a2 2a3 aj1 2aj bj2 (3) a2 2a3 aj1 2aj bj1 (3) a2 a3 aj1 aj bj 0 0 0
(3)
For 4 d j, it is straightforward but tedious to show (by taking the row sums
159
of M ) that X = BAj C m BAd Ad C m B Aj is given by
j2 i
2(d3l)
aji
j3
i=0 l=0
2d3 a2 + 2
d3
j3 i=0 j3
2(d4i) 2
aji aji
i 1 i=0 l=0
2d4+l fj,nji 2 . . . 2d4+l a2+i 2d4+l a2+i

d4+l
d2 di2
..
2(di3l) 2(di3l) . . .
(d3l)+
aj1i
i=0 l=0
a2 +
(d4i)+
i 1
i=0
ajl ajl
l=0 +
b1 b2
(d)
d2 di2
ai
(d)
i=1
i=0 l=0
i=0
l=0
i 1 i=0 l=0
. . . 2(d3l) 2d3 a2 0 0
+
. . . a3i 2d3 a2 + 2d4 a3 2d3 a2 2d3 a2 0
i 1 i=0 l=0 i 1
2 di2 2(di3l)+ a i=0 1 di2 2(di3l)+ a d2 l=0
. . . bj2 bj1 bj
(d) (d) (d)
jl
i=0 l=0
i=0
jl
2d3 a3 2d4 a2 0
l=0
2(d3i) 0
aji
i=0
where (x)+ = max{x, 0}. 1, , j + 1, and c denotes columns from 1, , j + 1), of X have the following
j+1r
It can be shown that for 2 d j, the entries xr,c (where r denotes rows from
properties:
1. For rows r = 1, , j, we have xr,1 = 2. Also, for rows r = 1, , j, xr,1 =
xr,i .
i=2
j+1r
xj,i .
i=2
3. The rst column has monotonically decreasing entries, i.e., x1,1 x2,1 xj+1,1 = 0.
4. The last column has monotonically decreasing entries since fi,nj+d fi+1,nj+d for any i = 1, , j.
j3 i
5. The (1, 1) entry is less than the (j, j + 1) entry, i.e., x1,1 = fj,nj+d = xj,j+1 .
i=0 l=0
2d3l fj,nji+l
Denote the row sum vector of M Aj by v and denote its last column by c. It can be checked for each j = 3, . . . , 7 that 2v bd r +
j c=jr+2
cj+1 , by evaluating for each row r = 1, . . . , j, 0.
xr,c 0. Then applying Lemma (85) yields the result that XAj
160
Applying the Caley-Hamilton Theorem, we have for d = j + 1, BAj C m BAj+1 Aj+1 C m B A2j
= =
BAj (C m B C m B) A2j + BAj (C m BA AC m B) A2j + 0, + BAj C m BAj Aj C m B A2j
BAj C m B I + A + + Aj I + A + + Aj C m B A2j
since we have shown above that each term is nonnegative. Suppose BAj C m BAd Ad C m B A2j 0 for some d = n > j + 1, then for
d = n + 1, we have
= =
BAj C m BAnj+1 Anj+1 C m B A2j + BAj C m BAnj+2 Anj+2 C m B A2j + 0, + BAj (C m BAn An C m B) A2j
BAj C m B Anj+1 + + An1 + An Anj+1 + + An1 + An C m B A2j
BAj C m BAn+1 An+1 C m B A2j
since each term is nonnegative by the inductive hypothesis. Proof of Lemma 53. From Eqns. (A.8) and (A.9), and substituting d = j g1 ,
we can write down the forms of C m BAjg1 and Ajg1 C m B.
For 0 < g1 < j, the matrix D = Ajg1 C m B C m BAjg1 is of the same form given in Eqn. (A.2), except that the signs are reversed and d = j g1 . If g1 = 0, it is straightforward to show BAg1 DAg1 is of the form
f3,(m+1)j f1,(m+1)j1 . . . . . . fj,(m+1)j f1,(m+1)j1 0 f1,(m+1)j1 0 0 f2,(m+1)j f1,(m+1)j1 f1,mj+1 f1,mj+1 f2,mj+1 . . .. . . . . . , f1,mj+1 fj1,mj+1 fj,mj+1 f1,mj+1 0 0 f1,mj+1
161
For 1 g1 j 1, Eqns. (A.6) and A.6 can be used to express the form of BAg1 DAg1 as [G|K], where G = [G |0]T is a (j + 1) g1 matrix, with G given as
f 1,mj+g1 +1 . .. . G = . . f1,(m+1)j2 f1,(m+1)j1 f1,mj+g1 f1,mj+1 f1,mj+2 . . . f1,(m+1)jg1 1 f1,(m+1)jg1 ,
G = [vg1 , vg1 1 , , v2 , v1 ].
f2,(m+1)j f3,(m+1)j . . . fjg ,(m+1)j 1 fjg +1,(m+1)j 1 fjg +1,(m+1)j 1 . . . fj1,(m+1)j fj,(m+1)j 0
and 0 is of size (g1 + 1) g1 . We can also write G in terms of column vectors The matrix K of size (j + 1) (j + 1 g1 ) is given by
f1,(m+1)j1 f1,(m+1)j1 . . . f1,(m+1)j1 f1,(m+1)j1
fjg +1,(m+1)j 1 fjg +1,(m+1)j1 1
.. .
f1,mj+g1 +1 f1,mj+g1 +1 . . . f1,mj+g1 +1

fjg +1,mj+g +2 1 1 fjg +1,mj+g +1 1 1
f1,mj+g1 +1 f2,mj+g1 +1 . . . fjg1 1,mj+g1 +1 fjg1 ,mj+g1 +1 fjg1 +1,mj+g1 +1 . . . fj1,mj+g1 +1 fj,mj+g1 +1 0
.. .
f1,mj+g1 +1
. . . fj1,(m+1)j fj1,(m+1)j1 fj,(m+1)j fj,(m+1)j1 0
. . . fj1,mj+g1 +2 fj1,mj+g1 +1 fj,mj+g1 +2 fj,mj+g1 +1 0
which we can condense in column vector form as K = v0 , v1 , , v(jg1 1) , v(jg1 ) . H is H = [vj+1 , vj , , vg1 +2 , vg1 +1 ], with the column vectors vi given below. Now let BAg1 DAj+1 = [H|G], with G as above. The (j + 1) (j g1 + 1) matrix
It is easily checked that the rst column of BAg1 DAg1 +1 can be expressed as follows using Eqns. (A.4) and (A.5):
T vg1 +1 =
f1,mj+g1 +1 , f1,mj+g1 +2 , , f1,(m+1)j , 0, , 0
0,
where the number of 0s is of length 1 if g1 = 0 and length g1 + 1 if 1 g1 j 1. Note that the non-zero entries of vg1 +1 are increasing with each row with vg1 +1 (r) 2vg1 +1 (r 1) for rows r = 1, . . . , g1 . The rst column of BAg1 DAg1 +2 is vg1 +2 = 2vg1 +1 v(jg1 ) . In fact, for g1 + 2 t j + 1, the rst column of BAg1 DAt
162
obeys the following recursion.

vt = 2vt1 v(jt+2) . (A.10)
Backtracking through the recursion, the explicit expression for vj+1 is

2jg1 f1,mj+g +2 2jg1 1 f2,mj+g +1 1 1 . . . jg 2 1 f1,(m+1)j 2jg1 1 fjg1 ,mj+g1 +1 jg1 1 2jg1 1t f1,mj+g1 +t t=1 = jg1 1 2jg1 1t fjg1 +1,mj+g1 +t fjg1 +1,(m+1)j t=1 jg1 1 2jg1 1t fjg1 +2,mj+g1 +t fjg1 +2,(m+1)j t=1 . . . jg1 1 jg1 1t 2 fj,mj+g1 +t fj,(m+1)j t=1 0 2jg1 f1,mj+g1 +1 2jg1 1 f1,mj+g1 +1
jg1 1 jg1 1t 2 f1,mj+g1 +t t=1 jg1 1 jg1 1t 2 f1,mj+g1 +t t=1
vj+1
To shorten notation, we write v to mean vj+1 as we will be dealing only with vj+1 in the following set of claims. We claim the following: 1. The entry v(1) > 0 for g1 j 3, and v(1) < 0 for 0 g1 j 4. 2. Let 1 r0 j g0 1. If v(r0 ) > 0, then for any rows r = r0 + 1, . . . , j g1 , v(r) > v(r 1). 3. For all j 1, the entry (a) v(j g1 2) 0 for 0 g1 j 3, (b) v(j g1 1) 0 for 0 g1 j 2, (c) v(j g1 ) 0 for 0 g1 j 1, 4. For j 7 and g1 j 2, the entry v(2) > 0. 5. If v(1) > 0, then v(j g1 ) is the largest positive entry, and v(j g1 + 1) is the negative entry with the largest magnitude. We say v is of form (I) when the rst
163
(j g1 ) entries are positive, the next g1 entries are negative and the (j + 1)-st entry is 0. If v(1) < 0, then v(j g1 ) is still the largest positive entry, but the negative entry with the largest magnitude is max{v(1), v(j g1 + 1)}. We say v is of form (II), by g1 negative entries and a 0 in the (j + 1)-st entry. when the rst entry is negative, the next (j g1 1) entries are positive, followed
6. Suppose v(1) > 0, so v is of form (I). We dene a positive entry v(k), 1 k jg1 to be good if v(k) |v(j g1 + 1)|. (A.11)
Thus, if v(k) were added to every entry of v, a nonnegative vector is obtained. Assuming j 7 and v is of form (II), so v(1) < 0. For 2 k j g1 , we dene
a positive entry v(k) to be good if
v(k) |v(j g1 + 1)| + 2|v(1)|. The addition of v(k) to every entry of v yields a nonnegative vector.
(A.12)
(a) Let v satisfy form (I), so j 3 g1 j 1, then the positive entry v(j g1 )
entry is good. When g1 = j 2, there are two positive entries in rows 1 and 2, entries in rows 1 to 3, with v(2) and v(3) both being good. with v(2) good and v(1) not good. When g1 = j 3, there are three positive
is good. When g1 = j 1, there is only one positive entry in row 1 and the
(b) Let v satisfy form (II), so 0 g1 j 4, then there are at least two good positive entries in v, namely in consecutive rows j g1 1 and j g1 . 7. Recall from Sections A.1.2 and A.1.1 that if M is pre-multiplied by a A (B), then the rst row of M contributes (does not contribute) to the value of the product AM (BM ). Let P = Pjg1 Pjg1 1 P2 P1 where Pi {A, B} for i = 2, . . . , j g1 1 and
P1 = A since we forbid two consecutive Bs.Pre-multiplying M by P , we can
associate Pi with the i-th row of M : If Pi = A, then the i-th row of M contributes
164
to the the product P M ; otherwise if Pi = B, then the i-th row of M does not contribute to the product P M . If Pi = A and the i-th entry in v is positive and good, then P BAg1 DAj+1 there are two consecutive good positive entries in v, then P BAg1 DAj+1 Proof of claims: Let S(g1 ) = which can be re-expressed as v(1) = 2
jg1 1 t=1
0. If 0.
2jg1 1t f1,mj+g1 +t for a xed j.
(1) Simplifying the expression for v(1), we obtain v(1) = 2jg1 1 f1,mj+g1 +1 S(g1 ),
jg1 2 jg1 2
f1,(m+1)j(jg1 1)
2k1 f1,(m+1)jk
k=1
the term gives
for g1 j 3, and we show below that v(1) < 0 for g1 j 4. First we lower bound
jg1 2 k=1
2k1 f1,(m+1)jk by (1 + 2 + + 2jg1 3 )f1,(m+1)j(jg1 2) , which 2k1 f1,(m+1)jk (2jg1 2 1)f1,(m+1)j(jg1 2) .
jg1 2
k=1
f1,(m+1)j(jg1 2) . Expanding the term f1,(m+1)j(jg1 2) using Eqn. (A.1) and bounding f1,mj+g1 +1 by way of Eqn. (A.4): f1,mj+g1 +1 2f1,mj+g1 , we obtain v(1) 2jg1 2 (f1,mj+g1 + f1,mj+g1 1 + f1,mj+g1 +1j ) + (3f1,mj+g1 + f1,mj+g1 1 + f1,mj+g1 2 + + f1,mj+g1 +1j ) 0, where the last inequality is obtained by checking that the coecients of each term f1,k for k = mj + g1 , mj + g1 1, . . . , (mj + g1 + 1 j) are at most 0. In fact, it suces to check the coecient of f1,mj+g1 , namely, 3 2jg1 2 0. The latter is true by the assumption g1 j 4. (2) We prove this by induction. Assume v(r0 ) > 0 for some 1 r0 j g1 1.
latter expression can be expressed as 2jg1 2 f1,(m+1)j(jg1 1) f1,(m+1)j(jg1 2) +
Using this, v(1) 2jg1 2 f1,(m+1)j(jg1 1) (2jg1 2 1)f1,(m+1)j(jg1 2) , and the
Then v(r0 + 1) = 2jg1 f1,mj+g1 +r0 +1 2jg1 1 fr0 +1,mj+g1 +1 S(g1 ), where the terms
165
f1,mj+g1 +r0 +1 and fr0 +1,mj+g1 +1 can be expanded using Eqn. (A.1) to obtain v(r0 ) + 2jg1
1 k=1j
f1,mj+g1 +r0 +k 2jg1 1 f1,mj+g1 +r0 .
Then using Eqn. (A.3) to bound f1,mj+g1 +r0 , we obtain

1
v(r0 + 1) v(r0 ) + 2 v(r0 ) + 2
jg1 k=1j 2
f1,mj+g1 +r0 +k 2jg1 f1,mj+g1 +r0 1 f1,mj+g1 +r0 +k > 0.
jg1 k=1j
Thus, v(j g1 ) > v(j g1 1) > > v(r0 ) > 0. claim (1). Now assume g1 j 4. In this case vj+1 (1) < 0, but we will show that vj+1 (j g1 2) = 2jg1 f1,(m+1)j2 2jg1 1 fjg1 2,mj+g1 +1 S(g1 ) > 0. First, use the inequality f1,(m+1)j1 2f1,(m+1)j2 to obtain vj+1 (jg1 2) 2jg1 f1,(m+1)j2 2jg1 1 fjg1 2,mj+g1 +1 S(g1 )2f1,(m+1)j2 +f1,(m+1)j1 , where the latter can be expressed as the sum of the terms (2jg1 1 4)f1,(m+1)j2 + (2jg1 1 )(f1,(m+1)j3 + f1,(m+1)j4 + f1,mj2 ), and 2jg1 1 fjg1 1,mj+g1 +1 2jg1 2 f1,mj+g1 +1 23 f1,(m+1)j4 22 f1,(m+1)j3 by rst splitting 2jg1 f1,(m+1)j2 into 2 2jg1 1 f1,(m+1)j2 and expanding one of the terms using its j-step recursive sum in Eqn. (A.1). Then vj+1 (j g1 2) (3a) When g1 = j 3, because vj+1 (j g1 2) = vj+1 (1), which is positive by
(a) (b)
2jg1 1 f1,mj+g1 + 2jg1 1 f1,mj+g1 0,
(2jg1 4)f1,(m+1)j2 2jg1 2 f1,mj+g1 +1 + 2jg1 1 f1,mj+g1
166
where in step (a), we verify that 2jg1 4 because g1 j 2, and drop all other terms that are positive except 2jg1 1 f1,mj+g1 . In step (b), we use Eqn. (A.4) to bound f1,mj+g1 +1 , i.e., f1,mj+g1 +1 2f1,mj+g1 +1 , so f1,mj+g1 +1 2f1,mj+g1 +1 . Thus v(j g1 2) > 0 for all g1 j 3. (3b) Part (3a) implies that v(j g1 1) and v(j g1 ) are positive for g1 j 3
by claim 2. When g1 = j 2, v(j g1 1) = v(1), which is positive by claim 1. Thus, v(j g1 1) > 0 for all g1 j 2. (3c) When g1 = j 1, v(j g1 ) = v(1), which is positive by claim 1. Combining
this with the parts (3a) and (3b), we obtain the result that v(j g1 ) > 0 for all g1 j 1. (4) It is straightforward but tedious to verify the (nitely many) cases.
From the explicit description of v, it is clear that the negative entries are located in down the rows. It is clear that the negative entry with the largest magnitude must be v(j g1 + 1). So, v is of form (I). j g1 2 by Claim 3. However, by Claim 2, the positive entries are increasing down the rows in v, thus v(j g1 ) is still the largest positive entry and v is of form (II). If v(1) 0 (which happens for g1 j 4), the entries in v become positive by row rows j g1 + 1 to j. Observe that the negative entries are decreasing in magnitude
(5) If v(1) > 0, then v(j g1 ) is the largest positive entry by claims 1 and 2.
However, the negative entry with the largest magnitude is either v(1) or v(j g1 + 1). (6a) When j 3 g1 j 1, v(1) > 0, so v is of form (I). To show that v(j g1 )
is good, we prove
= 2jg1 f1,(m+1)j 2(2jg1 1 fjg1 ,mj+g1 +1 + S(g1 ))
v(j g1 ) + v(j g1 + 1)
= 2jg1 (f1,(m+1)j1 + f1,(m+1)j2 + + f1,mj+1 + f1,mj ) (2jg1 2)f1,(m+1)j1 2jg1 1 f1,mj+g1 +1
(a) (b)
(2jg1 fjg1 ,mj+g1 +1 + 2jg1 1 f1,mj+g1 +1 + + 4f1,(m+1)j2 + 2f1,(m+1)j1 )
(2jg1 2 2jg1 1 )f1,mj+g1 +1 0,
167
In step (a), we compare the coecients of each f1,t term and use f1,mj+g1 +1 to upper bound fjg1 ,mj+g1 +1 . In step (b), it is clear that 2jg1 2 2jg1 1 0 for any j g1 . Thus, vj+1 (j g1 ) is good.
v(jg1 ) = v(2). This is the only positive good entry since v(1)|v(3)| = f1,(m+1)j1 f1,(m+1)j2 + + f1,(m+1)j4 ), which is < 0. Thus, v(1) is not good. f3,(m+1)j1 f3,(m+1)j , which can be expressed as f1,(m+1)j1 f3,(m+1)j1 (f1,(m+1)j1 +
When g1 = j 1, v(j g1 ) = v(1) is the only positive entry. When g1 = j 2,
When g1 = j 3, v(j g1 ) = v(3) is good. We show below that v(2) is also good. v(2) |v(4)|
= 23 f1,(m+1)j1 22 f2,(m+1)j2 2f1,(m+1)j2 f1,(m+1)j1 f4,(m+1)j1 f4,(m+1)j

(m+1)j3 (m+1)j2 (m+1)j1
= 7f1,(m+1)j1 4
mj1
f1,k 2f1,(m+1)j2
k=mj+2
f1,k
f1,k
k=mj+3
= 6f1,(m+1)j1 4f1,(m+1)j2 6(f1,(m+1)j3 + f1,(m+1)j4 + + f1,mj+3 ) 5f1,mj+2 4(f1,mj+1 + f1,mj + f1,mj1 ),
which can be reduced to v(2) |v(4)| = 6(f1,(m+1)j2 + f1,(m+1)j3 + + f1,mj1 ) 4f1,(m+1)j2 5f1,mj+2 6(f1,(m+1)j3 + f1,(m+1)j4 + + f1,mj+3 ) 4(f1,mj+1 + f1,mj + f1,mj1 ) 2f1,(m+1)j2 + f1,mj+2 + 2(f1,mj+1 + f1,mj + f1,mj1 ) > 0.
(6b) When g1 j 4, v(1) < 0, v is of form (II).
168
Let g1 j 4. We show below that the entry v(j g1 1) satises Eqn. (A.12). v(j g1 1) |v(j g1 + 1)| 2|v(1)|
(a)
2jg1 2 f1,(m+1)j1 + 2jg1 1 f1,mj+g1 +1 4S(g1 ) 2jg1 2 f1,(m+1)j1 2jg1 1 f1,mj+g1 +1

(m+1)j2
22 f1,(m+1)j1 + 23 f1,(m+1)j2 + + 2jg1 1 f1,mj+g1 +2 2)f1,(m+1)j1 + 2

3 jg1 1 k=mj2
(b) =
(2
jg1 1
f1,k
(c) (d)
(2jg1 1 6)f1,(m+1)j1 0.
2 f1,(m+1)j1 + 2 f1,(m+1)j2 + + 2jg1 1 f1,mj+g1 +2
In step(a), we subtract the term (2jg1 1 )fjg1 +1,mj+g1 +1 from (2jg1 )f1,mj+g1 +1 and applied Eqn. (A.4) to f1,(m+1)j which exceeds fjg1 +1,(m+1)j . In step (b), we expand 2jg1 1 f1,(m+1)j1 using Eqn. (A.1). In step (c), some of the positive terms are dropped, namely the terms containing f1,k with k = mj2, mj1, . . . , (m+1)j2 since their coecients are positive. In step (d), the coecient of f1,(m+1)j1 is positive if g1 j 4. shown that for g1 j 4, there are two consecutive positive good entries in vj+1 , (7) If Pi = A and the i-th entry in v is positive and good, then P v implies that P vj implies P vt P aj /2 but aj 0, so P vj It then follows immediately that v(j g1 ) is also good by Claim 2. Thus, we have
namely in rows j g1 and j g1 1.
0 by denition 0 0
of goodness. Note that using the recursion on vj+1 , we have P vj+1 = P (2vj aj ) 0. By the same argument, P vj+1 0.
nonnegative, we can conclude that Thus P BAg1 DAj+1
0 for t = j, j 1, . . . , g1 + 1. In addition, since the matrix G is already
If there are two consecutive good positive entries in vj+1 , say at rows n and n+1, then either Pn = A and Pn+1 = B or Pn = B and Pn+1 = A since two consecutive Bs in P are forbidden. It follows immediately that P BAg1 DAj+1 This completes the rst set of claims. 0.
169
entry at row 2, so if P2 = B, then negative entries will still persist in the product P BAg1 DAj+1 . To deal with this, postmultiply BAg1 DAj+1 by another A to obtain the following rst column vector, which we call vj+2 , i.e., vj+2 = 2vj+1 v0 . Evolving the vector vj+2 further by postmultiplying the matrix BAg1 DAj+2 with
Since from claim (6a), the case when g1 = j 2 yields only one good positive
2j, the rst column vt of BAg1 DAt satises
more As, we obtain a recursive expression as follows. For any j + 2 t j + g1 + 2 =
vt+1 = 2vt vtj1 .
(A.13)
Note that the vectors being subtracted on the right are nonnegative. Evolving v2j further, we obtain v2j+2 = 2v2j+1 vj . Thus the matrix BAg1 DA2j+2 = [v2j+2 , v2j+1 , , vj+3 , vj+2 ]. Second Set of Claims:
The following claims deal only with the special case g1 = j 2. 0 since vj+2
(i) Any premultiplication factor P (with P1 = A) will result in P vj+2 has positive good entries in rows 1 and 2.
(ii) The negative entry of v2j with the largest magnitude is in row 3 while the rst two rows in v2j are positive. For j 7, v2j (1) + v2j (3) 0 and thus v2j has positive good entries in rows 1 and 2. (iii) For j 7, any premultiplication factor P (with P1 = A) will make P v2j This in turn implies that that P vt P BA
j2
0.
DA
2j
0.
0 for t = 2j 1, 2j 2, . . . , j + 3, and thus
Proof of Second Set of Claims: (i) We verify that the rst two positive entries in vj+1 have become good entries. vj+2 (1) |vj+2 (3)| = 2vj+1 (1) + f2,(m+1)j (2vj+1 (3) f3,(m+1)j ) = 2(f1,(m+1)j1 f3,(m+1)j1 f3,(m+1)j ) + f2,(m+1)j + f3,(m+1)j 2(f1,(m+1)j f3,(m+1)j ) 0,
170
is now good. For the entry vj+2 (2),
since f2,(m+1)j f3,(m+1)j and f1,(m+1)j f3,(m+1)j . Thus, the positive entry vj+2 (1)
vj+2 (2) |vj+2 (3)| = 2vj+1 (2) + f3,(m+1)j (2vj+1 (3) f3,(m+1)j ) = 2(vj+1 (2) vj+1 (3)) + 2f3,(m+1)j 0, since vj+1 (2) is a good, positive entry. Thus, vj+2 (2) remains good. By Claim 7, two consecutive good entries is a sucient condition for P vj+2 However, P vj+2 P BAg1 DAj+2 0). 0 does not imply P vj+1 0.
0 (so we cannot conclude that
(ii) We write down an explicit expression for v2j , which is obtained by backtracking through the recursion in Eqn. (A.13).
4f1,(m+1)j 3f2,(m+1)j1 f 3,(m+1)j1 f3,(m+1)j f4,(m+1)j1 f4,(m+1)j j1 2 . . . fj1,(m+1)j1 fj1,(m+1)j fj,(m+1)j1 fj,(m+1)j 0 f1,(m+1)j2 f1,mj+2 f1,mj+1 f2,(m+1)j f1,(m+1)j1 f1,mj+3 f1,mj+2 f3,(m+1)j 0 0 f3,(m+1)j 0 0 j4 j3 j2 f 0 +2 0 2 0 2 4,(m+1)j 2 . . . . . . . . . . . . 0 0 f 0 j,(m+1)j 0 0 0 0 f1,(m+1)j1
v2j
largest magnitude.
Observe that v2j (1) 0 and v2j (2) 0, and v2j (3) is the negative entry with the
To simplify the expression v2j (1) + v2j (3) = 2j1 f1,(m+1)j1 + 2j2 f2,(m+1)j
171
j3 i=0
2i f1,(m+1)ji2 2j1 f3,(m+1)j1 2j2 f3,(m+1)j , we have v2j (1) + v2j (3)
j3 (a) =
2j1 f1,(m+1)j1 + 2j2 f1,mj+1 2

j1
i=0
2i f1,(m+1)ji2 2j1 f3,(m+1)j1

j3
(b) =
f1,mj + 2
j1
f1,mj1 + 2
j2
f1,mj+1
2i f1,(m+1)ji2 .
i=0
(A.14)
Step (a) is due to f2,(m+1)j f3,(m+1)j = f1,mj + f1,mj1 . Step (b) is due to f1,(m+1)j1
(m+1)j1 k=mj+1 f1,k (m+1)j2 f3,(m+1)j1 = k=mj1
(m+1)j1 k=mj+2 f1,k = f1,mj+1 . (m+1)j2 f1,k k=mj+1 f1,k =
Note that if v2j (1) + v2j (3) 0, then v2j (2) + v2j (3) 0, since v2j (2) > v2j (1). Next we will show that if j 7, Eqn. (A.14) satises
j3
j1
f1,mj + 2
j1
f1,mj1 + 2
j2
f1,mj+1
2i f1,(m+1)ji2 .
i=0
(A.15)
It is straightforward to show that the expression on the left reduces to 5(2j3 )f1,mj+1 + (2j2 + 2j3 )f1,(m1)j + 2j2 f1,(m1)j1 , by applying Eqn. (A.3) repeatedly. By analyzing the terms in the expression on the right, we have
j3
(A.16)
i=0
2i f1,(m+1)ji2 = 2j3 f1,mj+1 + 2j4 f1,mj+2 + + 2f1,(m+1)j3 + f1,(m+1)j2 .
Note that the above expression contains only 1 term if j = 3, 2 terms if j = 4, and in general, n terms if j = n + 2. We can express each f1,mj+t term in terms of
172
f1,mj+1 , again using Eqn. (A.3) repeatedly as follows.

t
j(t+2)
f1,mj+t = 2
j3
f1,mj+1
2jk f1,(m1)j+(k2)
k=4
if j t + 2 3.
Using this, we can manipulate the expression on the right for j 3 to become (j 2)2j3 f1,mj+1 (j 3)2j4 f1,(m1)j+2 + (j 4)2j5 f1,(m1)j+3 + + 22 f1,(m1)j+(j3) + f1,(m1)j+(j2) (A.17)
Comparing the expressions in (A.16) and (A.17) term by term, it is easy to see that for j 7, the expression on the left exceeds that on the right. This is because the number of 2j3 fmj+1 on the right is at most 5, and thus cancels out with the same term on the left hand side. But the expression on the left now has strictly positive terms while that on the right is composed of a sum of strictly negative terms. This yields inequality (A.15).
(iii) For j 7, we note that any premultiplication factor P (with P1 = A) that makes P v2j P vj2 /2 Eqn. (A.13). 0 since vj2 is a nonnegative vector. Thus, this implies that for 0 due to the recursive relation on the vt in 0 implies P (2v2j1 vj2 ) 0,which in turn implies P v2j1
t = 2j 1, 2j 2, . . . , j + 3, P vt
any premultiplication factor P under the restriction that there are no two consecutive Bs in P , will make P BAg1 DA2j+2 Proof of Lemma 54. 0. This concludes the proof.
Thus, by the second set of claims, we have shown that in the case when g1 = j 2,
Let g1 , . . . , g3 denote three consecutive gaps (from right to
left) with size < j. In the framework of Lemma 53 for j = 3, we can see by inspection that it omits the cases where (g3 , g2 , g1 ) = (0, 0, 2), (2, 0, 2), (1, 0, 2) and (g2 , g1 ) = (0, 1). These cases are overcome by localized mappings given by the following four
173
matrix inequalities: BBA3 BA3 C m BA5 A2 BA3 BA3 C m BA5 A2 BABA3 BA3 C m BA4 BA3 BA3 C m BA4 BBBA2 C m BA9 A2 BBA2 C m BA9 BABBA2 C m BA10 BBAC m BA9 . (A.18) (A.19) (A.20) (A.21)
Note that the matrices A and B used here are of size 4 4 as they belong to MTR(3), cases (0, 0, 2), (2, 0, 2), (1, 0, 2), and (0, 1) respectively.
while m is any integer 0. Inequalities (A.18), (A.19), (A.20) and (A.21) handle the
To show Inequality (A.18), rst we write down the matrix on the left BC m+3 A2 , which can be expressed as
(f 3,3(m+3)+2 + f3,3(m+3)2 ) f3,3(m+3)+1 f3,3(m+3) f3,3(m+3)1 0 0 0 0 0 0 0 0 (f2,3(m+3)+2 + f2,3(m+3)2 ) f2,3(m+3)+1 f2,3(m+3) f2,3(m+3)1 .
It is straightforward to derive the matrix on the right M = BBBA2 C m BA9 from the form of C m and using results from Sections A.1.1 and A.1.2. Thus, M is of the form
M (1, 1) M (1, 2) M (1, 3) M (1, 4) 0 0 0 0 0 0 0 0 0 0 0 0 ,
where
M (1, 1) = f1,3m+10 + 6f1,3m+2 + 2f1,3m+3 + 3f1,3m+5 + f1,3m+6 , M (1, 2) = f1,3m+9 + 3f1,3m+2 + f1,3m+3 + 2f1,3m+5 , M (1, 3) = f1,3m+8 + f1,3m+2 + 2f1,3m+5 , M (1, 4) = f1,3m+7 + f1,3m+2 + f1,3m+3 ,
174
We can easily check that the rst row of BC m+3 A2 exceeds that of the rst row of M entry by entry using Eqns. (A.1) and (A.5) to simplify the expressions. To show Inequality (A.19), rst we write down the matrix on the left A2 C m+3 A2 , which can be expressed as
(f 1,t+3 + f1,t+2 + 2f1,t1 ) (f1,t+2 + f1,t+1 ) (f1,t+1 + f1,t ) (f1,t + f1,t1 ) (f1,t+3 + f1,t+2 + 2f1,t1 ) (f1,t+2 + f1,t+1 ) (f1,t+1 + f1,t ) (f1,t + f1,t1 ) (f1,t+3 + f1,t1 ) f1,t+2 f1,t+1 f1,t (f1,t+4 + f1,t ) f1,t+3 f1,t+2 f1,t+1 ,
with t = 3(m+3). It is straightforward to derive the matrix on the right A2 BBA2 C m BA9 , which is given as
s t u v s t u v s t u v s t u v ,
with
s = 3f1,3m+10 + 2f1,3m+9 + 3f1,3m+7 + 2f1,3m+5 + 21f1,3m+4 + 16f1,3m+2 , t = 3f1,3m+9 + 2f1,3m+8 + 3f1,3m+5 + 13f1,3m+4 + 8f1,3m+2 , u = 3f1,3m+8 + 2f1,3m+7 + 7f1,3m+4 + f1,3m+3 + 4f1,3m+2 , v = 3f1,3m+7 + 2f1,3m+6 + 3f1,3m+3 + 5f1,3m+2 + 2f1,3m+1 ,
and
s = 2f1,3m+10 + f1,3m+9 + 2f1,3m+6 + 5f1,3m+5 + 11f1,3m+4 + 10f1,3m+2 , t = 2f1,3m+9 + f1,3m+8 + 3f1,3m+5 + 6f1,3m+4 + 5f1,3m+2 , u = 2f1,3m+8 + f1,3m+7 + 4f1,3m+4 + f1,3m+3 + 3f1,3m+2 , v = 2f1,3m+7 + f1,3m+6 + 2f1,3m+3 + 3f1,3m+2 + f1,3m+1 .
To show Inequality (A.20), rst we write down the matrix on the left A2 BAC m+3 A,
175
which can be expressed as

2f 2,t+2 + f1,t+1 2f2,t+2 + f1,t+1 f2,t+2 + f1,t+1 2(f2,t+2 + f1,t+1 ) 2(f2,t+1 + f1,t ) 2(f2,t + f1,t1 ) 2(f2,t1 + f1,t2 ) 2f2,t+1 + f1,t 2f2,t+1 + f1,t f2,t+1 + f1,t 2f2,t + f1,t1 2f2,t + f1,t1 f2,t + f1,t1 2f2,1 + f1,t2 2f2,t1 + f1,t2 (f2,t1 + f1,t2 ,
with t = 3(m + 3). It is easy to write out the entries of the expression on the right BABBA2 C m BA10 , which is given as
e f g h , e f g h 0 0 0 0 e f g h
with
e = f1,3m+11 + f1,3m+10 + 3f1,3m+7 + f1,3m+5 + 16f1,3m+4 + 11f1,3m+2 + 3f1,3m+1 , f = f1,3m+10 + f1,3m+9 + f1,3m+6 + 2f1,3m+5 + 9f1,3m+4 + 6f1,3m+2 + 2f1,3m+1 ,
g = f1,3m+9 + f1,3m+8 + f1,3m+5 + 5f1,3m+4 + 3f1,3m+2 + f1,3m+1 , h = f1,3m+8 + f1,3m+7 + 3f1,3m+4 + f1,3m+2 .
It is straightforward to check that that the entry-by-entry dierence of the third rows of A2 BAC m+3 A and BABBA2 C m BA10 are nonnegative.
To show Inequality (A.21), rst we write down the matrix on the left C m+3 A, which can be expressed as
f 2,3(m+3)+1 f2,3(m+3) f2,3(m+3)1 f2,3(m+3)2 f3,3(m+3)+1 f3,3(m+3) f3,3(m+3)1 f3,3(m+3)2 0 0 0 0 f1,3(m+3)+1 f1,3(m+3) f1,3(m+3)1 f1,3(m+3)2 .
It is straightforward to derive the matrix on the right BBAC m BA9 , which is given
176
as
with
a b c d , 0 0 0 0 0 0 0 0
a b c d
a = f1,3m+9 + 6f1,3m+1 + 2f1,3m+2 + 3f1,3m+4 + f1,3m+5 , b = f1,3m+8 + 3f1,3m+1 + f1,3m+2 + 2f1,3m+4 , c = f1,3m+7 + f1,3m+1 + 2f1,3m+4 , d = f1,3m+6 + f1,3m+1 + f1,3m+2 ,
Since the rst two rows of BBAC m BA10 are the same, it suces to check that the dierence between the second rows of C m+3 A and BBAC m BA10 are nonnegative for Inequality (A.21) to hold. This is straightforward to verify.
Appendix B Long Proofs for Chapter 7

We present the proofs of Theorems 71, 72 and 73 of Chapter 7 below. Proof of Theorem 71:. Let M = max(m) and s = min(m). Let l0 be the smallest integer such that l0 p s + 1. We rst show that for any integer k 1, there is a 1-1 mapping from strings of length kp in TMTR(m) into strings of length (k + l0 )p that satisfy TMTR(m ). Let w be a string of length kp satisfying TMTR(m). Let the position indices for Thus mv = s and mu = M . The image sequence w of length (k + l0 )p is indexed w range from 0 to kp 1. Let v = i1 + lp and u = i2 + lp for all l = 0, . . . , k 1.
all 0 i kp 1. Similarly, let r denote the TMTR values of the image sequence
and ri mi for all other position indices i. This is depicted for one pair of (v, u) as w . The mapping is constructed so that rv = s + 1 and ru M 1 for all v and u,
from l0 p to kp 1. Let r denote the TMTR values of the word w , so ri mi for
follows.
Position index TMTR of w TMTR of w
v 1
v s s+1
u M M 1
u + 1
First, we will dene the mapping l which maps strings of length (k+l0 )p satisfying TMTR((m )l+l0 mkl ) to strings of the same length satisfying TMTR((m )l+l0 +1 mk(l+1) ) 177
178
APPENDIX B. LONG PROOFS FOR CHAPTER 7
for the l-th (v, u) pair, and show that l is one-to-one. The overall mapping is then dened as = k1 0 , where is the mapping that prepends l0 p zeros to a given sequence of length kp. also one-to-one. Since and l for l = 0, . . . , k 1 are one-to-one mappings, it will follow that is Given a string w of length (k + l0 )p (indexed from l0 p to kp 1) satisfying
TMTR((m )l mkl ), dene l (w) = w as follows. First let z denote the following ( possibly empty) substring of w. z= w[v s 1, u M 1] if M = s + 2 or u M 1 < v + 1 ,
w[v s 1, v + 1]
if M s + 3 and u M 1 v + 1.
Then dene the mapping l for interval l as follows. (i) If w has < M 1s ending at u, then w = l (w) = w. (ii) If w has exactly M 1s ending at u, there are two cases: (a) If M = s + 2 or u M 1 < v + 1, then let w [v s 1, u] = 01s+1 0 z w[v + 2 + |z|, u] with w = w outside of this interval. (b) If M s + 3 and u M 1 v + 1, then let w [v s 1, u] = 01s+1 0 w[v + 2, u |z|] z with w = w outside of this interval. This forces exactly (s + 1) 1s ending at position v in the image string w , when there are exactly M 1s ending at position u in string w. To preserve the information
that is lost due to setting rv = s + 1, the string z is mapped into the space between
positions v + 2 and u. We need to check that z ts in this space. when M = s + 2.) Thus, z ts between positions v + 2 and u, though not necessarily tight; any extra space is lled in by w[v + 2 + |z|, u]. In case (ii)(b), |z| = s + 3. and M s + 3 together imply that u v 1 M + 1 s + 4. Again, z ts between positions v + 2 and u, since the inequalities u M 1 v + 1 In case (ii)(a), |z| = (uv)(M s)+1 uv1 since M s+2. (Equality holds
179
Next, let n = (m )l+l0 +1 mk(l+1) . We need to check that under l , rv = s + 1 and ru M 1, and for all other positions i, ri ni . In fact, it suces to show this for
v s 1 i u 1, since l only changes w in this interval.
1s ending at v forces the rv = s + 1 = nv . The 0 label in w at positions v s 1 and v + 1 imply that rvs1 = rv+1 = 0, so clearly rvs1 nvs1 and rv+1 nv+1 .
ru M 1 (by construction) and rv m + 1 trivially. In case (ii), the run of (s + 1)
In case (i), w = w. Clearly, ri ni for all i = s 1 to kp 1. In particular,
For positions v s 2 i v 1 in w , there are at most s 1s ending at positions i,

and so ri s ni . For the remaining positions from v + 2 to u, we need to consider
cases (ii)(a) and (ii)(b) separately.
Here, |z| = uv1, so w [v+2, u] = z. But in w, z ended at position uM 1 v+1.

By denition of s and M , s+1 ruM 1 M 1, which implies ru M 1 = nu since
For case(ii)(a), rst consider the subcase when M = s + 2 and u M 1 v + 1.
z is mapped into w to end at position u. We claim that for positions v +2 i u1,
then suces to show that there is no run of M 1s ending anywhere between positions v + 2 and u 1 in w . Observe that the string z does not contain any run of M 1s even though the values of ri for v s 1 i v 1 are not known. The length
ri ni . To see this, recall that for w, ri = s + 1 = M 1 = ni for v + 1 i u 1. It
between positions v s1 and v is exactly s+2 = M . But rv = s for w, which means
that there can be at most s + 1 1s followed by a zero ending at v. Thus, for positions v + 1 i u M 1, ri s + 1, and so there is no run of M 1s ending anywhere
there cannot be a run of M 1s ending anywhere in the interval w [v + 1, u], so ri ni
within this interval in w. Correspondingly, with w [v + 2, u] = z and w [v + 1] = 0, for v + 2 i u 1.
so z terminates exactly at position u M 1 in w, which satises rv = s. This

means that z must contain a zero, so ru M 1 = nu . Then w[v + 2, u] = 1uv1 and w [v + 2, u] = z together imply that for positions v + 2 i u 1, ri ni . If
But u v 1 M = s + 2. If u v 1 = M , rewriting this yields v = u M 1,
The next subcase is when uM 1 < v +1 and M = s+2. Again, w [v +2, u] = z.
at u. As before, w [v + 2, u] = z implies ri ni for v + 2 i u 1.
u v 1 < M , then w [v + 1] = 0 eliminates the possibility of a run of M 1s ending
180
u v 2 M 1. If u M 1 v 1, then u M + 1 v + 1, which implies that

all 1s. Thus, ru M 1 = nu , and it is clear that ri ni for v + 2 i u 1.
If uM 1 < v+1 and M s+3, it is easy to verify that |z| = uv(M s)+1
w [v + 1] = 0 eliminates the possibility of a run of M 1s ending at u even if z contains
ri ni for v + 2 i u 1.
substring of w that ends at position v and so rv s. Thus, ru M 1 = nu , and
position u in w is not possible since |z| = m + 2 and must contain a 0 since z is a
If u M 1 = v, then u M = v + 1, so w[v + 2, u] = 1M . A run of M 1s ending
Since z is a substring of w which ended at position v + 1, where s + 1 rv+1 M 1, The fact that w[u |z| + 1, u 1] = 1uv+|z|1 while w [u |z| + 1, u 1] = z
For case (ii)(b), when M s + 3 and u M 1 v + 1, then |z| = s + 3 M .
z must contain a 0. When z is mapped to w , it is clear that ru M 1 = nu .
and w [v |z|] = w[v |z|] = 0 implies that ri ni for v |z| + 1 i u 1.
w [v + 2, u |z|] = w[v + 2, u |z|].
For the remaining positions v + 2 i u |z|, it is clear that ri ni since
To show that l is 1-1, we show that given a w satisfying TMTR(x), there is a unique pre-image w satisfying TMTR(y), where x = (m )l+l0 +1 mk(l+1) ) and y = comparing vectors x and y, we can determine the positions v = i1 + lp and u = i2 + lp (m )l+l0 mkl ). The position indices for w and w range from l0 p to kp 1. By
cases (i) and (ii) by observing if there is a run of s + 1 1s ending at position v in w . If there is no such run, we are in case (i), and w = w . Otherwise, we are in case (ii), so w[l0 p, v s 2] = w [u + 1, kp 1]. Using the values of u, v, M, s, we can further
where xv = s + 1, yv = s, xu = M 1, and yv = M . We can distinguish between
distinguish between cases (ii)(a) and (ii)(b) to reconstruct w between positions vs2 and u. Namely, in case (ii)(a), we can determine z = w [v + 2, v + 1 + |z|], and thus and assign w[v s 1, u] = zw [v + 2, u |z|]1M . assign w[v s 1, u] = z01M . In case (ii)(b), identify the string z = w [u |z| + 1, u],
kp satisfying TMTR(m) into strings of length (k + l0 )p satisfying TMTR(m ). Thus
The overall mapping = k1 0 is thus 1-1, mapping strings of length
181
letting NS (l) denote the number of strings of length l satisfying S, we have NTMTR(m) (kp) NTMTR(m ) (kp + l0 p) 2l0 p NTMTR(m ) (kp). Taking log on both sides and dividing by kp yields log NTMTR(m) (kp) log(l0 p) log NTMTR(m ) (kp) + . kp kp kp Taking limits as k , we obtain capTMTR (m) capTMTR (m ). Proof of Theorem 72:. Applying the mapping in the proof of Theorem 71 repeatedly to the vector m, we obtain a m with mi = t for all i, or mi {t, t 1} capTMTR (m ) capMTR (t), where t = m.
for all i, where t is some positive integer. Each time, m = m, so capTMTR (m) Proof of Theorem 73:. We rst check the cases for which m < 2. For m = (1, 2),
the largest positive real root of its characteristic equation, as given by Eqn. (7.9), is 31/2 < 1.75 < (1.5). For m = (0, 1, 2), the largest positive real root of its characteristic equation, Eqn. (7.8), is 22/3 < 1.618 < (1). When m = (0, 1, 2, 3), the largest positive real root of its characteristic equation is 23/4 < 1.754 < (1.5). (We exclude the case m = (0, 1) since the largest positive real root of its characteristic equation is 21/2 > 1.39 > (0.5).) r = (M + 1) mod p, so M + 1 = qp + r. The characteristic equation for TMTR(m)
p qp
Now assume p 2 and m = M (p 1)/2 2. Let q = (M + 1)/p, and
for m satisfying the tightness condition with equality is given by Eqn. (7.5), and can be rewritten as z qp (2p 1) (z zp ) ) (2p 2p1r ) = 0. Multiplying by (1z p ) throughout (1z and simplifying, we obtain (2p 2p1r ) + (2p1r 1)z p + z M +1r (z p 2p ) = 0. Let g(z) = (2p 2p1r ) + (2p1r 1)z p + z M +1r (z p 2p ).
Let f (z) = z m+2 2z m+1 + 1, where m = M (p 1)/2 as above, so f (z) =
equation of MTR(m). The equation f (z) = 0 has a unique solution z = z(m) in (1, 2) and the solution is increasing as a function of m (see [11][Proof of Lemma 2]).
z M +1r z r(p1)/2 (z 2) + 1. Note that if m Z, then f (z) = 0 is the characteristic
Let z0 = 1.83, chosen so that capMTR (2) > z0 . It follows that the root of f in (1, 2)
182
implies f () < 0, because f is continuous and f (2) = 1 > 0.
is greater than z0 since m 2. It suces to show that for any [z0 , 2), g() = 0 Assume that g() = 0, where is in [z0 , 2). Then M +1r = (2 (2p 2p1r ) + (2p1r 1)p (p 2p )
p 2p1r )+(2p1r 1)p
(p 2p )
Substituting the above equation into f , we obtain f () =
p1 2
( 2) + 1.
We will show that f () < 0. This is equivalent to showing p 2p (2p 2p1r ) + (2p1r 1)p ( 2)r
p1 2
> 0.
(B.1)
For p = 2, consider the two cases r = 0 and r = 1. When r = 1, Inequality (B.1) since < 2. The latter is equivalent to ( 4)( 1) = 2 5 + 4 < 0, which is reduces to ( 2) ( + 2) 31/2 > 0, which is equivalent to ( + 2) 31/2 < 0,
true for (1, 4).
which is equivalent to ( + 2) (2 + 2 )1/2 < 0, which is in turn equivalent to 1.5874 < z0 .
When r = 0, Inequality (B.1) reduces to ( 2) ( + 2) (2 + 2 )1/2 > 0,
4 3 4 + 4 > 0. The latter is true for (z0 , 2) since its largest real root is
( 2)p2p1 since 2 < 0. Thus the LHS of Inequality (B.1) is at least
For p 3, rst note that p 2p = ( 2)(p1 + p2 2 + p3 22 + + 2p1 )

p1
( 2)p2p1 (2p 2p1r ) + (2p1r 1)p ( 2)r 2 p1 1 = ( 2)2p1r p2r (2r+1 1) + (1 p1r )p r 2 , 2 so it suces to show (2r+1 1)r
p1 2
+ (1
1 2p1r
)r+
p+1 2
> p2r ,
(B.2)
183
For p < 14, this inequality is easily checked by explicit calculation using (2r+1 1) min 1.83r
p1 2
, 2r
p1 2
+ 1
1 2p1r
(1.83)r+
p+1 2
as a lower bound to the LHS of Inequality (B.2) by letting p range from 3 to 8 and r range from 0 to p 1. For r = p 1, Inequality (B.2) reduces to (2p 1) 2 > p2p1 , which is true since 2p 1 > 2p1 and (p1)/2 > ( 1.83)p1 > 1.352p1 > p for p 8. ity (B.2) since the term (2r+1 1) For r < p 1, we will show 1
1 2p1r r p1 2
Assume p 14, and consider the two cases r = p 1 and r < p 1 separately.
p1
r+
p+1 2
> p2r , which implies Inequal-
is also positive.
Note that 1 1/(2p1r ) 1/2, so it suces to show 2

r
p+1 2
> 2p.
(B.3)
We can then bound the LHS of Inequality (B.3) as follows 2

r
p+1 2
p2
p+1 2
= 22p (3p3)/2 22p z0 (3p3)/2 . Note that 2z0 (3p3)/2 = 2(z0 3/2 )p z0 3/2 (2.475p )(0.8) > p 2p for p 14 since
Inequality (B.3).
(2.475/2)p = 1.2375p > 1.25p for p 14. Thus 22p z0 (3p3)/2 > 2p, which proves
The proofs of the various inequalities in Proposition 76 are very similar, so we shall only prove part (1) in detail. There are two steps, the rst being to derive the characteristic equation of the TMTR(m) constraint using Theorem 66. This involves constructing the VLG of its standard trellis presentation by selecting |m| trellis states, m, counting the paths and path lengths of rst return to the Rome set enables us to namely {(0, 0), (1, 0), . . . , (m, 0)} as the Rome set. For a given congruence class of
derive the matrix whose determinantal equation is described by Eqn. (7.4).
184
Once the characteristic equation of the constraint is determined, the second step uses algebraic techniques similar to that of the proof of Theorem 73. Namely, for two polynomials f (z) and g(z), to show that the largest real root of f exceeds that of g in some interval [a, b], we show that for the largest real root of g in [a, b], f () < 0 and f (b) > 0. To prove part (1) of Proposition 76, we require the following lemma. Lemma 86 Let 0 be the largest positive real root of the characteristic equation of TMTR(t, t, t + 1, t + 1). Then 0 is given by the largest positive real root of the following equations (one per congruence class of t). 3 + 12z 4+8s 16z 4+12s + z 8+12s = 0, for t 0 (mod 4)(B.4) for t 1 (mod 4)(B.5) for t 2 (mod 4)(B.6) for t 3 (mod 4)(B.7)
4 + z 4 + 16z 4+4s + 2z 8+4s 16z 8+8s + z 12+8s = 0,
2 + 7z 4+4s + 6z 8+8s 16z 8+12s + z 12+12s = 0,
1 10z 4+4s + 24z 8+8s 16z 12+12s + z 16+12s = 0,
Proof. First dene the polynomials p(z), q(z) and r(z) as follows.
4s
p(z) = q(z) =
z i =
i=0 4s4
z (4s+4) 1 z 4 1 z 4s 1 z 4 1
z i = z i =
i=0 4s+4
r(z) =
i=0
z (4s+8) 1 . z 4 1
Construct the VLG of the standard trellis presentation by choosing the 4 trellis lengths of rst return to the Rome set, we form the matrix (one for each congruence identity matrix. class of t) whose determinantal equation is given by Eqn. 7.4. Let I denote the 4 4 states {(0, 0), (1, 0), (2, 0), (3, 0)} as the Rome set. By counting the paths and path
185
For t = 4s + i, where i = 0, 1, 2, 3, the determinantal equations are given by |Vi I| = 0 respectively, with the matrices Vi (i = 0, 1, 2, 3) listed below. 3 z q(z) z 4 q(z) z 1 p(z) z 2 p(z) V0 = 2 z p(z) z 3 q(z) z 4 q(z) z 1 p(z) z 1 p(z) z 2 q(z) z 3 q(z) z 4 q(z) z 4 q(z) z 1 p(z) z 2 p(z) z 3 p(z) z 4 q(z) z 1 p(z) z 2 q(z) z 3 q(z) ,
3 z p(z) z 4 q(z) z 1 p(z) z 2 p(z) V1 = 2 z p(z) z 3 q(z) z 4 q(z) z 1 p(z) . 1 2 3 4 z p(z) z p(z) z q(z) z q(z) 3 z p(z) z 4 q(z) z 1 p(z) z 2 p(z) V2 = 2 z p(z) z 3 p(z) z 4 q(z) z 1 p(z) z 1 p(z) z 2 p(z) z 3 p(z) z 4 p(z) 3 z p(z) z 4 p(z) z 1 p(z) z 2 p(z) V3 = 2 z p(z) z 3 p(z) z 4 p(z) z 1 r(z) z 1 r(z) z 2 p(z) z 3 p(z) z 4 p(z) and (B.4) respectively for t = 4s, 4s + 1, 4s + 2, 4s + 3. Proof of Part (1) of Proposition 76. Eqn. (B.6) from Lemma 86 as follows z 12+8s 16z 8+8s + 2z 8+4s + 16z 4+4s + z 4 4 = 0 = = z 4s+6 4z 4s+4 + z 2 + 2
2s+1 2s+1 4s+4 4s+42k 4s+4
z 4 p(z) z 1 p(z) z 2 p(z) z 3 p(z)
z 4 p(z) z 1 p(z) z 2 p(z) z 3 p(z)
It is easy to check that the determinantal equations expand to Eqns. (B.4), (B.5), (B.4),
(1a) Let t = 4s + 2. We can factorize the
z 4s+6 + 4z 4s+4 + z 2 2 = 0 z 3
k=1
z
k=1
(1)k z 4s+42k 2
= 0.
186
Let f (z) = z t+2 3
k=1
t+2 1 2
Note that f (z) = g(iz) and g(z) = f (iz), so the moduli of the roots of f (z) are the
z t+22k 2 and g(z) = z t+2 3
k=1
t+2 1 2
(1)k z t+22k 2.
same as the moduli of the roots of g(z). But f (z) is the characteristic polynomial of TMTR(t, t + 1) by Eqn. 7.6. Thus, capTMTR (t, t + 1) = capTMTR (t, t, t + 1, t + 1) . (1b) From Eqn (7.6), the characteristic polynomial of TMTR(t, t + 1) for t = 4s is given by q(z) = z 4s+2 3
2s k=1
z 2k 2 which simplies to z 4s+2 3 (z
4s+2 z 2 )
z 2 1
2. Dene
the polynomial g(z) as g(z) = (z 2 1)q(z) = 2 + z 2 4z 4s+2 + z 4s+4 .
Eqn. (B.4), the largest positive real root of f (z) = 0 equals the largest positive real root of the characteristic equation of TMTR(t, t, t + 1, t + 1) for t = 4s. Note also that g(z) = 0 has the same largest root as q(z) = 0. Observe that g(2) = 6 > 0, and g( 2) = 4(1 22s ) 0 (with strict inequality for all s 1). The derivative g (z) is
Let f (z) denote the function f (z) = 3+12z 4+8s 16z 4+12s +z 8+12s . From Lemma 86
2z + 4z 4s+1 ((z 2)(z + 2)s + (z 2 2)) > 0 for z 2, so g is increasing on (2, ). Thus, the largest root of g is contained in [ 2, 2). Perform a change of variables on f and g by letting y = z 2 . We obtain f (y) = 3 + 12y 2+4s 16y 2+6s + y 4+6s = 3 + 12y 2+4s + y 2s+1 (y 4)
y 4s+1 (y + 4) .
and g(y) = 2 + y + y 2s+1 (y 4). So we can restrict attention to nding a root in [2, 4).
2+4s
Setting f (y) = 0, the solution y = y0 satises y 2s+1 (y 4) = (3+12y ) . Then, y 4s+1 (y+4)
2+4s (3+12y0 ) . 4s+1 y0 (y0 +4)
substituting the above into g(y), we obtain g(y0 ) = 2 + y0
We claim that g(y) < 0 for y in the open interval (2, 4). This is because the
function y 4s+1 (y 4)(y 2) 3 < 0 for y (2, 4). Rewriting y 4s+1 (y 4)(y 2) 3 that 2 + y as y 4s+3 6y 4s+2 + 8y 4s+1 3, yields (y 2 + 6y + 8)y 4s+1 < 3 + 12y 2+4s , which implies
(3+12y 2+4s ) y 4s+1 (y+4)
< 0, which is equivalent to g(y) < 0.
Since y0 falls into the range (2, 4), this implies that g(y0 ) < 0 for f (y0 ) = 0.
Thus, the largest root of g is strictly to the right of the largest root of f , hence capTMTR (t, t + 1) > capTMTR (t, t, t + 1, t + 1) for t = 4s. 7z 4s+4 +2. From Lemma 86 Eqn. (B.5), the largest positive real root of f (z) = 0 equals (1c) Let t = 4s + 1. Let f denote the function f (z) = z 12s (z 12 16z 8 ) + 6z 8s+8 +
187
the largest positive real root of the characteristic equation of TMTR(t, t, t + 1, t + 1) for t = 4s + 1. Observe that f (2) > 0 while f (1.7) < 0 (It is easily checked that for s = 0, the largest positive real root of TMTR(1, 1, 2, 2) is 1.7424). We compute the derivative f (z) = 4z 4s+3 7s + 7 + (12s + 12)z 4s+4 + z 8s+4 ((3s + 3)(z 4 16) + 16) > 0 for z 2, so f is increasing on [2, ). Thus, the largest positive real root of f lies in From Eqn (7.6), the characteristic polynomial of TMTR(t, t + 1) for t = 4s + 1 is
1 1 given by q(z) = z z 4s+2 3 z z2 1 . For q(z) = 0, z = 0 or z 4s+2 3 z z2 1 = 0, of
4s+2 4s+2
the open interval (1.7, 2).
which the latter reduces to
z 4s =
z4
Thus, substituting Eqn (B.8) into f (z), we obtain f (z) = 3 4 4z 2 z

3
3 . 4z 2 3 4 4z 2 z
2
(B.8)
(z 12 16z 8 ) + 6z 8
+ 7z 4
z4
3 4z 2
+ 2.
We claim that f (z) < 0 for z (1.7, 2). Note that showing f (z) < 0 is equivalent to showing (z 4 4z 2 )f (z) > 0 for z (1.7, 2). But it is easily checked that (z 4 4z 2 )f (z) of f is strictly to the right of that of q. simplies to 8z 6 (z 2 4)2 (z 2 1) > 0, hence proving the claim. Thus, the largest root Let t = 4s + 3. Let f denote the function f (z) = z 12s+12 (z 4 16) + 24z 8s+8
10z 4s+4 + 1. From Lemma 86 Eqn. (B.7), the largest positive real root of f (z) = 0 equals the largest positive real root of the characteristic equation of TMTR(t, t, t + 1, t + 1) for t = 4s + 3. Also observe that f (2) > 0, while f (1.9) < 0 (It is easily checked that for s = 0, the largest positive real root of TMTR(1, 1, 2, 2) is 1.9473). We compute the derivative f (z) = 4z 4s+3 2(s + 1)(24z 4s+4 5) + z 8s+8 (z 4 (3s + 4) 48(s + 1)) > 0 for z 2, so f is increasing on [2, ]. Thus, the largest positive real root of f lies in
188
the open interval (1.9, 2). From Eqn (7.6), the characteristic polynomial of TMTR(t, t + 1) for t = 4s + 3 is given by q(z) = z z 4s+4 3 z
4s+4 2s+1 k=0
z 2k
1 = z z 4s+4 3 z z2 1 . For q(z) = 0, z = 0 or
4s+4
4s+4 1 3 z z2 1
= 0, of which the latter reduces to z 4s+4 = 3 . 4 (B.9)
z2
Thus, substituting Eqn (B.9) into f (z), we obtain f (z) = 3 24 z

3
(z 4 16) + 24
3 24 z
10
z2
3 4
+ 1.
We claim that f (z) < 0 for z (1.9, 2). Note that showing f (z) < 0 is equivalent to showing (z 2 4)f (z) > 0 for z (1.9, 2). It turns out that
(z 2 4)f (z) = 27(z 2 4)(z 2 + 4) + 246 + (z 2 4), = (4 z 2 ) 27z 2 + 107 + 246 > 0,
for z < 2, thus proving the claim. Thus, the largest root of f is strictly to the right of the largest root of q. Hence, capTMTR (t, t + 1) < capTMTR (t, t, t + 1, t + 1) for t = 4s + 1 or t = 4s + 3. Parts (2) through (6) of Proposition 76 can be proved in the same manner.
Bibliography
[1] R. L. Adler, D. Coppersmith, and M. Hassner, Algorithms for sliding block codesAn application of symbolic dynamics to information theory, IEEE Trans. Inform. Theory, vol. 29, no. 1, pp. 522, Jan. 1983. [2] K. Anim-Appiah and S. McLaughlin, Constrained-input turbo codes for (0, k) rll channels, in Conf. on Inform. Sciences and Systems, 1999. [3] , Towards soft output app decoding for nonsystematic nonlinear block codes, in Proc. 37th Allerton Conf. on Commun., Control, and Computing, 1999. [4] , Turbo codes cascaded with high-rate block codes for (0, k)-constrained channels, IEEE J. Select. Areas Commun., pp. 677685, Apr. 2001. [5] L. Bahl, J. Cocke, F. Jelinek, and J. Raviv, Optimal decoding of linear codes for minimizing symbol error, IEEE Trans. Inform. Theory, pp. 284287, Mar. 1974. [6] A. Bassalygo, Correcting codes with an additional property, Prob. Inform. Transm., vol. 4, no. 1, pp. 15, 1968. [7] R. Bellman, Introduction to Matrix Analysis, 2nd ed. McGraw-Hill, 1970. [8] J. Bergmans, Digital Baseband Transmission and Recording. Kluwer Academic Publishers, 1996. 189
190
BIBLIOGRAPHY
[9] C. Berrou, A. Glavieux, and P. Thitimajshima, Near shannon limit errorcorrecting coding and decoding, in Proc. IEEE Intl. Conf. Commun., May 1993, pp. 10641070. [10] W. Bliss, An 8/9 rate time-varying trellis code for high density magnetic recording, IEEE Trans. Magnetics, vol. 33, no. 5, pp. 27462748, Sept. 1997. [11] J. Campello de Souza, B. H. Marcus, R. New, and B. A. Wilson, Constrained systems with unconstrained positions, IEEE Trans. Inform. Theory, vol. 48, no. 4, pp. 866879, Apr. 2002. [12] P. Chaichanavong and B. H. Marcus, Optimal block-type-decodable encoders for constrained systems, IEEE Trans. Inform. Theory, vol. 49, no. 5, pp. 1231 1250, May 2003. [13] R. D. Cideciyan and E. Eleftheriou, Combined modulation/parity codes for magnetic recording, in Proc. IEEE ISIT, June 2003, p. 48. [14] , Codes satisfying maximum transition run and parity-check constraints, in Proc. IEEE Intl. Conf. on Commun., June 2004, pp. 635639. [15] R. D. Cideciyan, E. Eleftheriou, B. H. Marcus, and D. S. Modha, Maximum transition run codes for generalized partial response channels, IEEE J. Select. Areas Commun., vol. 19, no. 4, pp. 619634, Apr. 2001. [16] R. Cideciyan, F. Dolivo, R. Hermann, W. Hirt, and W. Schott, A prml system for digital magnetic recording, IEEE J. Select. Areas Commun., vol. 10, no. 1, pp. 3856, 1992. [17] T. H. Cormen, C. E. Leiserson, and R. L. Rivest, Introduction to Algorithms. The MIT Press, 1998. [18] A. Dasdan and R. K. Gupta, Faster maximum and minimum mean cycle algorithms for system performance analysis, IEEE Trans. Computer-Aided Design of Integrated Circuites and Systems, vol. 17, no. 10, Oct. 1998.
BIBLIOGRAPHY
191
[19] A. Dasdan, A. Mathur, and R. Gupta, Faster maximum mean cycle algorithms, preprint at http://www1.ics.uci.edu/gupta/projects.html, 2003. r [20] T. Duman and E. Kurtas, Comprehensive performance investigation of turbo codes over high density magnetic recording channels, in Proc. IEEE Globecom, 1999, pp. 744748. [21] J. L. Fan, Constrained Coding and Soft Iterative Decoding. Publishers, 2001. [22] J. L. Fan and A. R. Calderbank, A modied concatenated coding scheme, with applications to magnetic storage, IEEE Trans. Inform. Theory, vol. 44, no. 6, pp. 15651574, Jul. 1998. [23] J. L. Fan, T. L. Poo, and B. H. Marcus, Constraint gain, IEEE Trans. Inform. Theory, vol. 50, no. 9, pp. 19892001, Sep. 2004. [24] J. Fan and J. Cio, Constrained coding techniques for soft iterative decoders, in Proc. Globecom, Rio de Janeiro, Brazil, 1999, pp. 723727. [25] J. Fan, E. Kurtas, A. Friedmann, and S. McLaughlin, Low density parity check codes for magnetic recording, in Proc. 37th Allerton Conf. on Commun., Control, and Computing, 1999. [26] Z. Fang, G. Mathew, and B. Farhang-Boroujeny, Joint turbo channel detection and rll decoding for (1, 7) coded partial response recording channels, in Proc. IEEE Int. Conf. Commun., May 2003. [27] K. K. Fitzpatrick and C. S. Modlin, Time-varying mtr codes for high density magnetic recording, in Proc. IEEE Globecom Telecomm. Conf., Phoenix, AZ, Nov 1997, pp. 12501253. [28] S. Halevy, J. Chen, R. Roth, P. Siegel, and J. Wolf, Improved bit-stung bounds on two-dimensional constraints, IEEE Trans. Inform. Theory, vol. 50, pp. 824 838, 2004. Kluwer Academic
192
BIBLIOGRAPHY
[29] H. M. Hilden, D. G. Howe, and . J. E. J.Weldon, Shift error correcting modulation codes, IEEE Trans. Magnetics, vol. 27, no. 6, pp. 46004605, Nov. 1991. [30] H. D. L. Hollmann, A block-decodable (d, k) = (1, 8) runlength-limited rate 8/12 code, IEEE Trans. Inform. Theory, vol. 40, no. 4, pp. 12921296, July 1994. [31] , On the construction of bounded-delay encodable codes for constrained systems, IEEE Trans. Inform. Theory, vol. 41, no. 5, pp. 13541378, Sept. 1995. [32] , Bounded-delay-encodable, block-decodable codes for constrained systems, IEEE Trans. Inform. Theory, vol. 42, no. 6, pp. 19571970, Nov. 1996. [33] K. A. S. Immink, Block-decodable runlength-limited codes via look-ahead technique, Philips J. Res., vol. 46, no. 6, pp. 293310, 1992. [34] , A practical method for approaching the channel capacity of constrained channels, IEEE Trans. Inform. Theory, vol. 43, no. 5, pp. 13891399, Sept. 1997. [35] , Codes for Mass Data Storage Systems. The Netherlands: Shannon Foundation Publishers, 1999. [36] K. Immink, Reed Solomon Codes and the Compact Disc. ch. 3. [37] H. Ito, A. Kato, Z. Nagy, and K. Zeger, Zero capacity region of multidimensional run length constraints, The Electronic Journal of Combinatorics, vol. 6(1), no. R33, 1999. [38] R. Karabed, P. H. Siegel, and E. Soljanin, Constrained coding for binary channels with high intersymbol interference, IEEE Trans. Inform. Theory, vol. 45, pp. 17771797, Sept. 1999. IEEE Press, 1994,
BIBLIOGRAPHY
193
[39] R. M. Karp, A characterization of the minimum cycle mean in a digraph, Disc.Math, vol. 23, pp. 309311, 1978. [40] A. Kato and K. Zeger, On the capacity of two-dimensional run-length constrained channels, IEEE Trans. Inform. Theory, no. 45, pp. 15271540, 1999. [41] W. Kautz, Fibonacci codes for synchronization control, IEEE Trans. Inform. Theory, pp. 284292, Apr. 1965. [42] J. Li, E. Kurtas, K. Narayanan, and C.N.Georghiades, On the performance of turbo product codes and ldpc codes over partial response channels, in IEEE Intl. Conf. Commun, 2001. [43] D. Lind and B. Marcus, An Introduction to Symbolic Dynamics and Coding. Cambridge University Press, 1995. [44] R. Lynch, E. M. Kurtas, A. Kuznetsov, E. Yeo, and B. Nikolic, The search for a practical iterative detector for magnetic recording, IEEE Trans. Magnetics, vol. 40, no. 1, pp. 213218, Jan. 2004. [45] D. J. C. Mackay and R. Neal, Near shannon limit performance of low density parity check codes, Electronic Letters, vol. 33, pp. 457458, 1997. [46] M. Mansuripur, Enumerative modulation coding with arbitrary constraints and post-modulation error correction coding and data storage systems, in Proc. SPIE, 1991, pp. 7286. [47] B. H. Marcus, R. M. Roth, and P. H. Siegel, Handbook of Coding Theory. Amsterdam, The Netherlands: Elsevier, 1998, ch. 20. [48] L. McPheters, S. McLaughlin, and K. Narayanan, Precoded prml, seriel concatenation and iterative (turbo) decoding for magnetic recording, in Intl. Conf. on Magnetics, 1999. [49] L. McPheters and S. McLaughlin, Turbo-coded optical recording channels with dvd minim mark size, IEEE Trans. Magnetics, vol. 38, no. 1, pp. 298302, Jan. 2002.
194
BIBLIOGRAPHY
[50] B. E. Moision, P. H. Siegel, and E. Soljanin, Distance-enhancing codes for digital recording, IEEE Trans. Magnetics, vol. 34, no. 1, pp. 6974, Jan. 1998. [51] J. Moon and B. Brickner, Maximum transition run codes for data storage systems, IEEE Trans. Magnetics, vol. 32, no. 5, pp. 39923994, Sept. 1996. [52] E. Ordentlich and R. Roth, Independent sets in regular hypergraphs and multidimensional runlength-limited constraints, SIAM J. Disc. Math., vol. 17, pp. 615623, 2004. [53] A. Patapoutian and P. V. Kumar, The (d, k) subcode of a linear block code, IEEE Trans. Inform. Theory, vol. 38, pp. 13751382, Jul. 1992. [54] P. Perry, Runlength-limited codes for single error detection in the magnetic recording channel, IEEE Trans. Inform. Theory, vol. 41, no. 3, pp. 809814, May 1995. [55] T. L. Poo, P. Chaichanavong, and B. H. Marcus, Trade-o functions for constrained systems with unconstrained positions, in Proc. IEEE Int. Symp. Inform. Theory, July 2004. [56] , Tradeo functions for constrained systems with unconstrained positions, accepted in IEEE Trans. Inform. Theory, April 2005. [57] T. L. Poo and B. H. Marcus, Optimal code rates and code congurations for maximum-transition-run and runlength-limited constrained systems with unconstrained positions, journal manuscript in preparation, 2005. [58] , Time-varying maximum transition run constraints, in accepted in IEEE Int. Symp. Inform. Theory, 2005. [59] , Time-varying maximum transition run constraints, journal manuscript in preparation, 2005. [60] R. Pyndiah, Near-optimum decoding of product codes: block turbo codes, IEEE Trans. Commun., vol. 46, Aug. 1998.
BIBLIOGRAPHY
195
[61] E. M. Reingold, J. Nievergelt, and N. Deo, Combinatorial Algorithms: Theory and Practice. Prentice-Hall, Inc., 1977. [62] W. Ryan, Concatenated codes for class iv partial response channels, IEEE Trans. Commun., vol. 49, no. 3, pp. 445454, Mar. 2001. [63] C. E. Shannon, A mathematical theory of communications, Bell Sys. Tech. Journal, vol. 27, pp. 379423, 1948. [64] H. Song, R. Todd, and J. Cruz, Low density parity check codes for magnetic recording channels, in Proc. IEEE Int. Conf. Commun., Sept. 2000. [65] T. Souvignier, A. Friedmann, M. Oberg, P. Siegel, R. Swanson, and J. Wolf, Turbo decoding for pr4: parallel vs. serial concatenation, in Proc. IEEE Intl. Conf. on Commun., 1999, pp. 16381642. [66] A. J. van Wijngaarden and K. A. S. Immink, Maximum runlength-limited codes with error control capabilities, IEEE J. Select. Areas Commun., vol. 19, no. 4, pp. 602611, Apr. 2001. [67] Z. Wu, Coding and Iterative Detection for Magnetic Recording Channels. Kluwer Academic Publishers, 2000. [68] Z. Wu and J. Cio, Low-complexity iterative decoding with decision-aided equalization for magnetic recording channels, IEEE J. Select. Areas Commun., vol. 19, no. 4, pp. 699788, Apr. 2001. [69] E. Yamada, T. Iwaki, and T. Yamaguchi, Turbo decoding with runlength limited code for optical storage, Jpn. J. Appl. Phys., vol. 41, no. Part 1, No. 3B, pp. 17531756, Mar. 2002. [70] E. Yeo, P. Pakzad, B. Nikolic, and V. Anantharam, Vlsi architectures for iterative decoders in magnetic recording channels, IEEE Trans. Magnetics, vol. 37, no. 2, pp. 748755, Mar. 2001.

Thesis

Transféré par

Informations du document

Description originale:

Copyright

Formats disponibles

Partager ce document

Partager ou intégrer le document

Options de partage

Avez-vous trouvé ce document utile ?

Ce contenu est-il inapproprié ?

Droits d'auteur :

Formats disponibles

Thesis

Transféré par

Droits d'auteur :

Formats disponibles

OPTIMAL CODE RATES FOR CONSTRAINED SYSTEMS WITH UNCONSTRAINED POSITIONS: AN APPROACH TO COMBINING ERROR CORRECTION CODES WITH

MODULATION CODES FOR DIGITAL STORAGE SYSTEMS

Tze-Lei Poo June 2005

c Copyright by Tze-Lei Poo 2005 All Rights Reserved

Brian H. Marcus Principal Adviser

John T. Gill III

Approved for the University Committee on Graduate Studies.

Soft Decoding in Concatenated Schemes . . . . . . . . . . . . . . . .

Optimal Code Rates over G . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Periodic Tradeo Functions . . . . . . . . . . . . . . . . . . . viii

Relation to Previous Denitions . . . . . . . . . . . . . . . . .

Tradeo Functions at the Maximum Insertion Rate . . . . . . . . . .

General Upper Bounds . . . . . . . . . . . . . . . . . . . . . . . . . . 111

Performance of some Upper and Lower Bounds . . . . . . . . . . . . 113

8 Future Work 8.1 8.2

A Long Proofs for Chapter 5

Tight forms of m for TMTR for p = 2, 3, 4 . . . . . . . . . . . . . . . 122

3.1 4.1 4.2 4.3 5.1 5.2

5.3 5.4 5.5 5.6 5.7 5.8

Shannon cover for TMTR(1, 2, . . . , p). . . . . . . . . . . . . . . . . . . 130

Shannon cover for TMTR(0, 1, . . . , p 1). . . . . . . . . . . . . . . . 130

A Typical Magnetic Storage System

Channel. The magnetic recording channel introduces intersymbol interference

1.1. A TYPICAL MAGNETIC STORAGE SYSTEM

user u bits {0, 1}

Binary signal mapper

x symbols {1, +1}

1.2. CONSTRAINED CODING FOR RECORDING CHANNELS

Constrained Coding for Recording Channels

1.2. CONSTRAINED CODING FOR RECORDING CHANNELS

1.3. CODE CONCATENATION

parity w(p) v ECC Encoder C2 Encoder

w(m) Hard ECC Channel Decoder v (p) Decoder w Decoder C2

Figure 1.7: Reverse concatenation scheme .

respectively, then the overall rate of standard concatenation is given by

1.4. SOFT DECODING IN CONCATENATED SCHEMES

Soft Decoding in Concatenated Schemes

Soft Demodulation Codes

w(m) Channel Soft Channel

w(m) Soft ECC Decoder

parity w(p) v ECC Encoder C2 Encoder

v (p) Decoder w Decoder C2

Figure 1.8: Soft Decoding in Reverse Concatenation.

1.4. SOFT DECODING IN CONCATENATED SCHEMES

since L modulated bits

The overall rate of this insertion scheme

by the capacity cap(MTR(j a)), it is desirable to chose small values of a, such as

1.4. SOFT DECODING IN CONCATENATED SCHEMES

Figure 1.10: Schematic diagram of bit insertion. a = 1, for higher rates.

1.4. SOFT DECODING IN CONCATENATED SCHEMES

to 1 , where is the insertion rate.

j 1 ( j+1 , j+1 cap(MTR(0))),

cap(MTR(j)) cap(MTR(j a)) a)) 2 Q P 1

L cap(MTR(j L+1 j+1a cap(MTR(j j+1

insertion scheme is contained within the framework of standard bit-stung.

1.4. SOFT DECODING IN CONCATENATED SCHEMES

h() for MTR(4) Simple bit insertion

It is straightforward to show by direct computation that

1.5. SUMMARY OF CONTRIBUTIONS

with constrained systems with unconstrained positions. This has been

1.5. SUMMARY OF CONTRIBUTIONS

compared it to previous denitions. We give a formal denition of the tradeo

1.5. SUMMARY OF CONTRIBUTIONS

Analyzed time-varying MTR constraints and related them to MTR con-