Signals, Information and Communications

Examples, Exercises and Extensions (X 3)

Erik G. Larsson
March 7, 2013

SOME HANDY FORMULAS

Trigonometric Identities

cos2 (x) + sin2 (x) =1

sin(x + y) = sin(x) cos(y) + cos(x) sin(y)
cos(x + y) = cos(x) cos(y) sin(x) sin(y)
sin(2x) =2 sin(x) cos(x)
cos(2x) = cos2 (x) sin2 (x) = 1 2 sin2 (x) = 2 cos2 (x) 1
1
sin(x) cos(y) = (sin(x + y) + sin(x y))
2
1
sin(x) sin(y) = (cos(x y) cos(x + y))
2
1
cos(x) cos(y) = (cos(x + y) + cos(x y))
2

Fourier Transform
Suppose x(t) and X( f ) constitute a Fourier transform pair,
X( f ) =F {x(t)} =

x(t) =F 1 {X( f )} =

x(t)e j2 f t dt,

and

X( f )e j2 f t d f .
1

Some Handy Formulas

Then
F {x(t)} = X( f )
 
f
1
,
F {x(at)} = X
a
a

a>0

F {x(t T )} = e j2 f T X( f )
1
F {x(t) cos(2 fct)} = (X( f fc ) + X( f + fc ))
2
1
F {x(t) sin(2 fct)} = (X( f fc ) X( f + fc ))
2j
If X1 ( f ) = F {x1 (t)} and X2 ( f ) = F {x2 (t)} are two Fourier transform pairs, then
F {(x1 x2 )(t)} = X1 ( f )X2 ( f )
F {x1 (t)x2 (t)} = (X1 X2 )( f )

x1 (t)x2 (t) dt

X1 ( f )X2 ( f ) d f

Some basic transform pairs:

x(t) = cos(2 fct)

x(t) = sin(2 fct)

x(t) = sinc(t)

1, |t| 1
2
x(t) =

0, otherwise

x(t) = e|t|
(
et , t 0
x(t) =
0,
otherwise
x(t) =

1
1 + t2

1
X( f ) = ( ( f fc ) + ( f + fc ))
2
1
X( f ) = ( ( f fc ) ( f + fc ))
2j

1, | f | 1
2
X( f ) =

0, otherwise
X( f ) = sinc( f )

X( f ) =

2
1 + 4 2 f 2

X( f ) =

1
1 + j2 f

X( f ) = e2 | f |

EXAMPLES, EXERCISES AND

EXTENSIONS

This document contains examples, exercises and extensions to supplement the book
Signals, Information and Communications.
References preceded by E are to this document. All other references are to the
book, printing December 2012.
The exercises are sorted chapter-by-chapter but apart from that they come in no
particular order.
Difficulty of exercises:
:

straightforward
normal
: hard
: small project
:

All exercises should be solvable by hand, without a calculator or computer.

If you cannot compute an answer exactly in closed form, a rough approximate

numerical answer with 1-2 significant digits, or a bound, or at least an order-ofmagnitude estimate, is sufficient. If you give an approximation, explain how accurate it is and why.
For tasks that involve more elaborate calculations, also explain in what ways you
can check that the final answer is reasonable.

This document does not seem to display or print properly in the Preview PDF
viewer under Mac OS. On Mac OS, use Adobe Reader instead. Unfortunately, this
document also does not seem to display correctly on the iPad.
3

Examples, Exercises and Extensions

Chapter 1
1. Energy retained in truncation. (abc , def , g )
Consider a signal x(t) and suppose we truncate it by setting,
(
x(t), |t| t0
x(t)
=
,
0,
otherwise

(E-1)

for some t0 .
For each of the following signals,

1, |t| 1
2
(a) x(t) =

0, otherwise
(b) x(t) = e|t|
(
cos(100 t),
(c) x(t) =
0,

|t| 100
otherwise

(d) x(t) = sinc(t)

(e) x(t) = sinc2 (t)
(f) x(t) = sinc(t 100) + sinc(t + 100)

(g) x(t) = F 1 {X( f )}, where

X( f ) =
for some F.

sinc( f ),

0,

F
,
2
otherwise
|f|

(E-2)

do the following:
f )|. When applicable, mark all zero crossings
(i) Sketch x(t), |X( f )|, x(t)
and |X(
carefully.
(ii) Determine (roughly) how small t0 can be in order for the relative energy lost
in the truncation,
R 2
R 2
x (t) dt x (t) dt
R
,
x

2 (t) dt

to be less than 1%. Summarize the results as rules of thumb.

(E-3)

Examples, Exercises and Extensions

2. Definitions of band-limitation. (a , b , c )
In the text we have introduced the notions of strict band-limitation and approximate
band-limitation. The concept of approximate band-limitation was not defined exactly, but had to be interpreted in the sense that when expanding x(t) as in (1.17)
on page 12, the energy of x(t) that is not captured by the expansion is negligible.
In practice, most signals are not strictly band-limited and it is useful to have a
more precise notion of approximately versus strictly band-limited. Three such
measures are
(i) The %-bandwidth, B . This is the bandwidth within which a fraction of
the signal energy is contained:
R B

2
B |X( f )| d f
R
2
|X( f )| d f

= .

(E-4)

(ii) The 3-dB bandwidth, B3 dB . This is the largest value of f , f 0, for which
r
1
|X( f )| =
max |X( f )|.
(E-5)
2 f , f 0
(iii) The effective bandwidth, defined as
sR
Beff =

2
2
f |X( f )| d f
R
2
|X( f )| d f

(E-6)

This bandwidth measure is reminiscent of the standard deviation of a probability density function. It naturally arises in some applications, see, for example, Section 4.6.
Compute B99% , B3 dB and Beff (if they exist!) for
(a) x(t) = sinc(t)

1, |t| 1
2
(b) x(t) =

0, otherwise
(
1 |t|, |t| 1
(c) x(t) =
0,
otherwise
Comment on the relation between B99% , B3 dB and Beff .

Examples, Exercises and Extensions

3. Definitions of time-limitation. (a , bc )
Starting from the definitions of bandwidth in Exercise 2, give corresponding definitions of
(a) time-duration T99% , for a signal that is symmetric in the sense that
|x(t)| = |x(t)|,

(E-7)

(b) time-duration T99% for a general signal

(c) time-duration Teff
4. Sampling an audio signal.
An audio signal of nominal bandwidth 20 kHz is truncated to the length T by using
a rectangular window, similarly to in Exercise 1. How fast is it appropriate to
sample this signal, if
(a) T = 0.5 millisecond
(b) T = 1 second
5. Dimensionality and sampling. (abcdef , g )
For each of the following signals,
(a) x(t) = sinc(t)
 t 
(b) x(t) = sinc
200

1, |t| 1
2
(c) x(t) =

0, otherwise
(
1 |t|, |t| 1
(d) x(t) =
0,
otherwise
(e) x(t) = sinc(t + 5000) + sinc(t 5000)
(
1 + cos( t) + cos(2 t) + cos(3 t),
(f) x(t) =
0,
(
sinc(t), |t| 3
(g) x(t) =
0,
otherwise

|t| 100
otherwise

Examples, Exercises and Extensions

do the following:
(i) Sketch x(t) and |X( f )| (for ag only).

(ii) Determine, approximately, the required sampling rate ( fs ) for sampling in

the time domain. Also, comment on what anti-alias filtering that would be
necessary. If the signal is not strictly band-limited, use B99% (see Exercise 2)
as a measure of its bandwidth.
(iii) If the signal is sampled in the frequency domain, how far apart should the
samples be taken? If the signal is not strictly time-limited, use T99% as a
measure of its time-duration.
(iv) Determine (roughly) the dimensionality of the signal. If the signal is not
strictly time-limited or band-limited, use T99% and B99% .
(v) Write out the expansions of the forms (1.17), (1.18), (1.38) and (1.39).
6. Direct demodulation, reversed.
Consider direct demodulation as presented in Section 1.5.1. Give a method to obtain x(t) from y(t) in (1.48) on page 21. Draw a block diagram of the method and
sketch all relevant Fourier transforms.
7. I/Q representation. (a , bc )
For each of the following signals, give the I and Q components xI (t) and xQ (t), as
well as the amplitude and phase, x(t)
and x (t). In all cases, fc = 45 Hz.
(a) x(t) = sin(100 t)
(b) x(t) = cos(110 t + /3)
(c) x(t) = sin(100 t) + cos(120 t + /5) (no need to compute x (t) here)
8. I/Q representation, delays.
Consider a narrowband signal x(t) defined by its I/Q components

1, |t| 1
2
xI (t) =

0, otherwise
(
1, 0 t 1
xQ (t) =
0, otherwise
and suppose fc = 100 Hz.

(E-8)

(E-9)

Examples, Exercises and Extensions

(a) Sketch xI (t), xQ (t), x(t),

x (t), and |X( f )|.

(b) Sketch x(t).
(c) Draw x(t) carefully around t = 0. Comment on the behavior of x(t) and x (t)
around t = 0.
y (t) and |Y ( f )|, to
(d) Let y(t) = x(t 1/400). Sketch y(t), yI (t), yQ (t), y(t),

within the approximations we use for small time-delays in Section 1.5.2.

9. I/Q representation, local symmetry.
A narrowband signal has the Fourier transform X( f ). For each of the signals defined as follows,

1 | f 11|, 10 f 12,
(a) X( f ) = 1 | f + 11|, 12 f 10,

0,
otherwise

10 f 11,

f 10,
(b) X( f ) = f 10, 11 f 10,

0,
otherwise
do the following:

(i) Sketch X( f ).
(ii) Suppose you want to form x(t) from its I and Q components xI (t) and xQ (t)
using an I/Q modulator, see Figure 1.8(a) on page 23. You can choose fc
freely.
Is there any way of choosing fc so that x(t) can be generated from only the I
or the Q component, that is, one of xI (t) or xQ (t) (but not both) is zero for all
t? Explain why, or why not.
10. Linearity of the I/Q representation.
Suppose x(t) and y(t) have the I/Q representations (xI (t), xQ (t)) and (yI (t), yQ (t)),
respectively, for some given fc . Give the I/Q representation of the signal
z(t) = ax(t) + by(t).

(E-10)

11. Tampering with an I/Q modulated signal.

Suppose xI (t) and xQ (t) are the I and Q components of a narrowband signal x(t),
with carrier frequency 50 Hz, see (1.49) on page 22. Let
y(t) = x(t) cos(20 t).

(E-11)

Examples, Exercises and Extensions

2 cos(2 fct + )

cos(2 fct)

xI (t)

xI (t)

LP
x(t)

x(t)
xQ (t)

xQ (t)

LP
sin(2 fct)

2 sin(2 fct + )

(a) I/Q modulator with carrier frequency fc .

(b) I/Q demodulator with carrier frequency fc and a

phase offset of radians.

Figure E-1. I/Q modulator/demodulator pair with a phase mismatch.

(a) Express yI (t) and yQ (t) in terms of xI (t) and xQ (t).
and x (t).
(b) Express y(t)
and y (t) in terms of x(t)
12. I/Q modulator/demodulator with oscillator phase mismatch.
Consider the I/Q modulator/demodulator pair in Figure E-1. Here, both the modulator and the demodulator are tuned to the same carrier frequency, fc . However, the
oscillator of the demodulator is not phase synchronized to that of the modulator,
and there is a phase offset of radians. This situation frequently arises in practice,
when the modulator and demodulator are driven by different crystals (they could be
kilometers apart). Moreover, delaying x(t) by a small amount also has the equivalent effect of introducing a phase offset. Hence, the output of the demodulator is
not equal to the input of the modulator, that is,
xI (t) 6= xI (t),

xQ (t) 6= xQ (t).

(E-12)

(a) Give an expression that relates xI (t) and xQ (t) to xI (t) and xQ (t).
(b) Suppose that is known, or that it can be determined by some appropriate
procedure at the demodulator. Propose a mechanism (give an explicit formula)
for recovering xI (t) and xQ (t) from xI (t) and xQ (t).
13. I/Q modulator/demodulator with oscillator frequency mismatch.
Consider the I/Q modulator/demodulator pair in Figure E-2. Here, the oscillator
frequency of the modulator, f1 , is not necessarily equal to that of the demodulator,

10

Examples, Exercises and Extensions

2 cos(2 f2t)

cos(2 f1t)

xI (t)

xI (t)

LP
x(t)

x(t)
xQ (t)

xQ (t)

LP
sin(2 f1t)

2 sin(2 f2t)
(b) I/Q demodulator with carrier frequency f2 .

(a) I/Q modulator with carrier frequency f1 .

Figure E-2. I/Q modulator/demodulator pair with a carrier frequency mismatch.

f2 . Similarly to in Exercise 12, this situation frequently arises in practice when the
modulator and demodulator are driven by different crystals. Hence, the output of
the demodulator is not equal to the input of the modulator, that is,
xI (t) 6= xI (t),

xQ (t) 6= xQ (t).

(E-13)

(a) Give an expression that relates xI (t) and xQ (t) to xI (t) and xQ (t).
(b) Suppose that f2 f1 is known, or that it can be determined by some appropriate
procedure at the demodulator. Propose a mechanism (give an explicit formula)
for recovering xI (t) and xQ (t) from xI (t) and xQ (t).
14. Energy of narrowband signal.
Let x(t) be a narrowband signal with energy Ex as defined in (1.1) on page 3. Let fc
be the carrier frequency, and let xI (t) and xQ (t) be the I and Q components of x(t) as
defined in Section 1.5.2. Furthermore, suppose that xI (t) and xQ (t) are band-limited
to [ fc /2, fc /2]. Show that
Ex =

1
ExI + ExQ .
2

(E-14)

Equation (E-14) has no particular physical meaning, it just arises as a consequence

of the way the I/Q representation is defined. But it is useful sometimes when doing
calculations.
15. Alternative derivation of the results of Section 1.4.2.

11

Examples, Exercises and Extensions

Consider a signal x(t) that is contained in the time-frequency space [0, T ] [B, B].
Suppose the signal is passed through a stable, linear time-invariant (LTI) system H
with impulse response h(t). Furthermore , suppose that T is much larger than the
time constants of the system H .
(a) Using the sinus-in sinus-out property of stable LTI systems, argue that
n  n o
n  n o
n (t) + Im H
n (t)
(h n )(t) Re H
n T n o
n T n o
(E-15)
(h n )(t) Im H
n (t) + Re H
n (t).
T
T

(b) Explain how (1.44)(1.45), see page 19, follow from (E-15).

16. Linear filtering of narrowband signals, alternative approach.

Consider again a narrowband signal x(t) that passes through an LTI filter with impulse response h(t), as in Section 1.5.2. Assume that xI (t) and xQ (t) are bandlimited to [ fc /2, fc /2]. Establish the relations (1.65) on page 27 by working in
the time-domain, i.e., write h(t) and x(t) on the form of (1.49) on page 22, and
compute the convolution (h x)(t).
Chapter 2
17. Dimensionality and sampling, PAM signals. (abc , d )
For each of the following pulse-amplitude modulated signals,

1, |t| 1
sn p(t nTs ), |t| 11

2 .
(a) x(t) = n=
, p(t) =

0, otherwise
0,
otherwise

(

s
p(t

nT
),
|t|

11
1 |t|, |t| 1
n
s

, p(t) =
.
(b) x(t) = n=

0,
otherwise

0,
otherwise
5

(c) x(t) =

(d) x(t) =

sn p(t nTs ),

n=5

sn p(t nTs ),

p(t) = sinc(t).

n=

|t| 10

0,

otherwise

p(t) = sinc(t).

Examples, Exercises and Extensions

12

do the following:
(i) Sketch x(t).
(ii) Sketch roughly the shape of |X( f )|.
(iii) Explain the requirements (sample rate) for sampling in the time domain.
(iv) Estimate the dimensionality of x(t). If the signal is not strictly time-limited
or band-limited, use T99% and B99% (see Exercise 2).
In all cases, the symbol time is Ts = 2 seconds and sn is an arbitrary sequence of
1.
18. Nyquist criterion. (abcdef , ghi )
Denote with B (t) the impulse response of the ideal lowpass filter HBLP {}. Consider a communication link with pulse-amplitude modulation, cf. Figure 2.3 on
page 35. For each of the following configurations,
(a) p(t) = sinc(t), h(t) = (t) = B (t), B = 3 Hz
(b) p(t) = sinc(t), h(t) = (t) = B (t), B = 1/2 Hz
(c) p(t) = sinc(t), h(t) = (t) = B (t), B = 0.1 Hz

1, |t| 1
2 ,
(d) p(t) =
h(t) and (t) are absent

0, otherwise
(
1 |t|, |t| 1
(e) p(t) =
,
h(t) and (t) are absent
0,
otherwise
(f) p(t) = sinc2 (t), h(t) = (t) = B (t), B = 1 Hz
(g) p(t) = sinc2 (t), h(t) = (t) = B (t), B = 1/2 Hz

1, |t| 1
2 ,
h(t) = (t) = B (t), B = 1 Hz
(h) p(t) =

0, otherwise
(
1 |t|, |t| 1
(i) p(t) =
,
h(t) = (t) = B (t), B = 1 Hz
0,
otherwise
determine for what values of Ts (if any) the Nyquist criterion is satisfied?

Examples, Exercises and Extensions

13

19. Nyquist criterion with shifted sampling.

Consider the communication link with pulse-amplitude modulation, in Figure 2.3
on page 35. Suppose the samples are taken at nTs + , where is a timing offset.
Then the noise-free part of z(nTs + ) depends on sm only for m = n if
(
1, n = 0
(E-16)
r(nTs + ) =
0, n 6= 0
for some , compare (2.13) on page 38. Modify (2.16) on page 39 accordingly.
20. Pulse shapes for communications, practical aspects.
Consider again the pulse shapes in Exercise 18. Discuss their suitability for use in
practice, with regard to sensitivity to a sampling time mismatch.

Chapter 3
21. Quantization SNR.
Consider an audio signal of length 2 hours and which occupies a bandwidth of 25
kHz. The signal is to be sampled in a way that no appreciable aliasing occurs, and
then quantized with uniform quantization, and finally stored in a memory. The SNR
desired after quantization must be at least 60 dB. Without any compression, how
much memory is needed?
22. SNR.
A signal y(t) = x(t) + w(t) has SNR 22 dB.
(a) What is the SNR in the signal 3y(t)?
(b) What is the SNR in the signal 2x(t) + w(t)?
23. Waveform SNR.
A lowpass waveform x(t) lives in a time-frequency space of time-duration 1 second
and bandwidth 100 Hz. Suppose x(t) represents a signal in an electrical circuit, as
defined in Section 3.3.1, and has energy 8 pJ.
(a) If x(t) is observed in the presence of additive white noise with temperature
Teff = 300 Kelvin, and the observed signal is band-limited and time-limited to
the same window that x(t) occupies, what is then the SNR, expressed in dB?
(Boltzmanns constant is kB 1.4 1023 J/K.)

Examples, Exercises and Extensions

14

(b) Suppose x(t) instead occupies 200 Hz, but has the same energy. What is the
SNR now?
(c) Generalize the calculation in (a)(b) and give the intuition behind. What is the
general conclusion?
24. Equivalent definitions of white noise.
Consider a random signal x(t). Prove that the coefficients xn in the expansion (1.3)
on page 7 are uncorrelated and have the same variance, if and only if the coefficients
x n , x n , in the frequency-domain expansion (1.31) on page 16 are uncorrelated and
have the same variance. You can assume that all coefficients have zero mean.
Hint: Express x(t) both in terms of {n (t)} and in terms of { n (t), n (t)}. Then
derive a linear relationship between {xn } and {xn , x n }.
Chapter 4
25. Dolphins.
A dolphin needs to navigate with a precision of two centimeters. Estimate the
frequency range of the sounds that it must produce. The speed of sound in water is
slightly less than 1500 meters per second.
26. Stepped-frequency waveform.
Compute an expression for the ambiguity function rx ( ) of a stepped-frequency
waveform, see (4.15) on page 69, for arbitrary N. Draw rx ( ). Comment on the
shape of rx ( ). Is it desirable?
27. An echo.
Consider linear, time-invariant system H which for any input signal x(t) yields the
output
y(t) = H {x(t)} = 3x(t 5) + x(t a).

(E-17)

The system H is fed with a signal x(t) that is designed suitable for time-delay
estimation.
(a) Suppose a = 7 and that z( ) is computed according to (4.5) on page 63. Sketch
the result.
(b) Suppose that we do not know a and that z( ) is computed according to (4.5) on
page 63. Explain how z( ) can be used to determine a.

15

Examples, Exercises and Extensions

(c) Suppose that both a and x(t) are unknown to us and that we compute the correlation between y(t) with itself:
z( ) ,

y(t)y(t + ) dt.

(E-18)

Explain how z( ) can be used to determine a. (You can assume that a > 5.)
28. Signals for time-delay estimation.
For each of the following signals,

1, |t| 1
2
(a) x(t) =

0, otherwise

sn p(t n), |t| 10.5 ,

(b) x(t) = n=

0,
otherwise
(
sinc(t), |t| 10
(c) x(t) =
0,
otherwise
(
cos(20 t), |t| 100
(d) x(t) =
0,
otherwise

1, |t| 1
100
p(t) =

0, otherwise

do the following:
(i) Comment on the bandwidth occupancy.
(ii) Sketch the ambiguity function.
(iii) Comment on the suitability (pros and cons) of the signal for use in a timedelay estimation application.
29. A time-frequency uncertainty relation.
In Section 4.5 we stated (somewhat imprecisely) and used an uncertainty relation,
which states that the bandwidth occupancy of a signal is approximately inversely
proportional to its time-duration. (This was also mentioned in Section 1.2.) To state
and prove a precise version of this relation, more exact definitions of band-limited
and time-limited are needed. But we can show one, rather weak, version of the
uncertainty relation as follows. Consider a signal x(t) and let X( f ) = F {x(t)}.

16

Examples, Exercises and Extensions

(a) Let

|x(t)| dt

,
maxt |x(t)|
R
|X( f )| d f
.
,
max f |X( f )|

(E-19)
(E-20)

Explain why and can be interpreted as measures of the time duration and
bandwidth of x(t).
(b) Explain why
max |x(t)|
t

|X( f )| d f .

(E-21)

(c) Explain why

max |X( f )|
f

|x(t)| dt.

(E-22)

(d) Use the result of (b)(c) to show that

1
.

(E-23)

30. Waveforms competing in a [0, T ] [B, B] space.

Consider waveforms x(t) that live in a [0, T ] [B, B] space. Suppose we are interested in waveforms that are orthogonal (orthonormal actually, but we do not need
that), that is,
(
Z T
1, k = n
xk (t)xn (t) dt =
.
(E-24)
0, otherwise
0

In the text, we have seen several examples of such orthogonal waveforms, especially n (t) in Section 1.3, and n (t), n (t) in Section 1.4.
(a) Give another example of a set of waveforms xn (t) that are orthogonal, i.e.,
satisfy E-24.
(b) Is it possible for the [0, T ] [B, B] space to host more than 2BT orthogonal
waveforms? If not, explain why this is so.

17

Examples, Exercises and Extensions

(c) In the design of a satellite navigation system, two satellites use the waveforms u(t) and v(t), respectively. Both u(t) and v(t) must live in the same
[0, T ] [B, B] space. Explain why it is desirable that u(t) and v(t) are mutually orthogonal. Also, explain why it is desirable that they are mutually orthogonal when time-shifted 1, 2, . . . , K time units, where K is an integer.
(d) How many different waveforms u(t) and v(t) are there that have this property?
(e) Discuss the implications for the design of navigation systems in practice.
Chapter 5
Some Useful Facts

21.44 e 2.72
22.59 6

21.59 3

26.64 100

31. Compressibility.
A die with 6 sides is rolled. The first 4 faces have probabilities 12.5% of coming
up, each, and the fifth and sixth faces have probability 25% of coming up.
(a) Viewed as a source, what is the entropy of the die?
(b) How much smaller or larger is the answer of (a) compared to the corresponding
answer for a fair die?
32. Runlength coding.
For each of the following strings,
(a) 0111111011101
(b) aacbbbbddaaac
describe what the alphabet is and carry out runlength encoding.
33. Huffman code.
Consider a source that generates symbols with the following probabilities:
P(x = a) =0.04
P(x = c) =0.16
P(x = e) =0.08

P(x = b) =0.02
P(x = d) =0.62
P(x = f) =0.08.

(E-25)

18

Examples, Exercises and Extensions

(a) Construct a Huffman code tailored to this source.

(b) Compare the expected codeword length of the Huffman code to the source entropy.
34. Huffman code, again.
Consider a source that generates symbols with the following probabilities:
P(x = a) =1/128
P(x = c) =61/128
P(x = e) =1/64

P(x = b) =1/128
P(x = d) =61/128
P(x = f) =1/64.

(E-26)

(a) Construct a Huffman code tailored to this source.

(b) Compare the expected codeword length of the Huffman code to the source entropy.
35. Huffman code, one more time.
Consider a source that generates symbols with the following probabilities:
P(x = a) =0.25
P(x = c) =0.25

P(x = b) =0.25
P(x = d) =0.25

(E-27)

(a) Construct a Huffman code tailored to this source.

(b) Compare the expected codeword length of the Huffman code to the source entropy. Why is the result natural?
36. A large text file.
A text file X.txt consisting of ten million characters aabcbaaab. . . was generated. Each character of the file was generated at random, independently of all
others. To generate each character, a fair coin was flipped. If it turned up heads,
then the character was taken to be a. Otherwise, a biased coin was flipped, such
that the probability of seeing a head is p = 1/64. If this coin came up heads, the
character was taken to be b, otherwise it was taken to be c.
Another text file of ten million characters, Y.txt, was generated by throwing a fair
three-sided die, that is letting each character be either a, b or c, with probability
1/3, and independently of all other characters.

Examples, Exercises and Extensions

19

(a) When X.txt and Y.txt were saved with ASCII encoding1 , both files had
the size 10,000,000 bytes2 . Explain why.
(b) If X.txt were compressed to the theoretical limit, how many binary digits
would be needed to represent it? How many bytes does this amount to?
(c) If Y.txt were compressed to the theoretical limit, how many binary digits
would be needed to represent it? How many bytes?
(d) Suggest a simple method for reducing X.txt and Y.txt to 2/8=25% of their
original size, that is 2,500,000 bytes, without loss.
(e) After running gzip3 on the file X.txt, its size was reduced to 1,673,113
bytes. In view of (a), (b) and (d), did gzip do a good job or not?
(f) After running gzip on the file Y.txt, its size was reduced to 2,291,129 bytes.
In view of (a), (c) and (d), did gzip do a good job or not?
(g) Compare the results of (e) and (f) and comment on the result.
(h) If you run gzip again on the gzip-compressed version of X.txt (or Y.txt),
what do you expect to happen and why?
37. Another large text file.
A large text file Z.txt was generated by repeating the following procedure one
million times. A fair coin is flipped. If it turned up heads, then a is written to the
file. Otherwise, another fair coin is flipped. If this coin comes up heads, then b b
(ten bs) are written to the file, otherwise c c (twenty cs) are written to the file.
(a) When Z.txt was saved with ASCII encoding, it had the size 7,988,822 bytes.
Explain why.
(b) If Z.txt were compressed to the theoretical limit, how many binary digits
would be required to represent it?
(c) Suggest a simple method for reducing Z.txt to 2,000,000 binary digits, that
is 250,000 bytes.
(d) After running gzip on the file Z.txt, its size was reduced to 351,993 bytes.
In view of (a)(c), did gzip do a good job or not?
1 ASCII

encoding is the classical format that computers use to store characters. Each character is represented by an integer from 0 to 255, hence requiring 8 binary digits. More recently, other formats, such as
Unicode, that use more than 8 digits and that can represent a wider variety of characters used in different
languages, have become widespread.
2 One byte is 8 binary digits.
3 With the best compression option.

Examples, Exercises and Extensions

20

38. A piece of classic literature.

The book Around the World in Eighty Days has about 400,000 letters.4
(a) If treated as a plain ASCII file, the file has a size of about 400,000 bytes. Explain why.
(b) Inspection of the file shows that the book in question consists only of az, A
Z, white space, ten or so different punctuation marks and the numerals 09;
that is, about 26 = 64 distinct characters in total.5 Suggest a simple method
for shrinking the ASCII file to 6/8=75% of its original length, that is 300,000
bytes, without losing any information.
(c) After running gzip on the book, its size was reduced to 149,000 bytes. By
running gunzip on the result, we get back the original book. State the fundamental theorem that makes it possible to compress data without loss like this.
(d) Use the observation in (c) to give a rough estimate of the entropy of English
text. Is your estimate an overestimate or an underestimate?
39. Monkey at a typewriter. (Solve Exercise 38 first.)
English text is generated by letting a monkey sit at a typewriter and type at random.
(a) If we let the monkey type 400,000 characters, what is the probability that it
produces Around the World in Eighty Days?
(b) If we let the monkey type 400,000 characters, what is the probability that it
produces any text that makes any sense?
Only order of magnitude estimates are sought. You can assume that the typewriter
offers 64 possible characters (az, AZ, white space and some punctuation characters) like in Exercise 38, and that entropy of English is 2 bits per character.
40. Ideal compression.
A wizard has invented an ideal compressor, that can take any English text of N
characters and compress it to 2NH binary digits without loss, where H is the entropy
of written English.
(a) Suppose we feed the compressor with a string that is not English text. What
must happen?
4 The

copyright has expired and full text is available from www.gutenberg.org.

73 to be exact, but let us use 64 just for the sake of the argument.

5 2 26 + 1 + 10 + 10 =

21

Examples, Exercises and Extensions

(b) Suppose we feed the compressor with Around the World in Eighty Days, then
perturb one binary digit of the result, and decompress the result. What do we
get?
(c) Explain why such a compressor cannot exist.
41. Entropy of natural language. (a , bc but hard without a pocket calculator)
We want to estimate the entropy of written English.
(a) Table 1 shows an estimate of the frequency of occurrence for different letters
(az) in English text. If H in (5.21) on page 92 (with M = 26) is evaluated
using values of pn from Table 1, we get H 4.17. Is this an overestimate or
underestimate of the entropy?
(b) The calculation in (a) is a bit cumbersome. It could be simplified by finding an
analytical approximation for the empirical distribution in Table 1. One pretty
god fit is
pn e0.13n2 .

(E-28)

(The result does not sum up to one, but that is not very important. We just want
a crude
approximation.) Use (E-28), together with a Riemann approximation
R
( ) of the sum in (5.21) on page 92, to compute H now.

(c) George Zipf discovered in 1935 that in most languages (English in particular),
the frequency of occurrence of the nth most common word is about 1/n. Words
like the and in are extremely common, and some words like signal are
quite uncommon except for in some textbooks. This law cannot be entirely
correct, since the series
n=1 1/n is divergent, but let us assume that it is valid
for n up to some number, say N. Then, the probability of the nth most common
word is
pn =

1/n
N
k=1 1/k

n = 1, . . . , N.

(E-29)

It is also known, that on the average the length of a word in written English is
about 4.5 letters.
Use these observations to estimate the entropy of English text, for N = 1000,
N = 10000 and N = 50000. The technique demonstrated in (b) is useful. Comment on the estimate you getis it likely to be an overestimate or an underestimate of the entropy? Does the answer depend a lot on the choice of N?

22

Examples, Exercises and Extensions

Letter %
e
12
i
7
h
6
c
3
w
2
v
1
j
0.1

Letter
t
n
d
m
g
k
z

%
9
7
4
3
2
0.7
0.07

Letter %
a
8
s
6
l
4
f
2
p
2
x
0.2

Letter %
o
8
r
6
u
3
y
2
b
1.5
q
0.1

Table 1. Approximate frequency of different letters (az) in English text. (The numbers
are rounded and do not exactly sum up to 1.)
(d) Compare the results of this exercise with that of Exercise 38 and discuss the
result.
The following integrals are useful:
Z x2
x1

eb
eaxb (ax b) dx =
(1 + b + ax2 )eax2 (1 + b + ax1 )eax1 (E-30)
a
 
Z x2

1
1
1
2
2
log
dx = (log(x1 )) (log(x2 )) .
(E-31)
x
2
x1 x

42. Approximation of

n

k .

Derive the approximation (5.12) on page 87.

43. A coin. (a , b )
A coin is tossed by dropping it from two meters height onto a concrete floor. You
watch the coin being tossed this way twenty times, and see following sequence:
00100000000000000000
(tails labeled with 0 and heads labeled with 1, as in the text).
If the coin is tossed again (in the same way), how large is the chance of seeing a 1,
if
(a) you know that the coin is fair,
(b) you do not know whether the coin is fair or not?

Examples, Exercises and Extensions

23

Chapter 6
44. Binary channel connected to a binary source.
Consider a binary channel that receives its input from a binary source, as in Figure 6.7 on page 125. We observe the output y but we do not know its input x.
(a) Suppose p = 1/2, 1 = 0.2 and 2 = 0.1, and we observe y = 0. What is the
probability that x was a 0?
(b) Suppose p = 1/2, 1 = 0 and 2 = 1/2, and we observe y = 1. What can we
say about x?
45. Counting.
Prove (6.30) on page 128 mathematically. Also, give an intuitive explanation for
(6.30).
46. Capacity of the binary symmetric channel.
Derive (6.37) on page 130. What is the intuition behind the solution?
47. Binary erasure channel.
A memoryless binary source feeds a binary erasure channel. Compute the probabilities that 0 and 1 are observed at the channel output, respectively.
48. Capacity of the binary erasure channel.
Compute the mutual information and capacity of the binary erasure channel, see
Figure 6.12 on page 136, for the case that 1 = 2 = .
49. Drilling a hole in an optical disc. (Solve Exercise 48 first.)
You are given an optical disc of diameter 5 cm that can store 1010 raw binary
digits, and that uses a state-of-the-art error correcting code for protection against
read-write errors.6 The requirement on the channel coding on the disc is that if the
disc has no other defects, it should be resilient enough against physical damage that
you can drill a 10 mm hole through it, and still all data should be recoverable.
But if the hole is larger, the disc may stop working.
6 An analysis of an actual CD-rom is more difficult. Drilling a large hole would cause the read head to
lose sync and moreover, the CD-rom standard does not use error correcting coding that comes close to the
theoretical limit. One reason for the latter is that the disc should be playable in near real time, that is, it
should be possible to listen to the contents without reading the whole disc into a memory and decoding it
first.

Examples, Exercises and Extensions

24

(a) Approximately, how many of the raw binary digits are lost due to the drilling
of the hole?
(b) State the fundamental theorem that makes it possible (in principle, ignoring the
difficulties with keeping the read-head on track) to tolerate the loss in (a).
(c) Suppose that the read head always reads out a binary digit correctly, except for
those lost by drilling of the hole. What is the maximum capacity of the disc, in
bytes, under the following different assumptions?
(i) The location of the 10 mm hole is exactly known at the time of encoding
and manufacture of the disc. Also, the read head can tell, whether it is
positioned over the hole or not.
(ii) The location of the hole is unknown at the time of encoding and manufacture of the disc. The read head can tell, whether it is positioned over
the hole or not. When the read head attempts to read out a binary digit
wiped out by the drilling of the hole, it reports that the digit is lost.
(iii) Like in (ii), the location of the hole is unknown at the time of encoding
and manufacture of the disc. However, here the read head is unable to tell,
whether it is positioned over the hole or not. When the read head attempts
to read a binary digit destroyed by the hole, the probability that a binary
digit being is read out incorrectly is 50%.
(d) Compare the results of (c-i)(c-iii). Also comment on the differences in encoding and decoding complexity between case (i) and cases (ii)(iii).
(e) Suppose that we live in a world where coding theory had not been invented,
so the only available error protection technique is repetition coding. Further,
suppose we want the disc to have the same storage capacity as in (c), but could
accept a residual error probability of 1043 . How large, in terms of physical
diameter, would the disc have to be now to tolerate the drilling of a 10 mm
hole, under
(I) the assumption in (cii) above,
(II) the assumption in (ciii) above.
Hints: In (c-ii), use the binary erasure channel. In (c-iii), use the binary symmetric
channel. In (e-I), how many of the repeated bits need be erased in order for the
result to be wrong (or undefined)? In (e-II), use (6.17) on page 119.
50. Efficiency of a mobile broadband system. 4G (LTE) mobile broadband7 uses
near-state-of-the-art channel coding. Under certain (very favorable) circumstances,
7 The

figures stated in this exercise here are not exact but the orders of magnitude should be correct.

25

Examples, Exercises and Extensions

the system operates at rate R = 1/2 and sends codewords of 100,000 binary digits
each millisecond. For the sake of this exercise you can model the transmission via
the binary symmetric channel, although reality is a bit more complicated.
(a) How many bits of information are transmitted per second?
(b) What raw channel error probability can the channel code tolerate, approximately?
(c) If coding theory had not been invented, so that we had to rely on repetition
coding, how much would we have to reduce the information rate? You can
assume that a residual error probability target of 1015 is acceptable, and use
the value of from (b).
51. RAM memory in a computer.
RAM memory in a computer usually includes error detection in the form of parity
digits. One parity digit is included for each 8-bit word. If b1 , . . . , b8 are the bits of
a word, then the parity digit is chosen as
p = b1 + + b8 ,

(E-32)

where addition is defined as in (6.20) on page 123.

(a) A 9-bit word (8 bits b1 , . . . , b8 plus a parity digit p ) is read out from the memory.8 What can be concluded if
p + b1 + + b8 = 0?

(E-33)

(b) A 9-bit word (8 bits b1 , . . . , b8 plus a parity digit p ) is read out from the memory. What can be concluded if
p + b1 + + b8 = 1?

(E-34)

(c) Viewed as a channel code, what are R and dmin ?

(d) How many digit errors in an 8-digit word can the parity check correct?
(e) How many digit errors in an 8-digit word can the parity check detect?
8 We use () to denote the digits read from the memory, as they may differ from the digits written to the
memory.

26

Examples, Exercises and Extensions

(f) Parity errors seem to be very rare. If an error occurs, usually the computers
operating system halts. Suppose that for a given computer, this happens once
every year of continuous use. If we view the memory as a binary symmetric
channel (for each stored binary digit), estimate the order of magnitude of .
As for a year of continuous use, you can assume that the computer reads out
one randomly chosen word from the memory every tenth clock cycle, that is,
about 108 times per second at a processor clock frequency of 1 GHz, each day
around the clock for a whole year.
(g) Suppose the memory in the computer can store 109 bytes, where each byte
consists of 8 data digits plus a parity digit. How many binary digits in total does
the memory contain? If all those digits were used efficiently, in the sense of the
channel coding theorem, how much more efficient than the parity checking
scheme would this be?
(h) Why yet do you think the parity checking mechanism is used?
52. Data center design. (abc , d ; solve Exercise 48 first.)
You are in charge of a big data center that has seven 100 Tbyte hard drives placed
in seven different countries. Your task is to design an error protection technique,
such that if one of the countries is hit by an earthquake and one of the hard drives
is destroyed, all data can still be recovered.
(a) Argue for why the binary erasure channel is an appropriate way of modeling
the data loss in case of an earthquake.
(b) Suggest a method using repetition coding. Comment on its efficiency, in relation to the channel coding theorem.
(c) Suggest a method using the (7,4)-Hamming code. Comment on its efficiency.
(d) How would you solve this task in practice?
53. Codebooks.
For each of the following codebooks,
(a)
X1 = (00000000)
X3 = (01001010)

X2 = (01110000)
X4 = (11001111)

27

Examples, Exercises and Extensions

(b)
X1 = (00000000)
X3 = (01001010)
X5 = (11001010)

X2 = (01000000)
X4 = (01001110)

(c)
X1 = (001000011000)
X3 = (010111111100)
X5 = (100010101001)

X2 = (010000011001)
X4 = (010010101010)

do the following:
(i) Determine the rate, R.
(ii) Determine the minimum distance, dmin .
(iii) Determine how many digit errors in a received string that the codebook can
correct.
(iv) Determine the order of magnitude of the error probability if the code is used
to communicate over a binary symmetric channel with raw error probability
.
(v) Compare the results of (i)(iii) to the corresponding characteristics of the
(7,4)-Hamming code, and to a repetition code with the same or a similar rate.
(vi) Comment on whether the codebook is good or not. If you conclude that it
is not good, can you suggest a way of modifying it in order to unilaterally
improve its rate and/or its error probability? Compare your modified code to
the (7,4)-Hamming code.
(vii) What do you conclude, is it easy or hard to invent a good (small) codebook?
54. Property of the Hamming distance.
Explain why the Hamming distance satisfies the triangle inequality, that is,
|X Y | |X Z| + |Z Y |
for any strings X, Y and Z consisting of binary digits.
Hint: Look at the elements of X, Y and Z one by one.

(E-35)

28

Examples, Exercises and Extensions

55. Technical point.

Prove each step of (6.66) on page 143.
Hint: Write up all the combinations of Y(n), Xm0 (n) and Xm1 (n) that can occur.
56. Binary Input Additive Gaussian Noise (BI-AGN) Channel. (abcd , e )
Consider a channel that takes a binary digit as input delivers
(
1 + w if a 0 was sent
y=
1+w
if a 1 was sent

(E-36)

as output, where w is zero-mean Gaussian noise with standard deviation . (This

channel is a special case of the additive Gaussian noise channel in Figure 6.15 on
page 139 and is called a binary-input additive Gaussian noise channel.)
(a) Suppose the transmitter flips a coin with bias p as in (6.22) on page 124 in
order to decide which digit to send. Write up an expression for the probability
density functions py (y|0 was sent), py (y|1 was sent) and py (y).
(b) Suppose y is observed at the output of the channel. Give an expression for the
probability that 0 was sent, and an expression for the probability that a 1 was
sent.
(c) Compute


P(1 was sent|y)

log
P(0 was sent|y)

(E-37)

Simplify the result for the special case that the coin is fair, p = 1/2.
(d) Suppose the coin is fair, so p = 1/2. Give a rule for how to detect the transmitted digit from y.
(e) How large is the capacity of the channel in (E-36)?
57. BI-AGN with repetition coding. (abc , de )
Consider the additive Gaussian noise channel and suppose we want to transmit a
binary string. We consider the following scheme: each digit b is mapped onto
s {1} (0 is mapped onto s = 1 and 1 is mapped onto s = +1). The symbol
s is then repeated V times and sent over the channel. For example, with V = 2, 0
results in {1, 1} being transmitted and 1 results in {+1, +1}.
The receiver sees

y i = s + wi ,

i = 1, . . . ,V,

(E-38)

29

Examples, Exercises and Extensions

where wi are independent, zero-mean Gaussian random variables with standard

deviation . Based on yi , we want to determine whether s = 1 or s = +1 was
sent. We call s the decision, or best guess, on s made by the receiver.
Suppose the transmitted digit is generated by a fair coin. Also, let V be odd.
(a) Suppose we use the rule derived in Exercise 56(d) for each yi , and then take
a majority decision. Give an expression for the probability that the result is
wrong.
(b) Compute



P(1 was sent|y1 , . . . , yV )
log
.
P(0 was sent|y1 , . . . , yV )

(E-39)

Use the result to construct a better rule for how to detect the transmitted digit
from y1 , . . . , yV . What is the probability that the result is wrong?
(c) How large is the capacity of the binary channel obtained after majority voting
in (a)?
(d) How large is the capacity of the channel defined by (E-38)?
(e) How much better is the decision rule in (b) than the one in (a)?

Chapter 7
58. Physical units.
Give the physical unit of each of the quantities that enter (7.28) on page 165 and
(7.32) on page 167.
59. Power versus bandwidth.
Consider the frequency-flat waveform channel, subject to additive white Gaussian
noise with energy N0 /2 per degree of freedom. Suppose we are given an energy
budget of E and have the choices of either (i) uniformly spreading this energy
over a time-frequency space of dimensionality 2BT , or (ii) uniformly spreading the
same energy over a time-frequency space of dimensionality BT . Operationally, this
means using waveforms with 2BT or BT degrees of freedom, respectively. Which
way is better and why?
60. Capacity of additive Gaussian white noise channel.

Examples, Exercises and Extensions

30

The text derives (7.28) on page 165 using a frequency-domain representation of the
transmitted waveform. Starting from (6.56) on page 139, derive the same result using a time-domain representation instead. (The resulting derivation is much simpler
than the one in the text, but it is harder to extend to the case of a frequency-selective
channel.)
61. Pre-ADSL age telephone modems.
Before ADSL became widespread, dialup modems were often used to connect personal computers to the Internet when an Ethernet connection was unavailable. The
limiting factors of that technology, for the upstream connection, were the analog
bandpass filter in the telephone switching station and the ensuing quantization of
the samples used on the network that connects the switches together. The bandpass
filter limits the signal to the interval [300, 3400] Hz (enough for decent quality
speech, but not really for music). The quantization represents each sampled value
with an integer from 1 to 256.
(a) Estimate how many bits/s a modem connection could achieve and compare
this to state-of-the-art of that time.9 Perform the required calculation in two
different ways:
(i) By calculating how fast the signal needs to be sampled at and how many
binary digits that are needed per second to represent the sample values.
(ii) By using the 6-dB-per-binary-digit rule-of-thumb for SNR after quantization, see Section 3.6, along with (6.56) on page 139.
Explain why you get (approximately) the same result.
(b) Estimate how much the data rate would have improved if the quantizer resolution would be doubled (512 steps instead of 256).
(c) Estimate how much the data rate would have improved if the filter bandwidth
would be doubled.
62. Greedy wireless service provider.
A service provider operates a point-to-point wireless link that uses 10 MHz of bandwidth and offers a data rate of 30 Mbits/s. The only impairment to the link is thermal noise. The link uses state-of-the-art technology and operates at 3 dB from the
theoretical limit.
9 There were several modem standards. One of the best available, V.92, offered 56 kbits/s on the downstream and 48 kbits/s on the upstream. Under good conditions there was no appreciable thermal noise, but
only quantization errors.

31

Examples, Exercises and Extensions

(a) What is P/(BN0 ) at the theoretical limit?

(b) What is P/(BN0 ) at the operating point of the link in question?
(c) What is the spectral efficiency, , of the link?
(d) Comment on the operation point of the link, is it power-limited or bandwidth
limited?
(e) If the link would operate at the theoretical limit (using the value of P/(BN0 )
actually required by the link in question, see (b)), what would the spectral efficiency then be?
(f) The service provider would like to double the data rate by increasing the transmit power but without buying more bandwidth (and still operating at 3 dB from
the limit). Is this possible, and if so, how much must the transmitter output
power be increased?
63. Greenish wireless service provider. (Solve Exercise 62 first.)
Consider the same service provider as in Exercise 62. In an effort to be green,
the service provider would like to reduce the transmitter output power by 10 dB,
without compromising the data rate (and still operate at 3 dB from the theoretical
limit). Is this possible, and if so, how much more bandwidth is required?
64. A case when repetition is not so bad.
Consider the additive Gaussian noise channel,
y k = sk + w k ,

(E-40)

and its capacity in bits per channel use, (6.56) on page 139. Suppose that we can
use the channel 109 times, and that we want to develop a codebook for this channel
that transmits symbols at rate R = 1/20 bpcu. Suppose that (practically) capacityachieving codebooks are available for rates down to 1/10, but good codebooks with
lower rates have not yet been invented.
Two engineers, Ann and Bob, propose two competing schemes to solve the task:
(A) Ann proposes to take an off-the-shelf codebook of rate 1/10, with codewords
of length 5 108 symbols, and then repeat each symbol twice, thereby using the
channel 109 times.
(B) Bob proposes to invest twenty person-years of manpower into inventing a new
(practically) capacity-achieving code for R = 1/20.
Which scheme do you support and why? Give a quantitative justification. If you
support Anns scheme, explain how the receiver should operate.

Examples, Exercises and Extensions

32

Chapter 8
65. Square roots.
In Chapter 8 we use square roots of complex numbers, see Footnote 1 on page 180.
Some care has to be exercised here. Consider:
p

1 = 1 = (1)(1) = 1 1 = i i = 1.
(!?)
(E-41)

Explain which step of (E-41) that is flawed and why.

66. Longer cable.

A 10 meters long cable with real-valued characteristic impedance Zc is fed with

an ideal voltage source ux (t) at one end and connected to a matched load with
impedance Z = Zc at the other end. Let uy (t) be the voltage measured over the load
and let H( f ) be the transfer function as defined in (8.1) on page 178. If the length
of the cable is doubled to 20 meters, describe quantitatively what happens to
(a) the characteristic impedance, Zc , of the cable
(b) the transfer function H( f ) (assuming the 20 meters cable is still connected to a
matched load)?
67. Mismatched cable segment.
A 10 meters long cable with real-valued characteristic impedance Zc is fed with
an ideal voltage source ux (t) at one end and connected to a matched load with
impedance Z = Zc at the other end. Let uy (t) be the voltage measured over the load
and let H( f ) the frequency response as defined in (8.1) on page 178.
Suppose the cable is cut in half and in between the two so-obtained 5 meter
long parts, another L meter long cable segment with real-valued characteristic
impedance Zc , Zc 6= Zc , is spliced in. Describe, as quantitatively as possible, what
happens to H( f ).
68. Very long cable connected to capacitive load.
A cable of length L meters and with characteristic impedance Zc = 50 is fed with
a voltage source ux (t) at the right end, and connected to a load with impedance
Z( f ) at the right end.
(a) If the load is matched, what is the frequency response H( f )?
(b) If the load is replaced by an ideal capacitor with capacitance C [F], what is
H( f ) now? You can assume that the cable is very long.

33

Examples, Exercises and Extensions

(c) Compare the frequency response H( f ) in (a) with that in (b). Interpret H( f )
in terms of the voltage of the load when ux (t) = cos(2 f t). Explain the result
intuitively.
69. Resistance of a cable.
A cable is connected to an ideal voltage source at one end and to a matched load
(Z( f ) = Zc ( f )) at the other end, so that (8.12) on page 180 holds.
a) Using Kirchoffs voltage law on the circuit constituted by the voltage source,
the cable and the load, explain why
q
H(0) = q

a(0)
b(0)

a(0)
b(0)

(E-42)

+ Rc

where Rc [] is the resistance of the cable, if fed with a constant direct current
(with frequency f = 0).
b) Suppose that the leakage between the conductors is negligible so that R p is extremely large. Use a first-order Taylor series to justify the approximation

1
a(0)b(0)L
p
H(0) = e
(E-43)

1 + a(0)b(0)L
c) By equating (E-42) and (E-43), establish that

Rc La(0) = LRs .

(E-44)

d) Explain the intuition behind (E-44).

70. Coherence time.
A wireless link operating at fc = 2 GHz comprises a stationary transmitter and a
mobile receiver located in an environment with rich scattering around the receiver.
The receiver moves with a velocity of 180 km/h. How long is the time experienced
between one fading dip and the next? Approximately, what is the coherence time
of the channel?
71. Coherence time, again.
A transmitter and a receiver are located as in Figure 8.4 on page 195, with =
45 . Determine the time between the fading dips, and comment on the channel
coherence time, if the the receiver moves with a velocity of 72 km/h

Examples, Exercises and Extensions

34

(a) upwards,
(b) to the left,
(c) southwest in an angle of 45 relative to the horizontal baseline.
The carrier frequency is fc = 1 GHz.
72. Coherence bandwidth.
Comment on the operating point of the following mobile communication systems,
in terms of frequency-flat or frequency-selective fading
(a) Digital AMPS,10 channel bandwidth 30 kHz
(b) GSM, channel bandwidth 200 kHz
(c) 4G (LTE), channel bandwidth 20 MHz (in one of the operating modes)
in the following environments:
(i) Strong line-of-sight between the base station and the terminal, no appreciable
scattering by other objects
(ii) Lots of scattering, no line-of-sight, |i j | in the order of 3 s (typical of
urban environments)
(iii) Lots of scattering, no line-of-sight, |i j | in the order of 20 s (occurs in
some mountainous environments)
You can disregard the effects of transmitter/receiver filters and antennas and consider only the actual propagation medium.
73. Derivation of the signal envelope in fading.
Prove (8.41) on page 191.
74. Varying propagation loss.
Consider the propagation loss in the terrestrial wireless channel. If r is 1 kilometer
and = 3.5, how far must the receiver move before the propagation loss is substantially affected, say before the received power has changed by 10%?
10 Digital

AMPS was the first north-American digital mobile telephony standard.

35

Examples, Exercises and Extensions

Chapter 9
75. Relation between AM and I/Q modulation.
(a) Give a method for generating a double-sideband AM signal, see Figure 9.2(b)
on page 208, using the I/Q modulator in Figure 1.8(a) on page 23.
(b) Give a method for generating a single-sideband AM signal, see Figure 9.2(c),
using the I/Q modulator in Figure 1.8(a).
76. Communicating a binary string.
A communication link as in Figure 9.3 on page 217 is used to transmit a string consisting of symbols from the alphabet {a, b, e, i, o, s, y}. At the end of the string,
the symbol  is appended, just to signify that it is the last symbol of the string.
The string of symbols is first compressed (first box in Figure 9.3) by using the
Huffman code defined by
a 1
o 00000

b 011
s 00001

e 010
y 00010

i 001
 00011.

(E-45)

The resulting string of binary digits is then padded with 0s, to make sure that its
length is a multiple of 4. The so-obtained string of binary digits is then encoded
(second box in Figure 9.3) with the (7,4)-Hamming code, in order to protect against
errors. The resulting string (whose length is a multiple of 7), is then mapped onto
symbols by first mapping each 0 onto 1 and each 1 onto +1. The resulting
symbols are then modulated (third box in Figure 9.3) by using pulse-amplitude
modulation as in Figure 2.3 on page 35, with the pulse shape

1, |t| 1
2 ,
p(t) =

0, otherwise

(E-46)

and Ts = 1 second. The modulated waveform is transmitted over a channel (the

fourth box in Figure 9.3). The receiver proceeds by sampling as in Figure 2.3 on
page 35. There is additive thermal noise present in the channel, but you can ignore
the effect of the filters h(t) and (t).

36

Examples, Exercises and Extensions

The following sequence of samples was seen after sampling at the receiver:
1.55
1.26
0.67
1.50
1.38

1.02 0.84 0.65 0.61 1.82

0.79
0.77 0.14 1.98 1.07 0.32 0.89
1.44 0.15 0.54 0.81 1.40 0.63
0.90 0.68 0.48 1.37
1.02
1.19
1.37 1.48 0.21 1.95 1.22 0.55

(a) Draw the tree of the form in Figure 5.4(b) on page 97 corresponding to (E-45),
and verify that the code is decodable symbol-by-symbol. Explain why, if this
were not the case, it could not be a Huffman code.
(b) Make a best guess on which message that was transmitted, using appropriate
mechanisms in the sixth and seventh box in Figure 9.3.
(c) Comment on how likely it is that your answer in (a) is correct.
(d) Make a rough (order-of-magnitude) estimate of the signal-to-noise ratio on the
channel.
(e) How much time and bandwidth does the transmitted waveform occupy, approximately?
(f) What is the dimensionality of the waveform that results from the pulseamplitude modulation?
Hint: In (d), look at the first (or one of the few first) 7-digit codeword. What symbols were actually transmitted? What are the values of the noise samples affecting
the received signal? How large (rough order of magnitude) is their variance?
77. Maximum-ratio combining.
A waveform x(t) contains a communication signalwith 2BT degrees of freedom
and energy Ex . Suppose the receiver has access to the two observations
y1 (t) = a1 x(t) + w1 (t),
y2 (t) = a2 x(t) + w2 (t),

(E-47)
(E-48)

where a1 and a2 are some constants and w1 (t) and w2 (t) are independent, white
thermal noise signals with energy N0 /2 per dimension. This is a good model for
a situation where x(t) is sent twice (with repetition), or a situation where x(t) is
received twice, for example at two different antennas of a wireless base station.
(a) What is the SNR in y1 (t)?

37

Examples, Exercises and Extensions

(b) What is the SNR in y2 (t)?

(c) What is the SNR in y1 (t) + y2 (t)?
(d) If the receiver forms the signal
z(t) = b1 y1 (t) + b2 y2 (t),

(E-49)

for some constants b1 and b2 , what is then the SNR in z(t)?

(e) If b1 and b2 are given, how should the receiver treat z(t) in order to recover
x(t)?
(f) Find the values of b1 and b2 that maximize the SNR in (d). Is the answer
unique, and if not, why not?
(g) If a1 and a2 fluctuate independently due to small-scale fading, as in Figure 8.5
on page 197, how likely is it that the SNR in y1 (t) experiences a dip? How
likely is it that the SNR of y2 (t) experiences a dip? How likely is it that the
SNR of z(t) experiences a dip? Under what physical conditions is it reasonable
to assume that a1 and a2 fluctuate independently?
Hint: Cauchy-Schwarz inequality,




(a1 b1 + a2 b2 )2 a21 + a22 b21 + b22

(E-50)

is useful. Equality in (E-50) holds if and only if a1 = b1 and a2 = b2 for some

.
78. Time-synchronization bursts in GSM.
GSM base stations periodically broadcasts synchronization bursts of duration 0.6
ms and bandwidth 200 kHz. Each synchronization burst contains a synchronization
waveform of length about 0.25 ms. This waveform is used by the terminals to timesynchronize themselves to the base stations.
(a) What properties should the synchronization waveform have?
(b) If the time-duration of the synchronization waveform would be doubled, what
potential effects would this have?
(c) If the bandwidth of the synchronization waveform would be doubled, what
potential effects would this have?
(d) If the synchronization bursts were used to locate the geographical position of
the terminal, by using a simple algorithm that correlates and samples at the
Nyquist rate, approximately what accuracy (order of magnitude) could be obtained? (You can ignore multipath propagation, interference and other realworld problems.)

Examples, Exercises and Extensions

38

79. Data bursts in GSM.

GSM uses a channel bandwidth of B = 200 kHz and transmits data bursts of length
about 0.6 ms. Each burst consists of a pilot waveform of duration about 0.1 ms, and
two information-carrying waveforms of duration about 0.25 ms. Each informationcarrying waveform is obtained by a nonlinear modulation technique, but the result
can be fairly well approximated by pulse-amplitude modulation with a symbol interval of Ts 3.7 s. The pilot waveform is located in the middle of the burst, with
one information-carrying waveform to its left and one on its right.
(a) Comment on the total duration of the burst (0.6 ms) in relation to the channel
coherence time in a typical use case.
(b) What is the dimensionality of a burst?
(c) With pulse-amplitude modulation, how many symbols could theoretically be
modulated into the data-bearing part of one burst? (Assume the Nyquist criterion is satisfied.)
(d) The data-bearing part of a real-world GSM burst contains 130 real-valued symbols modulated with pulse-amplitude modulation, of which 114 contain actual
data, the others are known to the receiver and inserted just to help the equalizer
at the receiving end. Does GSM make efficient use of the degrees of freedom
available in the channel? If not, what are the reasons for the lack of inefficiency?
(e) The pilot waveform is also (approximately) obtained by pulse-amplitude modulation. Why do you think the pilot waveform is located in the middle of the
burst? How many modulated symbols does the pilot waveform consist of?
(f) How much overhead (in %) is spent on channel training?
(g) If the channel coherence bandwidth is 50 kHz, approximately how many pilot
symbols are spent per coherence interval?
(h) If the channel impulse response h(t) is sampled and represented by samples
taken at the Nyquist frequency (for a signal with 200 kHz bandwidth), how
many samples are needed, approximately? (A typical commercial GSM implementation models the impulse response by five time-domain samples.)
80. Channel training using pilots.
Consider the estimation of a channel impulse response h(t) from a received pilot
sequence using (9.32) on page 230. Suppose x p (t) lives in the time-frequency space
[0, T ] [B, B].

39

Examples, Exercises and Extensions

(a) We must assume that h(t) lives in the same space [0, T ] [B, B]. Why is this
a reasonable assumption?
(b) We can assume, without loss of generality, that y(t) has been truncated and
filtered to fit into the time-frequency space [0, T ] [B, B]. Why?
(c) Under the assumptions made, write up y p (t), h(t) and x p (t) in terms of expansions by sampling in the time-domain.

(d) Under the assumptions made, write up y p (t), h(t) and x p (t) in terms of expansions by sampling in the frequency-domain.
(e) Formulate (9.32) on page 230 in terms of a least-squares regression problem on
vector-matrix form, using the result of (c). Give an explicit solution.
(f) Formulate (9.32) in terms of a least-squares regression problem on vectormatrix form, using the result of (d). Give an explicit solution.
81. Transmitted adaption with inverse filtering reduces received power.
Consider the inverse filtering approach towards elimination of channel distortion in
Section 9.3.2. Quantitatively compare
(i) the received useful energy in the nominal system, that is, the energy of (h
x)(t) in (9.14) on page 215, and
(ii) the received useful energy in the system that uses transmitter adaptation, that
is, the energy of x(t) in (9.19) on page 219.
Discuss the result.
Hint: Cauchy-Schwartz inequality,
Z

f (x)g(x) dx

2

Z

f (x) dx

Z

g (x) dx ,

(E-51)

may be useful. Equality in (E-51) holds if and only if f (x) = g(x) for some .
82. The matched filter.
Consider pulse-amplitude modulation of a single lonely symbol s, with the pulse
shape p(t). The received signal is
y(t) = s p(t) + w(t),

(E-52)

where w(t) is noise. Suppose w(t) is white over the bandwidth that p(t) occupies,
and has energy N0 /2 per dimension. The receiver detects s by using the architecture

40

Examples, Exercises and Extensions

in Figure 2.3 on page 35. Let us ignore the presence of the channel filter h(t) for
the moment. Then, the sample of z(t) taken at time t = 0 is
z=

( )y( ) d = s + v,

(E-53)

( )p( ) d ,

(E-54)

( )w( ) d .

(E-55)

where

v=

A reasonable measure of how accurately s can be detected from z is the signal-tonoise-ratio

2
.
E [v2 ]

(E-56)

(a) Compute (E-56) explicitly, expressed in terms of p(t) and (t).

(b) If we can choose (t) freely, what choice of (t) maximizes (E-56)? Is the
answer unique? If not, why not?
(c) The optimal choice of (t) is called the matched filter. Why do you think it has
this name?
(d) Can the result be easily generalized to the case with a channel filter h(t), that
we neglected in (a)(c) above?
(e) Can the result be easily generalized to the case when the pulse-amplitude modulation involves many symbols; that is, (2.1) on page 32 is used instead of
(E-52)?
 
Hints: To work out E v2 in (a), use a sampled representation as in Section 1.3.4
and exploit the fact that samples taken 1/(2B) seconds apart are independent, or
as an equivalent alternative, use the formulas for energy of a filtered signal in Section 3.7. In (b), use Cauchy-Schwarz inequality, see (E-51).
83. Information content in a waveform in the frequency domain.
A communication link transmits a single binary digit b by sending the waveform
x(t) where
(
1, |t| 12
x(t) = x0 (t) ,
,
(E-57)
0, otherwise

41

Examples, Exercises and Extensions

if b = 0 and
x(t) = x1 (t) ,

(q

1
1+2 ,

0,

|t| 21 +

(E-58)

otherwise

if b = 1, where is a constant chosen when the system is designed. The receiver

sees
y(t) = x(t) + w(t),

(E-59)

where w(t) is noise.

(a) Sketch x0 (t), x1 (t), |X0 ( f )| and |X1 ( f )|.

(x0 (t) x1 (t))2 dt, as a function

(b) Compute the squared Euclidean distance
of , and based on (9.11) on page 213, argue that this distance is a measure of
how easy it will be for the receiver to discriminate between x0 (t) and x1 (t).

(c) Suppose that in order to make sure that the transmitted waveform is bandlimited, HBLP {x(t)} is transmitted instead of x(t), for some bandwidth B. How
much energy of x0 (t) respectively x1 (t) is lost in the lowpass filteringif B = 10
Hz?

(HBLP {x0 (t)} HBLP {x1 (t)})2 dt. Compare with the results of
(d) Compute
(b) and (c).

(e) What part of the spectrum of X0 ( f ) and X1 ( f ) carries the information about b?
(f) Relate the observations to the different bandwidth definitions in Exercise 2.
84. Archiving information for the future.
Suppose we wanted to archive, for future generations, humanitys worldwide collective production of information, including books; newspapers; webpages; Facebook content; blogs; digital photos and videos taken; music produced; and similar.
One difficulty with doing that is that stored information is subject to degradation
of the physical medium. For example, the plastics in CD-roms degrades or cracks
with time; magnetic tapes or disks are subject to random demagnetization; and paper yellows or may dissolve if subjected to moisture. To safely store information for
a long time or forever, it will have to be periodically replicated. In an ideal world,
if the information is stored in digital form, such replication is possible without any
degrading of quality. However, this assumes that the replication is performed while
the information is still readable.

42

Examples, Exercises and Extensions

(a) Give an estimate (to the best you can) of how much information is generated
every day.
(b) Give an estimate of the entropy of the information in (a).
(c) Give some arguments for how fast the cumulative amount of information will
grow with time. The use of Internet seems to have grown exponentially over
the last two decades or so.11 The world population also seems to grow exponentially, though it is less clear for how long that will or can last.
(d) Suppose optical discs are used to store the information and suppose that the
probability that an optical disc is still readable after t years is et/T , where T
is a constant (a couple of hundred years, say). Argue for why this is is this a
reasonable model.12
(e) Suppose T = 100 years, and that each disc is replicated each years. What is
the probability that a disc is readable when it is due for duplication? What is
the probability that a disc is readable at its Kth duplication?
(f) If = 1 year, and we accept to lose a disc with probability 1% after 1000 years,
how long time (that is K ) can we wait between each replication?
(g) What is the answer to (f) if we instead can only accept to lose a disc with
probability 1010 ?
(h) Is the conclusion of (f) and (g) compatible with the assumption on growth in
(c)?
(i) Could channel coding techniques in any way be used to alleviate the problem,
and if so, how?
(j) How should the formats used to encode the information (e.g., the compression
algorithms) be documented?
(k) Does it seem fundamentally possible to store all information that we create
forever? (Is it desirable?)
11 The amount of information that flows through the Internet has been estimated to be some 100 Terabytes

per second (2012). In 1993, the corresponding volume of Internet traffic was only 100 Terabytes per year.
12 The exponential distribution is quite common to model the lifetime of physical objects. One justification for this model is as follows: suppose
P(object alive at time t|object alive at time t 1) = 1 p,
for some p and that the object is alive at time t = 0. Then
P(object alive at time t) = (1 p)t ,
which decays exponentially with t.

Examples, Exercises and Extensions

43

Mixed Exercises
85. Nyquist criterion and sampling.
Pulse-amplitude modulation is used to create a lowpass waveform that lives in the
time-frequency space [0, T ] [B, B], using a pulse shape p(t) and symbol time
such that the Nyquist criterion is satisfied.
(a) How many degrees of freedom does the waveform created this way have?
(b) How many symbols can be modulated this way?

SOLUTIONS AND FURTHER

DISCUSSION

59. Equation (7.21) on page 162 gives the number of bits that the channel can support
during the time-frequency interval in question. In (i), En = E/(2BT ). In (ii), En =
E/(BT ). The energy of the noise per degree of freedom is the same in (i) and (ii).
Since




E
E
2BT log2 1 +
BT log2 1 +
,
2BT N0 /2
BT N0 /2
alternative (i) is better.
61. (a-i) Each second we can take 2 (3400 300) = 6200 samples. This gives
6200 8 49, 600 bits/s
(a-ii) During T seconds, we can fit in a waveform with 2BT degrees of freedom.
Hence during T seconds, the theoretical limit predicts that we can transfer


E
1
2BT log2 1 + 2
2

bits, where E/ 2 is the SNR per degree of freedom. Using the rule-of-thumb,
the SNR is approximately 86 = 48 dB, that is, 104.8 in linear scale. Inserting
an SNR of 104.8 , and B = 3100, we find that each second we can transfer


1
2B log2 1 + 104.8 = 3100 4.8 log2 (10) 3100 4.8 3.3 49, 000
2

bits.

44

45

Solutions and Further Discussion

The results of (i) and (ii) are about the same (as expected), even though they
were obtained by two totally different routes of calculation. Of course, the
calculation in (ii) is a bit rough. In particular, it assumes Gaussian noise, not
uniformly distributed noise. However, in a first-order estimate, this does not
make a big difference.
(b) This gives one extra bit per sample, thus a rate of
6200 9 = 55, 800 bits/s.
(c) Doubling the number of samples, we get
2 6200 8 = 99, 200 bits/s.
76. (a) The code is uniquely decodable symbol-by-symbol. This is always the case for
a Huffman code, see Chapter 5.
(b) For each sample, say z, seen at the receiver, we decide that the corresponding
binary digit most likely was 0 if z < 0 and 1 otherwise. Then, execute the
standard steps of decoding the (7,4)-Huffman code and the Hamming code by
looking in the table (E-45). After (7,4)-Hamming decoding we obtain
0001011 0010101 0000000 1000111 1100001
which corresponds to the uncoded, zero-padded string
0001 0010 0000 1000 1100
This in turn corresponds to the message
yes
(c) It is pretty likely that we have decoded correctly. If at some point the channel had introduced more errors than what the (7,4)-Hamming code is able to
correct (that is, one digit error per codeword), then very likely we would have
lost sync in the Huffman decoding and not ended up with a message string
terminated by .
(d) To get an idea of the SNR, let us look at the first 7-digit codeword. The transmitted symbols must be
1

1 1

1 1

The noise samples therefore must be

0.55

0.02 0.16

0.35 0.39 0.82

0.21

Solutions and Further Discussion

46

These noise samples are zero mean. A (very) rough at-a-glance estimate
of the standard deviation would be 0.5. That would give an SNR of
10 log10 (1/0.52 ) = 6 dB. (The noise samples were actually generated with this
standard deviation.)
(e) The waveform is 35 seconds long. Its 99%-bandwidth (cf. Exercise 2) is about
10 Hz.
(f) The dimensionality (with a 99%-bandwidth measure) is about 2 35 10 =
700.