Detection and Estimation of Signals

in Noise
Dr. Robert Schober
Department of Electrical and Computer Engineering
University of British Columbia

Vancouver, August 24, 2010

Contents
1 Basic Elements of a Digital Communication
System

1.1 Transmitter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

1.2 Receiver . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

1.3 Communication Channels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

1.3.1

Physical Channel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

1.3.2

Mathematical Models for Communication Channels . . . . . . . . . . . .

2 Probability and Stochastic Processes

2.1 Probability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

9
9

2.1.1

Basic Concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

2.1.2

Random Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

2.1.3

Functions of Random Variables . . . . . . . . . . . . . . . . . . . . . . . 21

2.1.4

Statistical Averages of RVs . . . . . . . . . . . . . . . . . . . . . . . . . . 26

2.1.5

Gaussian Distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

2.1.6

Chernoff Upper Bound on the Tail Probability . . . . . . . . . . . . . . . 39

2.1.7

Central Limit Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

2.2 Stochastic Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42

2.2.1

Statistical Averages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

2.2.2

Power Density Spectrum . . . . . . . . . . . . . . . . . . . . . . . . . . . 50

2.2.3

Response of a Linear TimeInvariant System to a Random Input Signal . 52

2.2.4

Sampling Theorem for BandLimited Stochastic Processes . . . . . . . . 56

2.2.5

DiscreteTime Stochastic Signals and Systems . . . . . . . . . . . . . . . 57

2.2.6

Cyclostationary Stochastic Processes . . . . . . . . . . . . . . . . . . . . 60

3 Characterization of Communication Signals and Systems

63

3.1 Representation of Bandpass Signals and Systems . . . . . . . . . . . . . . . . . . 63

3.1.1

Equivalent Complex Baseband Representation of Bandpass Signals . . . 64

3.1.2

Equivalent Complex Baseband Representation of Bandpass Systems . . . 74

3.1.3

Response of a Bandpass Systems to a Bandpass Signal . . . . . . . . . . 76

Schober: Signal Detection and Estimation

3
3.1.4

Equivalent Baseband Representation of Bandpass Stationary Stochastic

Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77

3.2 Signal Space Representation of Signals . . . . . . . . . . . . . . . . . . . . . . . 85

3.2.1

Vector Space Concepts A Brief Review . . . . . . . . . . . . . . . . . . 85

3.2.2

Signal Space Concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88

3.2.3

Orthogonal Expansion of Signals . . . . . . . . . . . . . . . . . . . . . . 90

3.3 Representation of Digitally Modulated Signals . . . . . . . . . . . . . . . . . . . 103

3.3.1

3.3.2

Memoryless Modulation . . . . . . . . . . . . . . . . . . . . . . . . . . . 104

3.3.1.1

M-ary PulseAmplitude Modulation (MPAM) . . . . . . . . . 104

3.3.1.2

M-ary PhaseShift Keying (MPSK) . . . . . . . . . . . . . . . 109

3.3.1.3

M-ary Quadrature Amplitude Modulation (MQAM) . . . . . . 112

3.3.1.4

MultiDimensional Modulation . . . . . . . . . . . . . . . . . . 115

3.3.1.5

M-ary FrequencyShift Keying (MFSK) . . . . . . . . . . . . . 116

Linear Modulation With Memory . . . . . . . . . . . . . . . . . . . . . . 122

3.3.2.1

3.3.3

Mary Differential PhaseShift Keying (MDPSK) . . . . . . . 122

Nonlinear Modulation With Memory . . . . . . . . . . . . . . . . . . . . 124

3.3.3.1

ContinuousPhase FSK (CPFSK) . . . . . . . . . . . . . . . . . 124

3.3.3.2

Continuous Phase Modulation (CPM) . . . . . . . . . . . . . . 128

3.4 Spectral Characteristics of Digitally Modulated Signals . . . . . . . . . . . . . . 141

3.4.1

Linearly Modulated Signals . . . . . . . . . . . . . . . . . . . . . . . . . 141

3.4.2

CPFSK and CPM

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 150

4 Optimum Reception in Additive White Gaussian Noise (AWGN)

151

4.1 Optimum Receivers for Signals Corrupted by AWGN . . . . . . . . . . . . . . . 151

4.1.1

4.1.2

Demodulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153
4.1.1.1

Correlation Demodulation . . . . . . . . . . . . . . . . . . . . . 153

4.1.1.2

MatchedFilter Demodulation . . . . . . . . . . . . . . . . . . . 158

Optimal Detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164

4.2 Performance of Optimum Receivers . . . . . . . . . . . . . . . . . . . . . . . . . 174

4.2.1

Binary Modulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174

4.2.2

Mary PAM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182

4.2.3

Mary PSK . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 186

Schober: Signal Detection and Estimation

4
4.2.4

Mary QAM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187

4.2.5

Upper Bound for Arbitrary Linear Modulation Schemes . . . . . . . . . . 188

4.2.6

Comparison of Different Linear Modulations . . . . . . . . . . . . . . . . 190

4.3 Receivers for Signals with Random Phase in AWGN . . . . . . . . . . . . . . . . 192

4.3.1

Channel Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192

4.3.2

Noncoherent Detectors . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194

4.3.2.1

A Simple Noncoherent Detector for PSK with Differential Encoding (DPSK) . . . . . . . . . . . . . . . . . . . . . . . . . . . 195

4.3.2.2

Optimum Noncoherent Detection . . . . . . . . . . . . . . . . . 201

4.3.2.3

Optimum Noncoherent Detection of Binary Orthogonal Modulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 205

4.3.2.4

Optimum Noncoherent Detection of OnOff Keying . . . . . . . 210

4.3.2.5

MultipleSymbol Differential Detection (MSDD) of DPSK . . . 212

4.4 Optimum Coherent Detection of Continuous Phase Modulation (CPM) . . . . . 218

5 Signal Design for Bandlimited Channels

225

5.1 Characterization of Bandlimited Channels . . . . . . . . . . . . . . . . . . . . . 226

5.2 Signal Design for Bandlimited Channels

. . . . . . . . . . . . . . . . . . . . . . 228

5.3 DiscreteTime Channel Model for ISIfree Transmission

6 Equalization of Channels with ISI

. . . . . . . . . . . . . 240
244

6.1 DiscreteTime Channel Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . 245

6.2 MaximumLikelihood Sequence Estimation (MLSE) . . . . . . . . . . . . . . . . 247
6.3 Linear Equalization (LE) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 257
6.3.1

Optimum Linear ZeroForcing (ZF) Equalization . . . . . . . . . . . . . 258

6.3.2

ZF Equalization with FIR Filters . . . . . . . . . . . . . . . . . . . . . . 263

6.3.3

Optimum Minimum MeanSquared Error (MMSE) Equalization . . . . . 268

6.3.4

MMSE Equalization with FIR Filters . . . . . . . . . . . . . . . . . . . . 278

6.4 DecisionFeedback Equalization (DFE) . . . . . . . . . . . . . . . . . . . . . . . 283

6.4.1

Optimum ZFDFE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 288

6.4.2

Optimum MMSEDFE . . . . . . . . . . . . . . . . . . . . . . . . . . . . 295

6.5 MMSEDFE with FIR Filters . . . . . . . . . . . . . . . . . . . . . . . . . . . . 301

Schober: Signal Detection and Estimation

EECE 564
Signal Detection & Estimation


Instructor: Dr. Robert Schober, Kaiser Building, Room 4103,

Phone: 604-822-3515, Email: rschober@ece.ubc.ca.

Office hours: (Almost) any time or after lecture.

Text book: John G. Proakis, Digital Communications, 4th

Edition, McGrawHill.

Course notes and homework assignments: Download from

http://www.ece.ubc.ca/elec564/

Additional reading:
1. Oppenheim and Willsky, Signals & Systems, 2nd Edition,
Prentice Hall.
2. Papoulis, Probability, Random Variables, and Stochastic
Processes, McGrawHill.
3. Moon and Stirling, Mathematical Methods and Algorithms
for Signal Processing, Prentice Hall.
4. Haykin, Communications Systems, 4th Edition, Wiley.
5. Wozencraft and Jacobs (1965), Principles of Communication
Engineering, Wiley.

Grading:
Homework Assignments 10%
Midterm
30%
Final Exam
60%

Schober: Signal Detection and Estimation

II

Course Outline
1. Basic Elements of a Digital Communication System
2. Probability and Stochastic Processes a Brief Review
3. Characterization of Communication Signals and Systems
4. Detection of Signals in Additive White Gaussian Noise
5. Bandlimited Channels
6. Equalization

Schober: Signal Detection and Estimation

Basic Elements of a Digital Communication

System

Info
Source

Source
Encod.

Chan.
Encod.

Digital
Mod.

Channel

Info
Sink

1.1

Source
Decod.

Chan.
Decod.

Digital
Demod.

Transmitter

a) Information Source
analog signal: e.g. audio or video signal
digital signal: e.g. data, text
b) Source Encoder
Objective: Represent the source signal as efficiently as
possible, i.e., with as few bits as possible
minimize the redundancy in the source
encoder output bits

Schober: Signal Detection and Estimation

c) Channel Encoder
Objective: Increase reliability of received data
add redundancy in a controlled manner to
information bits
d) Digital Modulator
Objective: Transmit most efficiently over the
(physical) transmission channel
map the input bit sequence to a signal waveform
which is suitable for the transmission channel
Examples: Binary modulation:
bit 0 s0(t)
bit 1 s1(t)
1 bit per channel use
M ary modulation:
we map b bits to one waveform
we need M = 2b different waveforms to represent
all possible bbit combinations
b bit/(channel use)

Schober: Signal Detection and Estimation

1.2

Receiver

a) Digital Demodulator
Objective: Reconstruct transmitted data symbols (binary or
M ary from channelcorrupted received signal
b) Channel Decoder
Objective: Exploit redundancy introduced by channel encoder
to increase reliability of information bits
Note:

In modern receivers demodulation and decoding is

sometimes performed in an iterative fashion.

c) Source Decoder
Objective: Reconstruct original information signal from
output of channel decoder

Schober: Signal Detection and Estimation

1.3

Communication Channels

1.3.1

Physical Channel

a) Types
wireline
optical fiber
optical wireless channel
wireless radio frequency (RF) channel
underwater acoustic channel
storage channel (CD, disc, etc.)
b) Impairments
noise from electronic components in transmitter and receiver
amplifier nonlinearities
other users transmitting in same frequency band at the same
time (cochannel or multiuser interference)
linear distortions because of bandlimited channel
timevariance in wireless channels
For the design of the transmitter and the receiver we need a simple
mathematical model of the physical communication channel that
captures its most important properties. This model will vary from
one application to another.

Schober: Signal Detection and Estimation

1.3.2

Mathematical Models for Communication Channels

a) Additive White Gaussian Noise (AWGN) Channel

s(t)

r(t)
n(t)

r(t) = s(t) + n(t)

The transmitted signal is only attenuated ( 1) and impaired
by an additive white Gaussian noise (AWGN) process n(t).
b) AWGN Channel with Unknown Phase

ej
s(t)

r(t)
n(t)

r(t) = ej s(t) + n(t)

In this case, the transmitted signal also experiences an unknown
phase shift . is often modeled as a random variable, which is
uniformly distributed in the interval [, ).
Schober: Signal Detection and Estimation

c) Linearly Dispersive Channel (Linear Filter Channel)

s(t)

c(t)

r(t)
n(t)

c(t): channel impulse response; : linear convolution

r(t) = c(t) s(t) + n(t)
Z
c( ) s(t )d + n(t)
=

Transmit signal is linearly distorted by c(t) and impaired by AWGN.

d) Multiuser Channel
Two users:

s1(t)
r(t)
s2(t)

n(t)

Kuser channel:
r(t) =

K
X
k=1

Schober: Signal Detection and Estimation

sk (t) + n(t)

e) Other Channels
timevariant channels
stochastic (random) channels
fading channels
multipleinput multipleoutput (MIMO) channels
...

Schober: Signal Detection and Estimation

Some questions that we want to answer in this course:



Which waveforms are used for digital communications?

How are these waveforms demodulate/detect?

What performance (= bit or symbol error rate) can be achieved?

Schober: Signal Detection and Estimation