Slides 4tl4 PDF

CoE 4TL4, Hamilton 200
DIGITAL SIGNAL PROCESSING

(DSP)
Alex Gershman
Department of Electrical and Computer Engineering

McMaster University
COURSE ORGANIZATION
Webpage:
http://www.ece.McMaster.CA
Assessment:
Assignments: 15%
Labs: 15%
Midterm: 30%
Final exam: 40%
TU Darmstadt
Elektrotechnik und Informationstechnik
COURSE ORGANIZATION
Teaching assistants:
Mohamed Saad (tutorial), CRL/B104, ext. 27904
Ramy Gohary (labs), CRL/B104, ext. 27904
Abbas Ebrahimi-Moghadam (labs), CRL/B109B, 22009
TU Darmstadt
LAB SESSIONS
There are eight possible lab session times (maximal capacity 13 persons per
session):
Friday, every other week starting from September 6
Monday, every other week starting from September 9
Tuesday, every other week starting from September 10
Wednesday, every other week starting from September 11
Friday, every other week starting from September 13
Monday, every other week starting from September 16
Tuesday, every other week starting from September 17
Wednesday, every other week starting from September 18
TU Darmstadt
LAB SESSIONS
Five labs, everybody works alone, lab report deadline is one week
Please, choose today one of six possible lab session times (use sign-up sheets
near CRL/222)!
TU Darmstadt
CALENDAR OF DSP COURSE
Introduction
Time-Domain Analysis of Discrete Signals and Systems
Frequency-Domain Analysis Using Fourier
and z-Transforms
Fast Fourier Transform (FFT) Processing
Design of Nonrecursive and Recursive Digital Filters
Random Signal Analysis
DSP Processors
Adaptive Filtering and Adaptive Signal Processing
Spectral Analysis and Signal Compression
TU Darmstadt
COURSE TEXTBOOK
Alan V. Oppenheim and Ronald W. Schafer

Discrete-Time Signal Processing
1st (1989) or 2nd (1999) Edition
Prentice Hall, NJ
ISBN 0-13-754920-2
http://www.prenhall.com
TU Darmstadt
1 INTRODUCTION TO DISCRETE SIGNALS

1.1 The scope of DSP
Examples of analogue signals appearing in nature:
electrical signals: voltages, currents, fields

acoustic signals: mechanical vibrations, sound waves
mechanical signals: displacements, velocities, forces, moments
Analogue processing may include the following operations:
linear: amplification, filtering, integration, differentiation

nonlinear: squaring, rectification, inversion
TU Darmstadt
Limitations of practical analogue processing:
restricted accuracy (e.g., component variations with time,

temperature, etc.)
restricted dynamic range
sensitivity to noise
inflexibility to alter or adjust the processing functions
problems in implementing accurate nonlinear and time-synchronized
operations
high cost of data storage and transmission
limited speed of operation
TU Darmstadt
DSP operations:
converting analogue signals into a digital (usually binary) sequence

performing all signal processing operations in the digital form
if necessary, converting the digital information back to analogue signal
A typical DSP scheme
ANALOG ADC DAC ANALO

INPUT ANALOG DSP ANALOG
OUTP
( ANALOG-TO- ( DIGITAL- TO -
FILTER DIGITAL PROCESSOR ANALOG FILTER
CONVERTER ) CONVERTER )
TU Darmstadt
Advantages of DSP:
digital data storage and transmission is much more effective than in the
analogue form
flexibility: processing functions can be altered or adjusted
possibility of implementing much more complicated processing functions
than in analogue devices
efficient implementation of fast algorithms and matrix-based processing
speed of digital operation tends to grow rapidly with the years of technical
progress
a very high accuracy and reliability is possible to achieve
dynamic range can be increased
signal multiplexing: simultaneous (parallel) processing
TU Darmstadt 1
Some application areas of DSP:
Music: recording, playback, mixing, synthesis, storage (e.g. CD-players)

Speech: recognition, synthesis (e.g. automatic speakers)
Communications and multimedia: signal generation, transmission,
modulation and compression, data protection via error-correcting signal
coding, (e.g. digital modems, TV and telephony, computers,
videoconferencing and Internet)
Radar: filtering, detection, feature extraction, localization, tracking,
identification (e.g. air-traffic control)
Image processing: 2-D filtering, enhancement, compression, pattern
recognition (e.g. satellite images)
Biomedicine: diagnosis, patient monitoring, preventive care
TU Darmstadt 1
1.2 Sampling and Analog-to-Digital Conversion
s(t)
sampling of
analog signal
t
t
TU Darmstadt 1
Converting an analog signal into a binary code
quantization levels
6
5
4
3
2
1
0
0 1 2 3 4 5 6 7 8 9 10 11
sampling instants
sample 2 2 3 5 5 4 3 3 4 2 0 3
values
binary 010 010 011 101 101 10 0 011 011 10 0 010 0 0 0 011
codes
errors
TU Darmstadt 1
digital
0 1 2 3 4 5 6 7
111
110
101
10 0
analog
0 11
010
0 01
000
Input-output characteristic of 3-bit quantizer

TU Darmstadt 1
1.3 Basic Types of Digital Signals

Unit-step function:
(
0, n < 0
u(n) =
1, n 0
Unit-impulse function:
(
1, n = 0
(n) =
0, n =
6 0
n
X
u(n) = (m) integration
m=
(n) = u(n) u(n 1) differentiation
TU Darmstadt 1
The unit-step and unit-impulse functions

u(n)
1
(n)
1
n
TU Darmstadt 1
Periodic signals:

x(t) = x(t + T ) x(n) = x(n + N )
Finite-energy signals:
Z
X

E= |x(t)|2 dt < E= |x(n)|2 <
n=
Finite-power signals:
Z T N
X
1 2 1
P = lim |x(t)| dt < P = lim |x(n)|2 <
T 2T T N 2N
n=N
TU Darmstadt 1
Harmonic signals: an example of importance
sum of
all waves second
wave
first third
wave wave
TU Darmstadt 1
Complex exponentials (cisoids):

x(t) = A ej(t+) x(n) = A ej(n+)
Sinusoids:

x(t) = A sin(t + ) x(n) = A sin(n + )
t , i.e. discrete = analog t
ej(n+) = cos(n + ) + j sin(n + )

cos(n + ) = {ej(n+) + ej(n+)}/2
sin(n + ) = {ej(n+) ej(n+)}/2j
TU Darmstadt 1
A sine wave as the projection of a complex phasor onto the imaginary axis
Im

A
Re t
TU Darmstadt 2
Differences between sampled exponentials and their analog counterparts:
analog exponentials and (co)sinusoids are periodic with T = 2/,

discrete sinusoids are not necessarily periodic (although their values lie on
a periodic envelope).
Periodicity condition:
x(n) = x(n + N ) = ejn = ej(n+N ) = exp{jN } = 1

2m m
= = or f = ( = 2f )
N N
(this result applies to sines and cosines as well!)
for sampled exponentials, the frequency should be measured in [radians]
rather than [radians per second]
digital signals have ambiguity
TU Darmstadt 2
The unit-step function and one of many analog

signals which can be drawn through its sample points
u(n) , x(n)
1
n
x(n) Ambiguity in digital sinusoids
TU Darmstadt 2
Ambiguity condition for digital sinusoids:
sin(1 t) = sin(2 t) , 1 6= 2 =
2f1 t = 2f2 t + 2m , m = . . . , 2, 1, 1, 2, . . . =
|f1 f2| = m/t , m = 1, 2, . . . ()

Example: lowpass signal (whose spectrum |X(f )|2 is concentrated in the
interval [F, F ])
2
| X(f) |
-F F f
TU Darmstadt 2
Taking f1 = F and f2 = F in (), we obtain that there is no ambiguity

if the signal is sampled with
1
fs = > 2F ()
t
where fs is the so-called sampling frequency.
Remarks:
The frequency fN = 2F is referred to as the Nyquist rate
Digital signal ambiguity is often termed as the aliasing effect
Equation () represents the particular formulation of the
Shannon-Nyquist-Kotelnikov Sampling Theorem
Well study the differences between digital and analog signals further using
frequency-domain signal analysis
TU Darmstadt 2
2 TIME-DOMAIN ANALYSIS
2.1 Linear Time-Invariant (LTI) Systems
Definition of a system:
y(n) = T {x(n)}
where T {} is an operator that maps an input sequence x(n) into an
output sequence y(n).
Linear system: A system (or processor) is linear if it obeys the principle of
superposition.
Principle of superposition: If the input of a system contains the sum of
multiple signals then the output of this system is the sum of the system
responses to each separate signal.
TU Darmstadt 2
A system is linear if and only if:
T {ax1(n) + bx2(n)} = aT {x1(n)} + bT {x2(n)}

= ay1(n) + by2(n)
a x1(n) + b x 2 (n) Linear system a y1( n) + b y2 (n)

T{. }
TU Darmstadt 2
Example: Let y(n) = x2(n) (i.e., T {} = ()2). Then,
T {x1(n) + x2(n)} = x21(n) + x22(n) + 2x1(n)x2(n)

6= x21(n) + x22(n)
Hence, this system is nonlinear!
A time-invariant system has properties unvarying with time, i.e.:
if y(n) = T {x(n)} = y(n k) = T {x(n k)}
Linear Time-Invariant (LTI) system is a system that is both linear and

time-invariant (sometimes referred to as a Linear Shift-Invariant (LSI)
system)
TU Darmstadt 2
2.2 Digital Signals via Impulse Functions

x(n)
-2 -1 0 1 2 n
x(- 2 ) (n+2)
n
x(- 1) (n+1)
n
x(0) (n)
n
x(1) (n- 1)
n
x(2) (n-2)
n

X
x(n) = x(k)(n k)
k=
TU Darmstadt 2
Let h(n) be the response of the system to (n).

Due to the time-invariance property, the response to (n k)
is simply h(n k) =
y(n) = T
{x(n)}
X
= T x(k)(n k)

k=

X
= x(k)T {(n k)}
k=
X
= x(k)h(n k)={x(n)} {h(n)} convolution sum
k=
TU Darmstadt 2
The sequence {h(n)} is commonly referred to as impulse response of the

LTI system
(n) LTI h (n)

1 system
n n
An important property of convolution:

X
X
{x(n)} {h(n)} = x(k)h(n k) = h(k)x(n k)
k= k=
= {h(n)} {x(n)} =
the order in which two sequences are convolved is unimportant!
TU Darmstadt 3
Other properties of convolution:
{x(n)}[{h1(n)} {h2(n)}]
= [{x(n)} {h1(n)}]{h2(n)} associativity
{x(n)}[{h1(n)} + {h2(n)}]
= {x(n)} {h1(n)} + {x(n)} {h2(n)} distributivity
x(n) y(n)
h1(n) h2 (n) = x(n) h2(n) h1(n)
y(n)
= x(n) h1(n)* h2 (n) y(n)
h1(n)
x(n) y(n) x(n) y(n)
= h1(n) +h2 (n)
h 2(n)
TU Darmstadt 3
Example: Convolution of two rectangles:
y(n) = {x(n)} {x(n)}
x(k) N= 6
1
0 5 k
y(n)
6
0 5 10 n
TU Darmstadt 3
x(-7 -k ) x(k)
1 n=- 7 y =0
x(-1 -k ) x(k) k
n=- 1 y=0
x(2 -k ) x(k) k
n =2 y= 3
x(5 -k ) x(k) k
n =5 y=6
x(k) x(8 -k ) k
n =8 y= 3
x(k) x(11-k ) k
y=
n = 11 0
x(k) x(17-k ) k
n =17 y=0
k
TU Darmstadt 3
Example: Convolution of two sequences {x(n)} = {. . . , 0, 1, 2, 3, 0, . . .}

and {h(n)} = {. . . , 0, 2, 1, 0.5, 0, . . .}
3
x(k)
h(k) k
3
x(-k) k
3
x(- 2 - k) k
3
x(2 - k) k
3
TU Darmstadt 3
h(k), x(n-k)
3
2 2
1 1 0.5
k
y(n)
8.5
5
4
2 1.5
TU Darmstadt 3
Result: LTI systems are stable if and only if

X
|h(k)| <
k=
Proof of if: Let the input x(n) be bounded so that |x(n)| < Lx,
n [, ]. Then
X

|y(n)| = h(k)x(n k)
k=
X
|h(k)||x(n k)|
k=
X
X
Lx |h(k)| = |y(n)| < if |h(k)| <
k= k=
TU Darmstadt 3
Now, it remains to prove that if

X
|h(k)| =
k=
then a bounded input can be found that will cause an unbounded output.
Consider
(
h(n)/|h(n)| , h(n) 6= 0
x(n) = =
0, h(n) = 0;

X
X
y(0) = x(k)h(k) = |h(k)| =
k= k=
P
if k= |h(k)| = , the output sequence is unbounded.
TU Darmstadt 3
Definition: A causal system is one for which the output y(n) depends on
the inputs {. . . , x(n 2), x(n 1), x(n)} only.
Result: An LTI system is causal if and only if its impulse response
h(n) = 0 for n < 0.
Proof of if: From the definition of a causal system,

X
y(n) = h(k)x(n k)
k=
X
= h(k)x(n k)
k=0
P1
Obviously, this equation is valid if k= h(k)x(n k) = 0 i.e., if
h(n) = 0 for n < 0.
TU Darmstadt 3
Now, it remains to prove that if h(n) 6= 0 for n < 0, than the system
can be noncausal. Let
h(n) = 0 , n < 1
h(1) 6= 0 =

X
y(n) = h(k)x(n k) + h(1)x(n + 1) =
k=0
y(n) depends on x(n + 1) = the system is noncausal.
TU Darmstadt 3
Example: An LTI system with

(
an , n 0
h(n) = anu(n) =
0 , n < 0;
since h(n) = 0 for n < 0, the system is causal

To decide on stability, we must compute the sum

(
X X 1 , |a| < 1
S= |h(k)| = |a|k = 1|a| =
k= k=0
, |a| 1;
the system is stable only for |a| < 1
TU Darmstadt 4
2.3 Linear Constant-Coefficient Difference (LCCD) Equations
Consider LTI systems satisfying

N
X M
X
a(k)y(n k) = b(k)x(n k) ARMA
k=0 k=0
Particular cases:
M
X
y(n) = b(k)x(n k) MA
k=0
N
X
a(k)y(n k) = x(n) AR
k=0
TU Darmstadt 4
Example:
n
X
y(n) = x(k) accumulator
k=
n
X n1
X
y(n) y(n 1) = x(k) x(k)
k= k=
n1
X n1
X
= x(n) + x(k) x(k)

k= k=
= x(n)
TU Darmstadt 4
x(n) y(n)
+
one-
sample
delay
y(n- 1)
TU Darmstadt 4
Property: MA systems are bounded-input bounded-output (BIBO) stable,

i.e.
XM X M

|y(n)| = b(k)x(n k) |b(k)| |x(n k)|<
k=0 k=0
for any bounded input |x(n)| < and coefficient sequence |b(n)| < .
Remark: AR systems may be unstable. For example, the system
y(n) = ay(n 1) + x(n)
is unstable for a 1, because y(n) is unbounded for bounded x(n).
TU Darmstadt 4
Property: MA systems have finite impulse response (FIR), whereas AR

systems have infinite impulse response (IIR).
Proof for MA systems:

0, n<0
hMA(n) = b(n) , 0 n M ; = FIR!

0, n>M
Proof for AR systems: y(n) depends on y(n k), k = 1, . . . , =

y(n) depends on x(n k), k = 0, . . . , = the impulse
response hAR(n) is infinite, i.e., is in general case nonzero for all n > 0.
TU Darmstadt 4
Result: Suppose that for a given input x(n) we have found one particular
output sequence yP(n) so that a LCCD equation is satisfied. Then, the
same equation with the same input is satisfied by any output of the form
y(n) = yP(n) + yH(n)
where yH(n) is any solution to the LCCD equation with zero input
x(n) = 0.
Remark: yP(n) and yH(n) are referred to as the particular and
homogeneous solutions, respectively.
TU Darmstadt 4
Proof of Result: Taking the sum of

N
X M
X
a(k)yP(n k) = b(k)x(n k)
k=0 k=0
XN
a(k)yH(n k) = 0
k=0
we obtain
N
X M
X
a(k)y(n k) = b(k)x(n k)
k=0 k=0
with y(n) = yP(n) + yH(n). Result is proven.
TU Darmstadt 4
Property: A LCCD equation does not provide a unique specification of the

output for a given input.
Corollary: Auxiliary information or conditions are required to specify

uniquely the output for a given input.
Example: Let auxiliary information be in the form of N sequential output

values. Then,
later values can be obtained by rearranging LCCD equation as a recursive
relation running forward in n
prior values can be obtained by rearranging LCCD equation as a recursive
relation running backward in n.
TU Darmstadt 4
LCCD equations as recursive procedures:
M
X N
X
b(k) a(k)
y(n) = x(n k) y(n k) forwards
a(0) a(0)
k=0 k=1
M
X N
X 1
b(k) a(k)
y(n N ) = x(n k) y(n k) backwards
a(N ) a(N )
k=0 k=0
TU Darmstadt 4
Example: First-order AR system y(n) = ay(n 1) + x(n) with the given

input x(n) = b(n 1) and the auxiliary condition y(0) = y0.
Forwards recursion:
y(1) = ay0 + b ,
y(2) = ay(1) + 0
= a(ay0 + b) = a2y0 + ab ,
y(3) = a(a2y0 + ab) = a3y0 + a2b ,
,
y(n) = any0 + an1b
TU Darmstadt 5
Remark that:
y(n 1) = a1 (y(n) x(n)) =
Backwards recursion:
y(1) = a1(y0 0)
= a1y0 ,
y(2) = a2y0 ,
y(3) = a3y0 ,
,
y(n) = any0
TU Darmstadt 5
Question: is this system LTI and causal?
Auxiliary result: A linear system requires that the output be zero for all
time when the input is zero for all time.
Proof: Represent zero input as a product of zero constant c = 0 and
(arbitrary) non-zero signal x(n):
c x(n) = 0 x(n) = 0
Hence, for a linear system
y(n) = T {c x(n)} = c T {x(n)} = 0 T {x(n)}= 0
Result is proven.
TU Darmstadt 5
From the backwards recursion it follows that
y(n) = any0 6= 0 for a 6= 0 , y0 6= 0
whereas x(n) = 0 , n 0 =
according to Result, the system is nonlinear!
The system was implemented in both positive and negative directions

beginning with n = 0 = the system is noncausal!
TU Darmstadt 5
The forwards-backwards recursion can be rewritten for arbitrary n as
y(n) = any0 + an1bu(n 1)
Hence, the shift of the input by n0 samples,

x(n) = x(n n0) = b (n n0 1), gives
y(n) = any0 + ann01bu(n n0 1)6= y(n n0) =
the system is not time invariant!
TU Darmstadt 5
Example: First-order AR system y(n) = ay(n 1) + x(n) with the given

input x(n) = b(n 1) and the auxiliary condition y(0) = 0.
Recursion:
,
y(1) = 0 ,
y(0) = 0 ,
y(1) = a 0 + b
= b,
y(2) = ab ,
,
y(n) = an1b
TU Darmstadt 5
This recursion can be rewritten as:
y(n) = an1b u(n 1) , n
It is easy to prove that this system is a causal LTI system =
Linearity, time-invariance, and causality depend on auxiliary conditions!
TU Darmstadt 5
3 FREQUENCY-DOMAIN ANALYSIS
3.1 Review of Continuous-Time Fourier Transform
Consider a continuous complex signal
x(t) [T /2, T /2]
Let us represent it using an arbitrary orthonormal basis n(t):

X
x(t) = nn(t) ()
n=
Orthonormality condition
Z T /2
1
n(t)k (t) dt = (n k)
T T /2
TU Darmstadt 5
Multiplying () with k (t) and integrating over the interval, we obtain

Z T /2 Z T /2 X
1 1
x(t)k (t) dt = nn(t)k (t) dt
T T /2 T T /2
n=
X 1 Z T /2
= n n(t)k (t) dt
n
T T /2
X
= n(n k) = k =
n=
the coefficients of expansion are given by:

Z T /2
1
k = x(t)k (t) dt
T T /2
TU Darmstadt 5
Result: The functions n(t) = exp{j2nt/T } are orthonormal at the

interval [T /2, T /2].
Proof:
Z T /2 Z T /2
1 1 2(nk)
j T t
n(t)k (t) dt = e dt
T T /2 T T /2
sin((n k))
= = (n k)
(n k)
Result is proven, i.e., we can take exponential functions
n(t) = exp{j2nt/T } as orthonormal basis = we obtain Fourier
Series!
TU Darmstadt 5
Fourier Series for a periodic signal x(t) = x(t + T ):

X
j 2n
T t
x(t) = Xn e
n=
Z T /2
1 j 2n
Xn = x(t) e T t dt
T T /2
. . . Complex Fourier
__ t) X1 series representation
exp ( j 2 of a periodic signal
T
. . . x(t)
exp (j ___
2 n t)
T
Xn
. . .
TU Darmstadt 6
Fourier coefficients can be viewed as a signal spectrum:

2n
Xn X(n) , where n = =
T
Fourier series can be applied for analysis of signal spectrum! Also, this
interpretation shows that periodic signals have discrete spectrum.
TU Darmstadt 6
Example: Periodic sequence of rectangles:

Z T /2
1 j 2n
Xn = x(t) e T t dt
T T /2
Z /2
1 j 2n
= A e T t dt
T /2
A sin(n T )
= real coefficients
T n T
Remarks:
in general Fourier coefficients are complex-valued
for real signals Xn = Xn
alternative expressions for trigonometric Fourier series exist, exploiting
summation of sine and cosine functions
TU Darmstadt 6
x(t)
A
_
T
_
_ _T t
2 2 2 2
Xn
n
TU Darmstadt 6
What about Fourier representations of nonperiodic continuous-time

signals?
Assuming a finite-energy signal and T in the Fourier series, we
obtain limT Xn = 0.
x(t) = 1 1 Xn
20
T= 4 A In the limit as T ,
A 5
the Fourier coefficients
tend to zero and vanish
1 1 t
4 A Xn
10
1 t
A Xn
20
1 t
TU Darmstadt 6
Trick: To preserve the Fourier coefficients from degradation at T ,

introduce
Z T /2
j 2n
Xn = T Xn = x(t) e T t dt
T /2
Transition to Fourier transform:
X() = lim Xn
T
Z T /2
j 2n
= lim x(t) e T t dt
T T /2
Z
= x(t) ejt dt

where the discrete frequency 2n

T becomes the continuous frequency
TU Darmstadt 6
Transition to inverse Fourier transform:

X
j 2n
T t
x(t) = lim Xn e
T
n=
X
Xn j 2n t
= lim e T
T T
n=
Z
1 2 2n
= X()ejt d d = , =
2 T T
Continuous-time Fourier transform (CTFT):
Z
X() = x(t) ejt dt
Z
1
x(t) = X()ejt d
2
TU Darmstadt 6
Example: Finite-energy rectangular signal:

Z
X() = x(t) ejt dt
Z /2
= A ejt dt
/2
sin( /2)
= A real spectrum
/2
Remarks:
in general Fourier spectrum is complex-valued
for real signals X() = X ()
TU Darmstadt 6
x(t)
A

_ 0 _ t
2 2
X()
A
2
__ 2 4
__ __

TU Darmstadt 6
Definition of Dirac delta-function:

( Z
, t = 0
(t) = , (t) dt = 1
0, t =6 0

Do not confuse continuous-time (t) and discrete time (n)!
Sifting property:
Z
f (t) (t ) dt = f ( )

TU Darmstadt 6
Delta-function in time domain:

Z
1
(t) = ejt d
2
The spectrum of (t t0):

Z
X() = x(t) ejt dt
Z

= (t t0) ejt dt

= ejt0
TU Darmstadt 7
Delta-function in frequency domain:

Z (
1 jt , = 0
() = e dt =
2 0, =
6 0
Let the signal be

x(t) = A ej0t
Then,
Z
X() = x(t) ejt dt

Z
= A ej(0)t dt

= A 2 ( 0)
TU Darmstadt 7
Harmonic Fourier pairs:
ej0t 2 ( 0)
cos(0t) [( 0) + ( + 0)]

sin(0t) [( 0) ( + 0)]
j
x(t) = cos(0 t) X()

1 () ()
t 0 0 0
T= 2
0
TU Darmstadt 7
Parseval theorem for CTFT:

Z Z
2 1
|x(t)| dt = |X()|2 d
2
Proof:

Z Z ZZ

1 1
|X()|2 d = x(t)x( ) ej(t ) dt d d
2 2

ZZ
1 Z

= x(t)x( ) ej(t ) d dt d

2

| {z }
(t )
Z
= |x(t)|2 dt

TU Darmstadt 7
Example: Parseval theorem for the harmonic Fourier pair

ej0t 2 ( 0). The first part of Parseval equality:
Z Z
|x(t)|2 dt = |ej0t|2 dt =

Redefine delta-function as the following limit:
() = lim p()
0
where the pulselike function
(
1 ,
p() = 2 2
0, otherwise
TU Darmstadt 7
The second part of Parseval equality

Z Z
1
|X()|2 d = 2 2( 0) d
2
Z
= 2 lim p2( 0) d
0
Z0+
2
1
= 2 lim 2
d
0
0
2
1
= 2 lim =
0
TU Darmstadt 7
Multiplication of two identical pulselike functions
p() p() p2()

1
2
1 1

x =

2 2 2 2 2 2
TU Darmstadt 7
3.2 Discrete-Time Fourier Transform

Represent continuous signal x(t) via discrete sequence x(n):

X
x(t) = x(n t)(t n t)
n=
X
= x(n)(t n t)
n=
Substituting this equation in CTFT, we obtain:
Z X
X() = x(n)(t n t)ejt dt
n=

X Z
X
= x(n) (t n t)ejt dt = x(n)ejn t
n= n=
TU Darmstadt 7
In DTFT, let us use the normed frequency i.e., let t:
DTFT = CTFT t
Hence, the last expression for X() can be rewritten as

X
X() = x(n)ejn
n=
X() is periodic with 2:

X
X() = x(n)ejn e|j2n
{z }
n= 1
X
= x(n)ej(+2)n = X( + 2)
n=
TU Darmstadt 7
Trick: use in DTFT only one period of X():

X
X() = x(n) ejn DTFT
n=
Z
1
x(n) = X()ejn d Inverse DTFT
2
Z
1
x(n) = X()ejn d
2
Z X
1
= x(m) ej(nm) d
2
m=
X Z
1
= x(m) ej(nm) d = x(n)
2
m= | {z }
(nm)
TU Darmstadt 7
Compare Fourier series and DTFT:

T /2
Z

X
j 2n 1 j 2n
x(t) = Xn e T t , Xn = x(t) e T t dt FS
T
n=
T /2

X Z
1
X() = x(n) ejn , x(n)= X()ejn d DTFT
2
n=
Observation: replacing in Fourier series
x(t)X() Xnx(n) t T 2
we obtain DTFT!!!
TU Darmstadt 8
An important conclusion follows: DTFT is equivalent to Fourier series but

applied to the opposite domain. In Fourier series, a periodic continuous
signal is represented as a sum of exponentials weighted by discrete Fourier
(spectral) coefficients. In DTFT, a periodic continuous spectrum is
represented as a sum of exponentials weighted by discrete signal values.
Remarks:
DTFT can be derived directly from the Fourier series
all developments for Fourier series can be applied to DTFT
the relationship between Fourier series and DTFT illustrates the duality
between time and frequency domains
TU Darmstadt 8
Parseval theorem for DTFT:

X Z
2 1
|x(n)| = |X()|2 d
2
n=
Proof:
Z Z X

X
1 1
|X()|2 d = x(n)x(m)ej(nm) d
2 2
n= m=
Z
X X 1
=
x(n)x (m) ej(nm) d
2
n= m= | {z }
(nm)

X
= |x(n)|2
n=
TU Darmstadt 8
When does the DTFT exist (|X()| < )?
Sufficient condition:

X
|x(n)|
n=
Proof:
X

|X()| = x(n)ejn
n=
X
|x(n)| |e jn|
| {z }
n= 1
X
= |x(n)|
n=
TU Darmstadt 8
Example: Finite-energy rectangular signal:

N/2 N/2
X X
X() = Aejn = A ejn
n=N/2 n=N/2

sin N 2+1
= A (N + 1)
(N + 1) sin 2

sin N 2+1
' A (N + 1) N +1
for
| 2{z }
wellknown function
Both functions look very similar in their mainlobe domain!
TU Darmstadt 8
x(n)
A
0 n
X()
A(N+1)
2
__ 2 4__
__
N+1 N+1 N+1
TU Darmstadt 8
3.3 Properties of DTFT
Linearity:
If X() = F{x(n)} and Y () = F{y(n)}
then a X() + b Y () = F{a x(n) + b y(n)}
Also if x(n) = F 1{X()} and y(n) = F 1{Y ()}
then a x(n) + b y(n) = F 1{a X() + b Y ()}
Proof: elementary (direct substitution).
TU Darmstadt 8
Time shifting:
If X() = F{x(n)} then X()ejm = F{x(n m)}
Also if x(n) = F 1{X()} then x(n m) = F 1{X()ejm}

Proof:

X
X
F{x(n m)} = x(n m ) ejn = x(k)ej(m+k
| {z }
n= k k=

X
= ejm x(k)ejk = X()ejm
k=
F 1{X()ejm} = F 1{F{ x(n m)}} = x(n m)
TU Darmstadt 8
Frequency shifting:
If X() = F{x(n)} then X( ) = F{x(n)ejn}
Also if x(n) = F 1{X()} then x(n)ejn = F 1{X( )}

Proof:

X
F{x(n)ejn} = x(n)ej()n = X( )
n=
F 1{X( )} = F 1{F{ x(n)ejn}} = x(n)ejn
TU Darmstadt 8
Time reversal:
If X() = F{x(n)} then X() = F{x(n)}
Also if x(n) = F 1{X()} then x(n) = F 1{X()}

Proof:

X
X
F{x(n)} = n )ejn =
x(|{z} x(m)ejm = X()
n= m m=
F 1{X()} = F 1{F{ x(n)}} = x(n)
TU Darmstadt 8
Differentiation in frequency:
dX()
If X() = F{x(n)} then j = F{n x(n)}
d
1 1 dX()
Also if x(n) = F {X()} then n x(n) = F {j }
d
Proof:

X
X d(ejn)
F{n x(n)} = nx(n)ejn = j x(n)
d
n= n=
d n
X dX()
= j x(n) e jn =j
d d
n=
1 dX()
F {j } = F 1{F{ n x(n)}} = n x(n)
d
TU Darmstadt 9
Convolution theorem:
If X() = F{x(n)} , H() = F{h(n)} ,

X
and y(n) = x(k)h(n k) = {x(n)} {h(n)}
k=
then Y () = F{y(n)} = X()H()
Convolution of sequences in time-domain is equivalent to multiplication of

the corresponding Fourier transforms in frequency domain.
TU Darmstadt 9
Proof of convolution theorem:

X
X
Y () = F{y(n)} = x(k)h(n k ) ejn
| {z }
n= k= m

X
X
= x(k)h(m)ej(m+k)
m=
k=

X
X

= x(k)ejk h(m)ejm

k= m=
= X()H()
TU Darmstadt 9
Windowing theorem:
If X() = F{x(n)} , W () = F{w(n)} ,
and y(n) = x(n)w(n)

Z
1
then Y () = F{y(n)} = X()W ( ) d
2

Multiplication of sequences in time-domain is equivalent to periodic

convolution of the corresponding Fourier transforms in frequency domain.
Proof: by means of direct substitution, similarly to the proof of convolution

theorem.
TU Darmstadt 9
Generalized Parseval theorem:
If X() = F{x(n)} , Y () = F{y(n)} ,

X Z
1
then x(n)y (n) = X()Y () d
2
n=
Proof: similarly to the proof of Parseval theorem.
TU Darmstadt 9
Summary of main properties of DTFT

Sequence x(n) Fourier Transform X()
a x(n) + b y(n) a X() + b Y ()
x(n) X ()
x(n) X ()
x(n m) ejmX()
ejnx(n) X( )
x(n) X()
dX()
n x(n) j d
{x(n)} {h(n)} X()H()
R
1 X()W ( ) d
x(n)w(n) 2 R
P 2 1 |X()|2 d
|x(n)|
Pn= (n) 1
2
R

() d
n= x(n)y 2 X()Y
TU Darmstadt 9
3.4 Frequency-Domain Representation

of Discrete-Time Signals and Systems
Recall impulse response h(n) of an LTI system:

X
y(n) = h(k)x(n k)
k=
Consider an input sequence x(n) = ejn, < n <

X n X
o
y(n) = h(k)ej(nk) = ejn h(k)ejk
k= k=
| {z }
H()
= ejnH()
TU Darmstadt 9
The complex function

X
H() = h(k)ejk
k=
is called the frequency response or the transfer function of the system.
Remarks:
The impulse response and transfer function represent a DTFT pair =
H() is a periodic function
The transfer function shows how different input frequency components are
changed (e.g., attenuated) at system output
This function will be very useful for the consideration of signal filtering
TU Darmstadt 9
Interpretation of impulse
and frequency responses
(n) h(n)
LTI system
jn
e jn LTI system
e H()
TU Darmstadt 9
Example: the delay system:
y(n) = x(n nd) with fixed integer nd
x(n) = ejn = y(n) = ej(nnd) = H() = ejnd

Since |H()| = 1, this system is frequency nonselective.
Examples of frequency selective systems will be given when the filtering

operation will be considered.
TU Darmstadt 9
3.5 Elements of Sampling Theory
Question: How are CTFT and Fourier series related for periodic signals?
Consider a signal x(t) with CTFT
X() = 2 ( 0)
The signal itself is determined via the inverse relation

Z
1
x(t) = 2 ( 0)ejt d = ej0t
2
TU Darmstadt 10
We know that periodic signals have line equispaced spectrum. Let X()
be a linear combination of impulses equally spaced in frequency:

X
X() = 2Xn( n0) ()
n=
Using inverse CTFT, i.e., applying it to each term in the sum, we obtain
X Z
1
x(t) = 2Xn( n0)ejt d
2
n=
X
= Xnejn0t exactly Fourier series!
n=
TU Darmstadt 10
Result: CTFT of a periodic signal with Fourier series coefficients {Xn}

can be interpreted as a train of impulses occurring at the harmonically
related frequencies with the weights {2Xn}.
Remark: Periodic impulse train () is neither absolutely nor square

summable. Hence, the CTFT is introduced and understood in a limiting
sense.
TU Darmstadt 10
Result: the Fourier transform of a periodic impulse train is a periodic

impulse train.
X X
2 2n
(t k t)
t t
k= n=
x(t)
t
t X()

2
__
t
TU Darmstadt 10
P
Proof: The impulse train k= (t k t) is a periodic signal with
period t = applying Fourier series, we obtain, that the Fourier
coefficients
Z t/2
1 j 2n t 1
Xn = (t)e t dt =
t t/2 t
>From (), we obtain

X
X() = 2Xn( n |{z}
0 )
n= 2
t

X
2 2n
=
t t
n=
Result is proven.
TU Darmstadt 10
Let

X
s(t) = (t nt)
n=
Introduce the modulated signal

X
xs(t) = x(t) s(t) = x(t) (t nt)
n=
Since x(t)(t t0) = x(t0)(t t0), we obtain

X
xs(t) = x(n t)(t nt)
n=
Using this modulated signal, we characterize the operation of sampling.
TU Darmstadt 10
x(t)
from analog to
digital signal using
t impulse train
s(t)
x s (t) t
t
x(n) t
TU Darmstadt 10
Use windowing theorem:

1
xs(t) = x(t) s(t) = Xs() = {X()} {S()}
2
Hence,
Z
1
Xs() = X()S( ) d
2
Z X
1 2 2n
= X() d
2 t t
n=
X
1 2n
= X
t t
n=
TU Darmstadt 10
Effect in frequency domain when sampling is done in time domain faster

2 > 2 )
than at Nyquist rate ( t M
X()
1
M 0 M
S()
. . . .
2__
0 __
2
t t
Xs()
__
1
. . t . .
M 0 M __
2
t
TU Darmstadt 10
Effect in frequency domain when sampling is done in time domain exactly

2 = 2 )
at Nyquist rate ( t M
X()
1
M 0 M
S()
. . . .
2__t 0 __
2 =2
t
Xs()
M
__
1
. . t . .
M 0 M 2M
TU Darmstadt 10
Effect in frequency domain when sampling is done in time domain slower

2 < 2 )
than at Nyquist rate ( t M
X()
1
M 0 M
S()
. . . .
2__ 0 2__

t t
1 Xs()
__
t
. . . .
0
note aliazing!!!
TU Darmstadt 11
Introduce a lowpass filtering operation. The spectrum of the filtered signal
Xf () =HLP()Xs()
where an ideal filter transfer function

(
t , c c
HLP() =
0, otherwise
with the cut-off frequency c
TU Darmstadt 11
Perfect recovery of an analog signal from its samples using an ideal

lowpass filter
X()
1
M 0 M
1 X () H ()
t s , LP
t
. . . .
C 0 C
Xf ()
1
M 0 M
TU Darmstadt 11
Poor recovery of an analog signal from its samples using an ideal lowpass
filter
X()
1
M 0 M
t
Xs(), HLP()
1
t
. . . .
C 0 C
Xf ()
1
C 0 C
TU Darmstadt 11
Perfect recovery of a cosine at 0: no aliasing because the sampling rate is

2 > 2 )
higher than Nyquist rate ( t 0
X()

0 0 0
X () H ()
t s , LP
t
t 0
t

Xf ()

0 0 0
TU Darmstadt 11
Poor recovery of a cosine at 0: the aliasing occurs because the sampling

2 < 2 )
rate is lower than Nyquist rate ( t 0
X()

0 0 0
X () H ()
t s , LP
t
t 0
t

Xf ()

2 0 2
( t 0) t 0
TU Darmstadt 11
Now, let us determine how to reconstruct a bandlimited signal from its

samples in the time domain.
Result: Having a signal sampled at a rate higher than Nyquist rate and
infinite number of its discrete values, the signal can be exactly recovered as

X sin[(t n t)/t]
xf (t) = x(n)
(t n t)/t
n=
This equation is referred to as the Shannon-Nyquist-Kotelnikov

interpolation formula.
TU Darmstadt 11
Proof: We have seen that the signal can be reconstructed in the frequency
domain using ideal lowpass filter:
Xf () = Xs()HLP()
In time-domain:
xf (t) = {xs(t)} {hLP(t)}

n X o
= x(n t)(t nt) {hLP(t)}
n=
Z X
= x(n t)( nt)hLP(t ) d
n=
X
= x(n)hLP(t n t)
n=
TU Darmstadt 11
Ideal transfer function:

(

t , t
HLP() = t
0, otherwise
Ideal impulse response

sin(t/t)
hLP(t) = ()
t/t
Now, insert () into the equation for xf (t). Result is proven.
TU Darmstadt 11
generic digital system structure
Antialiasing Reconstruction
x(t) filter
x(n) y(n) filter
y(t)
DSP
and ADC (DAC)
TU Darmstadt 11
Sampling rate reduction by an integer factor:
xd(n) = x(nL) decimation, downsampling
x(n) x(nL)
L
sampling sampling
interval t interval L t
.
To avoid aliasing, the signal x(n) should be filtered with the cutoff
frequency M < /t = for decimated signal, L times lower
cutoff frequency d,M < /Lt is required, so that
M = L d,M
TU Darmstadt 12
Consider the DTFT of xd(n):

L1
X
1 2i
Xd() = X =
L L
i=0
the spectrum for the decimated sequence xd(n) represents the sum of
scaled and shifted copies of the spectrum for the conventional sequence
x(n)!
For the DTFT-frequencies, the non-aliasing conditions are:
M < , Ld,M <
TU Darmstadt 12
The effect of frequency scaling for the decimated sequence: no aliasing

because LM <
X()
M 0 M 2
Xd()
0 L M 2
TU Darmstadt 12
The effect of frequency scaling for the decimated sequence: the aliasing
occurs because LM >
X()
M 0 M 2
Xd()
0 LM 2
TU Darmstadt 12
Increasing the sampling rate by an integer factor:
x(n) = x(n t) xi = x(n t0) ,

t0 = t/L interpolation, upsampling
Interpolation formula can be used:

X
X
sin[(n kL)/L] sin[(n m)/L]
xi(n) = x(k) = xp(m)
(n kL)/L (n m)/L
k= m=
{xp(m)} = {. . .,x(1), 0,.

| {z. .,0} , x(0), 0,.
| {z. .,0} ,x(1), 0,.
| {z. .,0} ,x(2),. . .}
L1 points L1 points L1 points
= the upsampling can be done by means of filtering of the
zero-padded signal xp(m) using the ideal lowpass filter with the cutoff
/L!
TU Darmstadt 12
Upsampling system
x(n) zero xp(n) lowpass xi (n)

padding filtering
TU Darmstadt 12
Associated sequences and spectra for upsampling with the factor L = 2
X()
x(n)
n Xp()
xp(n)
n Xi ()
x i (n)
TU Darmstadt 12
4 z-TRANSFORM
4.1 Definition and the Regions of Convergence
Consider a continuous-time LTI system with x(t) = est:
Z
y(t) = h( )x(t ) d
Z

= h( )es(t ) d

Z
= est h( )es d = H(s)est =
| {z }
H(s)
Z
H(s) = h(t)est dt , s = + j Laplace transform

TU Darmstadt 12
The Laplace transform is extremely useful tool for continuous-time LTI

system analysis. What about discrete-time LTI system analysis?
Let the system input is a complex exponential signal:
x(n) = z n = y(n) = H(z) z n ,

X
H(z) = h(n)z n transfer function =
n=
we can introduce

X
X(z) = Z{x(n)} = x(n)z n ztransform
n=
TU Darmstadt 12
Relationship between the z-transform and DTFT: substitute z = rej :

X n
X(z) = x(n) rej
n=
X
= {x(n)rn}ejn
n=
= F{x(n)rn} =
z-transform of an arbitrary sequence x(n) is equivalent to DTFT of the

exponentially weighted sequence x(n)rn.
If r = 1 then

X(z) = X() = F{x(n)} =
z=ej
DTFT corresponds to the particular case of z-transform with |z| = 1!
TU Darmstadt 12
Complex z-plane. The z-transform reduces to the DTFT for values of z on

the unit circle
unit circle Im z=e j

1
Re
TU Darmstadt 13
Question: When does the z-transform converge?
>From the relationship between DTFT and z-transforms it follows that in

the general case of finite-energy signals, the z-transform does not converge
for all values of z = there is a range of values of z (referred to as the
Region Of Convergence (ROC)) for which |X(z)| < .
Example: the z-transform of the signal x(n) = anu(n)

X
X
X(z) = anu(n)z n= (az 1)n
n= n=0
For convergence, we require that

X
|az 1|n <
n=0
TU Darmstadt 13
Recall that

(
X 1 , |c| < 1
|c|n = 1|c| =
n=0
, |c| 1
the ROC is determined by |z| > |a|
Im
a 1 Re
TU Darmstadt 13
Example: now, let the signal be x(n) = anu(n 1)

X X1
X(z) = anu(n 1)z n = anz n
n= n=
X
X n
= anz n = 1 a1z =
n=1 n=0
the ROC is determined by |z| < |a|
Im
1
a Re
TU Darmstadt 13
4.2 Properties of the ROC

Property 1: The ROC of X(z) consists of a ring in the z-plane centered
about the origin.
Im
Re
TU Darmstadt 13
Property 2: The ROC does not contain any poles.
Property 3: If x(n) is of finite duration, then the ROC is the entire

z-plane except possibly z = 0 and/or z = .
N2
X
X(z) = x(n)z n finite duration signal
n=N1
Particular cases:
if N1 < 0 and N2 > 0 then the ROC does not include z = 0 and z =
if N1 0 then the ROC includes z = , but does not include z = 0
if N2 0 then the ROC includes z = 0, but does not include z =
TU Darmstadt 13
Property 4: If x(n) is a right-sided sequence, and if the circle |z| = r0 is

in the ROC, then all finite values of z for which |z| > r0 will also be in
the ROC.

X
X(z) = x(n)z n right sided sequence
n=N1
Particular cases:
if N1 < 0 then the ROC does not include z =
if N1 0 then the ROC includes z =
TU Darmstadt 13
Property 5: If x(n) is a left-sided sequence, and if the circle |z| = r0 is in

the ROC, then all values of z for which 0 < |z| < r0 will also be in the
ROC.
N2
X
X(z) = x(n)z n left sided sequence
n=
Particular cases:
if N2 > 0 then the ROC does not include z = 0
if N2 0 then the ROC includes z = 0
TU Darmstadt 13
Property 6: If x(n) is a two-sided sequence, and if the circle |z| = r0 is in

the ROC, then the ROC will be a ring in the z-plane that includes the
circle |z| = r0.

X
X(z) = x(n)z n two sided sequence
n=
Any two-sided sequence can be represented as a direct sum of a right-sided

and left-sided sequences = the ROC of this composite signal will be
the intersection of the ROCs of the components.
TU Darmstadt 13
To property 1: a) the ROC must be a connected region, b) the ROC

cannot be nonsymmetric
Im Im
Re Re
a) b)
TU Darmstadt 13
To property 6: intersection of the ROCs of right-sided and left-sided

sequences
Im Im Im
Re Re
U
Re
=
TU Darmstadt 14
To properties 2, 4, 5, 6:hpole-zero pattern and threei possible ROCs that

1
1 1
correspond to X(z) = (1 3 z )(1 1.3z ) 1
Im Im Im
1 Re 1 Re 1 Re
a) right-sided b) left-sided c) two-sided
TU Darmstadt 14
4.3 The Inverse z-Transform

We obtained that

X(z) = F{x(n)rn}
z=rej
Applying the inverse DTFT, we get
x(n) = rnF 1Z{X(rej )}

1
= r n X(rej )ejn d
2
Z
1 j )(rej )n d
= X(re
|{z} |{z}
2 I z z
1
= X(z)z n1 dz dz =jrej d
2j
TU Darmstadt 14
Remarks on inverse z-transform:

H
dz denotes integration around a closed circular contour centered at
the origin and having the radius r
the value of r must be chosen so that the contour of integration |z| = r
belongs to the ROC
contour integration in the complex plane may be a complicated task
simpler alternative procedures exist for obtaining the sequence from its
z-transform
TU Darmstadt 14
4.4 Alternative Methods for Inverse z-Transform

Inspection method: consists simply of becoming familiar with (or
recognizing by inspection) certain transform pairs.
Examples:
1
anu(n) 1
, |z| > |a|
1 az
(n m) z m , z 6= 0 if m > 0 , z 6= if m < 0
[sin ]z 1
sin(n) u(n) 1 2
, |z| > 1
1 [2 cos ]z + z
TU Darmstadt 14
Extended inspection method: consists of expressing a complicated

X(z) as a sum of simpler terms and then applying to each term the
inspection method.
Example:
X(z) = z 2(1 0.5z 1)(1 + z 1)(1 z 1)

= z 2 0.5z 1 + 0.5z 1 =
x(n) = (n + 2) 0.5(n + 1) (n) + 0.5(n 1)
TU Darmstadt 14
4.5 Properties of z-Transform
Linearity:
If X(z) = Z{x(n)} and Y (z) = Z{y(n)}
then a X(z) + b Y (z) = Z{a x(n) + b y(n)}
Also if x(n) = Z 1{X(z)} and y(n) = Z 1{Y (z)}
then a x(n) + b y(n) = Z 1{a X(z) + b Y (z)}
with ROC = ROCx ROCy
TU Darmstadt 14
Time shifting:
If X(z) = Z{x(n)} then X(z)z m = Z{x(n m)}
Also if x(n) = Z 1{X(z)} then x(n m) = Z 1{X(z)z m}
with ROC = ROCx except for possible
addition/deletion of z = 0 or z =
Example: for |z| > 0.25, consider

1 1 1
X(z) = = z
z 0.25 1 0.25z 1
= z 1Z{0.25nu(n)} = x(n) = 0.25n1u(n 1)

TU Darmstadt 14
Multiplication by exponential sequence:
If X(z) = Z{x(n)} then X(z/z0) = Z{x(n)z0n}
Also if x(n) = Z 1{X(z)} then x(n)z0n = Z 1{X(z/z0)}
with ROC = |z0| ROCx
Differentiation of X(z):
dX(z)
If X(z) = Z{x(n)} then z = Z{n x(n)}
dz
1 1 dX(z)
Also if x(n) = Z {X(z)} then n x(n) = Z { z }
dz
with ROC = ROCx
TU Darmstadt 14
Example: Starting with the known transform pair

1
u(n) 1
, |z| > 1
1z
determine X(z) of
n n
x(n) = 2rn cos(n)u(n) = rej u(n) + rej u(n)
Using the exponential multiplication property, we have

n 1
re j u(n) , |z| > |r|
1
1 z re j
n 1
re j u(n) , |z| > |r|
1
1 z re j
Using the linearity property, we obtain

1 1
X(z) = + , |z| > |r|
1 z 1rej 1 z 1rej
TU Darmstadt 14
Conjugation of a complex sequence:
If X(z) = Z{x(n)} then X (z ) = Z{x(n)}
Also if x(n) = Z 1{X(z)} then x(n) = Z 1{X (z )}
with ROC = ROCx
Time reversal:
If X(z) = Z{x(n)} then X(1/z) = Z{x(n)}
Also if x(n) = Z 1{X(z)} then x(n) = Z 1{X(1/z)}
with ROC = 1/ROCx
TU Darmstadt 15
Example: Consider the sequence
x(n) = anu(n)
which is a time-reversed version of

1
y(n) = anu(n) Y (z) = 1
, |z| > |a|
1 az
>From the time reversal property
X(z) = Y (1/z)
1
= , |z| < |a1|
1 az
TU Darmstadt 15
Convolution of sequences:
If X(z) = Z{x(n)} and Y (z) = Z{y(n)}
then X(z)Y (z) = Z{{x(n)} {y(n)}}
Also if x(n) = Z 1{X(z)} and y(n) = Z 1{Y (z)}
then {x(n)} {y(n)} = Z 1{X(z)Y (z)}
with ROC = ROCx ROCy
TU Darmstadt 15
Example: Evaluate the convolution of
x(n) = anu(n) and y(n) = u(n)
for |a| < 1. The z-transforms are

1
X(z) = 1
, |z| > a
1 az
1
Y (z) = 1
, |z| > 1 =
1z
z2
Z{{x(n)} {y(n)}} = X(z)Y (z) =
(z a)(z 1)

1 1 a
= 1
1
, |z| > 1
1a 1z 1 az
TU Darmstadt 15
Using the linearity property and the standard z-transform pairs, we obtain
that
1
{x(n)} {y(n)} = u(n) an+1u(n) , |z| > 1
1a
Pole-zero ROC plot for the z-transform of the convolution of the

sequences u(n) and an+1u(n)
Im
a 1 Re
TU Darmstadt 15
4.6 Analysis of LTI Systems Using z-Transforms
From the convolution property
y(n) = {h(n)} {x(n)} Y (z) = H(z)X(z)
H(z) = Z{h(n)} transfer function
Interpretation of the transfer function
zn LTI system H(z)zn
TU Darmstadt 15
Result: A discrete-time LTI system is causal if and only if the ROC of its
transfer function is the exterior of a circle including infinity.
Proof: follows from the ROC properties and from the fact that h(n) is a
right-sided sequence.
Result: A discrete-time LTI system is stable if and only if the ROC of its
transfer function H(z) includes the unit circle |z| = 1.
Proof: elementary, based on the definition of stability.
TU Darmstadt 15
Example:
1
H(z) =
1 [2r cos ]z 1 + r2z 2
Im Im
r
r 1 Re
1 Re
stable system( r< 1) unstable system (r> 1)

TU Darmstadt 15
Recall on LCCD equations of ARMA processes

N
X M
X
a(k)y(n k) = b(k)x(n k) ()
k=0 k=0
Taking z-transforms of both sides of ()
N
X M
X
a(k)Z{y(n k)} = b(k)Z{x(n k)}
k=0 k=0
and using time-shifting property, we obtain
N
X M
X
Y (z) a(k)z k = X(z) b(k)z k
k=0 k=0
TU Darmstadt 15
Hence, the transfer function of an ARMA process

PM k
Y (z) k=0 b(k)z B(z)
H(z) = = PN =
X(z) a(k)z k A(z)
k=0
X(z) __
1 Y(z)
B(z) A(z)
TU Darmstadt 15
5 DISCRETE FOURIER TRANSFORM AND FF

5.1 Discrete Fourier Transform (DFT)
Recall the DTFT

X
X() = x(n) ejn
n=
The DTFT is not suitable for a practical DSP because:

in any DSP application, we are able to compute the spectrum only at
specific discrete values of
any signal in any DSP application can be measured only in a finite number
of points
TU Darmstadt 16
A finite signal measured at N points:

0, n<0
x(n) = y(n) , 0 n (N 1)

0, nN
where y(n) are the measurements taken in N points.

Let us sample the spectrum X() in frequency domain so that
2
X(k) = X(k ) , = =
N
N
X 1
j2 kn
X(k) = x(n) e N DFT
n=0
TU Darmstadt 16
Result: the inverse DFT is given by

N
X 1
1 j2 kn
x(n) = X(k) e N
N
k=0
Proof:

1 X X
N 1 N 1
km
j2 N j2 kn
x(n) = x(m) e e N
N
k=0 m=0

N
X 1 1 NX
1 k(mn)
= x(m) ej2 N = x(n)
N
m=0 k=0
| {z }
(mn)
TU Darmstadt 16
The DFT transform:

N
X 1
j2 kn
X(k) = x(n) e N analysis
n=0
N
X 1
1 j2 kn
x(n) = X(k) e N synthesis
N
k=0
Alternative formulation:
N
X 1
j 2
X(k) = x(n) W kn W = e N
n=0
N
X 1
1
x(n) = X(k) W kn
N
k=0
TU Darmstadt 16
k0 Schematic
W =1 representation
x(0) of DFT
. . . X(k)
k(N-1)
W
x(N-1 )
TU Darmstadt 16
An important property of DFT-spectrum:

N
X 1 (k+N )n
j2
X(k + N ) = x(n) e N
n=0

NX1
j2 kn
N ej2n
= x(n) e
n=0
= X(k) ej2n = X(k) =
the DFT-spectrum X(k) is periodic with period N (recall that

DTFT-spectrum is periodic as well, but with period 2)
TU Darmstadt 16
Example: DFT of rectangular pulse

(
1 , 0 n (N 1)
x(n) =
0, otherwise
N
X 1
j2 kn
X(k) = e N = N (k) =
n=0
the rectangular pulse is interpreted by the DFT as a spectral line at the
frequency = 0
TU Darmstadt 16
DFT and DTFT of a rectangular pulse (N = 5)
x(n) signal for DFT (N= 5)

1
signal for DTFT
01 4
X(k) , X()
5
0 5 10
0 2 4
TU Darmstadt 16
Example (continued): What happens with the DFT of this rectangular

pulse if well increase N , i.e., include in the number of signal points severa
zeros? That is, let the DFT sequence be
{y(n)} = {x(0), . . . , x(M 1), |0, 0,{z

. . . , 0} }
N M positions
where x(0) = = x(M 1) = 1. Hence, the DFT:

N
X 1 M
X 1
j2 kn j2 kn
Y (k) = y(n) e N = e N
n=0 n=0

sin kM
N k(M 1)
e j
= N
k
sin N
TU Darmstadt 16
DFT and DTFT of a rectangular pulse with zero-padding (N = 10,

M = 5)
x(n) zero-padded
signal for DFT (N=10)
1
01 4 9
Y(k) , X()
5
0 5 10
0 2
TU Darmstadt 16
Corollary: using more and more zero-padding on analyzed sequence, we are

able to approximate its DTFT better and better
Corollary: zero-padding cannot improve the resolution of spectral

components because the resolution is proportional to 1/M rather than
1/N
Remark: zero-padding will be a very important tool for a fast

implementation of DFT
TU Darmstadt 17
5.2 Matrix Formulation of DFT

Introduce the N 1 vectors
x = [x(0), x(1), . . . , x(N 1)]T

X = [X(0), X(1), . . . , X(N 1)]T
and the N N matrix

W0 W0 W0 W0
0
W W1 W2 W N 1
0
W = W W2 W4 W 2(N 1)

2
W 0 W N 1 W 2(N 1) W (N 1)
TU Darmstadt 17
DFT in a matrix form:
X = Wx
Result: The inverse DFT is given by

1 H
x= W X
N
Proof: elementary, using the direct checking of the fact that
WH W = WWH = N I, where I is the identity matrix.
DFT Inverse DFT

x X X 1 H x
W NW
TU Darmstadt 17
5.3 Frequency Interval and Frequency Resolution

Since the DFT is periodic with the period N , its frequency resolution
1
f ' [Hz]
N t
and covered frequency interval
1
F ' N f = [Hz] =
t
the frequency resolution is determined only by the inversed length of the
observation interval, whereas the frequency interval is determined only by
the inversed length of sampling interval = increasing the sampling
rate we can expand the frequency interval, and increasing the observation
time, we can improve frequency resolution!
TU Darmstadt 17
Question: Does zero-padding alter the frequency resolution?

The answer is negative because the resolution is determined only by the
length of observation interval, and, obviously, zero-padding does not
increase this length
Example: Let two complex exponentials with the close frequencies

f1 = 10 Hz and f2 = 12 Hz be sampled with the sampling interval
t = 0.02 seconds and let us consider various data lengths
N = 10, 15, 30, 100, 300 with zero-padding of each data to 512 points.
TU Darmstadt 17
DFT with N = 10 and zero-padding to 512 points. The signals are

unresolved because f2 f1 = 2 Hz < 1/(N t) = 5 Hz
15
N=10
10
|DFT(x)|
0
25 20 15 10 5 0 5 10 15 20 25
FREQUENCY (HZ)
TU Darmstadt 17
DFT with N = 15 and zero-padding to 512 points. The signals are still
unresolved because f2 f1 = 2 Hz < 1/(N t) ' 3.3 Hz
16
N=15
14
12
10
|DFT(x)|
0
25 20 15 10 5 0 5 10 15 20 25
FREQUENCY (HZ)
TU Darmstadt 17
DFT with N = 30 and zero-padding to 512 points. The signals are

resolved because f2 f1 = 2 Hz > 1/(N t) ' 1.7 Hz
40
N=30
35
30
25
|DFT(x)|
20
15
10
0
25 20 15 10 5 0 5 10 15 20 25
FREQUENCY (HZ)
TU Darmstadt 17
DFT with N = 100 and zero-padding to 512 points. The signals are well
resolved because f2 f1 = 2 Hz > 1/(N t) = 0.5 Hz
110
N=100
100
90
80
70
60
|DFT(x)|
50
40
30
20
10
0
25 20 15 10 5 0 5 10 15 20 25
FREQUENCY (HZ)
TU Darmstadt 17
DFT with N = 300 and zero-padding to 512 points. The signals are fine
resolved because f2 f1 = 2 Hz 1/(N t) ' 0.17 Hz
N=300
300
250
200
|DFT(x)|
150
100
50
0
25 20 15 10 5 0 5 10 15 20 25
FREQUENCY (HZ)
TU Darmstadt 17
Better representation of the previous DFT plot by means of increasing the

number of zeros: zero-padding to 2048 points.
300 N=300
250
200
|DFT(x)|
150
100
50
0
25 20 15 10 5 0 5 10 15 20 25
FREQUENCY (HZ)
TU Darmstadt 18
5.4 Interpretation of DFT via Discrete Fourier Series

Construct a periodic sequence by repeating the finite sequence x(n),
n = 0, . . . , N 1:
{x(n)} = {. . . , x(0),
| . . . {z
, x(N 1)}, x(0),
| . . . {z
, x(N 1)}, . . .}
{x(n)} {x(n)}
The discrete version of the Fourier Series can be written as

X
j2 kn
x(n) = Xk e N
k
1 X j2 kn
= X(k) e N
N
k
where X(k) = N Xk
TU Darmstadt 18
Remarking that
(k+mN )n
j2 kn
W kn = e N = ej2 N = W (k+mN )n
for integer values of m, we obtain that the summation in the Discrete

Fourier Series (DFS) should contain only N terms:
N 1
1 X j2 kn
x(n) = X(k) e N DFS
N
k=0
The DFS coefficients are given by
N
X 1
j2 kn
X(k) = x(n) e N inverse DFS
n=0
TU Darmstadt 18
Proof:

1 N
pn
N
X 1 N
X 1
X
j2 kn 1 j2 N j2 kn
x(n) e N = X(p) e e N
N
n=0 n=0 p=0

N
X 1 1 NX
1 (pk)n
= X(p) ej2 N = X(k)
N
p=0 n=0
| {z }
(pk)
TU Darmstadt 18
The DFS pair:

N
X 1
j2 kn
X(k) = x(n) e N analysis
n=0
N
X 1
1 j2 kn
x(n) = X(k) e N synthesis
N
k=0
Remarks:
The DFS and DFT pairs are identical except for the fact that the DFT is
applied to a finite (nonperiodic) sequence x(n), whereas the DFS is
applied to a periodic sequence x(n)
The conventional (continuous-time) FS represent a periodic signal using an
infinite number of complex exponentials, whereas the DFS represent such
a signal using a finite number of complex exponentials
TU Darmstadt 18
5.5 Properties of the DFT
Linearity:
If X(k) = DF T {x(n)} and Y (k) = DFT {y(n)}
then a X(k) + b Y (k) = DF T {a x(n) + b y(n)}

where the lengths of both sequences should be equalized by means of
zero-padding.
Also if x(n) = DFT 1{X(k)} and y(n) = DFT 1{Y (k)}
then a x(n) + b y(n) = DF T 1{a X(k) + b Y (k)}
TU Darmstadt 18
Circular shift of a sequence:
If X(k) = DF T {x(n)}
j2 km
then X(k)e N = DF T {x ((n m)mod N ) }
Also if x(n) = DF T 1{X(k)} then

1 j2 km
x ((n m)mod N ) = DF T {X(k)e N }
where the operation mod N is exploited for denoting the periodic

extension x(n) of the signal x(n):
x(n) = x(n mod N )
TU Darmstadt 18
conventional shift circular shift
n
n
0 N n
TU Darmstadt 18
Proof of the circular shift property:
N
X 1 N
X 1
x((n m)mod N ) W kn = x ((n m)modN ) W k(nm+m)
n=0 n=0
NX1
= W km x((n m)modN )W k(nm
n=0
NX1
= W km x((n m)modN )
n=0
W k(nm)mod N = W kmX(k)
where we use the facts that W k(l mod N ) = W kl and that the order of
summation in DFT does not change its result
TU Darmstadt 18
Frequency shift (modulation):
If X(k) = DF T {x(n)}
j2 mn
then X((k m)mod N ) = DFT {x(n)e N }

j2 mn
x(n)e N = DF T 1{X((k m)mod N )}
Proof: similar to that for the circular shift property
TU Darmstadt 18
Parseval theorem:
N
X 1 N
X 1
1
x(n)y (n) = X(k)Y (k) general form
N
n=0 k=0
NX1 N
X 1
1
|x(n)|2 = |X(k)|2 specific form
N
n=0 k=0
Proof: Using the matrix formulation of the DFT, we obtain

H
H 1 H 1 H
y x = W Y W X
N N
1 H H 1 H
= 2
Y WW
| {z } X = Y X
N NI N
TU Darmstadt 19
Conjugation:
If X(k) = DF T {x(n)} then X ((N k)mod N ) = DF T {x(n
x(n) = DF T 1{X ((N k)mod N )}

Proof:

N
X 1 N
X 1 N
X 1
x(n)W kn = x(n)W kn = x(n) W
|
(k mod N )n
{z }
n=0 n=0 n=0 W kn

N
X 1
= x(n)W [(N k)mod N ]n= X ((N k)modN )
n=0
TU Darmstadt 19
Circular convolution:
then X(k)Y (k) = DFT {{x(n)}

{y(n)}}
Also if x(n) = DF T 1{X(k)} and y(n) = DF T 1{y(k)}
y(n) = DFT 1{X(k)Y (k)}

then x(n)
Here
stands for circular convolution defined by
N
X 1
{x(n)}
{y(n)} = x(m)y((n m)mod N )
m=0
TU Darmstadt 19
Illustration of circular convolution for N = 8:

n=0 0 n=1 0 n=2 0 n=3 0
7 1 7 1 7 1 7 1
1 0 7 2 1 0 3 2 1 4 3 2
6 2 6 2 6 3 7 2 6 4 2 6 5 1 2
0
3 4 5 4 5 6 5 6 7 6 7 0
5 3 5 3 5 3 5 3
4 4 4 4
n= 4 0 n=5 0 n=6 0 n=7 0

7 1 7 1 7 1 7 1
5 4 3 5 4
6 7 6 5 0 7 6
6 6 2 2 6 7 3 2 6 0 4 2 6 1 5 2
7 0 1 0 2 1 2 3 2 3 4
1
5 3 5 3 5 3 5 3
4 4 4 4
x(n) spread clockwise

y(n) spread counterclockwise
TU Darmstadt 19
Example: Let the circularly convolved sequences are
{x(n)} = {1, 1, 1, 1, 1, 0, 1, 2}
{y(n)} = {5, 4, 3, 2, 1, 1, 0, 1}
and {z(n)} = {x(n)}

{y(n)}. Then
z(0) = x(0)y(0) + x(1)y(7) + x(2)y(6) + x(3)y(5) + x(4)y(4)

+ x(5)y(3) + x(6)y(2) + x(7)y(1) = 1
z(1) = x(0)y(1) + x(1)y(0) + x(2)y(7) + x(3)y(6) + x(4)y(5)
+ x(5)y(4) + x(6)y(3) + x(7)y(2) = 1
z(2) = x(0)y(2) + x(1)y(1) + x(2)y(0) + x(3)y(7) + x(4)y(6)
+ x(5)y(5) + x(6)y(4) + x(7)y(3) = 6
TU Darmstadt 19
Example (continued):
z(3) = x(0)y(3) + x(1)y(2) + x(2)y(1) + x(3)y(0) + x(4)y(7)

+ x(5)y(6) + x(6)y(5) + x(7)y(4) = 4
z(4) = x(0)y(4) + x(1)y(3) + x(2)y(2) + x(3)y(1) + x(4)y(0)
+ x(5)y(7)+x(6)y(6)+x(7)y(5) = 5
z(5) = x(0)y(5)+x(1)y(4)+x(2)y(3)+x(3)y(2)+ x(4)y(1)
+ x(5)y(0)+x(6)y(7)+x(7)y(6) = -8
z(6) = x(0)y(6)+x(1)y(5)+x(2)y(4)+x(3)y(3)+ x(4)y(2)
+ x(5)y(1)+x(6)y(0)+x(7)y(7) = 4
z(7) = x(0)y(7)+x(1)y(6)+x(2)y(5)+x(3)y(4)+ x(4)y(3)
TU Darmstadt 19
+ x(5)y(2)+x(6)y(1)+x(7)y(0) = 7
Example (continued): illustration of the circular convolution process

7
0 4 6
5
12 3
x(m)
0 0 0 0
2 2 2
3 3 3
5 5 5
6 6
6 y((-m) modN)
7 4 7 4 7 4
(i.e., n= 0)
1 1 1
0 0 0
2 2 2
3 3 3
5 5 5
6 6 6
7 4 7 4 7 4
y((1-m) modN)
(i.e., n=1)
1 1 1 1
TU Darmstadt 19
Proof of circular convolution property:

N
X 1 N
X 1
DFT {{x(n)}
{y(n)}} = x(m)y((n m)mod N )W kn
n=0 m=0
| {z }
{x(n)}
{y(n)}

N
X 1 N
X 1
= y((n m)mod N )W knx(m)
m=0 n=0
| {z }
Y (k)W km
N
X 1
= Y (k) x(m)W km = X(k)Y (k)
|m=0 {z }
X(k)
TU Darmstadt 19
Multiplication:
1
then X(k)
Y (k) = DF T {x(n)y(n)}
N
Also if x(n) = DF T 1{X(k)} and y(n) = DF T 1{y(k)}
1 1
then x(n)y(n) = DF T { X(k)
Y (k)}
N
Proof: similar to that for the circular convolution property
TU Darmstadt 19
5.6 The Fast Fourier Transform
Let us now introduce the following new notation

j 2
XN (k) = X(k) , k = 0, . . . , N 1 , WN = e N
Before, we used simpler notation X(k) and W (without the subscript N )

because the sequence length was not important. Now, it is important!
Assume that
N = 2r , r = log2N
i.e., that the transformed sequence length is an integer power of 2.
If the original sequence length does not satisfy this assumption, it can be
always padded with a proper number of zeros so that the length of
zero-padded sequence will satisfy this assumption!
TU Darmstadt 19
The idea of FFT:

N
X 1
XN (k) = kn
x(n)WN Npoint DFT
n=0
X X
= x(n)WN kn + x(n)WN kn
n even n odd
N/21 N/21
X X (2l+1)k
= 2lk
x(2l)WN + x(2l + 1)WN
l=0 l=0
N/21 N/21
X X
= 2 lk
x(2l)( WN ) + WN k x(2l + 1)( WN2 )lk
|{z} |{z}
l=0 WN/2 l=0 WN/2
N/21 N/21
X X
= lk k
x(2l)WN/2 + WN lk two N/2po
x(2l + 1)WN/2
l=0 l=0
TU Darmstadt
Elektrotechnik und Informationstechnik DFT0s!!! 20
In a short notation:
kH
XN (k) = GN/2(k) + WN N/2(k)
where GN/2(k) and HN/2(k) are the N/2-point DFTs involving x(n)
with even and odd n, respectively
x(0) GN/2 (0) XN (0)
x(2) GN/2 (1) WN0 XN (1)
x(4) N point DFT 1
GN/2 (2) WN XN (2)
2
x(6) GN/2 (3) WN2 XN (3)
N=8 3
WN
x(1) HN/2 (0) XN (4)
x(3) HN/2 (1) WN4 XN (5)
x(5) N point DFT WN5
HN/2 (2) XN (6)
2 WN6
x(7) HN/2 (3) XN (7)
WN7
TU Darmstadt 20
Corollary: any N -point DFT with even N can be computed via two
N/2-point DFTs. In turn, if N/2 is even then each of these N/2-point
DFTs can be computed via two N/4-point DFTs and so on. In the case
N = 2r , all N, N/2, N/4 . . . are even and such a process of splitting
ends up with 2-point DFTs!
2-point DFT
4-point DFT
FFT (Cooley and Tukey)
2-point DFT
8-point DFT
2-point DFT
4-point DFT
2-point DFT
16-point DFT
2-point DFT
4-point DFT
2-point DFT
8-point DFT
2-point DFT
4-point DFT
2-point DFT
TU Darmstadt 20
Flow graph of 8-point FFT

x(0)
X(0)
WN0 WN0 WN0
x(4)
WN4 X(1)
WN2 WN1
x(2) X(2)
WN4
WN0 WN2
x(6)
WN4 WN6
X(3)
WN3
x(1)
WN4 X(4)
WN0 WN0
x(5)
WN4 WN5 X(5)
WN2
x(3)
WN4 WN6
X(6)
WN0
x(7) X(7)
WN4 WN6 WN7
TU Darmstadt 20
Flow graph of basic butterfly computation:

l
WN
WN(l+N/2)
Computational complexity: each stage has N complex multiplications and
N complex additions and there are log2N stages = the total
complexity (additions and multiplications) is
CFFT = N log2N
Compare it with the complexity of the straightforward DFT
CDFT = N 2
TU Darmstadt 20
For large N (i.e., N 1),
log2N N = CFFT CDFT
Example: For N = 210 = 1024, CDFT = 220 ' 106, whereas

CDFT = 10 1024 ' 104, i.e., the reduction is about 2 orders of
magnitude!
TU Darmstadt 20
6 DIGITAL FILTERS
6.1 What is Filtering
Definition: Digital filtering is just changing the frequency-domain
characteristics of a given discrete-time signal
Filtering operations may include:

noise suppression
enhancement of selected frequency ranges or edges in images
bandwidth limiting (to prevent aliasing of digital signals or to reduce
interference of neighboring channels in wireless communications)
removal or attenuation of specific frequencies
special operations like integration, differentiation, etc.
TU Darmstadt 20
Recall causal LTI-systems:

X
y(n) = h(k)x(n k) = {x(n)} {h(n)}
k=0
Y () = H()X()
x(n) y(n)
H()
The impulse response values {h(0), h(1), h(2), . . .} can be interpreted as

filter coefficients
TU Darmstadt 20
6.2 Finite and Infinite Impulse Responses

Definition: If {h(n)} is an infinite duration sequence, the corresponding
filter is called an infinite impulse response (IIR) filter. In turn, if {h(n)} is
a finite duration sequence, the corresponding filter is called a Finite
Impulse Response (FIR) filter.
h(n) h(n)
FIR IIR
n n
TU Darmstadt 20
A very general form of digital filter can be obtained from the familiar
equation (recall LTI-systems)
N
X M
X
a(k)y(n k) = b(k)x(n k) ARMA
k=0 k=0
where {x(n)} is the filter input signal and {y(n)} is the filter output
signal. As obtained before, the transfer function corresponding to this
equation is the following rational function
PM k
k=0 b(k)z B(z)
H(z) = PN =
k A(z)
k=0 a(k)z
If N = 0, the filter becomes a FIR (nonrecursive) one
If N > 0, the filter becomes a IIR (recursive) one
TU Darmstadt 20
6.3 Filter Specifications

Basic filter types:
lowpass filters (to pass low frequencies from zero to a certain cut-off
frequency C and to block higher frequencies)
highpass filters (to pass high frequencies from a certain cut-off frequency
C to and to block lower frequencies)
bandpass filters (to pass a certain frequency range [min, max], which
does not include zero, and to block other frequencies)
bandstop filters (to block a certain frequency range [min, max], which
does not include zero, and to pass other frequencies)
TU Darmstadt 21
ideal ideal
H() lowpass H() highpass
0 C 0 C
ideal ideal
H() bandpass H() bandstop
0 min max 0 min max

TU Darmstadt 21
Frequency responses of practical filters are not shaped in straight lines,

i.e., they vary continuously as a function of frequency: they are neither
exactly 1 in the passbands, nor exactly 0 in the stopbands
Lowpass filter specifications:
1 2 |H()| 1 + 1 , [0, P ] ; 0 |H()| 3 , [S

H()
transition
1+ 1 region
1 2
passband
region
stopband
region
3
P S
TU Darmstadt 21
Definition: The quantity max{1, 2} is called passband ripple, and the

quantity 3 is called stopband attenuation
These filter parameters are usually specified in dB:
AP = max{20log10(1 + 1), 20log10(1 2)} PB ripple in d

AS = 20log103 SB attenuation in d
Example: let 1 = 2 = 3 = 0.1 =
AP = max{0.828, 0.915} dB = 0.915 dB

AS = 20 dB
TU Darmstadt 21
BANDSTOP FILTER BANDPASS FILTER
H() H() transition

region
transition
region
transition
1+ 1 regions
1 2
passband passband
region 1 passband region
region 2
stopband stopband
stopband region 1 region 2
region
3
P1 S1 S 2 P2 S1 P P S
1 2 2
TU Darmstadt 21
6.4 The Phase Response and Distortionless Transmission

In most filter applications, the magnitude response |H()| is of primary
concern. However, the phase response may be also important.
H() = |H()|ej()
where
= angle{H()} phase response
Definition: If a signal is transmitted through a system (filter) then this
system is said to provide a distortionless transmission if the signal form
remains unaffected, i.e., if the output signal is a delayed and scaled replica
of the input signal.
TU Darmstadt 21
Two conditions of distortionless transmission:

the system must amplify (or attenuate) each frequency component
uniformly, i.e., the magnitude response must be uniform within the signal
frequency band.
the system must delay each frequency component by the same value.
2 1.5
1.5
1
1
0.5
0.5
0
0.5
0.5
1
1
FIRST SIGNAL 1.5 FIRST SIGNAL

1.5 SECOND SIGNAL SECOND SIGNAL
SUM SIGNAL SUM SIGNAL
2 2
0 100 200 300 400 500 600 700 800 900 1000 0 100 200 300 400 500 600 700 800 900 1000
TU Darmstadt 21
Definition: The group delay of a filter is the relative delay imposed on a

particular frequency component of an input signal.
We can represent an arbitrary phase response in the form
() = ()
where () is the group delay measured in samples.
Result: To satisfy the distortionless response phase condition, the group

delay must be frequency-independent, i.e., uniform for each frequency =
() = = const
A filter having this property is called a linear-phase filter because phase

varies linearly with the frequency
TU Darmstadt 21
Hence in general, a linear-phase frequency response is given by
H() = |H()| ej
Example: Ideal lowpass with linear phase

(
ej , || < C
H() =
0, C < || <
The corresponding impulse response
sin {C (n )}
h(n) =
(n )
Taking an integer delay = m, we have
sin {C (2m n m)} sin {C (m n)}
h(2m n) = = = h(n)
(2m n m) (m n)
TU Darmstadt 21
Hence, the response is symmetric about n = m = in that case we

can define a zero-phase system
sin(C n)
H() = H() ejm = |H()| , h(n) =
n
Definition: A system is referred to as a generalized linear-phase system if
its frequency response can be expressed in the form
H() = A() ej( ) ()
with a real function A() and = const, = const.
TU Darmstadt 21
Example: consider a discrete-time implementation of the ideal

continuous-time differentiator
d
yc(t) = xc(t) = H() = j
dt
Restrict the input to be bandlimited
(
jc , |c| < /t
Hc(c) =
0 , |c| /t
The corresponding discrete-time system has frequency response

j
H() = , || <
t
Hence, H() has the form () with = 0, = /2, and
A() = /t = This system is a generalized linear-phase filter!
TU Darmstadt 22
6.5 FIR Filters

The transfer function:
H() = h(0) + h(1)ej + + h(N 1)ej(N 1)

N
X 1
= h(n)ejn FIR
n=0
Advantages of FIR Filters:
FIR filters are stable
they can be designed to have linear phase or generalized linear phase
there is great possibility in shaping their magnitude response
they are convenient to implement
TU Darmstadt 22
Impulse response truncation method:
Step 1: Chose the desired amplitude response according to filter class

(lowpass, highpass, etc.)
Step 2: Chose the filters phase characteristics: integer or fractional
group delay, initial phase
Step 3: Write the ideal frequency response as Hid() =
Aid() exp{j(id id)} and compute the ideal impulse response
hid(n) using the inverse DTFT
Step 4: Truncate the impulse response by taking
(
hid(n) , 0 n N 1
h(n) =
0, n>N 1
TU Darmstadt 22
Example: Let the lowpass ideal frequency response be

(
ej0.5N , 0 /4
H() =
0, otherwise
or, in terms of f ,
(
ejf N , 0 f 0.125
H(f ) =
0, otherwise
Then, the ideal impulse response is

sin((n 0.5N )/4)
hid(n) =
(n 0.5N )
Consider different filter orders N = 11, N = 41, and N = 161
TU Darmstadt 22
Truncated impulse response (N = 11) and corresponding approximation

of lowpass frequency response
1.4
0.25
1.2
0.2
ABS. VALUE OF FREQUENCY RESPONSE

0.15 1
TRUNCATED IMPULSE RESPONSE
0.1
0.05 0.8
0.6
0.05
0.1
0.4
0.15
0.2
0.2
0.25
0
0 1 2 3 4 5 6 7 8 9 10 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
TIME INDEX FREQUENCY
TU Darmstadt 22

1.4
0.25
1.2
0.2

0.15 1
0.1
0.05 0.8
0.6
0.05
0.1
0.4
0.15
0.2
0.2
0.25
0
0 5 10 15 20 25 30 35 40 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
TU Darmstadt 22

1.4
0.25
1.2
0.2

0.15 1
0.1
0.05 0.8
0
0.6
0.05
0.1
0.4
0.15
0.2
0.2
0.25
0
0 20 40 60 80 100 120 140 160 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
TU Darmstadt 22
Gibbs phenomenon: oscillations at the edges of the pass- and stopband

cannot be reduced by the increase of N ! For any N , these oscillations
correspond to about 0.09 passband ripple and stopband attenuation
parameters! = the impulse response truncation method is easy but
not a very good way of filter design
The Gibbs phenomenon

1.09
-0.09
TU Darmstadt 22
FIR filter design using windows:
Step 1: Define the ideal frequency response as in the impulse response

truncation method
Step 2: Obtain the impulse response hid(n) of this ideal filter as in the
impulse response truncation method
Step 3: Compute the coefficients of the filter by
(
w(n)hid(n) , 0 n N 1
h(n) =
0, n>N 1
where w(n) is some chosen window function
TU Darmstadt 22
Window properties:
windows are always real and symmetric, i.e., they must satisfy
w(n) = w(N n 1), n = 0, . . . , N 1
for either even or odd N . If the ideal FIR filter has linear phase, symmetric
windows do not affect this property.
windows must be positive to avoid any changes of sequence polarities
a good window must satisfy the tradeoff between the width of the
mainlobe and the magnitudes of the sidelobes in the frequency domain
a good window must have a smooth transition at the edges of sequence
in order to reduce the truncation effect
the window length must correspond to the sequence length
TU Darmstadt 22
Classical window types (have been derived based on intuition and educated
guesses):
Rectangular window (corresponds to the natural truncation like in the
impulse response truncation method!)
(
1, 0 n N 1
w(n) =
0, otherwise
Triangular (or Bartlett) window

2n/(N 1) , 0 n (N 1)/2
w(n) = 2 2n/(N 1) , (N 1)/2 < n (N 1)

0, otherwise
TU Darmstadt 23
Bartlett window compared to rectangular window in frequency domain

(N = 41)
1
0
0.9
0.8
0.7
50
BARTLETT WINDOW
0.6
W(f) (DB)
0.5
0.4
100
0.3
0.2
0.1
0 150
0 5 10 15 20 25 30 35 40 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5
SAMPLE INDEX FREQUENCY
n o
W (f )[dB] = 10 log10 |W (f )|2 = 20 log10 {|W (f )|}
TU Darmstadt 23
Hanning (or Hann) window

(
0.5 (1 cos[2n/(N 1)]) , 0 n N 1
w(n) =
0, otherwise
Hamming window
(
0.54 0.46 cos[2n/(N 1)] , 0 n N 1
w(n) =
0, otherwise
Blackman window
( h i h i
0.42 0.5 cos N2n
1 + 0.08 4n , 0 n N 1
N 1
w(n) =
0, otherwise
TU Darmstadt 23
Hanning window compared to rectangular window in frequency domain

(N = 41)
1
0
0.9
0.8
0.7
50
HANNING WINDOW
0.6
W(f) (DB)
0.5
0.4
100
0.3
0.2
0.1
0 150
0 5 10 15 20 25 30 35 40 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5
TIME SAMPLE INDEX FREQUENCY
TU Darmstadt 23
Hamming window compared to rectangular window in frequency domain

(N = 41)
1
0
0.9
0.8
0.7
50
HAMMING WINDOW
0.6
W(f) (DB)
0.5
0.4
100
0.3
0.2
0.1
0 150
0 5 10 15 20 25 30 35 40 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5
TU Darmstadt 23
Blackman window compared to rectangular window in frequency domain

(N = 41)
1
0
0.9
0.8
0.7
50
BLACKMAN WINDOW
0.6
W(f) (DB)
0.5
0.4
100
0.3
0.2
0.1
0 150
0 5 10 15 20 25 30 35 40 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5
TU Darmstadt 23
Modern window types (have been derived based on optimality criteria):

Kaiser window: minimize the width of the mainlobe under the constraint
that the window length be fixed and the energy in the sidelobes not exceed
a given percentage of the total energy.
Dolph-Chebyshev window: minimize the width of the mainlobe under the
constraint that the window length be fixed and the sidelobe level not
exceed a given value.
Kaiser and Dolph-Chebyshev windows provide more flexibility than the
classical windows because the required tradeoff between the mainlobe and
sidelobes can be achieved!
TU Darmstadt 23
Dolph-Chebyshev window with 30 dBs of ripple compared to the

rectangular window in frequency domain (N = 41)
1
0
0.9
0.8
DOLPHCHEBYSHOV WINDOW
0.7
50
0.6
W(f) (DB)
0.5
0.4
100
0.3
0.2
0.1
0 150
0 5 10 15 20 25 30 35 40 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5
TU Darmstadt 23
Dolph-Chebyshev window with 100 dBs of ripple compared to the

rectangular window in frequency domain (N = 41)
1
0
0.9
0.8
DOLPHCHEBYSHEV WINDOW
0.7
50
0.6
W(f) (DB)
0.5
0.4
100
0.3
0.2
0.1
0 150
0 5 10 15 20 25 30 35 40 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5
TU Darmstadt 23
Now, let us take Dolph-Chebyshev window and design the lowpass filter
using the window method with N = 11, N = 41, and N = 161.
Remember that
sin((n 0.5N )/4)
hid(n) =
(n 0.5N )
We take the window length equal to the length of hid(n).
TU Darmstadt 23
Truncated and windowed impulse response (N = 11) and corresponding

approximation of lowpass frequency response (window ripple 30 dB)
1.4
0.25
1.2
TRUNCATED AND WINDOWED INPULSE RESPONSE
0.2

0.15 1
0.1
0.05 0.8
0.6
0.05
0.1
0.4
0.15
0.2
0.2
0.25
0
0 1 2 3 4 5 6 7 8 9 10 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
TU Darmstadt 24

1.4
0.25
1.2
0.2

0.15 1
0.1
0.05 0.8
0.6
0.05
0.1
0.4
0.15
0.2
0.2
0.25
0
0 5 10 15 20 25 30 35 40 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
TU Darmstadt 24

1.4
0.25
1.2
0.2

0.15 1
0.1
0.05 0.8
0
0.6
0.05
0.1
0.4
0.15
0.2
0.2
0.25
0
0 20 40 60 80 100 120 140 160 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
There is no Gibbs phenomenon!
TU Darmstadt 24
Optimization-based methods: the idea is to find the best approximation to

the ideal frequency response for a given fixed N , i.e. to find
(
hd(n) , 0 n N 1
h(n) =
0, n>N 1
which minimizes the squared error

Z
2 1
= |Hd() Hid()|2 d
2
Another powerful criterion is the so called minimax criterion: optimize the
impulse response by minimizing the maximal error between the ideal and
designed frequency responses.
TU Darmstadt 24
6.6 IIR Filters

We start with analog IIR filters and then find ways to transform a given
analog IIR filter to a similar digital filter.
A lowpass Butterworth filter is defined as:
1
|H()|2 = 2N
()
1 + (/0)
where N is the integer which will soon be seen to be the filter order
H() 2
1
N= 3
0
TU Darmstadt 24
To obtain the transfer function of the Butterworth filter, define s = j

and substitute it to ()
1 1
H(s)H(s) = 2N
=
1 + (s/j0) 1 + (1)N (s/0)2N
The 2N poles of this function:

(N + 1 + 2k)
sk = 0 exp j , 0 k 2N 1
2N
TU Darmstadt 24
2
POLES OF H(s)
Im (s) Im (s)
N=3 N=4
0
Re (s) Re (s)
POLES OF H(s)
N=3 Im (s) N=4 Im (s)
Re (s) Re (s)
TU Darmstadt 24
The Butterworth transfer function can be expressed as:

Y s
k
H(s) = , s = j
s sk

(2k + 1)
sk = 0 exp j + , 0k N 1
2 2N
It is possible to determine 0, N , and H(s) which correspond to |H()|2
satisfying required specifications.
Example: N = 1, then s0 = 0 ej . Therefore

1 2 1
H(s) = = |H(s)| =
1 + s/0 1 + (/0)2
TU Darmstadt 24
A lowpass Chebyshev filter is defined as:

1
|H()|2 = 2 (/ )
1 + 2TN 0
where Chebyshev polynomial of degree N
(
cos (N arccos x) , |x| 1
TN (x) =
cosh (N arccosh x) , |x| > 1
H() 2 N=2 H() 2 N=3
p s p s
TU Darmstadt 24
Summary of analog filter design
the s-plane poles and the transfer function of Chebyshev filter can be
obtained in the way similar to that for Butterworth filter
other analog filter structures are known, such as Bessel and elliptic filters
an important question is what kind of transformation can be used to
obtain digital IIR filters from the corresponding analog filters
TU Darmstadt 24
We wish to find some transformations of H(s) to H(z), where H(z) is a

digital filter transfer function
Consider bilinear transformation which represents an algebraic unique

mapping between the points in the s-plane and those in the z-plane. This
transformation corresponds to replacing s by
2 z1 2 1 z 1
=
t z + 1 t 1 + z 1
TU Darmstadt 25
Substitute s = ja (with analog frequency a) and z = exp{jd} (with

digital frequency d) in the introduced bilinear transformation:
2 exp{jd} 1
ja =
t exp{jd} + 1
2 exp{jd/2} (exp{jd/2} exp{jd/2})
=
t exp{jd/2} (exp{jd/2} + exp{jd/2})
2j
= tan{d/2}
t
Hence, the bilinear transformation gives us the following mapping between
analog and digital frequencies:

2 at
a = tan{d/2} d = 2 arctan
t 2
TU Darmstadt 25
= 2arctan(a t 2)
d

TU Darmstadt 25
Bilinear transformation warps the digital frequency with respect to analog

frequency. The nonlinear warping function is 2 arctan{0.5 a t}.
To design lowpass filter with the digital transition-region frequencies P

and S , we find the analog prewarped frequencies
2 2
P = tan {P /2} , S = tan {S /2} ()
t t
and design the analog filter using them in chosen specification. Then, the
analog filter can be transformed to the digital one as:

H(z) = H(s) ()
s=(2[z1])/(t[z+1])
TU Darmstadt 25
IIR filter design using bilinear transformation:
Step 1: Convert each specified edge-band (transition region) frequency

of the desired digital filter to a corresponding edge-band frequency of an
analog filter using ()
Step 2: Design an analog filter H(s) of the desired type, according to
the transformed specifications
Step 3: Transform analog filter H(s) to a digital filter H(z) using
()
TU Darmstadt 25
Another method of IIR filter design does not require analog filter design
and exploits the ARMA model
>From LCCD equations, we know that the ARMA model reads

N
X M
X
a(k)y(n k) = b(k)x(n k)
k=0 k=0
This model corresponds to the IIR filter with the frequency response
PM k
k=0 b(k)z
H(z) = PN
a(k)z k
k=0
TU Darmstadt 25
Minimizing
Z PM jk 2
2 k=0 b(k)e
|| = Hid() PN d
jk
k=0 a(k)e
we obtain desired filter parameters a(k), k = 1, . . . , N and b(k),
k = 1, . . . , M
Efficient numerical optimization methods can be formulated to minimize
this function
TU Darmstadt 25
6.7 Realizations of digital filters
unit delay
x(n) y(n)=x(n- 1)
t
BASIC
adder/summator BUILDING
x1(n) BLOCKS
x2(n) y(n) = x1(n)+x2(n)+x3(n)
+
x3(n)
multiplier by a constant
x(n) a y(n) = a x(n)
TU Darmstadt 25
Example 1: First-order FIR filter
H(z) = b(0) + b(1)z 1 , y(n) = b(0)x(n) + b(1)x(n 1)
x(n) b(0) y(n)

+
t
b(1)
TU Darmstadt 25
Example 1: First-order IIR filter

1
H(z) = 1
, y(n) = a(1)y(n 1) + x(n)
1 + a(1)z
x(n) y(n)
+
t
- a(1)
TU Darmstadt 25
The general FIR filter case:

M
X M
X
y(n) = b(k)x(n k) , H(z) = b(k)z k
k=0 k=0
x(n) b(0) y(n)

+
t
b(1)
t
b(M- 1)
t
b(M)
TU Darmstadt 26
The IIR filter case (N = M ):

M M PM
X X b(k)z k
k=0
y(n) = b(k)x(n k) a(k)y(n k), H(z) = PM
a(k)z k
k=0 k=1 k=0
x(n) b(0) y(n)

+
t
b(1) -a(1)
+
t
b(M) + -a(M )
TU Darmstadt 26
7 RANDOM SIGNAL ANALYSIS

7.1 Random Signals and Their Statistical Description
Let X be a random variable with the probability distribution
PX (x) = Probability{X x}
Probability density function (pdf):

PX (x)
pX (x) =
x
where
Z x0
PX (x0) = pX (x) dx

TU Darmstadt 26
Since PX () = 1, we obtain
Z
pX (x) dx = 1

Simple interpretation:
Probability{x /2 X x + /2}
pX (x) = lim0

p (x) SURFACE =Probability{x
X 1< X< x 2}
x1 x2 x
TU Darmstadt 26
Mathematical expectation of an arbitrary function f (X):

Z
E{f (X)} = f (x) pX (x) dx

Mean: Z
mX = E{X} = x pX (x) dx

Variance:
2 = E{(X E{X})2} = E{X 2} E{X}2
var{X} = X
Sample mean (recall MA systems!):

N
X 1
1
mX = x(n)
N
n=0
TU Darmstadt 26
Sample variance:
N
X 1
2 1
X = (x(n) mX )2
N 1
n=0
The weighting 1/N 1 is used instead of 1/N to make the sample
variance unbiased:
2 } = 2
E{X X
For the complex random variable:
2 = E{(X E{X})(X E{X})}
var{X} = X
= E{|X|2} |E{X}|2
TU Darmstadt 26
Random process (vector) X = (X(0), . . . , X(N 1))T
PX(x) = Probability{X(0) x(0) and X(1) x(1)

and and X(N 1) x(N 1)}
Probability density function (pdf):

N PX(x)
pX(x) =
x(0) x(N 1)
In the case of two random variables X = (x, y)T
pX(x) = pXY (x, y)
Variables are statistically independent when
pX,Y (x, y) = pX (x) pY (y) = E{xy} = E{x} E{y}

TU Darmstadt 26
Expectation of a function f (X):

Z Z
E{f (X)} = f (x) pX(x) dx

In the case of two random variables X = (x, y)T

Z Z
E{f (X, Y )} = f (x, y) pX,Y (x, y) dx dy

Correlation
Z Z
rX,Y = E{XY } = x y pX,Y (x, y) dx dy

In the complex case
Z Z
rX,Y = E{XY } = x y pX,Y (x, y) dx dy

TU Darmstadt 26
Covariance matrix in the real case:
R = E{XXT }
Covariance matrix in the complex case:
R = E{XXH }
Structure of the covariance matrix:

r(0) r(1) r(N 1)

r(1) r(0) r(N 2)
R=

r(N 1) r(N 2) r(0)
where r(i) = E{x(k)x(k + i)} and the process is assumed to be in
wide sense stationary
TU Darmstadt 26
Result: Covariance matrix is Hermitian and nonnegative definite

To prove Hermitian property, we must show that RH = R:
H
RH = E{XXH } = E{(XXH )H } = E{XXH } = R
To prove nonnegative definiteness property we must show that

dH Rd 0 N 1 vector d
dH Rd = dH E{XXH }d = E{|dH X|2} 0
Corollary: Covariance matrix eigenvalues are real and nonnegative
TU Darmstadt 26
7.2 Adaptive Filtering

x(n) x(n-1) x(n-2) x(n-N+1)
t t t t
w1 w2 w3 wN
y(n)

The weight vector
W = (w1, w2, . . . , wN )T
is tuned according to some adaptation criterion
TU Darmstadt 27
Observations vector
Xk = (x(k), x(k 1), . . . , x(k N + 1))T

= s(k)
| {zaS} + n(k)
|{z}
desired signal noise and interference
Complex filter output
y(k) = WH Xk
The key idea of adaptive filtering is to maximize the desired signal and
reject (as much as possible) everything else (noise, interference).
Assume that we know the frequency S of the desired signal and let us
maximize the Signal to Noise Ratio (SNR)
|WH aS |2 |WH aS |2
SNR = H 2
= H
,
E{|W n(n)| } W RnW
TU Darmstadt 27
Noise (and interference) covariance matrix:
Rn = E{n(n)nH (n)}
where E{} is statistical expectation

Known desired signal vector
T
aS = 1, ejS , . . . , ej(N 1)S
Characteristics of the interfering signals and noise are assumed unknown!

SNR maximization is equivalent to
min WH RnW subject to WH aS = const

W
TU Darmstadt 27
The (optimal) solution is
Wopt = R1
n aS Wiener filter
( does not affect SNR and, therefore, = 1 can be taken)

Optimal (maximally achievable) SNR can be obtained by inserting the
Wiener solution into the SNR expression:
H a |2
|Wopt S (aH R1a )2
S n S
SNRopt = H R W
= 1a
= aH R1a
S n S
Wopt n opt aH R
S n S
In practice, the matrix R1

n is unavailable (the only available are the
vectors Xk )
TU Darmstadt 27
Adaptive algorithm:
Wk+1 = Wk + (aS (XH

k Wk )Xk )
Proof of convergence: Subtract Wopt from the both sides of the

algorithm:
H
Wk+1 Wopt = Wk Wopt +aS Xk Xk Wk Wopt +Wopt
Take expectation and remark that
Vk = E{Wk Wopt}, Rn = E{Xk XH

k }
where we assume that the observation Xk does not contain the desired
signal (this assumption can be relaxed later)
TU Darmstadt 27
Then, the last expression can be rewritten as
Vk+1 = [I Rn] Vk + (aS RnWopt)

| {z }
= aS RnR1
n aS = 0
where we assume that the random vectors Xk and Wk are statistically

independent (so that E{Xk XH k Wk } = R E{Wk }).
Hence, we obtain the following iterative equation:
Vk+1 = [I Rn] Vk
Sufficient condition of convergence:
||Vk+1|| < ||Vk || , k
TU Darmstadt 27
Equivalently, the kth power of the matrix I Rn must become zero as

k becomes large. This is equivalent to the condition
2
0<<
max
where max is the maximal eigenvalue of Rn.
In practice, a proper (stronger) condition is

2
0<<
trace{Rn}
TU Darmstadt 27
Remark: Sometimes, it is easier to represent the observations as
Xk = (x(k), x(k + 1), . . . , x(k + N 1))T
rather than in time-reversed order
Xk = (x(k), x(k 1), . . . , x(k N + 1))T
It is straightforward to reformulate the algorithm for this case. The

definitions of Wk and aS must be correspondingly modified.
Example: Let
x(n) = exp(j2f
| {z S n)} + 100
| exp(j2f
{z I n)
} + (n)
|{z}
desired signal interference noise
Let us plot the so-called learning curve (SNR vs. k) for N = 8,

fS = 0.1, fS = 0.3, and different values of
TU Darmstadt 27
Comparison of convergence for 1 = 1/(50 trace{Rn}),

2 = 1/(15 trace{Rn}), and 3 = 1/(5 trace{Rn})
20
15
10 OPTIMAL SNR
5 MU3
MU2
0
SNR (DB)
MU1
5
10
15
20
25
30
0 1000 2000 3000 4000 5000 6000 7000
NUMBER OF ITERATION
TU Darmstadt 27
Frequency response in steady-state

0
10
ABSOLUTE VALUE OF FREQUENCY RESPONSE

20
30
40
50
60
70
80
0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5
FREQUENCY
TU Darmstadt 27
7.3 Elements of Spectral Analysis

Definition of the power spectral density for a finite power random signal:

1 NX 1 2
jn
P () = lim E x(n)e
N N
n=0
P() average power

between 1 and 2
1 2
TU Darmstadt 28
Periodogram:
1 N
X 1 2

Pp() = x(n)ejn
N
n=0
periodogram is very similar to squared and normed DFT

can be computed using FFT with zero-padding
its variance does not reduce with N !
Example:
x(n) = A exp(j2fS n) + (n)
where fS = 0.3 and is zero-mean unit-variance complex noise.
TU Darmstadt 28
Periodogram (A = 1, N = 100)
150
100
PERIODOGRAM
50
0
0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5
FREQUENCY
TU Darmstadt 28
1000
900
800
700
600
PERIODOGRAM
500
400
300
200
100
0
0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5
FREQUENCY
TU Darmstadt 28
250
200
150
PERIODOGRAM
100
50
0
0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5
FREQUENCY
TU Darmstadt 28
Periodogram (A = 0.1, N = 100)

10
6
PERIODOGRAM
0
0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5
FREQUENCY
TU Darmstadt 28

14
12
10
PERIODOGRAM
0
0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5
FREQUENCY
TU Darmstadt 28

4
3.5
2.5
PERIODOGRAM
1.5
0.5
0
0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5
FREQUENCY
TU Darmstadt 28
Refined periodogram (Bartlett method)

x(n)
x 1(n) x 2(n) x L(n)

Based on dividing the original sequence into L = N/M nonoverlapping
subsequences of length M , computing periodogram for each subsequence,
and averaging the result:
L
X
1
PB () = Pl () ,
L
l=1
1 M
X 1 2

Pl () = xl (n)ejn
M
n=0
TU Darmstadt 28
Averaged periodogram (A = 0.1, N = 10000, M = 1000)

12
10
AVERAGED PERIODOGRAM
0
0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5
FREQUENCY
TU Darmstadt 28
Further refinements of periodogram (Welch method)

x(n)
x 1(n) x 2(n) x L(n)

Welch method refines the Bartlett periodogram by:
using overlapping subsequences
windowing of each subsequence
Remark: There are many other advanced spectral analysis methods which
perform significantly better than the periodogram methods
TU Darmstadt 29
Welch periodogram (A = 0.1, N = 10000, M = 1000) with 2/3

overlap and Hamming window
4
3.5
3
WELCH PERIODOGRAM
2.5
1.5
0.5
0
0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5
FREQUENCY
TU Darmstadt 29
Another alternative way to define the power spectral density:

X
P () = r(k) ejk
k=
Z
1
r(k) = P ()ejk d
2
where the correlation function
r(k) = E{x(n)x(n k)} = E{x(n)x(n + k)}
Useful property:
r(k) = r(k)
The two definitions of the power sprectral density are identical if r(k)
decays sufficiently fast.
TU Darmstadt 29
Correlogram:
N
X 1
Pc()= r(k)ejk
k=N +1
where we can use either unbiased or biased estimates of r(k).
Unbiased estimate:
(
1 PN 1 x(i)x(i k) , k 0
r(k)= N k i=k
r(k) , k<0
Proof of the fact that it is unbiased:

N
X 1 N
X 1
1 1
E{r(k)} = E{x(i)x(i k)} = r(k) = r(k)
N k N k
i=k i=k
TU Darmstadt 29
Example: Let the observation contains four points

{x(0), x(1), x(2), x(3)}.
Unbiased estimate:
1
r(0) = [x(0)x(0) + x(1)x(1) + x(2)x(2) + x(3)x(3)]
4
1
r(1) = [x(1)x(0) + x(2)x(1) + x(3)x(2)]
3
1
r(2) = [x(2)x(0) + x(3)x(1)]
2
r(3) = x(3)x(0)
TU Darmstadt 29
Biased estimate:
(
1 PN 1 x(i)x(i k) , k 0
r(k)= N i=k

r (k) , k<0
Prof of bias:
N
X 1 N
X 1
1 1 N k
E{r(k)} = E{x(i) x(i k)} = r(k) = r(k)
N N N
i=k i=k
Biased estimate is asymptotically unbiased:

N k
lim r(k) = r(k)
N N
TU Darmstadt 29
Example (contd): four-point sequence.
Biased estimate:
1
r(0) = [x(0)x(0) + x(1)x(1) + x(2)x(2) + x(3)x(3)]
4
1
r(1) = [x(1)x(0) + x(2)x(1) + x(3)x(2)]
4
1
r(2) = [x(2)x(0) + x(3)x(1)]
4
1
r(3) = x(3)x(0)
4
Hence, least reliable estimates are weighted by lowest weights!

TU Darmstadt 29
Advantage of biased estimate: it is more reliable than the unbiased one

because it assigns lower weights to the poorer estimates of long correlation
lags.
Is there any relationship between the periodogram and correlogram?
TU Darmstadt 29
Result: Correlogram computed through the biased estimate of r(k)

coincides with periodogram.
Proof: Consider the auxiliary signal

N 1
1 X
y(m) = x(k)(m k)
N k=0
where {x(k)} are fixed constants and {(k)} is a unit-variance white

noise:
r(m l) = E{(m)(l)} = (m l)
TU Darmstadt 29
The signal y(m) can be viewed as the output of the filter with the transfer
function
N 1
1 X
X() = x(k)ejk
N k=0
Relationship between filter input and output PSDs:

X
Py () = |X()|2P()= |X()|2 r(k)ejk
k=

X
= |X()|2 (k)ejk = |X()|2
k=
2
NX1
1 jn

=Pp()
= x(n)e
N

n=0
TU Darmstadt 29
Now, it remains to prove that Py () = Pc(). Remark that
ry (k) = E{y(m)y(m k)}

1 X
N 1 NX1
= E x (p)(m p) x(s)(m k s)
N
p=0 s=0
N
X 1 N
X 1
1
= x(p)x(s) E{(m p)(m k s)}
N
p=0 s=0
N
X 1 N
X 1
1
= x(p)x(s) (p k s)
N
p=0 s=0
N 1
(
1 X rx(k) , 0 k N 1
= x(p)x(p k) = biase
N 0, kN
p=k
TU Darmstadt 30
Inserting the last result in the first definition of PSD, we obtain

X
Py () = ry (k)ejk
k=
N
X 1
= rx(k)ejk = Pc()
k=N +1
Summarizing, we have proven the following two equalities for the spectrum
of the auxiliary signal y(n):
Py () = Pp() , Py () = Pc() =
Pp() = Pc()
TU Darmstadt 30
Refined correlogram (Blackman-Tukey method):
r(k) is a poor estimate for higher lags k. Hence, let us truncate it (use
M N points)
Let us use some lag window
M
X 1
PBT () = w(k)r(k)ejk
k=M +1
TU Darmstadt 30
7.4 Signal and Image Compression

Definition: Compression is the operation of representing the N -bit
information using Nc bits.
N
compression ratio =
Nc
Principle of compression by orthonormal transforms:
compressed orthonormal
signal basis original
signal
=
TU Darmstadt 30
Original signal:
x = (x(0), x(1), . . . , x(M 1))T
Compression:
X = (X(0), X(1), . . . X(L 1))T = TH x
where
M N
=
L Nc
and T is the M L transformation matrix. The signal X can be
interpreted as a vector of coefficients of expansion using orthonormal basis
T:
x = TX
TU Darmstadt 30
Signal reconstruction:
orthonormal
recovered basis
signal compressed
= signal
Orthonormal transform compression is lossy!
transmission
x(n) or storage ~
x(n)
compression reconstruction
TH of T
compressed
data
TU Darmstadt 30
Good candidates for the transform X:

DFT
DCT
Karhunen-Loeve (principal eigenvectors based) transform
SVD of a N N matrix
N
X
R= iUiViH 1 2 . . . N
i=1
Compressed representation
M
X
R' iUiViH , M <N
i=1
TU Darmstadt 30
Example: the original (noisy) signal

2.5
1.5
1
ORIGINAL SIGNAL
0.5
0.5
1.5
2.5
100 200 300 400 500 600 700 800 900 1000
DISCRETE TIME
TU Darmstadt 30
Reconstructed signal after compression with the compression ratio = 10

2.5
1.5
1
RECONSTRUCTED SIGNAL
0.5
0.5
1.5
2.5
100 200 300 400 500 600 700 800 900 1000
DISCRETE TIME
TU Darmstadt 30
Reconstructed signal after compression with the compression ratio = 100

2.5
1.5
1
RECONSTRUCTED SIGNAL
0.5
0.5
1.5
2.5
100 200 300 400 500 600 700 800 900 1000
DISCRETE TIME
TU Darmstadt 30
CONCLUSIONS
many of you will work on DSP problems in future
when working with digital signals, do not forget their sampling rate!
we considered only the most general topics of DSP
to broaden and deepen your knowledge in specific DSP topics, an
advanced DSP course might be needed
DSP makes fun!
TU Darmstadt 31

Slides 4tl4 PDF

Transféré par

Informations du document

Titre original

Copyright

Formats disponibles

Partager ce document

Partager ou intégrer le document

Options de partage

Avez-vous trouvé ce document utile ?

Ce contenu est-il inapproprié ?

Droits d'auteur :

Formats disponibles

Slides 4tl4 PDF

Transféré par

Droits d'auteur :

Formats disponibles

CoE 4TL4, Hamilton 200

DIGITAL SIGNAL PROCESSING

Department of Electrical and Computer Engineering

CALENDAR OF DSP COURSE

Alan V. Oppenheim and Ronald W. Schafer

1 INTRODUCTION TO DISCRETE SIGNALS

Examples of analogue signals appearing in nature:

electrical signals: voltages, currents, fields

Analogue processing may include the following operations:

linear: amplification, filtering, integration, differentiation

Limitations of practical analogue processing:

restricted accuracy (e.g., component variations with time,

converting analogue signals into a digital (usually binary) sequence

A typical DSP scheme

ANALOG ADC DAC ANALO

Some application areas of DSP:

Music: recording, playback, mixing, synthesis, storage (e.g. CD-players)

1.2 Sampling and Analog-to-Digital Conversion

Converting an analog signal into a binary code

Input-output characteristic of 3-bit quantizer

1.3 Basic Types of Digital Signals

The unit-step and unit-impulse functions

Harmonic signals: an example of importance

Complex exponentials (cisoids):

t , i.e. discrete = analog t

ej(n+) = cos(n + ) + j sin(n + )

Differences between sampled exponentials and their analog counterparts:

analog exponentials and (co)sinusoids are periodic with T = 2/,

x(n) = x(n + N ) = ejn = ej(n+N ) = exp{jN } = 1

The unit-step function and one of many analog

Ambiguity condition for digital sinusoids:

|f1 f2| = m/t , m = 1, 2, . . . ()

Taking f1 = F and f2 = F in (), we obtain that there is no ambiguity

A system is linear if and only if:

T {ax1(n) + bx2(n)} = aT {x1(n)} + bT {x2(n)}

a x1(n) + b x 2 (n) Linear system a y1( n) + b y2 (n)

Example: Let y(n) = x2(n) (i.e., T {} = ()2). Then,

T {x1(n) + x2(n)} = x21(n) + x22(n) + 2x1(n)x2(n)

Hence, this system is nonlinear!

A time-invariant system has properties unvarying with time, i.e.:

if y(n) = T {x(n)} = y(n k) = T {x(n k)}

Linear Time-Invariant (LTI) system is a system that is both linear and

2.2 Digital Signals via Impulse Functions

Let h(n) be the response of the system to (n).

The sequence {h(n)} is commonly referred to as impulse response of the

(n) LTI h (n)

the order in which two sequences are convolved is unimportant!

Other properties of convolution:

Example: Convolution of two rectangles:

y(n) = {x(n)} {x(n)}

Example: Convolution of two sequences {x(n)} = {. . . , 0, 1, 2, 3, 0, . . .}

Result: LTI systems are stable if and only if

Now, it remains to prove that if

Example: An LTI system with

since h(n) = 0 for n < 0, the system is causal

the system is stable only for |a| < 1

2.3 Linear Constant-Coefficient Difference (LCCD) Equations

Consider LTI systems satisfying

Property: MA systems are bounded-input bounded-output (BIBO) stable,

y(n) = ay(n 1) + x(n)

is unstable for a 1, because y(n) is unbounded for bounded x(n).

Property: MA systems have finite impulse response (FIR), whereas AR

Proof for AR systems: y(n) depends on y(n k), k = 1, . . . , =