Stationary Random Process and Digital Signal Processing - Theory

NOTES ON
STATIONARY RANDOM PROCESS AND

DIGITAL SIGNAL PROCESSING
Prepared by Le Thai Hoa
2004
1 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing
1. STATIONARY RANDOM PROCESS

1.1. Basic concepts
(1) Continuous random process:
xk (t ) {x1 (t ), x2 (t ), x3 (t )}...xK (t )} , t
Where: { }: Ensemble of sample functions xk(t)
k: Index of sample function (k=1,2,3K)
t: Time variable
Random process {xk(t)} = Ensemble of sample function xk(t)

(2) The random process is called as the K-variate random process (multi-variate
random process)
Time shift
xk(t)
kth sample function
x1(t)
1st sample function

t
t
t+
Ensemble (sample records) of random signal
(3) For discrete sample function, discrete values of any sample random function are
measured at certain time points t1, t2, t3, tN (N: number of sampling values of
sample function)
xk (t ) xk (t1 ), xk (t 2 ), xk (t3 )}...xk (t N )} : Discrete sample function

1.2. Classification of random process
(1) Classification of random process can be widely expressed as follows
Random process
or Stochastic field
Stationary process
Ergodic process
Gaussian process
Non-Gaussian process
Non-ergodic signals
Non-stationary
processes
Classification of random processes
1.3. Representation of random process:

(1) Time-domain representation (as raw formats and sources)
(2) Frequency-domain representation (due to Fourier Transform)
(3) Time-frequency representation (due to Wavelet Transform)
1.4. Characteristics of random process

Basic statistical characteristics of two arbitrary random processes x k (t ) and
y k (t )
(1) Mean value (Expectation): First-order statistical moment
x (t ) E[ x k (t )] Lim N
1
N
(t ) dt
y (t ) E[ y k (t )] Lim N
1
N
(t )dt
(2) Variance and covariance: Second-order moment
2 x (t ) E[( xk (t ) x (t )) 2 ] E[ x 2 k (t )] k2 (t )
: Variance
2 y (t ) E[( y k (t ) y (t )) 2 ] E[ y 2 k (t )] y2 (t )
: Variance
C xy (t ) E[( xk (t ) x (t ))( y k (t ) y (t ))]

: Covariance
E[ xk (t ) * y k (t )] x (t ) * y (t )
C xx ( ) E[( xk (t ) x (t ))( xk (t ) x (t ))]
: Covariance
C yy ( ) E[( yk (t ) y (t ))( y k (t ) y (t ))]
: Covariance
Note: Zero mean value process xk(t): x (t ) 0
2 x (t ) E[( xk (t )) 2 ]
: Variance
2 y (t ) E[( y k (t )) 2 ]
: Variance
C xx (0) 2 x (t )
C yy (0) 2 y (t )
(3) Mean square and root mean square
C xx (t ) E[ x 2 k (t )]
: Mean square
C yy (t ) E[ y 2 k (t )]
: Mean square
Note: Zero mean value process xk(t): x (t ) 0

Variance 2 x (t ) = Mean Square Cxx(t)
(4) Correlations: Second-order moment
Rxx ( ) E[ xk (t ) * xk (t )]
: Autocorrelation
R yy ( ) E[ y k (t ) * y k (t )]
: Autocorrelation
Rxy ( ) E[ xk (t ) * y k (t )]
: Cross-correlation
: arbitrary time (time shift or time lag)
Note:
C xx ( ) Rxx ( ) 2 x
C yy ( ) R yy ( ) 2 y
C xy ( ) Rxy ( ) x y
Zero mean random process:
x (t ) 0 , y (t ) 0
C xx ( ) Rxx ( )
C yy ( ) R yy ( )
C xy ( ) Rxy ( )
(5) Correlation coefficients
C xx ( ) Rxx ( ) x2
xx ( )
Rxx ( )
Rxx ( )
yy ( )
xy ( )
C yy ( )
R yy ( )
C xy ( )
Rxy ( )
: Auto-correlation coefficient
R yy ( ) y2
R yy ( )
: Auto-correlation coefficient
Rxy ( ) x y
Rxy ( )
: Cross-correlation coefficient
xx ( )
yy ( )
xy ( )
C xx ( ) C xx ( )
Rxx ( )
2x
C yy ( )
R yy ( )
C xy ( )
Rxy ( )
C yy ( )
2y
C xy ( )
x y
Note 1:
i)
C xx ( ) R xx ( )
ii)
C xx ( ) C xx (0)
iii)
Rxx ( ) Rxx (0)
iv)
C xx (0) x2 and C yy (0) y2
v)
| C xy ( ) |2 C xx (0) * C yy (0)
vi)
| C xy ( ) |2 x2 y2
vii)
| Rxy ( ) |2 Rxx (0) * R yy (0)
viii)
Rxx ( ) Rxx ( ) and R yy ( ) R yy ( )
Rxy ( ) R yx ( )
Note 2:
i)
C xx (0) x2
: Variance
ii)
C yy (0) y2
: Variance
iii)
Rxx (0) E[ x 2 (t )] x2
: Mean square
iv)
R yy (0) E[ y 2 (t )] y2
: Mean square
(6) Power spectral density (PSD) function in frequency-domain
1.5. Power spectral density (PSD)

PSD function can be computed by following methods: i) Via correlation function (by
definition), ii) Via Fourier transform and iii) Via filter-squaring-averaging computation
(1) Spectra via correlation (by Fourier Transform of correlation)

By definition of spectral density through the Fourier Transform:
S xx ( f ) Rxx ( ) * e j 2f d
: Auto-spectral density function
S yy ( f ) R yy ( ) * e j 2f d
: Auto-spectral density function
S xy ( f ) Rxy ( ) * e j 2f d
: Cross-spectral density
function
Inverse Fourier Transform:
Rxx ( ) S xx ( f ) * e j 2f df
: Auto-correlation
R yy ( ) S yy ( f ) * e j 2f df
: Auto-correlation
Rxy ( ) S xy ( f ) * e j 2f df
: Cross-correlation
Sxx(f), Syy(f), Sxy(f): Two-sided spectra, f[-,]
One-sided spectral densities
S xx ( f ) Rxx ( ) * cos 2f d 2 Rxx ( ) * cos 2f d
Rxx ( ) 2 S xx ( f ) * cos 2f d
0
Changing the two-sided spectral density Sxx(f) with f[-,] to the one-sided spectral
density Gxx(f) with f[0,]
Gxx ( f ) 2 S xx ( f )
G yy ( f ) 2 S yy ( f )
Gxy ( f ) 2 S xy ( f )
Gxx(f)=2Sxx(f): One-sided
Spectra
Sxx(f): Two-sided
[-,0]
[0,]
Thus,
Gxx ( f ) 4 Rxx ( ) * cos 2f d ; f[0,]

0
Rxx ( ) Gxx ( f ) * cos 2f df ; f[0,]

0
Real part and imaginary part of one-sided cross-spectral density:
f(Hz)
Gxy ( f ) 2 Rxy ( ) * e j 2f d C xy ( f ) iQxy ( f )
C xy ( f ) : Co-spectrum
Qxy ( f ) : Quadratic spectrum
Writing in standard form:
G xy ( f ) | G xy ( f ) | e
j xy ( f )
Where:
| Gxy ( f ) | C xy2 ( f ) Qxy2 ( f )
xy ( f ) tan 1
Qxy ( f )
C xy ( f )
C xy ( f ) | Gxy ( f ) | cos xy ( f )
Qxy ( f ) | Gxy ( f ) | sin xy ( f )
Rxy ( ) [C xy ( f ) cos 2f iQxy ( f ) sin 2f ]df

0
Rxy (0) C xy ( f ) df
0
(2) Spectra via Fourier transform

Fourier Transform (Kinchint-Weiners pair):
X k ( f ) xk (t )e j 2ft dt
0
xk ( f ) X k ( f )e j 2ft df
0
Spectral density function:
Gxx ( f ) 2 LimT
1
E[| X k ( f ) |2 ]
T
G yy ( f ) 2 LimT
1
E[| Yk ( f ) |2 ]
T
Gxy ( f ) 2 LimT
1
E[ X k ( f ) * Yk ( f )]
T
1.5. Coherence
Coherence plays the same role as the correlation coefficient. The correlation coefficient
is expressed in time domain, whereas the coherence in frequency domain.
(1) Correlation coefficient:
| Rxy ( ) |2 Rxx (0) * R yy (0)
| E[ x(t ) * y (t )] |2 E[ x 2 (t )] * E[ y 2 (t )]
| C xy ( ) |2 C xx (0) * C yy (0) 2 x * 2 y
| E[( x (t ) x ) * ( y (t ) y )] |2 E[( x (t ) x ) 2 ] * E[( y (t ) y ) 2 ]
Thus,
xy ( )
C xy ( )
x * y
: Correlation coefficient
0 xy ( ) 1
xy ( ) 0 : x(t), y(t) Uncorrelated;

xy ( ) 1 : x(t), y(t) Correlated;
(2) Coherence:
| Gxy ( f ) |2 Gxx ( f ) * G yy ( f )
| Gxy ( f ) |2
xy ( f )
Gxx ( f ) * G yy ( f )
0 xy ( f ) 1
(3) Role of coherence function
xy ( f ) (constant-parameter linear systems) can be
interpreted as fractional portion of the mean square value at the output y(t) that is
contributed by the input x(t) at frequency value f. In contrast,
the quantity
[1 xy ( f )] is the portion of mean square value of output y(t) not be contributed by

input x(t) at frequency f.
Note: The role of coherence function

coefficient function
2 xy ( )
xy ( f ) is similar to the correlation
. In constant-parameter linear systems, the
coherence has some following possibilities:

2
1)
xy ( f ) 0 : x(t) and y(t) uncorrelated (unrelated)
2)
xy ( f ) 1 : x(t) and y(t) correlated (unrelated)
3)
0 xy ( f ) 1 : x(t) and y(t) some possible situations exist:
a. Extra noise
b. Non-linear system between input x(t) and output y(t)
c. SIMO or MISO
xy
(f)
xy ( f )
C xy ( f ) 2 iQxy ( f )
Gxx ( f ) * G yy ( f )
C xy ( f ) 2
Gxx ( f ) * G yy ( f )
: Coherence
C xy ( f ) : Co-spectrum (Real part of Cross-spectrum)

Gxx ( f ), G yy ( f ) : Auto-spectra
2. DIGITAL SIGNALS AND CLASSIFICATION

2.1. Digital signals
(1) The signals and data measurements are the similar concepts for almost cases in the
physical measurements and experiments. The classification of signals is important
to the digital signal processing (DSP) or data processing, especially, this closely
relates to the digital filters and discrete signal analysis.
2.2. Classifications
(1) Signals can be commonly classified in the engineering application by some follows
categories: i) Continuous (analogue) and discrete signals (digital), ii) Deterministic
and random signals
Electric signals
(by data acquisition)
Analogue Signals
(Continuous)
Sampling and
A/D conversion
Discrete Signals
(Digital)
Data Analysis and
Post-data processing
Analogue and discrete signals

Deterministic signals
Periodic signals
Sinusoidal signals
(2 cycle)
Complex periodic signals
(T cycle)
Nonperiodic signals
Almost periodic signals
Transient signals
Branches of deterministic signals

Random signals
Stationary signals
Ergodic signals
Non-ergodic signals
Nonstationary signals
Specific classification of
nonstationary signals
Branches of random signals
Notes and comments:

i) Sinusoidal (harmonic) signals (2-periodic signals):
ii) Complex periodic signals (T-periodic signals):
x (t ) x (t 2 n )
x(t ) x (t nT )
The complex periodic signals may be expanded by a Fourier series into the
combination of harmonic components (sine and cosine functions) as follow:
x(t ) X 0 cos(
2
t 0 )
T
a0
1
1
x (t )
[an cos 2n t bn sin 2n t ]
2 n 1
T
T
iii)
(Original signals)
(Fourier series)
Almost-periodic signals: can be expressed by the sum of sine functions that

their frequencies are not periodic.
iv)
Transient signals: can be defined as totally non-periodic signals (In other

word, the transient signals can be considered as the deterministic signals but
out of any kinds of periodic and almost-periodic signals). Apart from
periodic and almost-periodic signals, however, the spectrum of transient
data only exists under form of continuous spectrum but the discrete spectrum
does not exist.
(2) The characteristics of the random signals can be clarified by some following
quantities:
i)
Amplitude distribution: Mean value (expectation) or root mean square

(standard deviation) [first moment]
ii)
Correlation functions (auto- and cross-correlations) [joint or second moment]
iii)
Power spectral density (PSD) [power contribution of each frequency

components]
(3) For discrete signals, the above-mentioned quantities can be expressed by formulas:
1
x
N
1
xi ; rms x
N
i 1
1
R xx ( )
N
1
Sx ( f )
N
i 1
x (t
i
) x i (t 0 )
i 1
N
x e
j 2nk / N
; n=[1,N]
k 1
(4) Some hints on types of random processes

i)
Stationary signals are that their mean value and correlation of discrete
signals do not vary on time.
ii)
Non-stationary signals, by contrast, their mean value and correlation vary on

time.
x (t ) x
R xx (t , ) R xx ( )
iii)
Weakly stationary signals (or stationary in the wide sense) are that mean
value and correlation (first and joint moments) are time invariant.
x (t ) x
R xx (t , ) R xx ( )
iv)
Strongly stationary signals (or stationary in the strict sense) are that all
moments and joint moments (not only first moment but also high-order
moments) are time invariant.
x (t ) x
R xx (t , ) R xx ( )
(first moment and joint moment)
th
i x (t ) x
R
v)
i th
xx
(t , ) R xx ( )
(ith moment and ith joint moment)
Ergodic signal is stationary one (Mean value and correlation are time
invariant), moreover, its mean value and correlation are constant with
different samples of signal.
x (k ) x
Rxx
(k )
( ) Rxx ( )
k: Index of kth sample of signal
vi)
Non-ergodic signal is stationary one (Mean value and correlation are time
invariant), however, its mean value and correlation are differed with
different samples of signal.
x (k ) x
R xx
vii)
(k )
( ) R xx ( )
Gaussian signal (Normal distributed signal) is ergodic stationary one with

zero-mean and standard deviation
x (k ) 0
(k )
(k )
Rxx ( ) C xx ( ) 2
viii)
Non-gaussian signal is ergodic stationary one with non zero-mean and

standard deviation
x (t ) x 0
C xx ( ) 2
Note 1:
Single random process: Signal of one physical quantity (phenomenon) at one
position
Multi-random processes: Signals of many physical quantities at different
positions
Sampling: Signal of random process at any time ( time interval)
Multi-dimensional process: are multi-variable function
Multi-variate process: Vector of many signals
Multi-variate and multi-dimensional process: Vector of many processes (signals)
in which each individual signal is multi-variable function
Ensemble: the collection of sample functions (any time interval) of one signal.
Thus random signal is the collection of these sample function of one signal
Note 2:
Process (Field): Display and illustration of one physical quantity at one position
(If measurement of the same physical quantity at one position differs from that
at another position due to its distribution and redistribution)
Sample: Display and illustration of a physical quantity at one position at any
time interval
Record: Discrete display of a physical quantity at one position and one time
Signal: Electric display (continuous or discrete) of a physical quantity at one
position and one time
Difference between ergodic and non-ergodic processes

Ergodic stationary random process
Non-ergodic stationary random process
- Different initial phase angles
- Different initial phase angles
- The same amplitude
- Different amplitude
- The same frequency
- The same frequency
Summary on classifications and definition of random signals

No.
1
Items
Stationary
Definition
Note
Mean value and correlation not Time-invariant

vary on time
Stationary-Ergodic

vary on time and sampling
Sampling-independant
Stationary-
Nonergodic
vary on time, but sampling
Weak stationary
Mean
value
and
Sampling-dependant
correlation Time-invariant of only
(first-order moments) not vary on first moments

time
5
Strong stationary
All first-order and high-order Time-invariant of first

moments not vary on time
and
moments
Non-stationary
Mean value and correlation vary Time-variant

on time
high-order
(5) The ensemble (collection of sampling records) of random signal can be divided by
two categories: Individual sample records (one process) and Multiple sample
records (many processes). The characteristics of individual and multiple sample
records of one and many processes can be expressed by figure hereinafter:
One sample of one process
Pairs of two samples of one

process or many processes
Individual Sample Records

Analysis
Multiple Sample Records
Mean Values and

Root Mean Square(RMS)
Probability Density
Functions
Auto-Correlation Function
Analysis
Auto-spectral Density
Analysis
Joint Probability Density

Functions
Cross-Correlation Function
Analysis
Cross-spectral Density
Analysis
Coherence Function
Analysis
Frequency Response
Function (FRF)
Characteristics of individual and multiple sample records of random signals
Note 3:
Coherence function: is the relation between the power auto-spectral density and
the power cross-spectral density.
Frequency response function (gain factors and phase factors): is also the linear
relation between the power auto-spectral density and the power cross-spectral
density.
(6) Applications of statistical functions

From correlation functions
i) Similarity between 2 signals or 1 signal at different positions and time delays
ii) Prediction of signals in noise, influence of noise
iii) Identification of propagation directions and velocities
iv) Measurements of time delays
From power spectral density (PSD) function

i) Power contribution of each frequency components
ii) Identifications of system properties and input signals from output ones
iii) Identification of noise and energy sources
From coherence function

i) Accuracy of linear input/output systems
ii) Identification of propagation directions and velocities
From frequency response function (FRF)

i)
Identification of relationship between input and output signals
2.3. Relationships of input and output signals

(1) The input and output signal systems can be commonly classified by: i) Single input
and single output systems (SISO);
ii) Single input and multi output systems
(SIMO); iii) Multi input and single output systems (MISO); iv.Multi input and
multi output systems (MIMO).
(2) In the practical applications, many signal channels are simultaneously measured at
various positions or different time delays. For many cases of MIMO systems, many
input and output signals can be correlated or uncorrelated measured simultaneously.
Frequency Response
Function (FRF)
Input signal
x(t)
Hxy(f)
Signal noise
n(t)
Output signal
y(t)
SISO system
Input signal
x1(t)
Hxy,1(f)
Signal noise
n(t)
Output signal
y1(t)
Input signal
x2(t)
Hxy,2 (f)
Output signal
y2(t)
Channel No.2
Output signal
yk(t)
Channel No.k
Channel No.1
..
Input signal
xk(t)
Hxy,k (f)
Independent MIMO system
3. DATA ACQUISITION, PROCESSING AND ANALYSIS

3.1. Introduction
(1) In order to understand and clarify the physical measurements, signal processing and
analysis for the buffeting experiments and other specified measurements as well in
the wind tunnel, it is important to understand the digital signal processing (DSP)
and measurement procedures and instruments in wind tunnels.
(2) This study hinges on some following points:
1) Instrumental systems their by function: data acquisition, A/D conversion, data
qualification and analysis
2) A/D conversion and sampling theorem for eliminating the aliasing errors
3) Data analysis techniques
3.2. Data processing procedure

(1) The digital signal processing procedure can be expressed by the following diagram:
Data Acquisition
Data Conversion
Data Qualification
Data Analysis
Transducer
A/D Conversion
Classification
Individual Sample
Records
Signal Conditioning
Aliasing Errors
Validation
Multiple Sample
Records
Signal Calibration
Quantization Errors
Editing
Signal processing procedures for measurement systems
(3) Three following main steps of the digital signal processing procedure will be
overviewed as follows:
3.2.1. Data acquisition

Transducer: Device and sensor to transform from any physical phenomena
(force, pressure and motions: displacement, velocity and acceleration) to
electric signals. Transducers commonly are employed two kinds of materials:
piezoelectric and strain-sensitive materials.
+) Piezoelectric materials: frm physical quantities to electric charge, such
as naturally polarized crystals like quartz and artificially polarized
ferroelectric ceramics like barium titanate.
+) Strain-sensitive materials: from physical quantities to resistance, such
as
metallic
like
copper-nickel
alloy
and
semiconductor
like
monocrystalline silicon.
Signal conditioning: Change from the electric signals (charge and resistance)
to voltage.
3.2.2. Data conversion

Analogue to digital converter (ADC): Transforms from continuous analogue
signal to digital signal
Aliasing errors: Eliminated by sampling theorem: sampling frequency (F) or
sampling time interval (t)
Note: ADC can be stored under two types of codes: binary and ASCII codes
Binary code: By numbers 0 and 1 (8 bytes)
ASCII code: By numbers from 0 to 9 (1byte). However, almost data of
discrete signals (after ADC) has been stored under this ASCII code in
application, because it is easily red by any applied programs such as Matlab
and Fortran for data post-processing.
3.2.3. Data qualification

Many imperfections of received signals with noise can be reduced by the
digital or analogue filters.
Data qualification consists of: classification, validation and editing.
3.2.4. Data analysis

Individual sample records (Uni-variate process):
Sample collection of
measurement data of one physical phenomenon at one point in various time

intervals.
Multiple sample records (Multi-variate processes): Sample collection of many
individual records of one physical phenomenon at numerous points or of some
physical phenomena at one point in various time intervals.
Individual Sample Records

Analysis
Multiple Sample Records
Mean Values and

Root Mean Square(RMS)
Cross-Correlation Function
Analysis
Auto-Correlation Function
Analysis
Auto-spectral Density
Analysis
Cross-spectral Density
Analysis
Coherence Function
Analysis
Fig. 2. Individual and multiple sample record analysis
Individual sample record and uni-variate process

Mean and root mean square (RMS) value computation:
Auto-correlation function computation
Auto-spectral density function computation
Multiple sample record and multi-variate processes

Cross-correlation function computation
Cross-spectral density function computation
Coherence function computation
3.3. Data analysis procedures

3.3.1. Data sampling and data preparation
Multi input and multi output system (MIMO) = Combination of independent single
input and single output systems (SISO)
Input signal ui (t)

Continuous Data
Frequency
Response Function
Signal Filter
A/D Conversion
Data Sampling
Output signal xn(t)

Discrete Data
n: Index of ensemble (sample record) or signal or random process
Multi sample record (of multi signals) = Collection of number of individual sample
records (of individual signal). Thus analysis of the multi sample record can be carried
out by analysis of the individual sample record and analysis of the pairs of correlated
two individual sample records.
Continuous Signal
A/D Conversion
Data Sampling
Discrete Signal
Discrete Data Record
Discrete valued signal (or random process):
xn
n=1,2,3 N (N: Number of samples)
Equally spaced time interval (T: sampling time or sampling period) of samples in
discrete signal:
Tn T0 n * T
n=1,2,3 N
T0: Initial time
Tn n * T
(T0 = 0)
T: Sampling period
NT: Total time length of discrete signal;
F: Sampling frequency (Hz);
F0: Fundamental sampling frequency;
Thus,
xi x(Tn ) x (T0 n * T )
N *T
1
T
F0
1
N *T
n=1,2,3 N
Noting that the Limit sampling frequency (the Nyquist frequency) for eliminating the
Aliasing Errors
FN
1
2 *T
3.3.2. Data standardization

The purpose of data standardization is to transform the original data record
new type of data record
x' n (can be called the fluctuating data record) that has the
zero mean value.
1) Mean value and root mean square of sample record
1
x
N
x n to
n=1,2,3N
(Mean value)
n1
1 N 2
s x SQRT [
x n]
N 1 n 1
n=1,2,3N (Root Mean Square
value)
Note: For the stationary ergodic data record (time-invariant and samplingindependent), it is very convenient and common to transform the initial data
record
xn to the new zero-mean data record x'n .
2) Establishment of the fluctuating data record (Zero-mean data record)
x'n :
n=1,2,3N
x'n xn x xn (nT ) x
n=1,2,3N
Having:
x' 0
2
x'
(Zero-mean value)
1 N 2
x' n
N 1 n 1
(Mean Square value or Variance)
1 N 2
s x ' x ' SQRT [
x' n ]
N 1 n 1
(Root Mean Square or Standard Deviation)
3) Fourier Transform (Discrete Fourier Transform-DFT):

Fluctuating data record (Zero-mean data record) or standardized data record
x'n
n=1,2,3N
Discrete Fourier Transform
1
X ( Fk )
[ x(n T ) * exp( j 2 F
n T )]
n 1
X(Fk): DFT at computational frequency
Fk
k: Index of discrete frequencies

Fk: Computational frequency; Fk k ( f )
Note:
i) The frequency space (f ) = The fundamental sampling frequency (F0)
f F0
1
NT
ii) Number of frequency space (K)
Fc Fs
F0
Fc : Cut-off frequency
Fs : Starting frequency
iii) Computational frequency Fk:
Fk Fs k ( f ) Fs
Fk
k
;
NT
k
;
NT
k=0,2,3K-1
k=0,2,3K-1
Note: Index of computational frequency starts from 0

Frequency value starts from Fs (???)
iv) DFT of x(t) at the computational frequency
k
1
X ( Fk ) X (
)
NT
k
1
X ( Fk ) X (
)
NT
[ x(n T ) * exp( j 2 NT n T )] ;k=0,2,3K-1

n 1
N
[ x(n T ) * exp( j 2 k n / N )] ;k=0,2,3K-1

n1
DFT
X(Fk)
X(F1)
X(Fi)
f(Hz)
F1=Fs
k=1
Fi=Fs+i/NT
k=i
1
k=1: X ( F1 ) X ( Fs )
F1=Fc
k=K
N
[ x(n T ) * exp( j 2 n / N )]
n 1
1
1 N
) [ x (n T ) * exp( j 2 2 n / N )]
k=2: X ( F2 ) X ( Fs
NT
n 1
2
1
)
k=3: X ( F3 ) X ( Fs
NT
[ x(n T ) * exp( j 2 3 n / N )]
n 1
Note: +) In the DFT formula, factor (1/) appears (DFT standardization)

+) In Matlab command, X=FFT(x) (without standardization)
Inverse Discrete Fourier Transform:
1
x( nT )
N
[ X ( NT ) * exp( j 2 k n / N )] ;
k 1
n=0,1,2N
4) Auto-correlation function
Auto-correlation function can be computed by i) Direct computation by definition and
ii) Indirect computation via FFT. Note that the second method is more efficiency
because of application of FFT algorithm, however, the first one is easier to compute
but more time consuming.
Method 1: Direct computation of auto-correlation function

x'n
n=1,2,3N
Auto-correlation function of data record x'n with delay s
1
R xx (s)
N
[ x
(n)
( s) x( n s ) ( s s )] (Auto-correlation functions)
n 1
s r * t
1
R xx ( r * t )
N r
N r
[ x (s) x
n
nr
( s s )]
n 1
5) Auto-spectral density function

Auto-spectral density function can be computed by i) Ensemble Averaging and ii)
Frequency Averaging
Method 1: Ensemble-averaging technique

x'n
n=1,2,3N
1
Definition of mean power: S xx E[ x (t )]
N
2
N 1
1
x
NT
n0
2
n
NT
(t ) dt
x(t)
2T
3T
4T 5T
S xx ( f )
1
NT
6T
(N-1)T NT
N 1
| X
( f ) |2
: Two-sided spectral density
n 0
X ( f ) : Fourier transform at frequency f
1 N 1
X ( f ) x n * exp[ j 2nk / N ] ; k=0,1,2N-1
N n 0
2
G xx ( f k ) 2S xx ( f k )
NT
N 1
| X
( f k ) | 2 : One-sided spectral density
n0
Computational procedures (Ensemble-averaging technique)

1) Data Sampling: N samples of process x(t) are picked out (Data record)
xn , n=1,2,3N
2) Data Standardization: Compute the mean value of data record ( x ), then
reconstruct the Fluctuating Data Record
( x'
x'n with
0)
zero-mean value
x'n , n=1,2,3N
3) Data Blocking: Taper data record by each data blocks using Window
functions
4) Fourier Transform: Compute the Fourier Transform at frequency fk,
k=1,2,3N by using FFT technique
X ( f k ) , k=1,2,3N
5) Scale Factor Adjustment of X ( f k ) : Scale factor of X ( f k ) due to the loss
by tapering operation. (By Hanning tapering: Scale factor by 8 / 3 )
6) Spectral Density: Estimate the spectral density
S xx ( f k ) from each data
blocks
4. CORRELATION FUNCTION
4.1. Introduction
(1) The correction functions evaluate the statistical independence between the signals
(time-dependant) or stochastic processes (space-dependant). The cross correlation
function is a measure to tell us how much two processes or two signals are like each
other, whereas the auto correlation function tell us how much a process or a signal at
time t is like itself at time t+ (: time shift or time lag) or how much a process or a
signal at location (x,y,z) is like itself at another location (x+x,y+y,z+z). In addition,
to evaluate how much two processes or two signals are like each other, the correlation
coefficient function also is given.
4.2. The discrete correlation function

The discrete correlation function of two processes or two signals x, y can be expressed
as follow:
R xy ( s ) E[ x( n ) y ( n s ) ] Lim N
1
N
N 1
[ x
(n)
( s) y ( n s ) ( s s)]
n 0
E[] : Expected value or mean value
For the number of samples is taken large enough, we have following approximations:
1
R xx (s)
N
[ x
(n)
( s) x( n s ) ( s s )] (Auto-correlation functions)
n 1
1
R xy (s )
N
[x
(n)
( s ) y ( n s ) ( s s )]
n 1
functions)
s : Spatial interval or time shift
(Cross-correlation
s: Spatial coordinates or time variable

N: Number of samples
s s
Span-wise
Note: 1) Corresponding values of signals are sampled at same time.

Rxx , Rxy
Rxx, Rxy vs. s
Rxx (si )
s
si
2) Rxx(s)=Rxx(-s) and Rxy(s)=Rxy(-s)

3) Above approximate formulas are accuracy only if N
Computational procedure of correlation function:

Step 1: Setting parameters
+) Number of sample: N
+) Time shift or spatial interval: s
Step 2: Sampling
+) Sampling signals x, y
Step 3: Computing correlation function
Step 4: Plotting Rxx, Rxy versus n [1,N]
4.3. The discrete covariance function

The discrete covariance function of two processes or two signals x, y can be expressed
as follow:
xy ( s ) E[ x( n ) ]E[ y ( n s ) ] Lim N
1
N
N 1
[ x
( n)
( s )] Lim N
n 0
1
N
N 1
[ y
( n s )
( s s ) ]
n 0
For the number of samples is taken large enough, we have following approximations:
1
xx ( s )
N
1
[ x ( n ) ( s )]
N
n 1
1
xy (s )
N
[ x
( n s )
( s s)] (Auto-covariance functions)
n 1
1
[
x
(
s
)]
(n)
N
n 1
[ y
( n s )
( s s )]
n 1
(Cross-covariance
functions)
s : Spatial interval or time shift
s: Spatial coordinates or time variable

If x, y are the zero-mean processes or signals, these mean that E[x(s)]=0 and E[y(s)]=0,
then the Root mean square (R.M.S) value must be replaced to the mean value (or
Expected value):
Auto-covariance function:
1
xx ( s ) SQRT {
N
1
[
x
(
s
)]}
SQRT
{
(
n
)
N
n 1
2
[ x
( n s )
( s s )]}
( n s )
( s s )]}
n 1
Cross-covariance function:
1
xy ( s ) SQRT {
N
1
[ x ( n ) ( s )]} SQRT {
N
n 1
2
[ y
n 1
Computational procedure of covariance function:

+) Number of sample: N
+) Time shift or spatial interval: s

Step 2: Sampling
+) Sampling signals x, y
Step 3: Computing covariance function
Step 4: Plotting xx, xy versus n [1,N]
4.4. The discrete correlation coefficient function

The discrete correlation coefficient function of two processes or two signals x, y can be
expressed as follow:
xx ( s)
xy (s)
Rxx ( s)
xx ( s)
Rxy ( s)
xy (s)
(Auto-correlation coefficient function)
(Cross-correlation coefficient function)
Note: ( s ) [ 1,1] : 1 Full-correlated; 0 No correlated
4.5. Examples
Example 1: Correlation function
Initial signals x,y
Cross-correlation Rxy by definition

0.2
1
0.5
0.1
-0.5
-0.1
-1
50
100
150
Number of sample
Auto-correlation Rxx,Ryy by Xcorr(x)
0.6
-0.2
50
100
150
Number of sample
Cross-correlation Rxy by Xcorr(x,y)
0.2
0.4
0.1
0.2
-0.1
-0.2
100
200
300
-0.2
100
200
36 | Le Thai Hoa - Number

Notes onofStationary
Signal
Processing
sample Random Process and Digital
Number
of sample
300
Example 2: Correlation of signal with noise
Initial sinal+noise
Correlation functions of signal with noise

1.4
2
1.5
1.2
1
1
0.5
0.8
0
-0.5
0.6
-1
0.4
-1.5
0.2
-2
0
-2.5
-3
2
Time (s)
3
-3
x 10
-0.2
0.2
0.4
Time (s)
0.6
0.8
5. FOURIER TRANSFORM
5.1. Introduction
(1) It is well known that the Fourier Series Transform and Fourier Spectral Transform
have been widely applied for almost kinds of natural and physical phenomena.
Applications and contributions of the Fourier Series and Fourier Transform concentrate
on the problem of the Digital Signal Processing (D.S.P) in the data processing and
analysis of measurements and the buffeting response prediction in which the spectral
representation cant be required.
(2) In summary, the Fourier Series and the Fourier Transform will be studied for such
purposes as follows:
1) Data processing and analysis of measurement processes in the Digital Signal
Processing (D.S.P)
2) Spectral representation in the buffeting response prediction
(3) Data measurements can be expressed under the continuous or discrete processes.
However, almost measured signals have been collected under the discrete signal for the
data post-processing and analysis. The discretization of measured signals is well
known as the sampling processes.
(4) Some following DSP techniques will be studied hereinafter:

1) The discrete Fourier series (DFS)
2) The discrete Fourier transform (DFT)
3) The fast Fourier transform (FFT)
4) Amplitude spectrum and phase spectrum
5) The discrete inverse Fourier transform (IFT)
6) The sampling technique
5.2. The Discrete Fourier Series

The Fourier series purposes to decompose any periodic or non-periodic signals into
combination of simple harmonic signals (sine and cosine functions). Post processing
and analysis on simple harmonic functions seem to be much easy than the original
signals. Mathematically, the Fourier series is known as the harmonic analysis.
T: Period (Time step of sampling) of a sample (s)
F: Frequency of a sample (Hz), F
1
T
NT: Fundamental period of series (s)

Fo: Fundamental frequency of series (Hz) Fo
1
NT
Discrete Fourier series is expressed as the sum of harmonic functions as follow:
a0 M 1
xn
[a m cos( mn 0T ) bm sin( mn 0T )]
2 m 1
0 : Fundamental frequency of series (rad/s), 0
2
2
Fo NT
m: Times of fundamental frequency (m=1: fundamental harmonic term with 0 ,

m=2: 2nd harmonic term with 2 0 , mth harmonic term with m 0 )
M: Cutting-off number of frequency
ao, am, bm: Fourier coefficients of series
n: Pointer of samples
a0 M 1
xn
[ a m cos( 2mn / N ) bm sin( 2mn / N )]
2 m1
Discrete Fourier Coefficients:
2
am
N
2
bm
N
N 1
cos( 2mn / N ) ;
sin( 2mn / N ) ;
0 m M 1
n 0
N 1
1 m M 1
n 0
Computational procedure:
-
Initial signal
Number of samples N
Number of series M
Step 2: Sampling
Step 3: Calculating Fourier Coefficients
-
am, bm
Step 4: Simulating Fourier series and Plotting Simulated signal vs. number of
samples (n)
5.3. The Discrete Fourier Transform

x(t): time-dependant signal or stochastic process
T: Sampling cycle (time interval for a sample)
F: Sampling frequency (F=1/T: number samples per a second)
N: Total samples
NT: Total time for sampling or fundamental period of transform
Fo: Fundamental frequency of transform F0
1
NT
Discrete Fourier Transform: (to frequency domain)

N
X ( k ) [ x ( nT ) e j k n T ]
n 1
1
X ( fk )
2
[ x(nT ) e
j 2 f k n T
n 1
(Note: In origin, n=0N-1)
k: Number of discrete frequencies in range

In Matlab: k k ( ) k 0 [k=1-K]: equal spacing of frequency range
0 : Fundamental frequency (rad/s), 0
2
2
Fo NT
Note: n (pointer of samples) and k (pointer of frequency) have different

meaning
Computational procedure:
- T (cycle sampling), N (numbers of samples)
- Freq. range: fs (starting freq.), fc (cut-off freq.)
- Number of freq. interval: K ( fc fs) / k
Step 2:
Loop 1: Frequency
For k=1 to K k k ( )
Loop 2: Sampling
N
For n=1 to N X (k ) x (nT )e jk ( ) nT

n 1
Step 3: Plotting X( k) versus ( k)
Putting 0
2
2
into formulas:
Fo NT
N
X (k 0 ) [ x ( nT )e jk0 nT ]
n 1
X (k 2F0 ) [ x (nT )e
jk
2
nT
NT
n1
X ( kF0 ) [ x( nT )e j 2kn / N ]
n 1
Fourier Series (in time domain)
a0 K
h(t )
[ ak cos( k k t ) bk sin( kk t )]
2 k 1
ak
2
Re{H ( k )}
N
bk
2
Im{ H (k )}
N
Note in processing using FFT:

1) 2 /( NT ) or F 1 /( NT )
2) Maximum frequency: Fmax F / 2 1 /(2T )
3) Spectrum calculated at certain frequencies: 0, F, 2F, 3F Fmax
Leakage Effect: Amplitudes will distribute on adjacent closed frequencies. This effects
occurs in cases that total sampling time NT does not coincide the integer multiple of
the sampling cycle T. To prevent the leakage effect by Averaging Method or Window
Functions
5.4. The Fast Fourier Transform

It is very well-known that the Fast Fourier Transform (FFT) algorithm has been
powerfully used for solving the Discrete Fourier Transform. FFT is not a new
transform itself due to using the same DFT formula, however, its FFT algorithm is
much faster than the conventional DFT. In principle, the FFT algorithm eliminates the
component repetition to make the faster computation.
N 1
j 2kn / N
]
DFT formula: X ( kF0 ) [ x ( nT )e
n 0
N 1
X k x nW Nkn ; k=[0,N-1]; WN e j 2 / N
n 0
Xk contains N2 components (N of xn and N of WN)

j
Note: e cos j sin = x + jy
(Eulers Formula)
e j 2kn / N cos(2kn / N ) j sin( 2kn / N )

N 1
2n
N
n 0
1 e
1 e
2N
N
j
2
N
Sum can be expressed in the complex plane of 2=3600 (N=8 for example)
Im
900 k=2
1350 k=3
j 2 2 / 8
450 k=1
e j
0
e j 2 0 / 8 0 k=0
1800 k=4
e j 2 5 / 8
e j 2 7 / 8
2250 k=5
3150 k=7
2700 k=6
Complex plane (N=8)
Complex plane
Re
Based on above-mentioned notes, we can easily obtain:
W Nm W Nrem ( m , N ) ; m[0,N-1]; rem(m,N) is the remainder after dividing m

by N
W Nm e ( j 2 / N ) m ; m[0,N-1]
W Nrem ( m , N ) e ( j 2 / N )( iN m ) ; i=1,2,.. Max(kn)
N
For example, N=8, n=[0:7], k=[0:7], max(kn)=49, components W are expressed by

the complex plane
W82 W810 W818

W81 W89 W817
W83 W811 W819
W80 W88 W816
W84 W812 W820

W85 W813 W821
W87 W815 W823

W86 W814 W822
Complex plane (N=8)

Thus, among 49 components, 8 of these are unique.
Then, suppose N is a multiple of 2, we decompose the samples into two vectors
containing even- and odd-numbered samples as follows:
N / 21
Xk
N / 21
x2 nW
2 kn
N
n 0
Due to:
W N2 kn ; k=[0:N-1]
2 n 1
n 0
W Nk / 2 W N2 k/ 2 , thus
N / 2 1
Xk
n 0
N / 2 1
x 2 nW
kn
N /2
W Nkn/ 2 ;
2 n 1
n0
N
2
Number of components reduces to ( ) 2 < N 2 of original problem. Depending on N,

efficiency of FFT algorithm is different. For example, if N=2K then complex
components in FFT is
N
N
2
K log 2 N (in comparison of N components in DFT)
2
2
5.5. The Discrete Inverse Fourier Transform

The discrete inverse Fourier transform of X(f) can be expressed as follows
X ( kF 0 )
[ x ( nT ) e
j 2 kn / N
] ; k=[0:N-1]:
n 01
1
x ( nT )
N
N 1
[ X ( kF )e
j 2 kn / N
] ; n=[0:N-1]: Inverse Transform
k 0
5.6. Amplitude spectrum and phase spectrum

The amplitude spectrum is defined as the vector of DFT component amplitudes,
whereas the phase spectrum is vector of DFT component phase angles in radian unit.
5.7. The Discrete Sampling Theory

The problem in the sampling technique is to require 2 necessary points:
1) Sampling values are required enough for the data processing and analysis.
2) Sampling the pick-up values from either continuous signals or discrete signals
represents the original one. This means that from the sampling values can
reconstruct the similar original signals.
The sampling parameter: Sampling rate or sampling frequency Fs(Hz).

The sampling theorem may be stated as follows: If continuous signals is sampled at a
sampling rate or sampling frequency that is twice higher than their highest frequency
component, then it is possible to recover and reconstruct the original signals from
samples
Fsampling 2 Fmax,signal
5.8. Examples
Example 1: D.F.S
1.2
0.9
0.8
Signal from
D.F.S
0.6
0.5
0.4
0.3
0.4
0.2
-0.2
0.1
10
20
30
40
number of samples
50
60
70
-0.4
0.4
10
20
30
40
number of samples
50
60
-0.2
70
1.2
1.2
1.2
0.8
0.8
0.6
0.4
0.2
-0.2
0.6
0.4
0.2
10
20
30
40
number of samples
50
60
70
-0.2
10
20
30
40
number of samples
50
60
70
10
20
30
40
number of samples
50
60
70
0.6
0.4
0.2
0
0.8
N u m b e r o f s e rie s M = 5 0
0.6
0.2
Initial Signal
Impulse Function
0.2
0.8
N u m be r of s erie s M = 5
0.6
0.7
Nu m be r of s erie s M = 3
Nu m be r of s erie s M = 1
0.8
-0.2
10
20
30
40
number of samples
50
60
70
-0.4
Initial signal (impulse function) and signal due to Discrete Fourier Series at various
number of series (M=1,3,5,10,10,50)
Example 2: FFT vs. SPECTRUM

signal
spectra by using FFT
20
8
6
10
Amplitude
f1=8Hz
f1=33Hz
2
Noise
-10
0.2
0.4
time [t]
signal
0.6
0.8
20
20
40
60
frequency [Hz]
spectrum by using spectrum
25
20
10
15
10
5
-10
0.2
0.4
time [t]
0.6
0.8
No noise
0
20
40
frequency [Hz]
60
Fourier Transform by FFT and Spectrum
Note:
Fourier Transform
Spectrum
CFT ( x) X ( j )
jt
x(t )e dt
CSPEC( x) X ( j ) x 2 (t )e jt dt
N 1
DFT ( x) X ( j ) x n e jnT
n 0
N 1
DSPEC ( x) X ( j ) x 2 n e jnT
n 0
Example 3: DFT
Signal
FFT command
1
0.6
0.8
0.4
0.6
0.2
0.4
0.2
-0.2
-0.4
DFT Magnitude
0
0.2
0.4
0.6
0.8
Number of samples or time interval
DFT by definition
0.6
0.4
20
40
Frequency Hz
Spectrum command
60
20
40
Frequency Hz
60
8
6
0.2
4
0
2
-0.2
-0.4
20
40
Frequency Hz
60
Results: +) Fast Fourier Transform (FFT) is not a new form of Fourier transform, the
N
applied formula is exactly such an expression
X (kF0 ) [ x (nT )e j 2kn / N ] , however,

n 1
the modified algorithm is used for much faster computation of DFT

+) Result by DFTs defined formula is exactly same to that by FFTs Matlab
command. Moreover, Magnitudes of DFT and FFT are absolute values (abs(H))
+) Results by DFT and FFT are different from that by Spectrum
N
Notes: +) DFT definition X (kF0 ) [ x (nT )e j 2kn / N ] :

n 1
H0=x*exp(-j*2*pi*k'*n/N);
n: pointer of sample, k: pointer of frequency, k must be transposed (k)
H0=H0/N; H0 must be standardized by dividing by N (number of samples)
Example 4: Amplitude and phase spectrum
0.6
0.8
0.4
Amplitude
Signal
Spectrum
1
0.6
0.4
0.2
0
-0.2
0.2
0.4
0.6
Number of samples
Phase spectrum
-0.4
0.8
20
15
Phase(rad)
Phase(rad)
0.2
0
-2
-4
fftshift(H)
20
40
Frequency(Hz)
Phase spectrum
60
10
5
unwrap()
Angle(H)
0
20
40
Frequency(Hz)
60
20
40
Frequency(Hz)
60
Example 5: Amplitude and phase spectrum
1.5
signal
4
FFT
phase spectrum
unwrapped p.s
4
150
2
100
0.5
-2
50
-0.5
50
100
signal
50
100
swapped FFT
-4
50
100
50
100
1.5
p.s of swapped FFT unwrapped p.s

4
150
2
100
0.5
-2
-0.5
50
100
50
100
-4
50
50
100
50
100
6. POWER SPECTRAL DENSITY FUNCTION

6.1. Introduction
(1) The random signals can not be determined exactly. It means that the random
signals always differ from each other at the different observations, moreover, the
random signals contain the random parameters that can not be described by the means
of analytical quantities and determinant-parameter methods, but they are only able to
be described by the terms of statistical parameters that can differ from one random
signal to another.
(2) Thus the questions are that what terms of statistical parameters can be able to
describe the characteristics of the random signals. It is well known that the two mostcommonly-used means are:
i)
amplitude distribution functions of random signal
ii)
power spectral density of random signal (or equivalent as the correlation

function)
(3) In the term of amplitude distribution, the commonly-used statistical quantities are:
i) mean value (expectation); ii) mean square (variance when zero-mean signals) or root
mean square (standard deviation when zero-mean signals).
(4) It is broadly said that the square of DFT magnitudes of any function x(t) is
considered as power contribution of any frequency components in x(t) over the
frequency domain.
6.2. Characteristics of amplitude distribution

The most-commonly-used statistical terms of the amplitude distribution characteristics
are defined underneath:
1
- Mean value (Expectation): x E[ x ]
N
- Mean square
1
E[ x ]
N
2
x
- Root mean square :
N 1
i 0
N 1
i 0
N 1
1
E[ x ] sqrt (
N
2
i 0
- Variance:
1
E[( x x ) ] E[ x ] [
N
2
x
- Standard deviation:
2
x
N 1
1
x i][
N
i 0
2
N 1
x ]
i 0
x sqrt ( E[( x x ) 2 ]) sqrt ( E[ x 2 ] x2 )
Thus zero-mean random signals ( x 0 ) have the following deductions:

Mean square = Variance
Root mean square = Standard deviation
y a x b
Linear relationship of y and x as y=ax+b, we easily obtain:
x2 a 2 x2
Some distribution probability functions of measurements are widely used as

i)
uniform probability distribution:
ii)
normal (Gaussian) probability distribution:
6.3. The Power Spectral Density (PSD)

As above-mentioned, the power of random signals is expressed by the square of DFT
magnitudes of x(t) at any time t, whereas the power spectral density is expressed in the
frequency domain . Some concepts of power have been used:
The instantaneous power of signal x(t) at any time t can be defined as:
Power | x(t ) |2
For the complex signals, we must use the mean power or expected power:
T /2
T /2
1
1
2
Mean power E[ x (t )] LimT
x
(
t
)
dt
x 2 (t ) dt
T T / 2
T T / 2
2
(when T is long enough)
1 N 1 2
1
Discrete mean power S xx E[ x (t )] x n
N n 0
NT
2
NT
(t ) dt
(Definition:This is only approximate estimation, that is why it is called as the

spectral estimation)
We have the discrete inverse Fourier transform:
1
x ( nT )
N
N 1
[ X ( kF )e
j 2 kn / N
k 0
Thus, the discrete mean power can be written as follows:
1
S xx
N
N 1
1
x
N
n 0
2
n
N 1
1
[
n 0 N
N 1
X (kF )e
0
j 2kn / N 2
k 0
N 1
1 N 1 N 1
1 N 1 N 1
j 2kn / N 2
S xx 3 [ X ( kF0 )e
] 3 X k X m e j 2 ( k m ) n / N
N n 0 k 0
N k 0 m 0
n 0
1
S xx 2
N
N 1
2
k
k 0
1 N 1 2
x k
We have: S xx
N k 0
As a result, the relationship between the mean power of signal in the time domain and
power spectrum, or the mean power of signal can be presented in the term of the power
spectrum. This expression has been well known as the Parsevals Theorem.
N 1
1 N 1
x Xk
N k 0
k 0
2
k
(Parsevals Theorem)
In DSP, it is defined the periodogram that has N components given by:
Pxx (k )
1
Xk
N
k=[0,N-1]
Pxx (k ) Pxx ( N k )
The power spectrum can be obtained:
1
S xx
N
N 1
x
k 0
N 1
xx
(k )
k 0
Note: In the same way that x k2 represents a measure of signal power at a point in
the time domain, Pxx(k) represents the measure of signal power at the point in
the frequency domain.
Therefore, the vector P xx=[Pxx(k)], k=[0:N-1] is considered as the measure of
estimation of the power spectral density (PSD) of signal x(t).
Some expressions in the Power Spectral Density (PSD) S of signal x(t)
N 1
1
Discrete PSD: 1) S xx 2
N
(Frequency domain)
k 0
Xk is DFT amplitude of x(t) at frequency k

N 1
1
S
2) xx
N
(Time domain)
k 0
xk is discrete value of signal x(t) at time t

N 1
1
3) S xx
N
k 0
1
Xk
N
Pxx (k )
N 1
1
Parsevals Theorem: x
N
k 0
2
k
1
S
Continuous PSD: 1) xx
NT
N 1
xx
S xx
(Periodogram)
k 0
; m=[0,N-1]
N 1
2
k
(Parsevals Equality)
(t ) dt
(Frequency domain)
k 0
NT
/T
1 / 2T
2)
(k )
xx
( f )df or S xx
xx
( ) d
/T
1 / 2T
The relationship of the power spectrum to the auto-correlation: The discrete autocorrelation of vector x from sampled values of signal x(t) with extended period has
been defined as follows:
1
R xx ( s )
N
N 1
x x
n
n 0
n s ;
s=[0:N-1]
By definition of the mean power, we have:
1
R xx (0)
N
N 1
S xx Mean power
n0
Discrete Fourier transform of the auto-correlation can be calculated by definition:
1
S R DFT {R xx ( s )}
N
N 1 N 1
j 2sk / N
x
x
e
n n s
s 0 n 0
k=[0:N-1]
N 1
X k x n e j 2kn / N ; k=[0:N-1]
(Definition of DFT)
n0
S R DFT {Rxx ( s )}
1
Xk
N
Pxx (k ) ;
k=[0:N-1]
In the conclusion, the periodogram of vector x with periodic extension is the DFT of
the auto-correlation function of x, that means:
S R DFT {Rxx ( s )}
1
2
X k Pxx ( k )
N
Role of the auto-correlation function gives us information on how much

dependence of given sample xn on nearby samples xn+1, xn+2, and so on.
The Cross Spectrum: involves two time-dependant signals (two time series). Suppose x
and y are vectors of length N sampled from two signals x(t) and y(t).
The cross-periodogram:
1
Pxy(k) XkY'k ;
N
k=[0:N-1]
Mean cross-power:
1 N1
Sxy xn yn
N n0
1 N1
1 N1
Sxy Rxy(0) 2 XnY'n Pxy(k)
N k0
N k0
Role of the cross spectrum gives us information on how the cross
correlation function is distributed over the frequency scale.
Coherence function: The magnitude of the cross periodogram Pxy(k), that is Pxy (k ) , is
the measure of the coherence of signals x(t) and y(t) at different frequencies.
Coherence function:
Coh 2 xy ( f )
Pxy ( f )
Pxx ( f ) Pyy ( f )
6.4. Examples
Example 1: FFT and SPECTRUM
Impulse signal
0.8
0.4
0.6
The same
0.2
0.2
0.5
0.4
0.3
0.4
FFT
DFT
Amplitude
0.5
Amplitude
0.1
0
0.5
0.8
50
2
50
Spectrum
8
6
Difference in shape?
Different in value? 4
0.4
x 10
0.6
0.2
0.2
0.1
-5
0.3
0
1
0 Process and Digital
50 Signal
0 Processing
56 | Le Thai Hoa - Notes
on 0.5
Stationary Random
Time (s)
Frequency (Hz)
Frequency (Hz)
50
Note: 1) Using FFT command

H=FFT(x);
H=[H(1) 2*H(2:N/2)]/N; %Vector length limit and
standardization
2) Using DFTs definition
H=x*EXP(-j*2*pi*k'*n/N);
%DFT by definition
H=[H(1) 2*H(2:N/2)]/N;
3) Using SPECTRUM command
S=SPECTRUM(x);
S=[S(1) 2*S(2:N/2)]; %Vector length limit
4) Using spectrums definition
S=ABS(H).^2;
%Square of absolute amplitude
S=[S(1) 2*S(2:N/2)]/ (N^2); % Vector length limit and

standardize
Example 2:FFT and SPECTRUM of some types of functions

Time intervals
Frequency intervals
Sampling process
(Step 2)
FFT
(Step 3)
Digital Signal or
Random Process
SPECTRUM
(Step 4)
normal signal
uniform signal
sine signal
impulse signal
10
10
0.5
-10
FFT
0.4
0.2
-2
1.5
50
50
1
0.5
0.5
0
40
40
0
50 0
200
20
20
100
0 spectrum 50
50
50
50
0
40
20
50
Example 3: Some built-in filter function (window functions) in FFT

normal signal
FFT
20
10
FFT with boxcar
10
10
0
-10
-20
FFT with hamming

5
4
50
FFT with bartlett
50
FFT with hann
3
2
1
0
50
50
50
7. COMPUTATION OF COHERENCE FUNCTION

7.1. Introduction
(1)
Coherence function: Defined as the measure to evaluate the statistical

independence of two stochastic processes or two digital signals (but in the
frequency-domain)
(2)
Coherence function: Computed by using the cross-spectrum (two signals) and

auto-spectrum (one signal) of two stochastic processes
(3)
Correlation function: Similar to the coherence function, the correlation function

also is the measure to evaluate the statistical independence of two stochastic
processes or two digital signals (but in the time-domain)
Inter-relation or inter-influence
Stochastic process A
or digital signal A
Stochastic process B
or digital signal B
Correlation function (time domain)

Coherence function (frequency domain )
7.2. Cross spectrum and coherence function

(1) The computation of cross spectra has been used in the field of Digital Signal
Processing (D.S.P) and Discrete Data Processing (D.D.P)
(2) The Auto-spectrum represents the Fourier transform of the Auto correlation
(3) The Cross-spectrum represents the Fourier transform of the Cross correlation
(4) The Cross-spectrum is the complex valued function whose magnitude and phase
are used in the signal processing to indicate the degree of correlation between two
signals
(5) The magnitude of the cross spectrum indicates whether frequency components of
one signal are associated with large or small amplitudes at the same frequency in
the second signal.
(6) The phase of the cross spectrum indicates whether the phase lag (tre) or lead (dan
truoc) of one signal with respect to the second signal for a given frequency
component.
(7) The cross-spectrum is used to determine the coherence function between two
signals
(8) Cross spectrum and coherence function can be determined by the MATLAB as
powerful engineering program for Digital Signal Processing
7.3. Computation of coherence function

(1) Let x(t) and y(t) be two discrete data measurements of two signals, the problem is
that to determine i) Cross spectrum and auto spectrum and ii) Coherence function
(2) The Fourier transforms of discrete signals

X(f)=F{x(t)}
Y(f)=F{y(t)}
(3) Auto spectrum of x(t) or y(t)

Sxx(f)=X(f) X*(f)
Syy(f)=Y(f) Y*(f)
(4) Cross spectrum of x(t) and y(t)
Sxy(f)=X(f) Y*(f)
Syx(f)=Y(f) X*(f)
(5) Coherence function
S xy ( f )
Coh xy ( f )
S xx ( f ) S yy ( f )
| S xy ( f ) | 2
| Cohxy ( f ) |
(Complex Coherence Function)
S xx ( f ) S yy ( f ) (Magnitude Squared Coherence Function)
Note: Here two discrete signals are taken into account. In cases, many pairs of two
discrete signals are counted, the summation must be used, called the average cross
spectrum and the average coherence function.
Complex cross spectrum and
coherence function consists of two parts: i) magnitude and ii) phase
(6) Some simplified formulas

2
Cohu1u 2
S u1u 2 ( )
( )
S u1 ( ) S u 2 ( )
Cohu1u 2 ( )
S u1u 2 ( )
S u1 ( ) S u 2 ( )
Cohu ,y ( )
S u ,y ( )
S u , 0 ( ) S u , y ( )
: The span-wise coherence
Where: Cou , y ( ) is co-spectrum (real part) Fourier-transformed from

correlation.
7.4. Examples
Example 1: FFT vs. SPECTRUM
signal
spectra by using FFT
20
8
6
10
4
0
2
-10
0.2
0.4
time [t]
signal
0.6
0.8
20
20
40
60
frequency [Hz]
spectrum by using spectrum
25
20
10
15
10
5
-10
0.2
0.4
time [t]
0.6
0.8
Input Signal
20
40
frequency [Hz]
60
Spectrum with FFT
20
8
6
10
4
0
2
-10
0.2
0.4
0.6
Time[s]
Input Signal
0.8
20
60
20
40
Frequency [Hz]
0.5
-0.5
40
62 | Le Thai Hoa - Notes on Stationary

Time[s]
Random Process and Digital Signal Processing
60

Stationary Random Process and Digital Signal Processing - Theory

Transféré par

Informations du document

Copyright

Formats disponibles

Partager ce document

Partager ou intégrer le document

Options de partage

Avez-vous trouvé ce document utile ?

Ce contenu est-il inapproprié ?

Droits d'auteur :

Formats disponibles

Stationary Random Process and Digital Signal Processing - Theory

Transféré par

Droits d'auteur :

Formats disponibles

NOTES ON

STATIONARY RANDOM PROCESS AND

Prepared by Le Thai Hoa

1. STATIONARY RANDOM PROCESS

Random process {xk(t)} = Ensemble of sample function xk(t)

1st sample function

Ensemble (sample records) of random signal

xk (t ) xk (t1 ), xk (t 2 ), xk (t3 )}...xk (t N )} : Discrete sample function

Classification of random processes

1.3. Representation of random process:

1.4. Characteristics of random process

(2) Variance and covariance: Second-order moment

C xy (t ) E[( xk (t ) x (t ))( y k (t ) y (t ))]

C yy ( ) E[( yk (t ) y (t ))( y k (t ) y (t ))]

Note: Zero mean value process xk(t): x (t ) 0

Note: Zero mean value process xk(t): x (t ) 0

Variance 2 x (t ) = Mean Square Cxx(t)

(4) Correlations: Second-order moment

: arbitrary time (time shift or time lag)

Rxx ( ) Rxx (0)

C xx (0) x2 and C yy (0) y2

| Rxy ( ) |2 Rxx (0) * R yy (0)

Rxx ( ) Rxx ( ) and R yy ( ) R yy ( )

(6) Power spectral density (PSD) function in frequency-domain

1.5. Power spectral density (PSD)

(1) Spectra via correlation (by Fourier Transform of correlation)

: Auto-spectral density function

: Auto-spectral density function

Sxx(f), Syy(f), Sxy(f): Two-sided spectra, f[-,]

One-sided spectral densities

S xx ( f ) Rxx ( ) * cos 2f d 2 Rxx ( ) * cos 2f d

Gxx ( f ) 4 Rxx ( ) * cos 2f d ; f[0,]

Rxx ( ) Gxx ( f ) * cos 2f df ; f[0,]

Gxy ( f ) 2 Rxy ( ) * e j 2f d C xy ( f ) iQxy ( f )

| Gxy ( f ) | C xy2 ( f ) Qxy2 ( f )

Rxy ( ) [C xy ( f ) cos 2f iQxy ( f ) sin 2f ]df

(2) Spectra via Fourier transform

Spectral density function:

(1) Correlation coefficient:

| Rxy ( ) |2 Rxx (0) * R yy (0)

| E[( x (t ) x ) * ( y (t ) y )] |2 E[( x (t ) x ) 2 ] * E[( y (t ) y ) 2 ]

xy ( ) 0 : x(t), y(t) Uncorrelated;

xy ( ) 1 : x(t), y(t) Correlated;

(3) Role of coherence function

xy ( f ) (constant-parameter linear systems) can be

[1 xy ( f )] is the portion of mean square value of output y(t) not be contributed by

Note: The role of coherence function

xy ( f ) is similar to the correlation

. In constant-parameter linear systems, the

coherence has some following possibilities:

xy ( f ) 0 : x(t) and y(t) uncorrelated (unrelated)

xy ( f ) 1 : x(t) and y(t) correlated (unrelated)

0 xy ( f ) 1 : x(t) and y(t) some possible situations exist:

C xy ( f ) : Co-spectrum (Real part of Cross-spectrum)

2. DIGITAL SIGNALS AND CLASSIFICATION

Analogue and discrete signals

Almost periodic signals

Branches of deterministic signals

Branches of random signals

Notes and comments:

Almost-periodic signals: can be expressed by the sum of sine functions that

Transient signals: can be defined as totally non-periodic signals (In other

Amplitude distribution: Mean value (expectation) or root mean square

Correlation functions (auto- and cross-correlations) [joint or second moment]