Académique Documents
Professionnel Documents
Culture Documents
(Couch book)
Fig1
Transmitter shapes the message to produce a signal suitable for the channel. (Shape it for the
characteristics of the channel.) Ex Telephony pressure changed to electricity, Digital signals we use
sampling to quantise, compression and encoding, and interleaving.
Channel is the medium used to transmit our signal(pair of wires(twisted pair, coax(having ground
being the shielding), band of frequencies(wireless), light (fiberoptic))
Receiver tries to construct the message.
Sink is the final message(any device that can read the message
Data
Data can be either analog or digital (voice, file)
Capture digital and convert to analogue
Or sample the analogue and convert to digital
Digital data can be either converted to digital or modulated into an analogue signal
Our goal is to transmit the information whilst having minimal deterioration of the signal
Some constraints are the allowable transmission energy(ex you do not want to drain the battery of
a mobile phone, however we would like to use as much power as we want), available bandwidth
(the more bitrate the better the bps is, (the wider the bandwidth also the more noise you capture)),
and cost (to build and to run)
SNR for analogue (the higher the signal above the noise the better (dB))
Probability of bit error in digital (bit error rate), (more of a statistical measure)
Information sent from a digital source when the jth message is transmitted is given by Eqn1
The more something is rare the more it gives you information
Depends on the likelihood of sending a message and not the possible interpretation
Ex1
Since words have the same probability then 10 bit words, thus no compression is possible.
If not equiprobable than some would have more than 10 others less
Ex1continued
Maximum entropy occurs when all the probabilities are equal
Source efficiency is Eq3
Entropy gives us the minimum number of bits needed to encode a symbol, if we go below our
entropy we might not be able to distinguish between the symbols
Properties of Entropy
Entropy is 0 when all probabilities are 0 except one (ex a white image(only white is 1 others are 0))
Data Rate
How fast can we send our bits
Most important part is the bandwidth, the bigger the bandwidth the better
It also depends on the number of levels of the signals, with 2 bits 4 messages, 4bits 16messages
etc less bits but more information
It also depends on the quality of the channel (wireless if afar we need to keep data for longer so
that it is read correctly thus slower bit rate)
Ex2
Assume that we have only signal and white Gaussian noise (white noise is noise where the power
is the same)
Shannon Eqn10
C gives us the limit and we need to keep our rate below C so that the probability of error
approaches 0
This is the ideal case
(Using coding techniques we can reduce the signal to noise ratio (ex CRC), using shielding, higher
power, higher bandwidth, cooling)
Ex3
Ex4
Transmission Impairment
Attenuation, Distortion, Noise
Attenuation is due to resistance, loss in energy along the medium. Thus we use amplifiers Fig2 It
will have automatic gain control
Distortion - our channel will be a bandpass filter Fig3 Propagation speed is different and filter
affects different frequencies differently
Noise - to combat noise we will be applying signal processing and coding techniques, and caused
by Thermal noise, induces noice, crosstalk, and impulse
Aperiodic
Does not exhibit a pattern or cycle that repeats over time Typical of digital signals
We can analyse the signals in the time domain and frequency domain
Deterministic signals can be completely describes mathematically (easily know what the signal will
look like and we can describe it using mathematics)
Non-Causal signal
Has non-zero values in both positive and negative time
Anti-causal
Has zero value for all positive values of time
(Since our channel is a band pass filter in practice we wont have from -infinity to infinity)
We are interested in the magnitude(because we are looking at the signal to noise ratio)
The time can be obtained from the spectrum using the inverse Fourier transform Eqn14
Conjugation Eqn19
Parsevals Theorem
This is given by: Eqn21
This gives an alternative in finding the energy using the frequency domain description
Total normalised energy is the area under the energy spectral density(This gives the normalised
energy so that we can determine)
(typically we want to balance our energy amongst our spectral)
+ or - used as needed
+ Assumes that the function is even
+ Can be verified using the Fourier Transform
Spectrum of a sinusoid
Find the spectrum of a sinusoidal signal having frequency f0 and peak value A
Ex5
Definn=ition
The convolution of a waveform w1(t) with a waveform w2(t) to produce a third waveform w3(t) is:
Eqn40
If discontinuous waveforms are convolved, we can use the equivalent integral
Eqn41
Steps involved
Reverse one of the signals ex w2 giving w2(-lambda)
Time shift w2 by t seconds gives us w2(-(lambda-t))
Multiply the result by w1 to obtain the integrand
Ex6
Definition:
The power spectral density (PSD) for a deterministic waveform is Eqn42
-where wT(t) <> WT(f) and Pw(f) is in Watts/Hz
Notes
-PSD is always a real non negative function of frequency
-PSD is not sensitive to the phase spectrum of w(t)
Autocorrelation function
Definition:
The autocorrelation of a real waveform is Eqn44
Moreover, PSD and autocorrelation function are Fourier transform pairs Eqn45
-Called Weiner-Khintchine theorem
PSD can be calculated by calculating the autocorrelation function and taking the Fourier Transform
Normalised power P=Rw(0)
Orthogonal Functions
Defition:
-Functions Eqn46 are said to be orthogonal with respect to each other over interval a< t < b if
Eqn47
-Moreover, if the functions in set Eqn48 are orthogonal, then they also satisfy Eqn49
Fourier series
A type of orthogonal series
-Functions used are either sinusoids or complex exponential functions
Theorem
-A physical waveform may be represented over the interval a< t < a=T0 by the complex
exponential Fourier series
Eqn 57
-where the complex coefficients are:
Eqn58
and w0 = 2pif0 = 2pi/T0
If w(t) is periodic with period T0, this Fourier representation is valid over all time
Notes
-a is arbitrary(can start forom wherever u need) usually a = 0 pr a =-T0/2
-f0 is fundamental
Nf0 is nth harmonic
C0 is Dc Value
Properties
-If w(t) is real => Eqn59
-If w(t) is real and even (w(t) = w(-t)) => lm[cn] = 0
-If w(t) is real and odd (w(t) = -w(t)) => Re[cn] = 0
-Parsevals Theorem Eqn60
-Complex Fourier series coefficients of a real waveform are related to the quadrature Fourier series
coefficients by Eqn61
-Complex Fourier series coefficients of a real waveform are related to the polar Fourier series
coefficients by Eqn 62
Distortion of signals
Aplitude Distortion
-If response is no flat
Phase distortion
-If phase response is not linear
and fs is a parameter that is assigned a value greater than 0. Moreover, if w(t) is band limited to B
Hz and fs >= 2B, equation becomes the sampling function representation, with Eqn 65 [an = w(n/
fs)]
Minimum sampling rate to reconstruct a band limited waveform without errors Eqn 66
This is called Nyguist frequency
Let us use N sample values to reconstruct a band limited waveform
-Assume we want only the interval symb2
-The sampling function series can be truncated to include only N of the symb1 functions that have
their peaks in symb2. Eqn67
This will produce a weighted sum of time-delayed (sin x) / x waveforms, with [an = w(n/fs)] Eqn68
Fig8
Fig9
(When u some the components together you get something that resembles the original waveforms
The Fast Fourier transform is normally used to reduce the time for computations(Using butterfly
smth smth)
DFT can be used to compute the continuous Fourier transform (therefore we can approx better our
analog continuous signals)
The time waveform w(t) is first windowed to have a finite number of samples, N
Eqn72
The Fourier transform of this new waveform is:
Eqn73
Fig 9
(A window is just a square pulse we are multiplying our sinc with the delta functions)
We can approximate the CFT by using a finite series to represent the integral Eqn74
-Where t= k . delta t, f=n?T, dt =delta t, and delta t = T/N
Thus Eqn75
Aliasing
-Spectrum of sampled waveform consists of replicating spectrum of unsampled signal about
harmonics of fs
If fs <2B, where B is the highest significant frequency in unsampled signal we will have
aliasing errors
-Decreasied with higher fs or resampling low-pass filter
Picket-fence Effect
-N-point DFT canot resolve components any closer than delta = 1/T spacing
-Improved by increasing T
If data length is T0 =< T we can extend T by adding 0s (zero padding). This reduces delta f
Notes:
-Delta f must satisfy Nyguist sampling condition
-T must give desired frequency resolution
-`number of data points N=T/(dleta t)
-DFT can also be used to find the coefficients of the complex Fourier series
Introduction to Modulation
We need to transmit more than one signal at a time. To recognise it from other signals
Introduction
Modulation is a process of adding information (signal), m(t), to a carrier signal
Equivalent representations:
Eqn78
And
Eqn79
Where Ex7
Waveforms are all baseband waveforms and, except for g(t), are all real
v(t) is a low-pass-to-bandpass transformation
Fig11
(To detect the signal from the modulated signal, you need a diode)
Fig12
(The limit is when the amplitude increases there may be an overlap and you will have distortion,
therefore it can only increase till it gets to 0)
If m(t) has a positive peak value of=1 and a peak negative of -1 => signal is 100% modulated
Fig13
(The bandwidth is now doubled since we have 2 of them, therefore in amplitude modulation we
need double the bandwidth)
This gives
Eqn 94
(The 1 is our wanted signal) (The cos(2wc t) is the unwanted and we can filter this out using a low
pass filter)
Fig 15
But Eqn100
Thus
Eqn 101
Photo1
Vestigial Sideband
DSB takes a lot of bandwidth but SSB is expensive to implement
Can be recovered using product detection or envelope detection (if large carrier is used)
Fig16
Angle Modulation
The complex envelope is given by: Eqn 105
Real part = R(t) =|g(t)| = Ac (constant) and phase (t) is a linear function of m(t)
Fig 17
FM Modulation
We have a variation about fc which is directly proportional to m(t)
Fig 18
We have a constant envelope -> constant power level Pav =0.5 Ac2
Note that g(t) is a nonlinear function of m(t) => no general formula for G(f) w.r.t. M(f) exist
Ex8
Where Eqn116
After some maths Eqn117
Jn() is a Bessel function of the first kind of nth order. We can evaluate this using tables or MATLAB
Fig19
Properties:
Jn() =J-n(), n even
Jn() = -J-n(), n odd
For <<0
J0() approx = 1
J1() approx = /2
Jn() approx = 0, n>=2
Note that discrete carrier term (at fc) is proportional to |Jo()| => depends on and fm
Given Eqn120
The expression for the modulated signal becomes Eqn 121
Fig 21
Fig 22
Eqn 126
-Since cos(x) and sin(x) are even an odd functions
Eqn 127
Fig24
(BPSK is better for transmission then OOK)
Differential PSK
PSK signal cannot be detected incoherently
Fig26
(Instead of a low pass filter we can use a matched filter - which gives you maximum signal to noise
ratio (maximum power transfer -shape of filter is closer to the shape of the signal))
Frequency-Shift Keying
Can be generated by switching between two f=different frequencies
- Discontinuous at switching times
Eqn 131
Continous-phase FSK signal is obtained by feeding data signal into a frequency modulator
This gives: Eqn 132 (Even though discontinuous theta of t is continuous since we have an integral)
Multilevel Modulation
We can have more than two modulation levels
We can generate this from a serial binary input stream using a DAC
Fig 28
A plot givers 4 points, one for each value corresponding to the 4 phases that theta can take
These can correspond to 0, 90, 180 and 270 or 45, 135, 225, and 315 (The Volts are represented by
phase)
Fig 29
MPSK
MPSK can be obtained using 2 quadrature carriers modulated by the x and y components of the
complex envelope Eqn 133
Channels
White Noise
Fig31
1/2 indicates that 1/2 of power is on the positive frequency spectrum and the other 1/2 on the
negative
Gaussian Noise
The distribution at any time instant is Gaussian
Fig33
The magnitude is que=antified by the Noise Spectral density, N0 (Can be represented by the spectral
density
Eqn143
Where Eqn144
Eqn 145
Fig 34
Use of a real continuous channel with bandwidth W, noise spectral density N0, and power P is
equivalent to N / T = 2W uses/s
Eb /N0
Assume that we encode the system to transmit binary source bits at a rate R bits/s
We measure the rate-compensated SNR by the ratio of the power per source bit to N0
Eqn 151
Capacity
The capacity for the continuous channel is Eqn152
Analysing this:
- Assume that we have a fixed power constraint
- What is the best bandwidth to use?
- Let W0 = P/N0 (bandwisth for which SNR = 1)
- Then C/W0 = (W/W0)log(1+W0/W))
- Thus capacity increases with W0 log e
- Better to transmit at low SNR over large bandwidth (spare spectrum)
- Otherwise high SNR at narrow bandwidth (most used)
- Constant spectrum is a limited resource and needs sharing
Probability if output is
Eqn 154
- where px is the probability distribution over the input
Fig 35
Fig 36
We obtain a joint probability set XY in which random variables x and y have joint distribution:
Eqn 155
Example
- consider a Binary Symmetric Channel with f = 0.2
- Let Px = {p0 = 0.75, p1 =0.24} (p0 is probability of 0 and p1 is probability of 1)
- Assume we receive a 1
Ex 9
Optimal input distribution of binary symmetric channel is uniform (due to channel symmetry)
Similarly, symmetry exists in the binary erasure channel -> optimal input distribution is uniform
Note:
- In binary symmetric channel, receiver has no knowledge on which bits are flipped
- In binary erasure channel, receiver knows which symbols are erased
- If transmitter is informed we can get the 1-f limit
Eye Diagram
Fig 38
Effect of channel filtering and noise can be seen by observing the received line code on oscilloscope
Multiple sweeps with each sweep triggered by a clock signal and sweep width is slightly larger than
Tb
Assessment of quality
Eqn 156
Eqn 157
Fig37
The power is
Eqn171
(With large SNR yn (noise) is smaller than Ac therefore last term can be neglected since becomes
very small)
Note: Product detector might be better option for Weak AM stations or AM data transmission systems
DSB-SC Systems
This is an AM signal in which the discrete carrier term has been removed
- Corresponding to infinite percent AM
Therefore SNR is
Eqn177
- Noise performance of DSB-SC is the same as baseband signalling systems but twice the
bandwidth -NT = 2B
SSB Systems
IF bandwidth si equal to B
Hilbert transform
Eqn182
Eqn183
Eqn184
PM Systems
Phase detector can be used to recover the modulation signal on a PM signal
Fig40
Complex envelope of PM signal is:
Eqn187
Dp is the phase sensitivity of phase modulator (rad/V)
For large (S/N)in, ThetaT(t) can be approximated using the vector diagram
Fig 41
Shows that unmodulated carrier suppresses the noise at output (quieting effect) when (S/N)in >> 1
PSD of n0 is
Eqn194
- PSD of bandpass input noise is
- No / 2 in passband of IF FIkter
- Zero outside passband
Fig42
But Dp = Betap / Vp
- Betap is the PM index and Vp is the peak value of m(t)
Thus: Eqn201
and noise:
Eqn209
- Assuming (S/N)in >> 1
Because of the derivative component, the PSD of noise is different from PM case
Eqn210
Fig 43
SNR at input is
Eqn214
Output to input SNR relation is:
Eqn215
Definition - Pulse Code Modulation (PCM) is essentially analogue-to-digital conversion where the
information contained in the instantaneous samples of an analogue signal represented by digital
words in a serial bitstream
If the digital words have n binary digits, we will have a set of M = 2n unique code words
- Each code word represents an amplitude level
Note that analogue signal can take an infinite number of levels -> digital code word word
representats a value which is closest to actual sampled value
- Called Quantisation
(more n more bits more accuracy)
Advantages of PCM
Relatively inexpensive digital circuitry
PCM signals coming from any analogue source can be merged with data signals and transmitted on
a common system (using for example TSM)
Fig 44
Quantization
Assuming that we have 8 levels, we get:
Fig 45
Steps in quantizer shown are all equal -> uniform(can be not equal more resolution in some parts)
PCM signal is obtained by encoding each quantised PAM signal into a digital word
We can represent anode samples using digital words that have a different base than 2
- Called multi-level signal
- Advantage: need less bandwidth
- Disadvantage: more complex circuitry
Bandwidth of binary PCM depends on bit rate and waveform pulse shape used for data
representation
Using dimensionality theorem, bandwidth of binary PCM is bounded by: Eqn 219
For example we can select a unipolar NRZ, a polar NRZ, or a bipolar RZ PCM waveform.
- Typical of cheap circuits
The null bandwidth is the reciprocal of pulse width -1 /Tb = R - for binary signalling
Fig47
Fiq 49
(the more levels the better the performance, more dBs but the bandwidth required becomes larger,
accuracy at the expense of bandwidth)
Effects of Noise
Recovered signal will be corrupted by noise
We also need sufficient bandwidth limitations - using anti-aliasing filter - and fast sampling - to have
negligible aliasing of noise
Recall that the bit error rate, Pe, depends on the energy per bit of signal, Eb, and on N0/2
Fig 50
For binary coding, assume PCM code words are related to samples by:
Eqn220
Thus Eqn225
With M = 2n = 2V/:
Eqn226
Thus: Eqn232
But 2n = M
Fig51
Assumption is that peak-to-peak level of analouge waveform, at input of PCM encoder is set to the design level
of quantiser
If peak input exceeds design peak V we get flat-tops near the peak values - overload noise
- Produces unwanted harmonic content
If input level is a relatively small value compared to the design level -> error values become not equally likely
from sample to sample
- Granular noise
- Reduced by increasing levels or non-uniform quantiser(Fig 52)
A near-constant input analogue waveform can generate oscillating sample outputs - hunting noise
- Cause a sinusoidal tone at 1/2 fs
- Reduced by filtering or ensure no vertical step at the constant value
Example(Bk pg 151)
Assume voice over telephone occupies the bandwidth 300 - 3400 Hz. The signal is converted to PCM for
transmission. Let us oversample at 8 ksamples/s. If each sample value has 8 bits, bit rate is:
Thus we need 64kHz to transmit the digital PCM signal. Peak signal-to-quantization noise power ration is:
Fig53
At the receiver, the binary signal plus noise results in a baseband analogue waveform:
Eqn240
Eqn241
r0(t0) is a random variable with a continuous distribution (noise corrupted the signal)
From figure assume that the polarity of the circuits is such that for a clean signal:
r0>= VT gives a binary 1
r0<VT gives a binary 0
The BER is
Eqn 244
Note:
- For bandpass signalling assumption is valid as we have linear filters with some gain
(superheterodyne)
- For bandpass signalling, superheterodyne is linear
- Yet, if we use AGC or limiters, receiver becomes nonlinear(depending on distance from
transmitter)
- If nonlinear detector is used (ex envelope detector), output noise is not Gaussian
Let lambda = -(r0 - s01) / sigma0 in 1st integral and lambda = (r0 - s02) / sigma0 in 2nd
Eqn 251
or
Eqn 252
This means that appropriate selection of VT can reduce the probability of error
To find this we need to evaluate: dPe / dVT = 0 (To find the optimal VT)
Eqn 253
Linear filter which maximises instatneOUS OUTPUT SIGNAL POWER WHEN COMPARED TO
THE AVERAGE NOISE POWER (SIGMA02 = N02(T)) IS MATCHED FILTER
Definition:
- The matched filter is a linear filter that maximises (S/N)out = s02(t0)/n02(t) and has a transfer
function given by EQn260
- where S(f) is the Fourier transform of the known input signal s(t) which has duration T seconds.
Pn(f) is the PSD of the input noise, t0 is the sampling time, and K ids an arbitrary real non-zero
constant
Therefore, for binary signalling corrupted by white Gaussian noise, the BER is: Eqn 265
- This is valid for matched-filter reception and using the optimum threshold
Fig56
Matched filter in processing circuits has to match the filtered wave shapes
Eqn 267
The signal output from the pre-whitening filter will spread beyond the T second interval
Effects can be reduced if duration of original signalling intervals is made less than T so that
spreading will remain within T
Unipolar Signalling
Fig57
Eqn 269
Let us assume that we have a low-pass filter with unity gain (H(f))
Fig 58
Polar Signalling
Fig 58
Bipolar Signalling
Fig59
For NRZ
- Binary 1s are given by alternating positive and negative values Eqn 278
- Binary 0s are given by a zero level Eqn 279
This gives:
Eqn281
Using calculus we find the optimum VT for minimum BER to. be: Eqn282
For bipolar RZ, Ed=A2T/4=2Eb => BER formula remains the same
On-Off Keying
The filter at the receiver can either be a low-pass filter or a matched filter
Binary-Phase-Shift Keying
BPSK is an antipodal signal with Eqn 297
Fig 63
Optimum threshold is VT = 0
Eqn 300
Case 2 - Matched filter
The energy in the difference signal at receiver input is: Eqn 301
`Note:
- Performance is same as for baseband polar signalling
- Is 3dB better than OOK
Frequency-Shift Keying
Coherent detection using two product detectors
LPF must act like a dual bandpass filter - one entered at f1 and the other at f2, with Bp = 2B
or Eqn 311
Consider 2deltaF = f1 - f2 = n/(2T) = nR/2
- Integral term goes to zero
- Required for s1(t) and s2(t) to be orthogonal
Notes
- Performance is equivalent to OOK (matched)
- 3dB below BPSK
However, circuits to implement the receivers are simpler than the coherent ones
On-Off Keying
Fig 65
Eqn 316
n(t) is a Gaussian process -> in-phase baseband component A + x(t) is also Gaussian process
- But mean is A not zero!
The integral containing the Bessel function cannot be evaluated in closed form
Since A / sigma >> 1, the integrand is negligible for values of r0 in the neighbourhood of A
- Lower limit can thus be replaced by -infinity and r0 / 2 pi simga2 A by 1/ 2 pi sigma2 Eqn 324
BER becomes
Eqn 325
Eqn 326
Eqn 329
- Average bit energy is Eb = A2T / 4, sigma2 = N0Bp
- R = 1/T is the bit rate of OOK signal
Fig 67
Filter bandwidth = Bp
Given this symmetry and the noise of upper and lower channels is similar => VT = 0
PDF of r0(t) conditioned on s1 and that on s2 are similar f(r0 | s1) = f(-r0 | s2)
r0(t) is positive when the channel output vU(t) is larger than vL(t)
When receiving a zero plus noise, output pf upper bandpass filter is only Gaussian noise
- In this case, output of upper envelope detector vU is noise that has a Rayleigh distribution Eqn
337
- where sigma2 = N0 Bp
VL ha s a Rician distribution
- We now have a sinusoid plus noise, thus:
Eqn 338
We now obtain:
Eqn 339
Notess:
- OOK and FSK are equivalent on an Eb/N0 basis
- For error performance, non-coherent FSK needs ~ 1dB more Eb/N0 than for coherent FSK for Pe
<= 10-4
- In practice, most FSK receivers are non-coherent
Fig 68
Assumptions
- Additive input noise is white and Gaussian
- Phase perturbation varies slowly (ensure constant phase reference)
- Transmitter carrier oscillator is stable
BER of suboptimal demodulator for large SNR and BT > 2/T is:
Eqn341
- For typical BT and Eb/N0 (in range BT = 3/T and Eb/N0 = 10, BER is approx. Eqn 342
- Therefore, the performance is similar to OOK and FSK
Comparing performance of BPSK and DPSK with optimum demodulation -> for same Pe, DPSK
needs ~ 1dB more Eb/N0 than BPSK (Pe 10-4)
Quadrature PSK
QPSK is a multilevel signalling technique
GPSK signal is
Eqn 344
A factor is the bit data (one for cosine and one for sine)
Note that for same bit rate R, bandwidths of BPSK and QPSK signals are NOT the same
- QPSK has half the bandwidth of BPSK for a given R
Minimum-Shift Keying
MSK is equivalent to QPSK, except that dat on x(t) and y(t) quadrate modulation components are
offset and equivalent data pulses shape is a positive part of a cosine function (not rectangular)
- PSD rolls off faster!
Optimal receiver is similar to QPSK but matched filter with cosine pulse shape is needed
Comparison
Fig71
(Polar is the best, Unipolar i the worst and Bipolar is in the middle, for Baseband)
However in some cases, simple formulas for upper bounds on the probability of symbol error can
be found
In low error conditions (Pe < 10-3), error symbol selected is usually nearest neighbour on
constellation (If higher it would go out of neghbourhoud)
Synchronization
We have 3 levels of synchronisation in digital communications systems
- Bit synchronization
- Frame or Word synchronisation
- Carrier synchronisation (oscillator with carrier frequency)
Bit synchronization is needed for clocking the sample-and-hold and the matched filter
All BER results assume that bit sync and carrier sync are noise-free
- If these are noise we have larger Pes
Frame or word sync is required by some systems to remark the serial data into digital words or
bytes
In other systems, block coding or convolutional coding is used. Frame allows for error detection
and correction
(performance curves will be given in the exams and the equations would be in them)
Source Coding
(How we are representing the data before we send it)
(Try to reduce the size of the data since we have high sampling rate to make it digital)
Introduction
Source data needs to be represented with good fidelity and sampling rate
Solution: Coding
- Reduce channel rate required
- Introduces some levels of loss and distortion
- Ruled by Entropy
(We remove any unnecessary data, looks into perceived content)
(sometimes we need lossless compression, which looks into statistical redundancies, example
when we compress a document)
Fig 74
Source coder is needed to transform the source information into a coded sequence whose values
are obtained from a code alphabet (known from both ends of the comms medium)
Similarly the decoder will estimate the source signal from the received coded sequence
- This may contain transmission errors
Fig 75
If symbols do not have the same probability of occurrence, we can assign different code lengths to
the symbols
- Shorter codes for the more frequent ones
- Longer codes to the rare symbols
Basic steps:
- Change representation
- Quantization
- Code assignment
Fig76
Huffman Coding
Consider the following example:
- Let symbols A, B. C, D, E, F have probabilities of transmission of 0.3, 0.2, 0.1, 0.15, 0.05, 0.2
- Using simple binary or Grey code assignment:
The algorithm construct the prefix code starting from the last bits of the least probable symbols
- Arrange source symbols in descending order of probability
- Create a new source with one less symbol by combining the last two codewords and assign two
bits
- Add the two probabilities to replace the previous two
- Select the two lowest probabilities in thew new list and combine, agin assign two bits
- Repearte this combination and bit assignment until probability is 1
- Encode each original signal into the binary sequence generated by the combinations, with the
first combination being the LSB
Fig 77
Note:
Eqn352
lfixed=3
Eqn353
HIHuff (If lower than we have some code words which are unrecognisable)
Run-length Coding
Simple scheme that converts a string of repeated characters into one with fewer characters
Fig 79
Golomb Coding
We can potentially have an infinite number of symbols (Therefore Hoffman coding this is hard)
n = qm + r is represented using:
Fig 81
Example:
- Take m = 6
- Let us encode the following symbols
Fig 82
How to choose m?
Eqn 355
Using graphs
Assuming m = 5
Fig 83
Input Output
Ooooo 1
Oooo1 O111
Ooo1 O11o
Oo1 O1o
O1 Oo1
1 Ooo
Input:
Fig84
Eqn 356
Eqn 357
Notes:
- Useful when one symbol is more likely than the others
- Need to find the best order (training or learning)
Error Resilience
(We are going to reintroduce redundancy, we need to correct errors using statistics by
reintroducing redundancy in the bit steam, example repeating the data/sending it two times)
Introduction
Communications over a channel is prone to errors
Can be done by
- Automatic Repeat Request
- Need a feedback channel
- When parity errors are detected, receiver requests transmitter to resend the data block
- Forward Error Correction
- No feedback channel
- Data is encoded such that receiver can detect and correct errors
- Adds redundancy to the data stream
Classified into
- Block codes
- Maps k input binary symbols into n output binary symbols
- Coder is a meatless system
- n > k, such as adding parity bits
- Denoted by (n, k) and has a code rate of R = k / n
- Convolution codes
- Accepts k binary symbols and outputs n binary symbols
- N depends on v + k inputs -> needs memory
- R=k/n
Definitions
Hamming weight of code word is the number of 1 bits in the word
- Example: 1011011 has a Hamming weight of 5
Hamming distance is the number of bit positions in which two code words differ
- Denoted by: d
- Example: for code words 101101 and 110111, d = 3
Some errors can be detected and corrected if d s + t + 1, s is the number of detectable errors
and t is the number that can. be corrected.
Block Codes
General code word is: i1i2i3 ikp1p2p3 pr
- k is the number of information bits
- r is the number of parity bits
- N = k + ris the total length of the code (n, k)
More parity bits => more redundancy => more detection and correction capability => more
bandwidth necessary!
Linear Block Codes
We use modulo 2 arithmetic on {0, 1}
- 0 + 0 = 0, 0 + 1 = 1 + 0 = 1, 1 + 1 = 0
- 0*0=0, 0*1 = 1*0 = 0, 1*1 = 1
If C is an (n, k) code, there exists code words g0,g1, , gk-1 that form a basis for the code
considered as a vector space over F2
Therefore an (n, k) linear code has 2k code words, where a code word represents 1 of 2k
messages
Associated with a block code generator G we have a matrix H called the parity check matrix
Example 1
Sometimes it is fit to have the original data explicitly in the code word
When G is systematic,
- H = In-k | -PT], for binary codes H = [In-k | PT]
Theorem
- Let a linear block code C have a parity check matrix H. The minimum distance C is equal to the
smallest positive number of columns of H which are linearly dependent
Proof
- Let columns of H be d0, d1, , dn-1, then
- C0d0 + c1d1 + +cn-1dn-1 = 0
- Let c be a codeword of smallest weight, w = w(c) (kirsty)
Thus the bound on the distance of a code is: dmin < n-k+1
- This is because H has n-k linearly independent rows
- It follows that any combination of n-k+1 column of H must be linearly independent
Hamming Code
Block code having distance of 3
Givent that d >= 2t + 1 => t = 1, a single error can be detected and corrected; up to d-1 can be
detected
Maximise P(r | c)
Receiver has to estimate the most likely code word and most likely error
The syndrome s = rH = eH
- This limits the possible error patterns
- Using linear algebra, a solution e1 gives a set of possible solutions S = e1 + C := {e1 + c}
- Maximum likelihood decoder estimates . Closest to r
- Equivalent to finding with smallest Hamming weight in S
Using linear algebra, we can show that there isa one-to-one connection between
co-sets and syndromes
If we have more than one vector with minimum weight, sel=ection of co-set leader
is done arbitrarily from these
To decode
- Compute the syndrome
- Identify co-set leader
- Estimate = r +
Ex 7
Notes:
- When decoding we only need first and last columns
- If redundancies are large we get a huge array - impractical
Decoder will make correct decision iff the actual error pattern is a co-set leader
For a binary symmetric channel we can easily find the probability of code word error
Eqn 360
- p is the crossover probability
A block code detects an error iff e is not itself a code word other than the all zero
one.
Thus:
Eqn 362
Cyclic codes
Ex 8
BCH Codes
Definition:
- Let n be a divisor od 2p - 1, pis an integer >= 3
- The finite field containing F2 has size 2p (Galois Field (2p))
- Let gamma be a primitive element of GF(2p) and Eqn
- Alpha has order n
- A t-error correction BCH code is the set of all binary n-tuples that lie in the null-
space of the (t x n) matrix
Note: If N denotes the null-space of the matrix over GF(2p), then BCH code C N
F2n.
BCH Codes
Example
- Single error correcting BCH having n = 15 ha g(kirsty)
(continued on paper)