Overview of finite precision numerical effects in implementing systems:

Finite precision means limiting the number of bits (used to represent a number) to a finite value. This results in some effects on the behavior of
digital systems. These are called finite precision numerical effecrs.
Quantization:
There are two types of quantization:
If there is a signal x[n], quantization modifies this value into a pre-defined set of allowed values. Let the new value be denoted Q(x[n]). The
difference between the actual value and the quantized value is called quantization error, denoted as e[n].
e[n] = Q(x[n]) x[n]
Below it is assumed that x[n] is truncated or rounded to a representation that uses B bits. 2s complement representation is also
assumed.
1)

Truncation
Limits of e[n]

2(B +1) <e [n]2(B+1 )

2(B +1)

Mean value of e[n] =

22 B
12

Variance of e[n] =

2)

Rounding
B

Limits of e[n]

2 <e [n]0

Mean value of e[n] = 0

Variance of e[n] =

22 B
12

There are three kinds of quantization errors: input quantization error, product quantization error, coefficient quantization error.
Input quantization error:
Input quantization error results from the analog-to-digital conversion process.
Propagation of input quantization error to output of digital filter
Because of quantization error in input to digital filter, there will be error in the output also.
Mean of output quantization error = (mean of input quantization error) x (sum of impulse response coefficients of digital filter)
Variance of output quantization error = (variance of input quantization error) x (sum of magnitude squared of impulse response coefficients of
digital filter)
Product quantization error:
If the inputs to multiplier are represented using B bits, then the product will have length 2B bits. But if the multiplier uses B bit output data line,
then the result of the multiplication will have to be quantized to B bits, the least significant B bits are discarded using either truncation or
rounding. This results in quantization error, ths is called product quantization error.
Diagram: Error Model for a multiplier

Diagram: Unquantized model

Diagram: Quantized model

Propagation of product quantization error to output of digital filter:

The product quantization error in each multiplier will result in a quantization error in the output of the digital filter.

If there are totally P multipliers in a filter, the variance (energy) of total product quantization error (noise) is

P 22 B
12

Variance of the output quantization error = (variance of product quantization error) x (sum of magnitude squared of impulse response coefficients
of the part of the network between the multiplier and the output)

Coefficient quantization error:

Filter coefficients also need to be quantized to a finite number of bits. A filter may have designed for a required specification, and the required
values of filter coefficients may have been designed. But when the coefficients are quantized, this results in a difference in behavior between the
desired filter and the actual filter. The poles and zeros of the actual filter will be different from those of the desired filter. This causes errors in the
frequency domain behavior of the filter. Poles and zeros move to new positions in the z-plane.

Actual System function

System function with quantized coefficients

Each root of the numerator and denominator is affected by coefficient quantization error. Hence, each pole and zero is affected by all the
coefficient quantization errors in the numerator and denominator respectively.
Cascade form implementation and Parallel form implementation are less affected by coefficient quantization errors than direct form
implementation.

Limit cycles:
For a stable IIR system, if input becomes zero and remains zero after a certain time, then usually the output also will gradually decay towards
zero. However, if the filter is implemented using finite precision, the output may not decay to zero even if the signal goes to zero. Instead, the
output may start to have an oscillatory behavior. This behavior is called limit cycle behavior, and the oscillations are called limit cycles.
There are two causes for limit cycles:
1)
2)

Quantization
Overflow in adder

Limit cycles occur only in IIR systems, since there is feedback only in IIR systems. FIR systems have no feedback and hence limit cycles do not
occur in FIR systems. This is one advantage of FIR systems.

In digital filters, overflow may occur at certain internal nodes such as outputs of adders. Overflow can be prevented by multiplying the signals by
a scaling factor so that the amplitude is reduced. This is called scaling.