Performance of Non-Parametric Spectral Estimators

KanagaRagav KanagaSubramanian, UFID: 31118749

Abstract
Non-parametric Spectral Estimators do not need any prior information about the signal and hence can be implemented entirely from the data alone. The ubiquitous problem of bias-variance tradeoff calls for the choice of one over the other depending on the application. This problem is analyzed in this report for different signals and the performances of them are compared in terms of their tradeoffs. Simulation examples that back up well established theories have been shown here.

Table of Contents
1) Introduction 2) Non-parametric Spectral Estimators: Periodogram and Modified Periodogram Blackmann-Tukey method Bartlett and Welch methods Refined Filter Bank method Capon Estimator 3) Analysis and Simulation Examples: Effect of Data Length on Peaky Spectra. Bias analysis of Peaky Spectra. Bias and Variance analysis of Broadband spectra. Summary of Inferences. 4) Conclusion and Future work 5) Acknowledgement 6) Reference

Introduction
Non-parametric spectral estimation methods need no apriori knowledge of the data. While the accuracy of estimation improves if we use the right model in parametric spectral estimation, in situations where there are no known statistics of the data beforehand non-parametric spectral estimation methods are the ones which are sought after. All the non-parametric methods aim to reduce the variance at the cost of resolution (bias) and viceversa. The traditional methods are based on the Fourier transform of the data. They include the Periodogram, modified Periodogram, Bartlett and Welch methods. These methods can also be interpreted interms of a filterbank approach for viewing the spectrum [1]. This kind of thinking gives rise to advanced methods such as the Refined Filter Bank, Capon and other class of Filter Bank Methods [1][2]. The report is organized as follows. In Section.2 a brief review of the theory of the methods used in this report is done. In Section.3 simulation examples that help to visualize the methods are carried out. Further study on these topics is motivated in Section.4.

Non parametric Spectral Estimators

Periodogram and Modified Periodogram:
The Periodogram was the earliest of spectral estimators introduced by Schuster in 1898. It can be defined as follows: ( )= where: | | (i) is the spectral estimate.

data signal; N total number of samples of the signal;

This definition of the Periodogram can be shown to be biased and is unbiased asymptotically [1]. The Periodogram can be shown to be the convolution of the original spectrum and a (sinc)2 function (The spectrum of a triangular(Bartlett) window). This leads to smearing of the original spectrum which causes resolution problems. This is addressed in part.1 of Section.3 with a simulation example. Also the sidelobes of the Bartlett window causes frequency content to be leaked to other frequencies. Both smearing and leakage effects contribute to the bias of the spectral estimator. To mitigate the leakage, a smoother data window that tapers smoothly can be used to for time-limiting the data. This gives rise to the modified Periodogram which can be defined as ( )= | |

where w(n) is the window function. The Fourier transform of all the other windows have a main lobe width greater than of that the Bartlett window. Therefore they cause a further loss of resolution as compared to the unmodified Periodogram. Both the Periodogram and modified Periodogram suffer from the disadvantage that their variance does not reduce even when the length of the data is increased. Therefore, other methods which decrease the variance, but at the cost of resolution has been proposed.

Blackmann-Tukey method:
An equivalent formula similar to (i) for estimating the power spectrum is given by: = This form is called the Correlogram. defined as:
= =
(

( )

(ii)

is called as the estimate of the signals auto-correlation function, is

=0 = 1

1 ( 1)

This form of autocorrelation estimate can be shown to be biased. But however since this estimator guarantees a non-negative spectral density we prefer this rather than the biased version of the estimate.

The Blackmann-Tukey (BT) method windows the autocorrelation function with a windowing function (called the lag window) and then computes the PSD (Power Spectral Density) using (ii). This provides us a control to reduce the variance in the frequency domain, but however windowing in time domain leads to added loss of resolution by smearing the spectrum and hence more bias in the frequency domain.

Bartlett and Welch Methods:

To reduce the variance of the Periodogram, the available data can be split up into K no. of smaller segments, (each segment containing M samples of the original data) their Periodogram can be computed and then averaged. It is proved in [3] that the reduction in variance of the Bartlett method as compared to the Periodogram is given by: ( ) = (iii) ( ) However since the number of data points in each segment available to compute the spectrum is reduced, the resolution of the method is also reduced by the same factor K. The Bartlett method on the other hand is not superior to the BT method in terms of reduction in variance (if only the first M lags of the autocorrelation estimate are used in the computation of the BT method). The Welch method is similar to Bartlett method by way of splitting the original data into segments but it also allows overlap between the segments and does a windowing on each segment. This leads to larger lags being weighed less, a effect of which reduces the variance as compared to the Bartlett method.

Refined Filter Bank Method:

In order to reduce the variance it is necessary to average the PSD over a band of finite interval. If this is followed the problem of spectral estimation can be restated into the problem estimating the power in narrow frequency bands [1]. Fig.1 above [3] explains the statement. It is necessary to divide by the bandwidth of the filter for normalization. Also all the parametric methods can be viewed as a variation of the filter bank approach. The PSD as shown in [1] is given as: = (iv)

where: h(t) is the filters co-efficients, N is the number of data samples and y(t) is the output of the filter. We look into the problem of finding the ideal BandPassFilter (BPF) that satisfies the objective: To maximize the amount of signal energy in the band of interest subject to the condition that the filter has a flat-top unity gain response in the passband. It has been shown in [1] that a filter that obeys these conditions can be found as the Slepian Sequences. If K Slepian Sequences are used on the data it is equivalent to increasing the bandwidth of the BPF K times and a single BPF being replaced by parallelizing a number of BPFs, equal to the number of Slepian Sequences used, and whose total bandwidth covers the bandwidth of the original single BPF. Therefore the PSD is computed by averaging the filters outputs. It is given by the formula: = (v)

As a final note, more the number of Slepian sequences used, lesser is the resolution and the variance. [1] provides more information on the same.

Capon Estimator:
The BPF in the filterbank approach should be designed as selective as possible to the input data. This is the Capon method and is explained in [4], [1]. The BPF filter for the Capon Method is given by the FIR filter h = [h0.h1.hm] and the co-efficients are equal to: =
( )

( )

(vi)

( ) ( ) ( ) = (vii) ( ) Where N is the number of data points, and a(w) is given by, = [1 ] The choice of m (time width of Capon filter) is important. The maximum value it can take is N/2. It is to be noted that larger is m, more number of terms get averaged in (vii) and hence more stable is the R matrix which in turn leads to lesser variance. The power spectral density is then given by =

( )

(viii)

where denotes the frequency bandwidth of the filter Capon filter and is taken to be 1/(m+1), by the rule of time-bandwidth prouct[1].

Analysis and Simulation Examples

Effect of Data Length on Peaky Spectra:
The Periodogram, on the average, of the finite sample data (0, 1 M-1) can be shown as the convolution of the original spectrum with a (sinc)2 function (spectrum of a triangular window). This causes the smearing of the original spectrum leading to resolution problems. To demonstrate this we consider the following signal: x[n] = cos(0.35*pi*n) + cos(0.40*pi*n) + 0.25 * cos(0.8*pi*n) + 0.1*v(n) ; n=0,1,. . ., M-1.

Thus x(n) is a sum of three sinusoids located at normalized frequencies of 0.35, 0.4 and 0.8 and embedded in v(n), which is a zero mean, white Gaussian noise. The width of the main lobe of the triangular window is twice that of the rectangular window is given by (3.62*pi)/M. To resolve the 2 peaks at Normalized frequencies 0.35 and 0.40 therefore the length of the data should obey the equation: 3.62 72.4. where =0.40-0.35=0.05*pi. Substituting this value in the above equation, we get The above example was carried out using a simulation for M=15, 30, 37 and 81. The following is the graph of the mean Periodogram after 50 Monte Carlo Simulations:

Fig.1: Variation of resolution with data length of a Periodogram.

It is clearly seen that for M=15, 30 the peaks are not distinguishable and are better after M=37. Beyond M=81, there was not much improvement in the separation of the peaks and hence M=81 was chosen as the number of data points needed for spectral resolution. This value is also in accordance with the required data length as shown above.

Bias analysis of Peaky Spectra:

In this section, bias analysis of estimators is carried out. The same data which was used in the previous section is also used here. 50 Monte Carlo simulations of each estimator is carried out. There are 81 data points. Since the data is peaky, ideally the spectral estimate should be peaked towards the location of sinusoids. Hence in this case, we can decide that if more is the smearing, lesser are the peaks resolved, and more is the bias.

Periodogram and Modified Periodogram:

As explained in the previous section, the true spectrum gets convolved with a (sinc)2 in the case of Periodogram. In the modified Periodogram, peaks in the periodogram have been suppressed by the data windowing operation; This enhancement is clearly at the expense of the frequency resolution (or smearing of the true peaks), which is to be expected [3].

Fig.2: Periodogram and Modified Periodogram

Blackmann-Tukey Method:
Here the autocorrelation function is windowed to reduce the variance. Since our criteria here is resolution, we use a window with the smallest ML width Hann window. It is to be noted that since the function is estimated with lesser and lesser data points at larger lags towards the end of the data, the variance would be more for them as compared to the estimate evaluated using lesser lags. Hence as a corollary, estimates found using autocorrelation function of lesser lags though being statistically stable increase the bias of estimation. This can be shown as in the below figure:

Fig.3: BT method for different lengths of the autocorrelation function.

Bartlett and Welch Methods:

Fig.4: Bartlett and Welch Methods with varying parameters

Bartlett method with just 2 segments each of 40 data points was able to resolve the two closely spaced peaks better than BT method, and also the peak at 0.8 was seen with the same amplitude as the BT. Hence in terms of bias, Bartlett method with 2 segments provided lesser bias when compared with BT estimator. If any further attempt to increase the no. of segments lead to reduction in resolution. To show the advantage of windowing provided by the Welch Method with 2 segments and a single sample overlap curve is plotted. The sidelobes reduce considerably but the bias is increased. More overlap produces statistically stable spectra at the expense of increased bias. This is also showed in the graph above.

Refined Filter Bank (RFB) method:

It is shown in [1] and as given by eqn.(v) that more the number of Sleppian Sequences used, more is the averaging across neighboring frequencies in the frequency domain and hence more is the smearing and hence in this case more is the bias. This is verified by the below graph. Also it is shown that for K=1, the behavior is more or less similar to the Periodogram methods.

Fig.5: RFB method with different Slepian Sequences.

Capon Method:
Since this method estimates the spectrum by taking into account the signal statistics this ultimately performs the best for peaky spectra in terms of bias. Even though the spectrum gets separated the peak energy of the sinusoids is reduced as compared to the other methods and in particular the sinusoid at f0=0.8 is heavily attenuated. This reduction in the peak amplitude is because the Capon filters not at these frequencies try to attenuate the sinusoids and hence an overall reduction in the peaks occurs. However since the separation between the closely spaced peaks is greater than 20dB, it should serve our purpose of identifying them. Also as the time aperture m of the filter increases (frequency bandwidth becomes smaller) bias starts to reduce at the expense of variance. The maximum value that m can take is N (=81) divided by 2 [1].

Fig.6: Capon Method for different time apertures.

Bias and Variance Analysis of Broadband spectra:

A broad band spectrum was generated using the equation: = + 0.55 1 + 0.15 2 = 0,1,2, . . 1

where e(n) is a unit variance zero mean Gaussian noise. 256 data samples were taken and the spectrum was realized using a 1024 point FFT in all cases. 1000 Iterations was done as a part of Monte-Carlo Analysis. What interests us here more in this example is we wish to analyze the variance properties of the estimators. First, we show the mean spectrum of all the estimators as below. They converge to the true mean. But it is surprising to see that Capon is a biased estimator and it has been proved in [5] that Capon is biased downwards and this is the only shortcoming of the estimator for practical purposes. Also since the signal is a broadband signal, we want the frequency bandwidth to be less sharper which would be attained for lower values of filter order m. And hence the filter with lesser m has lower bias as compared with the filter with higher m. This is shown in Fig.10 clearly. The variance of all the estimators is then plotted below.

Fig.7: Periodogram, Modified Periodogram and BT

Fig.8: Bartlett and Welch Methods

Fig.9: RFB Method

Fig.10: Capon Method

The powers in dB of the fist standard deviation of all the estimators are also shown.

Fig.11: Variance of Estimators

It is noted that the Capons method produced the minimum variance. It also has to be stated that BT and Welch reached variances lesser than the Capon method, however the bias produced by the estimators in such conditions would make them less preferable. BT method was also experimented with other windows. But again

given the optimal tradeoff for the signal at hand, Capon performed the best. Another observation about the Capon estimator is that lower the time aperture, according to the equation (viii), better would be the estimate of R and hence more stable would be the estimate. This has been clearly seen in the below graph.

Fig.12: Effect of time aperture versus variance of the estimator

Final Inferences:
Capon performs the best for both peaky spectra where bias (resolution) was important and for wideband spectrum (though biased) where minimum variance was desired. For peaky spectra the next choices are the Periodogram and equally good is the RFB method. Bartlett and Welch methods have to be tuned for the situation at hand and then used.

Conclusion and Future Work

The Capon being the best is still biased. It is shown in [5] that a better estimator would be APES. Therefore this study can be extended to cover the scope of APES also. Further these could be tested on narrowband spectrum and also in the presence of increased noise power.

Acknowledgement
The author would like to thank Dr. Jian Li, Electrical and Computer Engineering, University of Florida for being the chief source of knowledge and also for the constant encouragement and motivation.

References
[1] [2] [3] [4] [5]

P Stoica and R Moses, SPECTRAL ANALYSIS OF SIGNALS, Prentice Hall, NJ, 2005 High-Resolution Nonparametric Spectral Analysis: Theory And Applications - Chapter.4 Erik G. Larsson and Jian Li, University of Florida Petre Stoica, Uppsala University Dimitris G.Manolakis, Vinay K.Ingle, Stephen M.Kogon, "Stastical and Adaptive Signal Processing Artech House, ch. 5, pp 194-254, 2005. J. Capon, High resolution frequency wave number spectrum analysis, Proc. IEEE 57 (1969) 1408 1418. P. Stoica, A. Jakobsson, and J. Li, Matched- Filterbank Interpretation of Some Spectral Estimators, Signal Processing, 66(1):4559, April 1998.