Using Radial Basis Probabilistic Neural Network
for Speech Recognition
Nima Yousefian, Morteza Analoui
Computer Department, Iran University of Science &Technology
Tehran, Iran
ni_yousefiyan@comp.iust.ac.ir, analoui@iust.ac.ir
Abstract— Automatic speech recognition (ASR) has been a
subject of active research in the last few decades. In this paper
we study the applicability of a special model of radial basis
probabilistic neural networks (RBPNN) as a classifier for
speech recognition. This type of network is a combination of
Radial Basis Function (RBF) and Probabilistic Neural
Network (PNN) that applies characteristics of both networks
and finally uses a competitive function for computing final
result. The proposed network has been tested on Persian one
digit numbers dataset and produced significantly lower
recognition error rate in comparison with other common
pattern classifiers. All of classifiers use Mel-scale Frequency
Cepstrum Coefficients (MFCC) and a special type of
Perceptual Linear Predictive (PLP) as their features for
classification. Results show that for our proposed network the
MFCC features yield better performance compared to PLP.
I. INTRODUCTION
A major problem in speech recognition system is the
decision of the suitable feature set which can accurately
describe in an abstract way the original highly redundant
speech signal. In non-metric spectral analysis,
mel-frequency cepstral coefficients (MFCC) are one of
the most popular spectral features in ASR. In parametric
spectral analysis, the LPC mel-cepstrum based on an all-
pole model is widely used because of its simplicity in
computation and high efficiency [1].
Another popular speech feature representation is known
as RASTA-PLP, an acronym for Relative Spectral
Transform - Perceptual Linear Prediction [2]. PLP was
originally proposed by Hynek Hermansky as a way of
warping spectra to minimize the differences between
speakers while preserving the important speech
information. RASTA is a separate technique that applies a
band-pass filter to the energy in each frequency sub-band
in order to smooth over short-term noise variations and to
remove any constant offset resulting from static spectral
coloration in the speech channel for example from a
telephone line[3]. RASTA-PLP outperforms PLP for
recognition of channel-distorted speech.
Many pattern classifiers have been proposed for speech
recognition. During the last several years, Gaussian
Mixture Models (GMMs) became very popular in Speech
Recognition systems and have proven to perform very
well for clean wideband speech [4]. However, in noisy
environment or for noisy band-limited telephone speech
the performance of GMM degrades considerably.
Another well known contemporary classification
technique, Vector Quantization (VQ) is tested with our
dataset and showed absolutely high recognition rate
between other classifiers. So we decided to design a new
probabilistic neural network (PNN) that uses a
competitive function for its transfer function such as VQ
networks. Results show that this network overcomes all
other pattern classifiers in recognition of Persian one digit
numbers dataset.
PNNs are known to have good generalization properties
and are trained faster than the back propagation ANNs.
The faster training is achieved at the cost of an increased
complexity and higher computational and memory
requirements [5].
II. SYSTEM CONCEPT
A. Dataset
A sequence of 10 isolated digits (0, 1, 2, …, 9) Voices
from 35 different speakers were recorded in Computer
Department of Iran University of Science and
Technology. All of voices saved as wave files with audio
sample size as 16 bit and audio sample rate 16 KHZ with
mono channel. So there are 350 wave files. We divided
them into two separate parts, 20 speakers (200 wave files)
for training and 15 remaining speakers (150 wave files)
for testing. So the ratio of train to test is 4:3.
B. Features Extraction
The goal of feature extraction is to represent speech
signal by a finite number of measures of the signal. This
is because the entirety of the information in the acoustic
signal is too much to process, and not all of the
information is relevant for specific tasks. In present ASR
systems, the approach of feature extraction has generally
been to find a representation that is relatively stable for
different examples of the same speech sound, despite
differences in the speaker or various environmental
characteristics, while keeping the part that represents the
message in the speech signal relatively intact.
 In this paper two well known methods are used. For
calculating MFCC features each speech signal is divided
into four equal length signals and 12 order coefficients for
each signal are computed by using 16 KHz as sampling
rate and hamming window and 0.5 for high end of highest
filter as a fraction of sampling rate. So 48 feature vectors
for each signal contribute in training and test phase.
A more sophisticated technique is the RASTA-PLP
approach. RASTA basically consists of additional steps
applied after the critical-band integration of PLP. Based
on an observation that human perception is more sensitive
to relative changes, the static or slow-changing part of the
signal (in each critical-band) is effectively filtered out.
Because most distortions due to unknown channel
characteristics are convolute, RASTA is best implemented
in a logarithmic domain of the spectrum, which makes the
different parts additive. Same as MFCC for calculating
RASTA-PLP coefficients, signal is separated into four
signals. Again 16 KHz sampling rate applied for
calculating 12 order coefficients of RASTA-PLP.
For some classifiers adding first and second order time
derivatives of coefficient results in better performance
than pure PLP [6]. But for calculating derivatives we
should be aware to take shorter window than normal. So
144 (12*4*3) feature vectors for each signal contribute in
training and test phase. This method is RASTA-Δ2 PLP.
C. Recognition Methodology
In multi-class mode such as our case, each classifier
tries to identify whether the set of input feature vectors,
derived from the current signal, belongs to a specific class
of numbers or not, and to which class exactly. For
samples that can not be realized as a specific class a
random class is selected.
III. CLASSIFIERS
Several classifiers are tested for mentioned dataset. The
structures of successful classifiers in recognition are
described in following subsections.
A. Gaussian Mixture Models
1) Structure: Gaussian mixture models (GMMs) are
one of the semi-parametric techniques for estimating
probability density functions (pdf) [7]. The output of a
Gaussian mixture model is the weighted sum of R
component densities, as shown in Fig.1. The training of
GMM can be formulated as maximum likelihood problem
where mean vectors, covariance matrices and prior
probabilities are estimated by Expectation-Maximization
(EM) algorithm. Given a set of N independent and
identically distributed patterns, Xi={ x(t) ; t=1,2, …, N}
associated with class ωi , it assumed that the class
likelihood function p(x(t) | ωi) for class ωi is a mixture of
p(x(t) | ωi)
Σ
p (θ1 | i | ωi) p(θ R | i | ωi)
…
p(x(t) | ωi , θ1 | i) p(x(t) | ωi , θR | i )
x(t)
Figure1: Architecture of GMM
Gaussian distributions, which denoted by:
R
p( x(t) | ωi ) = ∑P(θ r | i | ωi ) p( x(t) | ωi ,θr | i ) (1)
r =1
where θ r | i represents the parameters of the rth mixture
component, R is the total number of mixture component.
p( x(t ) | ωi , θ r | i ) = N ( μ r | i , Σ r | i ) (2)
typically N( μ r | i , Σ r | i ) is a Gaussian distribution that has
mean μ r | i and covariance Σ r | i .last equation denotes
probability density function of the rth component and
finally P (θ r | i | ω i ) is the prior probability of the rth
component.
2) Effect of parameters in recognition: Here we
assume number of mixtures R=3. Increasing this
parameter dose not has significant effect on performance.
The method of initializing mean and variance is
important. After examining various methods, k-harmonic
initialization method produced best results in comparison
to k-means and randomly selected data points algorithms.
The number of iterations set to 10 or it will stop when the
increase in log likelihood falls below 0.001.
In our work in the training phase, GMM usually
recognizes all samples with no mistake so recognition rate
is nearly 1, which overcomes all other classifiers in this
case. GMM is one of the classifiers in our work that gets
better result with RASTA-PLP than MFCC.
 Figure 2: Architecture of LVQ
B. Learning Vector Quantization
1) Structure: LVQ or Learning Vector Quantization
is a prototype-based supervised classification algorithm.
LVQ can be understood as a special case of an artificial
neural network, more precisely; it applies a winner-take-
all Hebbian learning-based approach. It is a precursor to
Self-organizing maps (SOM) and related to the k Nearest
Neighbor algorithm (k-NN) [8].
An LVQ network has a first competitive layer and a
second linear layer. The competitive layer learns to
classify input vectors in much the same way as the
competitive layers of Self-Organizing Nets. The linear
layer transforms the competitive layer's classes into
target classifications defined by the user. The network is
given by prototypes W= (w (i),..., w(n)). It changes the
weights of the network in order to classify the data
correctly. For each data point the prototype (neuron) that
is closest to it, is determined (called the winner neuron).
The weights of the connections to this neuron are then
adapted, i.e. made closer if it correctly classifies the data
point or made less similar if it incorrectly classifies it.
For a test sample that LVQ can not relate it to any class a
random class is considered. An advantage of LVQ is that
it creates prototypes that are easy to interpret for experts
in the field.
2) Effect of parameters in recognition: There are
various types of LVQ. In this work LVQ1 is applied for
classification. The number of neurons in first layer set
to number of training speakers. It is better to set learning
rate near zero, increasing it has unexpected influence in
accuracy of recognition. For initializations of the weights
LVQ should know the prior class Percentages, as the
same probability for each 10 classes we set this
parameter to 0.1 for all classes. Total number of epochs
for training is 150.Increasing it has no improvement in
performance of recognition.
C. Other Classifiers
Some other neural networks such as RBF, ARTMAP,
and MLP have been tested with our dataset. All of these
networks earned accuracy rate below 70 % in the test
phase. But in RBF network if we modify transfer
function of layer two with special linear layer and set
layer weights as training data great growth in
performance occurs. This is one of the points that lead us
to proposed probabilistic neural network in this study.
K-means is a clustering method. Here is a definition of
a special model of supervised k-means, which can
produce accuracy result for training and test phase. If a
random sample from each of 10 classes been selected and
use them as initialization seed points for k-means we can
train data and get accuracy recognition rate for these
data. If number of selecting random seed points repeats
more, recognition rate may improve. After getting best
performance in training, related seed points considered as
centers for calculating a test sample distances to these
centers separately. So for each test signal Euclidean
distance between its coefficients with all ten centers
coefficients is calculated and the center with minimum
distance is considered as the class of test sample.
IV) COMPETITIVE RADIAL BASIS PNN
As described in previous section LVQ and GMM have
best recognition rate over all other classifiers. This point
result in designing a probabilistic neural network that
uses Gaussian distribution as its probability kernel
function of network and a competitive function as its
transfer function like first layer of LVQ. In following
subsections, network procedure and architecture will be
described in details.
 Radial Basis Layer
Input
IW 1,1
Q*R
P
||dist|| n1
R*1 Radbas
Q*1
1 b1
R Q*1
Figure 3: Architecture of proposed radial basis PNN
R=number of elements in input data
Q=number of input / target Pairs=number of neurons in layer1
L=number of classes of input data=number of neurons in layer 2
A. Procedure
The radial basis probabilistic neural network (RBPNN)
model is in substance developed from the radial basis
function neural network (RBFNN) and the probabilistic
neural network. Therefore, the RBPNN possesses the
common characteristic of the original two networks, i.e.,
the signal is concurrently feed-forwarded from the input
layer to the output layer without any feedback
connections. On the other hand, the RBPNN, to some
extent, decreases the two original models demerits.
When an input is presented, the first layer computes
distances from the input vector to the training input
vectors, and produces a vector whose elements indicate
how close the input is to a training input. The second
layer sums these contributions for each class of inputs to
produce as its net output a vector of probabilities.
Finally, a competitive transfer function on the output of
the second layer picks the maximum of these
probabilities, and produces a 1 for that class and a 0 for
the other classes.
B. Network Architecture
Illustration of the network architecture is in figure 3. It
is assumed that there are Q input vector/target vector
pairs. Each target vector has L elements. One of these
elements is 1 and others are 0. Thus, each input vector is
associated with one of L classes. The first-layer input
weights, IW1,1 are set to the transpose of the matrix
formed from the Q training pairs, P'. This is exactly what
mentioned in previous section for growing performance
of RBF. When an input is presented the ||dist|| function
Competitive Layer
1 n2 L*1
a
LW 2,1 C
Q*1 L *1
a 2= y
L*Q
produces a vector whose elements indicate how close the
input is to the vectors of the training set. These elements
are multiplied, element by element, by the bias and sent
to the Radbas transfer function. Radbas function is
defined with following equation:
−n2
Radbas (n) = e (3)
An input vector close to a training vector is represented
by a number close to 1 in the output vector a1. If an input
is close to several training vectors of a single class, it is
represented by several elements of a1 that are close to 1.
ai1 =Radbas ( || IWi 1,1 – P|| bi1) (4)
The second-layer weights, LW2,1 are set to the matrix T
of target vectors. Each vector has a 1 only in the row
associated with that particular class of input, and 0's
elsewhere. The multiplication Ta1 sums the elements of
a1 due to each of the L input classes. Finally, the second-
layer transfer function, compete, produces a 1
corresponding to the largest element of n2, and 0's
elsewhere.
a 2 = compet ( LW 2,1 a1 ) (5)
where compet is a transfer function. compet (N) takes
one input argument, a matrix of net input vectors and
returns output vectors with 1 where each net input vector
has its maximum value, and 0 elsewhere.
Thus, the network has classified the input vector into a
specific one of L classes because that class had the
maximum probability of being correct. So each input
certainly has a class as output.
 V) EXPERIMENTS and RESULTS
A. Classifiers Comparison
As mentioned in previous sections 350 wave files
containing voices of 35 speakers divided into 2 parts,
train and test with ratio of 4:3. All of speakers are male.
Here 4 classifiers with best performances are selected.
We name the edited version of k-means that introduced
in third section edited k-means or Ek-means. The result
from this classifier is desirable and nearly better than
GMM.
In the training nearly all of described classifiers
recognized training patterns with accuracy above 95%.
Best performance belongs to GMM with accuracy near 1
in this phase.
In the test phase each classifier has been tested with
each of feature vectors described before. Performance of
different classifiers in the test phase for all of the three
introduced feature types can be observed in table 1.
Obviously RBPNN with MFCC gets best result
between all other methods. But it is noticeable that for all
classifiers adding time derivations of PLP to itself has
not improvement effect in recognition rates. Ek-means
like GMM and LVQ produce better outputs with PLP
than MFCC. The proposed method is the only one that
gets better accuracy with MFCC.
As we described in feature extraction subsection
original signal is divided into four parts. This is because
of average of numbers of phoneme in numbers from 1 to
10 in Persian is about four. But some classifiers produce
better output by setting this factor to three. So in table1
overall performance for each classifier is calculated in
best situation.
Table1: Performance Comparison (%)
Features MFCC RASTA-
PLP
RASTA-
Δ2 PLP
RBPNN 99.1 95.3 90.1
LVQ 96.7 97.3 86.3
GMM 90.8 91.7 87.2
Ek-means 94.3 96.8 78.3
B. Capability of Proposed Method
There is something remarkable about accuracy of this
version of RBPNN. If all of the three other classifiers
compared in table1 combine to make a single classifier
with majority vote approach, it will gain accuracy about
98.2% which is still about 1% less than proposed
competitive RBPNN.
Some methods have been proposed for improving
performance of RBPNN. Recursive Orthogonal Least
Squares Algorithms combined with Micro-genetic
Algorithms (μ-GA) were tested in [9] for optimizing and
improvement in results of spirals classification problem
and IRIS Classification problem. Results show that, after
optimization, quality of network grows and it is certainly
much better than traditional PNN and RBF networks.
Such approaches can be considered in future works.
ACKNOWLEDGMENT
The authors would like to thank Dr. A.Akbari for his
permission to use wave files as dataset. The authors
would also like to thank the reviewers for their comments
that greatly improved the manuscript.
REFRENCES
[1] Harosha Matsumoto, Masanora Moroto, “Evaluation of MEL-LPC
cepstrum in a large vocabulary continuous speech recognition”,
Acoustics, Speech, and Signal Processing Proceedings IEEE
International Conference on Volume 1, Issue, 2001, pp. 117-120, 2001.
[2] Edric Gaudard , Guillermo Aradilla ,” Speech Recognition based on
Template Matching and Phone Posterior probabilities” , IDIAP-Com
07-02, 2007.
[3] H. Hermansky, ”Perceptual Linear Predictive (PLP) Analysis of
Speech.”, The Journal of the Acoustical Society of America, Volume
87, Issue 4, pp.1738-1752 ,April 1990.
[4] Todor Ganchev, nastasios,Tsopanoglou, Nikos Fakotakis, George
Kokkinakis, “Probabilistic neural networks combined with GMMs for
speaker recognition over telephone channels”, 14 International
Conference on Digital Signal , Volume II, pp.1081-1084 ,July 2002.
[5] Todor Ganchev, Dimitris K. Tasoulis, Michael N. Vrahatis,
“Locally Recurrent Probabilistic Neural Network for Text-Independent
Speaker Verification”, in Proc. of the EuroSpeech, 2003, vol. 3, pp.
762-766, 2003.
[6] Qi Li, Frank K. Soong, Olivier Siohan, “An Auditory System-based
Feature for Robust Speech Recognition”, 7th European Conference on
Speech Communication and Technology, Aalborg-Denmark, pp. 619-
621, September 2001.
[7] K.K. Yiu, M.W. Mak, C.K. Li, “Gaussian mixture models and
probabilistic decision-based neural network for pattern classification: A
comparative Study”, Neural Computing and Applications 8(3): pp. 235-
245, 1999.
[8] Kosaka, T, Omatu, S, “Classification of the Italian Lira using the
LVQ method”, Systems, Man, and Cybernetics, 2000 IEEE
International Conference on Volume 4, pp. 2769 - 2774, 2000
[9] Wenbo Zhao, De-Shuang Huang, Lin Guo, “Optimizing Radial
Basis Probabilistic Neural Networks Using Recursive Orthogonal Least
Squares Algorithms Combined with Micro-Genetic Algorithms”,
Neural Networks, 2003. Proceedings of the International Joint
Conference on Volume 3, pp. 2277 - 2282, July 2003
[10] Leszek Rutkowski, “Adaptive Probabilistic Neural Networks for
Pattern Classification in Time-Varying Environment”, IEEE
TRANSACTIONS ON NEURAL NETWORKS, VOL. 15, NO.4, pp.
811-827, 2004.
[11] J.Barry Gomm, Ding Li Yu, “Selecting Radial Basis Function
Network Centers with Recursive Orthogonal Least Squares Training”,
IEEE Trans. Neural Networks, vol.lI,N0.2 , March 2000.
[12] F. Gorunescu, “Architecture of probabilistic neural networks:
estimating the adjustable smoothing factor,” Research Notes in
Artificial Intelligence and Digital Communications, 104, pp. 56-62,
2004.