Food Measure DOI 10.

1007/s11694-012-9129-9

ORIGINAL PAPER

Classication of dates varieties and effect of motion blurring on standardized moment features
Gabriel Thomas A. Manickavasagan R. Al-Yahyai

Received: 30 July 2012 / Accepted: 11 October 2012 Springer Science+Business Media New York 2012

Abstract Computer vision technology has been used as a successful non-destructive quality assessment tool for various food products. In general, several features are extracted from the images of interest, and used for the classication models. Furthermore, in most of the studies, static images have been used in the calibration and evaluation models. Classication models with a reduced number of features, and a mechanism to test the capability of the algorithm for moving objects by means of simulating the blurring effect on the static images would be benecial to determine the performance of the system in real-time quality monitoring in industries. Using three date varieties as model food, motion was simulated for the dates images and a successful neural network classier was designed with only three statistical features (mean, standard deviation, and skewness). The reduced number of features and simplicity of the classier yielded a solution that can be potentially implemented in hardware fast enough so that to consider the case of classication of the dates in a conveyor

belt. To test the solution under such conditions, a blurring degradation function was used to verify that the classier would work. The effects that motion blurring causes to these statistical moments in a general sense were examined using random numbers drawn from the distribution in the Pearson system. Because motion blurring showed a tendency to change the distribution to a Gaussian density, the same features and classier yielded similar results despite of motion. Keywords Classication Neural network Bayes classier Pearson random numbers Statistical moment Image motion

Introduction In computer vision (CV) technology, objects are imaged and analyzed to characterize their quality. This technique has great potential to be used as a non-destructive and objective quality measurement method. It is a reliable technique for the measurement of various quality attributes of agricultural and food products [14]. Attribute characterization using images taken while the objects are at static could be implemented at quality control laboratories. However, the efciency of the developed classication models for the objects moving on a conveyor at various speeds would be highly benecial to determine their ability for online quality monitoring in the real-time production facilities in food industries. Therefore the objective of this study was to determine the classication effects caused by simulated motion of date varieties with three grayscale features at static and motion blurred conditions. Motion was simulated via low pass ltering which system response is usually calculated in order to eliminate motion bluring

G. Thomas (&) Department Electrical and Computer Engineering, Faculty of Engineering, University of Manitoba, E3-555 Engineering and Information Technology Complex, Winnipeg, MB R3T5V6, Canada e-mail: Gabriel.Thomas@ad.umanitoba.ca; thomasg@cc.umanitoba.ca A. Manickavasagan Department of Soils, Water and Agricultural Engineering, College of Agricultural and Marine Sciences, Sultan Qaboos University, P.O. Box 34, 123 Al-Khoud, Sultanate of Oman R. Al-Yahyai Department of Crop Science, College of Agricultural and Marine Sciences, Sultan Qaboos University, P.O. Box 34, 123 Al-Khoud, Sultanate of Oman

123

G. Thomas et al.

using deconvolution techniques. Different methods propose alternative models for the low pass impulse response of the system and the expression that corresponds to linear motion in one direction as dened in [12] is used here; a conveyor belt motion scenario reduces the model to this case only. Dates are used as model images and this technique can be used as such in other products. Date is an important commodity in Oman, and around 50 % of the cultivable lands are under date palm vegetation. Although annual production of dates in Oman is 255,891 Mt, only 9,000 Mt (2.53.5 % of production) is exported due to various reasons [5]. Quality assurance has always been a major problem for Omani dates to compete in international markets [6]. Varietal purity, color, uniformity of size, and absence of defects are some of the important quality parameters for dates in domestic and international market. Generally, manual grading of dates is followed in handling and processing facilities to identify date varieties. This method has many constraints such as subjectivity, inuence of mental stress, inuence of environment, efciency of individuals at various times of the shift, and so on. An automated variety identication method using CV technique would be benecial to the date industries in Oman which would correspond to a rst step to assess the quality by avoiding contamination from different varieties. Materials and methods Image acquisition Three date varieties namely Khalas, Fard, and Madina were used in this study. Samples for each variety were obtained from at least three shops in Oman, and the varietal purity was conrmed by a date variety expert at Sultan Qaboos University. A conglomerate sample of 108 dates was taken for each variety (n = 108 for each variety) and the sample was imaged (single date images) with a color camera (model: D3, Nikon, Japan, Resolution4,256 9 2,832 pixels). The date samples were illuminated with halogen lamp (Visa tech, model SOLO 1600B) during imaging. Then all images were converted into gray scale images using Matlab software, and analyzed. Feature selection As the background of these images was deemed to have little information regarding the classication of each date, a thresholding operation was done in order to eliminate it based on the maximization of the between-class variance. The method is well known as Otsus method [7] and it is briey described next.

Starting by considering an image with background class dened as wb, and object in class wo, with probabilities P P Pwb T1 f zi and Pwo L f zi where i1 iT z denotes discrete image intensity, f(zi), i = 1, 2, , L is the corresponding histogram normalized to have area equal to one, and L is the number of distinct intensity levels; the threshold T is found by maximizing r2 T P B pwb lb ltot 2 pwo ltot 2 where lb T1 zi f zi , i1 P P lo L zi f zi and ltot L zi f zi . T i1 Statistical features in the form of standardized moments were to be considered [8]: cn ln Ez mn rn rn 1

where E is the expected value, r the standard deviation and ln is the nth moment about the mean. For n = 3 one can nd skewness that measures the asymmetry of the probability density function of the random variable associated to the image values. For n = 4 the value is known as kurtosis which also evaluates the shape of a distribution by quantifying its peakness. Because processing speed was important, out of the mean, standard deviation, skewness, and kurtosis, we decided to work with only three of them which would eliminate approximately 25 % of the computations. Then, for feature evaluation, a selection scheme with a backward sequential search starting with a full feature set and sequentially removing features was considered to be a practical solution for the decision of what features to work with. We decided to use a new feature selection scheme that is based on fuzzy entropy measures with a similarity classier as presented by Luukka [9], which is briey described next. The method starts by forming an ideal vector vi = (v1(f1), , vi(ft)) that represents the class I as good as possible by calculating the mean values of available vectors in each class vi(fi) for i = 1, 2, , t features. Afterwards, the similarities Shx; vi between the sample x to be classied and the ideal vectors v need to be calculated as: Shx; vi
t 1X 1 jxfr vfr j t r1

In order to calculate the relevance of the features, fuzzy entropy values are calculated with similarity values lA xj as suggested by DeLuca and Termini [10]:  n X lA xj log l xj A H 1 A 3 1 lA xj log 1 lA xj j1 where low entropy indicates high similarity values and high entropy values are obtained otherwise.

123

Dates varieties and effect of motion blurring

Similarly, fuzzy entropy as suggested by Parkash et al. [11] was used in this work which follows the expression:  n X plA xj p 1 l A xj sin 1 H 2 A sin 2 2 j1 4 Effect of motion To validate the use of the classier in motion conditions, a simulation was conducted where the blurring degradation caused by motion in one direction was implemented in the frequency domain as [12]: H u; v Te sin puaejpua pua 5
Fig. 1 Gray scale images of date samples (rst row Fard, second row Khalas, third row Madina)

where u and v are normalized frequency samples, Te is the camera exposure time, and a is the rate of movement in the x axis direction: xo(t) = at/T. When t = T the image has been displaced by a total distance a.

120

Khalas (blue) Fard (green) Madina (red)

Mean

Results and discussion

Value

110 100 90 80 70 60 0 20 40 60 80 100

Evaluation of features Figure 1 illustrates three samples of each variety of dates. The standardized moments were calculated only within the pixels located within the date and not the background and were computed for one date image at a time. From the small image sample, it can be inferred that features such as the mean value would work for the class Fard, since they look darker than the other two classes. Figure 2 shows the mean values m for 108 images of each class calculated with the expression mean m = ltot. Note how in Fig. 2 the values for class Fard do appear darker and dissimilar than the other two. We tested the four features (mean, standard deviation, skewness, and kurtosis) and evaluated them according to Eqs. (3) and (4). The selection process consistently eliminated the values of kurtosis. With this result, the next section elaborates on the design of the classier using only the mean, standard deviation, and skewness as inputs. Classier selection and classication results As described in the previous section, three features were to be used in the nal classier. Figure 3 shows a scatter plot of the features and albeit three clusters are visible, there is still some overlapping. A Lillilifors test [13] of the default null hypothesis that the values of the different features come from a Gaussian distribution passed the test conrming the Gaussianity of the clusters. Afterwards, a Bayes

Sample Index

Fig. 2 Average gray level values of date samples

classier for Gaussian pattern classes under the condition of a 01 loss function was implemented forming three decision functions dj(x) for j = 1, 2, 3 of the form [12]:   dj x ln P wj h 1=2lnCj  T i 1=2 x mj C1 x mj 6 j where mj = Ej{x} are the means of the feature grouped in vector x, and Cj = Ej{(x2mj)(x2mj)T}. This classier was tested using 80 samples for training and 28 samples for testing. A 65.48 % of correct classications were obtained this way, and in particular there were only two misclassication for the class Madina yielding a 92.86 % of accuracy for this variety. When kurtosis was used instead of skewness, a performance of 35.71 % was obtained. This conrmed that the selection made by Luukkas approach was correct.

123

G. Thomas et al.
0.8 0.6

Khalas Madina Fard

MSE

0.4 0.2

Train Test

skewness

2 0 -2 35 30 25 80 100 120

10

15

20

25

Epoch

mean

standard deviation 20
15

60

Fig. 4 Mean squared error (MSE) using a two layer neural network with ve hidden neurons. Matlab denes MSE as the measurement of the networks performance according to the mean of squared classication errors

Fig. 3 Sctatter plot of the selected features for date samples

Table 1 Mean squared error (MSE) obtained while using a two layer neural network with different number of hidden neurons No. of neurons n=3 MSE (%) Time (s) 12.73 1.0915 n=5 11.04 1.0130 n = 10 12.27 1.0031 n = 15 13.62 1.0441 n = 20 13.79 1.0711

than 0.025 s using an Asus Ee Slate tablet computer with an Intel i5 1.33 GHz processor and 4 GB of RAM memory. At this point the specied classication efciency of more than 90 % was achieved with a simple neural network architecture to allow the option of a future implementation in hardware rather than in a computer based software solution. Thus, the solution was considered to be fast enough to actually be implemented in a scenario where a conveyor belt with continuous motion could be used. Effect of motion

At this point, a two layer neural network was used with hyperbolic tangent sigmoid transfer functions using Matlab as the software platform for development. The training algorithm used was the scaled conjugate gradient. In order to dene the number of hidden neurons N, neural networks were trained 30 times with different numbers of hidden neurons. The initial weights are calculated randomly by Matlab and these averages would give an idea of the best number of hidden neurons to be used. Because only three features are used, the expectation was that the network would not need many neurons. Table 1 shows the average results after training the different networks 30 times with respect to mean squared error (MSE) as well as time required for training. Because of the small numbers of neurons used, the training time was very short for all of them. From these results, it was decided that the number of hidden neurons to use was ve, not only because it had the minimum MSE value in Table 1 but also because of the computational savings of not having to compute more multiplications once the network had to perform classications. Figure 4 shows the performance obtained using such a network. The total number of epochs needed was 23, for each class 65 out of 108 images were used for training, best performance was obtained with MSE values of 3.81 % for testing, 9.7 % for training and a needed training time of 0.78 s. Once trained, classications were performed in less

Figure 5 shows blurring simulated using Eq. (5) for a = 20 for the images shown in Fig. 1. Because a bright illumination set up was contemplated so that to eliminate any possible shadows of the dates in bulk, faster shutting camera times are expected and the blurring in Fig. 5 was considered an extreme case. In order to have an idea of the effects this blurring will cause to the features, 5,000 images of size 256 9 256 formed by random numbers drawn from the distribution in the Pearson system with specied l, r, c3, and c4, were generated and blurred for different values of a. Figure 6 shows how these features changed as the blurring progresses. As the degradation function in (5) is a low pass lter, blurring can be seen as a weighted averaging and no changes in mean values are expected. The slight changes in Fig. 6a correspond to the darkening of the beginning and nal values of the blurred images caused by discrete implementation of the lter that viewed in the discrete time domain, convolution values have this windowing effects. The standard deviation is reduced as expected after low pass ltering and skewness tend to go to zero, making the samples more Gaussian as what we would see by averaging random samples and explained by the central limit theorem. We used these features using Bayes classiers in order to identify the image blurring effects on classication.

123

Dates varieties and effect of motion blurring

0.7 0.65

correct classification

0.6 0.55 0.5 0.45 0.4 0.35 0 5 10 15 20 kurtosis skewness

motion rate a

Fig. 7 Bayes classier results for different motion rates Fig. 5 Motion blurred images of date samples (rst row Fard, second row Khalas, third row Madina)
0.45 0.4 0.35 0.3 a=4.8495 a=9.899 a=14.9495 a=20

(a)
mean

129 128 127 126

MSE
0 5 10 15 20

0.25 0.2 0.15 0.1

(b)
standard deviation

6 4 2 0

0.05 0 0 10 20 30 40 50 60

Epoch

Fig. 8 Mean squared error (MSE) obtained while using a two layer neural network with blurred images (results of test images)
0 5 10 15 20

(c)
skeewnes

0.1

0.05

10

15

20

motion value a

Fig. 6 Results of motion blurring on selected features of date samples

Results with accuracies similar to the ones found in Pydipati [14] for detection of citrus disease were found using a simpler neural network and reduced number of features. Because of this simplicity, real time classication under motion blurring caused by examining the dates on a conveyor belt was deemed feasibly. Thus, we developed a methodology to introduce motion via a degradation function implemented in the frequency domain to conrm the robustness of the proposed method.
Acknowledgments We thank The Research Council (TRC) of Sultanate of Oman for funding this study (Project No. RC/AGR/ SWAE/11/01-Development of Computer Vision Technology for Quality Assessment of Dates in Oman).

Figure 7 shows the results when using both skewness and kurtosis. As it can be seen, motion blurring at these rates did not cause any major differences and it was further conrmed that rejecting the kurtosis feature was correct as indicated by the feature selection section. Similar two layer networks as the one used in Fig. 4 were used for different values of a, and Fig. 8 shows the consistent good results.

References
1. A. Manickavasagan, G. Sathya, D.S. Jayas, N.D.G. White, J. Cereal Sci. 47, 518 (2008)

123

G. Thomas et al. 2. S. Mahesh, D.S. Jayas, J. Paliwal, N.D.G. White, Sens. Instrum. Food Qual. Saf. 5, 1 (2011) 3. M.I. Mladenov, S.M. Penchev, M.P. Dejanov, M.S. Mustafa, Sens. Instrum. Food Qual. Saf. 5, 111 (2011) 4. S.R. Delwiche, M.S. Kim, Y. Dong, Sens. Instrum. Food Qual. Saf. 5, 63 (2011) 5. FAO Statistics (2007), http://faostat.fao.org/site/342/default.aspx. Accessed 24 May 2012 6. A.S. Al-Marshudi, Tropicultura 20, 203 (2002) 7. N. Otsu, IEEE Trans. Syst. Man Cybernet. 9, 62 (1979) 8. J.F. Kenney, E.S. Keeping, Mathematics of Statistics, Pt. 1, 3rd edn (Princeton, NJ, 1962) 9. P. Luukka, Expert Syst. Appl. 38, 4600 (2011) 10. A. DeLuca, S. Termini, Inform. Cont. 20, 301 (1971) 11. O.M. Parkash, P.K. Sharma, Mahajan, Inf. Sci. 178, 2389 (2008) 12. R.C. Gonzalez, R.E. Woods, Digital Image Processing,, 2nd edn. (Prentice Hall, NJ, 2001) 13. H.W. Lilliefors, J. Am. Stat. Assoc. 62, 399 (1967) 14. R. Pydipati, T.F. Burks, W.S. Lee, Trans. ASAE 48, 2007 (2005)

123