The Online Journal on Power and Energy Engineering (OJPEE) Vol. (1) No.

(4)
Reference Number: W09-0040 125
CCDRice Grain Positioning Algorithm for Color
Sorter with Line CCD Camera
Lei Yan, Sang-Ryong Lee, Seung-Han Yang, Choon-Young Lee
Department of Mechanical Engineering, Kyungpook National University, Daegu, Korea
Abstract- Positioning is one of the most important
operations in color sorting systems for falling grain which
directly affects the performance of these systems. Line-
scan Charge Coupled Device (CCD) image sensor is
widely used in the color sorting systems, however, the
processing algorithms are much complex. In this paper, a
complete set of algorithms is presented for the above
systems. First, the Line-Scan Merging (LSM) algorithm, a
novel connected component labeling algorithm that can be
applied on line scan CCD imaging system, is proposed.
The end points of objects will be given by LSM. Second,
the direct least squares ellipse-fitting (direct ellipse-fitting
for short) algorithm is applied to position the single grain
with the end points of it from the result of LSM. Finally,
the direct ellipse-fitting algorithm is modified to position
the connected grains to solve the problem of difficulty in
segmenting the connected objects and accurately
positioning them. The experimental results show that the
proposed algorithms can be successful in positioning the
falling grain.
Keywords- Color Sorter, Line Scan Camera,
Mechatronics, Image Processing.
I. INTRODUCTION
For industrial inspection, machine vision systems have
been successfully implemented to improve the quality of
products. There are many applications such as fabric
inspection [6], web inspection [16][19], inspection of printed
circuit boards [10][20], quality inspection of food products
[2][5][18], gap detection [12], wafer surface inspection [11],
defect detection in textured materials [9] and other industrial
fields that demand for quality control of their
products[1][13][14][15][21][22].
A typical machine vision system will consist of several
components: One or more digital image sensors with suitable
optics for acquiring images; Cam-era interface for digitizing
images (widely known as a "frame grabber"); a processor
(such as a DSP) ; Suitable, often very specialized, light
sources (LED illuminators, fluorescent or halogen lamps)
etc.. For the digital image sensors, the area array sensors with
two-dimensional arrays of pixels in varying resolutions, are
widely used in the consumer digital cameras and parts
inspection, gauging in the industry. They excel in imaging
still or slow-moving objects. However, using area arrays to
image high-speed motion of objects or continuous webs is
problematic. It is harder and more expensive to get good
images because area arrays need short exposure times as well
as shuttering, synchronization equipment, or powerful,
strobed illumination, all of which add to systems complexity.
In contrast, line scan sensors can be well-used on imaging
high-speed objects or webs, and they can work well with low
illumination. With a single line of pixels, line scan sensors
build continuous images not limited in their vertical
resolution. Based on these reasons, line scan sensors are
suitably used in the machine vision systems that require high
real-time performance.
The color sorter, one of the machine vision systems, has
been developed and applied throughout the world in last
several decades. The color sorter studied in this paper is
applied to remove the foreign materials from the good grains,
especially the rice grains according to the different colors.
Fig. 1 shows the working principle of the color sorter.
Figure (1): Working principle of a color sorter
The grain samples are fed into input hopper automatically.
Then, via the in-feed vibrator, the grains are fed onto a flat
and channeled gravity chutes. The surface of the chutes is
smooth enough to reduce the force of friction between the
grains and the chutes. The grains are then passed into an
optical inspection area, where a decision on whether to accept
or reject each grain was made.
This paper is organized as follows. In the next section,
Line-Scan Merging algorithm is proposed, followed by the
description of direct ellipse-fitting algorithm. Subsequently
the modified ellipse fitting algorithm is presented. The
experimental results are then reported and our conclusions are
drawn finally.
II. LINE-SCAN MERGING ALGORITHM
The meaningful information of an image (that all pixels
belonging to connected components are assigned a unique
The Online Journal on Power and Energy Engineering (OJPEE) Vol. (1) No. (4)
Reference Number: W09-0040 126
label) can be extracted with applying the labeling operation.
Labeling of connected components in an image is one of the
most fundamental operations [4], which is required in almost
all image-based applications such as fingerprint identification,
automated inspection, target recognition, handwriting
recognition, face identification and medical image analysis.
Usually the labeling algorithms can be divided into two-scan
or raster scan[7][8], multi-scan and trace type [3], which are
not useful in the applications where the whole image is not
stored in the RAM, so one-scan labeling algorithms are
always demanded.
This Line-Scan Merging algorithm is a novel one-scan
algorithm for labeling connected compo-nents. The
advantages of this algorithm can be summarized in the
following four items. First, this algorithm can be directly used
on the digital imaging system with line scan CCD sensor.
Second, it scans the whole image only once which makes it
suitable for real-time system. Third, it can directly work on
gray image, unlike conventional algorithms that can only deal
with binary images, which needs to convert the image into a
binary one before labeling. Finally, it is easy and convenient
to compute shape characteristics (such as area, center position
and aspect ratio etc.) and other post-processing with the
output information of LSM.
Figure (2): Image processing to find end points
Definitions
Temporary Object: The continuous connected components
in the scanning line are recorded as one temporary object.
Existing Object: All of the connected components with the
same existing label consist of one existing object.
Final Object: The existing object which has finished being
labeled is named as the final object.
In this algorithm, two sets of labels are used - the
temporary label and the existing label.
Properties
Property 1. If there is no existing object that is connected
with the temporary one, it will be recognized as the new
existing object and recorded with a new existing label.
Property 2. If there is no temporary object that is connected
with the existing one after scanning one row, then the existing
one is done labeling.
Implementation
Fig. 2(a) is the original image of rice grain. The red end
points (Fig. 2(b)) show the scanning line. The black end
points show the end points of existing object which has been
scanned. Fig. 3 shows the flowchart of the detailed process.
Figure (3): The flowchart of line-scan merging algorithm
III. DIRECT ELLIPSE-FITTING ALGORITHM
From the processing result of LSM, the coordinates of end
points of objects are stored as the representation of those
grains. Next, the direct ellipse-fitting algorithm is applied.
A general conic can be described by an implicit second-
order polynomial:
2 2
( , ) 0 F x y ax bxy cy dx ey f = + + + + + = (1)
And
( , )
i i
F x y
is called the algebraic distance of a point pi =
(xi, yi) to the conic
( , ) 0 F x y =
. Eq.(1) can be represented in
vectorial form:
, ) , 0 F = = k x x k (2)
where
j
T
a b c d e f = k
and
2 2
1 x xy y x y =
]
x
are the
vectors of the coefficients and the coordinates of the data
points on the conic section respectively.
Fitting of a general conic to a set of points pi, i = 1, 2 m
can be approached by minimizing the sum of squared
algebraic distances of the points to the conic which is
represented by
2
1
( , )
m
i i
i
E F x y
=
=

(3)
which can be reformulated in vectorial form as
2
|| || E = Dk
(4)
where the design matrix D of the size 6 m ,
2 2
1 1 1 1 1 1
2 2
2 2
1
1
1
i i i i i i
m m m m m m
x x y y x y
x x y y x y
x x y y x y

(
(
(
=
(
(
(
(
]
D


(5)
The Online Journal on Power and Energy Engineering (OJPEE) Vol. (1) No. (4)
Reference Number: W09-0040 127
An ellipse can be achieved with the well-known
appropriate constraint, namely, the discriminate
2
4 0 b ac < (6)
It is free to arbitrarily scale the parameters of Eq. (2), so the
equality constraint
2
4 1 ac b = (7)
is imposed. This is a quadratic constraint which may be
expressed in the matrix form 1
T
= k Ck as
0 0 2 0 0 0
0 1 0 0 0 0
2 0 0 0 0 0
1
0 0 0 0 0 0
0 0 0 0 0 0
0 0 0 0 0 0
T

(
(
=
(
(
(
]
k k
(8)
The vector k could be calculated based on following
equations:
= Sk Ck (9)
1
T
= k Ck (10)
where S is the scatter matrix of the size 6 6 ,
T
= S D D (11)
and is an eigenvalue for S.
The sum of squared algebraic distances of the points to the
conic can be derived as
2
|| ||
=
T T T T
T
E

= = = =
=
Dk k D Dk k Sk k Ck
k Ck
(12)
So the eigenvector of S corresponding to the minimal
positive eigenvalue represents the best-fit ellipse for the
given set of points.
In this application, the end points of the rice grain obtained
from the result of LSM are used as the given points to fit
ellipse. Then the parameters of fitted ellipses will be
achieved.
Subsequently, the area and the overlap degree of these
grains are calculated to classify the grains into single one and
connected one by the threshold values of the area pixels and
the overlap degree. These threshold values were determined
with about 800 grains including both single cases and
connected cases. The following equation is used to measure
the overlap degree

=

(13)
where is the number of the set of connected grain pixels;
and is the number of the set of fitted ellipse pixels. Fig. 4
shows the overlap degree of fitted ellipses.
If the grain is single one, the center coordinates of best-fit
ellipse will be output as the center of that; otherwise, the end
points from the result of LSM will be used to conduct
modified direct ellipse-fitting algorithm to separate the
connected grains and position them.
Figure (4): Determination of overlap degree
Figure (5): Modified ellipse fitting algorithm for connected
grains
IV. MODIFIED DIRECT ELLIPSE-FITTING ALGORITHM
FOR CONNECTED GRAINS
When the flow rate of rice grains is 19.5 kg/min, about
93.4% of all grains were presented as single grains in color
sorter. Also, the majority of connected grains appeared in
groups of two, with less than 5% of them appeared in three
or four. Thus, it is necessary to separate the connected grains
image to position them respectively. In this paper, the
algorithm is developed with focus on separating two
connected grains.
For the connected grains, there were some methods
proposed to separate their images. In [17], the curvature of
every boundary point was computed, and the potential nodes
with the minimum curvature were identified as the terminal
points of segmentation line to separate the connected grain.
However, the threshold value of the curvature was difficult
to determine. Moreover, the combinations of potential nodes
failed with wrong pairing sometimes, which resulted in
wrong segmentation.
Shatadal developed a mathematical morphology-based
algorithm to disconnect the conjoint regions in an image of
connected grain, but it was time-intensive and did not
perform well on the slender grains.
= 0.9726 = 0.8209
Input the end points
Sort the end points by order
Fit ellipse with the
six end points
Is the overlap degree
large than
counter = counter + 1
counter == M
yes
no
yes
no
Clustering the N
fitted ellipses
Get all the clusters with a
number of the fitted ellipses
that is is larger than j
Compute the cluster
centers
Combine two of the clusters, and
compute the overlap degree, to get
the maximum of that
Then the two cluster centers
will be given as the final fitted
ellipses

The Online Journal on Power and Energy Engineering (OJPEE) Vol. (1) No. (4)
Reference Number: W09-0040 128
Zhang developed an ellipse fitting algorithm to separate
connected grains in images. Nevertheless, the clustering
method of fitted ellipses did not perform well because the
fitted ellipses were dispersed and the requirement of final
representative ellipses selection.
Figure (6): (left top)the connected grains (right top) the
result of line scan merging (left bottom) the fitted ellipses
(right bottom) the representative ellipses
V. RESULTS AND DISCUSSION
The images captured with developed image acquisition
board were processed by the proposed algorithms and were
analyzed by Matlab 7.0, including 521 single grains and 160
connected grains. The deviation from the center of fitted
ellipse to the center of gravity of grain, called center
deviation, should be less than 2.5 pixels in x and y directions
for the requirement of this kind of color sorter. In case of
single grains, the center of gravity of grain was calculated by
the following equation
( , ) ( , )
1 1
( , ) ,
c c
x y R x y R
x y x y
N N
e e

=
`
)

(14)
where N is the pixel number of single grain R. Table 1 shows
the statistical results of center deviation, from which it can be
seen that the maximum deviations in x and y directions are
less than one pixel, so the accuracy of these algorithms was
100% for single grains. In case of connected grains, their
centers of gravity were estimated by artificial method. The
centers were marked by 3 people and then the coordinates
were identified by Windows Paint. The average values of the
marked results were considered as the centers of gravity of
grains. The proposed algorithms were then conducted 3 times
to extract the centers of fitted ellipses for the grains.
The proposed algorithms are useful when the flow rate of
rice grains is less than 19.5 kg/min, and the connected grains
are less than 7% of all grains.
In this work, even though the case of more than two
connected grains is not discussed, the proposed algorithms
can be easily applied with a minor modification on sorting
part for end points is a little modified. However, the
computation is time-consuming and the occurrence is so less
that can be negligible. Thus, such case is not considered in
this paper.
Figure (7): Positioning of grains
VI. CONCLUSION
Positioning is an essential processing step for the color
sorter. A complete set of algorithms was pro-posed to
position rice grains. Following conclusions can be drawn
from this work:
(1) The LSM algorithm, a novel algorithm of labeling
connected components for line CCD sensor, was
proposed. It is simple and easy to be con-ducted. Also, it
takes up less storage space compared with the
conventional methods.
(2) With the area and overlap degree of rice grain, the cases
of single one and connected one were classified
successfully.
(3) The coordinates of the end points of rice grains were
applied to fit ellipse to get the position of single grains by
direct ellipse-fitting algorithm. The accuracy of the
positioning algorithms was 100% for single grains.
(4) The direct ellipse-fitting of algorithm was modified to
position connected grains, and the results showed a
96.36% successful rate.
ACKNOWLEDGEMENT
"This work was partially supported by the Grant of the
Korean Ministry of Education, Science and Technology" The
Regional Core Research Program
REFERENCES
[1] Aguilera, J.M., Cipriano, A., Eraa, M., Lillo, I., Mery,
D., Soto, A., Valdivieso ,C., 2007. Computer Vision for
Quality Control in Latin American Food Industry, A
Case Study. En International Conference on Computer
Vision: Workshop on Computer Vision Applications for
Devel-oping Countries, Ro de Janeiro, Brazil.
[2] Brosnan ,T., Sun, D.W., 2004. Improving quality
inspection of food products by computer vision-a
review. J. Food Eng., 61(1): 3-16.[doi:10.1016/S0260-
8774 (03)00183-3
The Online Journal on Power and Energy Engineering (OJPEE) Vol. (1) No. (4)
Reference Number: W09-0040 129
[3] Chang, F., Chen, C.J., Lu C.J., 2004. A linear-time
compon- ent-labeling algorithm using contour tracing
technique. Comput. Vis. Image Und., 93(2): 206- 220.
[4] Gonzalez, R.C., Woods, R.E., 2008. Digital Image
Processing. Prentice-Hall, Upper Saddle River.
[5] Gui, J.S., Rao, X.Q., Ying, Y.B., 2007. Fruit shape
detection by level set. J. Zhejiang Univ. Sci. A
8(8):1232-1236 [doi:10.1631/jzus.2007.A1232]
[6] Hamed, S.S., James, S.G., Goddard, Jr., 1999.Vision
system for on-loom fabric inspection. IEEE Trans. on
Ind. Appl.35(6):1252-259. [doi:10.1109/28.806035]
[7] He, L.F., Chao, Y.Y., kenji, S., 2008. A run-based two-
scan labeling algorithm. IEEE Trans. on Image Process.
17(5):749-756.
[8] He, L.F. Chao, Y.Y., Kenji, S., Wu, K., 2009. Fast con-
nected-component labeling. Pattern Recogn., 42(9):1
977-1987.
[9] Kumar, A., Pang, G.K.H., 2002. Defect detection in
textured materials using optimized filters. Systems,
Man, and Cybernetics, Part B, IEEE Trans. on Ind.
Appl.32(5): 553-570.
[10] Loh, H.H., Lu, M.S., 1999.Printed circuit board
inspection using image analysis. IEEE Trans. on Ind.
Appl.35(2): 426-432..
[11] Li, H., Lin ,J. C., 1994. Using fuzzy logic to detect
dimple defects of polished wafer surfaces. IEEE Trans.
on Ind. Appl., 30(2):317-323.
[12] Liu, C.T., Yang Y.Y., Lin S.Y., 2008. Development of
an Automatic online gap-detection scheme for levitated
in-dustrial steel-plate conveyance system. IEEE Trans.
on Ind. Appl.44(2):517-524.
[13] Lumia, R., Shapiro, L., Zuniga, O., 1983. A new
connected components algorithm for virtual memory
computers. Comput. Vis. Graph. Image Process.,22(2):
287-300.
[14] Manohar, M., Ramapriyan, H.K., 1989. Connected
component labeling of binary images on a mesh
connected massively parallel processor. Comput. Vis.
Graph. Image Process., 45(2):133-149.
[15] Shatadal, P., Jayas, D. S.; Bulley, N. R., 1995. Digital
image analysis for software separation and classification
of touching grains: I. Disconnect algorithm. Trans. of
the ASAE., 38(2): 635-643.
[16] Sorebo, J.H., Lorenz, R.D., 2005. Web inspection using
gradient-indexed optics., IEEE Trans on, Ind. Appl.,
41(6):1476-1482.
[17] Visen, N.S., Shashidhar, N.S., Paliwal, J., Jayas, D.S.,
2001.Identification and segmentation of occluding
groups of grain kernels in a grain sample image. J. agric.
Eng. Res. 79(2):159-166.
[18] Xun, Y., Zhang, J.X., Li, W., Cai, W.G., 2006.
Automatic system of seeds refined grading based on
machine vision. The Sixth World Congress on
Intelligent Control and Automation, p.9686-9689.
[19] Yang, Y.Z., Wu, C.Y., Hu X.D., 2005. Study of Web-
based integration of pneumatic manipulator and its
vision posi-tioning. J. Zhejiang Univ. Sci. A, 6A(6):543-
548.
[20] Wu, F.P., Zhang ,X.M., Kuan ,Y.C., He ,Z.Z., 2008. An
AOI algorithm for PCB based on feature extraction. The
7th World Congress on Intelligent Control and
Automation, 2008. WCICA 2008. p.240-247.
[21] Wu, K.S., Otoo, E., Kenji, S., 2009. Optimizing two-
pass connected-component labeling algorithms. Pattern
Anal. Appl., 12(2):117-135.
[22] Zhang, G., Jayas D. S., White N. D.G.., 2005.
Separation of touching grain kernels in an image by
ellipse fitting al-gorithm. Biosyst. Eng., 92(2):135-142.