Académique Documents
Professionnel Documents
Culture Documents
Automation in Construction
journal homepage: www.elsevier.com/locate/autcon
A R T I C L E I N F O A B S T R A C T
Keywords: Network level evaluation of pavement surface drainage plays a crucial role in the improvement of pavement
Pavement surface drainage safety and reducing accident rates. Hydroplaning, as the main considered cause of accidents in wet weather
Automated system conditions, is a consequence of the low quality draining in the pavement surface. Since no automated system
Image processing currently exists for the pavement drainage evaluation, this work was conducted to present a new system to assess
the quality of the surface drainage process. To this end, an innovative device is presented to simulate the sa-
turation condition of the pavement surface and acquire photos from the drainage process of the pavement
surface after saturation. Next, an image processing method was applied to produce proper indices for drainage
quality assessment. The preprocessing and enhancement of images was performed using shearlet transform. The
rate of surface drainage progress was evaluated by three indices extracted from the images. Finally, pavements
were classified into three categories according to the indices extracted for their surface drainage. The validation
of the proposed method by the confusion matrix shows the high performance of the system in simulation and
assessment of surface drainage of the road pavements.
⁎
Corresponding author.
E-mail addresses: behrouz.mataei@aut.ac.ir (B. Mataei), moghadas@aut.ac.ir (F. Moghadas Nejad), zahedi@razi.ac.ir (M. Zahedi), h-zakeri@aut.ac.ir (H. Zakeri).
https://doi.org/10.1016/j.autcon.2017.11.010
Received 21 November 2016; Received in revised form 3 July 2017; Accepted 11 November 2017
0926-5805/ © 2017 Elsevier B.V. All rights reserved.
B. Mataei et al. Automation in Construction 86 (2018) 240–255
241
B. Mataei et al. Automation in Construction 86 (2018) 240–255
1.2. Problem statement scales and locations but also across various orientations and highly
anisotropic shapes. Due to their richer structure, these more sophisti-
Although many researchers tried to present methods and devices for cated multiscale systems are able to overcome the poor directional
a better evaluation of pavement surface features, surface drainage as an sensitivity of traditional multiscale systems. Thus, they have been used
important subject in road safety has not been studied yet. The standard to derive several state-of-the-art algorithms in image and signal pro-
procedure for assessment of pavement surface features is to perform it cessing [44,45].
on saturated pavements. Hydroplaning, which is considered the main Shearlets, in particular, offer a unique combination of remarkable
cause of accidents in wet weather conditions, leads to the loss of friction features in that they have a simple and well-understood mathematical
and thus disables the vehicle from responding to actions such as structure derived from the theory of affine systems [43,46]; they pro-
steering, braking, and accelerating. The occurrence of this phenomenon vide optimally sparse representations, in a precise sense, for a large
has a great relationship with the quality of pavement surface drainage. class of images and other multidimensional data where wavelets are
Although the assessment of the drainage of the pavement surface is of suboptimal [47,48]; and the directionality in these systems is controlled
highest significance, no automated system currently exists for the by shear matrices rather than rotations. This last property, in particular,
drainage evaluation. This study presents an automated system based on enables a unified framework for a continuous and discrete setting since
image processing that allows simulating surface drainage of the pave- shear transformations preserve the rectangular lattice which is an ad-
ments and presents indices for surface drainage evaluation. The pro- vantage in deriving reliable digital implementations [49,50].
posed system consists of an innovative device and a software sum- Shearlets are mathematically defined as follows:
marized in Fig. 1. For all functions fϵL2(R2), the shearlet transform can be written as:
SHψ f (a, s, t ) = < f , ψa, s, t > (1)
1.3. Objective of the research
Where ψast is the affine function which is defined as follows:
The objective of this paper is to propose a device to prepare in- −3
formation from pavement drainage by saturating the surface and image ψa, s, t = a 4 ψ (Ss−1Aa−1 (x − t ))
acquisition from the drainage process, and then use image processing a 0 ⎞
techniques to process and analyze the pavement images and present a where Aa = ⎛
⎝0 a ⎠
, Ss =
1 s
0 1 ( )
and ψϵL2(R2).a represent image scale
new method for evaluation of pavement surface drainage. Also the ef- parameter, s represents shear direction, and t represents translation.
ficacy of texture characteristics in surface drainage is evaluated. The function ψ should meet the following conditions:
The rest of this paper is organized as follows: Section 2 describes
(ε ) = ψ (ε1, ε2) = ψ (ε1) ψ
ε2 ε
theoretical information. Section 3 presents the proposed method in
detail. Finally, Section 4 presents the discussion and conclusion.
1- ψ 1 2 ( )
1
ψ = Fourier transform of the function ψ,
ε = (ε1 , ε2 )ϵ R2 , ε2 ≠ 0.
2. Theoretical information 2- ψ1ϵL2(R) is a continuous wavelet, satisfying Calderon condition
∞
242
B. Mataei et al. Automation in Construction 86 (2018) 240–255
Fig. 2. The tilings of the frequency plane made by the shearlets (n = 2).
Table 2
List of different multiscale geometric analysis methods in the field of asphalt pavement distress detection, classification, and evaluation.
1 Wavelet [55–61] Regular in shape; Not sparse for 2-D signals, which leads to a low accuracy for
Compact support; crack representation;
Efficient for objects with point singularities; No smoothness along the edge;
Decomposes an image into subimages; Fail to represent objects with anisotropic elements;
Decomposes an image according to scale and Non-geometrical; and
translation; and Limited capability in dealing with directional information due
Ability to deliver highly sparse approximations to their isotropic support and limited directional sensitivity.
2 Curvelet [58,62–64] High directional sensitivity; Curvelet transform is not directly constructed in the discrete
High anisotropy; domain and implementation involves a low efficiency;
Availability of fast transform algorithms; and Non-adaptive methods; and
Tree-structured filter bank Pseudo-Gibbs artifacts
3 Contourlet [62] The combination of a multi-scale and a Less clear directional features;
directional filter bank in the discrete domain Non-adaptive methods; and
Large computation
4 Ridgelet [58,59,62,64] High directional sensitivity Pseudo-Gibbs artifacts
Highly anisotropic;
Availability of fast transform algorithms
5 Bandelet [62] Adaptive method; Regular with compact support;
Time-frequency localization; The algorithm of searching the best Bandelet basis is very
Directionality; complicated
Anisotropy; and
Strict sampling
6 Shearlet [53,54,60,62,64] Less pseudo-Gibbs artifacts;
High anisotropy;
Adaptive method;
time-frequency localization;
Directional filter bank;
Different decomposition scales;
Optimal approximation; and
Unique combination of mathematical rigidness
7 Beyond (Directionlet, Brushlet, [64]
Tetrolet, Wedgelet, Grouplet)
other singularity identifications [51,52]. Because of this property, In this investigation, a discrete shearlet transform is utilized to
shearlet transform was found useful for edge detection and anomaly process the pavement surface images. This method is based on a
detection [53,54]. Laplacian pyramid combined with appropriate shearing filter. Fig. 2
243
B. Mataei et al. Automation in Construction 86 (2018) 240–255
shows the schematic representation of the shearlet transform. As shown frame, with all imaging components and saturating tools that are placed
in Fig. 2, this transform decomposes the image into two subbands by inside, was developed in this work. The proposed device consists of
the Laplacian pyramid, namely a low-pass and a high-pass subband. several parts which are explained in the following.
The high-pass subband means the difference between the original
image and the low-band pass image. Then, the high-pass subband is 3.1.1. Imaging equipment
imported into appropriate shearing filters to complete multidirectional A digital camera is embedded in a device that is able to capture
decomposition. Similarly, the same process is also executed on the low- images and videos with desired quality and resolution from the pave-
pass subband to obtain a multi-scale decomposition. ment surface and surface drainage process in adjustable time intervals.
However, to the best of our knowledge, no work has been done on
evaluation of drainage in asphalt pavement surface using shearlet 3.1.2. Saturation equipment
transform (Table 2). This system consists of reservoir, pump, pipes and waterjets that are
designed to simulate saturation conditions. Many calculations were
3. Proposed method done to adjust the spray intensity, angle of spraying, and installation
height of waterjets. Finally, two waterjets were installed on two sides of
3.1. Hardware the inner section of the device to prepare the best condition for sa-
turation.
A hardware set was manufactured to obtain images from the pa-
vement surface drainage. This system is based on a digital camera that 3.1.3. Lighting system
records the pavement surface texture and drainage process. A dynamic To prepare a constant lighting condition without efficacy of the
244
B. Mataei et al. Automation in Construction 86 (2018) 240–255
surrounding light, a system consisting of LED lamps was designed to To perform the data acquisition process, the pavements were in-
provide a stable condition for imaging without any reflection and ne- itially saturated by the saturation tools placed in the device, and then
gative effects in the recorded images. Using this system, image acqui- the imaging process started immediately after saturation by 2-second
sition can be performed at all ambient light conditions. intervals until the end of the drainage process. To facilitate the analysis
process of the images taken by the device, a special white liquid was
3.1.4. Temperature recording system used for saturation to create a contrast with the aggregates color, pro-
A system with four thermometers is designed to record the tem- vide a better visualization for water drainage, and extruding of the
perature of the pavement surface during the images acquisition and surface aggregates from the liquid. Fig. 5 presents an example of images
transfer them to the computer to monitor how temperature influences taken from the surface drainage of a pavement using the proposed
the surface drainage of pavements. device.
245
B. Mataei et al. Automation in Construction 86 (2018) 240–255
Fig. 5. Images taken from the surface drainage of a pavement using the proposed device with 2-s intervals.
246
B. Mataei et al. Automation in Construction 86 (2018) 240–255
Table 3
The global comparison of the performance of transforms.
Transform Filter Time (s) Psnr (db) SNR (db) MSE MAE UQI SSIM
Wavelet dmey 2.36 25.5104 16.4626 6.2025e + 004 232.2252 0.9708 0.9995
Ridgelet Db1 1.7 101.2772 92.2320 6.1974e + 004 232.1057 1 1
Curvelet dmey 120 34.5235 25.4757 6.2025e + 004 232.2252 0.9952 0.9999
Shearlet pyrexc 6.3 309.9993 300.9446 6.2028e + 004 232.2581 1 1
Contourlet Db1 13.7 297.4252 288.3705 6.2028e + 004 232.2581 1 1
aggregates were extracted and detected by shearlet transform. Fig. 10 the water during the drainage and trends of water discharge process
shows edge detection and feature extraction of the pavement surface by from the pavement surface over time. Fig. 11 presents a granulometry
shearlet transform. This figure illustrates the high efficiency of shearlet diagram of the pavement surface aggregate protruding from the water
transform in the detection of edges and surface texture features. over time (2-second intervals) for a given pavement. The lowest curve is
Extracting granulometry of the whole pavement surface images related to the granulometry diagram of the pavement surface in the
taken from saturation time until completion of drainage may present saturation time (0 s) and the other curves from bottom to top refer to
the difference of the protruded aggregates of the pavement surface from the granulometry diagram over time.
247
B. Mataei et al. Automation in Construction 86 (2018) 240–255
Fig. 10. Pavement surface edge detection and shearlet coefficients extraction.
The granulometry diagram provides valuable information about 3.3.2.2. Pavement surface drainage. To develop an automated method
surface drainage. As shown in Fig. 11, the granulometry curves have for pavement surface drainage assessment, three general rules were
raised which means that the area of protruded aggregates is increasing applied:
with time. The rate of this increase is higher in the first seconds but
decreases with the passage of a certain amount of time. This trend 1- The volume of surface water decreases by time.
implies that water is drained from the pavement surface after a while 2- The number of protruding aggregates increases by time, but after a
and drainage continues with a low speed. Clearly, the section with while, due to the decreasing water head, a number of aggregates will
higher drainage speed (in this diagram about 12-16th seconds) is more join together and form a new object with a larger area. Thus, the
important in terms of pavement drainage evaluation. After passing this number of objects will decrease at some points.
section, the occurrence probability of hydroplaning is drastically re- 3- The area of objects will increase by time. In some pavements, the
duced. Thus, it can be concluded that the lower the time of this section, presence of obstacles on the path of water drainage causes water
the faster the drainage of water. returning and sectional decreasing of the objects area.
248
B. Mataei et al. Automation in Construction 86 (2018) 240–255
Fig. 11. Granulometry diagram of the pavement surface aggregates protruding from the water.
Fig. 12 presents a schematic explanation for these three rules. These indices can be calculated as below:
Fig. 13 also presents a 3D presentation of water level changes and n
∑i = 0 (Ni )
number and area of protruded objects over time. the rate of protruded objects numbers = QN = ,
According to the rules listed, three indices were presented to surface t
drainage evaluation: n
∑i = 0 (A i)
the rate of protruded objects area = QA = ,
1- The number of protruded objects over time; t
2- area of protruded objects over time; and the rate of protruded objects surfaces scattering = QP
3- Surface scattering of protruded objects over time. n n
(∑i = 0 (Ni )/ ∑i = 0 (Ai ) )
= .
t
249
B. Mataei et al. Automation in Construction 86 (2018) 240–255
1- The first section has a higher change in growth rate of objects than
the second section and drainage speed is higher while in the second
section drainage has a constant rate and drainage is almost stopped.
2- The time of minimum point occurrence is an important issue. The
lower the time of minimum point, the higher the speed of the
drainage.
3- In the minimum point of each drainage rate pavement surface, water
is partially available on the pavement surface. Thus, water is
drained and the risk of hydroplaning is minimized.
4- Some drainage diagrams of pavement surfaces have a negative
Fig. 13. 3D presentation of water level fluctuations and number and area of protruded minimum point. In such pavements, drained water returns due to
objects over time. the existence of obstacles in the discharging route. This returned
Table 4
The indices calculated for a pavement surface for 30 s after saturation.
Time(s) 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30
QN 49.5 132 44.5 57.5 − 5.5 − 3.5 − 13.5 34 − 19 − 30 −4 − 16.5 11.5 −29 −4
QA 1469 2682 2768 769 1974 1639 1217 290 955 502 680 514 163 664 185
QP ∗10−4 169 246 80 373 − 14 − 11 − 55 586 − 99 − 298 − 29 − 161 353 −218 − 108
250
B. Mataei et al. Automation in Construction 86 (2018) 240–255
Fig. 15. A drainage rate diagram of pavement surfaces with coarse and fine texture.
water is placed in the drainage process once more and increases the occurs in lower time and it can be concluded that drainage speed in
drainage time. pavements with fine texture is higher that drainage speed in coarse
5- The drainage speed has a relationship with the surface texture of the texture pavements (Fig. 15).
pavement. In the pavements with coarse texture, the minimum point
251
B. Mataei et al. Automation in Construction 86 (2018) 240–255
Table 5 To use the system results and make it work on full scale, the data
The variation range of indices for three class of pavements. (images) of surface drainage of the intended pavement is obtained by
the presented device. These images are analyzed by the presented
Type of pavement surface drainage QN QA QP
quality method and the characteristics of surface drainage of the pavement will
be examined. Finally, three indices related to pavement surface drai-
Good > 25 > 1000 0.015 < , < 0.02 nage are calculated and the drainage quality is placed in the range
Normal > 21 > 900 0.013 < , < 0.018
presented for three classes of pavements. Thus, the drainage quality of
Bad < 21 < 900 < 0.013
the pavement is characterized.
The numbers are related to mean of calculated parameters over time for every pavement.
3.3.4. The method validation
Table 6 In this section, the performance of the presented system is evaluated
Confusion matrix. using the confusion matrix. This matrix provides the basis for evalu-
ating the performance of any classifier through its four components,
System result
i.e., True Positive (TP), False Negative (FN), False Positive (FP), and
Reference images Good Normal Bad True Negative (TN). Among others, classification accuracy is the major
Good 12 2 0 parameter to judge the efficiency of a classifier. Classification accuracy
Normal 3 11 3 of a classifier on a given data set refers to the percentage of test set
Bad 0 2 17
tuples that is correctly classified by the classifier. It reflects how well
the classifier recognizes tuples of various classes. This measure is de-
Table 7 fined by:
Confusion table for “good” class.
TP + TN
accuracy (ACC) =
TP = 12 FN = 2 P+N
A number of measures of system performance can be defined based
FP = 3 TN = 28
on the confusion matrix. Some common measures are a sensitivity or
true positive rate (TPR), specificity (SPC), True Negative Rate (TNR),
Table 8 and precision or positive predictive value (PPV).
Confusion table for “normal” class. Sensitivity is a measure of the ability of a prediction model to select
instances of a certain class from a data set, defined as:
TP = 11 FN = 6
TP
FP = 4 TN = 29 Sensitivity =
TP + FN
Specificity is a measure of the ability of a prediction model to select
Table 9 instances of other classes in their own classes. This parameter is defined
Confusion table for “bad” class. as:
TP = 17 FN = 2 TN
Specifity =
FP + TN
FP = 3 TN = 23
Precision is a measure of the accuracy by which a specific class is
predicted. It is defined by:
Table 10
Performance evaluation of the presented system. TP
precision =
TP + FP
Type of pavement surface Accuracy Precision Sensitivity Specificity
drainage quality A confusion matrix has two dimensions; one dimension is indexed
by an actual class of an object and the other is indexed by the class that
Good 88.9% 80% 85.7% 90.3%
the classifier predicts. Table 6 presents the confusion matrix for this
Normal 80% 73.3% 64.7% 87.8%
Bad 88.9% 85% 89% 88.5%
work. For this evaluation, reference images were used as a comparison
system 86% 79.4% 80% 88.9% with the method results. In Table 7 the confusion table for "Good" class
and in Table 8 and Table 9 the confusion tables for respectively
"Normal" and "Bad" classes are presented.
Based on these facts, it is possible to assess the surface drainage As shown in Table 10, the presented system showed an 86% accuracy,
quality of different pavements. 79.4% precision, 80% sensitivity, and 88.9% specificity. Thus, it can be
concluded that the system presents appropriate indices and has a high per-
3.3.3. Final surface drainage evaluation formance in simulation and assessment of surface drainage of pavements.
In the previous sections, three parameters (e.g., the growth rate of
protruded objects number, the growth rate of protruded objects area, 3.4. User interface
and the growth rate of protruded objects surface scattering) were in-
troduced as indices for evaluation of pavement surface drainage. To produce a good user interface, a graphical user interface (GUI)
These parameters were calculated for surface drainage of 40 pave- was designed (Fig. 16). This GUI is user-friendly and bears all necessary
ments. The drainage process of pavements is reviewed by experts in the information. The operation starts by loading the pavement image (“load
pavement surface and pavements were classified into three classes: pavement” command). Then, the system analyzes the series of images
good, normal, and bad. taken from the drainage process and the surface drainage diagram is
The variation range of three indices for every class was calculated displayed as the “analyze” command. Finally, three indices calculated
by assessing the indices for different pavements according to their for the current pavement are shown and the drainage situation is
surface drainage situation. Table 5 presents the variation range of in- evaluated and classified into one of three good, normal, or bad classes
dices for three classes of pavements. by the “pavement drainage situation” command.
252
B. Mataei et al. Automation in Construction 86 (2018) 240–255
Fig. 16. The graphical user interface of the presented automated system.
4. Conclusions issue in road safety assessment. The results of the proposed system can
be implemented in the network-level and project-level pavement
In this study, an automated system was developed based on image management systems for road safety enhancement.
processing techniques to evaluate the pavement surface drainage. The
results indicate the acceptable performance of the proposed system. 5. Future work
Briefly, the shearlet transform was found as an effective tool for
denoising and compressing the pavement images taken by the proposed Despite all advantages mentioned for the proposed method, it is a
hardware. The changes in the granulometry of the protruded aggregates few steps taken toward the application of new methods and technolo-
over time with 2-s intervals were recorded and the granulometry dia- gies for assessing pavement surface drainage. There are many methods
gram of every section was plotted. Moreover, the diagram of pavement and devices for evaluation of pavement texture characteristics, such as
surface drainage rate was obtained by the differences of the granulo- skid resistance and roughness. The majority of these methods are based
metry diagram. Analysis of these diagrams provides valuable informa- on the dry situation of the pavement surface. Also, other methods which
tion about the drainage pattern of pavements. Finally, three indices work on wet pavements cannot simulate the drainage process of the
were presented for evaluation of surface drainage based on the number, pavement from saturation to completion of drainage. By now, there is
area, and surface scattering of the protruded aggregates from the water no method and device proposed for evaluation of pavement surface
over time. Evaluating the performance of the proposed system with drainage. Therefore, the proposed method is the first system for eval-
reference images using the confusion matrix presented a considerable uating surface drainage of pavement. There are many options to de-
86% accuracy for the proposed system. velop the present method to achieve more efficient results. The future
In this work, considerable time and effort were allocated to attain work will focus on using new technologies to prepare better data for
reliable results which cannot be easily affected by environmental fac- processing and analysis like using scanning laser, Kinect camera, and
tors. Although the image acquisition process needs short-time traffic other equipment to provide more accurate visualization and analysis of
controlling (because drainage needs time to accomplish), the analysis the surface in three dimensions (3D), or in the processing step, finding
process of the proposed system was very fast (less than a minute). This better filters for image enhancement using transforms, extracting more
short period allows giving immediate responses to a given index during appropriate indices for drainage evaluation, etc.
real-time applications.
The advantage of this system is its fast processing, operation ease, References
and user-friendly function. Besides, it is able to predict the occurrence
probability of the hydroplaning phenomenon, which is an important [1] Ibrahim M. Asi, Evaluating skid resistance of different asphalt concrete mixes,
Build. Environ. 42 (1) (2007) 325–329, http://dx.doi.org/10.1016/j.buildenv.
253
B. Mataei et al. Automation in Construction 86 (2018) 240–255
254
B. Mataei et al. Automation in Construction 86 (2018) 240–255
org/10.1016/j.patrec.2006.04.013. [61] Gitta Kutyniok, Wang-Q Lim, Rafael Reisenhofer, Shearlab 3D: Faithful digital
[58] Jalal Fadili, Jean-Luc Starck, Curvelets and ridgelets, Encyclopedia of Complexity shearlet transforms based on compactly supported shearlets, ACM Trans. Math.
and Systems Science, Springer, New York, 2009, pp. 1718–1738, , http://dx.doi. Softw. 42 (1) (2016) 5, http://dx.doi.org/10.1145/2740960.
org/10.1007/978-0-387-30440-3_111. [62] Ke Xu, Shunhua Liu, Yonghao Ai, Application of shearlet transform to classification
[59] Daqi Zhang, et al., Automatic ridgelet image enhancement algorithm for road crack of surface defects for metals, Image Vis. Comput. 35 (2015) 23–30, http://dx.doi.
image based on fuzzy entropy and fuzzy divergence, Opt. Lasers Eng. 47 (11) (2009) org/10.1016/j.imavis.2015.01.001.
1216–1225, http://dx.doi.org/10.1016/j.optlaseng.2009.05.014. [63] Wang-Q Lim, Nonseparable shearlet transform, IEEE Trans. Image Process. 22 (5)
[60] James M. Murphy, Jacqueline Le Moigne, David J. Harding, Automatic image re- (2013) 2056–2065, http://dx.doi.org/10.1109/TIP.2013.2244223.
gistration of multimodal remotely sensed data with global shearlet features, IEEE [64] Laurent Jacques, et al., A panorama on multiscale geometric representations, in-
Trans. Geosci. Remote Sens. 54 (3) (2016) 1685–1704, http://dx.doi.org/10.1109/ tertwining spatial, directional and frequency selectivity, Signal Process. 91 (12)
TGRS.2015.2487457. (2011) 2699–2730, http://dx.doi.org/10.1016/j.sigpro.2011.04.025.
255