Vous êtes sur la page 1sur 7

INTRODUCTION

The selected paper proposes a framework for continuous real-time traffic pattern analysis using
sparse video surveillance information. The sparsity of surveillance cameras poses the big challenge
as it does not provide a complete picture of real-time traffic patterns. Due to high deployment
overheads and dynamic nature of road networks, dense distribution of video surveillance cameras is
not feasible. Hence to overcome this limitation, most of the current systems focus on extracting the
patterns from the historical data available at a particular location. But this method fails to provide
real-time picture. This paper proposes the solution that not only works with limited video
surveillance information but also claims to provide real-time traffic analysis and inference.

Goal of the system is to learn the traffic pattern, approximate the missing surveillance information
and infer the real-time traffic volume.
Input:
 The GPS data collected from 4,303 taxicabs of a year
 Video surveillance information from 44 camera-equipped road intersections
Above information was collected for the particular period i.e., from May 1, 2015 to April
30, 2016
Output:
The system gives the calculated traffic volume value as the output for given intersection at
given time.

PROPOSED HYPOTHESES

According to the authors of the paper raise the concern as to how prediction can be made if some
parts of surveillance information is missing. They have analysed the results and methodology used
by existing systems and after clearly identifying the limitations of the prediction by existing
systems and reasoning out the causes behind drawbacks, they have proposed an algorithm to
overcome current challenges. Thus in this paper, they claim that the proposed framework not only
eliminates the existing issues but also outperforms the competitor algorithms.

Though the hypotheses were not stated explicitly in the paper, they can be inferred as follows:

 The proposed solution framework is performs better while predicting traffic volume with
missing surveillance information than existing approaches.

 The proposed TISV algorithm achieves better accuracy for the real-time intersection traffic
volume inference than the chosen competitors under different day and time settings and
varying factors.

Details of the above stated hypothesis are as follows:


Existing approaches: Neural network based methods, spatial topology based methods
TISV: Traffic Volume Inferring with Sparse Video Surveillance Cameras algorithm

The above hypotheses can be represented as


H0: muold = munew (null hypothesis)
H1: muold < munew (alternate hypothesis)

muold => avg accuracy of competitor algorithms or avg performance of existing approaches
munew => avg accuracy of TISV algorithm or avg performance of solution framework
TESTING OF HYPOTHESES

The experiments were performed on real world data to prove the effectiveness of the approach. In
order to test the hypothesis, an experiment was carried out on the Suzhou Industrial Park in China
covering about 45.5 km 2.
The details of the experimental setup are as follows:

Experimental setup
The aim of the experiment was to predict the traffic volume at given intersection at given time, i.e.
to predict the traffic volume Vj(Δt) where j is given camera-free intersection and Δt is given time
interval.

Road Networks
The experiment was performed on the main urban area with a road network of 107 road
intersections and 298 road segments. Among the 107 intersections, 44 were equipped with fixed
video surveillance systems.

Taxicab Systems
For the purpose of training the transition probabilities, historical records of 4,303 taxicabs were
collected. All taxicabs were equipped with a GPS-based navigation system and 3G/4G network
which uploaded their ID, location, speed, direction etc.in every two minutes.

An example illustrating the experimental setup is shown in Figure, where camera-equipped


intersections are marked by red dots, and the verifying intersections are illustrated by blue dots.

Metric used for comparison


Accuracy ratio used to denote the inference accuracy and used as metric for comparing the
performances of algorithms.
The scatter plots depicting the impact by the varying factors on the accuracies of TISV algorithm
and its competitors were drawn and analyse to study various effects.
Statistics
The evaluation process followed for this project did not include any statistical hypothesis testing. It
is identified as one of the limitation of the evaluation method used in this paper. Statistical testing of
hypothesis is necessary to know if the observations obtained from the experiment performed are not
due to any noise and are really significant to consider.

EVALUATION AND CONCLUSIONS DRAWN

Evaluation:

The dataset of taxicabs was used to train the transition probabilities, then use the surveillance data
to build the traffic estimation model. 35% camera-equipped intersections were randomly selected as
verifying intersections (analogous to test data).

It was assumed that the surveillance results of the chosen intersections are missing. The proposed
TISV algorithm was implemented to infer and predict the missing traffic volume information. The
result was then compared with ground value and accuracy ratio was computed.

Similarly, the performances of competitor algorithms were measured and accuracy was noted for
further comparison.

Later, 10 days were randomly selected including both workdays and weekends, to test the accuracy
of inference of the real-time traffic volumes. In all the experimental settings, time slots from 7 am
to 8 pm, was considered for each day and the algorithm was evaluated for its effectiveness by
comparing its accuracy with accuracies of several competitors in following settings:

 impact of day type


 impact of time period
 impact of number of camera equipped intersections
 impact of traffic volumes
 impact of density of camera equipped intersections

Conclusion:

Following conclusions were drawn from the experimental evaluation of the framework:
 It can be observed that the accuracy of TISV method is steadily above 70%.
 Accuracy and stability of TISV model is better than baseline models
 Proposed solution can improve estimation accuracy by up to 30%
 Proposed solution outperforms all competitors and achieves an accuracy as high as 75%
 Inference accuracy works better in rush hours
 Performances of all algorithms in morning rush hours is better than those in afternoon
 Accuracy increase with increased percentage of camera-equipped intersections solution
general to day types
 Performance of solution increases with increase in actual volume flows
 Increase in density of camera equipped sections results in increase in inference accuracy
In my opinion, the conclusions drawn from the observations are not justified by the results of the
evaluation. The evaluation focuses on proving an alternate hypothesis i.e. proposed TISV is better
than its competitors. But it fails provide information regarding veracity of this proof. In other
words, it fails to prove if the results are statistically significant and are not due to any noise.

STRENGTHS AND WEAKNESSES OF EVALUATION

Strengths:
 The experiment was chosen to be performed in the main urban area of China in order to
test the framework in worst-case traffic scenario. Hence testing system the real-time
setup and data could give a better picture of system’s robustness.
 In the beginning of the experiment, an area with at most 41% camera-intersections was
considered. And in the later part of the experiments, number of camera-equipped
intersections was reduced to merely 15%. Thus the evaluation of the system did
consider the possible scenarios such non-urban areas where sparsity of surveillance
cameras is very low.
 The GPS data of taxicabs was used as a historical data for training purpose. This is a
good part on ethics point of view as the personal GPS data from individual’s cars was
avoided to be used in the experiment.
 While designing the experiment, the process of randomization was taken into account.
For example randomly chosen 35% intersections, randomly chosen 10 days for
conducting the experiment etc. This has reduced the chances of biased data.
 During the experimental design, quite a good number of possible impacting factors
were considered. The experiment tries to cover the impacts of weekends, time of the
day, percentages of camera-equipped intersections etc. on the traffic flow, in order to
observe the system’s performance in adverse conditions.
 In the analysis phase of the evaluation, the alternate hypothesis was proved successfully
by comparing the performance of proposed algorithms with competitor algorithms.

Weaknesses
 There was no proof given anywhere throughout the evaluation that there is no
contribution of noise i.e. biased data in the results while concluding the observed
accuracies.
 The system was evaluated against the chosen baseline algorithms. However, there is no
mention of these competitor algorithms being the gold standard. Hence one cannot
ignore the possibility of a better algorithm that might exist to which the proposed
algorithm was not compared with.
 While designing the experiment, there is no consideration of extreme cases like
prediction of traffic flows for an area with all camera-free intersections. This is
necessary to know the breakpoint of the system in an adverse extreme condition such as
power outage.
 Also, the current experimental design does not comply with the standard experimental
design patterns such True experimental design or Quasi-experimental design. The
compliance with standard design pattern ensures that the system was evaluated in a
known scientific way.
 Statistical hypothesis testing was not performed while evaluating the system. Statistical
hypothesis testing provides a way to ensure the results are not due to noise. Thus, in this
case, there is no means to know if the observed results are statistically significant or not.
 Also, the evaluation process does not take into account the qualitative analysis of the
system. It is necessary to know when the system does not work and eventually to know
why the system does not work.

IMPROVEMENTS

The evaluation performed in this paper for the traffic analysis framework with sparse surveillance
information is found with some weaknesses which are mentioned in the previous section. In this
section, I have attempted to suggest some improvements which can help evaluate the system for its
effectiveness in a better way.

Firstly, there should be a proper evaluation strategy drafted before starting the evaluation.
Evaluation strategy is a plan that specifies “The collection and analysis of information to determine
answers to formative and summative evaluation questions in order to understand whether and how a
program [AI system] is meetings its stated objectives and its outcomes and impacts ” [5]. A proper
and clear documentation stating the evaluation strategy in terms aim, timings, methods, instruments
and information can be prepared. It should be carefully drafted as it can “make or break” the
system. Also, the existing experimentation method evaluates the traffic analysis system for its
effectiveness and efficiency but fails to consider the usability aspect of the evaluation. Thus to
overcome this limitation, the system should be tested with human participants. It can be done in
following ways
 Survey and questionnaires
 Asking users their opinions
 Asking experts their opinions

Secondly, the statistical hypothesis testing should be performed to ensure the quality of the results.
It is done to to check if the results are not due to noise and are statistically significant. Statistical
analysis tools such as R can be used to perform hypothesis testing. The separate datasets of
algorithms used and their accuracies for every setting can be prepared. Then ANOVA(ANalysis Of
VAriation) test can be used to perform comparison of accuracy means between proposed algorithm
and competitors. T-test can also be used to compare the means between only 2 algorithms. An
example dataset is shown below:
Day# Accuracy
Ratio(TISV)
Day1 0.75
Day2 0.79
.
Day10 0.80

Day# Accuracy
Ratio(K-means)
Day1 0.55
Day2 0.57
.
Day10 0.60

Thirdly, a research regarding the baseline algorithms can be done to check if they represent the gold
standard solution for the given the problem. Also, the experiment should be designed in such a way
that it complies with the standard available experimental design pattern. It helps evaluating the
system in proper scientific way. The qualitative analysis of errors and poor performances can be
performed to give insights on when the system works and when it doesn’t. It helps to give
researchers ideas about how to improve the system and also helps users/readers understand when to
use the system.

Lastly, a concrete conclusion along with the supporting results should be provided to prove if the
hypothesis claimed at the start of the experiment is true. The evaluation should revolve around the
testable hypothesis which should be clearly stated before evaluating the system. The evaluation
should also consider factors such as replication and negative results while testing the hypothesis
statistically.

References:
[1] Real-time Traffic Pattern Analysis and Inference with Sparse Video Surveillance
Information, Yang Wang and Yiwei Xiao and Xike Xie and Ruoyu Chen and
Hengchang Liu, IJCAI, 2018
[2] http://www.businessdictionary.com/definition/data-analysis.html
[3] https://www.statisticshowto.datasciencecentral.com/probability-and-statistics/
[4] https://ehudreiter.com/tag/evaluation/
[5] Markiewicz, 2014, p10
Appendix

1.Specifications of competitor algorithms


The performance of the proposed TISV algorithm was evaluated against the performances of OKA
(One Kilometer Average), LR (Linear Regression), K-means and LSTM (Long Short-Term
Memory) strategies. The OKA strategy assumes the real-time traffic volume of a camera-free
intersection equals to the average traffic volume of all camera-equipped intersections within one
kilometer.
The LR strategy establishes an linear regression model for each camera-equipped intersection, and
assumes its nearest camera-free intersection follows the same linear regression model. The K-means
strategy partitions all intersections into clusters and assume the traffic volume of a camera-free
intersection equals to the one of its cluster center.
The LSTM strategy approximates the traffic flow of a camera-free intersection by its nearest
camera-equipped intersection.

2. Important research questions


While performing the evaluation, I came across some of the important reserach questions that can
be considered.
Is really sparsity of camera distribution unavoidable?
Do prediction using historical data not provide clear picture of real-time traffic?
Is prediction performed without handling missing data wrong?
Do trajectories and traffic flows vary with different settings of time interval

Vous aimerez peut-être aussi