Measuring the Performance of IEEE 802.

11p Using ns-2 Simulator for Vehicular Networks

Todd Murray, Tammy Murray, Michael Cojocari, and Huirong Fu, Member of IEEE
Abstract The 802.11p protocol, also known as Wireless Access for the Vehicular Environment (WAVE), has recently gained momentum in the area of research and development. The WAVE protocol provides enhancements to the physical (PHY) and medium access control (MAC) layers of the existing 802.11 wireless standards. These enhancements are required to support the Intelligent Transportation Systems (ITS) initiatives of the US Department of Transportation regarding vehicle-tovehicle, vehicle-to-infrastructure, and infrastructure-to-vehicle communication. Many research groups have contributed to the development of the protocol. Many of the same individuals have worked to extend the ns-2 network simulator to correctly simulate wireless mobile networking, specifically Vehicle AdHoc Networks (VANETs). The objective of this research project is to measure the performance of the WAVE protocol at the MAC layer, using the ns-2 simulator. Specifically, the simulations measure aggregate throughput, average delay, and packet loss metrics. Index Terms Vehicular networks, simulation, modeling, traffic management, 802.11p, WAVE

I. INTRODUCTION
1

The US Department of Transportation is currently working on initiatives for vehicle-to-vehicle (V2V), vehicleto-infrastructure (V2I), and infrastructure-to-vehicle (I2V) communication known as Intelligent Transportation Systems (ITS). Wireless Access for the Vehicular Environment (WAVE) is an integral part of this research and development. The main goal of using communication technology in vehicles is to improve passenger safety and reduce fatalities, which is done by broadcasting vehicular intentions and actions to other vehicles in the surrounding power range [1]. These transmissions can be relevant to safety/driving information as well as having other infotainment possibilities (with priority on the safety/driving information). WAVE, or 802.11p, is a standard which provides enhancements to the physical (PHY) and medium access control (MAC) layers of the 802.11 protocol. The PHY layer uses seven 10 MHz channels in the 5.9GHz band, comprised of one control channel and six service channels.

The MAC layer in WAVE is identical to the IEEE 802.11e Enhanced Distributed Channel Access (EDCA) Quality of Service (QoS) extension [7]. While WAVE is designed to support multiple channels and prioritization of messages using Access Categories (ACs), the current implementation of ns-2 does not support channel management functions. Our simulation uses nodes which are configured to broadcast 500 byte messages, at 1 second intervals, on one of the WAVE service channels (CH 176) at 5.880 GHz. Adding channel management functions to ns-2, and measuring the performance of WAVE operating between channels, would be an interesting extension of our current research. We used a combination of the latest network simulation tool ns-2 (version 2.31 with modified PHY and MAC support [4][5]), C++, TCL and AWK to simulate and analyze a WAVE environment of 2-200 vehicles with varying intervehicle distances (5, 12, 19, and 26 meters) moving at varying speeds (0, 24, 48, 72, 96, and 120 km/h). For each combination of the variable parameters we executed twenty runs of the simulator to obtain an average. We used the average output of these five-second simulation iterations to develop an analysis of aggregate throughput, average delay, and packet loss statistics. We hope to have furthered the research on WAVE and the 802.11p protocol with this project. We have supplied some initial analysis on the effect of varying speed and vehicle density (number of vehicles and the distance between each vehicle) that can be expanded upon by including other factors such as multi-channel operation and other road configuration scenarios. The remainder of this paper is structured as follows. In Section II we explain how we used ns-2, C++, TCL, and AWK to implement and analyze our simulations. In Section III we provide an analysis of the performance evaluation of IEEE 802.11p. In Section IV we further discuss the related works and further reading on the WAVE topic. Finally, we conclude in Section V. II. IMPLEMENTATION Using a combination of ns-2, C++, TCL, and AWK we simulated and analyzed a basic one-lane road with varying conditions. Our variable parameters were the number of vehicles, the average distance between vehicles, and vehicle speed. We performed experiments on traffic scenarios of 2200 vehicles, moving at a rate of 0-120 km/h in increments of 24 km/h, with an average inter-vehicle distance of 5 to 26 meters in increments of 7 meters. There are 4,776 (199
498

This work was partly supported by the National Science Foundation under Grant No. 0716527, Michigan Space Grant Consortium Research Seed Grant, and Oakland University Faculty Research Fellowship (FRF). Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation. The authors are with Oakland University, Rochester Hills, MI, 48309, USA. The authors may be reached by email: fu@oakland.edu, phone: 248-370-4456, or fax: 248-370-4625.

978-1-4244-2030-8/08/$25.00 2008 IEEE.

vehicles * 4 distances * 6 speeds) combinations of the selected variable parameters. Our primary TCL script, 80211p_Performance.tcl, configures the simulation to use the latest PHY support for the WAVE standard, the Phy/WLANPhy module. We accepted all default configuration values for this module, except two of them. We set the freq_ parameter to 5.88e+9, or 5.88 GHz, to represent operation on service channel 176. We also set the transmission power of each node, Pt_, to 5.0e-2 to create a communication range of approximately 350 meters. The 802.11p protocol uses the basic channel access mechanisms of 802.11 without modification, so we used the standard wireless channel module, Channel/WirelessChannel. We used the enhanced MAC module, Mac/80211ext, which more realistically models a WAVE environment. Recent research has shown that a fading radio propagation model, such as the Nakagami model, is best for simulation of a WAVE environment [8],[9]. As a result, we have configured our simulation to use the Nakagami propagation model. We developed a C++ application that generates a traffic scenario file for each simulation run. The traffic scenario file contains the initial (x, y) coordinate of each node at ground level, or z = 0. The file also provides node movement instructions that specify the target destination (x, y) of each node and the speed at which the node moves toward that destination, in m/s. The following code segments are from the traffic scenario file generation application. (See TrafficFileGen.cpp for full source)
... numVehicles = atoi(argv[1]); // Grab params maxVehicleDistance = atof(argv[2]); vehicleMPH = atoi(argv[3]); ... for (int i = 0; i < numVehicles; i++) { // Number between 0 and 1 double d = (rand() / ((double)RAND_MAX)); sprintf(xPos, "%.2f", ((i * maxVehicleDistance) + 0.25) + ((d * maxVehicleDistance) 0.25) ); fout << "$vehicle_(" << i << ") set X_ " << xPos << endl; fout << "$vehicle_(" << i << ") set Y_ 10.0" << endl; fout << "$vehicle_(" << i << ") set Z_ 0.0" << endl << endl; } // Output vehicle movements vehicleMPS = (vehicleMPH * 1.6093439999999999 * 1000.0 / 3600.0); // Convert MPH to meters/sec fout << "# Set vehicle movements, " << vehicleMPH << " MPH" << endl; for (int i = 0; i < numVehicles; i++) { fout << "$ns_ at 0.0 \"$vehicle_(" << i << ") setdest 2999.0 10.0 " << vehicleMPS << "\"" << endl; } ...

The following command generates a traffic scenario file for 5 vehicles, which are an average distance of 12 meters apart, moving at 48 km/h (30 mph):
./TrafficFileGen 5 12 30

The output is similar to the following:

... # Set vehicle locations $vehicle_(0) set X_ 11.21 $vehicle_(0) set Y_ 10.0 $vehicle_(0) set Z_ 0.0 ... # Set vehicle movements, 30 MPH $ns_ at 0.0 "$vehicle_(0) setdest 2999.0 10.0 13.4112" ...

We developed a C++ application that executes our primary TCL script through ns-2 for all 4,776 possible combinations of the variable parameters. For each simulation run, the TCL script creates the network nodes, links, trace objects, simulation parameters, traffic scenario file, executes ns-2, and executes AWK scripts to parse the ns-2 trace files. Each AWK script appends one line per simulation into data files. Separate data files are built for aggregate throughput, average delay, and packet loss. The following code segments are from the driver application. (See 80211p_PerfDrv.cpp for full source)
... // Grab params numVehicles = atoi(argv[1]); // Grab params maxVehicleDistance = atof(argv[2]); vehicleMPH = atoi(argv[3]); for (sp = 0; sp <= vehicleMPH; sp += 15) { for (dist = 5; dist <= maxVehicleDistance; dist += 7) { for (veh = 2; veh <= numVehicles; veh++) { cout << "Vehicles: " << veh << endl; char cmd[64]; sprintf(cmd, "<path>/80211p_Performance.tcl %d %.2f %d", veh, dist, sp); std::system(cmd); cnt++; cout << endl; } } } ...

The following command is used to run all 4,776 simulations:

./80211p_PerfDrv 200 26 75

The 80211p_PerfDrv application cycles through the given scenarios and calls the TCL script, 80211p_Performance.tcl, with the varying parameters. The TCL script then opens the appropriate traffic scenario file, runs the simulation, and calculates aggregate throughput, average delay, and packet
499

loss from the simulation trace files. The following code segments are from this TCL script. (See 80211p_Performance.tcl for full source)
... # Parameters set val(numVehicles) [lindex $argv 0]; set val(maxVehicleDistance) [lindex $argv 1]; set val(vehicleMPH) [lindex $argv 2]; set val(seed) [lindex $argv 3]; ... # Redefine some parameters Phy/WLANPhy set freq_ 5.88e+9 ;#5.880 GHz ... Propagation/Nakagami set use_nakagami_dist_ true ... set val(traceFile) "80211p_ $val(numVehicles)nodes_ $val(maxVehicleDistance)_ $val(vehicleMPH).trc" set val(thruFile) "80211p_Thruput.out" set val(delayFile) "80211p_Delay.out" set val(pktLossFile) "80211p_PacketLoss.out" set val(trafficFile) "80211p_traffic_ $val(numVehicles)nodes_ $val(maxVehicleDistance)_ $val(vehicleMPH).tcl" ...

source $trafficFile

The script then runs the simulator and sets up several AWK calls to process the trace file generated by the simulation. The AWK scripts generate data files that are later graphed and analyzed.
... # Calculate throughput stats puts "Calculating thruput..." exec awk -v simtime=$stop -v numVehicles= $numVehicles -v maxVehicleDistance= $maxVehicleDistance -v vehicleMPH=$vehicleMPH -f 80211p_Thruput.awk $traceFile >> $thruFile ...

III. SIMULATION ANALYSIS First, we analyzed the effect of varying vehicle speed for a given average inter-vehicle distance, 12 meters. For each performance metric, we averaged the results of twenty simulation runs and graphed the average values for the 0, 48, and 96 km/h speeds. Within each metric, we found that the data series were very similar, and conceive that the series would further converge with additional simulation runs. Fig. 1 shows the average results of the packet loss metric for the 0, 48, and 96 km/h data series, with an average 12 meter inter-vehicle distance.
8.0 7.0 6.0 5.0 Loss (%) 4.0 3.0 2.0 1.0 0.0 4 36 68 100 132 Number of Vehicles
0 km/h 48 km/h 96 km/h

The above code stores the passed parameters into local variables, redefines the frequency, defines the trace file used as output by ns-2, defines the data files generated by the AWK scripts from the ns-2 trace file, and also defines which traffic file should be used for the simulation. The TCL script then creates an instance of the network simulator object, the topography, and configures the nodes.
... # Create simulator instance set ns_ [new Simulator] # Setup topography object set topo [new Topography] ... # Define node configuration $ns_ node-config -adhocRouting $val(rtg) -llType $val(ll) \ -macType $val(mac) \ -ifqType $val(ifq) \ -antType $val(ant) \ -propType $val(prop) \ -phyType $val(netif) \ -channel $chan \ -topoInstance $topo \ -agentTrace OFF \ -routerTrace OFF \ -macTrace ON \ -phyTrace ON ...

Packet Loss varying vehicle speed at 12 meters

164

196

Fig. 1. Packet loss as a function of vehicle speed, average inter-vehicle distance fixed at 12 meters

After the configuration is complete the TCL script generates the relevant traffic scenario by calling the C++ traffic scenario generator application.
# Vehicle location and movements from traffic # file, generate traffic file here exec ./TrafficFileGen $numVehicles $maxVehicleDistance $vehicleMPH

We also analyzed the impact of varying speed on aggregate throughput and average delay. Like packet loss, we found that the varying speed did not significantly affect either metric. These results are what we would intuitively expect. Based on a transmission rate of 6 Mbps, and a packet size of 500 bytes, the lowest theoretical transmission delay is 0.667 ms (4000 bits / 6,000,000 bits/sec). During this delay, a vehicle moving at a rate of 96 km/h (26.67 m/s) would move only 1.8 cm. Fig. 2 shows the average results of the aggregate throughput metric, and Fig. 3 shows the average results of the average delay metric, for the 0, 48, and 96 km/h data series, with an average 12 meter inter-vehicle distance.

500

Throughput varying vehicle speed at 12 meters

90 80 70

Throughput varying distance between vehicles at 48 km/h

4 Mbps Mbps

60 50 40 30 20

1 10 0 2 10 18 26 34 Number of Vehicles
0 km/h 48 km/h 96 km/h

0 42 50 4 36 68 100 132 Number of Vehicles

5m 12m 19m 26m

164

196

Fig. 2. Aggregate throughput as a function of vehicle speed, average intervehicle distance fixed at 12 meters
Average delay varying vehicle speed at 12 meters

Fig. 4. Aggregate throughput as a function of vehicle density, vehicles moving at 48 km/h

0.72870 0.72860 0.72850 Delay (ms) 0.72840 0.72830 0.72820 0.72810 0.72800 4

36

68

100 132 Number of Vehicles

48 km/h 96 km/h

164

196

0 km/h

Fig. 3. Average delay as a function of vehicle speed, average inter-vehicle distance fixed at 12 meters

Based on our finding that vehicle speed does not have a significant impact on any of the performance metrics, we performed the remainder of our analysis using a single vehicle speed, 48 km/h. Using the average results of the simulation runs, we calculated the systems aggregate throughput from the perspective of the receiver. We included only bytes successfully received by the receivers MAC layer in our aggregate throughput statistic. An event of type r and a trace level of MAC identified such events in the ns-2 trace file. The statistic can be thought of as a goodput measure for the MAC layer. As the number of vehicles increased, the aggregate throughput consistently increased. We observed the same effect for all simulated average inter-vehicle distances: 5, 12, 19, and 26 meters. Fig. 4 shows the average results of aggregate throughput for these inter-vehicle distances, at 48 km/h.

Due to the fact that each vehicle is broadcasting, the data series exhibit a nonlinear form. Each vehicle is sending a message to n 1 other vehicles, where n is the number of vehicles in a given simulation, resulting in n * (n 1) sending attempts per transmission interval. The receivers MAC layer will not successfully receive all of these sending attempts, but the results still show the non-linear effect. The lower inter-vehicles distance series display larger increases in aggregate throughput because added vehicles are more likely to be within the communication range of any other vehicle. Average delay was not significantly affected by vehicle density. Fig. 5 represents a traffic scenario where the vehicles are moving at 48 km/h. The number of vehicles represented, as well as the average distance between vehicles, is varied. In a scenario with few vehicles and smaller inter-vehicle distance, the average delay is slightly lower than that observed with a larger inter-vehicle distance with the same number of vehicles. However, as the number of vehicles increases, all of the average delay data series approach the same asymptotic value of approximately 0.7286 ms 0.7287 ms.
0.72870 0.72860 0.72850 Delay (ms) 0.72840 0.72830 0.72820 0.72810 0.72800 4 36 68 100 132 Number of Vehicles
12m 19m 26m

Average delay varying distance between vehicles at 48 km/h

164

196

5m

Fig. 5. Average delay as a function of vehicle density, vehicles moving at 48 km/h

501

Our analysis of packet loss only considered packets that were sent but never received due to any reason other than insufficient power at the receiver. In the trace file, we excluded any drop events with a reason code of either PND or DND, which are reason codes that indicate that there is insufficient power to successfully receive the packet. We included any other drop events, such as those with a RXB (busy receiving) or TXB (busy transmitting) reason code. Including packets which are dropped due to insufficient power causes the packet loss probability to shoot as high as 75% in lower density situations, which is what we expect, but not what we are interested in. Fig. 6 shows a generally higher packet loss rate as the number of vehicles increases. We measured packet loss probability as the ratio between: 1. The number of packets dropped due to a reason other than insufficient power at the receiving node, and 2. The number of packets transmitted less the number of packets dropped due to insufficient power at the receiving node.
8.0 7.0 6.0 5.0 Loss (%) 4.0 3.0 2.0 1.0 0.0 4 36 68 100 132 Number of Vehicles
5m 12m 19m 26m

vehicle distance series, with the 5m series never really flattening, and only crossing over all other series at the 184 vehicle mark. IV. RELATED WORK AND FURTHER READING In [1] the author presents an analytical and simulative review of the performance of the WAVE protocol in a multichannel environment. The work provides an evaluation of throughput, collision probability and delay based on the Access Categories (ACs) specified in [2]. The author uses the OMNeT++ Simulation Environment for the simulation portion of the analysis. Our work limited the number of simulation nodes to a maximum of 200. We imposed this limitation because the execution time of simulations using the ns-2 simulator increases significantly as the number of nodes increases. A hybrid approach using simulation and statistical models, as described in [3], can be used to significantly speed up the simulations. Many researchers have contributed to the development of the ns-2 simulator, especially in the area of the implementation of the 802.11 protocol. Enhancements have been made to the PHY and MAC layers of the 802.11 protocol [4],[5],[6] to more closely reflect the realities of the 802.11p protocol. V. CONCLUSION AND FUTURE WORK During the implementation, we found 2 software bugs in ns-2 that we were required to isolate and correct before continuing. We modified the VSC agent object in ns-2 to correct a bug in the code that was causing each node to only send out a single message in broadcast mode. If broadcast mode is enabled, each node should repeatedly send out broadcast messages based on a configured time interval. After correcting the bug, each node began functioning as expected. We also modified the CMU trace object in ns-2 to include IP information in the new CMU trace format for VSC agents. The authors of the VSC agent may have intentionally excluded IP information from the traces for their own purpose. However, our delay and packet loss AWK scripts, which parse the trace files, rely on the packet sequencing information provided by the IP logging to tie a drop or receive event back to a specific send event. Overall, vehicle speed proved to be an insignificant factor when judging aggregate throughput, average delay, and packet loss. On the other hand, vehicle density was shown to be a significant factor for all of the performance metrics. As we increased the vehicle density in the simulation, we found that aggregate throughput increased, packet loss increased and average delay converged. Further research can be done to test aggregate throughput, average delay, and packet loss. Some factors we could not include in this research were multi-channel operation and other road configurations. REFERENCES
[1] S. Eichler, Performance Evaluation of the IEEE 802.11p WAVE Communication Standard, in Proc. of the 1st IEEE International

Packet Loss varying distance between vehicles at 48 km/h

164

196

Fig. 6. Packet loss as a function of vehicle density, vehicles moving at 48 km/h

The packet loss probability varies between 4% and 8% for the selected inter-vehicle distances as the number of vehicles approaches 200. For all inter-vehicle distances, the packet loss rate was less than 0.5% until approximately 20 vehicles were introduced. Overall, what we observe here is what we would expect intuitively; as the traffic scenario becomes more dense we would expect each node to drop more packets due to the fact that each node is potentially busy processing receive events for packets sent by an increasing number of peer nodes. We also observed that the relative ordering of the packet loss statistic changed as the number of vehicles increased. For example, at 52 vehicles the relative ranking of packet loss, from lowest to highest, based on inter-vehicle distance was: 5m (0.6%), 12m (1.9%), 19m (2.2%), and 26m (2.4%). Contrast this ranking with that of the 196 vehicle scenario: 26m (4.4%), 19m (5.6%), 12m (7.1%), and 5m (7.7%). Generally, we observed that the packet loss of higher intervehicle distance series flattened sooner than lower inter502

Symposium on Wireless Vehicular Communications (WiVeC), Sept. 2007. [2] IEEE 1609.4, IEEE Trial-Use Standard for Wireless Access in Vehicular Environments (WAVE) - Multi-Channel Operation, IEEE Intelligent Transportation Systems Council, 2006. [3] M. Killat, F. Schmidt-Eisenlohr, H. Hartenstein, C. Rossel, P. Vortisch, S. Assenmacher, and F. Busch, Enabling Efficient and Accurate LargeScale Simulations of VANETs for Vehicular Traffic Management, in Proc. of the fourth ACM international workshop on Vehicular ad hoc networks, Sept. 2007, pp. 29-38. [4] F. Schmidt-Eisenlohr, J. Letamendia-Murua, M. Torrent-Moreno, and H. Hartenstein, Bug Fixes on the IEEE 802.11 DCF module of the Network Simulator ns-2.28, University of Karlsruhe, Tech. Rep. TR-2006-1, Jan. 2006. [5] F. Schmidt-Eisenlohr, M. Torrent-Moreno, T. Tielert, J. Mittag, and H. Hartenstein, Cumulative Noise and 5.9GHz DSRC Extensions for ns2.28, University of Karlsruhe, Tech. Rep. TR-2006-21, Oct. 2006. [6] Q. Chen, D. Jiang, V. Taliwal, and L. Delgrossi, IEEE 802.11 based vehicular communication simulation design for ns-2, in Proc. of the 3rd International Workshop on Vehicular Ad Hoc Networks (VANET), Sept. 2006, pp. 5056. [7] Standards Committee, Wireless LAN Medium Access Control (MAC) and Physical layer (PHY) specifications: Amendment 8: Medium Access Control (MAC) Quality of Service Enhancements, IEEE Computer Society, 2005. [8] F. Schmidt-Eisenlohr, M. Torrent-Moreno, J. Mittag, and H. Hartenstein, Simulation platform for inter-vehicle communications and analysis of periodic information exchange, in Proc. 4th Conference on

Wireless On demand Network Systems and Services (WONS), Obergurgl, Austria, pp. 50--58, Jan. 2007. [9] V. Taliwal, D. Jiang, H. Mangold, C. Chen, and R. Sengupta, Empirical Determination of Channel Characteristics for DSRC Vehicle-tovehicle Communication, in Proceedings of the ACM International Workshop on Vehicular Ad Hoc Networks (VANET), Philadelphia, PA, USA, Oct. 2004.

503