Vous êtes sur la page 1sur 10

This article has been accepted for publication in a future issue of this journal, but has not been

fully edited. Content may change prior to final publication. Citation information: DOI
10.1109/TWC.2014.2329568, IEEE Transactions on Wireless Communications
IEEE TRANSACTIONS ON WIRELESS COMMUNICATION, 1

Link Correlation Aware Opportunistic Routing in


Wireless Networks
Shuai Wang, Student Member, IEEE, Anas Basalamah, Song Min Kim, Shuo Guo, Yoshito Tobe
and Tian He, Member, IEEE

Abstract—Recent empirical studies have shown clear evidence links are independent when they estimate the transmission cost
that wireless links are not independent and that the packet from a upstream node to the next-hop candidate forwarder set.
receptions on adjacent wireless links are correlated. This finding Recent studies [8]–[10], however, provide clear evidence
contradicts the widely held link-independence assumption in the
calculation of the core metric, i.e., the expected number of that wireless links are not independent because of cross-
transmissions to the candidate forwarder set, in Opportunistic technology interference and correlated shadowing. Cross-
Routing (OR). The inappropriate assumption may cause serious technology interference, which is caused by the external signal
estimation errors in the forwarder set selection, which further in the unlicensed shared spectrum, can lead to correlated pack-
leads to underutilized diversity benefits or extra scheduling et losses since the high-power interferer’s signal may corrupt
costs. We thus advocate that opportunistic routing should be
made aware of link correlation. In this paper, we propose a nearby low-power links simultaneously. On the other hand,
novel link-correlation-aware opportunistic routing scheme, which correlated shadowing, a channel propagation phenomenon that
significantly improves the performance by exploiting the diverse nearby links are affected by the same shadower, may also
low correlated forwarding links. We evaluate the design in a introduce correlated packet losses to wireless networks.
real-world setting with 24 MICAz nodes. Testbed evaluation and The finding of the link correlation phenomenon has sig-
extensive simulation show that (i) higher link correlation leads to
fewer diversity benefits, and (ii) with our link-correlation-aware nificant impacts on network protocols that utilize concurrent
design, the number of transmissions is reduced by 38%. wireless links, which include but are not limited to (i) tradition-
al network protocols such as broadcast [11], multi-cast [12],
Index Terms—Opportunistic routing, Link correlation, Wire-
less networks, Protocol design. and multi-path routing [13], or (ii) diversity-based protocols
such as opportunistic routing [1]–[3], network coding [14],
and hybrid routing [15]. Ignoring this phenomenon may cause
I. I NTRODUCTION
serious estimation errors in modeling, which further leads to
PPORTUNISTIC Routing (OR), originally proposed by
O S. Biswas et al. in [1], has great potential to improve
the network performance. The basic idea of OR is fairly s-
underutilized benefits or extra costs.
For example, when the packet loss patterns of the candidate
forwarders are highly positive correlated (which means that
traightforward. Given a source and a destination in a multi-hop they lose the same packets), the performance of OR is the
wireless network, instead of preselecting a single specific node same as the traditional shortest path protocol (which uses the
to be the next-hop forwarder, a set of candidate forwarders node with the best link among the candidates), since there are
are selected to deliver the packets. Taking advantage of the no diversity benefits to exploit from the candidate forwarder
reception diversity in the candidate forwarder set, OR defers set. In this example, ignoring link correlation brings OR no
the selection of the next hop for a packet until it acquires benefits but extra candidate set schedule costs.
knowledge about the set of candidate forwarders that have Little research has been conducted to exploit link correlation
received that packet. to improve the performance of network protocols [8], [10].
Since the strength of OR comes from the packet reception In this paper, we introduce link correlation to improve OR’s
diversity of the candidate forwarder set, the candidate selection performance by optimizing the forwarder set selection and
becomes one of the key issues in OR. The selection of avoiding duplicate forwarding. Under link correlation, the
different candidates has a high effect on the performance forwarder set selection algorithm prioritizes low correlated
of OR. Extensive candidate selection algorithms based on nodes to increase the level of diversity while ensuring that
the key metric, the expected number of transmissions, have neighboring nodes are close enough to each other such that the
been proposed in the literature [1]–[7]. In these studies, the forwarded packets would be heard and duplicates are avoided.
researchers explicitly or implicitly assume that the wireless In summary, our contributions are as follows:
S. Wang, S. M. Kim and Tian He are with the Department of Computer • We reveal the impact of link correlation upon OR. A
Science and Engineering, University of Minnesota, Minneapolis, MN, USA. novel link correlation aware metric is proposed to capture the
E-mail: {shuaiw, ksong, tianhe}@cs.umn.edu. expected number of any-path transmissions.
A. Basalamah is with the Department of Computer Engineering, Umm Al
Qura University, Makkah, Saudi Arabia. E-mail: ambasalamah@uqu.edu.sa. • With the link correlation aware metric, we propose a new
S. Guo is with Arista Networks, San Francisco, CA, USA. E-mail: candidate forwarder selection algorithm to help OR fully
sguo@cs.umn.edu. exploit the diversity benefit of the wireless broadcast medium.
Y. Tobe is with the Department of Information Systems and Media Design,
Tokyo Denki University, Tokyo, Japan. E-mail: yoshito tobe@osoite.jp • We evaluate our work extensively with testbed implemen-
tations and simulations. The experiment results identify the

1536-1276 (c) 2013 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See
http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI
10.1109/TWC.2014.2329568, IEEE Transactions on Wireless Communications
IEEE TRANSACTIONS ON WIRELESS COMMUNICATION, 2

1 1

2 2
Node ID

Node ID
3 3

4 4

5 5

10 20 30 40 50 60 70 80 90 100 10 20 30 40 50 60 70 80 90 100
Packet ID Packet ID
(a) Real trace with correlated links (b) Synthesized traces with independent links

Fig. 1. A comparison between the link correlation and the link independent packet reception pattern where the black band indicates a packet loss: (a) the
real trace from a 802.15.4 testbed, and (b) the synthesized independent trace. Compare these two traces, we find that they are quite different. There are many
long black bands in the real trace, which indicates that a packet is lost at multiple receivers and packet receptions are correlated.

limitation of the traditional OR under the appearance of link realistic since a large number of candidates may introduce
correlation. With our link correlation aware design, the number large schedule overheads and duplicate transmissions.
of transmissions of OR is significantly reduced. Our work is different from the previous OR schemes which
The remainder of the paper is structured as follows. Sec- implicitly or explicitly assume that packet receptions cross
tion II reviews the related work. Section III presents the multiple receivers are independent when they exploit the diver-
motivation. Section IV introduces the metric, followed by sity benefit of the wireless broadcast medium. We propose a
its implementation in Section V. Experiment results from link correlation aware OR scheme to fully exploit the potential
the testbed and simulation are shown in Sections VI and diversity benefit.
Sections VII. Finally, Section VIII concludes the paper. • Link Correlation: Until recently, wireless links were al-
ways considered to be independent. Recent studies [8], [9],
II. R ELATED W ORK however, have proven the existence of link correlation. In [9],
We begin with a brief survey of prior work on opportunistic the authors derive the κ factor that is used to measure the
routing and link correlation. correlation among links. They show how the κ factor could
• Opportunistic Routing: The majority of previous studies in affect diversity based protocols such as OR. The authors
OR are devoted to candidate selection [1], [4]–[7], reception of [10] use models to generate link correlations based on
acknowledgement [16], forwarder coordination [17], [18], and distance where negative correlations are not accounted for.
rate control [19]. In this work we focus on the fundamental They follow a similar approach that developed in the mobile
issue – the candidate selection in OR. ExOR [1], the primary communication literature [21], [22], which is solely based
OR protocol, uses the single path ETX to select candidates, on correlated shadowing. The authors propose a link probing
where ETX is the average number of transmissions required protocol to help collect link correlations, but the protocol does
to send a packet through a link. The ETX value of a single not show how links are selected.
path is the sum of the ETX for each link in that path. Using While both [9], [10] briefly mention link correlation in
the single path ETX as a metric for candidate selection is the OR setting, we complement their work by providing (i)
an approximation since it cannot capture the opportunistic detailed analyses of the impact of link correlation on OR, (ii) a
paths. To account for the multiple paths that could be used correlation aware candidate selection algorithm, (iii) a detailed
by the candidates, expected any-path transmission (EAX) [20] illustration on how and when link correlation actually helps,
is used in [4]–[7] to capture a more accurate expected number (iv) models to generate and simulate link correlation and (v)
of transmissions. real implementations to demonstrate the effectiveness of a link
For example, H. Dubois et.al. propose least-cost opportunis- correlation aware design.
tic routing (LCOR) [6] which takes EAX as the metric to select
the candidate sets. Similar to the well-known Bellman-Ford III. M OTIVATION
algorithm, LCOR exhaustively searches all possible candidates In this section we first report the existence of link correla-
sets to find the paths with minimum transmissions. Its compu- tion. We then introduce the OR framework and explain how
tational cost increases dramatically in dense networks because wireless diversity serves OR. Finally, we illustrate the impact
of the exponentially explosion in the exhaustive search. In min- of link correlation on the diversity benefit of OR.
imum transmission selection (MTS) [5], the authors compute
the transmission cost with EAX from the destination back to
the source, using a dynamic programming formulation analo- A. Existence of Link Correlation
gous to the Dijkstra’s algorithm. The authors in [7] investigate In wireless networks, when a sender transmits a packet, the
the candidate selection with identical maximum candidate packet reaches to multiple receivers because of the broadcast
set sizes. The result shows that if the maximum number of medium. Link correlation is a phenomenon that the packet
candidates is not limited, different OR algorithms have almost receptions across multiple receivers have certain correlation
the same performance. They prove that this assumption is not and are not independent. To verify the existence of link

1536-1276 (c) 2013 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See
http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI
10.1109/TWC.2014.2329568, IEEE Transactions on Wireless Communications
IEEE TRANSACTIONS ON WIRELESS COMMUNICATION, 3

correlation, we conduct an experiment with six MICAz nodes. receptions. In OR, when one candidate fails in receiving a
In the experiment, the sender is placed in the center while packet, the other candidates may receive the packet. In the
the other five nodes are randomly deployed as receivers. The example in Figure 3, if node f1 fails to receive a packet,
sender broadcasts 1000 packets. Each packet is identified by candidate f2 may receive the packet. Similarly if f2 loses the
the sequence number. All the receivers record the reception packet as well, f3 may probably receive it. In other words, the
results through the packet sequence number. probability that all candidates lose the packet becomes quite
Figure 1(a) plots the packet reception patterns observed low because of the packet reception diversity among multiple
from the testbed. The black bands indicate packet losses while candidates.
the white bands indicate successful packet receptions. From The optimistic view of diversity comes from the assumption
Figure 1(a), we find that packet receptions are correlated. that the packet receptions cross multiple wireless links are
Specifically, there are a large number of long black bands independent. Under such an assumption, the packet loss in a
which indicate that an individual packet is lost at multiple receiver has no relationship with the packet losses in other
nodes. When packet receptions are assumed to be independent, receivers.
we synthetically generate the packet reception pattern which Let s be the source and d be the destination. Suppose Fs,d
is shown in Figure 1(b). The comparison indicates that there is the set of candidate next-hop forwarders from s to d, and
exist a big gap between the link independence model and the fi is the candidate with priority i (with 1 being the highest
real link reception patterns. The experiment results reaffirm priority). Assume that the packet delivery probability from s
the observation reported in recent studies [8], [10], [23]. to fi is ps,fi , and the ACK delivery probability from fi to s
is pfi ,s .
Sender DATA

Candidate F1 ACK DATA


f1
0.5 Correlated
Candidate F2 ACK
Pr(f1|f2)=100%
Candidate F3 ACK s 0.5 f2
Independent
Fig. 2. An example showing the OR operation: candidates defer the ACK 0.45 Pr(f2|f3)=50%
according to their priorities and the candidate with the highest priority will f3
send the data first.

Fig. 3. An OR example with two candidates. When the link independent


model is applied, the candidate set {f1 , f2 } will be selected since it only
B. Opportunistic Forwarding Framework counts link quality. Since the candidates f1 and f2 are perfect positive
The opportunistic packet forwarding process is shown in correlated, the function of the set {f1 , f2 } will be the same as f1 or f2 .
The link correlation aware design will choose {f2 , f3 } as the candidate set,
Figure 2. The sender selects a subset of nodes as candidate which fully exploits the diversity benefit.
next-hops and assigns a priority to each of them. When the
sender transmits a packet, it includes the ordered candidate We now mathematically demonstrate the impact of diversity
forwarder set in its headers. Each candidate that receives upon OR with the example in Figure 3. In the example, the
the packet responds with an ACK. To avoid the feedback size of the candidate set is set to be two, i.e., no more than
implosion, candidates defer their ACKs according to their two nodes are allowed to be candidate forwarders. Let us
priorities in a TDMA-like approach. Since the candidates are start by selecting the first forwarder. The expected number of
likely to hear each other’s ACK, they should include in their transmissions required for candidate f1 or f2 to successfully
ACKs a list of higher priority candidates. In OR, only the receive a packet from s is 1/ps,f1 = 1/ps,f2 = 1/0.5 = 2.
candidate with the highest priority forwards the packet to the Similarly, the expected number of transmissions required by
next hop. Other candidates refrain from forwarding the packet node f3 is 1/ps,f3 = 1/0.45 = 2.22. Obviously selecting node
as long as they overhear a higher priority ACK. Duplicate f1 or f2 would be the optimal choice. Now consider the case
forwarding by more than one candidate could happen if a that a second node is added to the candidate set. The expected
lower priority candidate cannot hear an ACK from a higher number of transmissions for receiving one packet from s to at
priority candidate. The whole forwarding process is initiated least one of the two candidates is given by
again by the sender as long as it does not receive an ACK
1
from any candidates. E(s, Fs,d ) = ∏2 . (1)
1− i (1 − ps,fi )
C. Diversity Benefits in OR In the example, E(s, Fs,d ) with candidate set {f1 , f2 }
Without considering link correlation, the previously OR equals 1.33. Similarly, E(s, Fs,d ) with candidate set {f1 , f3 }
schemes do not fully capture the diversity benefit of the wire- or {f2 , f3 } is 1.38. As a result, under link independence
less broadcast medium. The following section will demonstrate model, selecting nodes f1 and f2 as our forwarders will be
the impact of diversity on OR. the best choice in reducing the transmission cost.
• Diversity with Link Independence Model: The strength • Diversity with Link Correlation Model: If links are
of OR comes from the diversity of the candidates’ packet not independent, the expected number of transmissions for

1536-1276 (c) 2013 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See
http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI
10.1109/TWC.2014.2329568, IEEE Transactions on Wireless Communications
IEEE TRANSACTIONS ON WIRELESS COMMUNICATION, 4

delivering one packet to at least one of the two candidates the event that a packet is successfully received by forwarder
is given by fi with one transmission. The intersection areas represent the
1 correlated events. For example, Es,f1 ∩ Es,f2 represents the
E(s, Fs,d ) = , (2) correlation between event Es,f1 and Es,f2 . Let α0 be the
1 − P r(Es,f1 , Es,f2 )
expected number of transmissions for s to successfully deliver
where Es,fi is the event that a transmission from source s a packet to the candidate set. α0 is the inverse of the total area
is lost at forwarder fi . In the example in Figure 3, the links in Venn diagram, which is given by
from s to f1 and f2 are 100% correlated. P r(Es,f1 , Es,f2 )
α0−1 =ps,f1 + ps,f2 + ps,f3 − P r(Es,f1 , Es,f2 )
equals 0.5. Thus, E(s, Fs,d ) with candidate set {f1 , f2 } turns
out to be 2, which is greater than E(s, Fs,d ) with candidate − P r(Es,f1 , Es,f3 ) − P r(Es,f2 , Es,f3 ) (4)
set {f2 , f3 } (i.e., 1.38). As a result, selecting nodes f2 and + P r(Es,f1 , Es,f2 , Es,f3 ).
f3 as candidates is the best choice under link correlation
To calculate α, we need to count in the lost ACKs from
model. From this example, we find that the link independence
each candidate to s by multiplying each term in Eq.(4) with
assumption overestimates the real diversity of wireless links.
the probability of successfully receiving the ACK. We thus
In the following section, we further analyze the OR framework
have
under the impact of link correlation.

IV. L INK C ORRELATION M ETRIC α−1 =ps,f1 pf1 ,s + ps,f2 pf2 ,s + ps,f3 pf3 ,s
In this section we analyze OR under the existence of link − P r(Es,f1 , Es,f2 )ps,f1 ps,f2
correlation. We explore the impact of link correlation on the − P r(Es,f1 , Es,f3 )ps,f1 ps,f3 (5)
candidate set selection process and reveal how link correla- − P r(Es,f2 , Es,f3 )ps,f2 ps,f3
tion awareness improves the performance of OR. Finally we
+ P r(Es,f1 , Es,f2 , Es,f3 )ps,f1 ps,f2 ps,f3 .
propose a novel link correlation aware opportunistic routing.
We define the expected number of any-path transmissions When the candidate f1 , f2 or f3 succeeds in receiving a
needed for reliably delivering a packet from the source s to packet, it will take over the forwarding process as long as it
the destination d, given the candidate set F = {f1 , f2 , . . . , fn } does not receive an ACK from a higher priority candidate. For
with the link correlation awareness, as E(s, F, d). In our candidate f1 , it forwards a packet as long as it receives it since
design, the computation of E(s, F, d) is recursive and is f1 has the highest priority. We thus have
executed at individual nodes independently. At the receiver ( )
d, obviously, E(s, F, d) is zero. The key idea is to radially βf1 = α0 · E(f1 , F, d) · (1 − γf1 ) , (6)
calculate E(s, F, d) starting from the destination d outward to
where α0 is multiplied because of the implicit condition that
the rest of the network. Specifically, we calculate E(s, F, d)
at least one of the candidates has received the packet, and
as follows:
γf1 is the probability that candidate f1 will not take the
E(s, F, d) = α + β, (3)
forwarding process, which equals 1 − ps,f1 . For the lowest
where α captures the expected number of transmissions for priority candidate f3 , the forwarding process will not happen
successfully transmitting a packet from s to at least one of when f3 fails to receive a packet, or when an ACK is received
the candidates and getting at least one acknowledgment. β from f1 or f2 . The probability that candidate f3 will not take
captures the expected number of transmissions for delivering the forwarding process γf3 can be calculated as follows:
the packet in turn from those candidates to the destination.
γf3 =1 − ps,f3 + pf1 ,f3 P r(Es,f3 , Es,f1 )
(7)
+ pf2 ,f3 P r(Es,f2 , Es,f3 ).
Pr(Es,f1 Es,f2 Es,f3)
f1 For candidate f2 , an ACK could be received explicitly from
2
s,f

Es,f1 candidate f1 or implicitly from candidate f3 when f3 receives


s,f 1

the packet from s and the ACK from f1 . γf2 is thus given by
1
s,f

s f2 d
s,f 3

Es,f2 Es,f3
γf2 = 1 − ps,f2 + pf1 ,f2 P r(Es,f1 , Es,f2 )
(8)
f3 Pr(Es,f2 Es,f3)
+(1 − pf1 ,f2 ) · pf1 ,f3 pf3 ,f2 P r(Es,f1 , Es,f2 , Es,f3 ).

(a) Topology (b) Joint Reception Similar to (6), we now obtain βf2 and βf3 where γf2 and
γf3 can be calculated with Eq.(7) and Eq.(8). Finally, β is the
Fig. 4. Three-candidate case example: (a) the topology; (b) the Venn diagram sum of βf1 , βf2 and βf3 .
of the candidate’s packet reception.
• n-candidate Case: We now extend the problem to n
• Three-candidate Case: we start by considering the 3- candidates. It turns out to be an inclusion-exclusion where we
candidate case in Figure 4, where the receptions of candidate should sum the probabilities of individual links but remove
f1 , f2 and f3 are partially correlated. Figure 4(b) shows the overlapped intersection:
the Venn diagram representing the events that candidates 1
successfully receive a transmission from the sender. Es,fi is α = ∑n k−1 P r(f k )
, (9)
k=1 (−1)

1536-1276 (c) 2013 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See
http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI
10.1109/TWC.2014.2329568, IEEE Transactions on Wireless Communications
IEEE TRANSACTIONS ON WIRELESS COMMUNICATION, 5

where f k ⊂ F = {f1 , . . . , fn } is any candidate forwarder set P r(Es,f1k , . . . , Es,fkk ). Assume a reception report of length L,
with size k, and P r(f k ) is the probability that the k candidate we have
forwarders successfully receive a packet and send back the
1∑
L
ACK. P r(f k ) is calculated as follows: P r(Es,f1k , . . . , Es,fkk ) = Bf (j)& . . . &Bfk (j), (13)
L j=1 1
∑ ∏
k
P r(f k ) = P r(Es,f1k , . . . , Es,fkk ) pfik ,s . (10) where Bfi (j) is a bit representing the candidate fi ’s re-
f k ⊂F i ception status of the jth packet. Bfi (j) = 1 represents
candidate fi receives the packet, otherwise Bfi (j) = 0. For
• Special Case: Eq.(9) includes the special link independence
example, in Figure 5, the sender s has three candidates.
case. When wireless links are independent, we have
We calculate s’s P r(Es,f1k , . . . , Es,fkk ) when k equals two,

k i.e., P r(Es,f12 , . . . , Es,f22 ). Suppose the reception report of
P r(Es,f1k , . . . , Es,fkk ) = ps,fik . candidate f1 is [1101], which indicates that f1 receives the
i 1st, 2nd, and 4th packets and misses the 3rd packet. When the
sender s receives the reception reports from the candidates, it
And P r(f k ) in Eq.(10) turns out to be uses Eq.(13) to calculate P r(Es,f12 , . . . , Es,f22 ), i.e.,
∑ ∏
k 1
P r(f k ) = ps,fik pfik ,s . P r(Es,f12 , Es,f22 ) = (1&0 + 1&1 + 0&0 + 1&1)
4
f k ⊂F i = 50%.
For the candidates which have received the packet from the
transmitter, the expected number of transmissions to forward
the packet to the next hop is calculated as follows: f1 [ 1 1 0 1 ]


n
( )
β = α0 · E(fi , F, d) · (1 − γfi ) . (11)
s f2 [ 0 1 0 1 ]
i=1

where γfi is the probability that candidate fi does not forward


the packet. To calculate γfi , we need to consider three cases: f3 [ 1 1 1 0 ]
(i) candidate fi loses the packet, (ii) candidate fi receives
an direct ACK from a higher priority candidate, and (iii) the Fig. 5. An example of calculating s’s P r(Es,f k , . . . , Es,f k ) for {f1 , f2 }.
1 k
candidate receives an indirect ACK through a low priority
candidate. Compared with the first two cases, the third case can
be ignored since it rarely happens. γfi thus can be calculated
A. Metric Overhead
using the following equation:
• Computational Cost: In OR, having a large set of can-

j<i
didates may reduce the number of transmissions from the
γ = (1 − ps,fi ) + (−1)k−1 P r(f k ). (12) source to the destination. On the other hand, it may bring
k=1
serious problems of increasing the schedule overhead among
Eq.(12) consists two parts, i.e., (1 − ps,fi ) and candidates as well as the chance of duplicated transmis-
∑j<i k−1 k
k=1 (−1)∑ P r(f ). The first part represents the the
sions, which may reduce the efficiency of OR. Moreover,
j<i k−1 the computational cost of searching the optimal candidate
first case. k=1 (−1) P r(f k ) describes the second case,
which represents the union of the probability of each higher set increases dramatically due to the exponentially increased
priority candidate in receiving the packet and sending back combination [6], [7]. In practice, the number of candidates that
the ACK. can be used is set to a small number, e.g., 4. Therefore, the
computation cost of our metric is low due to the small size of
the candidate set.
V. I MPLEMENTATION
• Communication Cost: Our metric needs to calculate link
In this section, we introduce the implementation detail of quality and link correlation which may change over time. We
the link correlation aware metric E(s, F, d). The calculation now discuss the overhead for maintaining the metric accurate.
of E(s, F, d) finally goes to find the link quality (i.e., ps,fi We conduct an experiment in a dynamic scenario. In the
and pfi ,s ) and the link correlation (i.e., P r(Es,f1k , . . . , Es,fkk )). experiment, the sender transmits packets every 0.2 seconds
Suppose each receiver maintains a packet reception report while the packet reception report is sent in every 20s. The total
(e.g., [1101]) recording the reception status of a fixed number number of packets sent is 8000. The candidates keep track of
(e.g., 4) of most recent packets. With the reception report, the received packets through the packet sequence numbers.
the link quality is given simply by the number of 1s in the The main communication overhead of our metric comes
reception report divided by the length of reception report. from the reception reports which are used to calculate link
The calculation of P r(Es,f1k , . . . , Es,fkk ) deserves a little more quality and P r(Es,f1k , . . . , Es,fkk ). The required information
explanation. Here we use an example to show how to calculate for link quality and P r(Es,f1k , . . . , Es,fkk ) is exactly the same,

1536-1276 (c) 2013 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See
http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI
10.1109/TWC.2014.2329568, IEEE Transactions on Wireless Communications
IEEE TRANSACTIONS ON WIRELESS COMMUNICATION, 6

i.e., the reception report. It not only helps to calculate link Algorithm 1 CANDIDATES SELECT(s,d,setsize)
quality, which only uses the bit information in rows in Fig- 1: F ⇐ ∅; F̂ ⇐ ∅; mp ⇐ ∞; mc ⇐ ∞
ure 5, but also provides information of links’ relationship, i.e., 2: for all v ∈ N (s) do
3: if ETX(v, d) < ETX(s, d) then
the bit information in columns in Figure 5.
4: F̂ ⇐ F̂ ∪ v
To collect reception reports, we adopt the free piggyback 5: end if
mechanism with normal traffic data, which has been already 6: end for
applied by the existing protocols [24] to measure link quality 7: while |F | < setsize do
8: cand ⇐ argminE (s, F ∪ c, d)
or to improve the robustness of the routing structure. The c∈F̂
binary reception report is small and is much less frequently 9: mc ⇐ E (s, F ∪ cand, d)
10: if mc < mp then
transmitted, therefore the overhead occupies a tiny fraction
11: F ⇐ F ∪ cand; F̂ ⇐ F̂ \cand
(0.9%) of the total energy cost according to our measurements. 12: mp ⇐ mc
13: else
100 14: E(s, F, d) ⇐ mp ; break
15: end if
16: end while
90
# of received pkts

80
We initialize the candidate set F̂ by adding nodes with smaller
70 ETX values. At this step, we create a directed acyclic graph
from the source s to the destination d and eliminate candidates
60
which have higher ETX values (Lines 1-6). In lines 7-16, we
50 Ground Truth
loop through the initial candidate set and find the candidate
Estimation with Correlated Metric with the minimum E(s, F, d) value. We add the node to our
Estimation with Independent Metric
40 candidate set F and remove it from the initial set F̂ . We loop
0 5 10 15 20 25 30
the above procedures until we find enough candidates to meet
Time (minutes)
the predesigned set size.
Fig. 6. Received pkts vs. Estimation with our link correlation metric
α0 and the link independent metric ETX: the estimation with the link
correlation metric closely follows the real one while the link independent
VI. T ESTBED E XPERIMENTATION
metric overestimates the number of received packets. In the real world environment, interference exits everywhere
because of the massive number of uncontrollably deployed
wireless devices sharing the same unlicensed spectrum. In
B. Metric Accuracy the testbed experiment, we adopt interferers to introduce
We run our experiments in a period of 30 minutes, during negatively correlated, positively correlated and uncorrelated
which reception reports (0.9% overhead) are used to fresh link links. We investigate the effect of various correlation degrees
quality and P r(Es,f1k , . . . , Es,fkk ) values. Figure 6 compares on the performance of our design.
the real values with the estimates using our link correlation
metric α0 and the link independent metric ETX. From Fig- i

ure 6, we find that α0 is accurate over time and the number


of received packets by the forwarder set (during 20s) closely
follows the number of sent packets (i.e., 100) divided by α0 .
s d
From Figure 6, we also find that the link independence
metric always overestimates the number of received packets.
This is because in the testbed environment, the links are
positive correlated because of cross-technology interference i

and correlated shadowing [8]–[10]. Under such an environ-


ment, the forwarders in the candidate set lose similar packets. (a) Physical deployment (b) Logical topology
With the link independence assumption, when one forwarder
Fig. 7. The testbed with 24 MICAz nodes.
fails to receive a packet, others may have a better chance of
receiving it. This optimistic view thus leads to an overestimate
of diversity benefit and transmission efficiency.

C. Metric Embedding
This section describes how we integrate the link correlation -1 0 1
aware metric E(s, F, d) into OR to select the candidate for-
(a) Negative correlated (b) Uncorrelated (c) Positive correlated
warder set. The key idea is to select candidates with good link
quality and prioritize them according to the E(s, F, d) value. Fig. 8. The interference pattern which is used to generate link correlation.
The design is specified by the pseudo code in Algorithm 1.

1536-1276 (c) 2013 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See
http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI
10.1109/TWC.2014.2329568, IEEE Transactions on Wireless Communications
IEEE TRANSACTIONS ON WIRELESS COMMUNICATION, 7

A. Generating Correlated Links correlated forwarders are selected. That’s because when one
We deploy 24 MICAz nodes in our indoor 24 feet by forwarder fails to receive a packet, the other is very likely
8 feet testbed, as shown in Figure 7(a). It forms a simple to receive it under negative correlated link correlation. We
two-hop network as shown in Figure 7(b), where we have also find that the improvement decreases when the correlation
one source s, one destination d, two interferers, and a set level increases and there are no improvement at all when the
of 20 candidate forwarders. The two interferers are placed links are perfectly correlated. That’s because when the packet
randomly within a pool of forwarders. They are used to receptions of the candidate are highly correlated, link quality
create interference of various patterns. Figure 8 shows selected becomes the only factor affecting the selection process, and
examples of interference patterns which may further lead to no diversity benefits can be exploited.
different degrees of link correlation.
5
We use the maximum transmission power, i.e., 0 dBm, to CU
100
CA Negative Correlation

Percentage of Improvement
Number of Transmissions
CA Negative Correlation CA Correlation 0
make sure that the links from the forwarders to the destination 4 CA Correlation 0 80

d are perfect. For the links from the source s to the candidate 3 60
forwarders, the transmission power is controlled carefully to 2 40
introduce the packet loss in the presence of the interference
1 20
signal. The source s sends packets with 2-byte data payload
in every 0.2s. In the default setting, the source node keeps on 0
1/4 1/3 1/2 2/3 3/4
0
1/4 1/3 1/2 2/3 3/4
sending packets until the destination returns an ACK. We use Link Quality Link Quality

802.15.4 channel 26 to avoid the effect of Wi-Fi interference. (a) Number of transmissions (b) Percentage of improvement

Fig. 10. The impact of link quality.


B. Performance Evaluation
In this section, we compare the performance of OR with • Varied Link qualities: In this experiment, we investigate
and without considering link correlation. We use “CA” to the impact of link quality on our design. The results are
represent the link correlation aware OR. “CU” means the shown in Figure 10 where Figure 10(a) shows the number
traditional correlation unaware OR. The size of forwarder set of transmissions needed by correlation aware and correlation
is two. In the experiment, we maintain the link quality from the unaware designs and Figure 10(b) shows the corresponding
source to candidate forwarders to be almost the same. In the improvement percentage introduced by our correlation aware
correlation unaware OR, two forwarders which are the most design. From the figures, we find that the correlation aware
positively correlated are selected. In correlation aware OR, scheme obtains significant improvement when link quality is
we select forwarders with different degrees of link correlation. low. For example, the link correlation aware OR obtains the
The experiment results are the average values taken from 1000 best performance – an improvement of 38% under 1/4 link
samples. In the following, we show the performance results of quality. In the environment where we introduce independent
the correlation aware and unaware design on the number of interferences, we observe that the performance gain of our
transmissions, energy consumption, and delivery ratio. design is still significant, and its performance gain is lower
than the negative link correlation scenario.
(" !%% )
*+,-)./01+23)%45
Percentage of Improvement

$" 15
(% 1.0
CU
Eenergy consumption (mJ)

'" 12 CA
'% 0.8
)*51/

&"
Delivery ratio

9
&% 0.6
%"
6 0.4
!" #%
3 0.2
" % )

! "#$ " "#$ ! ! !"# !"$ % !"$ !"# ! 0 0.0


)*++,-./0*12)*,33040,1/ Correlation Level Send Receive CU CA
(a) Correlation distribution (b) Performance improvement

Fig. 9. The impact of correlation level. Fig. 11. Energy Consumption. Fig. 12. Delivery Ratio.

C. Results on Transmissions D. Results on Energy Consumption


• Varied Correlation Levels: We start off by showing the Figure 11 plots the experiment results of the correlation
impact of link correlation degree on OR. To focus on the aware and unaware designs on energy consumption from both
effect of link correlation, link quality is kept constant at 0.5. sending and receiving aspects. From the figure, we can see
Figure 9(a) plots the distribution of pairwise link correlations that on average the sender takes 11.34 mJ, including Clear
in the presence of the interference pattern of Figure 8(a). Channel Assessment (CCA) and transmitting operations, to
The achievable performance gain of the correlation aware deliver one packet to the destination with correlation unaware
OR is shown in Figure 9(b). From the figure, we find that OR. With our correlation aware design, this part energy con-
OR obtains the maximum 33% improvement, when negatively sumption deduces to 8.13 mJ, which thus saves 28.3% energy

1536-1276 (c) 2013 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See
http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI
10.1109/TWC.2014.2329568, IEEE Transactions on Wireless Communications
IEEE TRANSACTIONS ON WIRELESS COMMUNICATION, 8

consumption. At the receiver side, the energy consumption B. Simulation Setup


for CU is 12.33 mJ while it’s 8.90 mJ for CA. This part We randomly generate different network topologies using
energy consumption includes listening and receiving. Our the Waxman model [27], where the nodes are uniformly
design improves 27.8% energy efficiency. distributed in the plane and edges are added according to the
probability that depends on the distance between the nodes.
E. Results on Delivery Ratio The network size is 50. We consider both the single-hop
and multi-hop scenarios. The logical topologies are shown in
In the default setting, we retransmit as many packets as
Figure 13, where a sender s has n initial candidate forwarders
possible until the source node receives an ACK from the
to the destination d. The sender s transmits 10,000 packets to
destination. In this experiment, we limit the maximum number
the destination d.
of retransmissions to three. The experiment result is shown in
OR obtains significant benefit when link qualities are low.
Figure 12. From the figure, the reliability of CA, and CU
That’s because when link quality is high, the forwarder set is
is 100%, and 89%, respectively. With the packet reception
not so necessary and its function is almost the same as the best
correlation information, our design helps opportunistic routing
link. The performance of OR is quite close to the traditional
find the suitable candidate forwarders and save the number of
shortest path routing under such scenarios. In the following
retransmissions.
simulations, we mainly focus on low link quality scenarios.
Both the links from the source to the forwarder set and the
VII. S IMULATION E VALUATION links from the forwarder set to the destination are set to be
In this section, we evaluate our link correlation aware design lossy. In our experiments, the default size of the candidate
in simulations with various network settings. forwarder set is two.

10 15
A. Link Correlation Generation Model
14
Number of transmissions

Number of transmissions
9
We generate correlated links using the sampling algorithm 13

for correlated Bernoulli random variables, described in [25]. 8


12

The inputs of the algorithm are the mean and the covari- 7 11

ance matrix of the joint Bernoulli distribution. It then uses 10


6
a dichotomized multivariate Gaussian distribution to sample 9
CU CA CU CA
the multivariate Bernoulli distribution. We need to choose
the covariance matrix carefully since it cannot be always (a) Single-hop scenario (b) Multi-hop scenario
associated with a valid Bernoulli distribution. We thus use the
algorithm in [26] to obtain the closest admissible matrix. The Fig. 14. Main simulation results in single-hop and multi-hop scenarios.
algorithm converts the inadmissible matrix using an iterative
projection algorithm into the closest unique admissible matrix
in the Euclidean norm. C. Main Performance Results
Link correlation can be either positive or negative depending The main experimental results in both single-hop and multi-
on the possible cause that affects the inter-link reception. To hop scenarios are shown in Figure 14. The box plot in
accommodate such correlations, we create a correlation matrix Figure 14(a) shows the experiment result in the single-hop
with the covariance matrix randomly selected from [−1, 1]. scenario where the average number of transmissions with CA
We generate a string of 10,000 sequences for each sender is 7.05, which is much less than CU’s 8.06. The CA design
to capture the link correlation. We then run simulations on obtains the improvement because it replaces the candidates
OMNeT++ and the Castalia Framework using the generated selected by previous link independent metric when it finds
correlated traces to sample transmission success or failure more diversity benefits can be exploited. The performance
events. of the two algorithms in the multi-hop scenario is shown in
Figure 14(b), where a similar result, i.e., a 15% improvement,
f1 f1 fi+1 is observed.

D. Impact of Candidate Set Sizes


In this experiment, we examine the performance of our
s d s d
design with different candidate set sizes. We investigate the
cases with 2, 3, and 4 candidates. The results are shown in
Figure 15. From Figure 15(a) we can see that increasing the
fn fi fn size of the candidate set would improve the performance of OR
with or without correlation awareness. This happens because
(a) Single-hop (b) Multi-hop the more links we have, the more diverse we can exploit. From
Fig. 13. The topologies used to investigate the effect of link correlation Figure 15(b), we find that the percentage of improvement of
awareness: (a) Single-hop and (b) Multi-hop. our design is reduced when we add more nodes since little
improvement room is left when we have enough candidates.

1536-1276 (c) 2013 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See
http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI
10.1109/TWC.2014.2329568, IEEE Transactions on Wireless Communications
IEEE TRANSACTIONS ON WIRELESS COMMUNICATION, 9

Empirical CDF Empirical CDF [11] W. Lou and J. Wu. On reducing broadcast redundancy in ad hoc wireless
1 1
networks. IEEE Transactions on Mobile Computing, 2002.
0.8 0.8 [12] Q. Huang, C. Lu, and G. Catalin Roman. Spatiotemporal multicast in
sensor networks. In SENSYS, 2003.
0.6 0.6
[13] D. Ganesan, R. Govindan, S. Shenker, and D. Estrin. Highly-resilient,
F(x)

F(x)
0.4
CU Select MAX 2
CA Select MAX 2 0.4 energy-efficient multipath routing in wireless sensor networks. SIGMO-
CU Select MAX 3 BILE, 5, 2001.
CA Select MAX 3 Select MAX 2
0.2
CU Select MAX 4
0.2
Select MAX 3
[14] S. Chachulski, M. Jennings, S. Katti, and D. Katabi. Trading structure
0
CA Select MAX 4
0
Select MAX 4 for randomness in wireless opportunistic routing. ACM SIGCOMM,
4.4 4.8 5.2 5.6 6 0 3 6 9 12 15 37(4):169–180, 2007.
x: Number of Transmissions X: Percentage of Improvement
[15] A. Khreishah, I. Khalil, and J. Wu. Distributed network coding-based
(a) The CDF of the number of trans- (b) The CDF of the percentage of opportunistic routing for multicast. In MobiHoc, 2012.
missions improvement [16] D. Koutsonikolas, C.C. Wang, and Y.C. Hu. Ccack: Efficient network
coding based opportunistic routing through cumulative coded acknowl-
Fig. 15. The impact of forwarder set sizes. edgments. In INFOCOM, pages 1–9. IEEE, 2010.
[17] E. Rozner, J. Seshadri, Y.A. Mehta, and L. Qiu. Soar: Simple oppor-
tunistic adaptive routing protocol for wireless mesh networks. IEEE
Transactions on Mobile Computing, pages 1622–1635, 2009.
VIII. C ONCLUSION [18] D. Liu, Z. Cao, J. Wang, Y. He, M. Hou, and Y. Liu. Dof: Dupli-
cate detectable opportunistic forwarding in duty-cycled wireless sensor
This paper extensively studies the impact of link correlation networks. In ICNP, pages 1–10, 2013.
on the performance of OR. We provide a detailed analysis of [19] X. Zhang and B. Li. Optimized multipath network coding in lossy
the OR framework under the influence of link correlation. We wireless networks. Selected Areas in Communications, IEEE Journal
on, 27(5):622–634, 2009.
find that diversity benefit is overestimated when we assume [20] Z. Zhong, J. Wang, S. Nelakuditi, and G.H. Lu. On selection of
that packet receptions of wireless links are independent. A candidates for opportunistic anypath forwarding. In SIGMOBILE Mob.
link correlation aware metric is thus proposed to improve Comput. Commun. Rev., pages 1–2. ACM, 2006.
[21] M. Gudmundson. Correlation model for shadow fading in mobile radio
the performance of OR by selecting the nodes with diverse systems. Electronics letters, 27(23):2145–2146, 1991.
low correlated links as forwarder candidates. We evaluate our [22] N. Patwari and P. Agrawal. Effects of correlated shadowing: Connec-
work with testbed implementation and extensive simulations. tivity, localization, and rf tomography. In IPSN, pages 82–93. IEEE
Computer Society, 2008.
The experiment result affirms the efficiency of our design in [23] S. Wang, S. Kim, Y. Liu, G. Tan, and T. He. Corlayer: A transparent link
capturing the full advantage of OR. correlation layer for energy efficient broadcast. In MOBICOM, 2013.
[24] Nouha Baccour, Anis Koubaa, Luca Mottola, Marco Zuniga, Habib
Youssef, Carlo Boano, and Mario Alves. Radio link quality estimation
ACKNOWLEDGEMENT in wireless sensor networks: a survey. ACM TOSN, 2012.
This work was supported in part by National Science [25] J.H. Macke, P. Berens, A.S. Ecker, A.S. Tolias, and M. Bethge.
Generating spike trains with specified correlation coefficients. Neural
Foundation grants CNS-0845994 and CNS-0917097. We also Computation, 21(2):397–423, 2009.
received partial support from McKnight Land-Grant Profes- [26] N.J. Higham. Computing the nearest correlation matrix—a problem
sorship and “The Shanghai Recruitment Program of Global from finance. IMA Journal of Numerical Analysis, 22(3):329, 2002.
[27] B.M. Waxman. Routing of multipoint connections. Selected Areas in
Experts”. Anas Basalamah’s work was funded by the Ful- Communications, IEEE Journal on, 6(9):1617–1622, 1988.
bright Visiting Scholar Program. Tian He is the corresponding
authors.

R EFERENCES
Shuai Wang received the B.S. and M.S. degrees
[1] S. Biswas and R. Morris. Opportunistic routing in multi-hop wire- from Huazhong University of Science and Technol-
less networks. ACM SIGCOMM Computer Communication Review, ogy, China. He is currently working toward the PhD
34(1):69–74, 2004. degree with the Department of Computer Science
[2] K. Zeng, Z. Yang, and W. Lou. Opportunistic routing in multi-radio and Engineering at the University of Minnesota,
multi-channel multi-hop wireless networks. In INFOCOM, pages 3512– Twin Cities. His research interests include big data
3521. IEEE, 2010. analysis, wireless networks and sensors, distributed
[3] C.P. Luk, W.C. Lau, and O.C. Yue. Opportunistic routing with direction- systems, network coding and related areas.
al antennas in wireless mesh networks. In INFOCOM, pages 2886–2890,
2009.
[4] Z. Zhong and S. Nelakuditi. On the efficacy of opportunistic routing.
In SECON’07, pages 441–450. IEEE, 2007.
[5] Y. Li, W. Chen, and Z.L. Zhang. Optimal forwarder list selection in
opportunistic routing. In MASS’09., pages 670–675. IEEE, 2009.
[6] H. Dubois-Ferrière, M. Grossglauser, and M. Vetterli. Valuable detours:
Least-cost anypath routing. Networking, IEEE/ACM Transactions on, Anas Basalamah is an Assistant Professor at the
19(2):333–346, 2011. Computer Engineering Department of Umm Al Qura
[7] L. Darehshoorzadeh, A.and Cerda-Alabern and V. Pla. Modeling and University. He is currently leading the Data Collec-
comparison of candidate selection algorithms in opportunistic routing. tion Research Thrust of KACST GIS Technology In-
Computer Networks, 55(13):2886–2898, 2011. novation Center at Umm Al-Qura University. He did
[8] T. Zhu, Z. Zhong, T. He, and Z.L. Zhang. Exploring link correlation for his M.S. and Ph.D. Degrees at Waseda University,
efficient flooding in wireless sensor networks. In USENIX NSDI, pages Tokyo in 2006, 2009 respectively. He worked as a
4–4. USENIX Association, 2010. Post Doctoral Researcher at the University of Tokyo
[9] K. Srinivasan, M. Jain, J.I. Choi, T. Azim, E.S. Kim, P. Levis, and and the University of Minnesota in 2010, 2011
B. Krishnamachari. The κ factor: inferring protocol performance using respectively. His areas of interest include Embed-
inter-link reception correlation. In Mobicom, pages 317–328, 2010. ded Networked Sensing, Real-world applications of
[10] A. Zubow, M. Kurth, and J.P. Redlich. Considerations on forwarder Networked Sensing, Ubiquitous Computing, Participatory and Urban Sensing.
selection for opportunistic protocols in wireless networks. EW, 2008.

1536-1276 (c) 2013 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See
http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI
10.1109/TWC.2014.2329568, IEEE Transactions on Wireless Communications
IEEE TRANSACTIONS ON WIRELESS COMMUNICATION, 10

Song Min Kim received B.S. and M.S. degrees


from Korea University, Korea in 2007 and 2009,
respectively. He is currently a PhD student in the
department of Computer Science and Engineering
at the University of Minnesota, Twin Cities. His re-
search interests include wireless network, protocols,
and embedded systems.

Shuo Guo received her B.S. in Electronic Engineer-


ing at Tsinghua University in 2006, and her Ph.D.
in the Department of Electrical and Computer Engi-
neering at the University of Minnesota, Twin Cities.
Her research includes Wireless Sensor Networks,
Vehicular Ad-Hoc Networks, and Real-time and
Embedded Systems. She received a best paper award
at IEEE MASS 2008 and has publication in many
premier sensor network journals and conferences

Yoshito Tobe is a professor at Aoyama Gakuin


University. After he obtained M.E. in electrical en-
gineering from the University of Tokyo in 1986,
he had been working on process-control local area
networks at Toshiba Corp. until 1997. He obtained
M.S.(1992) and Ph.D.(2000) from Carnegie Mellon
University and Keio University, respectively. His
research ranges from wireless sensor networks to
urban-area networks. He is a member of IEEE.

Tian He is currently an associate professor in the


Department of Computer Science and Engineering at
the University of Minnesota-Twin City. He received
the Ph.D. degree under Professor John A. Stankovic
from the University of Virginia, Virginia in 2004.
Dr. He is the author and co-author of over 150
papers in premier network journals and conferences
with over 13,000 citations (H-Index 44). Dr. He has
received a number of research awards in the area of
networking, including five best paper awards, NSF
CAREER Award, K. C. Wong Award, and McKnight
Land-Grant Professorship Award. Dr. He served a few program/general chair
positions and on many program committees in international conferences, and
also currently serves as an editorial board member for six international jour-
nals including ACM Transactions on Sensor Networks. His research includes
wireless sensor networks, cyber-physical systems, intelligent transportation
systems, real-time embedded systems and distributed systems.

1536-1276 (c) 2013 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See
http://www.ieee.org/publications_standards/publications/rights/index.html for more information.

Vous aimerez peut-être aussi