Technology

gy Evaluation for Time Sensitive

Data Transport
Report and status for subtask in JRA1
Nordunet Conference 2012, Oslo

Henrik Wessing, Task leader

Time Sensitive Transport - Agenda
Motivation and objectives
Is it worth establishing circuit based paths or do we survive with the
packet based solutions?
Technologies and layering
Delay verification through test equipment and OAM
Physical layer
OTN layer
MPLS-TP and PBT
Measurements
Conclusions and continued work
Low delay is a key property
Online music production (LOLA)
Low Latency audio and video conferencing
Delay < 35-40 ms (for network)
Jitter < 3 ms
Characterised by many small packets
Telemedical surgery Picture from presentation by Claudio Allochio, GARR

Roundtrip delay < 150 ms

Gb/s bandwidth due to no compression
Interconnection of data centres
NREN operating as network for DC.
Banking
Stock exchange algoritms can utilise reduction of delays
in msec and usec range
Not a clear NREN task!
According to The Telegraph:
that a one millisecond advantage could be worth up to
$100m a year to the bottom line of a large hedge fund.
And more and more services emerges

Delay tolerant Delay sensitive

R emote backup
ve
Err or Sensitiv

3D streaming
Bandwidth

Grid computing
Internet surfing with Thin Clients
video
Tele Health Variety of services
Internet surfing
B

Email
Mobile gaming Different requirements

Ultra HD TV End to end QoS in focus

Errorr tolerant

Stereoscopic TV
ndwidth

(U)H D Video Stream Video Conferencing

Delay key issues
Video Streaming IP Telephony
Ban

Home monitoring Online D istributed

Environments
Technologies investigated
CCampus networks
t k / Metro
M t networks
t k
Traditionally packet based (L3/L2) depending on operational use
Core / NRENs (GEANT)
Packet
P k t or circuitbased
i itb d
Methodology
Layered approach from physical layer and up
Qualifying and quantifying delay parameters

Service (IP)

Layer 2 (ETH / MPLS-TP)

L1 Subwavelength
S b l th layer
l (OTN - ODUk)

L0 DWDM layer
Measuring delay External test or OAM
Delay measurements using external test equipment
Delay measured between Maintenance association End Points
Different levels acoording to domain

From MEF 35
OAM Delay and loss statistics
Metro Ethernet Forum specifies delay statistics (MEF 1010.2)
2)
Frame delay (range/mean/jitter) - histograms
Frame Loss Ratio - counters
Availability
One way FD utilising timestamps and sequence numbers (MEF 35 IA)
Real data or as added synthetic frames
Synchroneous clocks required (or estimated from two way FD)
Performance Monitoring solutions
Choice depending on single or dual ended
Synthetic frames must match real frames
Communication messages
Delay Measurements Message (DMM)/ Delay Measurement Response (DMR)
Results
R lt stored
t d in
i bi
bins ffor each
h measurementt interval
i t l

PM MEG Measurement PMFunction(s) Mandatory

Solution Type(s) TechniqueforLoss orOptional
pointtopoint Synthetic SingleEndedDelay
PM1 Mandatory
multipoint Testing SingleEndedSyntheticLoss
pointtopoint
PM2 n/a DualEndedDelay Optional
multipoint
Countingg
PM 3
PM3 pointtopoint
i tt i t Si l E d d S i L
SingleEndedServiceLoss O ti
Optional
l
ServiceFrames
Dedicated test equipment

Agilent (now Ixia) N2X test solution

Main advantage: You define which packet sizes and load to test with
RFC2544 compliant
Measuring loss, signal power, delay, jitter etc.
From services to fibres
Physical layer (L0)
Physical layer (L0)
Propagation delay in fiber. Well
defined.
1 km approximately 5 usec
Delay in modulation formats
depending on technology in sub usec
range
g
QPSK,QAM, OFDMA

For medium to long range applications:

Propagation is the only significant L0 delay
Optical Transport Network (OTN) L1
Physical Media Independent layer (L1) - general
Mapping from higher layer to L1
Forward Error Correction
OTN
Basically a technology to replace SDH/SONET better suited for
ethernet transport.
Timestamping for Delay Measurement Message and Delay
measurement reply (DMM/DMR)
Vendor support: All major vendors support ODU switching
OTN (II) Measurement scenarios
Lab measurements using Alcatel Switches Loops in Delay contribution
Triangular setup triangle
1 1 GFP mapping and demapping
CPH - CPH -HAM
p in triangle
Loops g to estimate delays
y from Total fibre propagation delay
mapping and FEC processing 2 x OTN switching latency
Expected delays
usec range (OTN mapping and processing) 2 1 GFP mapping and demapping

Results
Res lts using
sing external
e te nal N2X teste
tester and OAM T t l fibre
Total fib propagation
ti d l
delay
information
Triangle fibre propagation delay

5 x OTN switching latency

N2X tester OTN switch 3 1 GFP mapping and demapping

Total fibre propagation delay

2 x Triangle fibre propagation delay

8 x OTN switching latency

OTN switch OTN switch

L2 MPLS TP and PBT Packet handling
MPLS TP
Two flavours: MPLS-TP and T-MPLS
No signalling protocol
Legacy Ethernet can be encapsulated in PW transported over MPLS LSP
Delay contributions
Label processing
Store and forward or wire speed as no CRC FCS
FCS

Lower priority packet may delay if in process FCS Payload

=
Supported mainly by ALUALU, Cisco
Cisco, Ericsson Payload
802.1ad
frame

PBT FCS
Payload
with or
without
FCS
Ethernet for transport purposes
TPID

C-TAG I-TAG 32 bits

VLAN -> Q-in-Q -> MAC-in-MAC
Payload
TPID TPID TPID 16 bits

Delay
D l contributions
t ib ti Q-TAG S-TAG B-TAG 16 bits
TPID TPID TPID TPID 16 bits
MAC table lookup SA SA SA B-SA 48 bits
Store and forward due to CRC DA DA DA B-DA 48 bits

Dependent on packet length

Lower priority packet may delay if in process
Supported mainly by Ciena, NSN and Huawei,
 PBT Lab scenarios

Simple PBT
Delay in 1, 2, 3, 4 and 5 switches
PBT sswitch
tc Results next p
page
g
PBT switch

N2X tester PBT switch

PBT switch
PBT switch

PBT over OTN

Mapping between OTN and PBT
- Crossconnect version
- Terminate in PBT version
PBT measurements Simple PBT
140,00
64bytes
120,00
512bytes
100,00 1500bytes

80,00

60,00

40,00

20,00

0,00
0 1 2 3 4 5

Delay in usec depending on number of PBT switches

For Jumbo frames of 9000 appr.
appr 80 usec per node (400 usec for 5 nodes)
Clear dependence on packet length
Processing max ethernet size packet equals 4-5 km transmission!!
For Jumbo: ~16
16 km transmission!
Dependence on load
All numbers in usec
Jumbo frames included only 25
in text
Vendor specific 20 64

Graph: Delay dependent on

512

1500
load 15

Delay
Load 64 512 1500 9000 10
10 9,203 13,77 22,741 82,785
20 9 208
9,208 13 779
13,779 22 743
22,743 82 787
82,787
30 9,221 13,784 22,748 82,781 5
40 9,214 13,766 22,74 82,779
50 9,232 13,781 22,732 82,773
60 9,233
, 13,781
, 22,732
, 82,776
, 0
70 9,325 13,783 22,746 82,787 10 20 30 40 50 60 70 80 90 100
80 9,439 13,795 22,74 82,791
90 9,561 13,818 22,746 82,775
100 10,157 14,742 23,602 83,554
Measurements with Transpacket H1 (L1/L2)
TRANSPACKET H1 - Add-drop muxponder
Allows wavelength-grade Quality of Service
(QoS).
Ultra-low latency, ultra-low latency variation
Combined circuit and packet switching
Including statistically multiplexed (SM) BE traffic
Scenario including
Three
Th ee Transpacket
T anspacket switches
s itches in series
se ies
Test traffic on one or more ports from Spirent test
equipment
g
Guaranteed traffic originatingg from iperf
p
Latency of SM varies heavily on load and GST traffic
Stress the importance of high priority
Conclusion and continued work
Subtask within GN3 JRA1 to identify and provide guidelines for choice between packet or circuit
based transport
Relevant application
LOLA Online entertainment production
Telemedical applications
Banking
Common understanding of the different layers achieved
Delay assessment using test equipment or inline OAM functions.
Methodology -> From L0 -> L3
Physical layer
OTN transport
MPLS-TP or PBT -
Measurements on PBT showing high dependence on packet lenghts.
lenghts
Work to be done in future
Conduct similar measurements with MPLS-TP
Include OTN in the loop
Verify PBT measurements with another vendor
Application on top LOLA?