IPTV Channel Change Testing

26601 W. Agoura Rd. Calabasas, CA 91302 (Toll Free US) 1.877.FOR.IXIA (Int'l) +1.818.871.1800 (Fax) 818.871.1805 www.ixiacom.com

Test Plan

Copyright 2006 by Ixia All rights reserved Ixia 26601 West Agoura Road, Calabasas, CA 91302 (877) FOR-IXIA This Test Plan contains a general outline for testing a particular technology. Not all the capabilities of Ixia technology have been exposed in this document. Please feel free to contact us if additional capabilities are required.

IPTV Channel Change Test Plan

Contents
IPTV Channel Change Test Plan. . . . . . . . . . . . . . . 1 Background. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 Performance Metrics . . . . . . . . . . . . . . . . . . . . . . . 4 Basic Setup . . . . . . . . . . . . . . . . . . . . . . . 5 Advanced Setup . . . . . . . . . . . . . . . . . . . . 6 1. Test 1 Maximum Client Capacity . . . . . . . . . . . 9 Objective . . . . . . . . . . . . . . . . . . . . . . . . . 9 Setup. . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 Input Parameters . . . . . . . . . . . . . . . . . . . 10 Methodology . . . . . . . . . . . . . . . . . . . . . 11 Results . . . . . . . . . . . . . . . . . . . . . . . . . . 12 2. Test 2 Maximum Transaction Rate Objective . . . . . . . . . . . . . . . . Setup. . . . . . . . . . . . . . . . . . . Input Parameters . . . . . . . . . . . Methodology . . . . . . . . . . . . . Results . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 . . . . . . . . 15 . . . . . . . . 15 . . . . . . . . 16 . . . . . . . . 17 . . . . . . . . 18 . . . . . . . 20 . . . . . . 20 . . . . . . 21 . . . . . . 22 . . . . . . 23 . . . . . . 24

3. Test 3 Optimal Client Response Time Objective . . . . . . . . . . . . . . . . . . Setup. . . . . . . . . . . . . . . . . . . . . Input Parameters . . . . . . . . . . . . . Methodology . . . . . . . . . . . . . . . Results . . . . . . . . . . . . . . . . . . . . 4. Test 4 Server Resiliency Objective . . . . . . . . . Setup. . . . . . . . . . . . Input Parameters . . . . Methodology . . . . . . Results . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . 25 . . . . . . . . . . . . . . 25 . . . . . . . . . . . . . . 25 . . . . . . . . . . . . . . 26 . . . . . . . . . . . . . . 27 . . . . . . . . . . . . . . 28

2006 by Ixia

p.3

www.ixiacom.com

IPTV Channel Change Test Plan

In the context of Triple Play services delivery, IPTV represents the most signicant challenge and often the most complex to test and verify. This channel changing performance test plan assists you in comprehensively qualifying the performance of one or more Devices Under Test (DUT) or Systems Under Test (SUT) in an Internet Protocol Television (IPTV) deployment. This test plan encompasses several key IPTV performance metrics that are critical in qualifying the DUT/SUT and ensuring a successful deployment. Each of the test contained here provides comprehensive information on stress testing the DUT/SUT to ensure that is it able to perform as specied, and able to handle the typical and extreme load conditions as required. For more information, see the Performance Metrics section later in this document. The recommended testing methodology presented here is also meant to be used as a guideline for creating more case-specic testing scenarios to further characterize the performance limits of the IPTV infrastructure.

2006 by Ixia

p.1

www.ixiacom.com

IPTV Channel Change Test Plan

Background
In the telecommunications marketplace today, traditional cable and telephone operators are aggressively staging and deploying converged networks that can deliver data, voice, and video services to their rapidly expanding customer base. Such networks are referred to as Triple Play networks. The effective delivery of the video component of Triple Play services, called IPTV, has a fundamental requirement to operate within a multicast-capable network, delivering broadcast service television over the IP network by ensuring the efcient use of network resources. Using this technology, IPTV services are delivered directly to the home and are intended to replace the traditional video delivery network. The gure below is a simplied view of a multicast network used to deliver a single channel (using a single multicast address) to a number of group members (or viewers of a particular channel). The ow of data from the Group Source to the Group Members follows the multicast routing tree. A multicast routing protocol such as PIM-SM/DM is used to create an efcient routing path for the delivery of multicast packets between routers. The host to edgerouter communication is generally facilitated using IGMPv2/v3.
Group Source

Core/Aggregation Router 1
Edge Router 1
Aggregation Router 2

IP Multicast Network

Edge Router 2

Edge Router 3

NonGroup Group Member Member Member

NonGroup Member Member

NonMembers

Figure 1. Simple multicast tree used to deliver single channel multicast trafc to hosts
2006 by Ixia p.2 www.ixiacom.com

IPTV Channel Change Test Plan

What is channel changing or channel zapping? Channel changing or channel zapping in the IPTV domain is the equivalent of surng channels on a television set in the traditional sense. Channel changing, then, always entails leaving one TV channel and joining/watching the next. Why is it important? In the traditional cable television network, channels were always present on the wire. For this reason, channel changing performance was generally not an issue. Switching from one channel to the next was usually instant, typically was not a source of user dissatisfaction. For satellite TV providers, on the other hand, the inherent distance that the video signals had travel always introduced a delay in switching between channels. The usual approach to reduce the perceived delay was to provide some kind of onscreen feedback when switching channels so that the user knew the channel change request had been received. Now, with video being delivered over an IP network, one that was not necessarily designed with video transport as a key goal, several optimizations and novel methods are being deployed throughout a providers network to improve the perceived instant channel change behavior that is expected by traditional television consumers. IGMP is the industry standard protocol used for host to router communication in a multicast network. Therefore, channel changing (CC) performance measured at an edge router or an aggregation device is intricately tied to inherent implementation and ne tuning of the IGMP protocol. The CC performance is also a function of the emulated end device receiving the multicast stream for nal output, such as a set-top box. For this reason, channel changing performance is vital in characterizing performance for devices such as DSLAMS or CMTS and is equally important for an end-to-end IPTV test. For more information on the specic performance metrics attainable by using channel changing proles to test the IPTV network, refer to the Performance Metrics section later in this document.

2006 by Ixia

p.3

www.ixiacom.com

IPTV Channel Change Test Plan

What are the channel changing (CC) requirements? The channel changing performance in an IP network is primarily the result of end-to-end processing of associated multicast protocols, packet switching, and the end-devices ability to decode or show the video with relative consistency. A deployed system must be able to performance predictably in a sustained mode. A sustained mode of operation refers to a realistic load condition, not a best-performance condition with light use. In addition to sustained performance, a deployed system must be resilient to fault conditions where peak loads may easily exceed the sustained performance limits. For a system to be resilient, it must be able to perform acceptably well and be able to recover from an overloaded condition. Channel surng presents varying load conditions for a multicast network. For example, if there is a sudden power outage in a neighborhood on a typical Sunday game day, a mass ood of trafc will be injected into the IP network requesting network conguration information followed by several channel changes to tune back to a desired channel by hundreds, if not thousands, of viewers at almost the same time. Testing such realistic load conditions is critical in ensuring optimal QoS on the network and help tune device settings. Requirements of good channel performance are characterized using the metrics in the Performance Metrics section later in this document.

2006 by Ixia

p.4

www.ixiacom.com

IPTV Channel Change Test Plan

Performance Metrics
To determine channel changing performance, several performance metrics are used. The following terms are dened and used to provide objective performance. Note that the terms dened here comprise generally accepted industry terms and protocol-specic terms that help characterize the performance. IGMP Join Latency - The time between a request to join a multicast group and the receipt of the rst byte of data for a multicast group. IGMP Leave Latency The time between a request to leave a multicast group and the receipt of the last byte of data for the multicast group Channel Overlap The duration of time when data is received for a new joined multicast group and a previously left multicast group. This time is usually zero units. Channel Switch Delay (STB dependent) An internal IGMP processing delay between a Leave and Join request. This value is ideally insignicant; however, it can be otherwise. Channel Change/Zap Delay The inter-channel change delay, which is the time between a channel Leave request sent and the receipt of the rst byte of data from the new multicast channel. It is the IGMP Join Latency + Channel Switch Delay (STB dependent). This value is ideally very close to the IGMP Join Latency; however, the STB can introduce a signicant delay. The following timing diagrams outline the delays for 2 channels.
Join Latency
Multicast Channel 1

Leave Latency
CSD Join Latency

Channel Overlap
Multicast Channel 2

Join Channel 1 IGMP Membership Report (*, C1)

Leave Channel 1 IGMP Leave Group (*, C1) Join Channel 2 IGMP Membership Report (*, C2)

CSD Channel Switch Delay (STB dependent)

Figure 2. IGMP Join and Leave latency timing diagram

2006 by Ixia

p.5

www.ixiacom.com

IPTV Channel Change Test Plan

The specic metrics outlined above must be available on a perchannel/per-subscriber basis. The various metrics make up the parts of a sum that help provide an at-a-glance health check for every subscriber and the channels being watched. It is not sufcient, though, to use such metrics alone to characterize the performance of an IPTV network/device. To isolate adverse network conditions causing unacceptable channel change performance, network centric measurements must be available to provide an at-a-glance health check of the network on a per-channel/per-subscriber. Media Delivery Index (MDI) A scoring mechanism that combines packet delay variation (jitter) and media packet loss to determine the quality of the network to transport good quality video. It is measured in milliseconds. MDI:DF (Delay Factor) Dened as cumulative IP jitter. It represents the time it would take to drain an output buffer and ensure good video playback. MDI:MLR (Media Loss Rate) Dened as the packet loss rate due to dropped packets, bad/corrupted packets, or out-of-sequence packets. The Media Delivery Index (MDI) is important because it characterizes the performance of the network and its ability to handle good video streams. The index provides two measures separately so that each IPTV device can be tested with various channel change patterns to help insolate device-specic issues. By identifying problems on an IPTV network device, it becomes easy to troubleshoot and optimize the conguration to provide optimal performance end-to-end. This approach can be extended to characterize the end-to-end channel change performance of an IPTV deployment.

2006 by Ixia

p.6

www.ixiacom.com

IPTV Channel Change Test Plan

1. Test 1 Optimal Channel Change Performance

Objective This test setup focuses on determining the optimal channel change performance with IPTV trafc running across one or more DUTs. Since IPTV subscriber has a different usage pattern, it is necessary to determine the end users experience based on realistic channel change patterns. It is also important to determine how the subscribers channel change performance changes as the complexity of the subscribers behavior (channel change prole) increases and as the number of subscribers increase. The variability of the CC patterns will have a direct impact on the performance of devices supporting such trafc. The importance of determining the optimal performance of an IPTV DUT is to ensure that it is within operating limits of providing excellent channel change performance experienced by subscribers. For example, is there channel overlap when users are rapidly surng channels? How does poor jitter experienced by one video stream affect the others on the same link? How is the transport of video streams affected by a growing number of subscribers? These are just some of the questions that can be answered by emulating realistic user channel change behavior to determine various acceptable levels of performance. This test emulates typical load conditions of 1000s of subscribers with multiple channel changing proles and 100s of video streams. Several iterations of the test will help ensure that the device/network has sufcient raw bandwidth to support the load, reveal at localized congestions, and assist in ne-tuning devices for maximum performance. The maximum number of users supported by the DUT/network does not necessarily translate to optimal channel change performance. For this reason, there is a trade-off between the best channel change performance with realistic subscriber loads and the maximum performance possible by the device/network. Both metrics have merit and both must be determined. This test is applicable to an IPTV device and can be extended to characterize optimal end-to-end performance.

2006 by Ixia

p.7

www.ixiacom.com

IPTV Channel Change Test Plan

Setup The setup requires at least 2 or more Ixia test ports to generate stateful IPTV trafc. A video server port is used to stream 100s of standard and high denition video streams. The topology presented here is representative of a typical Triple Play deployment network. There are several congurations possible and this test setup is used as a guideline. The video subscribers are connected to an IGMPv3 enabled core router and the video server is connected to a carrier core router. The core multicast network has a 4Gbps backbone and it runs BGP for route determination and PIM-DM as the multicast protocol.
IXIA

IGMP Control
Subscriber Router

IXIA

LACP 4Gbit Trunk

IP Multicast Network

Carrier Router

Multicast Streams

1 3 5

Tri g Ou t Gn d

Tri

4 6

3 5

4 6

g Ou t Gn d

LM1000GBIC-2

LM1000GBIC-2

Ixia Clients Port(s)

IxLoad Video Clients

IxLoad Video Server

Ixia Server Port(s)

Figure 3. Topology for Test 1 Optimal Performance

2006 by Ixia

p.8

www.ixiacom.com

IPTV Channel Change Test Plan

Input Parameters Parameter Client Conguration Client Network Description 50 1000+ users per test port One or more VLAN with untagged and/or 802.1q tagged packets per VLAN Sequential channel surng for 10 20 sec Random channel surng for 10 sec 20 min Other random proles as per testers requirements JOIN channels as per Channel change Prole selected per run The IGMP options below may be congured as desired. Consider the specic trade-offs for each options before enabling/disabling them. Unsolicited Response Mode General Query/Group Specic Response Mode Immediate Response Suppress Reports At least one. More ports can be used to scale the test At least one 100+ video streams. Use MPEG2-TS with synthetic payload at 3.75Mbps (for SD). Use higher bitrate video streams to simulate HD streams (9.8Mbps+) Multicast group address congurable as per network design. Must have sufcient count to ensure all video streams have a multicast group address Iterative method to determine the maximum simulated users

Client Channel Change Proles

Client Command-list Client IGMP parameters

Client/Server Test Ports Video Server Video Server Conguration

Test Objective

Table 1. Input Parameters for Test 1 Optimal Performance

2006 by Ixia

p.9

www.ixiacom.com

IPTV Channel Change Test Plan

Methodology 1. Before starting the test, determine the baseline performance characteristics of the test equipment. This will ensure that the upper limits of the test equipment are known and that the performance per-port or per-system is well dened. 2. Once the baseline is established using the test ports, congure the test setup network or the IPTV DUT. To support multicast trafc, at least one switch will be required to support IGMPv2 or v3. In addition, a multicast protocol must be running to efciently deliver multicast trafc. 3. Set up the test, and congure the parameters for the protocols as outlined in Input Parameters for the Client Trafc. Choose the version of IGMP supported by the switch. Also, ensure that the IGMP parameters are using default values as specied in RFC 2236 (IGMPv2) or RFC 3376 (IGMPv3). The command-list is where the channel changing prole can be congured. The desirable options here are to run the test iteratively and modify the channel changing proles to simulate different load conditions on the multicast fabric. Some channel changing proles can include: A set of subscribers surng through channels 1-50 for 10-20 seconds each. A set of subscribers surng through channels 51-100 for 1-30 seconds each. A set of subscribers emulating a typical habit of watching channel A for 30 minutes with frequent channel surng within that time for commercial breaks. The channel-changing proles must be iteratively set up to test the multicast network and monitor an average subscribers channel change performance. Ensure that there are sufcient ports to run the test. 4. Congure the video server trafc to serve 100s of video streams. Refer to the Input Parameters to congure the stream contents appropriately.

2006 by Ixia

p.10

www.ixiacom.com

IPTV Channel Change Test Plan

5. Congure the Client and Server networks to ensure end-toend connectivity. Conrm that multicast protocol is running and reachable end-to-end. 6. Set up the test objective to determine the maximum number of sustained users. 7. Run several iterations of the test to determine the maximum performance of the DUT/network (in handling # of subscribers and # of simultaneous channels). However, consider that the maximum number of subscribers supported by the DUT/network does not necessarily translate to optimal channel change performance. There is a trade-off between the best channel change performance with realistic subscriber loads and the maximum performance possible by the network. Both metrics have merit and both must be determined. 8. Adjust the IGMP parameters to optimize channel change latency. Vendor implementations of IGMP may vary and specic optimized congurations must be determined from conguration and user guides.

2006 by Ixia

p.11

www.ixiacom.com

IPTV Channel Change Test Plan

Results This test was used to characterize various subscriber load conditions and determine the channel change performance experienced by each subscriber. In addition, network ow quality of video streams were examined per stream to determine how the device/system performed when it was incrementally loaded with more subscribers, more channels, and more complex subscriber channel change patterns. The optimal channel change performance was a trade-off between the maximum performance possible by the DUT/ network without packet loss and the acceptable channel change latencies under realistic load conditions. Both metrics have merit. Knowing the peak performance of the DUT/network ensures that extreme load conditions are serviceable during failure or startup times. However, at peak performance a subscribers channel changing performance may be less than optimal. Optimal channel changing performance was measured at an equilibrium point that was below the maximum device performance but still simulated realistic subscriber load conditions. A brief result of three runs is presented below:
Stream Bit Rate (bps) 3750007 3750007 3750007 MDI-DF (us) 2835 2835 2835 MPEG2 TS Loss 0 0 0 Inter Pkt Arrival Time (ns) 2805680 2807180 2216420 Packet Latency (ns) 38160 38320 38540 Join Latency (ms) 69 108 100 Leave Latency (ms) 8479 8460 5045

Stat Name RUN 1/ User 1 Channel 1 RUN 2/User 1 Channel 1 RUN 3/User Channel 1

MDI-MLR 0 0 0

Jitter (ns) 280 1780 300

Table 2. Test 1 channel change performance for multiple runs The rst run simulated 300 users watching 100 channels for 1020 seconds each. The second run increased the number of users to 1200. The jitter increased considerably with the increase in subscriber count. The third run reduced the subscriber count to 800 and modied the channel change behavior for different sets of subscribers.

2006 by Ixia

p.12

www.ixiacom.com

IPTV Channel Change Test Plan

If the subscribers were increased to more than 1200, there were MPEG2 TS losses seen. The optimal conguration for the network was 800 subscribers all having different channel changing proles. To further characterize the channel change performance, realtime statistics of the Channel Change/Zap Delay were actively monitored. A snippet of these real-time statistics for the third run is presented below.

Figure 4. Real-time channel change/zap delay for test 1, run 3 The average jitter distribution for several channel requests is presented below for the third run, and 99.87% of the packets had jitter below 50 us, which is very good.

Figure 5. Jitter distribution for test 1, run 3

2006 by Ixia p.13 www.ixiacom.com

IPTV Channel Change Test Plan

2. Test 2 Single Subscriber Experience

This test is composed of a series of runs with 1000s of subscribers with various channel change patterns. The test will assess the experience of a single subscriber who is watching a few channels while several other subscribers place different load requirements on the multicast network. In this way, the test setup will assess the join/leave latency and the channel change/zap delay, including MDI scores. Setup The setup will require at least 2 or more Ixia test ports to generate stateful IPTV trafc. A video server port is used to stream 100s of standard and high denition video streams. The topology presented here is representative of a typical Triple Play deployment network. There are several congurations possible and this test setup is simply used as a guideline. The video subscribers are connected to an IGMPv3 enabled core router and the video server is connected to a carrier core router. The core multicast network has a 4Gbps backbone and it runs BGP for route determination and PIM-DM as the multicast protocol.
IXIA

IGMP Control
Subscriber Router

IXIA

LACP 4Gbit Trunk

IP Multicast Network

Carrier Router

Multicast Streams

1 3 5

Tri g Ou t Gn d

Tri

4 6

3 5

4 6

g Ou t Gn d

LM1000GBIC-2

LM1000GBIC-2

Ixia Clients Port(s)

IxLoad Video Clients

IxLoad Video Server

Ixia Server Port(s)

Figure 6. Topology for Test 2 Single Subscriber Experience

2006 by Ixia

p.14

www.ixiacom.com

IPTV Channel Change Test Plan

Input Parameters Parameter Client Conguration Client Network Description 50 1000+ users per test port One or more VLAN with untagged and/or 802.1q tagged packets per VLAN 1 subscriber with xed channel change pattern Other random proles as per testers requirements JOIN channels as per Channel change Prole selected per run The IGMP options below may be congured as desired. Consider the specic trade-offs for each options before enabling/disabling them. Unsolicited Response Mode General Query/Group Specic Response Mode Immediate Response Suppress Reports At least one. More ports can be used to scale the test At least one 100+ video streams. Use MPEG2-TS with synthetic payload at 3.75Mbps (for SD). Use higher bitrate video streams to simulate HD streams (9.8Mbps+) Multicast group address congurable as per network design. Must have sufcient count to ensure all video streams have a multicast group address Iterative method to determine the optimal single subscriber performance (using maximum simulated users)

Client Channel Change Proles

Client Command-list Client IGMP parameters

Client/Server Test Ports Video Server Video Server Conguration

Test Objective

Table 3. Input Parameters for Test 2 Single Subscriber Experience

2006 by Ixia p.15 www.ixiacom.com

IPTV Channel Change Test Plan

Methodology 1. Congure the test setup network or the IPTV DUT. To support multicast trafc, at least one switch will be required to support IGMPv2 or v3. In addition, a multicast protocol must be running to efciently deliver multicast trafc. 2. Set up the test, and congure the parameters for the protocols as outlined in Input Parameters for the Client Trafc. Choose the version of IGMP supported by the switch. Also ensure that the IGMP parameters are using default values as specied in RFC 2236 (IGMPv2) or RFC 3376 (IGMPv3). The command-list is where the channel changing prole can be congured. The desirable options here are to run the test iteratively and modify the channel changing proles to simulate different load conditions on the multicast fabric. Some channel changing proles can include: One subscriber (pilot) watching a few channels for long durations. A set of subscribers emulating many channel change patterns to stress the multicast network differently. The channelchanging proles must be iteratively set up to test the multicast network to monitor a single subscribers channel change performance for the iterations. Depending on how the clients proles, ensure that there are sufcient ports to run the test. 3. Congure the video server trafc to serve 100s of video streams. Refer to the Input Parameters to congure the stream contents appropriately. 4. Congure the Client and Server networks to ensure end-toend connectivity. Conrm that multicast protocol is running and reachable end-to-end. 5. Set up the test objective to determine the maximum number of sustained users. 6. Run several iterations of the test to determine the channel change performance for a single pilot subscriber. Change the multicast load conditions by iteratively changing the other subscribers channel change patterns and monitor the pilot users Join/Leave and Change Change/Zap delays.

2006 by Ixia

p.16

www.ixiacom.com

IPTV Channel Change Test Plan

Results The objective of this test was to assess an single subscribers channel change performance under various loads by changing the channel change proles of the subscribers. The real-time nature of monitoring a single subscribers Join/ Leave latencies is essential. An instantaneous view of the realtime information of the single subscriber is presented below.
Stream Bit Rate (bps) 3750004 3750004 MDI-DF (us) 2229 2233 MPEG2 TS Loss 0 0 Inter Pkt Arrival Time (ns) 2216220 2217140 Packet Latency (ns) 38360 38020 Join Latency (ms) 100 170 Leave Latency (ms) 4503 6901

Stat Name RUN 1/ User 1 RUN 2/User 1

MDI-MLR 0 0

Jitter (ns) 198 422

Table 4. Test 2 channel change performance for multiple runs The rst run included 500 users with varying channel changing patterns. The second run increased the subscriber count to 1000. From the table above, it can be seen that increasing the subscribers and hence the load on the multicast network, increased the subscribers perceived Join/Leave latencies by 70% and 65% respectively. The channel change/zap delay observed for the single subscriber was similarly affected. The DUT was not tested for raw performance. Therefore, the behavior of a single subscriber being affected by other subscribers is primarily a result of the network having no QoS setup to police any trafc. Generally, the QoS on the network must be set up so that the single subscriber sessions are not affected by other subscribers on the same link. By monitoring a single subscriber, it is possible to troubleshoot devices that may not be observing the QoS settings or trafc policy properly. Such monitoring, however, may not be possible if the only metric observed is the optimal channel change for all aggregate subscribers.

2006 by Ixia

p.17

www.ixiacom.com

IPTV Channel Change Test Plan

3. Test 3 Triple Play Trafc

This test extends the IPTV channel change performance test by provisioning data and voice trafc on the same link as the video streams to assess the performance of this additional trafc. In a deployed network, the convergence of data, voice, and video trafc creates a realistic scenario where the video streams are sharing resources with data and voice trafc. The trafc types are very different from each other, and each requires a different level of service from the network. To ensure that each of the devices on the converged network is working properly as part of the IPTV solution, it is necessary to test several Triple Play scenarios where the channel change behavior of several subscribers creates realistic load conditions. This is in addition to the data and possibly voice trafc that must transit the same link, either from the same subscriber or other subscribers sharing an uplink from an aggregate router. By emulating various load conditions on the Triple Play network, optimizations can be made to ensure that the QoS on the devices in the network is working as expected. Setup The setup will require at least 3 or more Ixia test ports to generate stateful Triple Play trafc. A video server port will be used to stream 100s of standard and high denition video streams. A second server port will accept data and voice trafc. At least one client test port will generate stateful video, voice, and data trafc based on the conditions in the Input Parameters table. The topology presented here is representative of a typical Triple Play deployment network. There are several congurations possible and this test setup should be used as a guideline. The video subscribers are connected to an IGMPv3 enabled core router and the video server is connected to a carrier core router. The core multicast network has a 4Gbps backbone, and it runs BGP for route determination and PIM-DM as the multicast protocol.
IXIA

IGMP Control
Subscriber Router

IXIA

LACP 4Gbit Trunk

IP Multicast Network

Carrier Router

Multicast Streams

1 3 5

Tri g Ou t Gn d

Tri

4 6

3 5

4 6

g t Ou d Gn

LM1000GBIC-2

LM1000GBIC-2

Ixia Clients Port(s)

IxLoad Video Clients

IxLoad Video Server

Ixia Server Port(s)

Figure 7. Topology for Test 3 Triple Play Trafc

2006 by Ixia p.18 www.ixiacom.com

IPTV Channel Change Test Plan

Input Parameters Parameter Client Conguration Client Network Description 50 1000+ users per test port One or more VLAN with untagged and/or 802.1q tagged packets per VLAN Sequential channel surng for 10 20 sec Other random proles as per testers requirements Client Command-list JOIN channels as per Channel change Prole selected per run FTP trafc varying bandwidth to simulate typical and adverse conditions HTTP trafc varying page retrievals SIP trafc to simulate voice-over-IP calls Client Parameters The IGMP options below may be congured as desired. Consider the specic trade-offs for each options before enabling/disabling them. Unsolicited Response Mode General Query/Group Specic Response Mode Immediate Response Suppress Reports FTP client use Passive mode SIP use UDP for signaling with RTP media (codec selectable)
Continued on next page

Client Channel Change Proles

2006 by Ixia

p.19

www.ixiacom.com

IPTV Channel Change Test Plan

Trafc Impairment (optional)

Introduce trafc impairment such as latency, jitter, packet drop proles and duplicate packets (to further stress the edge devices in processing packets with errors or inherent delays/jitter than may cause queuing problems on the interface buffers) At least one. More ports can be used to scale the test At least two. A dedicated video server port and a dedicated data and voice trafc port 100+ video streams. Use MPEG2TS with synthetic payload at 3.75Mbps (for SD). Use higher bitrate video streams to simulate HD streams (9.8Mbps+) Multicast group address congurable as per network design. Must have sufcient count to ensure all video streams have a multicast group address

Client/Server Test Ports Video Server

Video Server Conguration

Test Objective

Iterative method to determine the maximum simulated users

Table 4. Input Parameters for Test 3 Triple Play Trafc

2006 by Ixia

p.20

www.ixiacom.com

IPTV Channel Change Test Plan

Methodology 1. Congure the test setup network or the IPTV DUT. For multicast trafc, at least one switch is required to support IGMPv2 or v3. In addition, a multicast protocol must be running to efciently deliver multicast trafc. 2. Set up the test, and congure the parameters for the protocols as outlined in Input Parameters table for the Client Trafc. Choose the version of IGMP supported by the switch. Also, ensure that the IGMP parameters are using default values as specied in RFC 2236 (IGMPv2) or RFC 3376 (IGMPv3). The command-list is where the channel changing prole should be congured. The desirable options here are to run the test iteratively, and then modify the channel changing proles to simulate different load conditions on the multicast fabric. Include data trafc such as FTP and/or HTTP with various page retrievals. Also, enable SIP voice-over-IP trafc (1 call per subscriber). 3. Congure the video server trafc to serve 100s of video streams. Refer to the Input Parameters table to congure the stream contents appropriately. Also congure the SIP callee and FTP server side trafc prole. 4. Congure the Client and Server networks to ensure end-toend connectivity. Conrm that a multicast protocol is running and reachable end-to-end. 5. Set up the test objective to determine the optimal subscriber channel change performance. The test Objective will vary for each of the activities dened (video, voice and data), depending on the activity and test iteration. The objectives will reect the load conditions required to effectively determine how background data and voice trafc affect an average subscribers channel change performance. 6. To test the capability of the edge routers in handling malformed packets and being able to manage its interface buffer queues properly, introduce impairment such as latency, jitter, duplicate packets or random drop packets.

2006 by Ixia

p.21

www.ixiacom.com

IPTV Channel Change Test Plan

Results The objective of this test was to determine the subscribers channel change performance on a converged network that is running video and voice trafc. Several test runs were conducted. Two of the runs are examined here. The results below show successful SIP calls during one of the test run 2 with 200 subscribers.

Figure 8. Completed SIP calls for test 3

2006 by Ixia

p.22

www.ixiacom.com

IPTV Channel Change Test Plan

HTTP transactions are presented below for the same test run.

Figure 9. HTTP transactions for test 3 The HTTP transaction latency for two runs is given below. The transaction latency increased on the second run as the FTP throughput was increased from 1.6Mbps to 24Mbps. The FTP trafc directly impacted the HTTP trafc (and the video trafc).

Figure 10. HTTP transaction latency for multiple runs A similar trend was seen on the voice calls. The per-call jitter increased an average of 18% between the rst and second run.

2006 by Ixia

p.23

www.ixiacom.com

IPTV Channel Change Test Plan

The impact of background data trafc was also apparent on the video streams.

Figure 11. Channel change/zap delay comparison between multiple runs The channel change/zap delay increased between runs as the data trafc on the link increased. In deployed systems, however, such behavior can be mitigated by designing and implementing strict QoS policies at the edge of the network. In addition, a persubscriber QoS model must be used to ensure that different trafc originating from the same subscriber is treated differently by the edge routers. This will allow predictable behavior to ensure that data trafc does not impede other more sensitive trafc such as video streams.

2006 by Ixia

p.24

www.ixiacom.com

About Ixia
Ixia is a leading provider of performance test systems for IP-based infrastructure and services. Its highly scalable solutions generate, capture, characterize, and emulate network and application trafc, establishing denitive performance and conformance metrics of network devices or systems under test. Ixias test systems are used by Network and Telephony Equipment Manufacturers, Semiconductor Manufacturers, Service Providers, Governments, and Enterprises to validate the functionality and reliability of complex IP networks, devices, and applications. Ixias Triple Play test systems address the growing need to test voice, video, and data services and network capability under realworld conditions. Ixias vision is to be the worlds pre-eminent provider of solutions to enable testing of next generation IP Triple Play networks. Ixias test systems utilize a wide range of industry-standard interfaces, including Ethernet, SONET, ATM, and wireless connectivity, and are distinguished by their performance, accuracy, reliability, and adaptability to the industrys constant evolution.

Contact Ixia

For more information, contact Ixia or visit our Web Site at http://www.ixiacom.com.

Ixia Worldwide Headquarters

Corporate Center 26601 W. Agoura Rd. Calabasas, CA 91302 (Toll Free North America) 1.877.367.4942 (Outside North America) +1.818.871.1800 (Fax) 818.871.1805 www.ixiacom.com Info: info@ixiacom.com Investors: ir@ixiacom.com Renewals: renewals@ixiacom.com Sales: sales@ixiacom.com Support: support@ixiacom.com Training: training@ixiacom.com

Ixia USA Sales

Phone: 1.866.355.4942

Email: sales@ixiacom.com Email: salescanada@ixiacom.com Email: saleschina@ixiacom.com

Ixia Canada Sales Ixia China Sales

Phone: 1.877.367.4942 Phone: +86.10.84549199

Ixia Europe, Middle East, & Africa Sales Phone: +44.1753.722056 Email: salesemea@ixiacom.com Ixia India Sales
Phone: +91.80.25633570 Email: salesindia@ixiacom.com Email: salesjapan@ixiacom.com Email: salesoceania@ixiacom.com Email: salessouthkorea@ixiacom.com Email: salesfederal@ixiacom.com

Ixia Japan Sales

Phone: +81.3.5365.4690

Ixia Oceania Sales Ixia South Korea

Phone: 1.818.292.1561 Phone: +82.11.897.1326

Ixia Federal Sales

Phone: 1.703.822.7527

 1998-2006 Ixia. All rights reserved. This publication may not be copied, in whole or in part, without Ixias consent. Ixia and its licensors retain all intellectual property rights in all products identied in this publication. Such products may be covered by one or more patents and/or pending patent applications, including but not limited to the following U.S. patents: 6,717,917; 6,408,335; 6,397,359; 6,061,725; 5,937,165; 5,881,237; and 5,838,919. All software and related documentation identied in this publication is licensed, not sold, pursuant to a separate license agreement between Ixia and the recipient. The recipients use of such software and documentation is subject to the terms of that agreement. Restricted Rights Legend Use, duplication, or disclosure by the U.S. Government is subject to the restrictions set forth in subparagraph (c)(1)(ii) of the Rights in Technical Data and Computer Software clause at DFARS 252.227-7013 and FAR 52.227-19. THIS PUBLICATION IS PROVIDED AS IS AND WITHOUT ANY WARRANTIES OF ANY KIND, EITHER EXPRESS OR IMPLIED. IXIA SPECIFICALLY DISCLAIMS ANY IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, OR NONINFRINGEMENT. THE INFORMATION HEREIN IS FURNISHED FOR INFORMATIONAL USE ONLY, IS SUBJECT TO CHANGE BY IXIA WITHOUT NOTICE, AND SHOULD NOT BE CONSTRUED AS A COMMITMENT BY IXIA. IXIA ASSUMES NO RESPONSIBILITY OR LIABILITY FOR ANY ERRORS OR INACCURACIES CONTAINED IN THIS PUBLICATION. Ixia, the Ixia four petal logo, and IxLoad are either trademarks or registered trademarks of Ixia in the United States and/or other countries. All other trademarks belong to their respective owners.