Deliverable D3.2 Technologies, Implementation Framework, and Initial Prototype (Initial Version)

Seventh Framework STREP No. 317846 D3.
2 Technologies, Implementation Framework,

and Initial Prototype
Commercial in Confidence

Version 0.8 Page 1 of 139
Copyright 2014, the Members of the SmartenIT Consortium

Socially-aware Management of
New Overlay Application Traffic with
Energy Efficiency in the Internet
European Seventh Framework Project FP7-2012-ICT- 317846-STREP

Deliverable D3.2
Technologies, Implementation Framework, and
Initial Prototype

The SmartenIT Consortium

Universitt Zrich, UZH, Switzerland
Athens University of Economics and Business - Research Center, AUEB, Greece
Julius-Maximilians Universitt Wrzburg, UniWue, Germany
Technische Universitt Darmstadt, TUD, Germany
Akademia Gorniczo-Hutnicza im. Stanislawa Staszica w Krakowie, AGH, Poland
Intracom SA Telecom Solutions, ICOM, Greece
Alcatel Lucent Bell Labs, ALBLF, France
Instytut Chemii Bioorganicznej PAN, PSNC, Poland
Interoute S.P.A, IRT, Italy
Telekom Deutschland GmbH, TDG, Germany


For more information on this document or the SmartenIT project, please contact:

Prof. Dr. Burkhard Stiller
Universitt Zrich, CSG@IFI
Binzmhlestrasse 14
CH8050 Zrich
Switzerland

Phone: +41 44 635 4331
Fax: +41 44 635 6809
E-mail: info@smartenit.eu
D3.2 Technologies, Implementation Framework,
and Initial Prototype Seventh Framework STREP No. 317846

Page 2 of 139 Version 0.8

Document Control

Title: Technologies, Implementation Framework, and Initial Prototype
Type: Public
Editor: George Petropoulos
E-mail: geopet@intracom-telecom.com
Author(s): George Petropoulos , Antonis Makris, Sergios Soursos, Lukasz Lopatowski,
Jakub Gutkowski, Thomas Bocek, Daniel Donni, Patrick Gwydion Poullie,
Guilherme Sperb Machado, Piotr Wydrych, Zbigniew Dulinski, Grzegorz
Rzym, David Hausheer, Fabian Kaup, Jeremias Blendin, Frederic Faucheux,
Ioanna Papafili, Gerhard Hasslinger
Doc ID: D3.2-v0.8

Amendment History

Version Date Author Description/Comments
v0.1 Sep 11, 2013 George First version, Table of contents
v0.2 Nov 28, 2013 George ToC update
v0.3 Feb 26, 2014 George, Antonis, Sergios, Lukasz,
Jakub, Thomas
Contributions to chapter 4
v0.4 Jun 6, 2014 George, Antonis, Lukasz, Jakub,
Thomas, Daniel, Piotr, Zbigniew, David,
Fabian, Jeremias
Contributions to chapter 3, 4 and 5.
v0.5 June 11, 2014 Lukasz, Antonis, Thomas, Frederic Contributions to chapter 5.
v0.6 July 2, 2014 George, Lukasz, Jakub, Patrick,
Guilherme, Piotr, Frederic
Contributions to all chapters.
v0.7 July 9, 2014 Ioanna, Gerhard Internal review.
v0.8 July 15, 2014 George, Lukasz, Patrick, Thomas, Piotr,
Fabian, Jeremias, Frederic
Internal review comments addressed, final version.

Legal Notices
The information in this document is subject to change without notice.
The Members of the SmartenIT Consortium make no warranty of any kind with regard to this document,
including, but not limited to, the implied warranties of merchantability and fitness for a particular purpose. The
Members of the SmartenIT Consortium shall not be held liable for errors contained herein or direct, indirect,
special, incidental or consequential damages in connection with the furnishing, performance, or use of this
material.

Seventh Framework STREP No. 317846 D3.2 Technologies, Implementation Framework,


Table of Contents
1 Executive Summary 6
2 Introduction 7
2.1 Purpose of the Deliverable D3.2 7
2.2 Document Outline 8
3 Architecture Overview 9
3.1 Entities 10
3.2 Interfaces 10
3.3 System Deployment 12
4 Technology Scouting 14
4.1 Programming Languages 14
4.2 Frameworks 16
4.2.1 JBoss Netty approach 16
4.2.2 OSGi-based approach 17
4.3 External interfaces libraries and tools 18
4.3.1 Social Monitor 18
4.3.2 Cloud Manager 20
4.3.3 QoS monitor 22
4.3.4 QoE monitor 27
4.3.5 Fixed/Mobile Network Traffic Manager 28
4.4 Internal interfaces protocol formats 32
4.4.1 Google Protocol Buffers 32
4.4.2 Apache CXF Distributed OSGi 33
4.4.3 JSON 33
5 Implementation Framework 34
5.1 Tools 34
5.1.1 Specification Language 34
5.1.2 Programming Languages 34
5.1.3 Libraries 35
5.1.4 Tool chain 36
5.1.5 Testbed 36
5.1.6 Version Control 37
5.1.7 Issue Tracking 38
5.1.8 Continuous Integration Server 38
5.2 Procedures 39
5.2.1 System Release Process 39
5.2.2 Development Process 40
5.2.3 Roles 40
5.2.4 Meetings 41
5.3 System tree layout 41
6 Entities 46
6.1 S-Box 46
6.1.1 Network Traffic Manager 46
6.1.2 Quality of Service Analyzer 58
6.1.3 Economic Analyzer 67
6.1.4 Database 73
6.1.5 Inter-SBox Communication Service 84
6.1.6 SBox-SDN Communication Service 87
6.1.7 User Interface 90
6.2 SDN Controller 108
6.2.1 SBox-SDN Server Interfaces 108
6.2.2 SDN Controller based on Floodlight 0.90 110


7 Summary and Conclusions 117
8 SMART Objectives Addressed 119
9 References 121
10 Abbreviations 125
11 Acknowledgements 127
Appendix A. History of system releases 128
A.1. Version 1.0 128
Appendix B. Internal Interfaces Protocol Format 136
B.1. Reference and Compensation Vectors JSON Format 136
B.2. Configuration Data JSON Format 136



(This page is left blank unintentionally.)


1 Executive Summary
The aim of Deliverable 3.2, entitled Technologies, Implementation Framework, and Initial
Prototype (Initial Version), is to describe the technology scouting, justify the technology
selections made, define the implementation framework and finally, describe the design
and development of the SmartenIT prototype for the first system release.
Deliverable 3.2 is the outcome of Task 3.2 Technology Scouting and Definition of
Implementation Framework, and includes first results of Tasks 3.3 Development of
Prototype Components and 3.4 Implementation of Selected Application, which will
produce and document the final version of the SmartenIT prototype and application
towards the end of the project.
Its the first outcome of the engineering aspect of Work Package 3 Architecture Design
and Engineering, providing technical details on the selected implementation framework
and the documentation of the first release of the SmartenIT software.
Hence, the goals of this deliverable are the following:
The summary of the technology scouting, including all the proposed technologies,
libraries and tools, which could be used to setup the implementation framework,
support the development of the SmartenIT prototype components and provide
and/or extend required existing functionalities.
The justification of choosing the final technologies and tools to be used throughout
the SmartenIT implementation, through a selection process which provides the pros
and cons of each approach, the prototypes requirements as expressed by the
mechanisms specification and, finally, the selection criteria.
The mapping of the latest SmartenIT architecture entities, components and
interfaces to what was actually implemented in the first release of the SmartenIT
software, as well as their deployment diagrams.
The definition of the implementation framework, including the methods, documents
and tools to support the development and integration process, as well as the
generated system tree, libraries and build commands to produce the release
artifacts.
The design and implementation details of each component, including their
interfaces to other components, sequence and class diagrams, and tests to ensure
that they operate as expected.
To conclude, this deliverable can stand as the engineering and implementation
documentation of the first release of the prototype, as well as the technology scouting
report, part of which will be deployed in future releases and documented in the final
deliverable, D3.4 Prototype Implementation, Validation and Selected Application.



2 Introduction
The overall objective of the SmartenIT project is to deal with the management of traffic
generated by applications running on the Cloud. In this sense, the project offers Cloud and
Overlay traffic management solutions for generated traffic that flows over the public
Internet. The proposed solutions consider the following objectives:
a) energy efficiency,
b) incentive compatibility for all stakeholders,
c) social awareness, and
d) QoE awareness.
They aim to improve the energy efficiency of network equipment and end-user devices, to
improve the QoE for users and efficiently manage the inter-domain and inter-cloud traffic
to benefit all the involved stakeholders.
A certain number of traffic management mechanisms were described and specified in
Deliverable 2.2 [1]. The Dynamic Traffic Management (DTM) mechanism was selected by
the consortium in the respective Tasks 2.3 and 2.4, as the first traffic management
mechanism to be implemented. DTM [1] is a generic concept addressing the problem of
minimizing the ISP's costs related to the amount of traffic in the network. The DTM
mechanism performs optimization to find a best solution over criteria and conditions
defined by the ISP.
Simultaneously with the research work performed in the context of WP2, the WP3 team
set up some methods and procedures to prepare and design the implementation of the
selected specified mechanisms:
Initial architecture design, documented in D3.1 [2] was adapted to host DTM-
required functionalities and components, and will be documented in the upcoming
Deliverable 3.3.
Partners were also involved in the technology scouting, investigating all available
existing technologies and tools, which could support the development, installation
and testing of any specified traffic management mechanism, and finally, selected
the most relevant ones, based on specific criteria.
A common implementation framework was initially agreed and deployed, and
specific roles, documents, and release/development cycles were defined, to ease
the development, integration and testing of the SmartenIT software.
The aforementioned methods and tools prepared the ground for the successful
implementation of the DTM mechanism, which is the first release of the SmartenIT
software.
2.1 Purpose of the Deliverable D3.2
This deliverable focuses on the technical documentation of the implemented SmartenIT
components, which host the DTM mechanism functionalities. It provides specific details on
their supported functionalities, interfaces exposed towards other components, their


implementation design, including class and sequence diagrams as well as algorithms and
pseudocode, and finally, the local tests performed to confirm their functional behavior.
In addition, the deliverable provides information on important engineering aspects:
a) the mapping of the DTM mechanism components to the latest SmartenIT
architecture,
b) the outcome of the technology scouting and the final selection of the technologies
and tools to be used throughout the SmartenIT implementation, and,
c) the definition of the implementation framework, which includes the technologies,
tools, as well as methods, roles, meetings and documents to support and ease the
implementation effort.
2.2 Document Outline
Chapter 3 briefly describes the subset of the SmartenIT architecture entities, components
and interfaces that are required to host the DTM mechanism functionalities, and presents
its deployment diagram.
Chapter 4 is the outcome of Task 3.2, which presents all the investigated technologies
and tools that could support the development and deployment of all proposed traffic
management mechanisms, including the DTM. For all approaches, their advantages and
disadvantages, as well as their relevance to SmartenIT are presented, including the
justification of the final selection.
The agreed implementation framework is then described in Chapter 5. It includes the tools
and methodologies to support the implementation effort in general, as well as the specific
libraries required for the first release. The generated system tree and build instructions are
also presented.
Chapter 6 provides the technical documentation of each entity and each component of the
first release, including design, development and testing details.
Chapter 7 provides a brief summary of the deliverables conclusions, including decisions in
the technology scouting, the implementation framework, and the implemented
components of the DTM mechanism.
The SMART objectives, which are defined in the Description of Work [89], are addressed
in Chapter 8.
Finally, the Appendix A includes the history of system releases, including the features,
issues, and integration test cases for each release, while Appendix B presents the
SmartenIT-internal protocol formats.


3 Architecture Overview
The initial SmartenIT architecture design was documented in D3.1 [2], which was
published during the early stages of the project. Since then, the architecture has evolved
to host all specified mechanisms, with new components and interfaces. The current status
of the system architecture, which will be documented in detail in deliverable D3.3, is
depicted in Figure 1. This section provides a quick overview of the current architecture and
shows how the DTM implementation maps to it.
The figure shows all the required components of the architecture as well as the necessary
interfaces. The color-coding of the components denotes whether a component:
already exists in external systems (white components),
is required to be implemented by SmartenIT (blue components),
is required by the DTM mechanism (orange components).

ALTO
E.g. inter-ALTO
OpenFlow
E.g. SNMP,
BGP, Netflow
Inter DC/Cloud/NaDa
communication
Datacenter/NaDa/Cloud Manager
Workflow
Manager
Load
Balancer
Traffic
Redirector
Hypervisor
Overlay
Manager
Social Analyzer
Economics
Analyzer
QoS/QoE
Analyzer
Energy
Analyzer
Network
Analyzer
Traffic Manager
Cloud Traffic Manager
Fixed/Mobile Network
Traffic Manager
Inter-domain
communication
QoE Monitor QoS Monitor
Switching/
Forwarding
Energy Monitor
Topology/
Proximity
Monitor
Social Monitor
Billing/
Accounting
SDN Controller
ISCS
ISCS
OSNs
(Facebook etc.)
E.g. FB API

Figure 1: The SmartenIT architecture.


3.1 Entities
The architecture diagram is instantiated into the entities presented in Figure 2. This
diagram includes both the architectural components, and the implementation-specific
ones, used to provide support functionalities. In addition, the implemented interfaces are
also defined. The white colored components were not implemented, but used, as well as
the grey colored Network Entity interfaces.
Network Entity
S-Box
Economics
Analyzer
Fixed/Mobile Network
Traffic Manager
DB
ISCS
QoS Analyzer
QoS Monitor
Switching/
Forwarding
snmp
db
ntm1
eca1
ntm2
Sbox-SDN
ssdn
ui
UI
iscs2
ntm3
iscs1
SDN Controller
SDN
Controller
Interfaces
openflow
sdn1
sdn2

Figure 2: The DTM mechanism entities/components/interfaces diagram.
Table 3-1 presents the list of the implemented entities and components that provide the
DTM functionalities, and the respective Deliverable chapter or section in which their
implementation details are further described.
Table 3-1: List of Entities for DTM implementation
Entity Components Reference
S-Box Inter-SBox Communication Service
(ISCS), Database (DB), User Interface
(UI), SBox-SDN, Economics Analyzer,
QoS/QoE Analyzer, Fixed/Mobile
Network Traffic Manager
Section 6.1
SDN Controller SDN Controller, Interfaces Section 6.2

3.2 Interfaces
The implemented interfaces presented in Figure 2 are defined in Table 3-2, also
referencing the section in which are further described.


Table 3-2: List of Interfaces for DTM implementation
Interface
ID
Component
providing the
interface
Component
using the
interface
Purpose Reference
ntm1 Network Traffic
Manager
QoS Analyzer X Vector update Section
6.1.1.2
Manager
Economic
Analyzer
Reference vector
update
Section
6.1.1.2
Manager
Inter-SBox
Communication
Service
Reception of reference
and compensation
vectors from remote
SBox
Section
6.1.1.2
eca1 Economic
Analyzer
QoS Analyzer X and Z Vector update Section
6.1.3.2
db Database Network Traffic
Manager,
Economic
Analyzer, QoS
Analyzer, User
Interface
Access to database
tables
Section
6.1.4.2
ssdn SBox-SDN
Communicatio
n Service
Network Traffic
Manager
Configuration data,
compensation and
reference vectors
preparation to be sent
to SDN Controller
Section
6.1.6.2
ui User Interface - Access to SBox User
Interface
Section
6.1.7.2
sdn1 SDN Controller
Interfaces
SBox-SDN
Communication
Service
Configuration data,
compensation and
reference vectors
update to SDN
Controller
Section
6.2.1.2
sdn2 SDN Controller SDN Controller
Interfaces
Configuration data,
compensation and
reference vectors
update
Section
6.2.2.2
iscs1 Inter-SBox
Communicatio
n Service
Inter-SBox
Communication
Service
Reference and
compensation vector
update to remote SBox
Section
6.1.5.2
iscs2 Inter-SBox
Communicatio
n Service
Network Traffic
Manager
Reference and
compensation vector
preparation to be sent
Section
6.1.5.2


to remote SBox
3.3 System Deployment
The deployment diagram of the first release of the SmartenIT software is presented in
Figure 3. The diagram presents the devices, processes and artifacts used, to deploy an
instantiation of the DTM mechanism.
More specifically, the SBox server is an x86 server with Ubuntu 14.04 [4] installed, and
requires the installation of Java [3] and the Jetty web server [14], in which the sbox.war
artifact is deployed. The sbox.war contains all the required components (UI and DB) for
the SmartenIT web application, exposing an HTTP/HTTPS web interface, accessible by
any web browser from the administrators PC. In addition, the main.jar is the main
executable of the SBox, including all components implementing DTM mechanisms
functionalities. It uses the JBoss Netty NIO framework [9] to expose the ISCS (Inter-SBox
Communication Service) interfaces using a custom SmartenIT-protocol over TCP, and
access the SDN Controllers REST API over HTTP. The Sqlite [45] is also used as the
SBoxs database.
The SDN Controller also requires an x86 server, with Ubuntu 14.04 [4] as its operating
system, requiring Java [3] to be installed. The floodlight.jar executes all the Floodlight [35]
SDN Controllers processes, extended with SmartenIT functionalities, and the Interfaces
component, which provides the SBox-SDN communication over HTTP.
Finally, the DTM mechanism requires also a network device, which is Openflow-
compatible [32], and also supports SNMP [38]. QoS Analyzer of SBox retrieves SNMP
measurements from the latter interface, while the network device also interacts with the
exposed Floodlight SDN Controllers interface using the OpenFlow protocol.



Sbox server
main.jar
SDN Controller Server
sqlite
floodlight.jar
Jetty web
Server
HTTP(S)
DB NTM EcA
QoA
Network Device
sbox.war
ISCS
SSDN
JDBC
Interfaces
User PC
Web Browser
HTTP
TCP
(Custom)
UDP
(SNMP)
TCP
(OpenFlow)

Figure 3: DTM mechanism deployment diagram



4 Technology Scouting
In the process of SmartenIT implementation framework definition, a few alternatives were
proposed by the consortium to be adapted and used in the development of the SmartenIT
components. These suggestions concerned the programming languages to be used for
entities development, the frameworks and/or containers to support and ease the
integration of all components, the libraries and tools that could be imported and used to
implement certain functionalities, and finally, the protocol serialization formats to define
the internal interfaces between SmartenIT entities.
The advantages and disadvantages of each suggested technology are identified and
briefly described in the sections below, as well as the justification for the final selection.
The SmartenIT consortium selected certain technologies which will be adapted in all
releases of the SmartenIT software, taking into account the following criteria:
Relevance to specified mechanisms,
Lack of restrictions,
Good performance,
Ease of prototyping and small learning curve,
Open-source nature,
Partners familiarity and expertise.
4.1 Programming Languages
Certain requirements for the selection of the programming language were identified at the
beginning of the technology scouting procedure:
The SmartenIT software system tree should consist of less than 2-3 system trees,
with a common implementation framework that spans as many as possible entities,
in order to ensure efficient implementation work, minimum development and
integration effort, and a homogeneous prototype.
The latest entities/component diagram implies that a lot of components share the
same or similar functionality across multiple entities. Taking into account the
aforementioned requirement, certain entities (e.g. SBox, uNaDa, and DataCenter)
should use the same implementation framework, in order to avoid code re-writing in
multiple programming languages.
Certain entities in the SmartenIT architecture impose hardware and software
limitations. While the SBox or the Datacenter management platform do not have
such restrictions, since they are installed on network providers (NSPs) and/or
Cloud Operators infrastructure, the uNaDa is installed at the end-users premises,
and thus, requires being a low-spec machine, such as a WiFi-router. Moreover,
besides the limited hardware characteristics such devices have (e.g. minimum
memory, disk storage, low-spec processors, etc.), they also deploy limited Linux
distributions, packaged with the minimum list of libraries. The same applies to the
network device, which could be any switch or router. In the case of the end-user
device, there are 2 options; (a) a mobile device, hence the application should be


implemented using the respective native API (e.g. Java for Android [6], Objective C
for iOS [7], etc.), and (b) a standard PC, thus the application could be implemented
using any programming language.
Taking into account the abovementioned requirements, a few programming languages
and frameworks were suggested for the implementation of SBox, uNaDa, and DC entities:
C++ for its high performance both in low- and high-spec devices, Python and Java for their
availability of frameworks and ease of development. Table 4-1 presents the proposed
approaches for each programming language, including tools and libraries that could be
used in implementation and integration.
Table 4-1: Programming Languages approaches
C++ approach Python approach
Linux
C++
Boost::asio [41] (async networking library)
CppCms [42] (web application framework)
Apache [43] (Sbox, DC), nginx [44] (uNaDa)
Sqlite [45] (DB)
Google protobuf [39] (data serialization)
CMake [46] (CP/CL build tool)
Google test [47] + gcov [48] (unit test + code
coverage)
Linux
Python
Twisted [49] (async networking library + web
server)
Django [50] (web application framework)
Apache [43] (Sbox, DC), nginx [44] (uNaDa)
Sqlite [45] (DB)
unittest [51] + unittest-xml-reporting (unit tests) +
coverage (code coverage)
Java approaches
JBoss Netty approach OSGi approach [8]
Linux
Java
Netty [9] (async nio framework)
Jetty [14] (web server) + HTML + JS + Bootstrap
(UI) [52]
SQLite/ H2 [16] /Derby [17] (DB)
Maven [19] (build tool)
Junit [20] (unit testing), Cobertura [23] / Sonar
[22] (quality reporting/code coverage)
Linux
Java
OSGI container Karaf [10] (one instance per
entity)
CXF DOSGi [11] (SOAP communication
between entities)
Tomcat [13] /Jetty [14] (web server) + JSF2 +
PrimeFaces [15] (UI)
H2 [16] /Derby [17] (DB) + EclipseLink JPA [18]
(persistence)
Maven [19] (build tool)
Junit [20] (unit testing), Cobertura [23] / Sonar
[22] (quality reporting/code coverage), Pax Exam
[21] (integration testing for OSGI)

After some internal discussions and tests indicating that Java could run in a low-spec
device, such as the Raspberry Pi [5], without significant performance issues and taking
into account partners expertise and familiarity, Java [3] was selected as the
programming language for the SBox, uNaDa, and DC entities.
With regards to end-user device, either (a) a mobile application or/and (b) a PC-based
client would be implemented. The consortium agreed to initially proceed with the mobile
application development, and if mechanisms specifications require a PC-based client as


well, it would be developed in future releases. Android [6] was selected for end-user
device due to its open-source nature, ease of development and partners expertise.
4.2 Frameworks
Two candidate frameworks/containers were proposed to be the basic implementation
frameworks for SBox, and uNaDa entities, the Apache Karaf OSGi container ([8], [10]),
and the NIO client-server framework JBoss Netty [9].
4.2.1 JBoss Netty approach
JBoss Netty [9] is a framework, written in Java and integrated as a Java API. It facilitates
the development of network applications. It is asynchronous, in the sense that function
calls do not block the caller until completion. It is also event-driven, meaning that thread
orchestration, synchronization and scheduling are based on events ordering. Besides
being a skeleton for network applications development, Netty supports a wide range of
well established standard protocols (e.g. TCP, UDP, and HTTP).
A Netty application is written as a plain, simple Java application. Its components become
integrated as explicit Java packages. Development of a Netty server is similar to writing a
socket server, with some additional framework-specific characteristics.
Netty is used to implement in Java, client server applications, either in a high (i.e.
application) or low (i.e. network) layer. Netty is proven to be suitable for low latency, high
scalability applications. It can be deployed even in low-spec hardware machines as for its
memory footprint is significantly low and its asynchronous nature permits better utilization
of resources.

Figure 4: Network application development stack on top of core functionalities provided by
Netty [9]
Netty has its own configurable model for events/messages/threads handling, separating
the interfaces from the logic and easing development. Due to its asynchronous nature, it
inherently supports multi-threading and concurrent connections handling.


4.2.2 OSGi-based approach
The core part of the OSGi-based approach is the Apache Karaf [10], which is a small and
lightweight OSGi based runtime. The OSGi [8] (Open Service Gateway Initiative)
technology is a set of specifications that define a dynamic component system for Java.
These specifications mainly aim to reduce software complexity by providing a highly
modular architecture. OSGi provides an environment for the modularization of applications
into smaller bundles (components). Each bundle is a dynamically loadable collection of
classes, jars, and configuration files that explicitly declare their external dependencies (in
Manifest file). Bundles have to be deployed and run on OSGi container like Apache Karaf.
Additionally to the modularization, this approach allows developers to benefit from several
interesting OSGi/Karaf features such as module versioning (multiple versions of the same
bundle may coexist within the same Java VM), or lifecycle management (i.e. bundles can
be remotely installed, started, stopped, updated, and uninstalled without requiring a
reboot).
Apache Karaf is a broader concept that determines how an application is developed,
tested and deployed. Using Karaf is not essential for the SmartenIT prototype but its
features may simplify the development and integration processes.
Following features may be helpful for the SmartenIT development process:
Karaf as a container all components and required libraries installed in one place,
Karaf as a dependency resolver bundle state (uninstalled, installed, resolved,
active, starting, stopping) determines whether all required dependencies are
installed,
Karaf as a logging system all logs in one place,
Karaf as an integration testing platform Pax Exam [21] library enables in-
container testing for OSGi meaning that for test purposes an instance of Karaf is
run in memory (with desired bundles installed), and communication and
interoperability between bundles can be tested, as if running in a real deployment
environment,
Easy provisioning Karaf provides a simple way to provision applications or
features. The provisioning system uses xml "repositories" that define a set of
features. In simple words, Karaf enables using xml file for defining a set of
components (with required libraries, configuration and configuration files) to be
installed using a single command,
Karaf as a multi-platform system each developer can install Karaf and use it in
development process on its own operating system (e.g. Windows XP, Windows 7,
Linux, Mac OS X),
Karaf as a lightweight container - clean binary distribution of Karaf 2.3.3 requires
only up to 20 MB disk space and JDK 1.5.x or greater, moreover after starting it
uses around 64 MB of RAM (checked on Ubuntu).
The use of Apache Karaf will require from developers some additional knowledge about
OSGi and Karaf itself. However the required knowledge is proportional to the number and
complexity of Karaf features to be used. OSGi bundles will be prepared by a Maven [19]
plug-in configured in the POM file. In most cases, no manual edition of Manifest files will
be required. Communication via services adds also some complexity (e.g. the necessity to


configure the blueprint file additional configuration file). This framework also determines
how integration tests are written and executed (using Pax Exam) which requires also
some efforts.
It is assumed that the SBox and uNaDa entities will lie more on the network layer,
interacting with network entities, e.g. routers, switches and external monitoring tools. The
communication with such elements, as well as the SmartenIT entities is considered as
performance-critical. Learning curve and ease of development were some additional
criteria required by the consortium, with regards to framework selection. Taking into
account the aforementioned factors, JBoss Netty [9] was selected by the consortium
as the core framework for the SBox and uNaDa entities.
4.3 External interfaces libraries and tools
4.3.1 Social Monitor
In order to monitor social activity in existing Online Social Networks (OSN), libraries are
required to be able to communicate to OSN APIs and to collect relevant information.
Some libraries are able to communicate to multiple existing OSNs, while others are
specific or specialized to one single OSN. For the scope of the SmartenIT Social Monitor
implementation, the chosen library is required to meet the following characteristics:
Flexibility: It is possible to easily extend the library and also send custom queries to
the OSN.
Open Source: The library is published under an open source license, maintained
regularly and well documented.
API Completeness: The library supports the most important functionalities of the
underlying OSN API.
Common OSN Data Model: The library is based on a general data model,
independent of the underlying OSN.
OSN Authentication: The library supports/handles OSN authentication (e.g. oAuth
1.2 [53]).
OSN Support: The library supports at least Facebook as the underlying OSN.
4.3.1.1 Social Monitor Libraries; Pro / Cons
Spring Social, RestFB and Twitter4J are the most used libraries for Facebook [59] and
Twitter [60], and are also officially listed by these OSNs. Facebook4J is rather new, but
developed by the same developer as Twitter4J, which is well structured and considered a
stable library. All four alternatives use a data model, which is specific for the underlying
OSN and would need adapters to be compliable with other OSNs. Except of RestFB, all
other presented libraries have OSN authentication support, which is a required feature,
but also could be easily integrated by third-party services (e.g. https://oauth.io/). A
comparison of these libraries is presented in Table 4-2.
Table 4-2: OSN monitoring libraries comparison


Spring Social [53]
Facebook4J
[55]
RestFB [56]
Twitter4J
[57]
Flexibility Yes
Yes, only
custom FQL
supported
Yes, custom
FQL, custom
open graph
Yes
Open Source Yes Yes Yes Yes
API
Completeness
Yes
Yes (no open
graph)
Yes Yes
Common OSN
Data Model
No No No No
OSN
Authentication
Yes Yes No Yes
OSN Support
Facebook, Twitter, LinkedIn,
(Incubation: TripIt, GitHub),
(many community implementa-
tions of other OSN providers)
Facebook Facebook Twitter

However, there are general issues with social monitoring as discussed in the next
paragraph. Those issues include resources limitations and development issues. While the
development issues can be solved with careful planning, the rate limitation cannot be
circumvented. Thus, the social monitor and analyzer components need to be developed
with those rate limitations in mind.
4.3.1.2 Social Monitoring Issues
Rate Limitations: OSNs limit the rate on which a user can access their API
endpoints. For example, Facebook limits the API to 600 calls per 10 minutes per
access token. This becomes critical when fetching a lot of data (e.g. retrieve all
friends (600+) of a Facebook user, search tweets based on a hashtag). Developers
should consider these limitations and work with timers and counters to avoid the
temporal or even final bans of an access token.
Testing: It is not an easy task to test the implemented functionality since most
OSNs do not allow the creation of test users. Therefore, developers have to work
with real user profiles. It is important to mention that the code should be reviewed
precisely before testing, since different user profiles may present slightly different
characteristics due to the OSN continuous development (e.g., new features just
available to a portion of users).
API Reliability: OSN APIs are not perfect, many bugs exist (such as [57]) and also
response times are not guaranteed. One has to deal with this uncertainty and take
into account whilst developing.
OSN Authentication: The most difficult part is the handling of the OSN
authentication. Some of the OSNs implement oAuth 1.0 (e.g. Twitter) others oAuth
2.0 (e.g. Facebook), and also proprietary solutions are used. It is preferred to use a
third-party service such as oauth.io, which already solved this problem and is easy
to integrate with minimum effort.


If the library should support as many OSNs as possible, then Spring Social should be
chosen. If the focus lies only on Facebook and Twitter, then it is recommended to work
with RestFB and Twitter4J, which have proven by the open-source community to be
well maintained and stable. Moreover, RestFB and Twitter4J are easy to use and they
support almost the complete underlying OSN API.
4.3.2 Cloud Manager
CloudStack [61] and OpenStack [62] provide an open source implementation of a fully
featured cloud software stack. Although both come with deployment features, as
discussed in Section 4.3.2.3, CloudStack is more stable and easier to deploy, but
OpenStack is more scalable. Since OpenStack was initiated by Rackspace and NASA and
is actively supported, deployed and developed by many other big stakeholders in the field,
such as SUSE Linux, Dell, Canonical, Citrix Systems, Hewlett-Packard, AMD, Intel, Red
Hat, and IBM, OpenStack's deployment and code quality rapidly increases.
Furthermore, as Section 4.3.2.4 discuses, CloudStack offers mechanisms that may ease
the deployment of the Cloud Manager in the case of cloud federation, which is a major
focal point of SmartenIT. Since SmartenIT aims to provide cloud mechanisms for fair
resource allocation, it is also highly important, that the chosen cloud software stack comes
with appropriate resource monitoring functions. Here OpenStack clearly outpaces
CloudStack, as discussed in Section 4.3.2.1. While CloudStack may outpace OpenStack
with respect to the ability of resource reallocation, these capabilities are constraint to
certain hypervisors, as discussed in Section 4.3.2.2. Further, if reallocations are
implemented on the hypervisor level, also OpenStack is highly capable.
4.3.2.1 Specific APIs for Resource Monitoring
To decide on resource allocation, an allocation mechanism needs information about the
current (and possibly past) resource utilization of VMs. OpenStack provides this
information via the Ceilometer component which is addressable through an API and highly
customizable.
CloudStack provides basic usage information through its dashboard. Further, the optional
component "cloudstack-usage" provides different access methods including an API, to
infer utilization data. However, this data is accessible only at user account level, which
means that, contrary to OpenStack, it is not possible to get necessary usage information
at VM level.
4.3.2.2 Reallocation
CloudStack provides advanced capabilities to reassign resources to VMs by an API, which
allows reallocations even for running VMs. Although CloudStack has an edge over
OpenStack, the functionality is only accessible in combination with the VMware or
XenServer hypervisor.
While OpenStack does not provide such fine grained resource reallocation features
natively, they can be implemented for many open source hypervisors. This means,
although CloudStack outruns OpenStack in terms of out-of-the-box resource reallocation
features, it is possible to implement the same functionality for OpenStack. Although
OpenStacks reallocation capabilities have therefore be implemented for hypervisors


specifically, CloudStack also only supports the functionality for two hypervisors out of the
box.
A way to implement resource reallocations in OpenStack, independent of the hypervisor,
would be by VM resizing. In particular, OpenStack provides VMs with predefined resource
sets, which are called flavors. Resources could therefore be reallocated by creating
customized flavors and resizing VMs to these as needed. The creation of flavors and
subsequent migration of a VM to the new flavor is handled by the component Nova and is
also addressable through an API. However, it requires restarting VMs and is therefore not
very flexible. Thus, although the resizing approach would virtually support every
hypervisor, it is planned to implement the reallocation functionality on hypervisor level.
4.3.2.3 DevStack; Pro / Cons
As mentioned, CloudStack outpaces OpenStack in terms of deployment ease. To address
this issue, the community created the DevStack [63] distribution of OpenStack. It is a
quick and (as described by the creators themselves) dirty way of deploying OpenStack.
The only goal is to setup OpenStack as quick as possible, hence all components are
initialized with default values and security is of no concern. Therefore, it is highly
discouraged to use OpenStack installed by DevStack as production environment.
However, DevStack will serve for initial deployment/test purposes well and mitigate the
deployment efforts compared to CloudStack.
4.3.2.4 Deployment
There are four available mechanisms to logically group an OpenStack deployment:
Regions, Cells, Availability Zones and Host Aggregates.
The first two mechanisms are used to control multiple cloud sites i.e. datacenters. Cells
rely on a hierarchical tree-like structure with only one master node to organize
datacenters. If datacenters are organized as Regions, each datacenter has its own master
node and its own API which allows better separation. The mechanisms may in particular
become relevant, when cloud federations are considered: as organizing datacenters by
regions allows for more heterogeneous cloud federations.
The other two mechanisms serve to organize a single master node and its corresponding
set of cloud services. Availability Zones allocate compute and storage nodes to logical
groups and allow for some form of physical isolation and redundancy when scheduling
VMs. Host Aggregates introduce the possibility of attaching additional meta-data (e.g.
"SSD_disks" or "Intel_AVX") to nodes such that they can be grouped to further ease
scheduling decisions.
The most favourable architecture for a multi-datacenter deployment would be based on
Regions to manage the different sites and Availability Zones to group nodes within
Regions as shown in Figure 5.


Figure 5: Multi-datacenter deployment architecture [62]
4.3.2.5 Cloud Manager Decision
As community support is a good indicator for future-proofness of software and OpenStack
obviously boasts this characteristic, OpenStack will be deployed by SmartenIT as it
clearly rules out CloudStack in terms of active contributors and enterprise support. Since
KVM is OpenStack's standard hypervisor and offers a lot of monitoring functionalities, and
is widely deployed, it will be deployed as the hypervisor. Table 4-3 lists the feature
highlights of the two alternatives sorted by relevance.
Table 4-3: Feature highlights of the two overlay manager alternatives
OpenStack CloudStack
Support by many big companies Stability
Rapidly growing deployment Ease of deployment
Ceilometer
4.3.3 QoS monitor
Depending on the network size and topology, network equipment and other aspects like
implemented Traffic Management Solution (TMS) specific requirements, network
monitoring information may be collected in different ways and from several types of
sources. Those may be roughly categorized in three sets: network devices, basic network
performance monitoring tools and more sophisticated and/or complex monitoring systems.
Typically network devices can be directly queried by means of SNMP [38] in order to
obtain information stored in MIBs. However in order to obtain any other specific monitoring
data form a device any of the implemented management protocols can be used to
connect to the device. Those would include among others: SSH [83], TL1 [84] and
NETCONF [37].


Selected measurement tools and monitoring systems that could be potentially used in the
SmartenIT prototype development are presented in the following sections. In the last
paragraph of Section 4.3.3.4 we present the final selection of tools and the reasoning
behind it.
4.3.3.1 Basic system monitoring tools
The utilization of entities in the SmartenIT architecture is required for the optimization of
service and content placement decisions. Therefore, a number of utilities and libraries
exist, simplifying the task of gathering the utilization of the different devices and device
types in a unified and platform independent way. These metrics are required for the model
based power estimation of participating devices, reducing the overall power consumption
to a minimum. The tools considered for deployment are listed in the following table.
Table 4-4: System utilization measurement tools
Tool name Available Metrics Language Platform Deployment
SIGAR [64] System utilization (CPU, RAM,
HDD, network, )
Java Multi-platform uNaDa or hosts
psutil [65] System utilization (CPU, RAM,
HDD, network, )
Python Multi-platform uNaDa or hosts

As the selected language is Java, the preferred system monitoring library is SIGAR.
4.3.3.2 Basic network measurement tools
A set of basic network monitoring tools covers most or even all network metrics that are
foreseen to be potentially collected and utilized by the implemented TMS. These include
packet delays, loss, data throughput, device/interface availability and others. The list of
selected example tools and their basic characteristics is provided in the following table.
The SmartenIT software would need to be able to interface with these tools to trigger
measurements and collect results. The way this would be done depends on the selected
tool(s) to be used and where given tool is deployed.
Table 4-5: Network measurement tools
Tool name Available metrics Deployment
Ping connectivity, packet loss, round-trip time On network devices or hosts
Traceroute path, round-trip time On network devices or hosts
Iperf [26] throughput, packet loss (UDP), and jitter Client-server application on hosts
OWAMP
[27]
one way delay Client-server application on hosts

4.3.3.3 Measurement tools for network and traffic analysis
Permanent measurements give information about network and application activity and
allow searching for trends in traffic volumes sent and level of applications usage.
Monitoring data is important for network administrators, since it can be used for anomaly


detection and user misbehavior. Moreover raw data is necessary for validation of
theoretical models and when searching for new modeling methods. Section 5 of
Deliverable 1.2 [31] provides some insight into traffic measurement and analysis
mechanisms and tools. Short descriptions of those tools are provided in the following table
complemented with additional information on sFlow network monitoring technology.
List of measurement tools suitable for data extraction in the context of traffic analysis is
provided in Table 4-6.
Table 4-6: Network and traffic analysis measurement tools
Tool name Available network performance
metrics
Purpose / usage
D-ITG Tool
[66]
throughput, delay, jitter, packet
loss
traffic generator that uses stochastic models (for
packet size and inter departure time) to imitate
real application behavior with built-in
measurement tool that can produce accurate
IPv4 and IPv6 applications workloads
Tstat Tool [67] - automatically correlates incoming and outgoing
packets combining them into flows using both
standard traffic measurement indices and more
sophisticated methods at L2-L7 layers level to
provide traffic evaluation
CoralReef Tool
[68]
- open source network analyzer
BLINC [85] - novel traffic classification approach operating at
three levels of the host behavior social,
functional and application
PERFAS delay, latency variation, packet
loss
performs active measurements on IP level,
installed on Deutsche Telekoms IP backbone
MIB/SNMP
Standards [38]
highly aggregated counter of
packets and bytes transmitted and
lost at interfaces
provide coarse grained statistics from network
equipment
NetFlow [81] - protocol dedicated to IP traffic information
collection and analysis which enables network
traffic accounting, usage-based network billing,
network planning, security, Denial of Service
monitoring capabilities, and network monitoring
sFlow [28] - multi-vendor sampling technology embedded
within switches and routers; provides the ability
to continuously monitor application level traffic
flows at wire speed on all interfaces
simultaneously; the sFlow Agent runs as part of
the network management software within a
device, it combines interface counters and flow
samples into sFlow datagrams that are sent
across the network to an sFlow Collector



4.3.3.4 External monitoring data sources and monitoring systems
Apart from directly interfacing with the monitoring tools it is also possible to interact with
monitoring systems or collect data from other sources, e.g. monitoring information
databases. Those databases, e.g. maintained by NOCs, can be for instance organized as
Round Robin Database (RRD) files for which special tools are used to store and retrieve
data [29].
Interacting with a deployed monitoring system would be particularly suitable when running
complex measurements in large and heterogeneous networks. It is possible to interface
with network management systems or monitoring middleware software that orchestrates a
number of underlying monitoring tools to run complex monitoring tasks. An example of
such a middleware is perfSONAR [30].
PERFormance Service-Oriented Network monitoring ARchitecture (perfSONAR) [30] is an
infrastructure for network performance monitoring. It consists of a set of services
delivering performance measurements in a federated environment. These services act as
an intermediate layer, between the performance measurement tools and the diagnostic or
visualization applications. perfSONAR is a joint collaboration between several partners
including GANT European project participants [88].
perfSONAR is a service-oriented architecture in which all services communicate with each
other using well-defined protocols. Figure 6 presents this architecture logically divided into
three layers. The lowest layer of the architecture contains measurement points individual
tools that take passive and active measurements of the network. The service layer is
made up of services that bind communication and management activities between the
measurement points and the user-focused tools.

Figure 6: perfSONAR Architecture [30]
Many perfSONAR services and monitoring applications have already been implemented
as standalone measurement tools [86], e.g. BWCTL [87] Measurement Point and
OWAMP Measurement Point.

Monitoring tool selection
Before the implementation of the DTM mechanism two basic measurement tools were
considered as main approaches for implementation in DTM monitoring modules: SNMP


[38] and NetFlow [81]. Below a short description of both protocols with explanation of
motivation for such choice is presented.
NetFlow offers per flow counters and reports without possibility to receive those reports
on-demand. NetFlow exporter sends flow statistics only when one of two counters will
trigger it: flow-active-timeout or flow-inactive-timeout. As presented in Figure 7 flow-active-
timeout is set as an example to 15s and flow-inactive-timeout to 20s. Time when collector
will receive flows statistics depends only on arrival of the first and last packet of particular
flow. If the flow is longer than flow-active-timeout the exporter sends statistics after 15s,
otherwise (i.e. if the flow is shorter than flow-active-timeout or period of time from last
export) flow-inactive-timeout triggers sending procedure.
+10 +10 +10 +10 +20 +20 +20 +20 +30 +30 +30 +30 +40 +40 +40 +40 +50 +50 +50 +50
Flow Y
Flow X
Flow Z
Reports received
Reports needed in DTM
15 s
15 s 15 s
20 s
t
20 s
20 s

Figure 7: NetFlow exporting process.
SNMP-based traffic measurements provide only information about interface/queue
statistics, i.e., number of transmitted/received packets or bytes. Reports can be
synchronized (send as a response for polling). Figure 8 presents example process of
receiving SNMP-based statistics from network equipment as a replay for polling by SNMP
agent. In addition, SNMP implementation is less complex and supported by almost any
network device.

+10 +10 +10 +10 +20 +20 +20 +20 +30 +30 +30 +30 +40 +40 +40 +40 +50 +50 +50 +50
Flow Y
Flow X
Flow Z
t
Reports needed in DTM
30 s 30 s

Figure 8: SNMP-based measurement.


The SmartenIT consortium aimed to select a measurement tool for network traffic that
addresses DTM specification requirements and is also lightweight. Taking into account
previously mentioned features of measurement tools, SNMP was selected.
4.3.4 QoE monitor
The QoE monitor can be anywhere in the network but more accurate results can be
obtained if it is located close to the video player in order to experience network
performance similar to those of the player. At best, QoE monitor should be on the video
player host itself.
Some open source libraries (e.g. IQA [71], VQMT [72]) are available but are only able to
provide results of full-reference quality assessment (i.e. similar to a diff between source
and received video), while it is important to support the no-reference quality assessment
case: the receiver evaluates the quality of what it receives without knowledge of the
original video file, as in many cases (i.e. use of Youtube), the original video file is not
known.
VitALU [73], a proprietary tool developed by ALBLF, is able to provide an estimation of the
Mean Opinion Score (MOS) value (a score estimated subjectively by human viewers)
without need for the reference video.
Table 4-7 presents the metrics that should be supported by the QoE monitor.
Table 4-7: Metrics for QoE monitoring tool
Category Metric Name
IP traffic metrics
Transport Protocol (HTTP Live, RTSP, RTP Multicast, Flash HTTP)
Number of Packets, Packets Retransmitted (%)
Lost/Disordered Packets (for UDP flows)
Network jitter
Average Bitrate
Video metrics
Video frame resolution (pixel)
Frame rate (Average and Std) (frame per second)
Video access time (s)
Video duration (s)
Video bit rate (bps)
Time distribution for each frame resolution / video bitrate (adaptive
streaming)
Number of freezes (video interruption)
Freeze duration (Average and Max) (s)
Number of erroneous frames
Late and erroneous frame ratio (%)
Compression Video Quality Score (Avg and Std) from 1 (very bad


quality) to 5 (very good quality)
Networked Video Quality Score (Avg and Std) from 1 (very bad
quality) to 5 (very good quality)
Audio metrics
Audio bitrate (bps)
Audio duration (s)
MOS (E-model)

The VITALU software supports all these metrics. It is implemented in C# and is targeted
for fixed clients (e.g. PC, server). A Windows executable (binary to be used with license by
Alcatel-Lucent
1
) can be provided along with open source DLLs (FFMPEG [74], Wireshark
[75], etc...). A version for the Linux operating systems exists but not all functionalities are
supported.
As it was developed by the consortium partner ALBLF, VitALU is selected to be used as
the QoE analyzing library in any traffic management mechanism, as it should be fairly
easy to make simple adaptations in order to be interfaced with SmartenIT entities or to
add new specific metrics. While the software itself cannot be released as open-source, it
relies on several other open source libraries. This software is intended to be simple of use:
all that is needed is the type of flow to analyze and understanding the resulting metrics. Of
course as being the originator of this software, ALBLF has a strong expertise on this
product.
In order to integrate VitALU into the java framework, the executable could be launched by
a Java application, or a wrapper could be provided. In case a mobile client is targeted, it is
possible to mimic VitALU measurements on a mobile by running it on a portable PC
connected with a 3G key. Another option is to perform an offline analysis by capturing the
packets, together with radio-related (e.g. base id, signal strength) and location (GPS)
information, on the mobile during the video rendering and later, send the file containing
the captured network data (in the standard PCAP format) corresponding to the monitored
flow to the VitALU server, e.g. via Wi-Fi.
4.3.5 Fixed/Mobile Network Traffic Manager
In general, the Fixed/Mobile Network Traffic Manager takes decisions related to traffic
management, e.g. connectivity set up between two network endpoints that should be
materialized at the network layer by the Switching/Forwarding component. So far two so
called Network Configuration Frameworks were considered in the project namely
OpenFlow-based Software Defined Networking (OpenFlow-based SDN) and Multi-protocol
Label Switching with Traffic Engineering capabilities (MPLS-TE). Those frameworks have
been already described in Sections 5.2 and 5.3 of Deliverable D2.2 [1], respectively.
Those descriptions contain basic technical information about related technologies and
mechanisms, brief explanation of their potential application to TMSs developed within the
project and a summary of conducted initial functional validation tests. Each of those
frameworks provides different capabilities which may be particularly suitable for different
TMSs.

1
ALBLF is a subsidiary company of Alcatel-Lucent


In the following sections the above mentioned frameworks are described focusing on their
technological aspects and their possible impact on the architecture and software
prototype. Please note that the network configuration framework has major impact on the
Network Traffic Manager and Switching/Forwarding components as well as the interface to
be used between them. We assume that the Switching/Forwarding component resides in
the network device thus the devices (either physical or virtual) must support the
technologies implied by the selected network configuration framework. This has a direct
impact on the testbed design done in the scope of Work Package 4.
4.3.5.1 OpenFlow-based SDN
OpenFlow Switch specification [32] covers the components and basic functions of the
switch as well as the OpenFlow protocol to manage the switch from a remote controller.
Multiple OpenFlow Switch specification versions exist: from version 1.0.0 up to the latest
version 1.4.0 released in October 2013. Each new version of the specification introduces
new features and capabilities. Nevertheless not all versions of OpenFlow are widely
implemented and supported by both virtual and physical network devices as well as
available software SDN controllers. What might be relevant from the project perspective is
that OpenFlow since version 1.1 supports basic operations related to MPLS.
OpenFlow is supported by a variety of physical network equipment vendors but also by
virtual switch implementations like Open vSwitch [33] which has been used during the
tests documented in D2.2 [1]. Mininet framework [34] was used for the creation of the
entire virtual network and may be found useful in the future work especially during
prototype validation.
Multiple SDN controllers supporting different OpenFlow versions exist. Both the Controller
and the underlying network devices will need to support the agreed version of OpenFlow.
Next, two of the available open SDN controllers are briefly described.
Floodlight
Floodlight [35] is an enterprise-class, Apache-licensed, Java-based OpenFlow Controller.
It is supported by a community of developers including a number of engineers from Big
Switch Networks. Floodlight is not just a plain OpenFlow controller. It realizes a set of
common functionalities to control and inquire an OpenFlow network, while applications on
top of it realize different features to solve different user needs over the network. Figure 9
shows the relationship among the Floodlight Controller, the applications built as Java
modules compiled with Floodlight, and the applications built over the Floodlight REST API.
The REST API is the recommended interface to develop applications utilizing Floodlight
supported features but it is also possible to add functionalities/applications inside the
controller software itself.
Floodlight controller supports OpenFlow version 1.0 only. The latest version of the
software has been released in October 2012.


Figure 9: Floodlight controller components diagram [35]
The advantages of using a Floodlight controller are: maturity and wide adoption of the
software, complete and easily accessed documentation of the REST API and clearly
documented guidelines for implementing controller extensions. On the other hand, it only
supports OpenFlow version 1.0, what may not be sufficient from the implemented TMS
point of view.
OpenDaylight
OpenDaylight [36] is an open source project with a modular, pluggable, and flexible SDN
controller platform at its core. OpenDaylight project is supported by many big companies
including leading network equipment vendors like Brocade, Cisco and Juniper. The
controller exposes open northbound APIs which are used by applications. Those APIs
support communication over the OSGi framework and bidirectional REST. The business
logic and algorithms reside in the applications. These applications use the controller to
gather network intelligence, run algorithms to perform analytics, and then use the
controller to orchestrate the new rules, if any, throughout the network.


Figure 10: OpenDaylight (Hydrogen) components diagram [36]
The controller platform itself contains a collection of dynamically pluggable modules to
perform needed network tasks. In addition, platform oriented services and other
extensions can also be inserted into the controller platform for enhanced SDN
functionality.
The southbound interface is capable of supporting multiple protocols (as separate plug-
ins), e.g. OpenFlow 1.0, OpenFlow 1.3, BGP-LS [82], etc. These modules are dynamically
linked into a Service Abstraction Layer (SAL). The SAL exposes device services to which
the modules north of it are written. The SAL determines how to fulfill the requested service
irrespective of the underlying protocol used between the controller and the network
devices.
The version 1.0 of the OpenDaylight controller software, named Hydrogen, was released
in February 2014. Three editions of the software were released, namely Base,
Virtualization and Service Provider. Hydrogen Base Edition is mainly meant for exploring
SDN and OpenFlow for proof-of-concepts or academic initiatives in physical or virtual
environments.
The main advantages of OpenDaylight controller are: support for OpenFlow 1.3, novel and
easily extensible modular architecture, its constantly rising popularity and wide support
provided by all major players from the network equipment vendors and network services
community. Its major downsides are on the other hand related to the very recent release
of the software which may still contain unidentified bugs and issues as well as the lack of
well-organized and complete documentation.


4.3.5.2 MPLS-TE
Multi-Protocol Label Switching [25] is a well-known standard and commonly used
technology in ISP networks. At its core, MPLS enables creation of virtual end-to-end
tunnels called Label Switch Paths (LSP). Over the years, MPLS has been enhanced with
additional features including the Traffic Engineering extensions that enable efficient and
reliable network operations while simultaneously optimizing network resources utilization.
Moreover, multiple connectivity applications have been defined that leverage MPLS as the
underlying technology, e.g., VPLS, BGP/MPLS VPN, that enable setting up different type
of connectivity services on top of the existing LSPs tailored for particular communication
requirements.
MPLS-TE [76] is suitable for any TMS that uses traffic differentiation, sets up dedicated
paths in the network, monitors the load on paths or makes QoS guarantees. LSPs might
be set up on-demand on per data transfer basis in the case when the delivery paths need
to be established for a considerably long time period. It is also possible to establish a
mesh of LSPs and provide periodical LSPs re-configuration what may be employed by
TMSs focused on load balancing and overall cost minimization.
In order to perform MPLS related configuration, software needs to communicate with
network devices using any of the available management protocols, e.g. NETCONF [37],
SNMP [38] or other.
Eventually, based on selected TMS requirements, network configuration by means of
OpenFlow-based SDN is being implemented. Taking into consideration the description of
both SDN controllers provided in section 4.3.5.1, the fact that OpenFlow 1.0 version is
sufficient, the importance of having clear and complete documentation and previous
experience of the development team, the Floodlight controller was selected by the
consortium as the SDN controller to be used as part of the SmartenIT prototype.
4.4 Internal interfaces protocol formats
The following sections describe the advantages and disadvantages of the 3 alternatives
for the protocol serialization and formatting that could be used in the internal SmartenIT
interfaces: the interfaces between the SmartenIT entities. In the end of the section, the
final selection is presented.
4.4.1 Google Protocol Buffers
Google Protocol Buffers [39] is a cross-language, cross-platform mechanism for serializing
structured data, ideal for communication protocol and data storage design and
development. Developers define the structure format in a text .proto file in a human-
readable fashion, which can then be compiled to most well-known and popular
programming languages, such as Java, C++, Python, and Ruby, generating the source
code to be integrated into the software project. The advantages of using the Google
Protocol buffers are (a) the simple and human-readable format definition, (b) the binary
format of the encoded messages, thus the small footprint when exchanged in a
communication channel, (c) the short time required for parsing and (d) the ease of using
the generated classes. On the other hand, the non-human readable format of the
exchanged messages could be considered as a disadvantage by certain developers.


4.4.2 Apache CXF Distributed OSGi
Apache CXF Distributed OSGi (DOSGi) [11] implements the OSGi Remote Service [8]
functionality using Web Services (since DOSGi release version 1.1 both SOAP and
RESTful services are supported). The core OSGi specification allows OSGi services to be
visible and accessible inside the OSGi container. The OSGi Remote Service specification
and consequently DOSGi extends this functionality and allows defining services inside one
container and using them in another one even if they are deployed on different physical
machines.
In the case of SmartenIT, DOSGi can be used for communication between entities in two
ways. If both entities are OSGi containers, the communication can be done as described
above, so for example one S-Box instance (Section 3) can get and use remote service
from another S-Box instance deployed on a different server. The other case would be to
use DOSGi only for creating and exposing Web Services (in cases where one of the
communicating entities is not an OSGi container).
The usage of DOSGi would be a natural choice, if OSGi-based framework would be
selected. It seamlessly integrates with OSGi/Karaf mechanisms and makes use of its
advanced features. If OSGi-based framework isnt selected, other more lightweight and
underlying framework independent approaches should be considered instead.
4.4.3 JSON
JSON [80], derived from JavaScript, is an open, text-based, cross-language, cross-
platform, human-readable data serialization format, ideal for network and web
applications. The existence of multiple libraries and tools for generating and parsing JSON
data in all programming languages, as well as its small footprint and human-readable
nature, have made JSON one of the most widely used data serialization formats.
The SmartenIT consortium aimed to select a data serialization format that would be
lightweight and used in all internal interfaces between SmartenIT entities to ensure the
homogeneity of the prototype. Taking into account that ALTO protocol also uses JSON for
its data exchange format, JSON was selected.



5 Implementation Framework
The implementation framework of the SmartenIT project includes all required tools to
design, develop, test and deploy the created software, and the necessary methods and
support platforms to organize and support the implementation process. These tools are
used by the SmartenIT implementation team to implement their respective components,
as well as by prototype integrators to produce a coherent, integrated system. This ensures
to some extent that developers and integrators use a common development, building, and
testing environment. Note that although testing is a core part of the implementation
process, the functional testing of the prototype falls under the WP4 tasks. Hence, in this
chapter we will briefly describe the testbed as part of the implementation framework. More
details on the design and set-up of the software validation testbed will be provided in the
upcoming Deliverable D4.1 (due to M24).
5.1 Tools
The implementation framework tools consist of a specification language for designing the
SmartenIT system, programming languages used to develop components, the libraries
used to support the software development, the tool chain for compiling and building the
source code, the version control platform for archiving source code, the issue tracking
system for organizing and planning the implementation work, as well as tracking and
resolving found bugs and issues, the continuous integration server for automatically
building systems components and the validation testbed created to validate each
releases features and functionalities.
5.1.1 Specification Language
The SmartenIT system is designed and modeled using UML specification language. The
consortium also agreed to use the Microsoft Visio [40] software, in order to achieve
homogeneity in created diagrams and figures. UML has been applied to component,
entity, class, sequence and deployment diagrams presented in previous, and the current
deliverable.
5.1.2 Programming Languages
The selected programming languages for the SmartenIT entities are presented in Table
5-1, and reflect the decision of Section 4.1.
Table 5-1: Entities programming languages
Entity Programming
Language(s)
Development Kit
SBox Java Oracle JDK 1.7.0_51 [3]
SDN Controller Java Oracle JDK 1.7.0_51 [3]



5.1.3 Libraries
Each components external dependencies are presented in Table 5-2. Most of these
libraries reflect the decisions of sections 4.1, 4.2 (JBoss Netty libraries), 4.3.3 (SNMP
library), 4.3.5 (Floodlight SDN Controller dependencies), and 4.4 (Jackson libraries for
JSON support). The rest of libraries are used to support certain features of the first
SmartenIT release.
Table 5-2: Components libraries
Entity Component/Module Libraries
SBox
Database
sqlite-jdbc-3.7.2
jdbi 2.53
jackson-annotations 2.3.3
Interfaces
(Inter-SBox and SBox-
SDN)
netty 4.0.18.Final
jackson-databind 2.2.3
wiremock 1.46
User Interface
javax.servlet 3.0
jsf-api, jsf-impl 2.2.6
QoS Analyzer snmp4j 2.2.5
Economic Analyzer commons-math3 3.0
Commons
slf4j 1.7.7
logback 1.1.2
commons-configuration 1.10
commons-logging 1.1.3
SDN
Controller
Floodlight
org.restlet, org.restlet.ext.jackson, org.restlet.ext.simple,
org.restlet.ext.slf4j 2.1-RC1
org.simpleframework 4.1.21
netty 3.2.6.Final
args4j 2.0.16
concurrentlinkedhashmap-lru 1.2
jython-standalone 2.5.2
libthrift 0.7.0
easymock 3.1
jsonassert 1.2.3
logback 1.0.0
slf4j 1.6.4
Interfaces
jackson-databind 2.2.3
jsonassert 1.2.3


5.1.4 Tool chain
Apache Maven [19] is used as the dependency management and build automation tool
for the SmartenIT prototype. Maven is actually a software project management tool, which
provides an expandable set of tools so that developers may define the build lifecycle,
phases and goals, the projects dependencies, build plug-ins that insert additional goals in
build phases and build profiles which may build the project in different ways.
Specifically, version 3.1.1 of Apache Maven is used for the building of the SmartenIT
project, including the following list of plug-ins to execute additional required functions:
maven-surefire-report-plugin (version 2.17), used to aggregate all JUnit test results
and present them in HTML format,
maven-javadoc-plugin (version 2.9), used to generate the Java API documentation,
cobertura-maven-plugin (version 2.6), used to generate and aggregate the
Cobertura coverage results,
maven-project-info-reports-plugin (version 2.1.2), used to include general project
information,
maven-site-plugin (version 3.3), which aggregates all the aforementioned reports
into a single project Maven website,
keytool-maven-plugin (version 1.3), used to generate a SmartenIT security
certificate,
jetty-maven-plugin (version 9.1.1.v20140108), used to run and deploy the web
applications of the SmartenIT prototype into a Jetty web server,
maven-compiler-plugin, used to compile the source of the project,
maven-assembly-plugin (version 2.4), used to create a single executable for each
entity and finally package and generate the SmartenIT software release artifact,
android-maven-plugin (version 3.8.0), used to run and test the Android application
of the end-user entity.
The projects system tree, dependencies and build/execution commands are presented in
Section 5.3.
5.1.5 Testbed
The testbed provides an environment for verification and experimentation, its basic
topology is depicted in Figure 11. The main technology used in the testbed is Linux; in
particular, the Ubuntu 14.04 distribution [4] is used. For managing virtual OS instances,
libvirt is used for KVM and the LXC tools for LXC.


C
Network Connections
uNaDa 2
uNaDa 3
Access Point 3
Datacenter 1
Switch 1
Datacenter 3 Datacenter 2
Switch 3 Switch 2
PC 2 PC 3
PC 1
Mobile Clients Home Clients
Access Point 2

Figure 11 : The basic testbed topology
Other operating systems different than Linux could be used, however, the prevalence of
Linux in research and industry makes it well suited for the task. Furthermore, the wide
range of virtualization options and its high performance are unparalleled. A number of
*NIX operating systems are free and open source as well. However, they lack
performance, software support and virtualization options.
L2 and OpenFlow switching is provided by Open vSwitch; L3 routing by Linux features.
Furthermore, standard network technologies are used, such as DNS, DHCP as well as
routing protocols such as BGP. Open vSwitch supports both L2 switch and OpenFlow
based forwarding. It is free and open source, while no other available software offers the
same level of features and this performance.
More details on the basic testbeds design and configuration will be provided in the
upcoming Deliverable D4.1 (due to M24).
5.1.6 Version Control
Apache Subversion [77] is used as the version control system of SmartenIT. It will be
used to store and maintain current and previous versions of the source code. Developers
can copy and modify files or folders in the code repository and still maintain full revision
history. Each change is identified by a revision identifier.


Developers will get access to the SmartenIT repository by the SVN administrator and can
extract, update and commit their code to the SmartenIT code repository using a client
based on their operating system, e.g. TortoiseSVN for Windows, XCode for Mac OS and
RapidSVN for Linux users.
The Subversion server is hosted and administrated by PSNC and can be accessed at
https://svn.man.poznan.pl/svnroot/SmartenIT/.
5.1.7 Issue Tracking
Atlassian Jira [78] is a software implementation management and issue-tracking tool,
used to organize and plan the work of developers and the integrator. Its main features that
are used during SmartenIT development and integration are:
Software implementation management and planning: Roadmap of milestones and
future versions of the software
Integration with the Subversion: Code changes and revision logs tracking
Ticketing: Bug and features tracking and management
Developers are provided access to JIRA by the JIRA administrator and can view their
tickets, milestones and other information from their web browser.
The Atlassian Jira server is hosted and administrated by PSNC and can be accessed at
https://jira.man.poznan.pl/en/.
5.1.8 Continuous Integration Server
Jenkins [79] is a continuous integration tool, used to automatically and frequently build
our system. Its main features are:
Integration with the Subversion: All artifacts essential to build our system should be
placed under code repository and system should be buildable upon every new code
checkout.
Build automation: Source code compiling, binary code packaging, deployment and
documentation generation can be automatically performed by calling single
commands in existing build scripts.
Automated Unit tests whenever developers commit code to the repository.
E-mail Notification and feedback to responsible developers whenever last build fails
or there are incompatible changes in code.
Continuous integration and testing statistics.
Developers will get access to Jenkins by the Jenkins administrator and can view the
projects build status, and other information from their web browser.
The Jenkins server is hosted and administrated by PSNC and can be accessed at
https://maven.man.poznan.pl/jenkins/job/SmartenIT/.


5.2 Procedures
A set of common software prototype development procedures was defined and agreed on
by the involved partners in order to assure a smooth and efficient prototype development
process and successful delivery of software in incremental releases.
This section includes information about the system release and development processes
(both represented in form of cycles) implemented in the project. Project partners play
specific roles in these processes. Specific types of meetings will be organized throughout
the implementation process.
5.2.1 System Release Process
The SmartenIT system release process will follow the agile development model, divided
into (approximately) 2 month release cycles. Each release cycle (Figure 12) would
consist of:
Feature planning, where the consortium would identify and prioritize features and
tasks for next release, and specify components mechanisms and interfaces in the
specification document (this phase takes place in feature planning meeting);
Development of the required features by the responsible partners (assignment of
components to partners and agreement on the implementation framework was
done prior to initialization of implementation);
Integration of the developed components and basic sanity tests;
System validation, where the developed system is tested in the testbed for the
required release features;
System release and preparation of system release report (current document D3.2
and upcoming Deliverable D3.4 update). In addition, a system release review
meeting takes place to identify and discuss problems encountered during this
release cycle.

Figure 12: System release cycle


5.2.2 Development Process
The concept of continuous integration will be adopted for the development process
within SmartenIT, in which developers commit frequently (daily if possible) their code,
aiming to prevent integration problems and identify issues rapidly. This process will be
supported by automated tools, e.g. Jenkins continuous integration server, while
developers are required to write unit tests, covering most parts of their code.
The implementation process (Figure 13) consists of 4 phases:
Developers write code and tests,
Perform unit tests locally,
If tests pass, then commit their code to the repository, and
In case the build fails they are notified to fix the issue, otherwise continue their
development work.

Figure 13: Development cycle
5.2.3 Roles
A set of roles involved in the overall implementation process was decided that would
support processes described above. Each role was assigned a list of responsibilities and
responsible project partner. Those roles and related assignments are presented in Table
5-3.
Table 5-3: Defined roles in the implementation process
Role Responsibilities
Responsible
Partner
Responsible
Person
Product
Manager

Sets the high-level requirements
and features of each release.
ICOM Spiros Spirou


Tools
Administrator
Manages and maintains
implementation framework tools.
Provides access to tools.
PSNC
(&ICOM)
Jakub Gutkowski
ukasz opatowski
George Petropoulos
Release
Master

Facilitates the system release
process.
Contact point of the
implementation team.
ICOM George Petropoulos
Implementatio
n Team
Consists of all the people,
responsible for the design,
development, integration, testing
and deployment of the
SmartenIT prototype.
UZH, TUD,
AGH, ICOM,
ALBLF,
PSNC
As appointed by
each Partner.
5.2.4 Meetings
In each release cycle defined in Section 5.2.1, 3 types of meetings will take place:
Feature planning meeting
o Long meeting before each release cycle, where product manager, release
master and implementation team decide releases features, break them into
smaller tasks, assign responsibilities and define the time plan.
Weekly meeting
o 15-minute meeting every week during each release cycle, organized by
release master where developers report their progress and problems since
last weekly meeting and state their plans for next week.
Release review meeting
o Short meeting after each release cycle, where product manager, release
master and implementation team review the release cycle, identify and
resolve issues.
o If possible, a prototype demonstration to product manager and the
consortium could take place.
o This meeting could be merged and co-located with feature planning meeting.
5.3 System tree layout
The SmartenIT source code is organized as a Maven [19] multi-module project. Each
entity (sbox, sdn, unada, enduser, network, dist) is also a Maven multi-module project,
which can be packaged as a single jar with the use of a certain maven plug-in (maven-
assembly-plugin).
Each entitys component (e.g. main, interfaces) is a Maven module, which may be divided
into further maven modules or packages, based on component owners preferences. In
the defined system tree, certain components were divided into sub-modules to isolate
developers work, as well as avoid possible dependencies conflicts.


Packages have the following structure: eu.smartenit.<parent module(s)>.
<current module>.package1, e.g. eu.smartenit.sbox.db.dao
The complete SmartenIT system tree, which also includes currently unimplemented
entities and components, is presented below. Parentheses define the type of packaging,
indicating whether a module consists of other sub-modules (pom packaging), is packaged
as a Java archive (jar), which aggregates all classes and configuration files into one single
executable or a Web application archive (war) which compresses all classes, web pages,
and resources into one executable which can be deployed to any web server.
smartenit (pom)
o sbox (pom)
main (jar)
interfaces (pom)
inter-sbox (pom)
o client (jar)
o server (jar)
sbox-sdn (jar)
db (pom)
dao (jar)
dto (jar)
web (war)
alto (jar)
ctm (jar)
ntm (jar)
na (jar)
qoa (jar)
eca (jar)
commons (jar)
o sdn (pom)
floodlight-0.90 (jar)
interfaces (pom)
sbox-sdn (jar)
o unada (pom)
o enduser (pom)
o network (pom)
o dist (pom)


The system can be built with the use of the following command at the smartenit base
directory:
mvn clean install
The aforementioned command, cleans all previously generated artifacts, compiles,
packages and installs all modules, as well as runs all test cases. To avoid executing all
test cases, the user may execute the following command:
mvn clean install -DskipTests
Besides, the user may generate the maven site for the smartenit project with the following
command:
mvn clean install site site:stage

Besides compiling and packaging all source code, the aforementioned command
aggregates and presents in html format the project-related information, e.g.
implementation team information (Figure 14), the Java API documentation (JavaDoc)
(Figure 15), the complete test results, and the Cobertura [23] coverage reports (Figure
16).

Figure 14: SmartenIT implementation team


Figure 15: SmartenIT JavaDoc API example

Figure 16: SmartenIT Cobertura coverage results for SBox entity


All the aforementioned commands also generate the SmartenIT software release artifact,
which is the smartenit-${project.version}.zip, created at the dist/target
directory.
For the first release, which implements the DTM mechanism, the zip file includes the
following executables and configuration files:
main.jar: The main executable of the SBox application, which initializes all
modules, and executes the DTM mechanism.
sbox.properties: It includes all the required SBox configuration parameters.
logback.xml: It defines the logging level of the SBox application.
web.war: Its the war file, deployed to the Jetty web server.
realm.properties, jetty.xml: Configuration files that override default Jetty
configuration.
floodlight-0.90.jar: The executable of the Floodlight SDN controller,
extended with SmartenIT functionality.
Assuming that all prerequisite software (JDK 1.7.0_51 and Jetty 9.1.1) is installed, the
release artifact may be installed to execute the DTM functionalities.


6 Entities
6.1 S-Box
The SBox entity is the coordinating entity of the DTM mechanism. SBox is responsible for
retrieving and aggregating the SNMP measurements from the network entities, calculating
the compensation and reference vectors, and finally, configuring the remote SDN
Controller. It includes certain components that are required to store and access
mechanism- and NSP-specific parameters (Database and User Interface), retrieve and
aggregate the monitoring data from the network entities (QoS Analyzer), perform
necessary calculations (Network Traffic Manager, Economic Analyzer) and also interact
with remote SBoxes and the SDN Controller (Inter-SBox and SBox-SDN Controller
Communication Services). Each components functionality, interfaces, design and finally
unit tests are further described in the following sections.
6.1.1 Network Traffic Manager
6.1.1.1 Functionality
The main functionalities provided by the Network Traffic Manager (NTM) can be grouped
into two sets. When NTM is running on SBox deployed in AS that receives data traffic
generated by one or multiple inter-DC communications, NTM provides the following
functionalities:
periodically receives information about link traffic vectors from QoS Analyzer
calculated after each reporting period and about reference vectors calculated by
Economic Analyzer after each accounting period;
calculates a new compensation vector after each link traffic or reference vector
update;
sends the compensation vector (with the reference vector after accounting period)
to remote SBox instances that are deployed in ASs from which the received data
traffic originated.
When considering SBox located at the data traffic sender end of an inter-DC
communication, the NTM is responsible for receiving compensation and reference vectors
from the Inter-SBox Communication Service server and distributing them to the underlying
SDN controllers (by means of SBox-SDN Client) that will distribute the outgoing traffic
among all configured tunnels accordingly.
In a general case, the SBox and therefore the NTM supports both ends of inter-DC
communication since DCs in a local AS will most likely both receive and send traffic
from/to remote ASs at the same time.
6.1.1.2 Interfaces
Network Traffic Manager provides in total 4 methods to be called by other S-Box
components. Two of them are implemented by DTMTrafficManager class:


public void updateXVector(XVector xVector) throws
IllegalArgumentException: Method called by the QoS Analyzer to update
information about link traffic vector calculated every reporting period.
o Input parameters:
xVector New link traffic vector
o Thrown exceptions:
IllegalArgumentException when given xVector argument is invalid
public void updateRVector(RVector rVector) throws
IllegalArgumentException: Method called by Economic Analyzer to update
information about reference vector calculated every accounting period.
o Input parameters:
rVector New reference vector
IllegalArgumentException when given rVector argument is invalid

The remaining two interface methods are implemented by
DTMRemoteVectorsReceiver class and are called by Inter-SBox Communication
Service server component.
public void receive(CVector cVector) throws
IllegalArgumentException: Method called to update information about
compensation vector calculated every reporting period in remote S-Box.
o Input parameters:
cVector Compensation vector calculated in remote SBox
IllegalArgumentException when given cVector argument is invalid
public void receive(CVector cVector, RVector rVector) throws
IllegalArgumentException: Method called to update information about
compensation and reference vectors calculated every reporting and accounting
period, respectively in remote S-Box.
o Input parameters:
cVector Compensation vector calculated in remote SBox
rVector Reference vector calculated in remote SBox
IllegalArgumentException when either of given vector arguments is
invalid


6.1.1.3 Design
The NTM component implementation consists of 20 classes. The main functionalities
provided by the NTM as described in Section 6.1.1.1 fall into two areas related with
reacting on incoming traffic volumes by providing compensation and reference vectors to
remote SBoxes and outgoing traffic distribution by means of SDN based on received
compensation and reference vectors. These functionalities are implemented by disjoint
sets of classes depicted in Figure 18 and Figure 21, respectively.
In Figure 17 a top level view on the NTM design is presented with classes that are used by
other components of the SBox:
NetworkTrafficManager: Main class of the Network Traffic Manager
component. Represents an entry point for the NTM component common for all the
implemented traffic management mechanisms. Manages the lifecycle of the major
NTM internal objects that implement traffic management mechanisms interfaces
and logic.
o public void initialize(): Method that should be called in order to
initialize the NTM component (e.g. read required data from data base) in
default DTM mode.
o public void initialize(NetworkTrafficManagerDTMMode
mode): Method that should be called in order to initialize the component in
provided NetworkTrafficManagerDTMMode mode.
o public DTMTrafficManager getDtmTrafficManager(): Returns
an instance of DTMTrafficManager class that should be used by QoS
Analyzer and Economic Analyzer to provide updated vectors to DTM logic
implemented as part of the Network Traffic Manager.
o public DTMRemoteVectorsReceiver getDtmVectorsReceiver():
Returns an instance of DTMRemoteVectorsReceiver class that should be
used by Inter-SBox Communication Service server to provide received
remote vectors to DTM logic implemented as part of the Network Traffic
Manager.
NetworkTrafficManagerDTMMode: Enumeration class that represents three
DTM running modes:
o TRAFFIC_SENDER: DTM instance is run on SBox that manages AS(s) that
only sends traffic to remote ASs. In this case some of the modules do not
need to be initialized.
o TRAFFIC_RECEIVER: DTM instance is run on SBox that manages AS(s)
that only receives traffic from remote ASs. In this case some of the modules
do not need to be initialized.
o TRAFFIC_SENDER_AND_RECEIVER: General case enabled by default.
DTMTrafficManager: Implements DTM external interface methods that are used
by QoS Analyzer and Economic Analyzer to update information about link traffic
and reference vectors calculated every reporting and accounting period,
respectively.


o public void initialize(): Method that should be invoked during
component initialization to load required data from the database and
populate internal structures.
o public void updateXVector(XVector xVector) throws
IllegalArgumentException: DTM external interface method used to
update information about link traffic vector calculated every reporting period.
Traffic vector values are accumulated from the beginning of accounting
period (reception of updated R vector). Triggers subsequent actions in a new
thread.
o public void updateRVector(RVector rVector) throws
update information about reference vector calculated every accounting
period. Link traffic vector value for given AS is reset. Triggers subsequent
actions in a new thread.
DTMRemoteVectorsReceiver: Implements DTM external interface methods that
are used by Inter-SBox Communication Service server side component to update
information about reference and compensation vectors calculated in remote
SBoxes.
o public void initialize(): Method that should be invoked during
component initialization to load required data from the database, populate
internal structures and initialize all SDN controllers.
o public void receive(CVector cVector) throws
update information about compensation vector calculated every reporting
period in remote SBox.
o public void receive(CVector cVector, RVector rVector)
throws IllegalArgumentException: DTM external interface method
used to update information about compensation and reference vectors
calculated every reporting and accounting period, respectively in remote
SBox.



Figure 17: NTM partial class diagram: top level
In Figure 18, a partial NTM class diagram is presented with the following classes
(DTMTrafficManager class was already described in the previous paragraph):
VectorProcessingThread: Base abstract class that implements the Runnable
interface. Each vector processing procedure implemented by the subclasses of this
base class is executed in separate thread of execution.
o public VectorProcessingThread(XVector xVector, RVector
rVector, List<SBox> remoteSboxes): The constructor with three
arguments: link traffic and reference vectors to be used during compensation
vector calculation and a list of SBoxes which should be updated once the
calculation is complete.
CVectorProcessingThread: Extends VectorProcessingThread class.
Implements new compensation vector calculation after new link traffic vector is
received followed by sending compensation vector to remote SBoxes. Those
actions are performed in separate thread.
o public void run(): Method launched when thread is started.
Calculates compensation vector with CVectorCalculator and sends it to
remote SBoxes using instance DTMVectorsSender.
CRVectorProcessingThread: Extends VectorProcessingThread class.
Implements new compensation vector calculation after new reference vector is
received followed by sending both compensation and reference vectors to remote
SBoxes.
o public void run(): Method launched when thread is started.
Calculates compensation vector with CVectorCalculator and sends both
vectors to remote SBoxes using instance of DTMVectorsSender.
CVectorCalculator: Implements compensation vector calculation logic.


o public CVector calculateCompensationVector(XVector
xVector, RVector rVector): Calculates compensation vector based
on provided arguments: link traffic vector and reference vector.
DTMVectorsSender: Implements methods that forward requests for inter-SBox
vector information update to the InterSBoxClient component.
o public void send(SBox sbox, CVector cVector): Method calls
proper method of the InterSBoxClient class instance in order to send
information about updated compensation vector to given remote SBox.
o public void send(SBox sbox, CVector cVector, RVector
rVector): Method calls proper method of the InterSBoxClient class
instance in order to send information about updated compensation and
reference vectors to given remote SBox.
InterSBoxClientFactory: Static factory class used by DTMVectorsSender to
obtain instances of InterSBoxClient class to be used for communication with remote
SBoxes.
o public static InterSBoxClient getInstance(): Method returns
an instance of the InterSBoxClient class according to the currently enabled
client creation mode. By default the so called unique client creation mode is
disabled.
VectorsContainer: Data container class used to store XVector and RVector
objects representing link traffic vectors and reference vectors organized in pairs
(XRVectorPair class) by AS number.
o public XRVectorPair
accumulateAndStoreUpdatedXVectorValues(XVector xVector):
Stores data from new link traffic vector in the container. If XRVectorPair for
given AS already exists it is updated, meaning that the current counter
values are increased by the values from the new link traffic vector. If not,
new XRVectorPair is created.
o public void resetXVectorValues(int asNumber): Resets the
counter values of link traffic vector stored for given AS.
o public XRVectorPair storeUpdatedRVector(RVector
rVector): Stores new reference vector (RVector) in the container. If
XRVectorPair for given AS already exists it is updated. If not, new
XRVectorPair is created.
o public XRVectorPair loadCurrentVectorPair(int asNumber):
Retrieves XRVectorPair for given AS from the container.
o public Set<Integer> getListOfAsNumbers(): Retrieves a list of all
ASs for which vector pairs are currently stored in the container.
XRVectorPair: Used to store two related vector objects (XVector and RVector)
representing link traffic vector and reference vector, respectively.
o public XRVectorPair(XVector xVector, RVector rVector):
The constructor with arguments that sets both vectors. Those vector can be
later on retrieved with appropriate getter method.


o public boolean areBothVectorsSet(): Checks whether both
vectors comprising the pair were already set.
o public int getAsNumber(): Returns the number of the AS for which
both of the vectors were calculated. Note: Both vectors should have the
same value of source AS number field.

Figure 18: NTM partial class diagram: traffic receiver side
Sequence diagrams in Figure 19 and Figure 20 present the NTM interactions with other
SBox components after reporting and accounting periods, respectively. Presented actions
are related to the NTM operations performed on the traffic receiving side of inter-DC
communication.
After reporting period elapses, the DTMQoSAnalyzer updates DTMTrafficManager
with a new X vector value. This triggers the execution of CVectorProcessingThread
which calculates a new C vector using CVectorCalculator and instructs the


DTMVectorSender to send the new value to a remote SBox. InterSBoxClient is
called to perform the actual C vector transfer.
After accounting period elapses, the EconomicAnalyzerInternal updates
DTMTrafficManager with a new R vector value. This triggers the execution of
CRVectorProcessingThread which calculates a new C vector and sends both newly
calculated C vector and received R vector to a remote SBox.

Figure 19: NTM sequence diagram after reporting period: traffic receiver end

Figure 20: NTM sequence diagram after accouting period: traffic receiver end
In Figure 21, a partial NTM class diagram is presented with the following classes
(DTMRemoteVectorsReceiver class was already described in the previous paragraph):
SDNConfigPusher: Implements methods that forward requests for SDN
Controller vector information update or configuration to the SboxSdnClient
component.
o public SDNConfigPusher(List<SDNController> controllers):
The constructor with arguments: list of SDN controllers to which subsequent
requests called on this object will be directed.


o public void updateVectorInfo(CVector cVector): Method uses
SboxSdnClient to update compensation vector information in selected SDN
controllers.
o public void updateVectorInfo(CVector cVector, RVector
rVector): Method uses SboxSdnClient to update compensation and
reference vectors information in selected SDN controllers.
o public void initializeSDNController(ConfigData data):
Method uses SboxSdnClient to initialize selected SDN controllers with
configuration data.
SDNClientFactory: Static factory class used by the SDNConfigPusher to obtain
instances of SboxSdnClient to be used for communication with remote SDN
controllers.
o public static SboxSdnClient getInstance(): Method returns an
instance of the SboxSdnClient according to currently enabled client creation
mode. By default the so called unique client creation mode is disabled.
ThetaCoefficientHandler: Implements logic for manipulating compensation
vector values based on theta coefficients.
o public CVector normalizeCVector(CVector cVector, RVector
rVector): Method normalizes compensation vector values based on theta
coefficients provided as part of reference vector for given AS.
SDNControllerContainer: Data container class used to store information
about SDN controllers that manage DA routers on which tunnels were configured
towards AS with given AS number.
o public void populateControllersFromDB(): Used to initialize
internal structures based on data stored in data base.
o public List<SDNController>
getControllersByRemoteASNumber(int asNumber): Returns a list
of SDN controllers that manage DA routers with tunnels towards given
remote AS.
o public List<SDNController> getAllControllers(): Returns a
list of all SDN controllers in local ASs.


Figure 21: NTM partial class diagram: traffic sender side
The sequence diagram in Figure 22 presents the Network Traffic Manager interactions
that take place on the traffic sending side of inter-DC communication. During NTM
component initialization, DTMRemoteVectorsReceiver is requested to initialize all SDN
controllers with initial configuration data loaded from the database. This request is passed
to SDNConfigPusher which in turn communicates with SboxSdnClient to transfer
required ConfigData to the SDN controller over HTTP. Later on during SBox operation,
DTMRemoteVectorsReceiver periodically receives from InterSBoxServer new C
and R vectors updated by remote SBoxes. SDNConfigPusher is used to pass those
vectors to appropriate SDN controllers. The actual transfer is done by SboxSdnClient.


Figure 22: NTM simplified sequence diagram: traffic sender side
6.1.1.4 Tests
In order to test the functionality of NTM a set of 10 test classes was implemented
comprising JUnit [20] tests for not only particular single methods or group of methods but
also simple workflows internal to NTM component. Mockito library [69] was used in several
cases to mock components external to NTM, e.g. Database, SBox-SDN Communication
Service and Inter-SBox Communication Service.
The implemented test classes with test methods are listed below (please note that all
provided names of test methods are preceded with public void in the source code):
NetworkTrafficManagerDTMTest contains following test methods for
NetworkTrafficManager class initialization of DTM modules in three modes as
specified in NetworkTrafficManagerDTMMode enum class:
o shouldCreateAndInitializeNTMInSenderOnlyMode()
o shouldCreateAndInitializeNTMInReceiverOnlyMode()
o shouldCreateAndInitializeNTMInGeneralMode()
o shouldCreateAndInitializeNTMInDefaultMode()
XVectorUpdateTest contains following test methods for workflow triggered by
DTMTrafficManager class after receiving updated link traffic vector from QoS
Analyzer component:
o shouldThrowExceptionOnVectorNull()
o shouldThrowExceptionOnInvalidVectorASNumberArgument()


o shouldThrowExceptionOnInvalidVectorValuesListArgument()
o shouldSkipCalculationAfterXVectorUpdate() throws
InterruptedException
o shouldCalculateCAndUpdateCVectorAfterXVectorUpdate()
throws InterruptedException
RVectorUpdateTest contains following test methods for workflow triggered by
DTMTrafficManager class after receiving updated reference vector from Economic
Analyzer component:
o shouldThrowExceptionOnVectorNull()
o shouldThrowExceptionOnInvalidVectorASNumberArgument()
o shouldThrowExceptionOnInvalidVectorValuesListArgument()
o shouldSkipCalculationAfterRVectorUpdate() throws
InterruptedException
o shouldCalculateCAndUpdateCRVectorsAfterRVectorUpdate()
throws InterruptedException
XRVectorsUpdateDetailedTest: contains following more detailed test
methods for workflow triggered by DTMTrafficManager class after receiving
updated traffic vectors from QoS Analyzer and Economic Analyzer component:
o shouldDistributeCRVectorsAfterAccountingPeriodEnd()
throws Exception
o shouldDistributeCVectorBeforeAccountingPeriodEndSingleXV
ector() throws Exception
o shouldDistributeCVectorBeforeAccountingPeriodEnd()
throws Exception
RemoteVectorsUpdateTest contains following test methods for workflow
triggered by DTMRemoteVectorsReceiver class after receiving new compensation
and reference vectors from remote SBoxes:
o shouldThrowExceptionOnCVectorNull()
o shouldThrowExceptionOnRCVectorsNull()
o shouldThrowExceptionOnInvalidCVectorASNumberArgument()
o shouldThrowExceptionOnInvalidCRVectorASNumberArgument()
o shouldThrowExceptionOnInvalidCVectorValuesListArgument()
o shouldThrowExceptionOnInvalidCRVectorValuesListArgument(
)
o shouldInitControllers()
o shouldProcessAndNotUpdateWhenNoSDNControllers()
o shouldProcessAndUpdateAfterRemoteCVectorReceived()
o shouldProcessAndUpdateAfterRemoteCAndRVectorsReceived()
CVectorCalculatorTest contains following test methods for compensation
vector calculation logic implemented in CVectorCalculator class:


o shouldCalculateFromTwoDimentionVectors()
o shouldCalculateWithOneXVectorValueZero()
o shouldCalculateWithEqualRAndXVectorValuesRatio()
o shouldCalculateWithAllXVectorValuesZero()
o shouldCalculateWithXVectorEqualRVactor()
o shouldCalculateFromFiveDimentionVectors()
o shouldReturnNullSinceVectorNull()
o shouldReturnNullSinceInvalidVectorSize()
o shouldReturnNullSinceVectorSizeBelowTwo()
o shouldReturnNullSinceInvalidLinkIDs()
ThetaCoefficientHandlerTest contains following test methods for the logic
implemented in ThetaCoefficientHandler class (this class and its tests will be
enhanced in future releases):
o shouldReturnTheSameCVectorIfThetaNull()
o shouldReturnTheSameCVectorIfThetaNotNull()
VectorsContainerTest contains following test methods for store and load
methods of VectorsContainer class:
o shouldCreateNewPairOnEmptyContainerWithUpdatedXVector()
o shouldCreateNewPairWithUpdatedXVector()
o shouldCreateNewPairOnEmptyContainerWithUpdatedRVector()
o shouldCreateNewPairWithUpdatedRVector()
o shouldUpdateExistingPairWithUpdatedXVector()
o shouldUpdateExistingPairWithUpdatedRVector()
o shouldReturnNullIfAsNumberIsNotSet()
SDNControllerContainerTest contains following test method for store and
load methods of SDNControllerContainer class:
o shouldPopulateContainerStructures()
RemoteSBoxContainerTest contains following test method for populate and get
methods of RemoteSBoxContainer class:
o shouldPopulateContainerStructures()
6.1.2 Quality of Service Analyzer
The Quality of Service Analyzer is responsible for periodical execution of the following
actions:
collection of incoming traffic counter values from all inter-domain links from a set of
configured BG routers;


collection of incoming traffic counter values from all tunnels from a set of configured
DA routers;
calculation of link traffic vector (X vector) and list of tunnel traffic vectors (Z vector)
for each local AS;
distribution of information about new traffic vectors to Economic Analyzer and
Network Traffic Manager.
The above actions are executed at intervals specified by the length of the reporting period.
Counter values collection from network devices is performed by means of SNMP.
Counters are read in multiple parallel threads (one thread per router).
6.1.2.2 Interfaces
Quality of Service Analyzer does not provide any methods to be called by other S-Box
components.
6.1.2.3 Design
Quality of Service Analyzer consists of 15 classes. Figure 23 presents 13 most relevant
classes with selected relationships between them. Please note that only public methods
are displayed.
Following is the description of those 13 classes:
DTMQosAnalyzer: Manages the lifecycle of DTM related modules of QoS
Analyzer component. Communicates with DTM modules in Economic Analyzer
(EconomicAnalyzer class) and Network Traffic Manager (DTMTrafficManager
class).
o public DTMQosAnalyzer(DTMTrafficManager trafficManager,
EconomicAnalyzer economicAnalyzer): The constructor with
arguments: instances of DTM external interface classes of Network Traffic
Manager and Economic Analyzer.
o public void initialize(): Initializes SNMPTrafficCollector class and
starts scheduled traffic counters monitoring tasks according to configured
time schedule parameters.
o public void updateXVector(XVector xVector): Updates Network
Traffic Manager (DTMTrafficManager class) with link traffic vector calculated
after reporting period.
o public void updateXZVectors(XVector xVector,
List<ZVector> zVectors): Updates Economic Analyzer with link traffic
vector and a list of tunnel traffic vectors calculated after reporting period.
SNMPTrafficCollector: Class used to schedule traffic counter collection tasks
and calculating traffic vectors once all counters are read after each reporting
period. Calculated vectors are provided to DTMQosAnalyzer for further distribution.
o public SNMPTrafficCollector(DTMQosAnalyzer analyzer): The
constructor with argument: instance of the DTMQosAnalyzer which should
be updated with calculated traffic vectors.


o public void configure(List<AS> systems,
List<DC2DCCommunication> communications): Populates helper
data structures with links and tunnels that will be monitored. Triggers
collection of SNMP OID numbers for links and tunnels counters.
o public void scheduleMonitoringTasks
(TimeScheduleParameters tsp): Schedules counters monitoring task
according to configured time schedule parameters.
o public void notifyNewCounterValues(int asNumber,
CounterValues counterValues): Calls MonitoringDataProcessor to
calculate new link traffic vectors with data fetched from counters. Updates
DTMQosAnalyzer with the new link and tunnel traffic vectors.
o public MonitoredLinksInventory getMonitoredLinks(): Getter
method for monitored links inventory.
o public MonitoredTunnelsInventory getMonitoredTunnels():
Getter method for monitored tunnels inventory.
TrafficCollectorTask: Task class responsible for collecting counter values
from BG and DA routers. Implements Runnable interface.
o public TrafficCollectorTask(SNMPTrafficCollector
trafficCollector, int asNumber): The constructor with arguments:
instance of the SNMPTrafficCollector from which information about
monitored links and tunnels will be read and which will be notified once all
counters are read; and number of the AS for which this task is launched
(relevant if SBox manages more than one AS).
o public void run(): Method launched when thread is started
responsible for fetching data from router counters and updating
SNMPTrafficCollector with collected data.
CounterCollectorThread: Thread class responsible for collecting counter
values from specific interfaces of a given router. Implements Callable interface.
o public CounterCollectorThread(List<Link> links, BGRouter
bgRouter): The constructor with arguments to be used when monitoring
BG router.
o public CounterCollectorThread(List<Tunnel> tunnels,
DARouter daRouter): The constructor with arguments to be used when
monitoring DA router.
o public CounterValues call() throws Exception: Thread's main
method responsible for fetching data from counters on given router and
updating TrafficCollectorTask.
SNMPWrapper: Wrapper class for the snmp4j library.
o public String snmpGet(OID oid, Target target): Triggers
snmpget command on given router specified by target argument and
given OID (object identifier). Returns the response received from the SNMP
agent.


o public List<String> snmpWalk(OID oid, Target target):
Triggers snmpwalk command on given router specified by target
argument and given OID. Returns the response received from the SNMP
agent.
o public void startClient() throws IOException: Starts SNMP
client.
o public void stopClient() throws IOException: Stops SNMP
client.
o public Target prepareTarget(String routerAddress, String
snmpCommunity): Prepares Target class instance with router address and
SNMP community to be used in snmpGet and snmpWalk method calls.
o public OID prepareOID(String oid): Prepares OID class instance
to be used in snmpGet and snmpWalk method calls.
SNMPOIDCollector: Helper class for preparing SNMP OID numbers for link and
tunnel counters.
o public SNMPOIDCollector(MonitoredLinksInventory links,
MonitoredTunnelsInventory tunnels): The constructor with two
arguments: links and tunnels to be monitored.
o public void collectOIDsForLinks() throws IOException:
Triggers OID information collection for all link counters from all BG routers.
o public void collectOIDsForTunnels() throws IOException:
Triggers OID information collection for all tunnel counters from all DA
routers.
MonitoringDataProcessor: Used to handle monitoring data (i.e. counter
values) and calculating link traffic vector and tunnel traffic vectors by means of
XVectorCalculator class and ZVectorCalculator class, respectively.
o public MonitoringDataProcessor(MonitoredLinksInventory
monitoredLinks, MonitoredTunnelsInventory
monitoredTunnels): The constructor with two arguments: all monitored
links and tunnels.
o public XVector calculateXVector(int asNumber,
CounterValues values): Calculates link traffic vector by means of
XVectorCalculator class based on provided counter values for inter-domain
links of given AS.
o public List<ZVector> calculateZVectors(int asNumber,
CounterValues values): Calculates a list of tunnel traffic vectors by
means of ZVectorCalculator class based on counter values for tunnels in
given AS.
VectorCalculator: Base abstract class for traffic vector calculators comprising
data structure used to store counter values from previous reporting periods.


o protected CounterValues getOrCreateLatestVectorValues
(int asNumber): Returns stored instance of CounterValues for given AS
if exists. If not, creates and returns new instance.
XVectorCalculator: Implements link traffic vector calculation based on current
counter values collected from links in given AS and previous values of those
counters. Extends VectorCalculator class.
o public XVector calculateXVector(int asNumber,
CounterValues values): Method calculates new link traffic vector for
given AS. It should be launched after each reporting period.
ZVectorCalculator: Implements tunnel traffic vector calculation based on
current counter values collected from tunnels in given AS and previous values of
those counters. Extends VectorCalculator class.
o public List<ZVector> calculateZVectors(int asNumber,
CounterValues values): Method calculates new set of tunnel traffic
vectors for given AS. Each ZVector class object corresponds to a single
DC2DCCommunication. It should be launched after each accounting period.
CounterValues: Data container class used to store counter values (as 64bit
numbers) on per LinkID and per TunnelID basis.
o public void storeCounterValue(LinkID linkID, long
counterValue): Stores counter value for given link in the structure.
o public void storeCounterValue(TunnelID tunnelID, long
counterValue): Stores counter value for given tunnel in the structure.
o public void addLinksAndTunnels(CounterValues
counterValues): Populates the local structure with provided links and
tunnels counters data.
o public Map<LinkID, Long> getLinkCounterValues(): Retrieves
counter values for all links stored in the structure.
o public Map<TunnelID, Long> getTunnelCounterValues():
Retrieves counter values for all tunnels stored in the structure.
o public Set<LinkID> getAllLinkIds(): Retrieves identifiers of all
links stored in the structure.
o public Set<TunnelID> getAllTunnelsIds(): Retrieves identifiers
of all tunnels stored in the structure.
o public Long getCounterValue(LinkID linkID): Retrieves 64-bit
counter value for given link from the structure.
o public Long getCounterValue(TunnelID tunnelID): Retrieves
64-bit counter value for given tunnel from the structure.
MonitoredLinksInventory: Data container class used to store information
about routers and monitored links on per AS number basis. Provides specific query
methods to retrieve information. Only selected query methods are described.


o public void populate(List<AS> systems): Method to be called to
populate inventory with monitored link information that is obtained from list of
local Autonomous Systems.
o public List<Link> getLinks(int asNumber): Retrieves all inter-
domain links for given local AS.
o public List<Link> getLinks(BGRouter bgRouter): Retrieves all
links from given BG router.
o public List<BGRouter> getBGRouters(): Retrieves all BG routers
stored in the structure.
o public List<BGRouter> getBGRoutersByAsNumber(int
asNumber): Retrieves all BG routers from given local AS.
o public Set<Integer> getAllAsNumbers(): Returns numbers of all
local ASs stored in the structure.
MonitoredTunnelsInventory: Data container class used to store information
about DA routers and monitored tunnels. Provides specific query methods to
retrieve information. Only selected query methods are described.
o public void populate(List<DC2DCCommunication>
communications): Method to be called to populate inventory with
monitored tunnels information that is obtained from list of
DC2DCCommunications terminated in local ASs.
o public List<Tunnel> getTunnels(int asNumber): Retrieves all
monitored tunnels from given local AS.
o public List<Tunnel> getTunnels(DARouter daRouter):
Retrieves all monitored tunnels from given DA router.
o public List<Tunnel> getTunnels(DC2DCCommunicationID id):
Retrieves all tunnels from inter-DC communication with given identifier.
o public List<DARouter> getDARouters(): Retrieves all DA routers
stored in the inventory.
o public List<DARouter> getDARoutersByAsNumber(int
asNumber): Retrieves all DA routers from given local AS.
o public List<DC2DCCommunicationID>
getAllDC2DCCommunicationIDs(int asNumber): Retrieves
identifiers of all inter-DC communications terminating in given local AS.
o public Set<Integer> getAllAsNumbers(): Retrieves numbers of all
local ASs stored in the inventory.



Figure 23: Quality of Service Analyzer class diagram
Sequence diagram in Figure 24 presents how Quality of Service Analyzer interacts with
other SBox components, i.e. the Economic Analyzer and Network Traffic Manager. After
the TrafficCollectorTask finishes its execution, meaning that all counter values are


read from monitored routers, it passes new counter values to the
SNMPTrafficCollector which in turn instructs MonitoringDataProcessor to
calculate new X and Z vectors. Later on the QoSAnalyzer is used to pass information
about those new vectors to proper EconomicAnalyzer and DTMTrafficManager
instances.

Figure 24: Quality of Service Analyzer sequence diagram

6.1.2.4 Tests
In order to test the functionality of the Quality of Service Analyzer a set of 13 test classes
was implemented comprising JUnit [20] tests for not only particular single methods or
group of methods but also simple workflows internal to QoS Analyzer component. Mockito
library [69] was used in several cases to mock components external to QoS Analyzer, e.g.
Database, Economic Analyzer and Network Traffic Manager.
The implemented test classes with test methods are listed below (please note that all
provided names of test methods are preceded with public void in the source code):
DTMQosAnalyzerTest contains following test method for DTMQosAnalyzer class
workflow launched after each reporting period including: new counter values
collection from routers, traffic vectors calculation and Economic Analyzer and
Network Traffic Manager updates:
o shouldUpdateXAndZVectors() throws Exception
SNMPTrafficCollectorTest contains following test methods for
SNMPTrafficCollector class:
o shouldCalculatXVextor()
o shouldCalculaZXVextors()
o shouldUpdateXVector()
o shouldUpdateZVectors()


o shouldTriggerFiveScheduledTasks()
o shouldPrepareLogForScheduleMonitoringTasks()
o shouldPrepareErrorForAsNumbersValidation()
o shouldCalculateInitialDelay()
TrafficCollectorTaskTest contains following test method for
TrafficCollectorTask class:
o shouldNotifyNewCounterValues()
TrafficCollectorTaskFailureScenarioTest contains following test method
for TrafficCollectorTask class:
o shouldMaxFetchingTimeExpire()
BGRouterCounterCollectorThreadTest contains following test methods for
collecting counter values from specific interfaces of BG routers:
o shouldParseCounter()
o shouldThrowExceptionForParseCounter()
o shouldThrowExceptionBecauseOfLackOfBGRouter() throws
Exception
o shouldThrowExceptionBecauseOfLackOfLinks() throws
Exception
o shouldCalculateCounterValues() throws Exception
DARouterCounterCollectorThreadTest contains following test methods for
collecting counter values from specific interfaces of DA routers:
o shouldParseCounter()
o shouldThrowExceptionForParseCounter()
o shouldThrowExceptionBecauseOfLackOfDARouter() throws
Exception
o shouldThrowExceptionBecauseOfLackOfTunnels() throws
Exception
o shouldCalculateCounterValues() throws Exception
SNMPWrapperTest contains following test methods for SNMP4J library wrapper
class named SNMPWrapper:
o shouldThrowExceptionForTimedOutForSnmpget() throws
Exception
o shouldThrowExceptionForNullResonseForSnmpget() throws
Exception
o shouldThrowExceptionBecauseOfErrorForSnmpget() throws
Exception
o shouldStarClient() throws Exception
o shouldStopClient() throws Exception


SNMPOIDCollectorTest contains following test methods for SNMPOIDCollector
class:
o shouldParseInterfaceNumber()
o shouldSetOutboundInterfaceCounterOID() throws
IOException
o shouldSetInboundInterfaceCounterOID() throws IOException
o shouldCollectOIDsForLinks() throws IOException
o shouldCollectOIDsForTunnels() throws IOException
o shouldThrowExceptionBecauseOfLackOfSnmpWalkResponse()
o shouldThrowExceptionBecauseOfLackOfPhysicalInterfaceName
()
XVectorCalculationTest contains following test methods for link traffic vector
calculation logic implemented by XVectorCalculator class and triggered by
MonitoringDataProcessor class:
o shouldCalculateFirstXVector()
o shouldCalculateSecondXVector()
o shouldCalculateLaterXVector()
ZVectorCalculationTest contains following test methods for tunnel traffic
vector calculation logic implemented by ZVectorCalculator class and triggered by
MonitoringDataProcessor class:
o shouldCalculateFirstZVectors()
o shouldCalculateSecondRoundOfZVectors()
CounterValuesTest contains following test methods for store and get methods
of CounterValue class:
o shouldStoreOneLinkAndTunnelEntry()
o shouldStoreTwoEntries()
o shouldReplaceOneOfTwoEntries()
o shouldValidateTwoCounterValueInstancesAsEqual()
MonitoredLinksInventoryTest contains following test method for population
and data retrieval methods of MonitoredLinksInventory data container class:
o shouldPopulateInventory()
MonitoredTunnelsInventoryTest contains following test method for
population and data retrieval methods of MonitoredTunnelsInventory data container
class:
o shouldPopulateInventory()
6.1.3 Economic Analyzer



The Economic Analyzer works out the strategy for traffic distribution for the next
accounting period. This is represented in a compact form of the reference vector. This
component collects the data expressing traffic amount transmitted through the inter
domain links during the accounting period.
To achieve this, the Economic Analyzer receives information about traffic amount from the
Network Analyzer (which collects data from network nodes) and accumulates the total
traffic volume in a given accounting Period.
It creates a cost map for traffic at the end of the accounting period and predicts the
possible optimal traffic pattern (the expected traffic amount on each inter domain link
which decreases total cost of transfer via these links). This information forms the basis for
calculating the reference vector. Each reference vector component represents expected
traffic amount on a particular inter domain link.
6.1.3.2 Interfaces
The economic analyzer provides one interface method to the other components as
follows:
EconomicAnalyzer provides the following method:
o void updateXZVectors(XVector X_in, List<ZVector>
Z_in_list): Accumulates the total traffic in a given accounting period
and correspondingly updates the X_V and Z_V vectors.
6.1.3.3 Design
The EconomicAnalyzer consists of 5 classes. Figure 25 presents all the involved
classes with selected relationships between them.


Figure 25: Economic Analyzer Class Diagram

Following is the description of those 5 classes:
EconomicAnalyzer: This is the main class that manages the calculation for the
RVector for a given set of Link IDs. The extensive calculation is done in the
EconomicAnalyzerInternal class instantiated within.
o EconomicAnalyzer (DTMTrafficManager dtmTrafficManager):
The constructor for the EconomicAnalyzer. Takes in the DTM Traffic
manager instance as the input variable in order to proceed to the following
methods eventually.
o void updateXZVectors(XVector X_in, List <ZVector>
Z_in_list): Accumulates the total traffic in a given accounting period


and correspondingly updates the X_V and Z_V vectors which are passed as
arguments.
EconomicAnalyzerInternal: This is the class that holds the economic analyzer
algorithm. It is responsible for the reference vector calculation.
o EconomicAnalyzerInternal(DTMTrafficManager
dtmTrafficManager, SimpleLinkID link1, SimpleLinkID
link2): The Constructor for EconomicAnalyzerInternal.
Responsible for initializing its member variables for subsequent calculations.
o RVector calculateReferenceVector(): Calculates the reference
vector and returns an RVector that contains the reference vector values.
The algorithm for the calculations is provided in steps 3-7 of Section
6.1.3.3.1.
o void updateXZVectors(XVector X_in, List <ZVector>
Z_in_list) : Accumulates the total traffic in a given accounting period
and correspondingly updates the X_V and Z_V vectors which are passed as
arguments. These calculations are part of the algorithm specified in steps 1-
2 of Section 6.1.3.3.1.
DInterval: This class represents an interval bounded by two alpha end points.
o DInterval (long lowerBound, long upperBound): The default
constructor. Creates a DInterval instance for a given set of lower and
upper bounds.
o boolean contains (long x): Returns true if the parameter value
passed as an argument is contained in the two bounds (including
boundaries).
o long getLowerBound(): Returns the lower bound of the interval.
o long getUpperBound(): Returns the upper bound of the interval
DAOFactory: Static factory class used to obtain an instance of a specific DAO
class.
o CostFunctionDAO getCostFunctionDAOInstance(): Returns
the locally stored instance of CostFunctionDAO.
o void setCostFunctionDAOInstance(CostFunctionDAO dao):
Method used to set a given instance of CostFunctionDAO to be returned
on each request.
Area: The class that represents the intervals which make up an area.
o Area(DInterval d1, DInterval d2): The constructor that is
responsible for creating an Area object.
o DInterval getD1(): Returns the interval on the x1 axis.
o DInterval getD2(): Returns the interval on the x2 axis.
6.1.3.3.1 Algorithm
The input data for the required calculations are presented below.


T, t parameters refer to the accounting and reporting period intervals and are
stored into the SBox database.
is the cost function for link s, while argument represents total traffic volume
transferred via link s during accounting period. Cost functions are stored and
retrieved from the SBox database.
vector is periodically provided by the QoS Analyzer (with t period). i refers to
the report from the period (and each vector component represents traffic on
a selected inter-domain link from period ).
vector is periodically provided by the QoS Analyzer (with t period and
= ). Each vector component represents sum of traffic going via a group of tunnels
traversing a respective single inter domain link.
, represent the tunnel identifiers acquired from the SBox database.
The algorithm calculates the vector, the inbound reference vector. Each vector
component represents optimized expected total inbound traffic on the selected inter-
domain link in the next accounting period.
The algorithm calculations are described below. The two first steps are repeated for each
accounting period T in the first time slot of the next accounting period.
1. Calculate total volume traffic vector :
.
2. Calculate total volume traffic vector received from the DC-B collected on DA-A
router:
.
Next steps (from 3 to 7) are done just after each accounting period T elapses.
3. Calculate the DC traffic manipulation freedom:
.
4. Construct the vector =(-S,S).
5. Identify areas for the optimization procedure.
Figure 26 presents an identification idea in the 2 dimensional case.


Figure 26: X Vector
The vector is presented in Figure 26. The and vectors determine the line on
which an optimal reference vector end must lay: ( . The algorithm
checks for optimal value on the mentioned line between points indicated by the
and vectors and for optimal cost in areas: , ,
Detailed procedure:
5a. Identify a set E={ | for i =1,,2and j =0,,3}, where
and In our example E={ } for the given cost functions.
5b. Identify intervals which will be used for optimization based on the E set.
For the chosen i for which , we take into account intervals
and . In our example we obtain following intervals: for i=1
we have , , and for i=2 we obtain , .
5c. For each find and for each find :
, identify that ,
, identify that .
In our example for we obtain and for we have .
5d. Define areas which will be used for optimization based on
identification: where i, j are related to the previous steps of the
algorithm. In our example we should use areas: and
. The only distinct areas are .
6. Calculate minimum for the function using simplex or interior point
method in each established area with constraint . The and
values representing minimum are components of the reference vector:
We obtain a separate reference vector for each area respectively: ,
, .
7. Chose the smallest reference vector from these calculated in the previous step:


6.1.3.4 Tests
In order to test the functionality of the EconomicAnalyzer a test class was implemented
comprising of JUnit [20] tests for not only particular single methods or group of methods
but also simple workflows internal to EconomicAnalyzer component. Mockito library
[69] was used in several cases to mock components external to EconomicAnalyzer,
e.g. the Database.
The implemented test classes with test methods are listed below
EconomicAnalyzerInternalTest: Test class for the
EconomicAnalyzerInternal.
o EconomicAnalyzerInternalTest(): The default constructor. Creates
an EconomicAnalyzerInternalTest instance.
o void firstTestData(): Tests the reference vector calculation using a
first set of test data.
o void secondTestData(): Tests the reference vector calculation using a
second set of test data.
o void thirdTestData(): Tests the reference vector calculation using a
third set of test data.
o void fourthTestData(): Tests the reference vector calculation using a
fourth set of test data.
o void fifthTestData(): Tests the reference vector calculation using a
fifth set of test data.
o void prepareTotalVolumeXVAfterAccountingPeriod(): Tests
whether the traffic volume accumulation is correct, when crossing the
accounting period threshold.
o void prepareTotalVolumeXVBeforeAccountingPeriod(): Tests
whether the traffic volume accumulation is correct, without crossing the
o void prepareTotalVolumeZVAfterAccountingPeriod(): Tests
whether the traffic volume accumulation is correct, when crossing the
o void prepareTotalVolumeZVBeforeAccountingPeriod(): Tests
whether the traffic volume accumulation is correct, without crossing the
accounting period threshold
6.1.4 Database
The Database (DB) component is responsible both for relational tables creation and the
application connection to the actual database. It is partitioned into the following packages:
dto: Contains all the Data Transfer Objects (DTO) of the database, namely all the
Java classes that correspond to the relational tables of the DB, and some additional


helper entities, which are used by certain components, but are not stored into the
database.
dao: Contains all the Data Access Objects (DAO) one for each table. Each of them
contains the methods provided to external components for reading from or writing
to the corresponding table. The dao package also includes the five subpackages
that follow.
o binders: Contains the binder classes.
o mappers: Contains the mapper classes.
o gen: Contains for each table an abstract class, behaving as interface, with
the actual necessary SQL commands wrapped each from a Java method
which is the one called by the above mentioned DAO method.
o util: Contains auxiliary, utility classes. For the moment it has only the
Tables class which creates and deletes all tables.
test: Includes the JUnit tests one for each DAO class.
The DB contains the SQL tables described in the Design Subsection 6.1.4.3.
6.1.4.2 Interfaces
The DB component provides its services by exposing for each table a number of methods.
These methods, which reside for each table-entity in the corresponding DAO class,
named <entity-name>DAO, constitute the DB interface to the other components. Each of
these methods wraps up one or more SQL commands, located in the corresponding
Abstract<entity-name>DAO. In the case of multiple SQL commands, these may reside in
more than one Abstract<entity-name>DAO.
There is a set of methods which are implemented by almost all DAOs, while few others
are more specific, more complicated therefore implemented usually by a single (or more
cooperating) DAO. The common methods are briefly described as follows.
Public void createTable(): The method that creates the corresponding
table.
Public void deleteTable(): The method that deletes the corresponding
table.
Public void insert(<entity type> e): The method that inserts an entity
to the corresponding table.
o Input parameter: The entity to be inserted.
Public List<entity type> findAll(): The method that finds all the
stored entities from the respective table.
o Returns: The list of stored entities.
Public <entity type> findById(<primary-key-type> pk): The
method that finds a stored entity by the corresponding tables primary key.
o Input parameter: The primary key.
o Returns: the entity indexed by the primary key.


Public <entity type> findLast(): The method that finds the last stored
entity.
o Returns: The most recent stored entity.
Public void update(<entity type> e):The method that updates a stored
entity into its table.
o Input parameter: The entity to be updated.
Public void deleteById(<primary-key-type> pk): The method that
deletes a stored entity by its table primary key.
o Input parameter: The primary key.
Public void deleteAll(): The method that deletes all the stored entities
from the respective table.

The DAO-specific methods and a short description for each of them follow:
TunnelDAO
o public List<Tunnel> findAllByDC2DCCommunicationID
(DC2DCCommunicationID dcID): The method that finds all stored
Tunnels by their DC2DCCommunicationID.
Input parameter: The DC2DCCommunicationID.
o public List<Tunnel> findAllByLinkID (SimpleLinkID
linkID)The method that finds all stored Tunnels by their linkID.
Input parameter: linkID The SimpleLinkID.
o public void deleteByDC2DCCommunicationID
(DC2DCCommunicationID dcID): The method that deletes all stored
Tunnels by their DC2DCCommunicationID.
o public void updateByDC2DCCommunicationID(SimpleTunnelID
tunnelID, DC2DCCommunicationID dcID): The method that updates
a stored Tunnel with its DC2DCCommunicationID.
Input parameter: The SimpleTunnelID.
ASDAO
o public List<AS> findLocalAs():The method that finds all the stored
local ASes from the AS table.
Returns: The list of stored local ASes.
o public List<AS> findRemoteAs(): The method that finds all the
stored remote ASes from the AS table.
Returns: The list of stored remote ASes.


o public List<AS> findAllAsesCloudsBgRoutersLinks(): The
method that finds all stored ASes, along with their attached BG Routers and
inter-domain links, with their traversing tunnels.
Returns: The list of ASes.
o public AS findASCloudsBgRoutersLinksByASNumber(int
asNumber)The method that finds an AS by its AS number, along with its
attached BG Routers and inter-domain links, with their traversing tunnels.
Returns: The AS.
CloudDCDAO
o public List<CloudDC> findRemoteClouds():The method that finds
all the stored remote Clouds from the Cloud table.
Returns: The list of stored remote Clouds.
o public List<CloudDC> findByASNumber(int asNumber): The
method that finds a stored Cloud by its AS number.
Input parameter: The AS number.
Returns: The list of stored clouds.
o public List<CloudDC> findLocalClouds(): The method that finds
all the stored local Clouds from the Cloud table.
Returns: The list of stored local Clouds.
CostFunctionDAO
o public void insertByLinkId(CostFunction c, SimpleLinkID
l): The method that inserts a CostFunction into the CostFunction table by
its link id.
Input parameter: The CostFunction to be inserted.
Input parameter: The SimpleLinkID.
o public CostFunction findByLinkId(SimpleLinkID l): The
method that finds a stored CostFunction by its linkID.
o public void deleteByLinkId(SimpleLinkID linkID): The
method that deletes a stored CostFunction by its linkID.
DARouterDAO
o public void insertBySDNControllerAddress(DARouter d,
String sdnAddress): The method that inserts a DARouter for a certain
SDN controller address into the DARouter table.
Input parameter:The DARouter to be inserted.
Input parameter: The SDN controller address.


o public void updateBySDNControllerAddress(String
daRouterAddress, String sdnAddress): The method that updates a
stored DARouter of a certain SDN controller address into the DARouter table
Input parameter: The address of the DARouter to be updated.
o public List<DARouter> findBySDNControllerAddress(String
sdnAddress): The method that finds a stored DARouter by its SDN
controller address.
Returns: The list of stored DARouters.
BGRouterDAO
o public void insertByASNumber(BGRouter bgrouter, int
asNumber):The method that inserts a BGRouter for an asNumber into the
BGRouter table.
Input parameter: The BGRouter to be inserted.
o public List<BGRouter> findByASNumber(int asNumber): The
method that finds the list of BGRouters by their AS number.
Returns: The list of stored BGRouters.
DC2DCCommunicationDAO
o Public List<DC2DCCommunication>
findAllDC2DCCommunicationsCloudsTunnels(): The method that
finds all the stored DC2DCCommunications with their attached clouds from
the DC2DCCommunication table.
Returns: The list of stored DC2DCCommunications.
LinkDAO
o public List<Link> findByBGRouterAddress(String
bgAddress): The method that finds all the stored Links by their BG route
address, as well as their traversing tunnels and assigned bg router.
Input parameter: bgAddress The BG router address.
Returns: The list of stored Links.
PrefixDAO
o public int insertByCloudName(NetworkAddressIPv4 p,
String cloudName): The method that inserts a Prefix of a cloud into the
Prefix table.
Input Parameter: p The Prefix to be inserted.
Input Parameter: cloudName The cloud name.


o public List<NetworkAddressIPv4> findAllByCloudName
(String cloudName): The method that finds all the stored Prefixes of a
cloud from the Prefix table.
Returns: The list of stored Prefixes.
o public void deleteByCloudName(String cloudName): The
method that deletes all the stored Prefixes of a cloud from the Prefix table.
SegmentDAO
o public void insertByLinkId(Segment s, SimpleLinkID
linkID): The method that inserts a Segment into the Segment table.
Input Parameter: s The Segment to be inserted.
Input Parameter: linkID The SimpleLinkID.
o public void insertBatchByLinkId(List<Segment> slist,
SimpleLinkID linkID): The method that inserts a batch of Segments
of a link into the Segment table.
Input Parameter: sList The Segments list to be inserted.
o public List<Segment> findAllByLinkId(SimpleLinkID
linkID): The method that finds all the stored Segments of a link from the
Segment table.
Returns: The list of stored Segments.
o public void deleteByLinkId(SimpleLinkID linkID): The
method that deletes all the stored Segments of a link from the Segment
table.
6.1.4.3 Design
The DB schema contains the following SQL tables depicted in Figure 27:
ASSYSTEM: The table where each Autonomous System is stored. A unique
number that identifies each AS, its ASNUMBER, it is tables primary key.
BGROUTER: The table where the Border Gateway Is stored. The prefix of the router
management IP address is the tables primary key.
CLOUDDC: The table where all the different clouds are stored. The cloud name is
the primary key.
COSTFUNCTION: The table where all the different cost functions are stored. The
pair <local link id, ISP name> is the primary key.


DAROUTER: The table where all the different Datacenter Attached Routers are
stored. The router IP address is the primary key.
DC2DCCOMMUNICATION: The table where the various connections between any
pair of datacenter are stored. Each such pair is uniquely identified by a combination
of 6 parameters: the communication number, the respective symbol, the name of
the local cloud, the name of the remote cloud, the local AS number and the remote
AS number.
ISP: The table where all the different ISPs are stored. The ISP name is the primary
key.
LINK: The table where all the different links are stored. The pair <local link id, ISP
name> is the primary key.
PREFIX: The table where all the different address prefixes we use are stored. An
auto incremented numerical identifier is the primary key.
SDNCONTROLLER: The table where all the different SDN controllers are stored. The
controller address is the primary key.
SEGMENT: The table where all the different segments are stored. An auto
incremented numerical identifier is the primary key.
TIMESCHEDULEPARAMETERS: The table where the single time schedule
parameters record is stored. An auto incremented numerical identifier is the primary
key.
TUNNEL: The table where all the different tunnels are stored. The pair <local link id,
ISP name> is the primary key.


Figure 27: DB schema
Figure 28 presents a class diagram including all data transfer objects and their
relationships. The class diagram maps to the aforementioned database schema.


Figure 28: Data Transfer Objects class diagram
Figure 29 presents the class diagram for the AS table-related classes. All other entities
conform to the same logic. For this simple case, 5 cooperating classes are involved to
provide read/write access to the respective table in a transparent fashion. Specifically, the
ASDAO class provides the necessary interface to all the other modules, and uses the
respective functions of the AbstractASDAO class, which execute all the SQL queries and
updates. Additional classes, BindAS and ASMapper provide the necessary mapping of
SQL result sets and columns to and from the respective DTO (AS class). The following
description of the aforementioned classes applies to all *, *DAO, Abstract*DAO, Bind* and
*Mapper classes.
ASDAO
o It is the only interface used by applications either to issue commands to or
receive responses from the database.
All its methods were described in Section 6.1.4.2.
AS
o All the ASDAO methods have arguments and/or return value an AS entity
object (the application language).
It is the DTO, therefore contains only constructors, getters and
setters.
AbstractASDAO


o Belongs to the gen package, contains all SQL queries, each one defines a
java method. These methods only get called from the ASDAO methods. The
ASDAO methods are the ones exposed to any application requesting to read
or write to the database.
BindAS
o Translator from application-entity to the database-table when commands
issued to the DB.
o It is an interface that includes a static class ASBinderFactory which
implements BinderFactory interface i.e a build() method that instantiate a
Binder object for the AS type using the bind() method whose description
follows
public void bind(SQLStatement q, BindAS bind, AS
arg): It maps the parameters of an sql statement in AbstractASDAO
into AS parameters.
ASMapper
o Translator from the database-table to application-entity dialect, when the DB
responds to an issued command, using the below described map method.
ASMapper: AS map(int index, ResultSet r,
StatementContext ctx): The method that translates a received
resultset into an AS object.
Input parameter: index The index.
Input parameter: r The received result set.
Input parameter: ctx The statement context.
Returns: The AS object.


Figure 29: AS table-related class diagram
6.1.4.4 Tests
In order to test the functionality of the DB provided services, test classes, one for each
relational table, therefore one for each DAO class, has been implemented. All test classes
include JUnit [20] tests for all methods the corresponding DAO supports. The
implemented test classes and the included test methods briefly described follow below.
ASTest
o public void testFunctions(): Tests all AS entity provided queries.
BGRouterTest
o public void testSimpleFunctions(): Tests all BGRouter entity
provided queries.
o public void testASBGRouterFunctions(): Tests the combined AS,
BGRouter query.
CloudDCTest
o public void testFunctions(): Tests all CloudDC entity provided
queries.


ComplexQueriesTest
o public void testFindAllAsesCloudsBgRoutersLinks(): Tests
AS in depth (i.e shows the relative clouds,bgrouters and links) query.
o public void
testFindAllDC2DCCommunicationsCloudsTunnels(): Tests
DC2DC in depth (i.e shows the relative clouds and tunnels) query.
DARouterTest
o public void testSimpleFunctions(): Tests all DARouterentity
provided queries.
o public void testDARouterSDNFunctions(): Tests the combined
DARouter, SDNController query.
DC2DCCommunicationTest
o public void testFunctions(): Tests all DC2DCCommunication
entity provided queries.
ISPTest
o public void testFunctions(): Tests all ISP entity provided queries.
LinkTest
o public void testFunctions(): Tests all Link entity provided queries.
o public void testComplexFunctions(): Tests complex Link
functions.
SDNControllerTest
o public void testFunctions(): Tests all SDNController entity
provided queries.
TimeScheduleParametersTest
o public void testFunctions(): Tests all TimeScheduleParameters
entity provided queries.
TunnelTest
o public void testFunctions(): Tests all Tunnel entity provided
queries.
VectorTest
o public void testFunctions(): Tests all Vector entity provided
queries.
6.1.5 Inter-SBox Communication Service
The Inter-SBox Communication Service component enables the communication of
information between several SBoxes.


6.1.5.2 Interfaces
The Inter-SBox Communication Service component exposes 2 interfaces, to be used by
other components, the client-side, which is implemented by the InterSBoxClient class,
which is called by the Network Traffic Manage and sends the reference and compensation
vectors to the remote SBox server, and the server-side, which is implemented by the
InterSBoxServer class, which receives this information and then forwards them to the
local Network Traffic Manager component to proceed with the execution of the
mechanism. These 2 classes and their respective methods are described below.
InterSBoxClient:
o public void send(CVector cVector, RVector rVector,
String dstIspId)
sends vectors cVector and rVector to interSBoxServer entity of the
SBox identified by dstIspId (IP_ADDRESS:PORT)
Input parameters:
cVector The compensation vector
rVector The reference vector
dstIspId The remote SBox identifier(IP_ADDRESS:PORT)
o public void send(CVector cVector, String dstIspId)
sends vectors cVector and rVector to interSBoxServer entity of the
SBox identified by dstIspId (IP_ADDRESS:PORT)
Input parameters:
dstIspId The remote SBox identifier(IP_ADDRESS:PORT)
InterSBoxServer:
o public void receive(CVector cVector, RVector rVector)
Receives the vectors cVector and rVector from InterSBoxClient entity
and forwards them to the linked DTMRemoteVectorsReceiver object
Input parameters:
6.1.5.3 Design
The Inter-SBox communication service consists of two main objects, which implement the
interconnection of two SBox entities, based on the JBoss Netty Framework [9] using
JSON description format. This format is presented in Appendix B.1. Each of these objects
defines an inner handler class to deal with multiple connections. An object representing
the generic message that is exchanged is also created.
InterSBoxClient The client side of the Inter-SBox communication service. It
implements methods for the exchange of the reference and compensation vectors.


Besides the methods defined in section 6.1.5.2, it creates an inner handler object of
type InterSBoxClientH.
InterSBoxServer The server side of the Inter-SBox communication service. It
implements methods for the exchange of the reference and compensation vectors.
Besides the methods defined in section 6.1.5.2, it creates an inner handler object of
type InterSBoxServerH. Both the InterSBoxServerH and
InterSBoxClientH classes implement the ChannelInboundHandlerAdapter
of the io.netty.channel package, and override methods to read from the
channel.
InterSBoxObject The inner object containing compensation vector and
reference vector sent between InterSBoxClient and InterSBoxServer.

Figure 30: Inter-SBox Communication Service Class Diagram


6.1.5.4 Tests
In order to test the functionality of the Inter-SBox communication service, a test class was
implemented, including JUnit [20]. Mockito library [69] was also used to mock the
functionality of a DTM vector receiver.
InterSBoxServerTest: the test starts a server waiting for connections on a
certain port, then launches a client that uses the send method with basic
reference and compensation vectors.
6.1.6 SBox-SDN Communication Service
The SBox-SDN communication service is the component responsible for the
interconnection of the SBox with the SDN controller. It is used by the Network Traffic
Manager component to provide the reference and compensation vectors to the SDN
controller, as well as the stored configuration data.
6.1.6.2 Interfaces
The SBox-SDN communication service provides 3 methods to be called by the Network
Traffic Manager, all included in the SboxSdnClient class:
public void distribute(SDNController sdnController, CVector
cVector, RVector rVector): The method that distributes the compensation
and reference vectors to the SDN controller.
o Input parameters:
sdnController The SDN Controller
public void distribute(SDNController sdnController, CVector
cVector): The method that distributes the compensation vector to the SDN
controller.
o Input parameters:
public void configure(SDNController sdnController, ConfigData
configData): The method that provides the configuration parameters to the SDN
controller.
o Input parameters:
configData The configuration data


6.1.6.3 Design
The SBox-SDN communication service consists of 3 classes, which implement the
interconnection of the 2 entities, based on the JBoss Netty Framework [9]. All 3 classes
and their connections are presented in Figure 31 and are also described below:
SboxSdnClient: The client side of the SBox-SDN controller communication. It
implements methods for the exchange of the reference and compensation vectors,
as well SDN controller configuration parameters. The JSON format of these
parameters is presented in the Appendix A.2.1 and A.2.2. Besides the methods
defined in section 6.1.6.2, it includes the following methods:
o public DefaultFullHttpRequest prepareHttpRequest(URI
uri, String content): The method that prepares the HTTP request
header.
o private void send(URI uri, String content) throws
InterruptedException: The method that sends the HTTP request to the
SDN controller server.
SboxSdnClientHandler: The handler class for the SBox-SDN controller
communication. It prints the server response for the sent HTTP request. The
SboxSdnClientHandler extends the SimpleChannelInboundHandler<HttpObject>
class from the io.netty.channel package, and overrides the following methods:
o public void channelRead0(ChannelHandlerContext ctx,
HttpObject msg) throws Exception: The methods that reads server's
response.
o public void exceptionCaught(ChannelHandlerContext ctx,
Throwable cause) throws Exception: The method handling
exceptions while connecting to server.
SboxSdnClientInitializer: The channel initializer for the client side of the
Sbox-SDN controller communication. It includes all the relevant JBoss Netty
handlers for the communication over HTTP. It extends the
ChannelInitializer<SocketChannel> class from the io.netty.channel package, and
overrides the following methods:
o public void initChannel(SocketChannel ch) throws
Exception: The method initializing the channel.


Figure 31: SBox-SDN communication service class diagram
6.1.6.4 Tests
In order to test the functionality of the SBox-SDN communication service, a test class was
implemented, including JUnit [20] tests for all methods of the SboxSdnClient class.
Wiremock library [70] was also used to mock the functionality of an SDN controller. The
implemented class and methods are the following:
SboxSdnClientTest:
o public void testPrepareHttpRequestCVector(): Checking
whether http request for c vector is correct.
o public void testPrepareHttpRequestRCVectors(): Checking
whether http request for c and r vectors is correct.
o public void testPrepareHttpRequestConfigData(): Checking
whether http request for config data is correct.
o public void testDistributeCVector(): Mocking sdn controller and
checking whether received request includes serialized c vector.


o public void testDistributeCRVectors(): Mocking sdn controller
and checking whether received request includes serialized r and c vectors.
o public void testConfigure(): Mocking sdn controller and checking
whether received request includes serialized configuration data.
6.1.7 User Interface
The User Interface (UI) of the SBox is the module responsible for configuring relevant
parameters for the execution of the DTM mechanism. It is operated by the Network
operator administrator, which is also the SBox administrator, and exposes a web interface
accessible by any web browser, where the following parameters may be inserted, modified
or deleted:
Local and remote autonomous systems that are involved in DTM mechanism
execution,
Local BG and DA routers,
SDN controllers and their attached routers,
Inter-domain links,
Tunnels,
Local cloud information,
Remote clouds,
Inter-cloud communications, and,
General DTM settings.
In the Section 6.1.7.2, all implemented pages are presented.
6.1.7.2 Interfaces
The Home page (Figure 32) is the introductory page of the SBox UI web application. The
SBox administrator will be able to navigate to all UI pages through the navigation bar and
its dropdown menus, residing in the upper part of the page.
In the Autonomous Systems page (Figure 33), the administrator may configure the ISPs
name, as well its local autonomous systems. For each autonomous system, the attached
SBox IP address is required. Besides, remote autonomous systems may be configured,
which will take part in inter-domain communications.
In the BG and DA routers page (Figure 34), the intra-domain routers can be configured.
BG routers insertion requires the number of the AS they belong, while both types of
routers require their SNMP community.
The administrator may configure the local SDN controllers parameters in the SDN
controllers page (Figure 35). For each SDN controller, the IP, REST and Openflow
address, REST and Openflow port, as well as its attached DA routers are required.


All inter-domain links are configured in the Links page (Figure 36). Link parameters
include: ID, ISP name, IP address, physical interface name, VLAN number, the BG router
it belongs, and a list of Segments (left border, right border, a and b constants) that
comprise its cost function.
In the Tunnels page (Figure 37), the administrator may insert, modify and delete traversing
tunnels. More specifically, information about tunnels name, number, source and
destination end addresses, physical interface name and the inter-domain link they
traverse are expected.
In the Local (Figure 38) and Remote (Figure 39) clouds page, the administrator configures
information about local and remote datacenters, that are located in local and remote
ASes, and are involved in inter-domain communications which are handled by the DTM
mechanism. Their respective autonomous systems and prefixes are also provided.
In the Inter-Datacenter page (Figure 40), all the inter-datacenter communications
parameters are configured, specifically the local and remote clouds and autonomous
systems, as well as their traversing tunnels.
Finally, in the Settings page (Figure 41), required DTM execution parameters are set: the
start date of the DTM execution, and the duration of the accounting and reporting periods.


Figure 32: Home page


Figure 33: Autonomous systems page


Figure 34: BG and DA routers page


Figure 35: SDN Controllers page


Figure 36: Links page


Figure 37: Tunnels page


Figure 38: Local clouds page


Figure 39: Remote clouds page


Figure 40: Inter-Datacenter communications page


Figure 41: DTM settings page
6.1.7.3 Design
The SBox User Interface is a web application designed and implemented based on the
Model-View-Controller (MVC) pattern, which defines 3 different but interconnected layers,
in order to separate data representation from business logic. In the generalized case, the
model layer includes all the application data, the view provides their output representation
and finally the controller is the coordinating component receiving user requests, calling
appropriate functions and returning results.
In the SBox User Interface, the model is represented by the Database component, which
consists of all the SBox database tables, their mapping to data transfer objects, as well as
the data access objects to query and update those tables. The database component was
described with details in Section 6.1.4.
The controller layer is implemented by the DashboardBean class, which includes all the
required functions that:
a) receive users requests and input from the web application pages,
b) interact with the database and,
c) finally return output and/or navigate to other pages.
Figure 42 presents the class diagram of the DashboardBean class, with its associations
with AS and ASDAO classes. Parameters and their getter/setter methods, as well as


associations with all the DAO and DTO classes are not included in this figure. The same
associations apply to all DTO and DAO classes of the Database component.

Figure 42: DashboardBean class diagram and sample association with ASDAO and AS
classes
The complete documentation for the DashboardBean class is presented below.


DashboardBean: It is the controlling class of the sbox web application, including
all parameters, and functions that are performed in the UI.
o public void init(): The method called when the view is constructed. It
reads the location of the database from a configuration file, and creates all
the database tables when called for a first time.
o public String updateIspName():The method that inserts/updates the
ISP information to the db.
o public String updateAS(): The method that inserts/updates as AS to
the db.
o public String editAS(AS a): The method that edits an AS.
o public String deleteAS(AS a): The method that deletes as AS from
the db.
o public String gotoIspPage(): The method that redirects to the
isp.xhtml.
o public String updateBGRouter(): The method that inserts/updates a
BG Router to the db.
o public String deleteBGRouter(BGRouter bg): The method that
deletes a BG Router from the db.
o public String updateDARouter(): The method that inserts/updates a
DA Router to the db.
o public String deleteDARouter(DARouter da): The method that
deletes a DA Router from the db.
o public String gotoRoutersPage(): The method that redirects to the
routers.xhtml.
o public String updateSDNController(): The method that
inserts/updates an SDN Controller to the db.
o public String editSDNController(SDNController s): The
method that edits an SDN Controller.
o public String deleteSDNController(SDNController s): The
method that deletes an SDN Controller from the db.
o public String gotoSDNControllersPage(): The method that
redirects to the sdncontrollers.xhtml.
o public String updateLink(): The method that inserts/updates a Link
to the db.
o public String editLink(Link l): The method that edits a Link.
o public String deleteLink(Link l): The method that deletes a Link
from the db.
o public void addSegment(): The method that adds a Segment to the
list of link's segments.


o public void deleteSegment(Segment s): The method that deletes a
Segment from the list of link's segments.
o public String gotoLinksPage(): The method that redirects to the
links.xhtml.
o public String updateTunnel(): The method that inserts/updates a
Tunnel to the db.
o public String editTunnel(Tunnel t): The method that edits a
Tunnel.
o public String deleteTunnel(Tunnel t): The method that deletes a
Tunnel from the db.
o public String gotoTunnelsPage(): The method that redirects to the
tunnels.xhtml.
o public void addLocalNetwork(): The method that adds a Network to
the list of local cloud's networks.
o public void deleteLocalNetwork(NetworkAddressIPv4 net):
The method that deletes a Network from the list of local cloud's networks.
o public String updateCloud(): The method that inserts/updates a
local CloudDC to the db.
o public String editCloud(CloudDC c): The method that edits a local
CloudDC.
o public String deleteCloud(CloudDC c): The method that deletes a
local CloudDC from the db.
o public String gotoLocalCloudsPage(): The method that redirects to
the localclouds.xhtml.
o public String updateRemoteCloud(): The method that
inserts/updates a remote CloudDC to the db.
o public String editRemoteCloud(CloudDC c): The method that
edits a remote CloudDC.
o public String deleteRemoteCloud(CloudDC c): The method that
deletes a remote CloudDC from the db.
o public void addRemoteNetwork(): The method that adds a Network
to the list of remote cloud's networks.
o public void deleteRemoteNetwork(NetworkAddressIPv4 net):
The method that deletes a Network from the list of remote cloud's networks.
o public String gotoRemoteCloudsPage(): The method that redirects
to the remoteclouds.xhtml.
o public String updateDC2DC(): The method that inserts/updates a
DC2DCCommunication to db.
o public String editDC2DC(DC2DCCommunication dc): The method
that edits a DC2DCCommunication.


o public String deleteDC2DC(DC2DCCommunication dc): The
method that deletes a DC2DCCommunication from the db.
o public void changeLocalAS(): The method that maps a specific local
cloud name to the local AS number.
o public void changeRemoteAS(): The method that maps a specific
remote cloud name to the remote AS number.
o public String gotoInterDatacenterPage(): The method that
redirects to the interdatacenter.xhtml.
o public String updateTimeParams(): The method that
inserts/updates the TimeScheduleParameters to the db.
o public String gotoSettingsPage(): The method that redirects to the
settings.xhtml.
o public <K> K[] listToArray(Class<K> c, List<K>
objectsList): The method that transforms an arraylist of objects to an
array of objects.
o public Map<String, Integer> cloudListToMap(List<CloudDC>
cloudList): The method that transforms an array list of CloudDC objects
to a map of <cloud name, as number>.
Figure 43 presents a simple sequence diagram of an AS insertion through the User
Interface, which is triggered when the updateAS function is called. DashboardBean
calls the insert(AS) function of DB components ASDAO class, which subsequently
calls the related method from the AbstractASDAO, which executes the AS table update.
Afterwards, the findAll() is executed, to get all the AS entries and populate the table
presented in the page. The same action applies for all pages and methods defined above.

Figure 43: AS insertion sequence diagram
Finally, the view layer consists of all the pages, files and forms for the representation of
existing entries in the SBox database, and users request handling. The list of files is as
follows:
admin/: Its the directory where all web application pages exist:


o index.xhtml: The home page summarizing the User Interface features
(Figure 32)
o interdatacenter.xhtml: The inter-datacenter communications page
(Figure 40). When the user inserts an inter-datacenter communication, the
updateDC2DC of the DashboardBean is called, while the list of stored
DC2DCCommunication entries is rendered. The user can also delete an
inter-datacenter communication by pushing the Delete button, which calls
the deleteDC2DC function. The changeLocalAS and changeRemoteAS
functions are also used to dynamically adjust local and remote AS numbers,
based on users selection of local and remote cloud names respectively.
o isp.xhtml: The autonomous systems page (Figure 33). The user may
update the ISPs name (the updateIspName function is called), check
stored autonomous systems, add new ones (updateAS), update (editAS)
or delete (deleteAS) them.
o links.xhtml: The links page (Figure 36). The SBox administrator can
insert new links (updateLink), as well as update (editLink) or delete
(deleteLink) existing ones. The addSegment and deleteSegment
support functions are also used to add and delete, respectively, a Segment
for a selected Link.
o localclouds.xhtml: The local clouds page (Figure 38). The existing local
clouds are presented, and the user may insert (updateCloud) a local cloud,
edit (editCloud) or delete (deleteCloud) a stored one.
o remoteclouds.xhtml: The remote clouds page (Figure 39). The existing
remote clouds are presented, and the user may insert
(updateRemoteCloud) a remote cloud, edit (editRemoteCloud) or delete
(deleteRemoteCloud) a stored one.
o routers.xhtml: The BG and DA routers page (Figure 34). In this page,
the SBox administrator can insert (updateBGRouter, updateDARouter)
new BG and DA Routers, as well as delete (deleteBGRouter,
deleteDARouter) stored ones.
o sdncontrollers.xhtml: The SDN Controllers page (Figure 35). The
SBox administrator may check stored SDN Controllers, add new ones
(updateSDNController), update (editSDNController) or delete
(deleteSDNController) them.
o settings.xhtml: The DTM settings page (Figure 41). In this page, the
DTM execution parameters are configured and stored in the database
(updateTimeParams).
o tunnels.xhtml: The Tunnels page (Figure 37). The SBox administrator
can insert new tunnels (updateTunnel), as well as update (editTunnel)
or delete (deleteTunnel) existing ones.
css/: This directory includes all the css files used to format and style the web
application pages. Besides, the 2 files (sbox-main.css and sbox-signin.css)


that describe the SBox UI style and format, the directory includes all the Twitter
Bootstrap [52] css and theme files.
fonts/: It includes all Twitter Bootstrap [52] fonts.
js/: This directory includes all the javascript files. These files are used to support
certain Twitter Bootstrap [52] components (bootstrap.js, bootstrap-
dropdown.js, bootstrap-modal.js, bootstrap-datetimepicker.js,
jquery.js, moment.js), and support the validation of the user input. In the
second case, the bootstrapValidator.js acts as the key element of forms validation,
highlighting correct/incorrect fields and enabling/disabling Submit button, while
additional javascript files (between.js, integer.js, ip.js, notEmpty.js)
provide individual fields validation, providing relevant error messages (Figure 44).

Figure 44: Validation example
WEB-INF/: This directory includes the web.xml file, including web applications
configuration. The web application supports FORM authorization method, which
means that a new or session-expired request will be always redirected to a sign-in
page to provide its username and password. Currently, access is limited only to
administrators, with credentials encrypted in web servers configuration files.


signin.jsp: The sign-in page of the web application. In case of incorrect
credentials it redirects user to signinError.jsp page, otherwise to
index.xhtml.
signinError.jsp: The error page, when the user inserts incorrect credentials.
6.1.7.4 Tests
No automated tests were implemented for the User interface. The web application was
tested during integration and system tests.
6.2 SDN Controller
The main purpose of the use of the SDN Controller is dynamic flow table management. It
reactively adds new flows into the switch's flow table i.e., when a packet from a new flow
reaches an OpenFlow-enabled switch, the packet is sent to the controller, which evaluates
it, sends appropriate flow rule to the switch, and lets the switch continue its forwarding.
The SDN Controller decides to which tunnel the particular flow will be sent by inserting
appropriate output action. The decision is based on the current traffic statistics and the
values of the Reference and Compensation vectors received from S-Box.
6.2.1 SBox-SDN Server Interfaces
The SBox-SDN Server Interfaces expose constants and helper methods used commonly
by the SBox and by the SDN controller.
6.2.1.2 Interfaces
The SBox-SDN Server exposes following constant URLs on which the SDN controller
listens:
public static final String BASE_PATH Base path URL.
public static final String DTM_R_C_VECTORS_PATH Path used by
server-side resource used for editing and retrieving RCVectors, i.e., RVector and
CVector.
public static final String DTM_CONFIG_DATA_PATH Path used by
server-side resource used for editing and retrieving ConfigData.
Moreover, the helper methods for serialization, deserialization, and conversion from tunnel
id to OpenFlow switch port id are exposed:
public static <T extends Serializable> String serialize(T
object) Generic serializator.
public static <T extends Serializable> T deserialize(String
text, Class<T> clazz) Generic deserializator.
public static short toPort(TunnelID tunnelID) Converter of tunnel id to OpenFlow
switch port id.


6.2.1.3 Design
The SBox-SDN interfaces have been implemented in four classes, all within package
eu.smartenit.sdn.interfaces.sboxsdn. They are presented in Figure 45.
public class URLs (non-instantiable) holds information on common constant
URLs (BASE_PATH, DTM_R_C_VECTORS_PATH, and DTM_CONFIG_DATA_PATH)
as described in Section 6.2.1.2.
public class Serialization (non-instantiable) defines serialization and
deserialization helpers (serialize and deserialize) as described in Section
6.2.1.2.
public class SimpleTunnelIDToPortConverter defines converter of
simple tunnel id object to OpenFlow switch port number. It contains one method:
o public static short toPort(TunnelID tunnelID) converts
tunnel id object to OpenFlow switch port number.
public final class RCVectors implements Serializable provides a
container for reference and compensation vectors. It contains two fields (with
getters and setters):
o private RVector referenceVector reference vector;
o private CVector compensationVector compensation vector.

Figure 45: SBox-SDN Server Interface classes
6.2.1.4 Tests
One JUnit [20] class with five test methods has been implemented:
public class SerializationTest:
o public void testSerialize() tests serialization of RCVectors and
ConfigData.
o public void testDeserialize() tests deserialization of RCVectors
and ConfigData.
o public void
testLinkBGRouterBiDirReferenceSerialization() tests if bi-


directional reference between Link and BGRouter are correctly
(de-)serialized.
o public void testTunnelLinkBiDirReferenceSerialization()
tests if bi-directional reference between Tunnel and Link are correctly
(de-)serialized.
o public void testCloudDCASBiDirReferenceSerialization()
tests if bi-directional reference between CloudDC and AS are correctly
(de-)serialized.
6.2.2 SDN Controller based on Floodlight 0.90
The SDN Controller is a component responsible for flow distribution in the remote
autonomous system (DTM extension allows also distribute flows in a local autonomous
system). The NTM in the remote autonomous system provides R and C vectors for SDN
controller. The SDN controller monitors traffic transferred via tunnels and periodically
retrieves statistics from OpenFlow switch attached to the data center. Relying on this
information, the SDN controller takes the decision which tunnel should transfer newly
created flows in data center and directs flows to a proper tunnel. The appropriate
OpenFlow rule is prepared by the SDN controller and sent to OpenFlow switch. The REST
communication pattern is used for data delivery from the S-Box to the SDN controller. The
SDN controller communicates with switches using the OpenFlow protocol. The SDN
controller provides generally following functionalities:
communication with selected components (northbound interface REST,
southbound interface OpenFlow),
traffic statistics collection from tunnels,
OpenFlow flow rules establishment.
6.2.2.2 Interfaces
The SDN controller runs a HTTP server on port 8080 and exposes two REST interfaces
under following URLs:
URLs.BASE_PATH + URLs.DTM_R_C_VECTORS_PATH used to send (using
POST request) and retrieve (using GET request) reference and compensation
vector in JSON-encoded RCVectors container; the reference vector may be set to
null in this case it is not updated.
URLs.BASE_PATH + URLs.DTM_CONFIG_DATA_PATH used to send (using
POST request) and retrieve (using GET request) configuration data in JSON-
encoded ConfigData object.
6.2.2.3 Design
The SDN controller is designed as an extension to the Floodlight 0.90 controller [35]. The
functionality has been implemented in packages
eu.smartenit.sdn.floodlight090.dtm and


eu.smartenit.sdn.floodlight090.dtm.restlet. The former consists of six
classes, and the latter of three classes. They are presented in Figure 46. The sequence
diagram for processing a new incoming packet is shown in Figure 47.
public class DTM (singleton) main DTM class; holds information on current
compensation vector, reference vector, and traffic transmitted; it contains following
methods:
o public void setConfigData(ConfigData configData)/public
ConfigData getConfigData() update/return configuration;
o public synchronized void setReferenceVector(RVector
referenceVector)/public RVector getReferenceVector()
update/return reference vector;
o public synchronized void setCompensationVector(CVector
compensationVector)/public CVector
getCompensationVector() update/return compensation vector;
o public synchronized void setSwitch(IOFSwitch sw)/public
IOFSwitch getSwitch() update/return switch on which DTM operates;
currently, only one switch is supported;
o public static DTM getInstance() returns DTM singleton; creates
one if necessary;
o public long[] getTransmittedBytes() returns traffic counters on
switch ports;
o public long[] getTransmittedBytesStart() returns traffic start
counters on switch ports;
o public boolean isCompensating() returns true if DTM is
compensating the traffic;
o public TunnelID getCompensatingToTunnel() returns the id of
the tunnel to which DTM is compensating; returns null if the traffic is not
being compensated;
o public synchronized TunnelID getTunnel() returns tunnel id for
a new flow (pseudocode is available in Secion 6.2.2.3.1);
public class DTM.PortStatisticsPoller extends TimerTask timer
task responsible for getting swithchport statistics every
PORT_STATISTICS_POLLING_INTERVAL ms; it contains one method:
o public void run() - requests port statistics (tx bytes) from switch and
saves the state upon success;
public class DTMFloodlightModule implements IFloodlightModule
Floodlight module responsible for adding message listener, switch listener, and
RESTlet routable; it contains following methods:
o public Collection<Class<? extends IFloodlightService>>
getModuleServices() and public Map<Class<? extends
IFloodlightService>, IFloodlightService>


getServiceImpls() do nothing and return null; this module does not
implement any services;
o public Collection<Class<? extends IFloodlightService>>
getModuleDependencies() returns a list of modules that this module
depends on, i.e., IFloodlightProviderService IRestApiService;
o public void init(FloodlightModuleContext context)
initializes DTM module;
o public void startUp(FloodlightModuleContext context)
starts DTM module;
public class DTMMessageListener implements IOFMessageListener
adds a new flow when a packet in message is received from a switch; it contains
following methods:
o public Command receive(IOFSwitch sw, OFMessage msg,
FloodlightContext cntx) processes new packet and add a flow for
it;
o public String getName() returns the listener name;
o public boolean isCallbackOrderingPrereq(OFType type,
String name) and public boolean
isCallbackOrderingPostreq(OFType type, String name)
return false - this listener does not have pre/post-requisites.
public class DTMSwitchListener implements IOFSwitchListener
listens for new switches and configures main DTM class.
o public void addedSwitch(IOFSwitch sw) fired when a switch is
connected to the controller, and has sent a features reply; propagates the
information to DTM;
o public void removedSwitch(IOFSwitch sw) fired when a switch is
disconnected from the controller; propagates the information to DTM;
o public void switchPortChanged(Long switchId) fired when
ports on a switch change (any change to the collection of OFPhysicalPorts
and/or to a particular port); does nothing;
o public String getName() returns the listener name;
public class Flows (non-instantiable) manages flows; contains one method:
o public static void add(IOFSwitch sw,
IFloodlightProviderService floodlightProvider,
FloodlightContext cntx, short outPort, OFPacketIn pi)
adds flow for given switch, outgoing port, and incoming packet; constructs
OFMatch from the incoming packet and instructs switch to send all packets
from the flow to be sent to given outgoing port; the received packet is sent
do switch for forwarding; the controller sends the switch two OpenFlow
messages: ofp_flow_mod (with OFPFC_ADD action) and
ofp_packet_out;


public class DTMRestletRoutable implements RestletRoutable
Configures RESTlet resources; contains two methods:
o public Restlet getRestlet(Context context) returns the restlet
that will map to the resources; two resources are mapped:
RCVectorsResource ConfigDataResource;
o public String basePath() returns the base path URL where the
router should be registered;
public class ConfigDataResource extends ServerResource Server-
side resource used for editing and retrieving ConfigData; contains two mehods:
o public String handleGet() returns JSON-serialized DTM
configuration data;
o public String handlePost(String text) sets JSON-serialized
new DTM config data; returns JSON-serialized current DTM config data;
public class RCVectorsResource extends ServerResource Server-
side resource used for editing and retrieving RCVectors, i.e., RVector and CVector;
contains two methods:
o public String handleGet() returns JSON-serialized current R and C
vectors;
o public String handlePost(String text) sets JSON-serialized
new R and C vectors; returns JSON-serialized current R and C vectors.
The Floodlight classes were all left unchanged. The only modifications to the controller
that have been done were the configuration files the DTM module has been enabled
and the Forwarding module has been disabled.


Figure 46: SDN Controller based on Floodlight 0.9 classes


Figure 47: Sequence diagram for processing a new packet.
6.2.2.3.1 Pseudocode for DTM.getTunnel()

r_inv[1] = R[2] / (R[1] + R[2])
r_inv[2] = R[1] / (R[1] + R[2])
if(compensating)
if traffic[tunnel] < C[tunnel] / r_inv [tunnel]
send_SDN_rule(tunnel);
else
compensating = false
reset_transmitted_bytes_start
if tunnel = 1
send_SDN_rule(2)
else
send_SDN_rule(1)
end if
else
if traffic[1] / (trafic[1] + traffic[2]) <= R[1] / (R[1] + R[2])
send_SDN_rule(1);
else
send_SDN_rule(2);
end if
end if



6.2.2.4 Tests
Three JUnit [20] classes were developed to test the implementation:
public class DTMTest main DTM class test; contains following tests:
o public void testGetInstance() tests if DTM.getInstance() returns the same
value in subsequent calls;
o public void testSetConfigData() tests the configuration data setter;
o public void testSetReferenceVector() tests the reference vector setter and
getter.
o public void testSetCompensationVector() tests the compensation vector
setter and getter;
o public void testSetSwitch() tests the DTM.setSwitch(IOFSwitch) method;
o public void testGetTunnel() tests the algorithm (DTM.getTunnel() method);
public class ConfigDataResourceTest test the server-side resource used for
editing and retrieving ConfigData; contains one test:
o public void testHandlePost() tests if the configuration data is deserialized
correctly and if it is passed to DTM instance;
public class RCVectorsResourceTest test the server-side resource used for
editing and retrieving R and C vectors; contains one test:
o public void testHandlePost() tests if the R and C vectors is deserialized
correctly and if they are passed to DTM instance.



7 Summary and Conclusions
In this deliverable we reported the engineering activities of the WP3, including the
technology scouting, development, integration and release processes.
Based on WP2 progress and specified traffic management mechanisms, a set of tools
and technologies were investigated to support the implementation and deployment of
these mechanisms, and finally, the consortium has selected a specific subset. Java will be
the programming language for most entities (SBox, uNaDa, SDN controller, end-user
device), due to its open-source nature, ease of development, libraries support and
partners familiarity. JBoss Netty NIO framework was selected as the key communication
framework, and all SmartenIT entities will be developed on top of this. JSON will be the
protocol serialization format for the internal entities interfaces, to ensure the homogeneity
of the prototype, taking into account that ALTO server/clients also use JSON. Finally, a
couple of additional decisions were made with regards to supporting libraries and tools,
e.g. Floodlight is preferred over Opendaylight as the SDN Controller, OpenStack will be
deployed in case a cloud management platform is required, SNMP for QoS
measurements, and RestFB library will be used to retrieve required information from the
Facebook API.
The WP3 engineering team agreed and set up a common implementation framework,
including specific tools and methodologies to ease and support the work of developers,
integrators and testers. The SmartenIT implementation cycle adopts the agile
development and continuous integration concepts, aiming at 2-month release cycles
and rapid and efficient code development, issue identification and resolution. To support
these concepts, a Subversion source code repository, the JIRA issue management
platform and a Jenkins continuous integration server were deployed in PSNCs premises.
In addition, certain people were given specific roles (product manager, tools administrator,
release master) to efficiently organize and manage the implementation team and ensure
the prototypes quality. Documents (specification and release reports) and meetings
(feature planning, weekly and review) were introduced to report progress in all
implementation cycle tasks. Finally, the system tree was generated, including all
necessary dependencies and required tools, to develop, build and deploy SmartenIT
modules.
The first version of the SmartenIT software implements the DTM mechanism components,
in 2 architecture entities: SBox and SDN Controller. SBox includes the following
components: The QoS Analyzer retrieves the required SNMP measurements from the
network entities, the Economic Analyzer calculates the reference vector based on links
costs and the Network Traffic Manager calculates the compensation vector and also
coordinates the interaction with the other entities (remote SBox, local SDN Controller).
The inter-SBox Communication Service is responsible for sending and receiving the
calculated reference and compensation vector to/from other SBoxes. The SBox-SDN
executes the same functionality towards the local SDN controller, and finally, UI and DB
provide the necessary interfaces to save and access required configuration. The SDN
controller is built on top of Floodlight controller source code to implement DTM
functionality, which translates received compensation and reference vectors into flow
rules. It also exposes interfaces towards the SBox to receive these vectors and additional
configuration parameters.


All the aforementioned tools and procedures aimed at and ensured a commercially
developed and integrated prototype, following industry standards. The first version of the
SmartenIT was released successfully, and now the project proceeds with the
implementation of additional mechanisms and features that highlight and fulfil the projects
objectives and vision.


8 SMART Objectives Addressed
This deliverable addresses certain SmartenIT SMART objectives defined in Section
B1.1.2.4 of the SmartenIT Description of Work (DoW) [89]. Specifically, the deliverable
addresses the Work Package 3 SMART Objective (Table 8-1), 2 overall SMART
objectives (Table 8-2) and one prototyping SMART objective (Table 8-3).
The WP3 Objective is defined in the DoW as follows:
Table 8-1: Work Package 3 SMART objective addressed.
Objective 3 SmartenIT will investigate economically, QoE, and energy-wise such models
in theory by simulations to guide the respective prototyping work in terms of
a successful, viable, and efficient architectural integration, framework and
mechanisms engineering, and its subsequent performance evaluation.
This deliverable is the first outcome of the engineering work performed in WP3, describing
the selected technologies (Chapter 4) and the overall implementation framework (Chapter
5) for all the SmartenIT software releases, to ensure and produce an integrated prototype,
following industry standards and implementation procedures. In addition, it provides the
technical documentation of the first selected, specified and implemented traffic
management mechanism, the DTM mechanism (Chapter 6). Future releases of the
SmartenIT software will implement additional mechanisms, specified and simulated by
WP2, which will cover more aspects of the aforementioned objective and also fulfill the
projects goals. Hence, this deliverable partially addresses Objective 3, in the sense that a
successful, viable, and efficient architectural integration, framework and mechanisms
engineering have been performed and will keep evolving as the work package
progresses.
In addition, the overall SmartenIT SMART objectives presented in Table 8-2 have been
addressed in the current document. The SmartenIT architecture has progressed since
Deliverable 3.1 [2], covering more aspects of the specified traffic management
mechanisms, and also the advancements and decisions made in the engineering work of
the consortium. The consortium decided and set up the implementation framework to
support the software implementation, fulfilling MS3.2. Certain entities, components and
interfaces of the SmartenIT architecture were developed in the first release of the
SmartenIT prototype, achieving the MS3.3, implementing the DTM mechanism. The WP3
engineering team currently progresses with the implementation of additional mechanisms,
as these were selected by the consortium, aiming to achieve the projects goals.
Table 8-2: Overall SmartenIT SMART objectives addressed.
Objective
No.
Specific
Measurable
Achievable Relevant
Timely
Deliverable
Number
Mile Stone Number
O3
Framework and
mechanism engineering
D3.2 Implementation
Advanced MS3.2, MS3.3
Architectural integration D3.2 Implementation Complex MS3.2, MS3.3


Finally, Deliverable 3.2 partially addresses the prototyping SMART objective O3.2, by
performing a comparison on the available tools and technologies to integrate with existing
network management platforms and finally selecting the most appropriate ones (Section
4.3.3.3). This SMART objective will be fully addressed in the upcoming deliverables and
milestones, as well as the final releases of the SmartenIT prototype.
Table 8-3: Prototyping SmartenIT SMART objectives addressed.
Objective
ID
Specific
Measurable
Achievable Relevant
Timely
Metric
Project
Month
O3.2
At which level to integrate
the proposed solution with
the existing network
management platforms?
Number of considered
management platforms,
number of comparison
studies to existing
solutions
Prototyping
T3.2, T3.3
Highly relevant
output of
relevance for
providers
M24


9 References
[1] The SmartenIT project: Deliverable D2.2 - Report on Definitions of Traffic Management
Mechanisms and Initial Evaluation Results; October 2013.
[2] The SmartenIT project: Deliverable D3.1 - Report on Initial System Architecture; April
2013
[3] Java Development Kit, a development environment for building applications, applets,
and components using the Java programming language, online. Available at:
http://www.oracle.com/technetwork/java/javase/downloads/jdk7-downloads-
1880260.html
[4] Ubuntu - a Debian-based Linux operating system, online. Available at:
http://www.ubuntu.com/
[5] Raspberry Pi - a low cost, credit-card sized computer, online. Available at:
http://www.raspberrypi.org/
[6] Android - a mobile operating system based on the Linux kernel that is currently
developed by Google, online. Available at: http://www.android.com/
[7] iOS - a mobile operating system developed by Apple Inc. and distributed exclusively
for Apple hardware, online. Available at: https://www.apple.com/ios/
[8] OSGi - The Dynamic Module System for Java, online. Available at:
http://www.osgi.org
[9] The Netty project, online. Available at: http://netty.io/
[10] Apache Karaf, online. Available at: http://karaf.apache.org/
[11] The Apache CXF Distributed OSGi subproject, online. Available at:
http://cxf.apache.org/distributed-osgi.html
[12] Apache ActiveMQ, online. Available at: http://activemq.apache.org/
[13] Apache Tomcat, online. Available at: http://tomcat.apache.org/
[14] Eclipse Jetty, online. Available at: http://www.eclipse.org/jetty/
[15] The Primeface JSF framework, online. Available at: http://primefaces.org/
[16] The H2 Database Engine, online. Available at:
http://www.h2database.com/html/main.html
[17] Apache Derby, online. Available at: http://db.apache.org/derby/
[18] EclipseLink JPA, online. Available at: http://www.eclipse.org/eclipselink/jpa.php
[19] Apache Maven, online. Available at: http://maven.apache.org/
[20] JUnit a programmer-oriented testing framework for Java, online. Available at:
http://junit.org/
[21] Pax Exam, online. Available at:
https://ops4j1.jira.com/wiki/display/PAXEXAM3/Pax+Exam
[22] SonarQube, online. Available at: http://www.sonarqube.org/


[23] Cobertura a code coverage utility for Java, online. Available at:
http://cobertura.github.io/cobertura/
[24] JaCoCo Java Code Coverage Library, online. Available at:
http://www.eclemma.org/jacoco/
[25] E. Rosen, A. Viswanathan and R. Callon, Multiprotocol Label Switching
Architecture; RFC 3031, January 2001.
[26] iperf - TCP and UDP bandwidth performance measurement tool., online. Available
at: https://code.google.com/p/iperf/
[27] S. Shalunov, B. Teitelbaum, A. Karp, J. Boote and M. Zekauskas, A One-way
Active Measurement Protocol (OWAMP); RFC 4656, September 2006
[28] sFlow, online. Available at: http://www.sflow.org/
[29] RRDtool logging and graphing, online. Available at: http://oss.oetiker.ch/rrdtool/
[30] PERFormance Service Oriented Network monitoring ARchitecture, online. Available
at: http://www.perfsonar.net/
[31] The SmartenIT project: Deliverable D1.2 - Report on of Cloud Service
Classifications and Scenarios; October 2013.
[32] OpenFlow: Open Switch specification, online. Available at:
https://www.opennetworking.org/sdn-resources/onf-specifications/openflow
[33] Open vSwitch, online. Available at: http://openvswitch.org/
[34] Mininet - An Instant Virtual Network on your Laptop, online. Available at:
http://mininet.org/
[35] Project Floodlight, online. Available at: http://www.projectfloodlight.org/floodlight/
[36] Opendaylight SDN Controller, online. Available at:
http://www.opendaylight.org/project
[37] R. Enns, Ed., "NETCONF Configuration Protocol", RFC 4741, December 2006
[38] J. Case, M. Fedor, M. Schoffstall, J. Davin, "A Simple Network Management
Protocol (SNMP)", RFC 1157, May 1990
[39] Protocol Buffers - Google's data interchange format, online. Available at:
https://developers.google.com/protocol-buffers/
[40] Microsoft Visio, online. Available at: http://office.microsoft.com/en-001/visio/
[41] Boost.Asio - C++ network and low-level I/O programming library, online. Available
at: http://www.boost.org/doc/libs/1_55_0/doc/html/boost_asio.html
[42] CppCMS - High Performance Web Development Framework, online. Available at:
http://cppcms.com/wikipp/en/page/main
[43] httpd - Apache Hypertext Transfer Protocol Server, online. Available at:
https://httpd.apache.org/
[44] nginx - HTTP and reverse proxy server, online. Available at: http://nginx.org/
[45] SQLite - SQL database engine, online. Available at: http://www.sqlite.org/
[46] CMake - Cross Platform Make, online. Available at: http://www.cmake.org/


[47] Google Test - C++ unit testing library, online. Available at:
https://code.google.com/p/googletest/
[48] gcov - Test Coverage Program, online. Available at:
https://gcc.gnu.org/onlinedocs/gcc/Gcov.html
[49] Twisted - Python event-driven networking engine, online. Available at:
https://twistedmatrix.com/trac/
[50] Django - Python Web framework, online. Available at:
https://www.djangoproject.com/
[51] unittest - Python Unit testing framework, online. Available at:
https://docs.python.org/2/library/unittest.html
[52] Bootstrap front-end framework, online. Available at: http://getbootstrap.com/
[53] OAuth Community Site, online. Available at: http://oauth.net/
[54] Spring Social, online. Available at: http://projects.spring.io/spring-social/
[55] Facebook4J - open-source Java Facebook monitoring library, online. Available at:
http://facebook4j.org/en/index.html
[56] RestFB - Facebook Graph API and REST API client in Java, online. Available at:
http://restfb.com/
[57] Facebook Bugs, online. Availables at:
https://developers.facebook.com/x/bugs/694034083948939
[58] Twitter4J - Java library for the Twitter API, online. Available at:
http://twitter4j.org/en/index.html
[59] Facebook, online. Available at: https://www.facebook.com/
[60] Twitter, online. Available at: https://twitter.com/
[61] Apache CloudStack: Open Source Cloud Computing, online. Available at:
http://cloudstack.apache.org/
[62] OpenStack: Open Source Cloud Computing Software, online. Available at:
https://www.openstack.org/
[63] DevStack - Deploying OpenStack for Developers, online. Available at:
http://devstack.org/
[64] SIGAR API: System Information Gatherer and Reporter library, online. Available at:
http://www.hyperic.com/products/sigar
[65] psutil: A cross-platform process and system utilities module for Python, online.
Available at: https://code.google.com/p/psutil/
[66] ITG, online. Available at: http://traffic.comics.unina.it/software/ITG/
[67] Tstat: TCP STatistic and Analysis Tool, online. Available at:
http://tstat.tlc.polito.it/index.shtml
[68] CoralReef, online. Available at: http://www.caida.org/tools/measurement/coralreef/
[69] Mockito - open source testing framework, online. Available at:
https://code.google.com/p/mockito/


[70] WireMock: Java library for stubbing and mocking web services, online. Available at:
http://wiremock.org/
[71] Image Quality Assessment (IQA) Open-source C library for measuring image
quality. Available at http://tdistler.com/iqa/
[72] VQMT: Video Quality Measurement Tool, online. Available at:
http://mmspg.epfl.ch/vqmt
[73] VitALU, Video Inspection Tool designed and used internally at Alcatel-Lucent,
http://www2.alcatel-lucent.com/techzine/an-innovative-tool-for-measuring-video-
streaming-qoe/
[74] FFmpeg: open source libraries and programs for encoding/decoding multimedia
data. Available at http://ffmpeg.org/
[75] Wireshark: open source libraries and programs for analyzing network
communications and protocols. Available at http://www.wireshark.org/
[76] Srinivasan, C., Viswanathan, A., and T. Nadeau, "Multiprotocol Label Switching
(MPLS) Traffic Engineering (TE) Management Information Base (MIB)", RFC 3812,
June 2004.
[77] Apache Subversion software, online. Available at: http://subversion.apache.org/
[78] Atlassian JIRA, online. Available at: https://www.atlassian.com/software/jira
[79] Jenkins open source continuous integration server, online. Available at:
http://jenkins-ci.org/
[80] Crockford, D., "The application/json Media Type for JavaScript Object Notation
(JSON)", RFC 4627, July 2006.
[81] B. Claise, Ed., "Cisco Systems NetFlow Services Export Version 9", RFC 3954,
October 2004
[82] Gredler, H. et al., North-Bound Distribution of Link-State and TE Information using
BGP, draft-ietf-idr-ls-distribution-05, May 2014.
[83] Ylonen, T., Lonvick C., The Secure Shell (SSH) Protocol Architecture, RFC 4251,
January 2006
[84] Telcordia Technologies, GR-831-CORE: Operations Application Messages -
Language For Operations Application Messages, November 1996
[85] Thomas Karagiannis, Konstantina Papagiannaki, and Michalis Faloutsos. 2005.
BLINC: multilevel traffic classification in the dark. SIGCOMM Comput. Commun. Rev.
35, 4 (August 2005), 229-240. DOI=10.1145/1090191.1080119
[86] perfSONAR software latest releases, online. Available at:
https://forge.geant.net/forge/display/perfsonar/Downloads
[87] BWCTL: Bandwidth Test Controller, online. Available at:
http://software.internet2.edu/bwctl/
[88] The GANT project, online. Available at: http://www.geant.net/about
[89] The SmartenIT Consortium: Grant Agreement for STREP: Annex I Description of
Work (DoW). 2012.


10 Abbreviations
ALTO Application Layer Traffic Optimization
API Application Programming Interface
AS Autonomous System
BG Border Gateway
BGP Border Gateway Protocol
DA Datacenter Attachment
DAO Data Access Object
DB Database
DC Datacenter
DCO Datacenter Operator
DHCP Dynamic Host Configuration Protocol
DLL Dynamic Link Library
DNS Domain Name System
DOSGi Distributed OSGi
DTM Dynamic Traffic Management
DTO Data Transfer Object
ECA Economic Analyzer
FQL Facebook Query Language
GPS Global Positioning System
HTTP Hypertext Transport Protocol
ISP Internet Service Provider
JDK Java Development Kit
JSF Java Server Faces
JSON Javascript Object Notation
KVM Kernel-based Virtual Machine
LSP Labeled-Switched Path
LXC LinuX Containers
MIB Management Information Base
MOS Mean Opinion Score
MPLS Multiprotocol Labeled Switching
MVC Model-View-Controller
NIO Non-blocking I/O
NOC Network Operations Center


NSP Network Service Provider
NTM Network Traffic Manager
OSN Online Social Network
OSGi Open Service Gateway Initiative
PC Personal Computer
POM Project Object Model
QOA QoS/QoE Analyzer
QoE Quality of Experience
QoS Quality of Service
REST Representational State Transfer
RFC Request for Comments
RRD Round Robin Database
RTP Real-time Transport Protocol
RTSP Real Time Streaming Protocol
SAL Service Abstraction Layer
SDN Software Defined Networking
SmartenIT Socially-aware Management of New Overlay Application Traffic with Energy
Efficiency in the Internet, EU STREP project
SNMP Simple Network Management Protocol
STREP Specific Targeted Research Project
TCP Transmission Control Protocol
TMS Traffic Management Solution
UDP User Datagram Protocol
UI User Interface
UML Unified Modeling Language
URL Uniform Resource Locator
VM Virtual Machine
VPLS Virtual Private LAN Service
VPN Virtual Private Network
WP Work Package


11 Acknowledgements
This deliverable was made possible due to the large and open help of the WP3 team of
the SmartenIT team within this STREP. Many thanks to all of them.


Appendix A. History of system releases
A.1. Version 1.0
Version 1.0 of the SmartenIT software implements the DTM mechanism functionalities. A
set of features were identified before the implementation start and were reported to the
JIRA issue management platform, to be resolved by responsible developers. The list of
features is presented in Figure 48.

Figure 48: v1.0 features
After the development completion, in which all features were implemented and tested and
respective issues were resolved, the integration phase began, in which components
communication would be tested, and a basic sanity testing would be performed. The
integration tests are either JUnit tests, including mock SmartenIT entities (e.g. mock SDN
Controller), JUnit tests, including deployed SmartenIT entities (e.g. SDN Controller, SNMP
daemon), and real test cases in a real testbed scenario. In the latter case, the testbed of


Figure 49 was created and configured to deploy the SmartenIT software and test a basic
subset of v1.0 functionalities. The integration testbed design and configuration was based
on the basic testbeds (Section 5.1.5) design and requirements, and contains only a
subset of machines and entities, to perform some basic sanity testing.

Figure 49: v1.0 integration testbed
Integration test cases are presented in Table 11-1. For each test, a short description, the
involved components, the testing environment, the required configuration and execution
procedure, and finally the expected results are presented. A few issues were identified,
which were reported to the JIRA issue management platform and are also presented in
Figure 50. Each responsible developer had to fix his assigned issues, until all integration
tests passed. Eventually, all issues were resolved.
Table 11-1: v1.0 integration test cases
Description Components Environment Configuration/Execution Expected Outcome
NTM
initialization
at sending
domain with
mock SDN
controller
NTM, DB,
SBox-SDN
JUnit test
case, with
mock SDN
controller
The smartenit.db file is
filled with all the domain-
required parameters. The
DB component is initialized
to point to this file.
NTM is initialized in
TRAFFIC_SENDER mode,
and is expected to extract
the required information
and send the configuration
data to the mock SDN
controller.
No exceptions or logs that
indicate configuration
failures.
Verify that mock SDN
controller config-data REST
API was called once and
with required header
parameters.
NTM and
inter-sbox
NTM, inter-
SBox, DB,
JUnit test
case, with


server
initialization
at sending
domain with
mock SDN
controller
SBox-SDN mock SDN
controller
TRAFFIC_SENDER mode.
Inter-SBox server is also
initialized including the
constructed NTM.
One instance of inter-sbox
client is initialized to send
sample reference and
compensation vectors.
NTM is expected to extract
from the database and
send the configuration data
to the mock SDN controller.
When the inter-sbox server
receives the reference and
compensation vectors, it
should send them to the
mock SDN controller.
failures.
parameters.
controller r-c-vectors REST
parameters.
NTM and
inter-sbox
server
initialization
at sending
domain with
mock SDN
controller,
with multiple
inter-sbox
client
requests
NTM, inter-
SBox, DB,
SBox-SDN
JUnit test
case, with
mock SDN
controller
TRAFFIC_SENDER mode.
initialized including the
constructed NTM.
Multiple instances (10) of
inter-sbox client are
initialized to send sample
reference and
compensation vectors.
NTM is expected to extract
from the database and
send the configuration data
to the mock SDN controller.
Each time that the inter-
sbox server receives the
reference and
compensation vectors, it
should send them to the
mock SDN controller.
failures.
parameters.
API was called 10 times
and with required header
parameters.
NTM
initialization
at receiving
domain, and
inter-sbox
initialization
at sending
NTM, inter-
SBox, DB
JUnit test
case
failures.
Inter-sbox server receives
the compensation vector.


domain TRAFFIC_RECEIVER
mode.
initialized, mocking the
sending domain.
Random X and R vectors
are updated to the NTM,
which is expected to
prepare the compensation
vector and send them to
the inter-sbox server.
NTM
initialization
at receiving
domain, with
inter-sbox
initialization
at sending
domain
NTM, inter-
SBox, DB
JUnit test
case
TRAFFIC_RECEIVER
mode.
sending domain.
Random X and R vectors
are updated to the NTM
multiple times (10), which is
expected to prepare the
compensation vectorS and
send them to the inter-sbox
server accordingly.
failures.
the compensation vector 10
times.
NTM
initialization
at receiving
domain,
NTM
initialization
at sending
domain, with
mock SDN
controller
NTM, inter-
SBox, DB,
SBox-SDN
JUnit test
case with
mock SDN
controller
This test case is supposed
to emulate 2 domains,
sending and receiving, and
also mock the SDN
controller at the sending
domain.
2 db files are prepared, one
for each domain, with
symmetric stored
information.
NTM at sending domain is
initialized in
along with the inter-sbox
server. It extracts the
required configuration data
and sends them to the
mock SDN controller, using
the SBox-SDN interface.
Then, NTM at receiving
domain is also initialized in
TRAFFIC_RECEIVER
mode, and it gets updated
with an R and X vector.
failures.
controller receives one
request for configuration
data, with all the required
http header parameters.
the compensation vector
and provides it to the NTM
at receiving domain.
parameters.


It should then calculate the
compensation vector and
send it along with the
reference one to the other
domain, through the inter-
sbox communication
service, and configure the
mock SDN Controller.
ECA and
NTM
initialization
at receiving
domain, with
inter-sbox
initialization
at sending
domain
ECA, NTM,
inter-SBox,
DB
JUnit test
case
TRAFFIC_RECEIVER
mode.
sending domain.
Random X and Z vectors
are updated to the ECA and
NTM, and it is expected
that NTM will prepare the
send them to the inter-sbox
server.
failures.
the compensation vector.
ECA and
NTM
initialization
at receiving
domain,
NTM
initialization
at sending
domain, with
mock SDN
controller
ECA, NTM,
inter-SBox,
DB, SBox-
SDN
JUnit test
case with
mock SDN
controller
also mock the SDN
domain.
symmetric stored
information.
initialized in
and sends them to the
mock SDN controller, using
the SBox-SDN interface.
Then, ECA and NTM at
receiving domain are also
initialized in
TRAFFIC_RECEIVER
mode, and they get
updated for multiple times
with random X and Z
vectors.
ECA should calculate the
failures.
controller receives one
request for configuration
data, with all the required
http header parameters.
the compensation and
reference vectors and
provides them to the NTM
at receiving domain.
parameters.


reference vector and NTM
should then calculate the
then send them to the other
sbox communication
service, and configure the
mock SDN Controller.
QoA
initialization
QOA, ECA,
NTM, DB
JUnit test
case with
snmp
deamon
running in a
remote VM
This test case should check
whether the major
components of the SBox
are initialized properly.
A smartenit.db file is filled
with all the domain-required
parameters. The DB
component is initialized to
point to this file.
QOA, ECA and NTM
(TRAFFIC_RECEIVER
mode) should be initialized
properly.
failures.
NTM
initialization
at sending
domain with
SDN
controller
NTM, DB,
SBox-SDN,
SDN
Controller,
SDN
Controller
interfaces
JUnit test
case, with
SDN
controller
running
The SDN Controller is
running, listening for
configuration data at port
8080.
and is expected to extract
and send the configuration
data to the SDN controller.
failures.
Check that SDN controller
config-data REST API was
called and receives the
configuration data without
any issues.
NTM
initialization
and r/c
vectors
update at
sending
domain with
SDN
controller
ECA, NTM,
DB, Inter-
SBox, SBox-
SDN, SDN
Controller,
SDN
Controller
interfaces
JUnit test
case, with
SDN
controller
running
also mock the SDN
domain.
symmetric stored
information.
initialized in
and sends them to the SDN
controller, using the SBox-
failures.
config-data REST API was
called and receives the
configuration data without
any issues.
r-c-vectors REST API was
called and receives the r
and c vectors without any
issues.


SDN interface.
Then, ECA and NTM at
receiving domain are also
initialized in
TRAFFIC_RECEIVER
mode, and they get
updated for multiple times
with random X and Z
vectors.
ECA should calculate the
reference vector and NTM
should then calculate the
then send them to the other
sbox communication
service, and finally update
the SDN Controller.
UI
configuratio
n
UI, DB Integration
testbed
Run the Sbox web
application and store all the
required domain
parameters.
indicate failures.
Inserted parameters stored
and presented as expected.
Basic Sanity
testing
ALL Integration
testbed
Run and configure 1 SBox
and 1 SDN controller, along
with 1 switch per domain.
Enable snmp daemon in
each switch.
Configure the database of
each SBox with the
required domain
information (Test case
#13).
Initialize the SBoxes and
the SDN Controllers.
SDN controllers should
initially receive the
configuration data.
QOA in receiving domain
reads counters and
provides X and Z vectors to
NTM and ECA. After the
accounting period, the R
and C vectors are sent to
the SBox of the sending
domain and then the SDN
controller is also updated.
indicate failures.
Compare logs and verify
that reference and
compensation vectors are
sent to the sending domain,
and then to the SDN
Controller.

The list of found issues during integration is presented in Figure 50.


Figure 50: v1.0 integration issues

Figure 51: Jenkins view
All issues were resolved and closed, and eventuall all integration tests were successful.
SmartenIT build is stable (Figure 51), and v1.0 was finally released.



Appendix B. Internal Interfaces Protocol Format
Finally, the JSON format of the internal interfaces protocol is presented, including some
sample values, as these were retrieved from the integration testing logs.
B.1. Reference and Compensation Vectors JSON Format
{"referenceVector":{
"vectorValues":[{"value":26511,"linkID":{"class":"eu.smartenit.sbox.db.d
to.SimpleLinkID","localLinkID":"link2","localIspName":"isp1"}},{"value":
20,"linkID":{"class":"eu.smartenit.sbox.db.dto.SimpleLinkID","localLinkI
D":"link1","localIspName":"isp1"}}],"sourceAsNumber":1,"thetaCollection"
:null},
"compensationVector":{
"vectorValues":[{"value":39340599,"linkID":{"class":"eu.smartenit.sbox.d
b.dto.SimpleLinkID","localLinkID":"link2","localIspName":"isp1"}},{"valu
e":-
39340599,"linkID":{"class":"eu.smartenit.sbox.db.dto.SimpleLinkID","loca
lLinkID":"link1","localIspName":"isp1"}}],"sourceAsNumber":1}}

B.2. Configuration Data JSON Format
{"inCommunicationList":[],
"outCommunicationList":[{
"id":{
"communicationNumber":1,
"communicationSymbol":"fb-drop",
"localAsNumber":2,
"localCloudDCName":"Facebook",
"remoteAsNumber":1,
"remoteCloudDCName":"Dropbox"},
"trafficDirection":"outcomingTraffic",
"remoteCloud":{"@id":"CloudDC@1","cloudDcName":"Dropbox","as":{"@id":"AS
@1","asNumber":1,"local":false,"bgRouters":null,"sbox":{"managementAddre
ss":{"prefix":"146.124.2.222","prefixLength":0}},"localClouds":null},"da
Router":{"managementAddress":{"prefix":null,"prefixLength":0},"snmpCommu
nity":null},"sdnController":{"managementAddress":{"prefix":null,"prefixL
ength":0},"daRouters":null,"restHost":{"prefix":null,"prefixLength":0},"
restPort":0,"openflowHost":{"prefix":null,"prefixLength":0},"openflowPor


t":0},"dcNetworks":[{"prefix":"1.1.1.1","prefixLength":32}]},
"localCloud":{"@id":"CloudDC@2","cloudDcName":"Facebook","as":{"@id":"AS
@2","asNumber":2,"local":true,"bgRouters":[{"@id":"BGRouter@1","manageme
ntAddress":{"prefix":"146.124.2.74","prefixLength":0},"snmpCommunity":"s
martenit","interDomainLinks":[{"@id":"Link@1","linkID":{"class":"eu.smar
tenit.sbox.db.dto.SimpleLinkID","localLinkID":"link1","localIspName":"is
p1"},"address":{"prefix":"146.124.2.74","prefixLength":0},"physicalInter
faceName":"eth0","vlan":2221,"inboundInterfaceCounterOID":null,"outbound
InterfaceCounterOID":null,"costFunction":{"type":"Cost-
function","subtype":null,"inputUnit":"bps","outputUnit":"euros","segment
s":[{"leftBorder":0,"rightBorder":10,"a":2.0,"b":1.0},{"leftBorder":10,"
rightBorder":20,"a":20.0,"b":1.0},{"leftBorder":20,"rightBorder":-
1,"a":-
2.0,"b":1.0}]},"bgRouter":"BGRouter@1","traversingTunnels":[{"@id":"Tunn
el@1","tunnelID":{"class":"eu.smartenit.sbox.db.dto.SimpleTunnelID","tun
nelName":"tunnel1","tunnelNumber":1},"link":{"@id":"Link@2","linkID":{"c
lass":"eu.smartenit.sbox.db.dto.SimpleLinkID","localLinkID":"link1","loc
alIspName":"isp1"},"address":{"prefix":null,"prefixLength":0},"physicalI
nterfaceName":null,"vlan":0,"inboundInterfaceCounterOID":null,"outboundI
nterfaceCounterOID":null,"costFunction":{"type":null,"subtype":null,"inp
utUnit":null,"outputUnit":null,"segments":[]},"bgRouter":{"@id":"BGRoute
r@2","managementAddress":{"prefix":null,"prefixLength":0},"snmpCommunity
":null,"interDomainLinks":null},"traversingTunnels":null},"sourceEndAddr
ess":{"prefix":"2.2.2.2","prefixLength":0},"destinationEndAddress":{"pre
fix":"1.1.1.1","prefixLength":0},"physicalLocalInterfaceName":"eth0","in
boundInterfaceCounterOID":null,"outboundInterfaceCounterOID":null}]},{"@
id":"Link@3","linkID":{"class":"eu.smartenit.sbox.db.dto.SimpleLinkID","
localLinkID":"link2","localIspName":"isp1"},"address":{"prefix":"146.124
.2.74","prefixLength":0},"physicalInterfaceName":"lo","vlan":21,"inbound
InterfaceCounterOID":null,"outboundInterfaceCounterOID":null,"costFuncti
on":{"type":"Cost-
1,"a":-
2.0,"b":1.0}]},"bgRouter":"BGRouter@1","traversingTunnels":[{"@id":"Tunn
el@2","tunnelID":{"class":"eu.smartenit.sbox.db.dto.SimpleTunnelID","tun
nelName":"tunnel2","tunnelNumber":2},"link":{"@id":"Link@4","linkID":{"c
lass":"eu.smartenit.sbox.db.dto.SimpleLinkID","localLinkID":"link2","loc
alIspName":"isp1"},"address":{"prefix":null,"prefixLength":0},"physicalI
nterfaceName":null,"vlan":0,"inboundInterfaceCounterOID":null,"outboundI
nterfaceCounterOID":null,"costFunction":{"type":null,"subtype":null,"inp
utUnit":null,"outputUnit":null,"segments":[]},"bgRouter":{"@id":"BGRoute
r@3","managementAddress":{"prefix":null,"prefixLength":0},"snmpCommunity
":null,"interDomainLinks":null},"traversingTunnels":null},"sourceEndAddr
ess":{"prefix":"2.2.2.2","prefixLength":0},"destinationEndAddress":{"pre
fix":"1.1.1.1","prefixLength":0},"physicalLocalInterfaceName":"lo","inbo
undInterfaceCounterOID":null,"outboundInterfaceCounterOID":null}]}]}],"s
box":{"managementAddress":{"prefix":"146.124.2.178","prefixLength":0}},"
localClouds":[{"@id":"CloudDC@3","cloudDcName":"Facebook","as":{"@id":"A
S@3","asNumber":2,"local":false,"bgRouters":null,"sbox":{"managementAddr
ess":{"prefix":"146.124.2.178","prefixLength":0}},"localClouds":null},"d
aRouter":{"managementAddress":{"prefix":"146.124.2.74","prefixLength":0}
,"snmpCommunity":null},"sdnController":{"managementAddress":{"prefix":"1
46.124.2.178","prefixLength":0},"daRouters":null,"restHost":{"prefix":nu
ll,"prefixLength":0},"restPort":0,"openflowHost":{"prefix":null,"prefixL


ength":0},"openflowPort":0},"dcNetworks":[{"prefix":"2.2.2.2","prefixLen
gth":32}]}]},"daRouter":{"managementAddress":{"prefix":"146.124.2.74","p
refixLength":0},"snmpCommunity":"smartenit"},"sdnController":{"managemen
tAddress":{"prefix":"146.124.2.178","prefixLength":0},"daRouters":null,"
restHost":{"prefix":"146.124.2.178","prefixLength":0},"restPort":8080,"o
penflowHost":{"prefix":"146.124.2.178","prefixLength":0},"openflowPort":
6633},"dcNetworks":[{"prefix":"2.2.2.2","prefixLength":32}]},"qosParamet
ers":null,
"connectingTunnels":[{"@id":"Tunnel@3","tunnelID":{"class":"eu.smartenit
.sbox.db.dto.SimpleTunnelID","tunnelName":"tunnel1","tunnelNumber":1},"l
ink":{"@id":"Link@5","linkID":{"class":"eu.smartenit.sbox.db.dto.SimpleL
inkID","localLinkID":"link1","localIspName":"isp1"},"address":{"prefix":
"146.124.2.74","prefixLength":0},"physicalInterfaceName":"eth0","vlan":2
221,"inboundInterfaceCounterOID":null,"outboundInterfaceCounterOID":null
,"costFunction":{"type":"Cost-
1,"a":-
2.0,"b":1.0}]},"bgRouter":{"@id":"BGRouter@4","managementAddress":{"pref
ix":"146.124.2.74","prefixLength":0},"snmpCommunity":null,"interDomainLi
nks":null},"traversingTunnels":[{"@id":"Tunnel@4","tunnelID":{"class":"e
u.smartenit.sbox.db.dto.SimpleTunnelID","tunnelName":"tunnel1","tunnelNu
mber":1},"link":{"@id":"Link@6","linkID":{"class":"eu.smartenit.sbox.db.
dto.SimpleLinkID","localLinkID":"link1","localIspName":"isp1"},"address"
:{"prefix":null,"prefixLength":0},"physicalInterfaceName":null,"vlan":0,
"inboundInterfaceCounterOID":null,"outboundInterfaceCounterOID":null,"co
stFunction":{"type":null,"subtype":null,"inputUnit":null,"outputUnit":nu
ll,"segments":[]},"bgRouter":{"@id":"BGRouter@5","managementAddress":{"p
refix":null,"prefixLength":0},"snmpCommunity":null,"interDomainLinks":nu
ll},"traversingTunnels":null},"sourceEndAddress":{"prefix":"2.2.2.2","pr
efixLength":0},"destinationEndAddress":{"prefix":"1.1.1.1","prefixLength
":0},"physicalLocalInterfaceName":"eth0","inboundInterfaceCounterOID":nu
ll,"outboundInterfaceCounterOID":null}]},"sourceEndAddress":{"prefix":"2
.2.2.2","prefixLength":0},"destinationEndAddress":{"prefix":"1.1.1.1","p
refixLength":0},"physicalLocalInterfaceName":"eth0","inboundInterfaceCou
nterOID":null,"outboundInterfaceCounterOID":null},{"@id":"Tunnel@5","tun
nelID":{"class":"eu.smartenit.sbox.db.dto.SimpleTunnelID","tunnelName":"
tunnel2","tunnelNumber":2},"link":{"@id":"Link@7","linkID":{"class":"eu.
smartenit.sbox.db.dto.SimpleLinkID","localLinkID":"link2","localIspName"
:"isp1"},"address":{"prefix":"146.124.2.74","prefixLength":0},"physicalI
nterfaceName":"lo","vlan":21,"inboundInterfaceCounterOID":null,"outbound
InterfaceCounterOID":null,"costFunction":{"type":"Cost-
1,"a":-
2.0,"b":1.0}]},"bgRouter":{"@id":"BGRouter@6","managementAddress":{"pref
ix":"146.124.2.74","prefixLength":0},"snmpCommunity":null,"interDomainLi
nks":null},"traversingTunnels":[{"@id":"Tunnel@6","tunnelID":{"class":"e
u.smartenit.sbox.db.dto.SimpleTunnelID","tunnelName":"tunnel2","tunnelNu
mber":2},"link":{"@id":"Link@8","linkID":{"class":"eu.smartenit.sbox.db.
dto.SimpleLinkID","localLinkID":"link2","localIspName":"isp1"},"address"
:{"prefix":null,"prefixLength":0},"physicalInterfaceName":null,"vlan":0,
"inboundInterfaceCounterOID":null,"outboundInterfaceCounterOID":null,"co
stFunction":{"type":null,"subtype":null,"inputUnit":null,"outputUnit":nu


ll,"segments":[]},"bgRouter":{"@id":"BGRouter@7","managementAddress":{"p
refix":null,"prefixLength":0},"snmpCommunity":null,"interDomainLinks":nu
ll},"traversingTunnels":null},"sourceEndAddress":{"prefix":"2.2.2.2","pr
efixLength":0},"destinationEndAddress":{"prefix":"1.1.1.1","prefixLength
":0},"physicalLocalInterfaceName":"lo","inboundInterfaceCounterOID":null
,"outboundInterfaceCounterOID":null}]},"sourceEndAddress":{"prefix":"2.2
.2.2","prefixLength":0},"destinationEndAddress":{"prefix":"1.1.1.1","pre
fixLength":0},"physicalLocalInterfaceName":"lo","inboundInterfaceCounter
OID":null,"outboundInterfaceCounterOID":null}]}]}

Deliverable D3.2 Technologies, Implementation Framework, and Initial Prototype (Initial Version)

Transféré par

Informations du document

Copyright

Formats disponibles

Partager ce document

Partager ou intégrer le document

Options de partage

Avez-vous trouvé ce document utile ?

Ce contenu est-il inapproprié ?

Droits d'auteur :

Formats disponibles

Deliverable D3.2 Technologies, Implementation Framework, and Initial Prototype (Initial Version)

Transféré par

Droits d'auteur :

Formats disponibles

Seventh Framework STREP No. 317846 D3.

2 Technologies, Implementation Framework,

Vous aimerez peut-être aussi