A Cluster-based Architecture for Distributed Automated Testing

Mr Dassen SATHAN Lecturer, University of Mauritius Mauritius B.Eng(UoM),MSc(UK)

Overview
       

Introduction Related works Proposed framework Components of proposed framework Flowchart of overall operation Pseudocode Results Conclusions

FIT Conference 2011- Islamabad

Introduction


What is Software Testing?

The process of operating software under controlled conditions with the intent of identifying bugs and getting them fixed. Levels of testing : Unit , Module, Sub-system, System

What is Automated software Testing?

Use of software tools to generate test cases, execute tests and generate report.

Why Automated software testing?

40% to 60% of entire software development time spent on testing. Increased functionality & complexity.

FIT Conference 2011- Islamabad

Distributed Automated Testing



Distributed Automated Testing involves distributing

automated tests across several machines, executing the tests in parallel and aggregating the results

FIT Conference 2011- Islamabad

Why Distributed Testing



Less time and money: Automated tests can be executed over and over again. More accuracy: Automated tests carry out the same steps accurately every time. Increased test coverage: The depth and scope of test coverage is increased. High Capability- Concurrency testing, stress test, Performance testing.

FIT Conference 2011- Islamabad

Agent Based Distributed Automated Testing



AGDAT is composed of the following agents:

Test master Agent: Deploy and schedule test tasks. Test results integration Agent: Collect and integrate the test log Test execution Agent: Execute test scripts and send results.

6 FIT Conference 2011- Islamabad

Multi Agent Framework



Multi Agent System is based on distributing a task among several agents.

The distributor agent gets as input the program to be tested, specification about the number of testing agents and the permitted time to complete the testing processes. Based on these specifications, the distributor selects the testing agents and assigns the tasks to them. Each testing agent can manage and control itself on a local dimension and interact directly with its clones to exchange, provide and receive services, data and knowledge.

The cloning agents execute the test cases and return it to the testing agents.
7

FIT Conference 2011- Islamabad

Multi agent Framework

inputs

Registration, prediction

Clone identification Clone Registration Test case execution Individual Test Report

Distributor agent

Test case generator

testing agent

Clone Specification Clone Specification

clone agents

FIT Conference 2011- Islamabad

Distributed Automated Regression Testing

 

Each application has a suite of distributed tests. Each test undergoes pre processing, execution and post processing phases to compute the results

9 FIT Conference 2011- Islamabad

Evaluation
AGDAT Architecture Framework Platform Expansion Test Language Ease of use Agent Hybrid Win, Mac Yes Java No MAS Agent Hybrid Linux, Win, Linux Yes Java Yes DART ClientServer Hybrid Linux Yes C No

FIT Conference 2011- Islamabad

10

Cluster-Based framework
A computer cluster is a group of linked computers, working together closely thus in many respects forming a single computer. Test/Results DB

Store Test/Results

Retrieve Test/Results Allocate Tests Failure Report

Create tests

View Results

Test Results GUI Base Node Cluster Node

FIT Conference 2011- Islamabad

11

Architecture of proposed framework

Base Node Test Creation Test Distribution Node Manager Cluster Head Test Distribution Node Manager Test Execution Test Execution Result Reporting Status Checking Cluster Nodes

Result Reporting

FIT Conference 2011- Islamabad

12

Base Node/Cluster Head



Test creation: The test creation component provides a graphical user interface (GUI) through which the user creates the test tasks. Test distribution: The test distribution component allocates test tasks among nodes across the network. Node Manager: The node manager constantly monitors the network, and when it finds that a node has failed or no longer responds, it inform the supervisor node about the failure. Result reporting: When the test tasks have been executed at the cluster nodes, the results are sent back to the cluster heads which forward it to the base node ;where the result reporting component puts in a suitable format to be presented to the user.

FIT Conference 2011- Islamabad

13

Cluster Nodes


Status checking The status checking component runs on the cluster nodes to register a particular node to the server, by checking the nodes status (ready or inuse) and resources availability (internet connectivity) to execute tests. Test execution The test execution component executes the tests at cluster nodes. It applies the test data in the test cases to the application under test (website), gets the response and sends it to the server.

FIT Conference 2011- Islamabad

14

Overall operation

FIT Conference 2011- Islamabad

15

Hierarchical Cluster Formation



Hierarchy is formed by merging available nodes s1Sn to a group based on a cost function. The cost function find the set {si, sj} which is the cheapest to merge. Pseudocode: Compute the proximity matrix containing the distance between each pair of nodes. Treat each as a cluster. Find the most similar pair of clusters using the proximity matrix. Merge these two clusters into one cluster. Update the proximity matrix to reflect this merge operation If all patterns in one cluster, top, else repeat step 2.
16

 1.

2.

3.

FIT Conference 2011- Islamabad

Test distribution
for (int i = 0; i < listofavailableclients.count(), i++) { Create a new Listoftest Create a database connection and get list of test next to execute GetNodeStatus if node = ready { Form clusters Get a row from the test cases Create a thread for executing test cases Execute test cases } Assumption : Test cases are stored in excel file and have more or less the same number of test scenarios.
17

FIT Conference 2011- Islamabad

Results


With a cluster-based framework, test execution is faster

25

20 Test execution time in mins

15

10

Client-server(3 clients) Clusterbased(7clients)

0 5 10 20 30 40 50 60 70

No. of test cases

FIT Conference 2011- Islamabad

18

Conclusion


Cluster based Framework handle effectively test case management, test distribution , test execution and result reporting. Fault tolerance for the time being is only a message sent to Base node about a node failure and the user has to re-allocate the test. Test case allocation is done manually since the excel files are created separately. An improvement would be to split test cases based on the performance of the nodes.

FIT Conference 2011- Islamabad

19

References


[1]LKDT: A Keyword - Driven Based Distributed Test Framework, Jie Hui, Lan Yuqing, Luo Pei, Guo Shuhang, Gao Jing, 2008 International Conference on Computer Science and Software Engineering. [2]DiPerf: An automated DIstributed PERformance testing Framework, Ioan Raicu, Catalin Dumitrescu, Matei Ripeanu, Ian Foster. [3]DART: Distributed Automated Regression Testing for Large-Scale Network Applications, Brent N. Chun, OPODIS 2004, Springer-Verlag. [4]Evolution of Agent Oriented Distributed Model for Software Testing, A Layered Approach, Dhavachelvan Ponnurangam , Uma Anbarasan, 2004. [5]Agent-based Distributed Automated Testing Executing Framework,Gao Jing, Lan Yuqing,IEEE 2009.

FIT Conference 2011- Islamabad

20

THANK YOU QUESTIONS & SUGGESTIONS

FIT Conference 2011- Islamabad

21