What is Framework?

A framework is considered to be a combination of set protocols, rules, standards

and guidelines that can be incorporated or followed as a whole so as to leverag
e the benefits of the scaffolding provided by the Framework.
Let us consider a real life scenario.
We very often use lifts or elevators. There are a few guidelines those are menti
oned within the elevator to be followed and taken care off so as to leverage the
maximum benefit and prolonged service from the system.
Thus, the users might have noticed the following guidelines:
Keep a check on the maximum capacity of the elevator and do not get onto an elev
ator if the maximum capacity has reached.
Press the alarm button in case of any emergency or trouble.
Allow the passenger to get off the elevator if any before entering the elevator
and stand clear off the doors.
In case of fire in the building or if there is any haphazard situation, avoid th
e use of elevator.
Do not play or jump inside the elevator.
Do not smoke inside the elevator.
Call for the help/assistance if door doesn t open or if the elevator doesn t work at
all. Do not try to open the doors forcefully.
There can be many more rules or sets of guidelines. Thus, these guidelines if fo
llowed, makes the system more beneficial, accessible, scalable and less troubled
for the users.
Now, as we are talking about Test Automation Frameworks , let us move our focus tow
ards them.
Test Automation Framework
A Test Automation Framework is scaffolding that is laid to provide an execution en
vironment for the automation test scripts. The framework provides the user with
various benefits that helps them to develop, execute and report the automation t
est scripts efficiently. It is more like a system that has created specifically
to automate our tests.
In a very simple language, we can say that a framework is a constructive blend o
f various guidelines, coding standards, concepts, processes, practices, project
hierarchies, modularity, reporting mechanism, test data injections etc. to pilla
r automation testing. Thus, user can follow these guidelines while automating ap
plication to take advantages of various productive results.
The advantages can be in different forms like ease of scripting, scalability, mo
dularity, understandability, process definition, re-usability, cost, maintenance
etc. Thus, to be able to grab these benefits, developers are advised to use one
or more of the Test Automation Framework.
Moreover, the need of a single and standard Test Automation Framework arises whe
n you have a bunch of developers working on the different modules of the same ap
plication and when we want to avoid situations where each of the developer imple
ments his/her approach towards automation.
Note: Take a note that a testing framework is always application independent tha
t is it can be used with any application irrespective of the complications (like
Technology stack, architecture etc.) of application under test. The framework s
hould be scalable and maintainable.
Advantage of Test Automation framework
Reusability of code
Maximum coverage
Recovery scenario
Low cost maintenance
Minimal manual intervention
Easy Reporting
Types of Test Automation Framework
Now that we have a basic idea of what is an Automation Framework, in this sectio
n we would harbinger you with the various types of Test Automation Frameworks th
ose are available in the market place. We would also try shed lights over their
pros and cons and usability recommendations.
There is a divergent range of Automation Frameworks available now days. These fr
ameworks may differ from each other based on their support to different key fact
ors to do automation like reusability, ease of maintenance etc.
Let us discuss the few most popularly used Test Automation Frameworks:
Module Based Testing Framework
Library Architecture Testing Framework
Data Driven Testing Framework
Keyword Driven Testing Framework
Hybrid Testing Framework
Behavior Driven Development Framework
(click on image to view enlarged)
Test Automation Frameworks 1
Let us discuss each of them in detail.
But before that I would also like to mention that despite having these framework
, user is always leveraged to build and design his own framework which is best s
uitable to his/her project needs.
#1) Module Based Testing Framework
Module based Testing Framework is based on one of the popularly known OOPs conce
pt Abstraction. The framework divides the entire Application Under Test into numbe
r of logical and isolated modules. For each module, we create a separate and ind
ependent test script. Thus, when these test scripts taken together builds a larg
er test script representing more than one modules.
These modules are separated by an abstraction layer in such a way that the chang
es made in the sections of the application doesn t yields affects on this module.
Test Automation Frameworks 2
Test Automation Frameworks 3
Pros:
The framework introduces high level of modularization which leads to easier and
cost efficient maintenance.
The framework is pretty much scalable
If the changes are implemented in one part of the application, only the test scr
ipt representing that part of the application needs to be fixed leaving all the
other parts untouched.
Cons:
While implementing test scripts for each module separately, we embed the test da
ta (Data with which we are supposed to perform testing) into the test scripts. T
hus, whenever we are supposed to test with a different set of test data, it requ
ires the manipulations to be made in the test scripts.
#2) Library Architecture Testing Framework
The Library Architecture Testing Framework is fundamentally and foundationally b
uilt on Module Based Testing Framework with some additional advantages. Instead
of dividing the application under test into test scripts, we segregate the appli
cation into functions or rather common functions can be used by the other parts
of the application as well. Thus we create a common library constituting of comm
on functions for the application under test. Therefore, these libraries can be c
alled within the test scripts whenever required.
The basic fundamental behind the framework is to determine the common steps and
group them into functions under a library and call those functions in the test s
cripts whenever required.
Example: The login steps can be combined into a function and kept into a library
. Thus all the test scripts those require to login the application can call that
function instead of writing the code all over again.
Test Automation Frameworks 4
Pros:
Like Module Based Framework, this framework also introduces high level of modula
rization which leads to easier and cost efficient maintenance and scalability to
o.
As we create common functions that can be efficiently used by the various test s
cripts across the Framework. Thus, the framework introduces a great degree of re
-usability.
Cons:
Like Module Based Framework, the test data is lodged into the test scripts, thus
any change in the test data would require changes in the test script as well.
With the introduction of libraries, the framework becomes a little complicated.
#3) Data Driven Testing Framework
While automating or testing any application, at times it may be required to test
the same functionality multiple times with the different set of input data. Thu
s, in such cases, we can t let the test data embedded in the test script. Hence it
is advised to retain test data into some external data base outside the test sc
ripts.
Data Driven Testing Framework helps the user segregate the test script logic and
the test data from each other. It lets the user store the test data into an ext
ernal database. The external databases can be property files, xml files, excel f
iles, text files, CSV files, ODBC repositories etc. The data is conventionally s
tored in Key-Value pairs. Thus, the key can be used to access and populate the dat
a within the test scripts.
Note: The test data stored in an external file can belong to the matrix of expec
ted value as well as matrix of input values.
------------
Test Automation Frameworks 5
Example:
Let us understand the above mechanism with the help of an example.
Let us consider the Gmail Login Functionality.
Step 1: First and the foremost step are to create an external file that stores t
he test data (Input data and Expected Data). Let us consider an excel sheet for
instance.
Test Automation Frameworks 6
Step 2: The next step is to populate the test data into Automation test Script.
For this purpose several API s can be used to read the test data.
1
public void readTD(String TestData, String testcase) throws Exception {
2
TestData=readConfigData(configFileName,"TestData",driver);
3
testcase=readConfigData(configFileName,"testcase",driver);
4
FileInputStream td_filepath = new FileInputStrea
m(TestData);
5
Workbook td_work =Workbook.getWorkbook(td_filepa
th);
6
Sheet td_sheet = td_work.getSheet(0);
7
if(counter==0)
8
{
9
for (int i = 1,j = 1; i <= td_sheet.getRows()-1; i++){
10
if(td_sheet.getCell(0,i).getContents().equalsIgn
oreCase(testcase)){
11
startrow = i;
12
arrayList.add(td_sheet.getCell(j,i).getConten
ts());
13
testdata_value.add(td_sheet.getCell(j+1,i).ge
tContents());}}
14
for (int j = 0, k = startrow +1; k <= td_sheet.getRows()-1; k++)
{
15
if (td_sheet.getCell(j,k).getContents()==""){
16
arrayList.add(td_sheet.getCell(j
+1,k).getContents());
17
testdata_value.add(td_sheet.getC
ell(j+2,k).getContents());}}
18
}
19
counter++;
20
}
The above method helps to read the test data and the below test step helps the u
ser to type in the test data on the GUI.
element.sendKeys(obj_value.get(obj_index));
Pros:
The most important feature of this framework is that it considerably reduces the
total number of scripts required to cover all the possible combinations of test
scenarios. Thus lesser amount of code is required to test a complete set of sce
narios.
Any change in the test data matrix would not hamper the test script code.
Increases flexibility and maintainability
A single test scenario can be executed altering the test data values.
Cons:
The process is complex and requires an extra effort to come up with the test dat
a sources and reading mechanisms.
Requires proficiency in a programming language that is being used to develop tes
t scripts.
#4) Keyword Driven Testing Framework
The Keyword driven testing framework is an extension to Data driven Testing Fram
ework in a sense that it not only segregates the test data from the scripts, it
also keeps the certain set of code belonging to the test script into an external
data file.
These set of code are known as Keywords and hence the framework is so named. Key
words are self-guiding as to what actions needs to be performed on the applicat
ion.
The keywords and the test data are stored in a tabular like structure and thus i
t is also popularly regarded as Table driven Framework. Take a notice that keywo
rds and test data are entities independent of the automation tool being used.
Test Automation Frameworks 7
Example Test case of Keyword Driven Test Framework
Test Automation Frameworks 8
In the above example keywords like login, clickLink and verifyLink are defined w
ithin the code.
Depending upon the nature of application keywords can be derived. And all the ke
ywords can be reused multiple times in a single test case. Locator column contai
ns the locator value that is used to identify the web elements on the screen or
the test data that needs to be supplied.
All the required keywords are designed and placed in base code of the framework.
Pros:
In addition to advantages provided by Data Driven testing, Keyword driven framew
ork doesn t require the user to possess scripting knowledge unlike Data Driven Tes
ting.
A single keyword can be used across multiple test scripts.
Cons:
The user should be well versed with the Keyword creation mechanism to be able to
efficiently leverage the benefits provided by the framework.
The framework becomes complicated gradually as it grows and a number of new keyw
ords are introduced.
#5) Hybrid Testing Framework
As the name suggests, the Hybrid Testing Framework is a combination of more than
one above mentioned frameworks. The best thing about such a setup is that it le
verages the benefits of all kinds of associated frameworks.
Test Automation Frameworks 9
Example of Hybrid Framework
Test sheet would contain both the keywords and the Data.
Test Automation Frameworks 10
In the above example, keyword column contains all the required keywords used in
the particular test case and data column drives all the data required in the tes
t scenario. If any step does not need any input then it can be left empty.
#6) Behavior Driven Development Framework
Behavior Driven Development framework allows automation of functional validation
s in easily readable and understandable format to Business Analysts, Developers,
Testers, etc. Such frameworks do not necessarily require the user to be acquain
ted with programming language. There are different tools available for BDD like
cucumber, Jbehave etc. Details of BDD framework are discussed later in Cucumber
tutorial. We have also discussed details on Gherkin language to write test cases
in Cucumber.

Components of Automation Testing Framework

(click on image to view enlarged)
Test Automation Frameworks 11
Though the above pictorial representation of a framework is self-explanatory but
we would still highlight a few points.
Object Repository: Object Repository acronym as OR is constituted of the set of
locators types associated with web elements.
Test Data: The input data with which the scenario would be tested and it can be
the expected values with which the actual results would be compared.
Configuration File/Constants/ Environment Settings: The file stores the informat
ion regarding the application URL, browser specific information etc. It is gener
ally the information that remains static throughout the framework.
Generics/ Program logics/ Readers: These are the classes that store the function
s which can be commonly used across the entire framework.
Build tools and Continuous Integration: These are the tools that aids to the fra
meworks capabilities to generate test reports, email notifications and logging i
nformation.
Conclusion
The frameworks illustrated above are the most popular frameworks used by the tes
ting fraternity. There are various other frameworks also in the place. For all t
he further tutorials we would base on the Data Driven Testing Framework.
In this tutorial, we discussed about the basics of an Automation Framework. We a
lso discussed about the types of frameworks available in the market.
Next Tutorial #21: In the next tutorial, we would briefly introduce you with the
sample framework, the MS Excel which would store the test data, excel manipulat
ions etc.
Till then feel free to ask your queries about automation frameworks.

Share97Tweet1+116Pin2Share92

38 comments ?
#1 Henk Noppe
Now I know all testing frameworks at one place with comparison. good work.
#2 Ummer
Keep up the good work.
Good pros and cons explanation.
This should be aware as a automation tester.
#3 Shruti Shrivastava
@Henk @Ummer
Thank you for the appreciative words.
Keep Learning.
#4 jyoti
Superb explanation please keep posting all about automation framework
Thanks & regards
Jyoti
#5 AbuAmmar
Good Work, Keep Doing.
I ve a question, I fwe want to secure our Framework scripts from being disclosed b
y any another party, is this possible?
I m using UFT and Vbs script files.
Thanks.
Abdelghany
#6 abc
i need solution of below questions.
1>if a similar function is taken across all the modules and tested .it is called
?
a)horizontal first testing
b)vertical first testing
c)sandwitch testing
d)critical path testing
2>which of the following testing confirm syntax,convention,format,style abbrevia
tion
a)security testing
b)performance testing
c)usebility testing
d)stress testing
3>windows resizes option suit contains which button /buttons according to window
s standard ?
a)minimize button
b)maximize button
c)close button
d)all of the above
#7 asha
Thanks so much.
#8 Aravind
Very helpful for the beginners of test automation.
Is there any post regarding test automation frame work which completely deals wi
th scripting languages like python.
#9 Amit
For Example : we have 1000+ testcases to do aumoted so we have to write 1000++ t
imes drier.findelement now I want to make one general method so whenever it requ
ire to write driver.findelement we can ingnor it y writing just that general met
hod name
Can you suggest me how to do it ??
#10 csda
sads
#11 Akien MacIain
For what it covers, this is a decent summary. From my point of view, this is a v
ery narrow definition of framework. My own definition also includes testing reso
urce management, artifact management, onboarding training for new resources, lab
resource management, coding standards, crisis management, prioritization polici
es, documentation procedures, test data management, and just so much more. I thi
nk you ve inspired me to write my own article. Thanks! I will probably link back t
o you.
#12 Muthuprasath
Thank a lot :)
#13 Ramesh
Awesome explaination
#14 Bharath
Helpful !!!
#15 Ram
Simple and Elegant for a beginner with no idea on What Testing Framework is?
Good Work, keep up doing this stuff for community
#16 Yen
Thank you so much! very good work.
#17 Hemalatha
Superb, very very helpful
#18 vijaya
can you please explain the hybrid framework structure
#19 Kajal
Very Informative..Thank you so much..I was looking for this kind of elabroted in
fo from long tym.
#20 rohini
must read tutorials. thanks for sharing
#21 Gaurav Khurana
Hi thanks for sharing this, iT was really good to know about something which we
already know but dont know the definiton of it.
whenever you are giving a code please give it end to end. as you have shared cod
e for reading from configuration file, but we are not able to implement it as we
don t have any instructions,etc to user that code which is written under the topi
c data driven testing
#22 SHailesh
Thanks a lot for clear n crystal explanation
I got rejected once for not explained well
now its clear .. Thanks one again ..best luck
#23 Mahesh
what about TestNG frame work
how cal the excel file for multiple test cases in testng frame plz give the solu
tion
#24 Poonam
hi, i have write code in a franmework, where i have made configure file, read cl
ass, driver initialize class, objects class, function class and 4 testNG class.
and run these 4 class using xml file. what kind of framework it is? TDD? BDD? pl
s reply soon
#25 Sreekanth Sunnapu
Nice explanation, really useful stuff
Thanks
#26 manju
very helpful
#27 Mallikarjun
Great job buddy !
#28 kashif
I have followed most of your selenium series. You have obviously put in a lot of
effort.
The only suggestion I can offer is that the content can be further improved thro
ugh a bit of editing, such as choice of words and sentence structure.
Keep up the good work!
#29 Anubha Gupta
very useful content many thanks.
#30 yomi
insightful and helpful
#31 Jigs
Hey Shruti, very well written article. I would suggest adding TDD as well since
you have added BDD. TDD is gaining a lot of steam now.
#32 NGP
Precise and nice explanation.
Thanks.
#33 akien maciain
Hi! You and I chatted very briefly some time back. I ve written some of the stuff
I was planning to about framework design, but I ve also had kind of a breakthrough
in my understanding of framework design. I d be inclined to chat with you about i
t, if you d care to ping me at akienm@gmail.com
#34 Deep
Great Work. Easy to Understand.
Thank you.
#35 bHIkshu
good explanation
beginners easy to understand.
#36 Magz
very informative and helpful
#37 Rick Stag
I see one important item missing here and that is the applications database. You
need to be able to have a known starting point with the application database. Y
ou should always be able to backup and restore the application database before s
tarting automation test scenarios. This is a best case scenario but should be a
high priority goal.
#38 Cal
They missed one, Model Based Testing framework. But otherwise a very good list o
f items and information. Definitely recommend using this as a reference to expla
in automation and frameworks to management and other non-technical people.
Leave a Comment
Name
Mail
Website (Optional)
Submit
Useful Links!
Home
Subscribe for FREE Updates
Testing RESOURCES
QA Testing TRAINING
Write and Earn Program
FREE eBooks
ALL Tutorials
Suggest New Tutorials
Contact Us
Help & Tutorials
ISTQB Study Guide
ISTQB Premium Study Guide
Free QA Training
Free Selenium Training
Free QTP Training
Free QC Training
HP LoadRunner Tutorials
JIRA Tutorials
101+ Interview Questions
Follow Us!
follow SoftwareTestingHelp
Explore Categories
Explore Categories
Adv
Stryka
uTest
TestLodge
Radview
Get Free eBooks &
Best Articles
Your Email:
Submit Form
Latest Articles!
Kanban vs Scrum vs Agile: A Detailed Comparison to Find Differences
7 Ways to Kick Start Your Manual Testing Career
7-Step Practical Implementation of Manual Testing Before Production Release
Manual Testing Tutorials for ALL the Testers: Getting the Basics Right
Integration of Jenkins with Selenium WebDriver: Step-by-Step Tutorial
How to Run Selenium WebDriver in Different Popular Browsers
How Should Testers Handle Boredom?

Best Online Training

Software Testing Online Training Course
Software Testing freelancing work

Search
ABOUT US | ARTICLES | CONTACT US | DIRECTORY | AFFILIATES | ADVERTISE | TESTING
SERVICES
ALL ARTICLES ARE COPYRIGHTED AND CAN NOT BE REPRODUCED WITHOUT PERMISSION.
ALL RIGHTS RESERVED. DO NOT COPY 2006 - 2017 SOFTWARE TESTING HELP READ OUR COPY
RIGHT POLICY | PRIVACY POLICY | LINK TO US