Software Testing

Software testing is a process used to identify the correctness, completeness, and quality of developed computer software. We do Software testing for following reasons. 1. To check the reliability of the software. 2. To be ensure that the software does not contain any bug which results in software failure 3. The software is developed according to its specification or not.

Principles of Testing:
There are seven principles of Software testing. 1. Testing shows presence of defects: Testing can show that defects are present, but cannot prove that there are no defects. Testing reduces the probability of undiscovered defects remaining in the software but, even if no defects are found, it is not a proof of correctness. 2. Exhaustive testing: Testing everything (all combinations of is impossible Inputs and preconditions) is not possible. Instead of exhaustive testing, we Use risks and priorities to focus testing efforts. 3. Early testing: Testing activities should start as early as possible in the software or system development life cycle and should be focused on defined objectives. 4. Defect clustering: A small number of modules contain most of the defects discovered during pre-release testing or show the most operational failures. 5. Pesticide paradox: If the same tests are repeated over and over again, eventually the same set of test cases will no longer find any new bugs. To overcome this, the test cases need to be regularly reviewed and revised, and new and different tests need to be written to exercise different parts of the software or system to potentially find more defects. 6. Testing is context: Testing is done differently in different dependent contexts. For example, safety-critical software is tested differently from an ecommerce site.

7. Absence-of-errors: Finding and fixing defects does not help if fallacy the system built is unusable and does not fulfill the users' needs and expectations.

SOFTWARE DEVELOPMENT LIFE CYCLE (SDLC)

The System Development Life Cycle framework provides a sequence of activities for system designers and developers to follow. It consists of a set of phases such as Requirement analysis, design, development, testing, and deployment. There are different Software Development lifecycle Models. Some of them are 1) 2) 3) 4) 5) 6) 7) Waterfall Model. Prototype Model. Incremental/Iterative Model. Spiral Model. V-Model. Agile Methodology. Fish Model.

Waterfall Model:
The waterfall Model illustrates the software development process in a linear sequential flow. In "The Waterfall" approach, the whole process of software development is divided into separate phases. The phases in Waterfall model are: Requirement Specifications , Software Design, Development, Testing & Maintenance. Any phase in this development process begins only if the previous phase is complete. So the name "Waterfall Model".

Requirement Analysis:
Involves carrying out detailed study of the customer requirements and arriving at the exact requirements of the proposed system. The phase involves freezing the requirements before the design phase begins.

System Design:
All the high-level and low level architectures are identified in this phase, this phase helps in specifying hardware and system requirements and also helps in defining overall system architecture. The system design specifications serve as input for the next phase of the model.

Development:
On receiving system design documents, the works divided in modules/units and actual coding is started. The system is first developed in small programs called units, which are integrated in the next phase.

Testing:
In this phase the code is tested to verify whether the software build according to user specifications .Any bugs or errors found in this stage are reported to development team to fix the bugs, in order to avoid any software failures.

Maintenance:
After successful testing the software is deployed to the customer and it is maintained for future development.

Advantages and Disadvantages

Advantages
1) It is very simple to implement 2) The amount of resources required to implement this model are minimal. 3) Documentation is produced at every stage of the software's development. This makes understanding the product designing procedure, simpler.

Disadvantages 1) Testing activity takes place very late in waterfall model. 2) In this model, if one phase is complete one cannot go back to previous
phase; if the design phase has gone wrong, things can get very complicated in the development phase. 3) The cost of defects is very high in this model as they are found in the end of the project life cycle.

Prototype Model:
Prototyping model was developed to counter the limitations of waterfall model. The basic idea behind prototyping model is that instead of freezing the requirements before any design or coding can begin, a blue print of the end product is built to understand the requirements, and it is given to customer, if the customer is satisfied then the project moves to next phase and follows the phases involved in waterfall model.

Advantages and Disadvantages

Advantages
1) Customer is able to see the blueprint of the final product, rather than at the end of project. 2) It reduces risk of failure, as potential risks can be identified early.

Disadvantages
1) It is a slow process. 2) Testing takes place after the development phase, so if any bugs found in the product can be difficult to fix due to time limitations. 3) Development costs are very high.

Incremental/Iterative Model
In Iterative Development Model the delivery of software is divided into increments or builds with each increment adding new functionality to the software product. Incremental development divides the system functionality into increments. In each increment, a part of functionality is delivered which involves the phases from the requirements to the deployment.

Advantages and Disadvantages

Advantages
1) Some working functionality can be developed quickly and early in the life cycle. 2) Results are obtained early and periodically and progress can be measured. 3) Risks are identified and resolved during an iteration; and each iteration is an easily managed milestone. 4) With every increment operational product is delivered.

Disadvantages
1) System design issues may arise because not all requirements are gathered. Up front for the entire life cycle. 2) Although cost of change is lesser but it is not very suitable for changing requirements. 3) More resources may be required.

Spiral Model
This model of development combines the features of the prototyping model and the waterfall model. The spiral model is intended for large, expensive, and complicated projects.

In the first quadrant of the spiral model each cycle begins with the identification of objectives for that cycle, the alternatives possible for achieving objectives and the constraints. The next step is to evaluate these different alternatives based on objectives and constraints. The evaluation in this step is based on the risk perception of the project. The next step is to develop strategies that resolve the risks. After this the software is developed keeping in mind the risks and finally next stages are planned.

Advantages and Disadvantages

Advantages
1) Changing requirements can be accommodated. 2) Customers can see the system early. 3) Risk analysis is done and measures are taken to overcome those risks.

Disadvantages
1) Very complex is process. 2) Not suitable to small projects. 3) Large number of intermediate stages requires excessive documentation.

V-Model
In V-Model testing begins as early as possible in the project life cycle. The VModel illustrates that testing activities (Verification and Validation) can be integrated into each phase of the development life cycle. Verification, part of testing is integrated in the earlier phases of the life cycle which includes reviewing end user requirements, design documents etc. Validation in v-model is to ensure that the product actually meets the users needs, and that the specifications were correct in the first place.

The V-model generally has four test levels. 1. Unit Testing 2. Integration Testing 3. System Testing 4. Acceptance Testing

Advantages and Disadvantages

Advantages
V Model is considered to be an extension of the Waterfall model so its a wellstructured method in which the different phases progress in a sequential manner. That means each phase begins only after the completion of the previous phase. The advantage is that it saves time and since the testing team is involved early on, they develop a very good understanding of the project at the very beginning.

Disadvantages
The biggest disadvantage of V-model is that its very rigid and the least flexible. If any changes happen mid way, not only the requirements documents but also the test documentation needs to be updated.

Agile Methodology
Now a days agile model is one of the most advance model used to develop large projects. Agile model is used because it has following features: 1. Customers are involved throughout the project. 2. Welcomes changes in requirements, even late in development process. 3. A working Software is constantly developed, in a couple of weeks to a couple of months. 4. Business people and the developers work together. 5. Face to face interactions with team members. 6. Progress of the project is measured with continuous deliverables of software. 7. Review meetings takes place constantly to improve product quality. 8. Agile model promotes very sustainable Development.

There are different types of methodologies used in agile model. Of those we use 1. Extreme programming (XP) and 2.Scrum methodology 1. Extreme programming(XP) : XP teams work in iterations that are one or two weeks long. XP teams allow change within their iterations as long as the team hasnt started work on a particular feature. Extreme Programming teams work in a strict priority order. Features to be developed are prioritized by the customer and the team is required to work on them in that order. XP teams prescribes engineering practices such as test driven development, focus on automated testing etc.

2. Scrum methodology: Scrum teams typically work in iterations called sprints that are from two weeks to one month long. Scrum teams do not allow changes into their sprints. Once the sprint planning meeting is completed teams are committed to develop the product. Scrum customer prioritizes the products to be developed but the team determines the sequence in which they will develop the product.

Fish Model
This model of software development resembles with Waterfall model. It contains all the sequential steps involve in waterfall model. It also involves simultaneous deployment of verification & validation processes. Verification and validation is done parallely by separate teams in each phase of the model. So there are two reports generated by the end of each phase, one for validation and one for verification. The structure of this model looks like a skeleton between two parallel lines, hence the name fish model. This process results in products of exceptional quality.

Advantages and Disadvantages

Advantages
1) Due to thorough verification and validation Fish Model yields a quality product.

Disadvantages
1) It is more time consuming. 2) It is more expensive.

TYPES OF TESTING
There are different types of testing. Some of them are:
1)

Unit Testing: Unit or component testing is testing a single module or one module at a time. Generally unit testing is done by the developer.

2)

Integration Testing: Integration testing is done by combining two or more modules and testing. In this type of testing we generally test interaction or interface between two modules. There are 4 types of integration testing: a) Top down: Testing takes place from top to bottom, following the control flow or architectural structure (e.g. starting from the GUI or main menu). Components or systems are substituted by stubs.
b) Bottom-up:

Testing takes place from the bottom of the control flow upwards. Components or systems are substituted by drivers.
c) Big Bang:

Here all component are integrated together at once, and then tested.
d) Sandwich Testing :

Sandwich Testing is an approach to combine top down testing with bottom up testing.

3)

System Testing: System testing is testing the functionality of whole system when the product is completely developed. Acceptance Testing: Acceptance testing is done to check whether the final product is meeting the Customer requirements or not.

4)

5)

Security Testing: Security testing is done to check whether the system protects the data or not. It involves Authentication, authorization and confidentiality. An example of security testing is a bank application Authentication involves a unique username and protected password for a specified it means no two users can have a same username. Confidentiality involves protecting the sensitive data of customer it can be a password a security code in case of a loss of password etc. are some examples.

6)

Compatibility Testing: Compatibility testing is done to check how an application runs or responds when it is open on different types of browsers.

7)

Load Testing: Load Testing is done to check how the application responds when it is subjected to specific load. Load testing comes under performance testing. For example checking a monster.com page when it is 10 users logged at a time and determine its performance. Checking for 100 users and determine its performance of application and so on. Stress Testing: Stress testing is done to determine the performance when it is put under a stress. It is also a type of performance testing. For example if the If an application can handle a maximum of 200 users at a time, so if the application is subjected to a stress 500 users logged in at same time, how the application performs under stress is stress testing.

8)

9)

Recovery Testing: Testing how well the system recovers from crashes, hardware failures, and other problems is called recovery testing.

10) Volume

Testing: Volume testing is testing a software application with a certain amount of data. 11) Regression Testing: Regression Testing is to check whether that a change, such as fixing a bug, did not introduce new faults or any affects on any other functionality in the product.

12) Re-Testing:

Re-testing or confirmation testing is to test whether the bug is fixed or not.

13) End

to End testing: End to end testing is Test the application from one end to another end at a time is called End to End testing. Generally, it is conducted before closing the testing. This test is conducted to validate the flow of data between modules. And know the behavior of the application. This is also part of the system testing. Testing: Database is to check whether the system is fetching the exact values from the database or not. Testing: Sanity testing is done to check one or very few functionalities of a system. It is a subset of acceptance testing. This type of testing is very narrow.

14) Database

15) Sanity

16) Smoke

Testing: Smoke testing is done to check the critical functionalities of the system. Smoke testing is scripted either using a written set of tests or an automated test. It is a subset of a regression testing. The objective of smoke is to determine the stability of system in order to perform more rigorous testing.

17) Static

Testing: Static Testing is done without executing the program. This type of testing does verification process. Static testing is about prevention of defects. Static testing gives assessment of code and documentation. Cost of finding defects and fixing is less. More reviews comments are highly recommended for good quality. Testing: Dynamic testing is done by executing the software code. Dynamic testing does validation process. Dynamic testing is about finding and fixing the defects. Dynamic testing involves test cases for execution. More defects are highly recommended for good quality. Testing: Ad hoc testing is an informal testing type which can aim to break the system. This testing is usually an unplanned activity. It does not follow any test design techniques to create test cases. Main aim of this testing is to find defects by random checking. The advantage of ad hoc testing is one can find critical defects easily. The main disadvantage is one cannot reproduce the defects since no documentation is involved.

18) Dynamic

19) Ad hoc

20) Exploratory

Testing: Exploratory testing is all about discovery, investigation and learning. Test cases are not created in advance but testers check system on the fly. They may note down ideas about what to test before test execution. Is not random testing but it is ad hoc testing with purpose of find bugs.

Major testing types

White Box Testing
White Box Testing (also known as Clear Box Testing, Glass Box Testing, Structural Testing) is a software testing method in which the internal structure/design/implementation of the item being tested is known to the tester. The tester chooses inputs to exercise paths through the code and determines the appropriate outputs. Programming knowledge is essential for tester to do white box testing. BLACK BOX TESTING AND TECHNIQUES: Black box testing treats the system as a black-box, so it doesnt explicitly use Knowledge of the internal structure or code. Or in other words the Test engineer need not know the internal working of the application. Main focus in black box testing is on functionality of the system as a whole. Each testing method has its own advantages and disadvantages. There are some bugs that cannot be found using only black box or only white box. Majority of the applications are tested by black box testing method. We need to cover majority of test cases so that most of the bugs will get discovered by black box testing. Black box testing occurs throughout the software development and Testing life cycle i.e, in Unit, Integration, System, Acceptance and regression testing stages. Advantages of Black Box Testing - Tester can be non-technical. - Used to verify contradictions in actual system and the specifications.

- Test cases can be designed as soon as the functional specifications are complete. Disadvantages of Black Box Testing - The test inputs needs to be from large sample space. - It is difficult to identify all possible inputs in limited testing time. So writing test cases is slow and difficult. - Chances of having unidentified paths during this testing. METHODS OF BLACK BOX TESTING: 1. Boundary Value Analysis: Many systems have tendency to fail on boundary. So testing boundary values of application is important. Boundary Value Analysis (BVA) is a Functional Testing technique where the extreme boundary values are chosen. Boundary values include maximum, minimum, just inside/outside boundaries, typical values, and error values. Limitations of Boundary Value Analysis Boundary value testing is efficient only for variables of fixed values i.e, boundary.

2. Equivalence Class Partitioning: Equivalence partitioning is a black box testing method that divides the input domain of a program into classes of data from which test cases can be derived. 3. Error guessing: This is a Test design technique where the experience of a tester is used to find the components of software where defects might be present.

It is mostly done by experienced testers who can use their past experience to find defects in software. Error guessing has no rules for testing, it only uses the testers previous skills. In error guessing testers can think of situations where software will fail. For example: 1. Division by zero 2. Pressing submit button on form without filling any entries. 3. Entering wrong data in the fields and checking software behaviour.

4. State Transition Diagram: State transition diagram is use to determine the behaviour of the system. We can also use state transition diagram to determine invalid system transitions. In this diagram all the states that occur are listed on the left side and all the event that cause transitions are placed on top of each state.

For example If we start at s_init state and to reach the final state s_final. System need to pass through each and every state that is in between s_init and s_final with the subsequent transitions occurring at each state. If any negative event occurs at state S3, it is represented with a transition T_3_1 and user is taken back to state S1. In this way we can study the behaviou of the system.

BUG LIFE CYCLE:

Bug can be defined as the abnormal behavior of the software. No software exists without a bug. The elimination of bugs from the software depends upon the efficiency of testing done on the software.

The different states of a bug can be summarized as follows:

1.New 2. Open 3. Assign 4. Test 5. Verified 6. Deferred 7. Reopened 8. Duplicate 9. Rejected 10. Closed

Description of Various Stages: 1. New: When the bug is posted for the first time, its state will be NEW. This means that the bug is not yet approved.

2. Open: After a tester has posted a bug, the lead of the tester approves that the bug is genuine and he changes the state as OPEN. 3. Assign: Once the lead changes the state as OPEN, he assigns the bug to corresponding developer or developer team. The state of the bug now is changed to ASSIGN. 4. Test: Once the developer fixes the bug, he has to assign the bug to the testing team for next round of testing. Before he releases the software with bug fixed, he changes the state of bug to TEST. It specifies that the bug has been fixed and is released to testing team.
5.

Deferred: The bug, changed to deferred state means the bug is expected to be fixed in next releases. The reasons for changing the bug to this state have many factors. Some of them are priority of the bug may be low, lack of time for the release or the bug may not have major effect on the software. 6. Rejected: If the developer feels that the bug is not genuine, he rejects the bug. Then the state of the bug is changed to REJECTED. 7. Duplicate: If the bug is repeated twice or the two bugs mention the same concept of the bug, then one bug status is changed to DUPLICATE. 8. Verified: Once the bug is fixed and the status is changed to TEST, the tester tests the bug. If the bug is not present in the software, he approves that the bug is fixed and changes the status to VERIFIED.

9. Reopened: If the bug still exists even after the bug is fixed by the developer, the tester changes the status to REOPENED. The bug traverses the life cycle once again. 10. Closed: Once the bug is fixed, it is tested by the tester. If the tester feels that the bug no longer exists in the software, he changes the status of the bug to CLOSED. This state means that the bug is fixed, tested and approved.

SOFTWARE TESTING LIFE CYCLE:

Requirement Analysis: During this phase, test team studies the requirements from a testing point of view to identify the testable requirements. The QA team may interact with various stakeholders (Client, Business Analyst, Technical Leads, System Architects etc) to understand the requirements in detail. Requirements could be either Functional (defining what the software

must do) or Non Functional (defining system performance /security availability). Test Planning: This phase is also called Test Strategy phase. Typically, in this stage, a Senior QA manager will determine effort and cost estimates for the project and would prepare and finalize the Test Plan. Test Case Development: This phase involves creation, verification and rework of test cases & test scripts. Test data , is identified/created and is reviewed and then reworked as well. Test Environment Setup: Test environment decides the software and hardware conditions under which a work product is tested. Test environment set-up is one of the critical aspects of testing process and can be done in parallel with Test Case Development Stage. Test team may not be involved in this activity if the customer/development team provides the test environment in which case the test team is required to do a readiness check (smoke testing) of the given environment. Test Execution: During this phase test team will carry out the testing based on the test plans and the test cases prepared. Bugs will be reported back to the development team for correction and retesting will be performed. Test Cycle Closure: Testing team will meet, discuss and analyze testing artifacts to identify strategies that have to be implemented in future, taking lessons from the current test cycle. The idea is to remove the process bottlenecks for future test cycles and share best practices for any similar projects in future.

TEST SCENARIO AND TEST CASE: Test scenario: What to test comes under test scenario. A Scenario is any functionality that can be tested. An item or a feature or a functionality to be tested in the application under test is called test scenario. Test case: How to test comes under test case. A test case has components that describes an input, action or event and an expected response, to determine if a feature of an application is working correctly.