Best Practices in Unit Testing Automation

Unit testing and Automation:

Once the source code has been generated it must be tested to uncover (and correct) as many errors as possible before delivery. In order to achieve this, testing must be conducted systematically & test cases must be designed using disciplined techniques so that they have a high likelihood of finding errors. Unit testing is one such testing technique which reveals errors in our application at early stage of software development process. Unit testing is a process of verification and validation of individual units of source code in isolation. The key idea is that each piece of code needs its own tests and the best person to test that code is the developer working on it. Hence, unit testing is typically performed by developers in a development cycle, rather than in the QA phase. By performing unit test, programmers can catch bugs early before they cause significant problems or delays in the later phases. However, manual process of writing test cases, running tests, monitoring the test results, and pinpointing further areas for testing consumes more time and effort. Hence automation is essential. First step towards the automation is usage of unit testing frameworks. They can be considered as intermediate between manual testing techniques and automated unit testing tools. These frame-works provide an interface to write and execute manually written tests. They define how to structure your test cases and provide the tools to run them. Though, effort and time required here is also higher, testing with frameworks ensures more coverage, reusability and repeatability of tests. Further, these tests can be easily managed and maintained throughout the product life cycle. Some of the commonly used unit testing frameworks are, JUnit -a simple unit testing framework for java standalone application Cactus -an extension of JUnit for unit testing server-side java code (Servlets, EJBs etc) CppUnit -a unit testing framework for C/C++ applications NUnit -a unit-testing framework for all .Net languages WiproUT is an automated java unit testing tool developed and owned by Tools Group, Wipro Technologies. It automatically generates and executes Junit, Cactus test cases by taking compiled source code as an input.

Strategy:
Effective testing is important, so is the strategy. By reviewing the Test Specification prior to testing, one can assess the completeness of tests and testing tasks. An effective test plan and procedure will lead to the orderly construction of the software and the discovery of errors at each stage in the construction phase. Points to be considered here are, 1. 2. 3. As a first step, applicability and scope of unit testing has to be defined The objectives of testing like test effectiveness, test coverage, the cost to find and fix defects should be clearly stated within the test plan in measurable terms. Selection of right tool is another important step at this stage. Below parameters can be considered while selecting a tool, a. Project environment and technology b. Amount of code to be written newly c. Amount of code that undergoes frequent changes d. Lifespan of product being developed e. Team size and effort estimated for the project f. Budget

4. 5.
6.

List of tools available at Tools_list can be referred for some of the commonly used unit testing tools. Regression testing definition, frequency, actions to be stated in test plan Test plan should be implemented effectively in the project with the tool selected Formal technical reviews should be conducted periodically to assess the test strategy and test cases

Unit test case design:

Test cases should be designed to ensure that individual units of program are structurally correct and function as expected. These tests should adequately exercise functionality of all internal components, program logic and loop conditions. Such test cases later become the foundation for Regression testing practice (regular, repeatable verification of the code base). Unit test design typically involves writing test harnesses, specifying input data and supplying stubs/mock objects for missing functions. Unit testing automation tools provide us unit testing capabilities in simplified manner. They come with the capability to generate test cases in a high level at the click of a button, by using code as input. To test the complex logic implemented code, tool generated tests can be customized to test all paths, conditions and logic.
Steps to design test cases in tool: 1. Define a project specific test configuration by defining,

2. 3. 4. 5. 6. 7.

-The scope of testing (what needs to be tested) -Number of assert statements per test -Timeout value for tests -Test runner (testing framework to be used for execution) etc Share the created configuration with the entire team Generate the test cases using defined configuration fie. Review the tests generated and update with application-specific input values Try to pass all feasible boundary values as test input values to uncover more errors. Consider negative input values s too Identify parameterizable fields in a application and generate parameterized tests (Data driven tests) Associate the data source with parameterized tests.

Unit test case execution and Defect closure:

Knowing the internal working & specified function that a product has been designed to perform, tests can be executed to demonstrate each function is fully operational while at the same time searching for errors in each function. Post test execution closure of defects has to be done with highest priority, The general steps in closing violations are, Isolate code along with unit test which has problem, Test isolated code to confirm it fails, Fix the bug, Run test to verify it succeeds. Tools provide an effective interface to run all tool generated/customized test cases with centralized testing, debugging and reporting environment. On executing tool generated/user defined/legacy tests it reports problems like functionality related issues, regression related issues, unexpected runtime exceptions organized by error categories and severities. Recommended steps to execute test cases and close defects using tool:
1.

Run the test cases generated/modified

2. 3. 4. 5. 6. 7.

Verify and validate the test execution results reported by tool Isolate the failures reported by tool, prioritize and fix them by editing the offending source-code. As a good practice, fix the problems related to exceptions first, then functionality problem followed by others Re-run the tests to ensure the correctness Baseline these tests for further regression testing Run the base-lined set of tests on regular basis to validate code behavior with changes Track this activity at project level through review meetings

Coverage analysis:
Code coverage analysis measures the degree to which the source code has been tested. It is the way to verify effectiveness of source code as well as test cases. In other words, the coverage of code is the best indication of how well the code has been tested. Coverage analysis results can also be used to identify unexecuted code of program (dead code). Most of the unit testing tools offer both customizable annotated representation of covered/ uncovered code and sharable coverage statistics. Coverage analyzer can be executed post test case execution to get the coverage details in which coverage at different levels including line coverage and decision coverage can be tracked. Recommended actions in case of lesser coverage: 1. Clean up the code by removing/modifying the unexecuted lines of code 2. Review tests and test input values to include missed conditions/scenarios 3. Parameterize the test input by associating with data-sources like excel, databases to pass wide range of input values 4. Use user defined objects, stubs and mock objects to handle units with domain specific objects and dependencies

Comparison of some commonly used tools:

Technol ogy Tools WiproUT JTest *Junit, *Cactus C++Test *CppUnit .Test *NUnit Project specific configuration Automatic Test Generation Test Execution Features of tools Data Coverage Sourcing Analysis Build integration Integratio n with IDE

Reports

Java

x x x

x x x

x x x x x

x x x

C/C++

.Net

* Frameworks

Conclusion:
All of us are interested in improving the quality of the applications we develop. However, formal manual testing methodologies are simply impractical to implement for 100% of code coverage. Tool-assisted process offers both an efficient and effective ways to identify and locate defects without requiring painstaking tasks. Further, the best tools alone arent enough; well-defined processes arent enough. Whats required is, A lifecycle approach to quality, Structured planning and design of the environment, Consistent use of best practices.