Vous êtes sur la page 1sur 50

1. What is bidirectional traceability?

Bidirectional traceability needs to be implemented both forward and backward (i.e., from
requirements to end products and from end product back to requirements).
When the requirements are managed well, traceability can be established from the source
requirement to its lower level requirements and from the lower level requirements back to
their source. Such bidirectional traceability helps determine that all source requirements have
been completely addressed and that all lower level requirements can be traced to a valid
source.
2. What is stub? Explain in testing point of view?
Stub is a dummy program or component, the code is not ready for testing, it's used for
testing...that means, in a project if there are 4 modules and last is remaining and there is no
time then we will use dummy program to complete that fourth module and we will run whole 4
modules also. The dummy program is also known as stub.
3. For Web Applications what type of tests are you going to do?
Web-based applications present new challenges, these challenges include:
- Short release cycles;
- Constantly Changing Technology;
- Possible huge number of users during initial website launch;
- Inability to control the user's running environment;
- 24-hour availability of the web site.

The quality of a website must be evident from the Onset. Any difficulty whether in response
time, accuracy of information, or ease of use-will compel the user to click to a competitor's
site. Such problems translate into lost of users, lost sales, and poor company image.

To overcome these types of problems, use the following techniques:


1. Functionality Testing
Functionality testing involves making Sure the features that most affect user interactions work
properly. These include:
· forms
· searches
· pop-up windows
· shopping carts
· online payments
2. Usability Testing
Many users have low tolerance for anything that is difficult to use or that does not work. A
user's first impression of the site is important, and many websites have become cluttered with
an increasing number of features. For general-use websites frustrated users can easily click
over a competitor's site.

Usability testing involves following main steps


· identify the website's purpose;
· identify the indented users ;
· define tests and conduct the usability testing
· analyze the acquired information

3. Navigation Testing
Good Navigation is an essential part of a website, especially those that are complex and
provide a lot of information. Assessing navigation is a major part of usability Testing.

4. Forms Testing
Websites that use forms need tests to ensure that each field works properly and that the forms
posts all data as intended by the designer.

5. Page Content Testing


Each web page must be tested for correct content from the user perspective for correct
content from the user perspective. These tests fall into two categories: ensuring that each
component functions correctly and ensuring that the content of each is correct.

6. Configuration and Compatibility testing


A key challenge for web applications is ensuring that the user sees a web page as the designer
intended. The user can select different browser software and browser options, use different
network software and on-line service, and run other concurrent applications. We execute the
application under every browser/platform combination to ensure the web sites work properly
under various environments.

7. Reliability and Availability Testing


A key requirement o a website is that it Be available whenever the user requests it, after 24-
hours a day, every day. The number of users accessing web site simultaneously may also affect
the site's availability.

8. Performance Testing
Performance Testing, which evaluates System performance under normal and heavy usage, is
crucial to success of any web application. A system that takes for long to respond may frustrate
the user who can then quickly move to a competitor's site. Given enough time, every page
request will eventually be delivered. Performance testing seeks to ensure that the website
server responds to browser requests within defined parameters.

9. Load Testing
The purpose of Load testing is to model real world experiences, typically by generating many
simultaneous users accessing the website. We use automation tools to increases the ability to
conduct a valid load test, because it emulates thousand of users by sending simultaneous
requests to the application or the server.

10. Stress Testing


Stress Testing consists of subjecting the system to varying and maximum loads to evaluate the
resulting performance. We use automated test tools to simulate loads on website and execute
the tests continuously for several hours or days.

11. Security Testing


Security is a primary concern when communicating and conducting business- especially
sensitive and business- critical transactions - over the internet. The user wants assurance that
personal and financial information is secure. Finding the vulnerabilities in an application that
would grant an unauthorized user access to the system is important.
4. Define Brain Stromming and Cause Effect Graphing?
BS:
A learning technique involving open group discussion intended to expand the range of available
ideas
OR
A meeting to generate creative ideas. At PEPSI Advertising, daily, weekly and bi-monthly
brainstorming sessions are held by various work groups within the firm. Our monthly I-Power
brainstorming meeting is attended by the entire agency staff.
OR
Brainstorming is a highly structured process to help generate ideas. It is based on the principle
that you cannot generate and evaluate ideas at the same time. To use brainstorming, you must
first gain agreement from the group to try brainstorming for a fixed interval (eg six minutes).

CEG:
A testing technique that aids in selecting, in a systematic way, a high-yield set of test cases
that logically relates causes to effects to produce test cases. It has a beneficial side effect in
pointing out incompleteness and ambiguities in specifications.
5. What is the maximum length of the test case we can write?
We can't say exactly test case length, it depending on functionality.
6. Password is having 6 digit alphanumeric then what are the possible input conditions?
Including special characters also Possible input conditions are:
1) Input password as = 6abcde (ie number first)
2) Input password as = abcde8 (ie character first)
3) Input password as = 123456 (all numbers)
4) Input password as = abcdef (all characters)
5) Input password less than 6 digit
6) Input password greater than 6 digits
7) Input password as special characters
8) Input password in CAPITAL ie uppercase
9) Input password including space
10) (SPACE) followed by alphabets /numerical /alphanumerical/
7. What is internationalization Testing?
Software Internationalization is process of developing software products independent from
cultural norms, language or other specific attributes of a market
8. If I give some thousand tests to execute in 2 days what do you do?
If possible, we will automate or else, execute only the test cases which are mandatory.
9. What does black-box testing mean at the unit, integration, and system levels?
Tests for each software requirement using
Equivalence Class Partitioning, Boundary Value Testing, and more
Test cases for system software requirements using the Trace Matrix, Cross-functional Testing,
Decision Tables, and more
Test cases for system integration for configurations, manual operations, etc.
10. What is agile testing?
Agile testing is used whenever customer requirements are changing dynamically

If we have no SRS, BRS but we have test cases does you execute the test cases blindly or do you
follow any other process.

Test case would have detail steps of what the application is supposed to do.
1) Functionality of application.
2) In addition you can refer to Backend, is mean look into the Database. To gain more
knowledge of the application.
11. What is Bug life cycle?
New: when tester reports a defect
Open: when developer accepts that it is a bug or if the developer rejects the defect, then the
status is turned into "Rejected"
Fixed: when developer make changes to the code to rectify the bug...
Closed/Reopen: when tester tests it again. If the expected result shown up, it is turned into
"Closed" and if the problem persists again, it's "Reopen".
12. What is deferred status in defect life cycle?
Deferred status means the developer accepted the bus, but it is scheduled to rectify in the
next build.
13. Smoke test? Do you use any automation tool for smoke testing?
Testing the application whether it's performing its basic functionality properly or not, so that
the test team can go ahead with the application. Definitely can use.
14. Verification and validation?
Verification is static. No code is executed. Say, analysis of requirements etc.
Validation is dynamic. Code is executed with scenarios present in test cases.
15. When a bug is found, what is the first action?
Report it in bug tracking tool.
16. What is test plan and explain its contents?
Test plan is a document which contains the scope for testing the application and what to be
tested, when to be tested and who to test.
17. Advantages of automation over manual testing?
Time saving, resource and money
18. What is mean by release notes?
It's a document released along with the product which explains about the product. It also
contains about the bugs that are in deferred status.
19. What is Testing environment in your company, means how testing process start?
Testing process is going as follows:
Quality assurance unit
Quality assurance manager
Test lead
Test engineer
20. Give an example of high priority and low severity, low priority and high severity?
Severity level:
The degree of impact the issue or problem has on the project. Severity 1 usually means the
highest level requiring immediate attention. Severity 5 usually represents a documentation
defect of minimal impact.
Severity is levels:

• Critical: the software will not run

• High: unexpected fatal errors (includes crashes and data corruption)


• Medium: a feature is malfunctioning

• Low: a cosmetic issue

Severity levels

• Bug causes system crash or data loss.

• Bug causes major functionality or other severe problems; product crashes in


obscure cases.

• Bug causes minor functionality problems, may affect "fit anf finish".

• Bug contains typos, unclear wording or error messages in low visibility fields.

Severity levels

• High: A major issue where a large piece of functionality or major system component is
completely broken. There is no workaround and testing cannot continue.

• Medium: A major issue where a large piece of functionality or major system component
is not working properly. There is a workaround, however, and testing can continue.

• Low: A minor issue that imposes some loss of functionality, but for which there is an
acceptable and easily reproducible workaround. Testing can proceed without
interruption.

Severity and Priority


Priority is Relative: the priority might change over time. Perhaps a bug initially deemed P1
becomes rated as P2 or even a P3 as the schedule draws closer to the release and as the test
team finds even more heinous errors. Priority is a subjective evaluation of how important an
issue is, given other tasks in the queue and the current schedule. It’s relative. It shifts over
time. And it’s a business decision.
Severity is an absolute: it’s an assessment of the impact of the bug without regard to other
work in the queue or the current schedule. The only reason severity should change is if we
have new information that causes us to re-evaluate our assessment. If it was a high severity
issue when I entered it, it’s still a high severity issue when it’s deferred to the next release.
The severity hasn’t changed just because we’ve run out of time. The priority changed.

Severity Levels can be defined as follow:


S1 - Urgent/Showstopper. Like system crash or error message forcing to close the window.
Tester's ability to operate the system either totally (System Down), or almost totally, affected.
A major area of the users system is affected by the incident and it is significant to business
processes.

S2 - Medium/Workaround. Exist like when a problem is required in the specs but tester can go
on with testing. Incident affects an area of functionality but there is a work-around which
negates impact to business process. This is a problem that:
a) Affects a more isolated piece of functionality.
b) Occurs only at certain boundary conditions.
c) Has a workaround (where "don't do that" might be an acceptable answer to the user).
d) Occurs only at one or two customers. or is intermittent

S3 - Low. This is for minor problems, such as failures at extreme boundary conditions that are
unlikely to occur in normal use, or minor errors in
layout/formatting. Problems do not impact use of the product in any substantive way. These
are incidents that are cosmetic in nature and of no or very low impact to business processes.
21. What is Use case?
A simple flow between the end user and the system. It contains pre conditions, post conditions,
normal flows and exceptions. It is done by Team Lead/Test Lead/Tester.
22. Diff. between STLC and SDLC?
STLC is software test life cycle it starts with

• Preparing the test strategy.

• Preparing the test plan.

• Creating the test environment.

• Writing the test cases.

• Creating test scripts.

• Executing the test scripts.

• Analyzing the results and reporting the bugs.

• Doing regression testing.

• Test exiting.

SDLC is software or system development life cycle, phases are...

• Project initiation.
• Requirement gathering and documenting.

• Designing.

• Coding and unit testing.

• Integration testing.

• System testing.

• Installation and acceptance testing. " Support or maintenance

23. How you are breaking down the project among team members?
It can be depend on these following cases----
1) Number of modules
2) Number of team members
3) Complexity of the Project
4) Time Duration of the project
5) Team member's experience etc......
24. What is Test Data Collection?
Test data is the collection of input data taken for testing the application. Various types and
size of input data will be taken for testing the applications. Sometimes in critical application
the test data collection will be given by the client also.
25. What is Test Server?
The place where the developers put their development modules, which are accessed by the
testers to test the functionality.
26. What are non-functional requirements?
The non-functional requirements of a software product are: reliability, usability, efficiency,
delivery time, software development environment, security requirements, standards to be
followed etc.
27. What are the differences between these three words Error, Defect and Bug?
Error: The deviation from the required logic, syntax or standards/ethics is called as error.

There are three types of error. They are:


Syntax error (This is due to deviation from the syntax of the language what supposed to
follow).
Logical error (This is due to deviation from the logic of the program what supposed to follow)
Execution error (This is generally happens when you are executing the same program, that time
you get it.)
Defect: When an error found by the test engineer (testing department) then it is called defect
Bug: if the defect is agreed by the developer then it converts into bug, which has to fix by the
developer or post pond to next version.
28. Why we perform stress-testing, resolution-testing and cross- browser testing?
Stress Testing: - We need to check the performance of the application.
Def: Testing conducted to evaluate a system or component at or beyond the limits of its
specified requirements
Resolution Testing: - Some times developer created only for 1024 resolution, the same page
displayed a horizontal scroll bar in 800 x 600 resolutions. No body can like the horizontal scroll
appears in the screen. That is reason to test the Resolution testing.

Cross-browser Testing: - This testing some times called compatibility testing. When we develop
the pages in IE compatible, the same page is not working in Fairfox or Netscape properly,
because
most of the scripts are not supporting to other than IE. So that we need to test the cross-
browser Testing
29. There are two sand clocks(timers) one complete totally in 7 minutes and other in 9-
minutes we have to calculate with this timers and bang the bell after completion of 11-
minutes!plz give me the solution.
1. Start both clocks
2. When 7 min clock complete, turn it so that it restarts.
3. When 9 min clock finish, turn 7 min clocks (It has 2 mints only).
4. When 7 min clock finishes, 11 min complete.
30. What is the minimum criteria for white box?
We should know the logic, code and the structure of the program or function. Internal
knowledge of the application how the system works what's the logic behind it and structure
how it should react to particular action.
31. What are the technical reviews?
For each document, it should be reviewed. Technical Review in the sense, for each screen,
developer will write a Technical Specification. It should be reviewed by developer and tester.
There are functional specification review, unit test case review and code review etc.
32. In what basis you will write test cases?
I would write the Test cases based on Functional Specifications and BRDs and some more test
cases using the Domain knowledge.
33. Explain ETVX concept?
E- Entry Criteria
T- Task
V- Validation
X- Exit Criteria
ENTRY CRITERIA: Input with 'condition' attached.
e.g. Approved SRS document is the entry criteria for the design phase.

TASK: Procedures.
e.g. Preparation of HLD, LLD etc.

VALIDATION: Building quality & Verification activities


e.g. Technical reviews

EXIT CRITERIA: Output with 'condition' attached.


e.g Approved design document
It is important to follow ETVX concept for all phases in SDLC.
34. What are the main key components in Web applications and client and Server
applications? (differences)
For Web Applications: Web application can be implemented using any kind of technology like
Java, .NET, VB, ASP, CGI& PERL. Based on the technology,We can derive the components.

Let's take Java Web Application. It can be implemented in 3 tier architecture. Presentation tier
(jsp, html, dthml,servlets, struts). Busienss Tier (Java Beans, EJB, JMS) Data Tier(Databases
like Oracle, SQL Server etc., )

If you take .NET Application, Presentation (ASP, HTML, DHTML), Business Tier (DLL) & Data Tier
( Database like Oracle, SQL Server etc.,)

Client Server Applications: It will have only 2 tiers. One is Presentation (Java, Swing) and Data
Tier (Oracle, SQL Server). If it is client Server architecture, the entire application has to be
installed on the client machine. When ever you do any changes in your code, Again, It has to be
installed on all the client machines. Where as in Web Applications, Core Application will reside
on the server and client can be thin Client(browser). Whatever the changes you do, you have to
install the application in the server. NO need to worry about the clients. Because, You will not
install any thing on the client machine.
35. If the client identified some bugs to whom did he reported?
He will report to the Project Manager. Project Manager will arrange a meeting with all the
leads (Dev. Manager, Test Lead and Requirement Manager) then raise a Change Request and
then, identify which all the screens are going to be impacted by the bug. They will take the
code and correct it and send it to the Testing Team.
36. What is the formal technical review?
Technical review should be done by the team of members. The document, which is going to be
reviewed, who has prepared and reviewers should sit together and do the review of that
document. It is called Peer Review. If it is a technical document, It can be called as formal
Technical review, I guess. It varies depends on the company policy.
37. At what phase tester role starts?
In SDLC after complition of FRS document the test lead prepare the use case document and
test plan document, then the tester role is start.
38. Explain 'Software metrics'?
Measurement is fundamental to any engineering discipline
Why Metrics?
- We cannot control what we cannot measure!
- Metrics helps to measure quality
- Serves as dash-board

The main metrices are :size,shedule,defects.In this there are main sub metrices.

Test Coverage = Number of units (KLOC/FP) tested / total size of the system
Test cost (in %) = Cost of testing / total cost *100
Cost to locate defect = Cost of testing / the number of defects located
Defects detected in testing (in %) = Defects detected in testing / total system defects*100
Acceptance criteria tested = Acceptance criteria tested / total acceptance criteria
39. Actually how many positive and negetive testcases will write for a module?
That depends on the module and complexity of logic. For every test case, we can identify +ve
and -ve points. Based on the criteria, we will write the test cases, If it is crucial process or
screen. We should check the screen,in all the boundary conditions.
40. What is Software reliability?
It is the probability that software will work without failure for a specified period of time in a
specified environment.Reliability of software is measured in terms of Mean Time Between
Failure (MTBF). For eg if MTBF = 10000 hours for an average software, then it should not fail for
10000 hours of continous operation.
41. What are the main bugs which were identified by you and in that how many are
considered as real bugs?
If you take one screen, let's say, it has got 50 Test conditions, out of which, I have identified 5
defects which are failed. I should give the description defect, severity and defect classfication.
All the defects will be considered.

Defect Classification are:


GRP : Graphical Representation
LOG : Logical Error
DSN : Design Error
STD : Standard Error
TST : Wrong Test case
TYP : Typographical Error (Cosmotic Error)
42. What the main use of preparing a traceability matrix?
Traceability matrix is prepared in order to cross check the test cases designed against each
requirement, hence giving an opportunity to verify that all the requirements are covered in
testing the application.
(Or)
To Cross verify the prepared test cases and test scripts with user requirements. To monitor the
changes, enhance occurred during the development of the project.
43. What is Six sigma? Explain.
Six Sigma:
A quality discipline that focuses on product and service excellence to create a culture that
demands perfection on target, every time.

Six Sigma quality levels


Produces 99.9997% accuracy, with only 3.4 defects per million opportunities.

Six Sigma is designed to dramatically upgrade a company's performance, improving quality and
productivity. Using existing products, processes, and service standards,
They go for Six Sigma MAIC methodology to upgrade performance.

MAIC is defined as follows:


Measure: Gather the right data to accurately assess a problem.
Analyze: Use statistical tools to correctly identify the root causes of a problem
Improve: Correct the problem (not the symptom).
Control: Put a plan in place to make sure problems stay fixed and sustain the gains.

Key Roles and Responsibilities:

The key roles in all Six Sigma efforts are as follows:


Sponsor: Business executive leading the organization.
Champion: Responsible for Six Sigma strategy, deployment, and vision.
Process Owner: Owner of the process, product, or service being improved responsible for long-
term sustainable gains.
Master Black Belts: Coach black belts expert in all statistical tools.
Black Belts: Work on 3 to 5 $250,000-per-year projects; create $1 million per year in value.
Green Belts: Work with black belt on projects.
44. What is TRM?
TRM means Test Responsibility Matrix.

TRM: --- It indicates mapping between test factors and development stages...

Test factors like:


Ease of use, reliability, portability, authorization, access control, audit trail, ease of operates,
maintainable... Like dat...
Development stages...
Requirement gathering, Analysis, design, coding, testing, and maintenance
45. What are cookies? Tell me the advantage and disadvantage of cookies?
Cookies are messages that web servers pass to your web browser when you visit Internet sites.
Your browser stores each message in a small file. When you request another page from the
server, your browser sends the cookie back to the server. These files typically contain
information about your visit to the web page, as well as any information you've volunteered,
such as your name and interests. Cookies are most commonly used to track web site activity.
When you visit some sites, the server gives you a cookie that acts as your identification card.
Upon each return visit to that site, your browser passes that cookie back to the server. In this
way, a web server can gather information about which web pages are used the most, and which
pages are gathering the most repeat hits. Only the web site that creates the cookie can read it.
Additionally, web servers can only use information that you provide or choices that you make
while visiting the web site as content in cookies. Accepting a cookie does not give a server
access to your computer or any of your personal information. Servers can only read cookies
that they have set, so other servers do not have access to your information. Also, it is not
possible to execute code from a cookie, and not possible to use a cookie to deliver a virus.
46. What is the difference between Product-based Company and Projects-based Company?
Product based company develops the applications for Global clients i.e. there is no specific
clients. Here requirements are gathered from market and analyzed with experts.
Project based company develops the applications for the specific client. The requirements are
gathered from the client and analyzed with the client.

What makes a good test engineer?

A good test engineer has a 'test to break' attitude, an ability to take the point of view of the

customer, a strong desire for quality, and an attention to detail. Tact and diplomacy are useful

in maintaining a cooperative relationship with developers, and an ability to communicate with

both technical (developers) and non-technical (customers, management) people is useful.


Previous software development experience can be helpful as it provides a deeper

understanding of the software development process, gives the tester an appreciation for the

developers' point of view, and reduce the learning curve in automated test tool programming.

Judgement skills are needed to assess high-risk areas of an application on which to focus

testing efforts when time is limited.

What makes a good Software QA engineer?

The same qualities a good tester has are useful for a QA engineer. Additionally, they must be

able to understand the entire software development process and how it can fit into the

business approach and goals of the organization. Communication skills and the ability to

understand various sides of issues are important. In organizations in the early stages of

implementing QA processes, patience and diplomacy are especially needed. An ability to find

problems as well as to see 'what's missing' is important for inspections and reviews.

What makes a good QA or Test manager?

A good QA, test, or QA/Test(combined) manager should:

• be familiar with the software development process

• be able to maintain enthusiasm of their team and promote a positive atmosphere, despite

• what is a somewhat 'negative' process (e.g., looking for or preventing problems)

• be able to promote teamwork to increase productivity

• be able to promote cooperation between software, test, and QA engineers

• have the diplomatic skills needed to promote improvements in QA processes

• have the ability to withstand pressures and say 'no' to other managers when quality is

insufficient or QA processes are not being adhered to

• have people judgement skills for hiring and keeping skilled personnel

• be able to communicate with technical and non-technical people, engineers, managers, and

customers.

• be able to run meetings and keep them focused

What's the role of documentation in QA?


Critical. (Note that documentation can be electronic, not necessarily paper.) QA practices

should be documented such that they are repeatable. Specifications, designs, business rules,

inspection reports, configurations, code changes, test plans, test cases, bug reports, user

manuals, etc. should all be documented. There should ideally be a system for easily finding and

obtaining documents and determining what documentation will have a particular piece of

information. Change management for documentation should be used if possible.

What's the big deal about 'requirements'?

One of the most reliable methods of insuring problems, or failure, in a complex software

project is to have poorly documented requirements specifications. Requirements are the

details describing an application's externally-perceived functionality and properties.

Requirements should be clear, complete, reasonably detailed, cohesive, attainable, and

testable. A non-testable requirement would be, for example, 'user-friendly' (too subjective). A

testable requirement would be something like 'the user must enter their previously-assigned

password to access the application'. Determining and organizing requirements details in a

useful and efficient way can be a difficult effort; different methods are available depending on

the particular project. Many books are available that describe various approaches to this task.

(See the Bookstore section's 'Software Requirements Engineering' category for books on

Software Requirements.)

Care should be taken to involve ALL of a project's significant 'customers' in the requirements

process. 'Customers' could be in-house personnel or out, and could include end-users, customer

acceptance testers, customer contract officers, customer management, future software

maintenance engineers, salespeople, etc. Anyone who could later derail the project if their

expectations aren't met should be included if possible.

Organizations vary considerably in their handling of requirements specifications. Ideally, the

requirements are spelled out in a document with statements such as 'The product shall.....'.

'Design' specifications should not be confused with 'requirements'; design specifications should

be traceable back to the requirements.

In some organizations requirements may end up in high level project plans, functional

specification documents, in design documents, or in other documents at various levels of


detail. No matter what they are called, some type of documentation with detailed

requirements will be needed by testers in order to properly plan and execute tests. Without

such documentation, there will be no clear-cut way to determine if a software application is

performing correctly.

'Agile' methods such as XP use methods requiring close interaction and cooperation between

programmers and customers/end-users to iteratively develop requirements. The programmer

uses 'Test first' development to first create automated unit testing code, which essentially

embodies the requirements.

What steps are needed to develop and run software tests?

The following are some of the steps to consider:

• Obtain requirements, functional design, and internal design specifications and other

necessary documents

• Obtain budget and schedule requirements

• Determine project-related personnel and their responsibilities, reporting requirements,

required standards and processes (such as release processes, change processes, etc.)

• Identify application's higher-risk aspects, set priorities, and determine scope and limitations

of tests

• Determine test approaches and methods - unit, integration, functional, system, load,

usability tests, etc.

• Determine test environment requirements (hardware, software, communications, etc.)

• Determine testware requirements (record/playback tools, coverage analyzers, test tracking,

problem/bug tracking, etc.)

• Determine test input data requirements

• Identify tasks, those responsible for tasks, and labor requirements

• Set schedule estimates, timelines, milestones

• Determine input equivalence classes, boundary value analyses, error classes

• Prepare test plan document and have needed reviews/approvals

• Write test cases

• Have needed reviews/inspections/approvals of test cases


• Prepare test environment and testware, obtain needed user manuals/reference

documents/configuration guides/installation guides, set up test tracking processes, set up

logging and archiving processes, set up or obtain test input data

• Obtain and install software releases

• Perform tests

• Evaluate and report results

• Track problems/bugs and fixes

• Retest as needed

• Maintain and update test plans, test cases, test environment, and testware through life cycle

What's a 'test plan'?

A software project test plan is a document that describes the objectives, scope, approach, and

focus of a software testing effort. The process of preparing a test plan is a useful way to think

through the efforts needed to validate the acceptability of a software product. The completed

document will help people outside the test group understand the 'why' and 'how' of product

validation. It should be thorough enough to be useful but not so thorough that no one outside

the test group will read it. The following are some of the items that might be included in a test

plan, depending on the particular project:

• Title

• Identification of software including version/release numbers

• Revision history of document including authors, dates, approvals

• Table of Contents

• Purpose of document, intended audience

• Objective of testing effort

• Software product overview

• Relevant related document list, such as requirements, design documents, other test plans,

etc.

• Relevant standards or legal requirements

• Traceability requirements

• Relevant naming conventions and identifier conventions


• Overall software project organization and personnel/contact-info/responsibilties

• Test organization and personnel/contact-info/responsibilities

• Assumptions and dependencies

• Project risk analysis

• Testing priorities and focus

• Scope and limitations of testing

• Test outline - a decomposition of the test approach by test type, feature, functionality,

process, system, module, etc. as applicable

• Outline of data input equivalence classes, boundary value analysis, error classes

• Test environment - hardware, operating systems, other required software, data

configurations, interfaces to other systems

• Test environment validity analysis - differences between the test and production systems and

their impact on test validity.

• Test environment setup and configuration issues

• Software migration processes

• Software CM processes

• Test data setup requirements

• Database setup requirements

• Outline of system-logging/error-logging/other capabilities, and tools such as screen capture

software, that will be used to help describe and report bugs

• Discussion of any specialized software or hardware tools that will be used by testers to help

track the cause or source of bugs

• Test automation - justification and overview

• Test tools to be used, including versions, patches, etc.

• Test script/test code maintenance processes and version control

• Problem tracking and resolution - tools and processes

• Project test metrics to be used

• Reporting requirements and testing deliverables

• Software entrance and exit criteria

• Initial sanity testing period and criteria


• Test suspension and restart criteria

• Personnel allocation

• Personnel pre-training needs

• Test site/location

• Outside test organizations to be utilized and their purpose, responsibilties, deliverables,

contact persons, and coordination issues

• Relevant proprietary, classified, security, and licensing issues.

• Open issues

• Appendix - glossary, acronyms, etc.

(See the Bookstore section's 'Software Testing' and 'Software QA' categories for useful books

with more information.)

What's a 'test case'?

• A test case is a document that describes an input, action, or event and an expected response,

to determine if a feature of an application is working correctly. A test case should contain

particulars such as test case identifier, test case name, objective, test conditions/setup, input

data requirements, steps, and expected results.

• Note that the process of developing test cases can help find problems in the requirements or

design of an application, since it requires completely thinking through the operation of the

application. For this reason, it's useful to prepare test cases early in the development cycle if

possible.

What should be done after a bug is found?

The bug needs to be communicated and assigned to developers that can fix it. After the

problem is resolved, fixes should be re-tested, and determinations made regarding

requirements for regression testing to check that fixes didn't create problems elsewhere. If a

problem-tracking system is in place, it should encapsulate these processes. A variety of

commercial problem-tracking/management software tools are available (see the 'Tools' section

for web resources with listings of such tools). The following are items to consider in the

tracking process:
• Complete information such that developers can understand the bug, get an idea of it's

severity, and reproduce it if necessary.

• Bug identifier (number, ID, etc.)

• Current bug status (e.g., 'Released for Retest', 'New', etc.)

• The application name or identifier and version

• The function, module, feature, object, screen, etc. where the bug occurred

• Environment specifics, system, platform, relevant hardware specifics

• Test case name/number/identifier

• One-line bug description

• Full bug description

• Description of steps needed to reproduce the bug if not covered by a test case or if the

developer doesn't have easy access to the test case/test script/test tool

• Names and/or descriptions of file/data/messages/etc. used in test

• File excerpts/error messages/log file excerpts/screen shots/test tool logs that would be

helpful in finding the cause of the problem

• Severity estimate (a 5-level range such as 1-5 or 'critical'-to-'low' is common)

• Was the bug reproducible?

• Tester name

• Test date

• Bug reporting date

• Name of developer/group/organization the problem is assigned to

• Description of problem cause

• Description of fix

• Code section/file/module/class/method that was fixed

• Date of fix

• Application version that contains the fix

• Tester responsible for retest

• Retest date

• Retest results

• Regression testing requirements


• Tester responsible for regression tests

• Regression testing results

A reporting or tracking process should enable notification of appropriate personnel at various

stages. For instance, testers need to know when retesting is needed, developers need to know

when bugs are found and how to get the needed information, and reporting/summary

capabilities are needed for managers.

What is 'configuration management'?

Configuration management covers the processes used to control, coordinate, and track: code,

requirements, documentation, problems, change requests, designs,

tools/compilers/libraries/patches, changes made to them, and who makes the changes. (See

the 'Tools' section for web resources with listings of configuration management tools. Also see

the Bookstore section's 'Configuration Management' category for useful books with more

information.)

What if the software is so buggy it can't really be tested at all?

The best bet in this situation is for the testers to go through the process of reporting whatever

bugs or blocking-type problems initially show up, with the focus being on critical bugs. Since

this type of problem can severely affect schedules, and indicates deeper problems in the

software development process (such as insufficient unit testing or insufficient integration

testing, poor design, improper build or release procedures, etc.) managers should be notified,

and provided with some documentation as evidence of the problem.

How can it be known when to stop testing?

This can be difficult to determine. Many modern software applications are so complex, and run

in such an interdependent environment, that complete testing can never be done. Common

factors in deciding when to stop are:

• Deadlines (release deadlines, testing deadlines, etc.)

• Test cases completed with certain percentage passed

• Test budget depleted


• Coverage of code/functionality/requirements reaches a specified point

• Bug rate falls below a certain level

• Beta or alpha testing period ends

What if there isn't enough time for thorough testing?

Use risk analysis to determine where testing should be focused.

Since it's rarely possible to test every possible aspect of an application, every possible

combination of events, every dependency, or everything that could go wrong, risk analysis is

appropriate to most software development projects. This requires judgement skills, common

sense, and experience. (If warranted, formal methods are also available.) Considerations can

include:

• Which functionality is most important to the project's intended purpose?

• Which functionality is most visible to the user?

• Which functionality has the largest safety impact?

• Which functionality has the largest financial impact on users?

• Which aspects of the application are most important to the customer?

• Which aspects of the application can be tested early in the development cycle?

• Which parts of the code are most complex, and thus most subject to errors?

• Which parts of the application were developed in rush or panic mode?

• Which aspects of similar/related previous projects caused problems?

• Which aspects of similar/related previous projects had large maintenance expenses?

• Which parts of the requirements and design are unclear or poorly thought out?

• What do the developers think are the highest-risk aspects of the application?

• What kinds of problems would cause the worst publicity?

• What kinds of problems would cause the most customer service complaints?

• What kinds of tests could easily cover multiple functionalities?

• Which tests will have the best high-risk-coverage to time-required ratio?

What if the project isn't big enough to justify extensive testing?

Consider the impact of project errors, not the size of the project. However, if extensive testing
is still not justified, risk analysis is again needed and the same considerations as described

previously in 'What if there isn't enough time for thorough testing?' apply. The tester might then

do ad hoc testing, or write up a limited test plan based on the risk analysis.

What can be done if requirements are changing continuously?

A common problem and a major headache.

• Work with the project's stakeholders early on to understand how requirements might change

so that alternate test plans and strategies can be worked out in advance, if possible.

• It's helpful if the application's initial design allows for some adaptability so that later changes

do not require redoing the application from scratch.

• If the code is well-commented and well-documented this makes changes easier for the

developers.

• Use rapid prototyping whenever possible to help customers feel sure of their requirements

and minimize changes.

• The project's initial schedule should allow for some extra time commensurate with the

possibility of changes.

• Try to move new requirements to a 'Phase 2' version of an application, while using the

original requirements for the 'Phase 1' version.

• Negotiate to allow only easily-implemented new requirements into the project, while moving

more difficult new requirements into future versions of the application.

• Be sure that customers and management understand the scheduling impacts, inherent risks,

and costs of significant requirements changes. Then let management or the customers (not the

developers or testers) decide if the changes are warranted - after all, that's their job.

• Balance the effort put into setting up automated testing with the expected effort required to

re-do them to deal with changes.

• Try to design some flexibility into automated test scripts.

• Focus initial automated testing on application aspects that are most likely to remain

unchanged.

• Devote appropriate effort to risk analysis of changes to minimize regression testing needs.

• Design some flexibility into test cases (this is not easily done; the best bet might be to
minimize the detail in the test cases, or set up only higher-level generic-type test plans)

• Focus less on detailed test plans and test cases and more on ad hoc testing (with an

understanding of the added risk that this entails).

What if the application has functionality that wasn't in the requirements?

It may take serious effort to determine if an application has significant unexpected or hidden

functionality, and it would indicate deeper problems in the software development process. If

the functionality isn't necessary to the purpose of the application, it should be removed, as it

may have unknown impacts or dependencies that were not taken into account by the designer

or the customer. If not removed, design information will be needed to determine added testing

needs or regression testing needs. Management should be made aware of any significant added

risks as a result of the unexpected functionality. If the functionality only effects areas such as

minor improvements in the user interface, for example, it may not be a significant risk.

How can Software QA processes be implemented without stifling productivity?

By implementing QA processes slowly over time, using consensus to reach agreement on

processes, and adjusting and experimenting as an organization grows and matures, productivity

will be improved instead of stifled. Problem prevention will lessen the need for problem

detection, panics and burn-out will decrease, and there will be improved focus and less wasted

effort. At the same time, attempts should be made to keep processes simple and efficient,

minimize paperwork, promote computer-based processes and automated tracking and

reporting, minimize time required in meetings, and promote training as part of the QA process.

However, no one - especially talented technical types - likes rules or bureacracy, and in the

short run things may slow down a bit. A typical scenario would be that more days of planning

and development will be needed, but less time will be required for late-night bug-fixing and

calming of irate customers.

What if an organization is growing so fast that fixed QA processes are impossible?

This is a common problem in the software industry, especially in new technology areas. There

is no easy solution in this situation, other than:


• Hire good people

• Management should 'ruthlessly prioritize' quality issues and maintain focus on the customer

• Everyone in the organization should be clear on what 'quality' means to the customer

How does a client/server environment affect testing?

Client/server applications can be quite complex due to the multiple dependencies among

clients, data communications, hardware, and servers. Thus testing requirements can be

extensive. When time is limited (as it usually is) the focus should be on integration and system

testing. Additionally, load/stress/performance testing may be useful in determining

client/server application limitations and capabilities. There are commercial tools to assist with

such testing. (See the 'Tools' section for web resources with listings that include these kinds of

test tools.)

How can World Wide Web sites be tested?

Web sites are essentially client/server applications - with web servers and 'browser' clients.

Consideration should be given to the interactions between html pages, TCP/IP communications,

Internet connections, firewalls, applications that run in web pages (such as applets, javascript,

plug-in applications), and applications that run on the server side (such as cgi scripts, database

interfaces, logging applications, dynamic page generators, asp, etc.). Additionally, there are a

wide variety of servers and browsers, various versions of each, small but sometimes significant

differences between them, variations in connection speeds, rapidly changing technologies, and

multiple standards and protocols. The end result is that testing for web sites can become a

major ongoing effort. Other considerations might include:

• What are the expected loads on the server (e.g., number of hits per unit time?), and what

kind of performance is required under such loads (such as web server response time, database

query response times). What kinds of tools will be needed for performance testing (such as web

load testing tools, other tools already in house that can be adapted, web robot downloading

tools, etc.)?

• Who is the target audience? What kind of browsers will they be using? What kind of

connection speeds will they by using? Are they intra- organization (thus with likely high
connection speeds and similar browsers) or Internet-wide (thus with a wide variety of

connection speeds and browser types)?

• What kind of performance is expected on the client side (e.g., how fast should pages appear,

how fast should animations, applets, etc. load and run)?

• Will down time for server and content maintenance/upgrades be allowed? how much?

• What kinds of security (firewalls, encryptions, passwords, etc.) will be required and what is it

expected to do? How can it be tested?

• How reliable are the site's Internet connections required to be? And how does that affect

backup system or redundant connection requirements and testing?

• What processes will be required to manage updates to the web site's content, and what are

the requirements for maintaining, tracking, and controlling page content, graphics, links, etc.?

• Which HTML specification will be adhered to? How strictly? What variations will be allowed

for targeted browsers?

• Will there be any standards or requirements for page appearance and/or graphics throughout

a site or parts of a site??

• How will internal and external links be validated and updated? how often?

• Can testing be done on the production system, or will a separate test system be required?

How are browser caching, variations in browser option settings, dial-up connection

variabilities, and real-world internet 'traffic congestion' problems to be accounted for in

testing?

• How extensive or customized are the server logging and reporting requirements; are they

considered an integral part of the system and do they require testing?

• How are cgi programs, applets, javascripts, ActiveX components, etc. to be maintained,

tracked, controlled, and tested?

Some sources of site security information include the Usenet newsgroup

'comp.security.announce' and links concerning web site security in the 'Other Resources'

section.

Some usability guidelines to consider - these are subjective and may or may not apply to a

given situation (Note: more information on usability testing issues can be found in articles

about web site usability in the 'Other Resources' section):


• Pages should be 3-5 screens max unless content is tightly focused on a single topic. If larger,

provide internal links within the page.

• The page layouts and design elements should be consistent throughout a site, so that it's

clear to the user that they're still within a site.

• Pages should be as browser-independent as possible, or pages should be provided or

generated based on the browser-type.

• All pages should have links external to the page; there should be no dead-end pages.

• The page owner, revision date, and a link to a contact person or organization should be

included on each page.

Many new web site test tools have appeared in the recent years and more than 280 of them are

listed in the 'Web Test Tools' section.

Software Life Cycle

The software life cycle typically includes the following: requirements analysis,
design, coding, testing, installation and maintenance. In between, there can
be a requirement to provide Operations and support activities for the
product.

FOREWORD

Beginners Guide To Software Testing introduces a practical approach to


testing software. It bridges the gap between theoretical knowledge and real
world implementation. This article helps you gain an insight to Software
Testing - understand technical aspects and the processes followed in a real
working environment.
Who will benefit?

Beginners. For those of you who wish to mould your theoretical software
engineering knowledge into practical approach to working in the real world.

Those who wish to take up Software Testing as a profession.

Developers! This is an era where you need to be an “All rounder”. It is


advantageous for developers to posses testing capabilities to test the
application before hand. This will help reduce overhead on the testing team.

Already a Tester! You can refresh all your testing basics and techniques
and gear up for Certifications in Software Testing

An earnest suggestion: No matter which profession you choose, it is


advisable that you posses the following skills:

- Good communication skills – oratory and writing


- Fluency in English
- Good Typing skills

By the time you finish reading this article, you will be aware of all the
techniques and processes that improves your efficiency, skills and confidence
to jump start into the field of Software Testing.[/size]

<!--[if !supportLineBreakNewLine]-->
1. Overview

The Big Picture


All software problems can be termed as bugs. A software bug usually occurs
when the software does not do what it is intended to do or does something
that it is not intended to do. Flaws in specifications, design, code or other
reasons can cause these bugs. Identifying and fixing bugs in the early stages
of the software is very important as the cost of fixing bugs grows over time.
So, the goal of a software tester is to find bugs and find them as early as
possible and make sure they are fixed.

Testing is context-based and risk-driven. It requires a methodical and


disciplined approach to finding bugs. A good software tester needs to build
credibility and possess the attitude to be explorative, troubleshooting,
relentless, creative, diplomatic and persuasive.

As against the perception that testing starts only after the completion of
coding phase, it actually begins even before the first line of code can be
written. In the life cycle of the conventional software product, testing begins
at the stage when the specifications are written, i.e. from testing the product
specifications or product spec. Finding bugs at this stage can save huge
amounts of time and money.

Once the specifications are well understood, you are required to design and
execute the test cases. Selecting the appropriate technique that reduces the
number of tests that cover a feature is one of the most important things that
you need to take into consideration while designing these test cases. Test
cases need to be designed to cover all aspects of the software, i.e. security,
database, functionality (critical and general) and the user interface. Bugs
originate when the test cases are executed.

As a tester you might have to perform testing under different circumstances,


i.e. the application could be in the initial stages or undergoing rapid changes,
you have less than enough time to test, the product might be developed
using a life cycle model that does not support much of formal testing or
retesting. Further, testing using different operating systems, browsers and
the configurations are to be taken care of.

Reporting a bug may be the most important and sometimes the most difficult
task that you as a software tester will perform. By using various tools and
clearly communicating to the developer, you can ensure that the bugs you
find are fixed.

Using automated tools to execute tests, run scripts and tracking bugs
improves efficiency and effectiveness of your tests. Also, keeping pace with
the latest developments in the field will augment your career as a software
test engineer.

What is software? Why should it be tested?

Software is a series of instructions for the computer that perform a particular


task, called a program; the two major categories of software are system
software and application software. System software is made up of control
programs. Application software is any program that processes data for the
user (spreadsheet, word processor, payroll, etc.).

A software product should only be released after it has gone through a proper
process of development, testing and bug fixing. Testing looks at areas such
as performance, stability and error handling by setting up test scenarios
under controlled conditions and assessing the results. This is why exactly any
software has to be tested. It is important to note that software is mainly
tested to see that it meets the customers’ needs and that it conforms to the
standards. It is a usual norm that software is considered of good quality if it
meets the user requirements.
What is Quality? How important is it?

Quality can briefly be defined as “a degree of excellence”. High quality


software usually conforms to the user requirements. A customer’s idea of
quality may cover a breadth of features - conformance to specifications, good
performance on platform(s)/configurations, completely meets operational
requirements (even if not specified!), compatibility to all the end-user
equipment, no negative impact on existing end-user base at introduction
time.

Quality software saves good amount of time and money. Because software
will have fewer defects, this saves time during testing and maintenance
phases. Greater reliability contributes to an immeasurable increase in
customer satisfaction as well as lower maintenance costs. Because
maintenance represents a large portion of all software costs, the overall cost
of the project will most likely be lower than similar projects.

Following are two cases that demonstrate the importance of software quality:

Ariane 5 crash June 4, 1996- Maiden flight of the European Ariane 5 launcher
crashed about 40 seconds after takeoff
- Loss was about half a billion dollars
- Explosion was the result of a software error
- Uncaught exception due to floating-point error: conversion from a 64-bit
integer to a 16-bit signed integer applied to a larger than expected number
- Module was re-used without proper testing from Ariane 4
- Error was not supposed to happen with Ariane 4
- No exception handler
Mars Climate Orbiter - September 23, 1999 - Mars Climate Orbiter,
disappeared as it began to orbit Mars.
- Cost about $US 125-million
- Failure due to error in a transfer of information between a team in Colorado
and a team in California
- One team used English units (e.g., inches, feet and pounds) while the other
used metric units for a key spacecraft operation.

What exactly does a software tester do?

Apart from exposing faults (“bugs”) in a software product confirming that the
program meets the program specification, as a test engineer you need to
create test cases, procedures, scripts and generate data. You execute test
procedures and scripts, analyze standards and evaluate results of
system/integration/regression testing. You also...

· Speed up development process by identifying bugs at an early stage (e.g.


specifications stage)
· Reduce the organization's risk of legal liability
· Maximize the value of the software
· Assure successful launch of the product, save money, time and reputation of
the company by discovering bugs and design flaws at an early stage before
failures occur in production, or in the field
· Promote continual improvement

What makes a good tester?

As software engineering is now being considered as a technical engineering


profession, it is important that the software test engineer’s posses certain
traits with a relentless attitude to make them stand out.
Here are a few.

· Know the technology. Knowledge of the technology in which the


application is developed is an added advantage to any tester. It helps design
better and powerful test cases basing on the weakness or flaws of the
technology. Good testers know what it supports and what it doesn’t, so
concentrating on these lines will help them break the application quickly.
· Perfectionist and a realist. Being a perfectionist will help testers spot the
problem and being a realist helps know at the end of the day which problems
are really important problems. You will know which ones require a fix and
which ones don’t.
· Tactful, diplomatic and persuasive. Good software testers are tactful
and know how to break the news to the developers. They are diplomatic while
convincing the developers of the bugs and persuade them when necessary
and have their bug(s) fixed. It is important to be critical of the issue and not
let the person who developed the application be taken aback of the findings.
· An explorer. A bit of creativity and an attitude to take risk helps the
testers venture into unknown situations and find bugs that otherwise will be
looked over.
· Troubleshoot. Troubleshooting and figuring out why something doesn’t
work helps testers be confident and clear in communicating the defects to
the developers.
· Posses people skills and tenacity. Testers can face a lot of resistance
from programmers. Being socially smart and diplomatic doesn't mean being
indecisive. The best testers are both-socially adept and tenacious where it
matters.
· Organized. Best testers very well realize that they too can make mistakes
and don’t take chances. They are very well organized and have checklists,
use files, facts and figures to support their findings that can be used as an
evidence and double-check their findings.
· Objective and accurate. They are very objective and know what they
report and so convey impartial and meaningful information that keeps politics
and emotions out of message. Reporting inaccurate information is losing a
little credibility. Good testers make sure their findings are accurate and
reproducible.
· Defects are valuable. Good testers learn from them. Each defect is an
opportunity to learn and improve. A defect found early substantially costs
less when compared to the one found at a later stage. Defects can cause
serious problems if not managed properly. Learning from defects helps –
prevention of future problems, track improvements, improve prediction and
estimation.

Guidelines for new testers·

. Testing can’t show that bugs don’t exist. An important reason for
testing is to prevent defects. You can perform your tests, find and report
bugs, but at no point can you guarantee that there are no bugs.
· It is impossible to test a program completely. Unfortunately this is not
possible even with the simplest program because – the number of inputs is
very large, number of outputs is very large, number of paths through the
software is very large, and the specification is subjective to frequent changes.

· You can’t guarantee quality. As a software tester, you cannot test


everything and are not responsible for the quality of the product. The main
way that a tester can fail is to fail to report accurately a defect you have
observed. It is important to remember that we seldom have little control over
quality.
· Target environment and intended end user. Anticipating and testing
the application in the environment user is expected to use is one of the major
factors that should be considered. Also, considering if the application is a
single user system or multi user system is important for demonstrating the
ability for immediate readiness when necessary. The error case of Disney’s
Lion King illustrates this. Disney Company released it first multimedia CD-
ROM game for children, The Lion King Animated Storybook. It was highly
promoted and the sales were huge. Soon there were reports that buyers were
unable to get the software to work. It worked on a few systems – likely the
ones that the Disney programmers used to create the game – but not on the
most common systems that the general public used.
· No application is 100% bug free. It is more reasonable to recognize
there are priorities, which may leave some less critical problems unsolved or
unidentified. Simple case is the Intel Pentium bug. Enter the following
equation into your PC’s calculator: (4195835 / 3145727) * 3145727 –
4195835. If the answer is zero, your computer is just fine. If you get anything
else, you have an old Intel Pentium CPU with a floating-point division bug.
· Be the customer. Try to use the system as a lay user. To get a glimpse of
this, get a person who has no idea of the application to use it for a while and
you will be amazed to see the number of problems the person seem to come
across. As you can see, there is no procedure involved. Doing this could
actually cause the system to encounter an array of unexpected tests –
repetition, stress, load, race etc.
· Build your credibility. Credibility is like quality that includes reliability,
knowledge, consistency, reputation, trust, attitude and attention to detail. It
is not instant but should be built over time and gives voice to the testers in
the organization. Your keys to build credibility – identify your strengths and
weaknesses, build good relations, demonstrate competency, be willing to
admit mistakes, re-assess and adjust.
· Test what you observe. It is very important that you test what you can
observe and have access to. Writing creative test cases can help only when
you have the opportunity to observe the results. So, assume nothing.
· Not all bugs you find will be fixed. Deciding which bugs will be fixed and
which won’t is a risk-based decision. Several reasons why your bug might not
be fixed is when there is no enough time, the bug is dismissed for a new
feature, fixing it might be very risky or it may not be worth it because it
occurs infrequently or has a work around where the user can prevent or avoid
the bug. Making a wrong decision can be disastrous.
· Review competitive products. Gaining a good insight into various
products of the same kind and getting to know their functionality and general
behavior will help you design different test cases and to understand the
strengths and weaknesses of your application. This will also enable you to
add value and suggest new features and enhancements to your product.
· Follow standards and processes. As a tester, your need to conform to
the standards and guidelines set by the organization. These standards
pertain to reporting hierarchy, coding, documentation, testing, reporting
bugs, using automated tools etc.

2. Introduction

Software Life Cycle


The software life cycle typically includes the following: requirements analysis,
design, coding, testing, installation and maintenance. In between, there can
be a requirement to provide Operations and support activities for the
product.

Requirements Analysis. Software organizations provide solutions to


customer requirements by developing appropriate software that best suits
their specifications. Thus, the life of software starts with origin of
requirements. Very often, these requirements are vague, emergent and
always subject to change.
Analysis is performed to - To conduct in depth analysis of the proposed
project, To evaluate for technical feasibility, To discover how to partition the
system, To identify which areas of the requirements need to be elaborated
from the customer, To identify the impact of changes to the requirements, To
identify which requirements should be allocated to which components.

Design and Specifications. The outcome of requirements analysis is the


requirements specification. Using this, the overall design for the intended
software is developed.

Activities in this phase - Perform Architectural Design for the software, Design
Database (If applicable), Design User Interfaces, Select or Develop Algorithms
(If Applicable), Perform Detailed Design.

Coding. The development process tends to run iteratively through these


phases rather than linearly; several models (spiral, waterfall etc.) have been
proposed to describe this process.

Activities in this phase - Create Test Data, Create Source, Generate Object
Code, Create Operating Documentation, Plan Integration, Perform Integration.

Testing. The process of using the developed system with the intent to find
errors. Defects/flaws/bugs found at this stage will be sent back to the
developer for a fix and have to be re-tested. This phase is iterative as long as
the bugs are fixed to meet the requirements.

Activities in this phase - Plan Verification and Validation, Execute Verification


and validation Tasks, Collect and Analyze Metric Data, Plan Testing, Develop
Test Requirements, Execute Tests.

Installation. The so developed and tested software will finally need to be


installed at the client place. Careful planning has to be done to avoid
problems to the user after installation is done.

Activities in this phase - Plan Installation, Distribution of Software, Installation


of Software, Accept Software in Operational Environment.

Operation and Support. Support activities are usually performed by the


organization that developed the software. Both the parties usually decide on
these activities before the system is developed.

Activities in this phase - Operate the System, Provide Technical Assistance


and Consulting, Maintain Support Request Log.

Maintenance. The process does not stop once it is completely implemented


and installed at user place; this phase undertakes development of new
features, enhancements etc.

Activities in this phase - Reapplying Software Life Cycle.

Various Life Cycle Models

The way you approach a particular application for testing greatly depends on
the life cycle model it follows. This is because, each life cycle model places
emphasis on different aspects of the software i.e. certain models provide
good scope and time for testing whereas some others don’t. So, the number
of test cases developed, features covered, time spent on each issue depends
on the life cycle model the application follows.
No matter what the life cycle model is, every application undergoes the same
phases described above as its life cycle.

Following are a few software life cycle models, their advantages and
disadvantages.

Waterfall Model

Strengths:
•Emphasizes completion of one phase before moving on
•Emphasises early planning, customer input, and design
•Emphasises testing as an integral part of the life cycle •Provides quality
gates at each life cycle phase

Weakness:
•Depends on capturing and freezing requirements early in the life cycle
•Depends on separating requirements from design
•Feedback is only from testing phase to any previous stage
•Not feasible in some organizations
•Emphasises products rather than processes

Prototyping Model

Strengths:
•Requirements can be set earlier and more reliably
•Requirements can be communicated more clearly and completelybetween
developers and clients
•Requirements and design options can be investigated quickly and with low
cost
•More requirements and design faults are caught early

Weakness:
•Requires a prototyping tool and expertise in using it – a cost for the
development organisation
•The prototype may become the production system

Spiral Model

Strengths:
•It promotes reuse of existing software in early stages of development
•Allows quality objectives to be formulated during development
•Provides preparation for eventual evolution of the software product
•Eliminates errors and unattractive alternatives early.
•It balances resource expenditure.
•Doesn’t involve separate approaches for software development and
software maintenance.
•Provides a viable framework for integrated Hardware-software system
development.

Weakness:
•This process needs or usually associated with Rapid Application
Development, which is very difficult practically.
•The process is more difficult to manage and needs a very different approach
as opposed to the waterfall model (Waterfall model has management
techniques like GANTT charts to assess)

Software Testing Life Cycle

Software Testing Life Cycle consist of six (generic) phases: 1) Planning, 2)


Analysis, 3) Design, 4) Construction, 5) Testing Cycles, 6) Final Testing and
Implementation and 7) Post Implementation. Each phase in the life cycle is
described with the respective activities.

Planning. Planning High Level Test plan, QA plan (quality goals), identify –
reporting procedures, problem classification, acceptance criteria, databases
for testing, measurement criteria (defect quantities/severity level and defect
origin), project metrics and finally begin the schedule for project testing. Also,
plan to maintain all test cases (manual or automated) in a database.

Analysis. Involves activities that - develop functional validation based on


Business Requirements (writing test cases basing on these details), develop
test case format (time estimates and priority assignments), develop test
cycles (matrices and timelines), identify test cases to be automated (if
applicable), define area of stress and performance testing, plan the test
cycles required for the project and regression testing, define procedures for
data maintenance (backup, restore, validation), review documentation.

Design. Activities in the design phase - Revise test plan based on changes,
revise test cycle matrices and timelines, verify that test plan and cases are in
a database or requisite, continue to write test cases and add new ones based
on changes, develop Risk Assessment Criteria, formalize details for Stress
and Performance testing, finalize test cycles (number of test case per cycle
based on time estimates per test case and priority), finalize the Test Plan,
(estimate resources to support development in unit testing).

Construction (Unit Testing Phase). Complete all plans, complete Test Cycle
matrices and timelines, complete all test cases (manual), begin Stress and
Performance testing, test the automated testing system and fix bugs,
(support development in unit testing), run QA acceptance test suite to certify
software is ready to turn over to QA.

Test Cycle(s) / Bug Fixes (Re-Testing/System Testing Phase). Run the test
cases (front and back end), bug reporting, verification, revise/add test cases
as required.

Final Testing and Implementation (Code Freeze Phase). Execution of all


front end test cases - manual and automated, execution of all back end test
cases - manual and automated, execute all Stress and Performance tests,
provide on-going defect tracking metrics, provide on-going complexity and
design metrics, update estimates for test cases and test plans, document test
cycles, regression testing, and update accordingly.

Post Implementation. Post implementation evaluation meeting can be


conducted to review entire project. Activities in this phase - Prepare final
Defect Report and associated metrics, identify strategies to prevent similar
problems in future project, automation team - 1) Review test cases to
evaluate other cases to be automated for regression testing, 2) Clean up
automated test cases and variables, and 3) Review process of integrating
results from automated testing in with results from manual testing.

Testing tools

Black box testing - not based on any knowledge of internal design or code. Tests are
based on requirements and functionality.
White box testing - based on knowledge of the internal logic of an application's code.
Tests are based on coverage of code statements, branches, paths, conditions.
Unit testing - the most 'micro' scale of testing; to test particular functions or code
modules. Typically done by the programmer and not by testers, as it requires detailed
knowledge of the internal program design and code. Not always easily done unless the
application has a well-designed architecture with tight code; may require developing test
driver modules or test harnesses.
Incremental integration testing - continuous testing of an application as new
functionality is added; requires that various aspects of an application's functionality be
independent enough to work separately before all parts of the program are completed, or
that test drivers be developed as needed; done by programmers or by testers.
Integration testing - testing of combined parts of an application to determine if they
function together correctly. The 'parts' can be code modules, individual applications,
client and server applications on a network, etc. This type of testing is especially relevant
to client/server and distributed systems.
Functional testing - black-box type testing geared to functional requirements of an
application; this type of testing should be done by testers. This doesn't mean that the
programmers shouldn't check that their code works before releasing it (which of course
applies to any stage of testing.)
System testing - black-box type testing that is based on overall requirements
specifications; covers all combined parts of a system.
End-to-end testing - similar to system testing; the 'macro' end of the test scale; involves
testing of a complete application environment in a situation that mimics real-world use,
such as interacting with a database, using network communications, or interacting with
other hardware, applications, or systems if appropriate.
Sanity testing - typically an initial testing effort to determine if a new software version is
performing well enough to accept it for a major testing effort. For example, if the new
software is crashing systems every 5 minutes, bogging down systems to a crawl, or
destroying databases, the software may not be in a 'sane' enough condition to warrant
further testing in its current state.
Regression testing - re-testing after fixes or modifications of the software or its
environment. It can be difficult to determine how much re-testing is needed, especially
near the end of the development cycle. Automated testing tools can be especially useful
for this type of testing.
Acceptance testing - final testing based on specifications of the end-user or customer,
or based on use by end-users/customers over some limited period of time.
Load testing - testing an application under heavy loads, such as testing of a web
site under a range of loads to determine at what point the system's response time
degrades or fails.
Stress testing - term often used interchangeably with 'load' and 'performance'
testing. Also used to describe such tests as system functional testing while under
unusually heavy loads, heavy repetition of certain actions or inputs, input of large
numerical values, large complex queries to a database system, etc.
Performance testing - term often used interchangeably with 'stress' and 'load' testing.
Ideally 'performance' testing (and any other 'type' of testing) is defined in requirements
documentation or QA or Test Plans.
Usability testing - testing for 'user-friendliness'. Clearly this is subjective, and will depend
on the targeted end-user or customer. User interviews, surveys, video recording of user
sessions, and other techniques can be used. Programmers and testers are usually not
appropriate as usability testers.
Install/uninstall testing - testing of full, partial, or upgrade install/uninstall processes.
Recovery testing - testing how well a system recovers from crashes, hardware failures,
or other catastrophic problems.
Security testing - testing how well the system protects against unauthorized internal or
external access, willful damage, etc; may require sophisticated testing techniques.
Compatibility testing - testing how well software performs in a particular
hardware/software/operating system/network/etc. environment.
Exploratory testing - often taken to mean a creative, informal software test that is not
based on formal test plans or test cases; testers may be learning the software as they
test it.
Ad-hoc testing - similar to exploratory testing, but often taken to mean that the testers
have significant understanding of the software before testing it.
User acceptance testing - determining if software is satisfactory to an end-user or
customer.
Comparison testing - comparing software weaknesses and strengths to competing
products.
Alpha testing - testing of an application when development is nearing completion; minor
design changes may still be made as a result of such testing. Typically done by end-
users or others, not by programmers or testers.
Beta testing - testing when development and testing are essentially completed and final
bugs and problems need to be found before final release. Typically done by end-users or
others, not by programmers or testers.
Mutation testing - a method for determining if a set of test data or test cases is useful,
by deliberately introducing various code changes ('bugs') and retesting with the original
test data/cases to determine if the 'bugs' are detected. Proper implementation requires
large computational resources.

Some more questions


Q - What are test case formats widely use in web based testing?
A - Web based applications deal with live web portals. Hence the test cases can be
broadly classified as - front end , back end, security testing cases, navigation based,
field validations, database related cases. The test cases are written based on the
functional specifications and wire-frames.

Q - How to prepare test case and test description for job application?
A - Actually the question seems to be vague,... see naukri is one of biggest job site
globally and it has is own complex functionality normally a Test case is derived from a
SRS (or) FRS basically and test description is always derived from a Test case. Test
description is nothing but the steps which has to be followed for the TC what u wrote.
And the TC is nothing which compares the expectation and the actual(outcome)result.

Q - What is the difference between Functional and Technical bugs? Give an


example for each.?
Functional Bugs : Bugs found when Testing the Functionality of the AUT.
Technical bugs: Related to Communication which AUT makes.Like H/W,DB ! where
these could not be connected properly.

Q - Give proper Seq. to following testing Types Regression, Retesting, Funtional,


Sanity and Performance Testing.?
A - The proper sequence in which these types of testing are performed is - Sanity,
Functional, Regression, Retesting, Performance.

Q - How u test MS- Vista without any requirement Doc.?


Know what change is being made from the older verison of Windows to the newer
version with the help of User Release notes thats released with Windows Vista. Based
on that, formulate the test cases and execute the same.

Q - What is verification? validation?


Verification typically involves reviews and meetings to evaluate documents, plans, code,
requirements, and specifications. This can be done with checklists, issues lists,
walkthroughs, and inspection meetings. Validation typically involves actual testing and
takes place after verifications are completed. The term 'IV & V' refers to Independent
Verification and Validation.

Q - How can new Software QA processes be introduced in an existing


organization?

A lot depends on the size of the organization and the risks involved. For large
organizations with high-risk (in terms of lives or property) projects, serious management
buy-in is required and a formalized QA process is necessary.

Where the risk is lower, management and organizational buy-in and QA implementation
may be a slower, step-at-a-time process. QA processes should be balanced with
productivity so as to keep bureaucracy from getting out of hand.

For small groups or projects, a more ad-hoc process may be appropriate, depending on
the type of customers and projects. A lot will depend on team leads or managers,
feedback to developers, and ensuring adequate communications among customers,
managers, developers, and testers.

The most value for effort will often be in (a) requirements management processes, with a
goal of clear, complete, testable requirement specifications embodied in requirements or
design documentation, or in 'agile'-type environments extensive continuous coordination
with end-users, (b) design inspections and code inspections, and (c) post-
mortems/retrospectives.

Q - Why is it often hard for management to get serious about quality assurance?

Solving problems is a high-visibility process; preventing problems is low-visibility. This is


illustrated by an old parable: In ancient China there was a family of healers, one of
whom was known throughout the land and employed as a physician to a great lord.
Q - What's an 'inspection'?

An inspection is more formalized than a 'walkthrough', typically with 3-8 people including
a moderator, reader, and a recorder to take notes.
The subject of the inspection is typically a document such as a requirements spec or a
test plan, and the purpose is to find problems and see what's missing, not to fix anything.
Attendees should prepare for this type of meeting by reading thru the document; most
problems will be found during this preparation. The result of the inspection meeting
should be a written report.

Q - What is a 'walkthrough'?
A 'walkthrough' is an informal meeting for evaluation or informational purposes. Little or
no preparation is usually required.

Q - What makes a good test engineer?


A good test engineer has a 'test to break' attitude, an ability to take the point of view of
the customer, a strong desire for quality, and an attention to detail. Tact and diplomacy
are useful in maintaining a cooperative relationship with developers, and an ability to
communicate with both technical (developers) and non-technical (customers,
management) people is useful. Previous software development experience can be
helpful as it provides a deeper understanding of the software development process,
gives the tester an appreciation for the developers' point of view, and reduce the learning
curve in automated test tool programming. Judgment skills are needed to assess high-
risk areas of an application on which to focus testing efforts when time is limited.

Q - What makes a good Software QA engineer?


The same qualities a good tester has are useful for a QA engineer. Additionally, they
must be able to understand the entire software development process and how it can fit
into the business approach and goals of the organization. Communication skills and the
ability to understand various sides of issues are important. In organizations in the early
stages of implementing QA processes, patience and diplomacy are especially needed.
An ability to find problems as well as to see 'what's missing' is important for inspections
and reviews.
Q - What is agile testing?
Agile testing is used whenever customer requirements are changing dynamically
If we have no SRS, BRS but we have test cases does you execute the test cases blindly
or do you follow any other process.
Test case would have detail steps of what the application is supposed to do.
1) Functionality of application.
2) In addition you can refer to Backend, is mean look into the Database. To gain more
knowledge of the application.
2. How you will know when to stop testing?
a:Testing will be stopped when we came to know that there are only some
minnor bugs which may not effect the functionality of the application and
when all the test cases has benn executed sucessfully.

3. What are the metrics generally you use in testing?


A:These software metrics will be taken care by SQA team
Ex:rate of deffect efficiency

5. What is ECP and how you will prepare test cases?


A:It is a software testing related technique which is used for writing
test cases.it will break the range into some wqual partitions.the main
purpose of this tecnique is
1) To reduce the no. of test cases to a necessary minimun.
2) To select the right test cases to cover all the senarios.

6. Test Plan contents? Who will use this doc?


A:Test plan is a document which contains scope,risk analysis.
for every sucess there should be some plan,like that for geting some
quality product proper test plan should be there.

The test plan Contents are:


1) introduction
a) Overview
b) Achronyms
2) Risk Analysis
3) Test items
4) Features and funtions to be test
5) Features and funtions not to be test
6) Test statergy
7) test environment
8) system test shedule
9) test delivarables
10) Resources
11) re-sumptiom and suspension criteria
12) staff and training

8. What are Test case preparation guidelines?


A:Requirement specifications and User interface documents(sreen
shots of application)

10. How u will do usability testing explain with example?


A:Mainly to check the look and feel,ease of
use,gui(colours,fonts,allignment),help manuals and complete end to
end navigation.

11. What is Functionality testing?


A:Here in this testing mainly we will check the functionality of the
application whether its meet the customer requirements are not.
Ex:1+1 =2.

12. Which SDLC you are using?


A:V model

13. Explain V & V model?


A:Verification and Validation Model.

14. What are the acceptance criteria for your project?


A:It will be specified by the customer what is his acceptance
criteria.Ex:if so and so functionality has worked enough for me.
15. Who will provide the LOC to u?
A:Loc (lines of code) it will depend on the any company standards they
are following.

16. How u will report the bugs?


A:by using Bug tracking tool like Bugzilla,test director again it may
depends on company,some companies they may use own tool.

17. Explain your organizations testing process?


A: 1) SRS
2) Planning
3) Test senario design
4) Test case design
5) Execution
6) Bug Reporting
7) maintainance