Bug report

Below sample bug/defect report will give you exact idea of how to report a bug in bug tracking tool. Here is the example scenario that caused a bug: Lets assume in your application under test you want to create a new user with user information, for that you need to logon into the application and navigate to USERS menu > New User, then enter all the details in the User form like, First Name, Last Name, Age, Address, Phone etc. Once you enter all these information, you need to click on SAVE button in order to save the user. Now you can see a success message saying, New User has been created successfully. But when you entered into your application by logging in and navigated to USERS menu > New user, entered all the required information to create new user and clicked on SAVE button. BANG! The application crashed and you got one error page on screen. (Capture this error message window and save as a Microsoft paint file) Now this is the bug scenario and you would like to report this as a BUG in your bug-tracking tool. How will you report this bug effectively? Here is the sample bug report for above mentioned example: (Note that some bug report fields might differ depending on your bug tracking system) SAMPLE BUG REPORT: Bug Name: Application crash on clicking the SAVE button while creating a new user. Bug ID: (It will be automatically created by the BUG Tracking tool once you save this bug) Area Path: USERS menu > New Users Build Number: Version Number 5.0.1 Severity: HIGH (High/Medium/Low) or 1 Priority: HIGH (High/Medium/Low) or 1 Assigned to: Developer-X Reported By: Your Name Reported On: Date Reason: Defect Status: New/Open/Active (Depends on the Tool you are using) Environment: Windows 2003/SQL Server 2005 Description: Application crash on clicking the SAVE button while creating a new user, hence unable to create a new user in the application. Steps To Reproduce: 1) Logon into the application 2) Navigate to the Users Menu > New User 3) Filled all the user information fields 4) Clicked on Save button 5) Seen an error page ORA1090 Exception: Insert values Error 6) See the attached logs for more information (Attach more logs related to bug..IF any) 7) And also see the attached screenshot of the error page. Expected result: On clicking SAVE button, should be prompted to a success message New User has been created successfully. (Attach application crash screen shot.. IF any)

Save the defect/bug in the BUG TRACKING TOOL. You will get a bug id, which you can use for further bug reference. Default New bug mail will go to respective developer and the default module owner (Team leader or manager) for further action.

Bug status description: These are various stages of bug life cycle. The status caption may vary depending on the bug tracking system you are using. 1) New: When QA files new bug. 2) Deferred: If the bug is not related to current build or can not be fixed in this release or bug is not important to fix immediately then the project manager can set the bug status as deferred. 3) Assigned: Assigned to field is set by project lead or manager and assigns bug to developer. 4) Resolved/Fixed: When developer makes necessary code changes and verifies the changes then he/she can make bug status as Fixed and the bug is passed to testing team.

5) Could not reproduce: If developer is not able to reproduce the bug by the steps given in bug report by QA then developer can mark the bug as CNR. QA needs action to check if bug is reproduced and can assign to developer with detailed reproducing steps. 6) Need more information: If developer is not clear about the bug reproduce steps provided by QA to reproduce the bug, then he/she can mark it as Need more information. In this case QA needs to add detailed reproducing steps and assign bug back to dev for fix. 7) Reopen: If QA is not satisfy with the fix and if bug is still reproducible even after fix then QA can mark it as Reopen so that developer can take appropriate action. 8 ) Closed: If bug is verified by the QA team and if the fix is ok and problem is solved then QA can mark bug as Closed. 9) Rejected/Invalid: Some times developer or team lead can mark the bug as Rejected or invalid if the system is working according to specifications and bug is just due to some misinterpretation. Choosing Test Data
In system testing, test data should cover the possible values of each parameter based on the the requirements. Since testing every value is impractical, a few values should be chosen from each equivalence class. An equivalence class is a set of values that should all be treated the same. Ideally, test cases that check error conditions are written separately from the functional test cases and should have steps to verify the error messages and logs. Realistically, if error test cases are not yet written, it is OK for testers to check for error conditions when performing normal functional test cases. It should be clear which test data, if any, is expected to trigger errors. Example equivalence classes: Strings

empty string String consisting solely of white space String with leading or trailing white space syntactically legal: short and long values syntactically legal: semantically legal and illegal values syntactically illegal value: illegal characters or combinations Make sure to test special characters such as #, ", ', &, and < Make sure to test "Foreign" characters typed on international keyboards

Numbers

empty string, if possible 0 in range positive, small and large in range negative, small and large out of range positive out of range negative with leading zeros syntactically invalid (e.g., includes letters)

Identifiers

empty string syntactically legal value syntactically legal: reference to existing ID, invalid reference syntactically illegal value

Radio buttons

one item checked nothing checked, if possible

Checkboxes

checked unchecked

Drop down menus

select each item in turn

Scrolling Lists

select select select select

no item, if possible each item in turn combinations of items, if possible all items, if possible

File upload

blank 0 byte file long file short file name long file name syntactically illegal file name, if possible (e.g., "File With Spaces.tar.gz")

How to write test cases? Here is a simple test case format Fields in test cases: Test case id: Unit to test: What to be verified? Assumptions: Test data: Variables and their values Steps to be executed: Expected result: Actual result: Pass/Fail: Comments:

So here is a basic format of test case statement: Verify Using [tool name, tag name, dialog, etc] With [conditions] To [what is returned, shown, demonstrated]

Verify: Used as the first word of the test case statement. Using: To identify what is being tested. You can use entering or selecting here instead of using depending on the situation.

For any application basically you will cover all the types of test cases including functional, negative and boundary value test cases. Keep in mind while writing test cases that all your test cases should be simple and easy to understand. Dont write explanations like essays. Be to the point. Try writing the simple test cases as mentioned in above test case format. Generally I use Excel sheets to write the basic test cases. Use any tool like Test Director when you are going to automate those test cases.

Feel free to comment below any query regarding test case writing or execution.

How to Report a Bug? Use following simple Bug report template: This is a simple bug report format. It may vary on the bug report tool you are using. If you are writing bug report manually then some fields need to specifically mention like Bug number which should be assigned manually. Reporter: Your name and email address. Product: In which product you found this bug. Version: The product version if any. Component: These are the major sub modules of the product. Platform: Mention the hardware platform where you found this bug. The various platforms like PC, MAC, HP, Sun etc. Operating system: Mention all operating systems where you found the bug. Operating systems like Windows, Linux, Unix, SunOS, Mac OS. Mention the different OS versions also if applicable like Windows NT, Windows 2000, Windows XP etc. Priority: When bug should be fixed? Priority is generally set from P1 to P5. P1 as fix the bug with highest priority and P5 as Fix when time permits. Severity: This describes the impact of the bug. Types of Severity:

Status:

Blocker: No further testing work can be done. Critical: Application crash, Loss of data. Major: Major loss of function. Minor: minor loss of function. Trivial: Some UI enhancements. Enhancement: Request for new feature or some enhancement in existing one.

When you are logging the bug in any bug tracking system then by default the bug status is New. Later on bug goes through various stages like Fixed, Verified, Reopen, Wont Fix etc. Click here to read more about detail bug life cycle. Assign To: If you know which developer is responsible for that particular module in which bug occurred, then you can specify email address of that developer. Else keep it blank this will assign bug to module owner or Manger will assign bug to developer. Possibly add the manager email address in CC list. URL: The page url on which bug occurred.

Summary: A brief summary of the bug mostly in 60 or below words. Make sure your summary is reflecting what the problem is and where it is. Description: A detailed description of bug. Use following fields for description field:

Reproduce steps: Clearly mention the steps to reproduce the bug. Expected result: How application should behave on above mentioned steps. Actual result: What is the actual result on running above steps i.e. the bug behavior.

These are the important steps in bug report. You can also add the Report type as one more field which will describe the bug type. The report types are typically: 1) Coding error 2) Design error 3) New suggestion 4) Documentation issue 5) Hardware problem Some Bonus tips to write a good bug report: 1) Report the problem immediately:If you found any bug while testing, do not wait to write detail bug report later. Instead write the bug report immediately. This will ensure a good and reproducible bug report. If you decide to write the bug report later on then chances are high to miss the important steps in your report. 2) Reproduce the bug three times before writing bug report:Your bug should be reproducible. Make sure your steps are robust enough to reproduce the bug without any ambiguity. If your bug is not reproducible every time you can still file a bug mentioning the periodic nature of the bug. 3) Test the same bug occurrence on other similar module: Sometimes developer use same code for different similar modules. So chances are high that bug in one module can occur in other similar modules as well. Even you can try to find more severe version of the bug you found. 4) Write a good bug summary: Bug summary will help developers to quickly analyze the bug nature. Poor quality report will unnecessarily increase the development and testing time. Communicate well through your bug report summary. Keep in mind bug summary is used as a reference to search the bug in bug inventory. 5) Read bug report before hitting Submit button: Read all sentences, wording, steps used in bug report. See if any sentence is creating ambiguity that can lead to misinterpretation. Misleading words or sentences should be avoided in order to have a clear bug report. 6) Do not use Abusive language: Its nice that you did a good work and found a bug but do not use this credit for criticizing developer or to attack any individual.

How to write software Testing Weekly Status Report

Posted In | Bug Defect tracking, Software Testing Templates, Testing Tips and resources

Writing effective status report is as important as the actual work you did! How to write a effective status report of your weekly workat the end of each week?

Here I am going to give some tips. Weekly report is important to track the important project issues, accomplishments of the projects, pending work and milestone analysis. Even using these reports you can track the team performance to some extent. From this report prepare future actionables items according to the priorities and make the list of next weeks actionable. So how to write weekly status report? Follow the below template: Prepared By: Project: Date of preparation: Status: A) Issues: Issues holding the QA team from delivering on schedule: Project: Issue description: Possible solution: Issue resolution date: You can mark these issues in red colour. These are the issues that requires managements help in resolving them. Issues that management should be aware: These are the issues that not hold the QA team from delivering on time but management should be aware of them. Mark these issues inYellow colour. You can use above same template to report them. Project accomplishments: Mark them in Green colour. Use below template. Project: Accomplishment: Accomplishment date: B) Next week Priorities: Actionable items next week list them in two categories: 1) Pending deliverables: Mark them in blue colour: These are previous weeks deliverables which should get released as soon as possible in this week. Project: Work update: Scheduled date: Reason for extending:

2) New tasks: List all next weeks new task here. You can use black colour for this. Project: Scheduled Task: Date of release: C) Defect status: Active defects: List all active defects here with Reporter, Module, Severity, priority, assigned to. Closed Defects: List all closed defects with Reporter, Module, Severity, priority, assigned to. Test cases: List total number of test cases wrote, test cases passed, test cases failed, test cases to be executed. This template should give you the overall idea of the status report. Dont ignore the status report. Even if your managers are not forcing you to write these reports they are most important for your work assessment in future.

Try to follow report writing routine. Use this template or at least try to report it in your own words about the overall work of which you can keep some track

Difference between Desktop, Client server and Web testing

Posted In | Basics of Software testing, Testing Interview questions, Web Testing

In this post I am gonna give answer to readers question. There was one question asked by Srividya: Q.What is difference between client server and Web Testing? Vijay: Well Srividya I would like to add one more testing type i.e Desktop Testing in this discussion. So now we have three testing types Desktop application testing, Client server application testing and Web application testing. Each one differs in the environment in which they are tested and you will lose control over the environment in which application you are testing, while you move from desktop to web applications.

Desktop application runs on personal computers and work stations, so when you test the desktop application you are focusing on a specific environment. You will test complete application broadly in categories like GUI, functionality, Load, and backend i.e DB.

In client server application you have two different components to test. Application is loaded on server machine while the application exe on every client machine. You will test broadly in categories like, GUI on both sides, functionality, Load, client-server interaction, backend. This environment is mostly used in Intranet networks. You are aware of number of clients and servers and their locations in the test scenario. Web application is a bit different and complex to test as tester dont have that much control over the application. Application is loaded on the server whose location may or may not be known and no exe is installed on the client machine, you have to test it on different web browsers. Web applications are supposed to be tested on different browsers and OS platforms so broadly Web application is tested mainly for browser compatibility and operating system compatibility, error handling, static pages, backend testing and load testing. I think this will have an idea of all three testing environment. Keep in mind that even the difference exist in these three environment, thebasic quality assurance and testing principles remains same and applies to all.

Bug Priority and Severity

Differentiate Priority and Severity. The effect of a bug on the software does not automatically correlate with the priority for fixing it. A severe bug that crashes the software only once in a blue moon for 1% of the users is lower priority than a mishandled error condition resulting in the need to re-enter a portion of the input for every user every time. Therefore: Track priority and severity separately, then triage appropriately. It helps to have input from others on the team on priority. The importance of a bug is a project decision, different from the bug's perception by the Customer. In some cases it makes sense to track Urgency, the customer's point of view, separately. Microsoft uses a four-point scale to describe severity of bugs. Severity 1 is a crash or anything that loses persistent data , i.e., messing up your files on disk. Sev 2 is a feature that doesn't work. Sev 3 is an aspect of a feature that doesn't work. Sev 4 is for purely cosmetic problems, misspellings in dialogs, redraw issues, etc. This system works very well. (Interestingly, sev 4 bugs end up getting set to priority 1 fairly often, because they are frequently very annoying to users, and fixing them is generally easy and doesn't destabilize things.) Keep clear the difference between severity and priority". The example given says "a

start-up splash screen with your company logo backwards and the name misspelled is purely a cosmetic problem. However, most companies would treat it as a highpriority bug." Bug Triage Meeting Severity & Priority

"Triage" is a medical term. It refers to dividing wounded or sick people into three categories: those who will die no matter what you do, those who will recover even if unaided, and those who will recover only if aided. In a situation where there's too much to do, you must concentrate on the third group.

Bug Triage Meetings (sometimes called Bug Councils) are project meetings in which open bugs are divided into categories. The most important distinction is between bugs that will not be fixed in this release and those that will be

There are three categories for the medical usage, software also three categories - bugs to fix now, bugs to fix later, and bugs we'll never fix

Triaging a bug involves:

Making sure the bug has enough information for the developers and makes sense Making sure the bug is filed in the correct place Making sure the bug has sensible "Severity" and "Priority" fields

Let us see what Priority and Severity means

Priority is Business; Severity is Technical

In Triages, team will give the Priority of the fix based on the business perspective. They will check How important is it to the business that we fix the bug? In most of the times high Severity bug is becomes high Priority bug, but it is not always. There are some cases where high Severity bugs will be low Priority and low Severity bugs will be high Priority.

In most of the projects I worked, if schedule drawn closer to the release, even if the bug severity is more based on technical perspective, the Priority is given as low because the functionality mentioned in the bug is not critical to business.

Priority and Severity gives the excellent metrics to identify overall health of the Project. Severity is customer-focused while priority is business-focused. Assigning Severity for a bug is straightforward. Using some general guidelines about the project, testers will assign Severity but while assigning a priority is much more juggling act. Severity of the bug is one of the factors for assigning priority for a bug. Other considerations are might be how much time left for schedule, possibly who is available for fix, how important is it to the business to fix the bug, what is the impact of the bug, what are the probability of occurrence and degree of side effects are to be considered.

Read the excellent article Arguing Apples and Oranges This article clearly explains the how Priority and Severity of the bug given. Some of the above points taken from this article

Many organizations mandate that bugs of certain severity should be at least certain priority. Example: Crashes must be P1; Data loss must be P1, etc. A severe bug that crashes the system only once and not always reproducible will not be P1, where as an error condition that results re-entry a portion of input for every user will be P1

Microsoft uses a four-point scale to describe severity of bugs and three-point scale for Priority of the bug. They are as follows

Severity ---------------

1. Bug causes system crash or data loss. 2. Bug causes major functionality or other severe problems; product crashes in obscure cases. 3. Bug causes minor functionality problems, may affect "fit anf finish". 4. Bug contains typos, unclear wording or error messages in low visibility fields.

Priority --------------1. Must fix as soon as possible. Bug is blocking further progress in this area. 2. Should fix soon, before product release. 3. Fix if time; somewhat trivial. May be postponed.