Which Test Cases to Automate?

Tests that need to be run for every build of the application (sanity check, regression test)
Tests that use multiple data values for the same actions (data driven tests)
Stress/load testing

Which Test Cases Not to Automate?

Usability testing--"How easy is the application to use?"

One-time testing
"ASAP" testing--"We need to test NOW!"
Ad hoc/random testing--Based on intuition and knowledge of application

Why Use Testing Tools?

Manual Testing:

Time consuming
Low reliability
Human resources
Inconsistent

Automated Testing:

Speed
Repeatability
Programming capabilities
Coverage
Reliability
Reusability

What is Rational Robot…

Automation tool used for Functional/Regression Testing.

Developed by Rational Software, now acquired by IBM.
Test scripts can be created and run.
For Windows based applications.
Supports Web, Java, Oracle Forms, Power Builder, Delphi and VB applications, .NET
applications.
How Rational Robot works…

Records how you work on your application.

Generate a script in VB like language called SQA Basic.
Stores the object's properties while recording and generates scripts capturing the user
actions performed on those objects
Enhancement in script can be done manually and verification points can be inserted.
Script can be saved and replayed anytime.
During playback, the Robot compares the actual application status with the stored
expected results and decides if the test Passes or Fails.
The test results can be viewed as a log using TestManager.

Steps in automation of an application functionality:

Setup a project and configure it using Rational Administrator.

Generate scripts and enhance them using Rational Robot in accordance with the deduced
test scenarios.
Run/Playback the scripts using Rational Robot.
Analyze the results using log provided by the TestManager
Enhance the scripts if needed and log the defect.

setup a rational project:

Set up test framework and add / register a project using Rational Administrator.
Configure the Project
Create Test Data Store as test assets.
Set up user groups and users for the project.

Creating User Groups and Users

You can regulate access to the Rational Test datastore by using users and groups. You
can set the Rational Test datastore privileges for a test user to create, modify and delete:
--Test planning assets
--Test implementation assets
--Test execution assets
--Test results analysis assets

Create TestScripts:

Steps in generating a script

--Decide the functionality you want to test
--Decide the verification point which you want in your test script
--Setting Record Options
--Record the scripts
--Customize the script if required

Setting Recording Options…

GUI recording options provide instructions to Robot about how to record and generate
GUI scripts.
Recommended to set these options before you begin recording.

To set GUI recording options:

Click Tools --> GUI Record Options.

What can be set ?

Name scripts automatically

Control how Robot responds to unknown object
Select an object order preference
Record Think Time

Record/Create Test Scripts

Start Rational Robot by choosing:

Start->Programs->Rational Software->Rational Robot

Log into a Rational Administrator Project to continue.

Click the Record GUI Script button on the toolbar.

Record/Create Test Scripts

On the screen that pops up, enter a name of the script (TestScript) and click OK.

By default, Robot minimizes its main window so you can work with the application-
under-test.

When in Record Mode, the toolbar will be on top and will record all key strokes and
mouse clicks. If you want to do something that you don’t want to record, remember to
pause first.

Click the Display GUI Insert Toolbar button on the GUI Record toolbar.
Start the application or other executable program to be tested by using one of the Start
buttons on the GUI Insert toolbar.
Start Application .
Start Java Application .
Start Browser.

Record/Create Test Scripts

To insert verification points, on the GUI Record Toolbar, click the “Display GUI Insert”
again. There are many buttons to set VPs.

Here are some of the available VP types and their buttons:

Alphanumeric - Verifies alphanumeric data. Used for edit boxes, pushbuttons.

Clipboard-Verifies the contents of Windows Clipboard.
Menu-Verifies the menu values and optionally their .
Object Data-Tests data content of objects.
Object Properties-Tests object attributes such as color, font, position.
Region Image-Graphically compares an area of the screen you specify.
Window Existence-Tests to see if a particular window does or does not exist.
Window Image-Graphically compares an entire window.

Record/Create Test Scripts:

Lets verify the data entered in the edit box (Emp#).

Click the Object Data VP button on the toolbar. Enter the name of verification point.
Click OK.

Drag and drop the “Object Finder Tool” over the object.

The window that appears shows you what data can be verified with this VP. Select
“contents” and Click OK.

You will now have the scripts ready with verification point in it.

Recording Modes:
Rational Robot has two recording modes

Object-Oriented Recording mode – It is the default method. Robot uses internal object
names to identify objects, instead of using mouse movements or absolute screen
coordinates.

Low-level recording mode – Records detailed mouse movements and keyboard actions
by screen coordinates and exact timing. Use low-level recording when you are testing
functionality that requires the tracking of detailed mouse actions ,such as in painting,
drawing, or CAD applications.

To switch between the two modes during recording, do one of the following:
--Press CTRL+SHIFT+R.
--Click the Open Robot Window button on the GUI Record toolbar (or press
CTRL+SHIFT+F) to bring Robot to the foreground. Click Record > Turn Low-Level
Recording On/Off.

Robot Inspector:

The inspector in Robot helps in identifying the objects and their properties present in the
screens of the AUT.

Clicking Tools -> Inspector, opens the Inspector window and all the properties of all the
windows open on the desktop are captured by it.

By pointing and clicking the icon of the Inspector on the desired object, all the child
properties of the object are captured with all the properties of the object, even hidden
ones.

It is not integrated with the Robot such that the properties which are captured, appear in
the Robot script automatically.

It is analogous to the GUI spy in WinRunner.

Verification Points:

Different verification points in Rational Robot are

Object Properties

Object data
Alphanumeric

Menu

Clipboard

Region Image

Window Existence

Window Image

File Comparison

File Existence

Web Site Scan

Web Site Compare

Inserting Verification points:

Click the Display GUI Insert Toolbar button on the GUI Record toolbar.

Click a verification point button on the GUI Insert toolbar.

Edit the name as appropriate(20 Characters maximum).

Optionally, set the Wait state.

Retry every

Timeout after

Set the Expected result.

Verification Point - Object Properties:

Captures and compares the properties of objects.

Robot displays a list of the captured objects and their corresponding properties.
Select the Objects and properties which you want to test.
Verification Point - Object data:

Captures and compares the data in objects.

Verification Points - Alphanumeric:

Captures and compares alphabetic or numeric values.

Can be used for Windows objects that Robot recognizes. These include: Checkbox,
Generic, Group Box, Label, Pushbutton, Radio Button, Toolbar, Window (caption).

Verification Points - Menu:

Captures and compares the menu title, menu items, shortcut keys, and the state of
selected menus.
Robot records information about the top menu and up to five levels of sub-menus.

Verification Points - Clipboard:

Captures and compares alphanumeric data that has been copied to the Clipboard.
To use this verification point, the application must supply a Copy or Cut capability so
that you can place the data on the Clipboard.

Verification Point – Region Image:

Captures and compares a region of the screen (as a bitmap).

For the verification point to pass, the location of the region and the screen resolution
should be the same during record and playback.

Masks and OCR regions can be created in this verification point.

Verification Point - Region Image (Masks):

Masks are used to "hide" the underlying masked area from comparison when scripts are
played back.
Use masks to ensure that certain regions will not be tested.
Application has a date field, user might want to mask it so that it won't produce a failure
every time the script is played back.

Verification Point - Region Image (OCR):

Robot uses Optical Character Recognition (OCR) regions to read the text within a
designated region and compare it in subsequent playbacks of the script.

Use OCR regions to verify proper operation of an application that

dynamically paints text in window areas or where the actual text is difficult to obtain.

Verification Point - Window Image:

Similar to Region Image verification point, but the user needs to point to the particular
window instead of specifying the co-ordinates of the image.

Masks and OCR regions can also be specified here

Note : Use Region Image and Window Image verification points only when verification
cannot be accomplished using object or window properties.

Verification Point - Window Existence:

Checks that the specified window is displayed before continuing with the playback.
The status can be normal, minimized, maximized, or hidden.

Verification Point – File comparison and Existence:

File Comparison
Compares two specified files during playback. The comparison is based on the contents
of the files and their sizes, not on the file names or dates.

File Existence
Verifies the existence of a specified file during playback. When you create the
verification point, you specify the drive, directory, and file names.

Verification Point - Web Site Scan:

Web Site Scan

Checks the contents of a website with every revision and ensures that changes have not
resulted in defects. You can check for the following:
– Internal and external links not found
– Pages with broken links or pages that have moved
– Orphans, viruses, and assertions
– ActiveScan entries
– Slowly downloading pages

Verification Point - Web Site Scan

When you record, you use Robot to scan your Web site through SiteCheck, which is the
Robot component for managing your intranet or World Wide Web site.

During playback, you can identify changes or unintentional errors in your Web site as it
evolves. This lets you correct errors before your site becomes public.

When you play back a Web Site Scan verification point, SiteCheck scans the site based
on the options that you selected when you recorded the verification point. If it finds any
defects, the verification point fails.

After you play back a Web Site Scan verification point, you can view the playback results
in the TestManager log.

Verification Point - Web site Compare:

Web Site Compare

Compares a baseline of a web site and compares it to the web site at another point in
time.

The SiteCheck Site Comparison Report details the difference between two sites and
provides basic site statistics for each site. For example, if you develop a Web site locally
and then move changes to your live Web site, you can compare the local version with the
live Web site to check for defects.

When you play back a Web Site Compare verification point, SiteCheck starts and
compares the baseline you selected with another Web site you selected when you
recorded the verification point. If it finds any defects, the verification point fails.

After you play back a Web Site Compare verification point, you can view the playback
results in the TestManager log.
Enhance the script:

Shell Scripting

Shell Scripting is used to group scripts into a single script. Shell Script plays back other
scripts in sequence.
The results from all scripts are stored in the same log, which simplifies results analysis.
To create a shell script, Click File > New > GUI Shell Script ,name the script and select
the scripts you want to add to the shell script.
The shell script contains a CallScript command followed by the name of each script that
you included.

Using Object Scripting commands to access objects and properties from within a script.
These commands let you work with an object’s properties. The Object Scripting
commands can only be used programmatically. Robot does not generate these commands
during recording.

SQAFindObject - Search for a specified object.

SQAGetChildren - Retrieve an array containing recognition methods that identify each of

an object’s child objects.

SQAGetProperty - Retrieve the value of the specified property.

SQAGetPropertyArray - Retrieve an array of values for the specified property.

SQAGetPropertyArrayAsString - Retrieve the string equivalent of an array of values for

the specified property.

Enhance the script:

SQAGetPropertyArraySize - Retrieve the number of elements in an array of property

values.

SQAGetPropertyAsString - Retrieve a property value in String form.

SQAGetPropertyNames - Retrieve an array containing the names of all the object’s

properties.

SQAInvokeMethod - Execute the specified method of an object.

SQASetProperty - Assign a value to a specified property.

SQAWaitForObject - Pause execution of the script until the specified object can be
found.

SQAWaitForPropertyValue - Pause execution of the script until a property is set to the

specified value.

Database connectivity and retrieval

These commands let you access information in a database from Robot script.

SQLClose - Close a data source connection.

SQLError - Return a detailed error message for ODBC functions.

SQLExecQuery - Execute an SQL statement.

SQLGetSchema - Obtain information about data sources, databases, terminology, users,

owners, tables, and columns.

SQLOpen - Connect to a data source for use by other functions.

SQLRequest - Make a connection to a data source, execute an SQLstatement, and return

the results.

SQLRetrieve - Return the results of a select that was executed by SQLExecQuery into a
user-provided array.

SQLRetrieveToFile - Return the results of a select that was executed by SQLExecQuery

into a user-specified file.

Enhance the script:

Inserting Log Messages

Inserting Comments
Inserting Delay Values
Using data driven approach supported by DATAPOOL
Inserting Custom Verification Points
Display messages in Console Window / Logs
Data Driven - Datapool:

Datapool:

Facilitates running the same test with different data and avoid hard coding in the script.

Can generate random data also.

Steps to use datapool

- Create a Datapool using Test Manager.

- Open the Datapool.
- Fetch the Datapool values.
- Assign the values to variables declared within the script.
- Close the Datapool.

Datapool…:

These commands let you access data in a datapool.

SQADatapoolOpen - Open the specified datapool.

syntax: return& = SQADatapoolOpen ("name$", [wrap], [sequence], [exclusive])
name$ - The name of the datapool to open.
wrap - Optional. Indicates whether the datapool cursor should return to the first row in
the row access order after the last row has been reached.
(True, False)
sequence - Optional. Determines row access order(SQA_DP_SEQUENTIAL,
SQA_DP_RANDOM, SQA_DP_SHUFFLE)
exclusive - Optional. Indicates whether the datapool cursor is shared with other users or
is exclusive for an individual user. (True, False)
SQADatapoolFetch - Move the cursor for the datapool to the next row.
syntax: return& = SQADatapoolFetch (datapool_id&)
datapool_id& - An ID returned by SQADatapoolOpen that represents an open datapool.

SQADatapoolClose - Close the specified datapool.

syntax: return& = SQADatapoolClose (datapool_id&)
datapool_id& - An ID returned by SQADatapoolOpen that represents an open datapool.
SQADatapoolRewind - Reset the cursor for the specified datapool.
syntax: return& = SQADatapoolRewind (datapool_id&)
datapool_id& - An ID returned by SQADatapoolOpen that represents an open datapool.
SQADatapoolValue - Retrieve the value of the specified datapool column.
syntax: return& = SQADatapoolValue (datapool_id&, column, value$) datapool_id& -
An ID returned by SQADatapoolOpen that represents an open datapool.
column - Specifies the name or ID of the datapool column to retrieve.
value$ - Contains the value from the datapool column upon successful return.

These commands require that you include the header file SQAUTIL.SBH with the
SQABasic command '$Include.

Example:This example opens a datapool named repo_dp1, moves the cursor to the next
row, retrieves the value from column 1, and then closes the datapool.

'$Include "sqautil.sbh"
DIM dp_id as Long
DIM dp_Result as Long
dp_id = SQADatapoolOpen ("repo_dp1", FALSE, SQA_DP_SEQUENTIAL,FALSE)
dp_Result = SQADatapoolFetch (dp_id)
dp_Result = SQADatapoolValue (dp_id, 1, dp_Value)
dp_Result = SQADatapoolClose (dp_id)

Debugging the Script…:

Robot includes a complete, built-in debugging environment to assist you during the
development phase of your GUI scripts
Breakpoints
Step Commands
Step (Go)
Step Into
Animate

Break Point : Stops a test at a specified line number in a test script

Animate: Use to play back the active script, displaying a yellow arrow in the left margin
of each line (or highlighting the line) as it executes.

View Variables: Use the Variables window to examine variable and constant values as
you play back GUI scripts during debugging.

Playing Back GUI Script…:

Restoring the Test Environment Before Playback

Setting GUI Playback Options

Playing Back a GUI Script

Analyze test results:

Rational Robot stores an expected result with the recorded test script

Rational Robot captures an actual result during playback

Rational Robot compares the actual result to the stored, expected result

Rational Robot reports the comparison output as PASS or FAIL in Rational Test
manager.

Click View -> Verification Point to analyze the failure of verification point in a
Comparator

You can use Rational ClearQuest with TestManager to track and manage defects and
change requests. With TestManager, you can submit

defects directly from a test log in ClearQuest.

Performance testing using Rational Robot:

Rational Robot can also be used for load and performance testing.

Click the Record Session (VU) button on the toolbar

Robot records all of a client's requests to the server and all of the server's responses from
the time you begin recording until the
time you stop recording

Use Rational TestManager to playback Virtual User (VU) script commands to determine
an application's performance speed and to detect

contention problems caused by multiple users performing actions simultaneously.

Advantages in using Rational Robot:

Improve the scope and frequency of regression testing.

Plan, track and execute regression testing through IBM Rational TestManager.
The Basic-based language is much more intuitive-whether someone has programming
language experience or not.

Supports testing applications built in a wide range of environments and languages,

including HTML and DHTML, Microsoft Visual
Studio .NET, Visual Basic and Visual C++ development systems, Java, Oracle
Developer/2000, PeopleSoft, Sybase PowerBuilder and Borland Delphi.

Limitations in using Rational Robot:

Image (picture) dependent tests – Robot cant capture a live image.

No GUI map – Test scripts (object names) need to be updated manually for every new
build.

External functions – Libraries need to be created first to develop external functions.

Result Log file – As default generated Result Log File output is not easy-to-read, after
recording, log file messages need to be

inserted into the script for proper analysis of the outcome.

Character based recording not supported (Eg: Extra PC Client)

Does not have a built in exception handling mechanism

Advanced script – After recording Robot script needs to be modified before playback