INTRODUCTION AUTOMATED UI TESTING

UI Testing with Visual Studio 2010

PUBLISHED BY Enovation Press, Corp. A Division of Software Solutions, Corp PO Box 46462 Madison, WI 53744 Copyright @2010 by Rigoberto Garcia All rights reserved. No part of the contents of this book may be reproduced or transmitted in any form or by any means without written expressed permission of the publisher.

Printed and bound in the United States of North America

This book expresses the authors views and opinions. The information contained in this book is provided without any express, statutory, or implied warranties. Neither the author, Software Solutions Corporation, Enovation Press Corporation, resellers, distributors nor partners will be held liable for any damages caused or alleged to be caused directly or indirectly by this book. Copyright 2010 by Enovation Press (Publisher); All rights reserved, Printed in the United States of America. Except as permitted under the Copyright Act of 1976, no part of this publication may be reproduced or distributed in any form or by any means, or stored in a database or retrieval system, without the prior written permission of the Publisher.

First Edition Paperback FIRST DRAFT ONLY

978-1-105-99506-4
Information has been obtained by Publisher from sources believed to be reliable. However, because of the possibility of human or mechanical errors by our sources, Publisher, or others, do not guarantee the accuracy, or completeness of any information included in this work and are not responsible for any errors or omissions or the results obtained from the use of such information Enovation Press, Inc. does not make any representations or warranties as to the accuracy, adequacy, or completeness of any information contained in this Work, and is not responsible for any errors or omissions.

About This Book

The idea behind this book is simple. I want you the reader/student to be able to see technology solutions, not technology systems. At times we get so focus on the mechanics of an idea that we tend to forget or ignore the big picture; what I call the business driven force or spirit of any technology based enterprise solution. Therefore, my strategy for this book is to provide you with the project management skills to be able to manage the creative process. We will first begin with an important Subject, Managing Innovation and Creativity. This book is part of a series of books that will take you from the management of a project to development of a solutions and finally thru the deployment and maintenance phases. Why do I begin our book with this topic? For the reason that businesses are constantly looking for innovative ideas that will put them one step ahead of their competitors it has become important to look far into the future in order to stay Ahead of the Curve. In other words in order to arrive at a truly new idea, a manager must understand how to manage the innovation process. This book will offer effective managerial strategies that address the organizational aspects of managing a creative team, supporting that creativity, moving an idea to implementation, and troubleshooting ideas to help guide a team through the innovation process. Managing innovation will not only make you a more valuable manager but will also help successfully navigate your team to implementing the new innovation. One clarification, most of us associate innovation with new product development or deployment. However, I challenge you to think of your new SQL Server or Server Framework deployment as part of an innovation process. A manager needs to understand the creative process and effectively allow for creativity to happen if he wants his team to create something that will set the company apart from the competition

How to Use This Book

Each chapter covers one broad topic or set of related topics. Each chapter is mean to give you a foundation on the technology your interested on deploying, in this manner I help you create a strong foundation in which you can based deployment decision on. The chapters are arranged in order of increasing proficiency; skills you acquire in one chapter are used and developed in

subsequent chapters. For this reason, you should work through the chapters in sequence, although the information on each chapter could be used stand alone also. I have organized each chapter into results-oriented topics. Topics include all the relevant and supporting information you need to master Automated UI Testing Using Visual Studio 2010, and System Center 2012 Virtual Machine Manager. All Content is based on framework 3.5 or Framework 4.0 and it cover heavily virtualization technology and SharePoint 2010 UI Testing and Server Core (Longhorn) technology. Activities or Labs found in this book, will allow you to apply this information to practical hands-on examples. At times I will use several business scenarios. These business scenarios immerse you in a comfortable environment where you are presented with situations that require you to make real world decisions based on the principles you have learned in the course of this book. Within these simulations, you will meet characters, have access to clear and timely feedback, and have the ability to review and change previous decisions. These business simulations are available on the support website at http://northamerica.softwaresolutioncorp.com/kb and can be downloaded from any computer, tablet or smart device. The course manual can be also found at the site and PowerPoint presentations. .

As a Review Tool
Any method of instruction is only as effective as the time and effort you are willing to investing it. In addition, some of the information that you learn in class may not be important to you immediately, but it may become important later on. For this reason, we encourage you to spend some time reviewing the topics and activities after the course. For an additional challenge when reviewing activities, try the Why To Do It column before looking at the How To Do It column.

As a Reference
The organization and layout of the book make it easy to use as a learning tool and as an after class reference. You can use this book as a first source for definitions of terms, background information on given topics, and summaries of procedures.

Table of Contents
Chapter 1....................................................................................................................................... 17 Why Automated Testing? ............................................................................................................. 17 Introduction............................................................................................................................... 18 Manual software testing ........................................................................................................... 18 Automated software testing tool .............................................................................................. 18 Improves Accuracy................................................................................................................. 19 Increases Test Coverage ........................................................................................................ 19 Does What Manual Testing Cannot ....................................................................................... 19 Helps Developers and Testers ............................................................................................... 19 Improves Team Morale ......................................................................................................... 19 UI Automation for Testing ......................................................................................................... 20 Implementing UI Automation................................................................................................ 21 UI Automation in a Client .......................................................................................................... 21 Programmatic Access ........................................................................................................ 22 Key Properties for Test Automation ................................................................................. 22 AutomationID.................................................................................................................... 22 ControlType....................................................................................................................... 23 NameProperty................................................................................................................... 23 Implementing UI Automation in a Test Application ......................................................... 23 Chapter 2....................................................................................................................................... 25 Application Lifecycle Management............................................................................................... 25 Scenario #1:....................................................................................................................... 27 The Client Tier ........................................................................................................................ 28 The Application Tier ............................................................................................................... 28 The Data Tier ......................................................................................................................... 29 Automatic Compiling ............................................................................................................. 29 Application Lifecycle Management ....................................................................................... 30 Microsoft Test Manager: .................................................................................................. 30

Visual Studio Premium: ..................................................................................................... 30 Visual Studio Ultimate ...................................................................................................... 30 Overview of Virtual Lab for Application Lifecycle Management .............................................. 32 What are Virtual Environments ............................................................................................. 34 Simple Virtual Environments ................................................................................................. 35 Complex Virtual Environments .............................................................................................. 36 Targeted Virtual Environments ............................................................................................. 36 Virtual Environments ................................................................................................................ 37 Virtual machines (Hyper-V or VMware) ................................................................................ 38 Virtual machine ................................................................................................................. 38 Host ................................................................................................................................... 38 Snapshot ........................................................................................................................... 39 System Center Virtual Machine Manager (SCVMM) ............................................................. 39 Template ........................................................................................................................... 39 Host group......................................................................................................................... 39 Library server .................................................................................................................... 39 Library share ..................................................................................................................... 39 Lab Management ................................................................................................................... 40 Virtual environment .......................................................................................................... 40 Physical environment........................................................................................................ 40 Lab Center ......................................................................................................................... 40 Team project library.......................................................................................................... 40 Creating and managing environments .................................................................................. 41 Testing ................................................................................................................................... 42 Build deployment .................................................................................................................. 43 Lab Management ................................................................................................................... 44 Chapter 3....................................................................................................................................... 45 ANATOMY of a Coded UI Test ....................................................................................................... 45 LESSON 1: Overview of Coded UI Test ...................................................................................... 47 Contents of a Coded UI Test.................................................................................................. 47 UIMap.Designer.cs ............................................................................................................ 48 Declarations section .............................................................................................................. 48 UIMap class............................................................................................................................ 48

UIMap methods ..................................................................................................................... 49 UIMap properties .................................................................................................................. 50 UIMap.cs ................................................................................................................................ 51 UIMap class............................................................................................................................ 51 CodedUITest1.cs .................................................................................................................... 52 CodedUITest1 class ................................................................................................................ 52 CodedUITest1 properties....................................................................................................... 52 CodedUITest1 methods ......................................................................................................... 52 UIMap.uitest .......................................................................................................................... 54 Chapter 4....................................................................................................................................... 56 Automated Test ............................................................................................................................ 56 Creating Automated Tests......................................................................................................... 57 Automated Testing Best Activities ........................................................................................ 58 Manual to Automated Test Conversion ................................................................................ 59 Testing Methods........................................................................................................................ 61 Use Cases or User Stories ................................................................................................. 61 Speed Up Testing User Interface ...................................................................................... 61 Improving Code Quality Using Unit Tests ......................................................................... 62 Test How Your Application Performs Under Heavy Loads ............................................... 62 Test the Functionality of the Application......................................................................... 62 Improving Database Code Quality Using Database Unit Tests: ........................................ 62 Converting Manual Test Cases to Automated Tests to Run As Part of a Test Plan .......... 62 Running Automated Tests from a Test Plan.............................................................................. 63 Best Practice Checklist ........................................................................................................... 64 Determine what roles you need in your environment ..................................................... 64 Create a physical environment .............................................................................................. 64 Create a test plan .............................................................................................................. 64 Associating automated test with a Test Case ........................................................................... 66 Testing Scenarios ................................................................................................................... 68 Coded UI Tests ....................................................................................................................... 68 Unit Tests ............................................................................................................................... 68 Database Unit Tests ............................................................................................................... 68 Load tests ............................................................................................................................... 68

10

Web Performance Tests ........................................................................................................ 68 Generic Tests ......................................................................................................................... 68 LESSON 4: Testing the Application ............................................................................................ 69 Planning your testing strategy ............................................................................................... 70 Planning your testing effort .............................................................................................. 70 Creating and managing tests ............................................................................................ 70 Setting up how to run tests or collect data remotely ....................................................... 70 Running Your Tests ........................................................................................................... 70 Submitting bugs ..................................................................................................................... 70 Tracking software quality.................................................................................................. 71 ................................................................................................................................................... 71 LAB: Automated Test Case ........................................................................................................ 72 Create and Configure Test Projects for Automated Tests ........................................................ 76 ............................................................................................................................................... 76 LAB 1: Create a Test Project When You Add a Test ............................................................... 77 ............................................................................................................................................... 77 LAB 2: Create a test project as you generate unit tests ........................................................ 78 ............................................................................................................................................... 78 LAB 3: Create a test project by using the Add New Project dialog box ................................ 79 LAB 4: Configure Test Project Creation ................................................................................. 80 Chapter 5....................................................................................................................................... 82 Tracking Software Quality ............................................................................................................. 82 Test Dashboard (Agile) .......................................................................................................... 83 SharePoint 2010 Developers Dashboards ............................................................................. 84 Counters in developer dashboard ......................................................................................... 84 Pre-Process counters ........................................................................................................ 84 Client Counters ................................................................................................................. 85 System Counter ................................................................................................................. 85 How to use Monitor in Code?................................................................................................ 85 Developer Dash Board in a custom Page .............................................................................. 86 STSADM Utility ....................................................................................................................... 86 PowerShell Commands .......................................................................................................... 87 LAB 1: Create the Portal in SharePoint and then fix up the missing documents ...................... 88

11

Integrate portal site with team project ................................................................................. 89 Download process template.................................................................................................. 90 Add Product and Iteration backlog........................................................................................ 90 Bind Iteration backlog workbook to the team project .......................................................... 91 Bind Product backlog workbook to the team project ........................................................... 93 Chapter 6....................................................................................................................................... 94 Create a Coded UI Test ................................................................................................................. 94 Introduction............................................................................................................................... 95 LAB: Create a coded UI test....................................................................................................... 97 Coding UI Test ........................................................................................................................... 98 LAB: Generating a coded UI test by recording the application under test ............................... 99 Chapter 7..................................................................................................................................... 104 UI Controls and Validation Code Using the Coded UI Test Builder ............................................ 104 Chapter 8..................................................................................................................................... 109 Data-Driven Coded UI Test ......................................................................................................... 109 Configuring the UI Test DataSource ........................................................................................ 111 LAB: Create the file and add it to your test project. ............................................................... 116 Action Recording ................................................................................................................. 117 LAB: Generate a Coded UI Test from an Action Recording .................................................... 118 Chapter 9..................................................................................................................................... 121 Actions and Properties in a Coded UI ......................................................................................... 121 Accessing Actions of UI Test Control ...................................................................................... 122 Chapter 10................................................................................................................................... 125 What to do if your UI Changes .................................................................................................... 125 LAB: Creating, Editing and Maintaining a Coded UI Test ........................................................ 126 Chapter 11................................................................................................................................... 135 Extending Coded UI Tests and Action Recordings to Support Microsoft Excel .......................... 135 Architecture Overview ............................................................................................................ 136 Create A Test Case ............................................................................................................... 137 Create The CodedUI Watin Test .......................................................................................... 137 Testing Using Virtual Environments ........................................................................................ 139 Running Tests .......................................................................................................................... 141 ................................................................................................................................................ 143

12

Database Unit Tests ................................................................................................................ 144 Chapter 12................................................................................................................................... 145 Running Automated Tests........................................................................................................... 145 Create Test Cases from Automated Tests Using TCM.exe ......................................................... 165 Introduction............................................................................................................................. 166 Common Parameters for the Procedures ............................................................................... 167 LAB: Create Test Cases from an Assembly of Automated Tests ............................................. 168 ................................................................................................................................................. 172 EXERCISE: ................................................................................................................................. 173

13

14

15

16

Chapter 1
Why Automated Testing?

17

Introduction
Every software development group tests its products, yet delivered software always has defects. Test engineers strive to catch them before the product is released but they always creep in and they often reappear, even with the best manual testing processes. Automated software testing is the best way to increase the effectiveness, efficiency and coverage of your software testing.

Manual software testing

is performed by a human sitting in front of a computer carefully going through application screens, trying various usage and input combinations, comparing the results to the expected behavior and recording their observations. Manual tests are repeated often during development cycles for source code changes and other situations like multiple operating environments and hardware configurations.

Automated software testing tool

Is able to playback pre-recorded and predefined actions compare the results to the expected behavior and report the success or failure of these manual tests to a test engineer. Once automated tests are created they can easily be repeated and they can be extended to perform tasks impossible with manual testing. Because of this, savvy managers have found that automated software testing is an essential component of successful development projects. Automated software testing has long been considered critical for big software development organizations but is often thought to be too expensive or difficult for smaller companies to implement. Visual Studio 2010 offers an affordable solution for testing that includes some of the best testing tools in the market

The Question is: What makes automated software testing so important to these successful companies? Automated Software Testing Saves Time and Money Software tests have to be repeated often during development cycles to ensure quality. Each time source code is modified software tests should be repeated. For every release of the software it may be tested on all supported operating systems and hardware configurations. Physically repeating these tests is costly and time consuming. Once created, automated tests can be run over and over again at no extra cost and they are much faster than manual tests. Automated software testing can decrease the time to run repetitive tests from days to hours.

18

A time savings that translates directly into cost savings.

Improves Accuracy
Even the most meticulous tester will make mistakes during monotonous manual testing. Automated tests perform the same steps precisely every time they are executed and never forget to record detailed results thru reporting strategies and monitoring webparts.

Increases Test Coverage

Automated software testing can increase the depth and scope of tests to help improve software quality. Drawn-out tests that are often avoided during manual testing can be run unattended. Automation can be deployed to support multiple devices with a diverse range of configurations. Automated software testing can monitor the application and report on memory utilization, object relationships, data tables, file contents, and object states to determine if the application is behaving as expected within the constraining set by the manufacturer. Automated software tests can easily execute thousands of different complex test cases during every test run providing coverage that is awkward with manual tests. Testers freed from repetitive manual tests are more productive in other phases of development or deployment, being able to create new automated software tests and deal with complex features of each specialized application.

Does What Manual Testing Cannot

Even when a large amount of developers are present in an architecture, they are not able to perform a controlled test of certain technologies such as web application. Web application might support thousands of requests per second generated by thousands of users, being able to replicate this environment in testing and automation the process is a crucial part of testing. Automated testing can simulate tens, hundreds, thousands and even millions of virtual users interacting with network or web software and applications real time.

Helps Developers and Testers

Shared automated tests can be used by developers to catch problems quickly before sending to QA. Tests can run automatically each time source code changes are checked in and notify the team or the developer if they fail. Features like these save developers time and increase their confidence.

Improves Team Morale

This is hard to quantity but weve experienced it first hand, automated software testing can increase team confidence. Automating tedious tasks with automated software testing gives the

19

development team time to spend on more challenging and rewarding projects. Team members increase their skill sets and self-assurance and, in turn, pass those gains on to their organization.

UI Automation for Testing

UI Automation delivers a unified object model that enables all user interface (UI) frameworks to expose complex and rich functionality in an accessible and easily automated manner. UI Automation was developed as a replacement to Microsoft Active Accessibility strategy. Active Accessibility is an existing framework designed to provide a solution for making controls and applications accessible. Active Accessibility was not designed with test automation in mind even though it evolved into that role due to the very similar requirements of accessibility and automation. UI Automation, in addition to providing more refined solutions for accessibility, is also specifically designed to provide robust functionality for automated testing. Scenario: Active Accessibility relies on a single interface to both expose information about the UI and collect the data required by AT products; UI Automation separates the two models: Provider Client

Together they are required to implement UI Automation for it to be useful as an automated test tool. UI Automation providers are applications such as Microsoft Word, Excel, and other third-party applications or controls based on the Microsoft Windows operating system. UI Automation clients include automated test scripts and assistive technology applications. There are multiple topics to consider in UI Automation for Testing, such as: UI Automation in a Provider UI Automation in a Client Related Tools and Technologies Security Related Topics For a UI to be automated, a developer of an application or control must look at what actions an end-user can perform on the UI object using standard keyboard and mouse interaction. Once these key actions have been identified, the corresponding UI Automation control patterns (that is, the control patterns that mirror the functionality and behavior of the UI element) should be implemented on the control. For example, user interaction with a combo box control

20

(such as the run dialog) typically involves expanding and collapsing the combo box to hide or display a list of items, selecting an item from that list, or adding a new value via keyboard input. Note With other accessibility models, developers must gather information directly from individual buttons, menus, or other controls. Unfortunately, every control type comes in dozens of minor variations. In other words, even though ten variations of a pushbutton may all work the same way and perform the same function, they must all be treated as unique controls. There is no way to know that these controls are functionally equivalent. Control patterns were developed to represent these common control behaviors.

Implementing UI Automation
As mentioned earlier, without the unified model provided by UI Automation, test tools and developers are required to know framework-specific information in order to expose properties and behaviors of controls in that framework. Since there can be several different UI frameworks present at any single time within Windows operating systems, including Win32, Windows Forms, and Windows Presentation Foundation (WPF), it can be a daunting task to test multiple applications with controls that seem similar. For example, the following table outlines the framework-specific property names required to retrieve the name (or text) associated with a button control and shows the single equivalent UI Automation property. UI Automation Control Type Button Button Image UI Framework Windows Presentation Foundation Win32 HTML Framework Specific Property Content Caption alt UI Automation Property NameProperty NameProperty NameProperty

UI Automation providers are responsible for mapping the framework-specific properties of their controls to the equivalent UI Automation properties.

UI Automation in a Client
The goal of many automated test tools and scenarios is the consistent and repeatable manipulation of the user interface. This can involve unit testing specific controls through to the recording and playback of test scripts that iterate through a series of generic actions on a group of controls. A complication that arises from automated applications is the difficulty synchronizing a test with a dynamic target. For example, a list box control, such as one contained in the Windows Task Manager, that displays a list of currently running applications. Since the items in the list box are dynamically updated outside the control of the test application, attempting to repeat

21

the selection of a specific item in the list box with any consistency is impossible. Similar issues can also arise when attempting to repeat simple focus changes in a UI that is outside the control of the test application. Programmatic Access Programmatic access provides the ability to imitate, through code, any interaction and experience exposed by traditional mouse and keyboard input. UI Automation enables programmatic access through five components:

The UI Automation tree facilitates navigation through the structure of the UI. The tree is built from the collection of hWnd's. For more information, see UI Automation Tree Overview Automation elements are individual components in the UI. These can often be more granular than an hWnd. For more information, see UI Automation Control Types Overview. Automation properties provide specific information about UI elements. For more information, see UI Automation Properties Overview. Control patterns define a particular aspect of a control's functionality; they can consist of property, method, event, and structure information. For more information, see UI Automation Control Patterns Overview . Automation events provide event notifications and information. For more information, see UI Automation Events Overview.

Key Properties for Test Automation The ability to uniquely identify and subsequently locate any control within the UI provides the basis for automated test applications to operate on that UI. There are several Microsoft UI Automation properties used by clients and providers that assist in this. AutomationID Uniquely identifies an automation element from its siblings. AutomationIdProperty is not localized, unlike a property such as NameProperty that is typically localized if a product gets shipped in multiple languages. See Use the AutomationID Property.

Note AutomationIdProperty does not guarantee a unique identity throughout the automation tree. For example, an application may contain a menu control with multiple top-level menu items that, in turn, have multiple child menu items. These secondary

22

menu items may be identified by a generic scheme such as "Item1, Item 2, Item3, etc.", allowing duplicate identifiers for children across top-level menu items. ControlType Identifies the type of control represented by an automation element. Significant information can be inferred from knowledge of the control type. NameProperty This is a text string that identifies or explains a control. NameProperty should be used with caution since it can be localized. Implementing UI Automation in a Test Application Add the UI Automation References. The UI Automation dll's necessary for UI Automation clients are listed here. UIAutomationClient.dll provides access to the UI Automation client-side APIs. UIAutomationClientSideProvider.dll provides the ability to automate Win32 controls. See UI Automation Support for Standard Controls. UIAutomationTypes.dll provides access to the specific types defined in UI Automation. This namespace contains everything UI Automation clients need to use the capabilities of UI Automation except text handling. This namespace contains everything a UI Automation clients need to use the capabilities of UI Automation text handling. Automated test scripts locate UI Automation elements that represent controls of interest within the automation tree.

System.Windows.Automation

System.Windows.Automation.Tex t Find controls of interest

There are multiple ways to obtain UI Automation elements with code.

Query the UI using a Condition statement. This is typically where the language-neutral AutomationIdProperty is used.

Note: An AutomationldProperty can be obtained using a

23

tool such as UISpy.exe(UI Spy) that is able to itemized the UI automation properties of a control

Use the TreeWalker class to traverse the entire UI Automation tree or a subset thereof. Track focus. Use the hWnd of the control. Use screen location, such as the location of the mouse cursor.

Obtain Control Patterns

Control patterns expose common behaviors for functionally similar controls. After locating the controls that require testing, automated test scripts obtain the control patterns of interest from those UI Automation elements. For example, the InvokePattern control pattern for typical button functionality or the WindowPattern control pattern for window functionality. Automated test scripts can now control any UI of interest from a UI framework using the information and functionality exposed by the UI Automation control patterns.

Automate the UI

24

Application Lifecycle Management

Chapter 2

25

LESSON 1:

Application Lifecycle Management (ALM) is a continuous process of managing the life of an application through governance, development and maintenance. ALM is the marriage of business management to software engineering made possible by tools that facilitate and integrate requirements management, architecture, coding, testing, tracking, and release management. A developer can apply proven Activities to manage application's lifecycle by using the suite of tools in Visual Studio Premium and Visual Studio Ultimate in combination with Visual Studio Team Foundation Server. Visual Studio allows the developer to better understand customer needs and more effectively design, implement, and deploy code.

26

Scenario #1: A quality assurance team, task with testing application to determine their production viability, they checked-in code, builds, and test results. QA Team might adopt Activities such as check-in code, builds and test results so that customers value more application and to certify the performance and more reliability of the solution. A QA Test Unit might use the following tools to achieve results:

Plan and track your project: Enact processes and monitor their quality based on customer requirements into working software. Design functionality: Create new or use existing assets Architectural Diagrams: architectural diagrams help communicate critical information about the software solution. Write, unit test, debug, analyze, and profiling: Prototypes for Customer approvals Version control: to manage your source code and other files. Build: verify that requirements have been fulfilled in each build. Test: performance and stress tests. Deploy into virtual environments: Advanced testing.

27

LESSON 2: Team Foundation Server

Visual Studio Team Foundation works in three-tier architecture: The Client Tier The application tier The data tier

The Client Tier

The client tier is used for creating and managing projects and accessing the items that are stored and managed for a project. TFS does not include any user interface for this tier, rather it exposes services which client applications can use to integrate VSTFS functionality with themselves. These web services are used by applications like Visual Studio Team System to use TFS as data storage back end or dedicated TFS management applications like the included Team Foundation Client. The web services are in the application layer.

The Application Tier

The application layer also includes a web portal and a document repository facilitated by Windows SharePoint Services. The web portal, called the Team Project Portal, acts as the central point of communication for projects managed by TFS. The document repository is used for both project items and the revisions tracked, as well as for aggregated data and generated reports.

28

The Data Tier

The data layer, essentially an SQL Server 2008 R2 SP1 Enterprise Edition installation, provides the persistent data storage services for the document repository. The data tier and application tier can exist on different physical or virtual servers as well, provided they are running Windows Server 2008 R2 or later. The data tier is not exposed to the client tier; only the application tier is.

Automatic Compiling
With Team Foundation Build, you can create build definitions to automate compiling applications, running associated tests, performing code analysis, releasing continuous builds, and publishing build reports. To build an application, you create a build definition to specify what projects to build, what triggers a build to run, what automated tests to run, and where to deploy the output. This information is stored in the data warehouse, from which it is retrieved when a build runs. After the build runs, data about the build results is stored back in the warehouse, where it is available to view through build reports. The following illustration shows the three main phases of building an application: Three phases of building an application

29

Application Lifecycle Management

You can integrate other aspects of the application development lifecycle with Team Foundation Server by using one or more other products in Visual Studio Premium and Visual Studio Ultimate. Microsoft Test Manager: enables you to manage and execute test cases outside Visual Studio, Create and manage physical or virtual environments. Visual Studio Premium: Provides a complete development toolset for building applications. Advanced coding, debugging, database, and testing tools Visual Studio Ultimate

simplifies solution development, Lowers risk and increases ROI return by providing tools for every stage of the lifecycle, from design and development through test and deployment. can design your application in UML,

30

Analyzes your existing software assets. defines software architecture Validate that design when your team checks in and builds with Team Foundation Server

The following table shows how these tools can be used throughout the application lifecycle. Application Lifecycle Plan and Track Design Develop Test Build Team Foundation Server Microsoft Test Manager Visual Studio Premium Visual Studio Ultimate

31

LESSON 3:
Overview of Virtual Lab for Application Lifecycle Management

32

Lab Management is an extension of Microsoft Test Manager that helps you to optimize the use of Microsoft Hyper-V technology to manage and use virtual machines in testing, building, and developing applications in Visual Studio 2010 or Visual Studio 2012. Visual Studio Lab Management is integrated with System Center 2012 Virtual Machine Manager (SCVMM) to enable you manage multiple physical computers that host virtual machines and to manage the storage of virtual machines, virtual machine templates, and other configuration files in SCVMM library servers.

33

What are Virtual Environments

Virtual environments are groups of virtual machines that are managed by Lab Management. Virtual environments enable you to: Reproduce the exact conditions of a bug or other development issue. Lab Management snapshots capture the state of all computers in the environment at a point in time. A snapshot of environment can be shared with members of a project team. A reference to the stored snapshot can be included in Visual Studio Team Foundation Server work item so that a copy of the environment can be created with a few clicks of the mouse. When Microsoft IntelliTrace data is included in the work item, the execution path of the application to the bug can be fully reproduced on the exact configuration where the bug occurred. Build, deploy, and test applications automatically in a clean environment. Using Lab Management workflows for Visual Studio Team Foundation Server builds, environments, snapshots and Microsoft Test Manager automated tests, you can stage daily builds of your application to a clean environment. After tests have been run, you

34

can store the environment in a snapshot and then restore the environment to its original state so that it can be used again. Reduce the time required to create and configure machines for testing an application. Using Lab Management stored virtual machines and templates; you can quickly deploy customized environments that recreate the configuration of customer's production environments. Run multiple copies of a test or development at the same time Using Lab Management stored virtual environments; you can deploy multiple copies of a virtual environment at the same time without the computer name conflicts that can limit the use of unmanaged virtual machines in a domain. Enable members of a team to create and manage virtual environments without requiring system administrator privileges. You can use Lab Management permissions to create role-based self- service groups of team members who can deploy, create and manage virtual environments without requiring system administrator privileges. You can also combine Lab Management permissions with Visual Studio Team Foundation Server and SCVMM permissions to meet the specific requirements of your organization.

Simple Virtual Environments

A testing professional can create a testing environment by using Microsoft Test Manager and assign virtual machines to each role that is required for the application that you intend to develop, test, or run. Scenario: A Development team might be coding a multi-tiered application that requires three roles: A desktop client, a Web server, and a Database server. By using Lab Management, you can create a virtual environment that assigns a virtual machine to each role, deploys each part of the application to the relevant virtual machine by using Team Foundation Build, and then runs the three virtual machines as a single instance of the application for testing. The following illustration shows a virtual environment that uses the three roles: desktop client, Web server, and database server.

35

Complex Virtual Environments

If your application is complex, you can assign multiple virtual machines to the same role in a virtual environment. Similarly, you might be developing an application that requires the same two roles to be deployed in two different topologies. Scenario: The data tier and the application tier might be located on the same machine in one topology, whereas the two tiers are located on separate machines in the other topology. The first topology is represented in one virtual environment, and the second topology is represented in a separate virtual environment. Your virtual lab now contains the two environments that you have to have for testing.

Targeted Virtual Environments

You can also have a virtual environment where only some components of the application are deployed whereas other components are shared across environments. For example, if your application needs a large database, you can decide to host a shared database on a physical machine. All virtual environments will have only virtual machines for the client and application tiers that can connect to the shared database as required.

36

LESSON 3:

Virtual Environments
You must understand some key concepts before you start to use Lab Management:

37

Virtual machines (Hyper-V or VMware)

Virtual machine A virtual machine is a software implementation of a computer that runs programs as if it is a physical computer. This enables you to run multiple virtual machines on one physical computer. You must install your virtual machine with an operating system and any other software that you require. Host The physical computer that runs the virtual machine

38

Snapshot A snapshot saves the state of your virtual machine at a specific point in time. You can revert your virtual machine to this snapshot whenever you want and then use this virtual machine from this known state when the snapshot was taken.

System Center Virtual Machine Manager (SCVMM)

Template A template is a generalized image of a virtual machine that has been stripped of identifying information such as the machine name and the product key. You can create multiple virtual machines from a template without creating computer name conflicts on a domain. Host group A group of physical computers that are hosts that can be used to run virtual machines. Host groups optimize the deployment of virtual machines on the collection of hosts for you, so that you do not have to select to which host each virtual machine is deployed. You can assign host groups to your team projects and team project collections in Team Foundation Server. Library server A computer that is used to store virtual machines, templates, and other resources. Library share A shared location on a library server where you can store your virtual machines, templates, and other resources. You can have multiple library shares. You can assign library shares to your team projects and team project collections in Team Foundation Server.

39

Lab Management
Virtual environment A collection of virtual machines that run on Hyper-V hosts, are managed by System Center Virtual Machine Manager, and which Lab Management manages as a logical unit. Each virtual machine that you add to the environment is used for a role in your application. For example, you might select a role of Web server for a virtual machine. Physical environment A collection of physical machines, each of which has a test agent installed. If you have virtual machines running on a non-Hyper-V platform, you can model them as physical machines in Lab Management. Lab Center The activity area in Microsoft Test Manager where you manage and operate the virtual machines and environments that are deployed on the SCVMM host groups of your team project. Team project library The activity area in Test Manager where you manage the stored virtual machines

40

A Testing Engineer can use either physical or virtual machines in creating and managing environments, testing, or build deployment.

Creating and managing environments

Creating virtual machines from a template, cloning environments using network isolation, VM operations such as Start/Stop/Snapshot/Pause are exclusively available for Hyper-V based virtual environments and not available for non-Hyper-V or physical environments.

41

Testing
Using Microsoft Test Manager, you can run tests on both virtual as well as physical environments. The capabilities are exactly the same here and this means that you can run automated and manual tests, and do rich diagnostic data collection from all sorts of environments Hyper-V, non-Hyper-V, and physical.

42

Build deployment
You can easily automate the build-deployment-testing workflow on Hyper-V virtual environments using the template shipped in TFS 2010 (labdefaulttemplate.xaml). For physical environments, such a template is not available out of the box. However, since this automation is based on Windows Workflow Foundation 4.0, you can customize the template (or create your own) using the activities in the toolbox, to automate build deployment and testing on physical machines. This means you can do the same for non-Hyper-V environments as well, however note that the virtual environment specific actions (restore to snapshot/start/stop environment) will not be available for physical and non-Hyper-V environments.

43

Lab Management
Lab Management works with both virtual environments hosted on Hyper-V and physical environments, but the capabilities of each type of environment are different. Virtual environments on non-Hyper-V hosts are treated the same as physical environments. The following table summarizes the different capabilities for each type of environment. Capability Virtual (Hyper-V) Physical Virtual (VMware)
Testing Run unit tests Run manual tests Run coded UI and other automated tests File rich bugs using diagnostic adapters Build deployment Automatic build-deploy-test workflow Environments creation and management Create environment from VM template Start/stop/snapshot environment Connect using Environment Viewer Clone environment using network isolation Supported Supported Supported Supported Supported Supported Supported Customization required Supported Supported Supported Customization required

Supported

Supported Supported Supported Supported

n/a Not supported Not supported n/a

Not supported Not supported Not supported Not supported

44

Chapter 3
ANATOMY of a Coded UI Test

45

46

LESSON 1: Overview of Coded UI Test

When you create a Coded UI Test in a testing project, several files are added to your solution. In this topic, we will use an example Coded UI Test to explore these files.

Contents of a Coded UI Test

When you create a Coded UI Test, the Coded UI Test Builder creates a map of the user interface under test, and also the test methods, parameters, and assertions for all tests. It also creates a class file for each test. File UIMap.Designer.cs Contents Declarations section UIMap class (partial, auto-generated) Methods Properties UIMap class (partial) CodedUITest1 class Methods Properties The XML map of the UI for the test. Editable? No

UIMap.cs CodedUITest1.cs

Yes Yes

UIMap.uitest

No

47

UIMap.Designer.cs This file contains code that is automatically created by the Coded UI Test Builder when a test is created. This file is re-created every time that a test changes, so that it is not a file in which you can add or modify code.

Declarations section
This section includes the following declarations for a Windows UI. C# using System; using System.CodeDom.Compiler; using System.Collections.Generic; using System.Drawing; using System.Text.RegularExpressions; using System.Windows.Input; using Microsoft.VisualStudio.TestTools.UITest.Extension; using Microsoft.VisualStudio.TestTools.UITesting; using Microsoft.VisualStudio.TestTools.UITesting.WinControls; using Microsoft.VisualStudio.TestTools.UnitTesting; using Keyboard = Microsoft.VisualStudio.TestTools.UITesting.Keyboard; using Mouse = Microsoft.VisualStudio.TestTools.UITesting.Mouse; using MouseButtons = System.Windows.Forms.MouseButtons; The Microsoft.VisualStudio.TestTools.UITesting.WinControls namespace is included for a Windows user interface (UI). For a Web page UI, the namespace would be Microsoft.VisualStudio.TestTools.UITesting.HtmlControls; for a Windows Presentation Foundation UI, the namespace would be Microsoft.VisualStudio.TestTools.UITesting.WpfControls.

UIMap class
The next section of the file is the UIMap class. [GeneratedCode("Coded UITest Builder", "10.0.21221.0")] public partial class UIMap The class code starts with a GeneratedCodeAttribute that is applied to the class, which is declared as a partial class. You will notice that the attribute is also applied to every class in this file. The other file that can contain more code for this class is UIMap.cs, which is discussed later. The generated UIMap class includes code for each method that was specified when the test was recorded.

48

public void LaunchCalculator() public void AddItems() public void VerifyTotal() public void CleanUp() This part of the UIMap class also includes the generated code for each property that is required by the methods. public virtual LaunchCalculatorParams LaunchCalculatorParams public virtual AddItemsParams AddItemsParams public virtual VerifyTotalExpectedValues VerifyTotalExpectedValues public virtual CalculateItemsParams CalculateItemsParams public virtual VerifyMathAppTotalExpectedValues VerifyMathAppTotalExpectedValues public UIStartMenuWindow UIStartMenuWindow public UIRunWindow UIRunWindow public UICalculatorWindow UICalculatorWindow public UIStartWindow UIStartWindow public UIMathApplicationWindow UIMathApplicationWindow

UIMap methods
Each method has a structure that resembles the AddItems() method. This is explained in more detail under the code, which is presented together with line breaks to add clarity. /// <summary> /// AddItems - Use 'AddItemsParams' to pass parameters into this method. /// </summary> public void AddItems() { #region Variable Declarations WinControl uICalculatorDialog = this.UICalculatorWindow.UICalculatorDialog; WinEdit uIItemEdit = this.UICalculatorWindow.UIItemWindow.UIItemEdit; #endregion // Type '{NumPad7}' in 'Calculator' Dialog Keyboard.SendKeys(uICalculatorDialog, this.AddItemsParams.UICalculatorDialogSendKeys, ModifierKeys.None); // Type '{Add}{NumPad2}{Enter}' in 'Unknown Name' text box

49

Keyboard.SendKeys(uIItemEdit, this.AddItemsParams.UIItemEditSendKeys, ModifierKeys.None); } The summary comment for each method definition tells which class to use for parameter values for that method. In this case, it is the AddItemsParams class, which is defined later in the UIMap.cs file, and which is also the value type that is returned by the AddItemsParams property. At the top of the method code is a Variable Declarations region that defines local variables for the UI objects that will be used by the method. In this method, both UIItemWindow and UIItemEdit are properties that are accessed by using the UICalculatorWindow class, which is defined later in the UIMap.cs file. Next are lines that send text from the keyboard to the Calculator application by using properties of the AddItemsParams object. The VerifyTotal() method has a very similar structure, and includes the following assertion code. // Verify that 'Unknown Name' text box's property 'Text' equals '9. ' Assert.AreEqual( this.VerifyTotalExpectedValues.UIItemEditText, uIItemEdit.Text); The text box name is listed as unknown because the developer of the Windows Calculator application did not provide a publicly available name for the control. The Assert.AreEqual method fails when the actual value is not equal to the expected value, which would cause the test to fail. Also notice that the expected value includes a decimal point that is followed by a space. If you ever have to modify the functionality of this particular test, you must allow for that decimal point and the space.

UIMap properties
The code for each property is also very standard throughout the class. The following code for the AddItemsParams property is used in the AddItems() method. public virtual AddItemsParams AddItemsParams { get { if ((this.mAddItemsParams == null)) { this.mAddItemsParams = new AddItemsParams(); }

50

return this.mAddItemsParams; } } Notice that the property uses a private local variable that is named mAddItemsParams to hold the value before it returns it. The property name and the class name for the object it returns are the same. The class is defined later in the UIMap.cs file. Each class that is returned by a property is structured similarly. The following is the AddItemsParams class. /// <summary> /// Parameters to be passed into 'AddItems' /// </summary> [GeneratedCode("Coded UITest Builder", "10.0.21221.0")] public class AddItemsParams { #region Fields /// <summary> /// Type '{NumPad7}' in 'Calculator' Dialog /// </summary> public string UICalculatorDialogSendKeys = "{NumPad7}"; /// <summary> /// Type '{Add}{NumPad2}{Enter}' in 'Unknown Name' text box /// </summary> public string UIItemEditSendKeys = "{Add}{NumPad2}{Enter}"; #endregion } As with all classes in the UIMap.cs file, this class starts with the GeneratedCodeAttribute. In this small class is a Fields region that defines the strings to use as parameters for the Keyboard.SendKeys method that is used in the UIMap.AddItems() method that was discussed earlier. You can write code to replace the values in these string fields before the method in which these parameters are used is called.

UIMap.cs
By default, this file contains a partial UIMap class that has no methods or properties.

UIMap class

51

This is where you can create custom code to extend the functionality of the UIMap class. The code that you create in this file will not be regenerated by the Coded UI Test Builder every time that a test is modified. All parts of the UIMap can use the methods and properties from any other part of the UIMap class.

CodedUITest1.cs
This file is generated by the Coded UI Test Builder, but is not re-created every time that the test is modified, so that you can modify the code in this file. The name of the file is generated from the name that you specified for the test when you created it.

CodedUITest1 class
By default, this file contains the definition for only one class. [CodedUITest] public class CodedUITest1 The T:Microsoft.VisualStudio.TestTools.UITesting.CodedUITestAttribute is automatically applied to the class, which allows the testing framework to recognize it as a testing extension. Also notice that this is not a partial class. All class code is contained in this file.

CodedUITest1 properties
The class contains two default properties that are located at the bottom of the file. They must not be modified. /// <summary> /// Gets or sets the test context which provides /// information about and functionality for the current test run. ///</summary> public TestContext TestContext public UIMap UIMap

CodedUITest1 methods
By default, the class contains only one method. public void CodedUITestMethod1()

52

This method calls each UIMap method that you specified when you recorded your test, which is described in the section on the UIMap Class. A region that is titled Additional test attributes, if uncommented, contains two optional methods. // Use TestInitialize to run code before running each test [TestInitialize()] public void MyTestInitialize() { // To generate code for this test, select "Generate Code for Coded // UI Test" from the shortcut menu and select one of the menu items. // For more information on generated code, see // http://go.microsoft.com/fwlink/?LinkId=179463 // You could move this line from the CodedUITestMethod1() method this.UIMap.LaunchCalculator(); } // Use TestCleanup to run code after each test has run [TestCleanup()] public void MyTestCleanup() { // To generate code for this test, select "Generate Code for Coded // UI Test" from the shortcut menu and select one of the menu items. // For more information on generated code, see // http://go.microsoft.com/fwlink/?LinkId=179463 // You could move this line from the CodedUITestMethod1() method this.UIMap.CloseCalculator(); } The MyTestInitialize() method has the TestInitializeAttribute applied to it, which tells the testing framework to call this method before any other test methods. Similarly, the MyTestCleanup() method has the TestCleanupAttribute applied to it, which tells the testing framework to call this method after all other test methods have been called. Use of these methods is optional. For this test, the UIMap.LaunchCalculator() method could be called from MyTestInitialize() and the UIMap.CloseCalculator() method could be called from MyTestCleanup() instead of from CodedUITest1Method1(). If you add more methods to this class by using the CodedUITestAttribute, the testing framework will call each method as part of the test.

53

UIMap.uitest
This is an XML file that represents the structure of the coded UI test recording and all its parts. These include the actions and the classes in addition to the methods and properties of those classes. The UIMap.Designer.cs file contains the code that is generated by the Coded UI Builder to reproduce the structure of the test and provides the connection to the testing framework. The UIMap.uitest file is not directly editable. However, you can use the Coded UI Builder to modify the test, which automatically modifies the UIMap.uitest file and the UIMap.Designer.cs file.

54

55

Chapter 4
Automated Test

56

Creating Automated Tests

There are several types of automated tests using Visual Studio Ultimate or Visual Studio Premium, which enable you to test your application more efficiently. Automated tests run test steps for you and determine whether the test passes or fails. These tests can be run more quickly and more frequently. Automated tests can quickly test whether your application is still working correctly after code changes have been made to your application. Automated tests are created using Visual Studio.

57

Automated Testing Best Activities

1. 2. 3. 4. 5. 6. Decide what Test Cases to Automate Test Early and Test Often Select the Right Automated Testing Tool Divide your Automated Testing Efforts Create Good, Quality Test Data Create Automated Tests that are Resistant to Changes in the UI

58

Manual to Automated Test Conversion

If you choose, a developer can convert manual test cases to automated tests by linking automation to the test case. Then these automated tests can be run using Microsoft Test Manager and the results can be reviewed as part of the test plan progress. The following table shows the types of automated tests that you can create and which types of tests can be run as part of a test plan: Type of test Coded UI Tests Unit Tests Database Unit Tests Load tests Description Tests the user interface by performing UI actions. Tests code at the method level. Tests a stored procedure, function or trigger in a database. Tests application performance and stress using unit tests, Web Performance tests or coded UI tests. Run tests from VS Yes Yes Yes Yes Run from a TP by associating a TM with TC Yes Yes Yes Not recommended because you cannot view the test as it runs.

59

Web Performance Tests Generic Tests

Used as part of load tests to test server responses for Web applications by sending and receiving http requests. Test functionality using API calls or command line tools for the application under test.

Yes

Yes

Not recommended because Web Performance tests are primarily intended to be used with load tests. Yes

Depending on what is need to be accomplish with UI testing will determine which types of automated tests you need to create. When you have created these tests, then you can decide if you want to report the results as part of a test plan. The tasks in the table below can help you determine your automated testing needs.

60

Testing Methods
Use Cases or User Stories After reviewing the users' needs for your application, you can create test cases to cover testing these needs. You can connect each test case to the relevant requirements, use cases or user stories, depending on your project management methodology. You can then associate an automated test with your test case. This links your automated tests to your requirements, use cases, or user stories. How to: Add Requirements or User Stories to Your Test Plan Creating a Test Plan Using Requirements or User Stories How to: Associate an Automated Test with a Test Case Speed Up Testing User Interface You can create automated user interface tests for your application. These tests can be used to check that there has been no regression or change in the user interface from coding changes. Testing the User Interface with Automated UI Tests

61

How to: Associate an Automated Test with a Test Case

Improving Code Quality Using Unit Tests You can automatically generate unit tests from your existing code, or you can create unit tests manually. Unit tests will help improve the quality of your code by ensuring that changes you make do not cause issues in existing functionality. Verifying Code by Using Unit Tests Test How Your Application Performs Under Heavy Loads You can create either unit tests or Web Performance tests and use these in load tests. With a load test, you can create a heavy load on your application by simulating multiple users. Then you can measure the performance of your application and check it can still run successfully with a large number of users accessing it. Testing Application Performance and Stress

Test the Functionality of the Application If you can access the functionality of your application using API calls or a command line interface, you can create automated tests to drive your application without using the user interface. Creating an Automated Test That Runs an Executable Using Generic Tests Improving Database Code Quality Using Database Unit Tests: You can create database unit tests to verify that the stored procedures, functions and triggers in the database are working correctly. These tests can be run to check that changes to the database or database code do not affect existing database code. Verifying Database Code by Using Unit Tests

Converting Manual Test Cases to Automated Tests to Run As Part of a Test Plan You can convert manual test cases to automated tests by adding the details for the automated test to the test case using Visual Studio. Or you can create a test case and add an automated test if you want to run these tests as part of a test plan using Microsoft Test Manager. How to: Associate an Automated Test with a Test Case Defining Your Testing Effort Using Test Plans

Managing Your Automated Tests You can organize automated tests by marking test methods with a test category attribute. You can then use these groupings to select the tests that you want to run. Or you can convert a test

62

case into an automated test and manage these tests using test plans and suites. You can also use test lists to manage your automated tests. Defining Test Categories to Group Your Tests Organizing Test Cases Using Test Suites Defining Test Lists to Group Your Tests

Running Automated Tests from a Test Plan

This is a guide for how to run automated tests as part of your test plan using Visual Studio Ultimate or Visual Studio Test Professional. The guide contains a checklist of the essential tasks that you must perform before you can run your automated tests using Microsoft Test Manager. The following illustration shows how automated tests might be run for a multi-tier application using a test controller and test agents. For a test plan, you have to use an environment with your test controller to run automated tests. An environment is the set of roles that are required to run a specific application and the machines to be used for each role. The set of roles used in the following illustration are Web Server, Database Server and Client.

When the tests are run as part of a test plan, you select test settings and environments for your test plan as shown in the following illustration. You configure a test plan using the Properties view in the Plan section of the Testing Center.

63

Best Practice Checklist

Determine what roles you need in your environment For example, it could be a Web server and a desktop client that runs a browser. Then install a test controller and register it with your Team Foundation Server. Install test agents on the computers that you want to use as part of your physical environment for testing. These computers can be physical or virtual machines. Setting Up Test Machines to Run Tests or Collect Data Installing and Configuring Visual Studio Agents and Test and Build Controllers

Create a physical environment

Create a test plan A test plan lets you specify what you want to test and how to run those tests. A test plan can be applied to a specific iteration of your project. You can have just one default test suite for your test cases, or you can create a test suite hierarchy. You can also select the default configurations to use to run the tests in your test plan. The test configuration informs the tester of the set up that is required for these tests. You can have one or multiple default configurations. A test result is recorded every time that you run the test with a specific configuration. A test plan enables you to measure your testing progress, based

64

on these test results for the test and configuration pairings, as shown in the following illustration. Create automated test settings for your test plan with a set of roles that match your physical environment. Add the test settings and environment to your test plan. Create Test Settings for Automated Tests as Part of a Test Plan

Create automated tests using Microsoft Visual Studio 2010. These tests might be unit tests or coded UI tests. Creating and Running Unit Tests for Existing Code Testing the User Interface with Automated UI Tests

65

Associating automated test with a Test Case

You might begin your testing cycle by creating a manual test case that you decide later is a good test to automate. You want to be able to continue to run that test as part of a test plan. Or you might have a manual test case that you used to create an action recording when you ran your test. Now you have created a coded UI test from that action recording and you want to associate this coded UI test with the original test case and run it as an automated test. For more information about action recordings and coded UI tests, see Recording and Playing Back Manual Tests and Testing the User Interface with Automated UI Tests. You associate an automated test with a test case using Visual Studio. After you create your automated test, you must check in your test project that contains the automated test and make sure that the test project is part of your build definition. Then you associate the automated test with a test case that has been added to a test plan that uses this build. For more information about checking in your test project and team build, see Add Files to Version Control and Building the Application. Also, you might have created a set of automated tests that you want to run as part of a test plan using Microsoft Test Manager. You can import these automated tests from an assembly and create test cases associated with each test using a command line tool for Microsoft Test Manager. For more information about how to import automated tests, see How to: Create Test Cases from an Assembly of Automated Tests Using tcm.exe. You can only associate the automated test with your test case using Visual Studio. You cannot make this association using Microsoft Test Manager. If you have an existing test case that you want to use, you must first open the test case using Visual Studio as shown in the following illustration.

66

Then you can associate the test method with your test case as shown in the following illustration.

If you need to create a new test case, then you can do this from the Test View window as described in the following procedure Create a Test Case From an Automated test. You must then add this test case to a test plan to be able to run it.

67

Note You can view the information from Microsoft Test Manager, but you cannot modify it.

Testing Scenarios
The following table shows the types of automated tests that you can create and which types of tests can be run as part of a test plan: Type of test Description Tests the user interface by performing UI actions. Tests code at the method level. Tests a stored procedure, function or trigger in a database. Tests application performance and stress using unit tests, Web Performance tests or coded UI tests. Used as part of load tests to test server responses for Web applications by sending and receiving http requests. Test functionality using API calls or command line tools for the application under test. Run from a test plan by associating a test method with a test case Yes Yes Yes Not recommended because you cannot view the test as it runs.

Coded UI Tests Unit Tests Database Unit Tests Load tests

Web Performance Tests Generic Tests

Not recommended because Web Performance tests are primarily intended to be used with load tests. Yes

Use the following procedures to associate an automated test with a test case that you can then run as part of your test plan:

Add the automated test to your existing test case Create a test case from an automated test Set up your test plan to use your build Create your test settings and environment to run your tests Run the automated test using Microsoft Test Manager View and update the test results

68

LESSON 4: Testing the Application

You can now be more productive throughout your testing lifecycle of planning, testing and tracking your progress by using Visual Studio Ultimate or Visual Studio Test Professional. These testing tools are integrated with Team Foundation Server, which lets you define your testing based on the same team projects that other areas of your organization are using. Both Microsoft Visual Studio 2010 Ultimate and Visual Studio Test Professional 2010 now include a new application called Microsoft Test Manager to help you define and manage your testing effort by using test plans. You create a test plan and add any test suites, test cases, or configurations that you need, as shown in the following illustration. The configurations are used to determine which set ups you want to use to run test suites or test cases:

When you have defined all these, you are ready for testing. When requirements or user stories, or features are ready to be tested, you can run your tests for each configuration that you specified. This plan enables you to measure your progress as you run your tests and report on how much testing remains. You can run manual tests from Microsoft Test Manager using the Microsoft Test Runner. You can also run automated tests from Microsoft Test Manager if the automation is associated with a test case. Results from running these tests will be associated with a test plan.

69

In addition, you can run automated tests from Visual Studio that are not associated with a plan. You can select to run the tests individually, as part of a check-in policy, or based on test categories. They can also be run as part of a build created by using Team Foundation Build, and from the command line. Because the testing tools are integrated with the other parts of Visual Studio Ultimate, you can save your test results to a database, generate trend and historical reports, and compare different kinds of data. For example, you might use the data to see how many and which bugs were found by your tests.

Planning your testing strategy

Use the following topics to help you with testing your application:

Planning your testing effort

You may decide to plan your testing based on either requirements or user stories. You can link your tests to these requirements or user stories and then plan which are the configurations that you want to use to run these tests. Alternatively, you can create test suites based on other criteria to use to plan your testing effort.

Creating and managing tests

You can create and manage both manual and automated tests as part of your plan. You can create test cases to be run as manual tests with individual test steps from Microsoft Test Manager. Automated tests, such as unit tests, load tests and automated UI tests, can be created by using Visual Studio. Later, if you want, you can convert test cases to automated tests by linking automation to the test case.

Setting up how to run tests or collect data remotely

You can use test settings and environments to set up what types of data to collect remotely, or how to impact the system, when you run your tests. For example, you may want to collect code coverage data, emulate a specific network connection, record your UI actions for a manual test, or collect diagnostic trace logs to help reproduce a complex bug.

Running Your Tests

You can run your tests from a test plan using Microsoft Test Manager and save these test results to use to measure your testing progress. You can run your tests using Visual Studio, or you can run your tests from the command line.

Submitting bugs
You can submit bugs with the detailed information that you collect when you run your tests using Test Runner. You can also submit bugs from Visual Studio or Microsoft Test Manager.

70

Tracking software quality

You can measure your testing readiness; track your testing progress and the status of bugs. The parameters in a test case are not used by any automated test that you associate with a test case. Iterations of a test case that use these parameters are for manual tests only.

EXERCISE:
1. To open the solution that contains the automated test, click File and point to Open and then click Project/Solution. The Open Project dialog box is displayed. 2. Select your solution, and then click Open. Note You must open your solution to be able to associate the automation with the test case. 3. If you know the work item of the test case, in Team Explorer, on the Team menu, click Go To Work Item. You can also right-click the Work Items node for your team project. 4. In the Go to Work Item dialog box, in the ID box, type the ID number of your test case. 5. Click OK to open the work item. Note You can also use a pre-existing query in the Team Queries or My Queries node to find the work item if you do not know the ID. For more information about how to use queries in Team Explorer, see Finding Bugs, Tasks, and Other Work Items. 6. To add the automated test, click the Associated Automation tab. 7. To find the automated test name, click the ellipsis (). The Choose Test dialog box is displayed. All the tests in the solution are shown in the list together with their associated test projects. Note If a test case already has an automated test associated with it, you must first remove this association before you can add a different automated test. Click Remove association to remove the existing automation. 8. Select the test, and then click OK. Note The value in Automation Status is automatically changed to Automated 9. To save the changes to the test case, click Save Work Item. The next part of the process is to set up your test plan to use your build.

71

LAB: Automated Test Case

EXERCISE 1:
To open the solution that contains the automated test, click File and point to Open and then click Project/Solution. The Open Project dialog box is displayed. Select your solution, and then click Open. Note You must open your solution to be able to associate the automation with the test case. To open the Test View window, click Test, point to Windows, and then point to Test View. The Test View window is displayed. To create a new test case for the automated test, right-click the test method in the Test View window and point to Create Test Case from Test. The fields in the Associated Automation tab are automatically filled in for the test method that you selected in the Test View window. Note The value in Automation Status is automatically changed to Automated Type the name of the test case in Title. To set the priority of the test case, click Priority. Note For more information about the fields in a test case, see How to: Create a Manual Test Case. To save the changes to the test case, click Save Work Item. You must add the test case to your test plan using Microsoft Test Manager. For more information, see How to: Add a Test Case to a Test Suite. The next part of the process is to set up your test plan to use your build. To set up your test plan to run the automated test that you have created, you must select the correct build definition used to build your automated test or a build definition that has the correct build drop location for your existing automated test assemblies. You must do this so that the automated test can be found in the share location for your build definition and then it can be run from Microsoft Test Manager. Important If you have multiple build configurations, the test assemblies to run the automated tests are searched for recursively from the root directory of the build drop folder. If it is important which assemblies are selected when you run your automated tests, you should use Run with options to specify the build configuration. For more information about how to do this, see How to: Override Settings in Your Test Plan For Test Runs

1.

2.

3.

4. 5.

6.

72

EXERCISE 2:
1. Open Microsoft Test Manager. Note To display the Microsoft Test Manager window, click Start, and then click All Programs. Point to Microsoft Visual Studio 2010 and then click Microsoft Test Manager 2010. 2. To select a test plan, click the down-arrow on the center group switcher and then click Testing Center. 3. On the center group menu bar, click Plan. 4. To set up your test plan to run the automated tests, click Properties and then click the drop-down arrow to the right of Filter for builds. The dialog box that shows build definition and quality is displayed. 5. To select the build definition that is used to build your automated tests, click Build definition. 6. Each build can be given a specific value to reflect the quality of the build. To select the quality of the builds you want to be able to view, click Build quality. 7. To save your changes, click Set build filter. 8. To select the most recent build to use with this test plan that includes the latest changes to the automated test, you must first click Save to save the plan and then click Modify. The Assign Build activity is displayed. You can compare your current build with a build you plan to take. The associated items list shows the changes to work items between the builds. You can then assign the latest build to take and use for testing with this plan. For more information about how to select a build, see Determining Which Builds Have Bug Fixes, New Features, or Requirements. 9. To close the Assign Build activity and return to the test plan properties, click the Close icon. 10. To save these changes for this test plan, click Save in the toolbar.

73

EXERCISE 3:
To run your automated tests, you must use a physical or virtual environment. You cannot run automated tests using Microsoft Test Manager without an environment. You must create an environment that contains the roles in your test settings and then use this environment in your test plan. For more information about how to create your environment and roles and test settings, see Setting Up Test Machines to Run Tests or Collect Data. Note If you want to run automated tests that interact with the desktop, you must set up your agent to run as a process instead of a service. For more information, see How to: Set Up Your Test Agent to Run Tests that Interact with the Desktop. 1. To create a physical environment, follow the steps in this topic: Creating a Physical Environment to Use for Testing. 2. If you are using Visual Studio Lab Management, you can create a virtual environment. To create a virtual environment, follow the steps in this topic: How to: Create an Environment from Virtual Machines or Templates. If you want more information about how to create virtual environments, see Virtual Environments Concepts and Guidelines. 3. To create your test settings, follow the steps in this topic: Create Test Settings for Automated Tests as Part of a Test Plan.

74

EXERCISE 4:
1. Open Microsoft Test Manager. Note To display the Microsoft Test Manager window, click Start, and then click All Programs. Point to Microsoft Visual Studio 2010 and then click Microsoft Test Manager 2010. 2. To run the automated test, click the down-arrow on the center group switcher and then click Testing Center. 3. On the center group menu bar, click Test. 4. (Optional) To override the build, the test settings or the environment to use for running the automated tests that you select in this test plan, right-click the test and then click Run with options. For example, if you want to run on a staging environment instead of your standard testing environment then you might select a different environment. From the Run options dialog box, you can change these settings, and then click Run to run the selected test. 5. To run the automated test without changing any options, right-click the test and then click Run. The Analyze Test Runs activity is displayed. It shows the progress of the test run that contains this test. The test result is displayed. It shows the details from the test run, the attachments for data collected for this test result, and the test results history for that test. You can close this view to return to the test run. 1. To save these changes for this test run, click Save in the toolbar.

75

Create and Configure Test Projects for Automated Tests

Test projects can contain tests of any type. You can create a test project explicitly. Or, if your solution contains no test projects when you create your first test, a test project is created automatically and the test is placed in it. Before you create a test project, you can choose to configure the way test projects are created. This means determining how many tests and what kinds of tests are automatically added to new test projects, in addition to the default language of the test project. To do this, you use the Options dialog box, which is available on the Tools menu. As described in the following procedures, you can create test projects in several ways: Create a test project when you add a test. When you create a new test, you can create new a test project into which the test will be added, or add the test to an existing test project that is already loaded.

Create a test project when you generate a unit test. When you generate unit tests, you can create a new test project into which the tests will be added, or add them to an existing test project. Using the Add New Project dialog box. With this method, you use a dialog box to choose the programming language of your test project. Configure test project creation. Set the programming language for new test projects, and whether one or more tests should be added to new test projects by default.

76

LAB 1: Create a Test Project When You Add a Test

1. Click Test and then click New Test. The Add New Test dialog box is displayed. 2. Under Add to Test Project, choose one of the options for creating a new test project. For example, click Create a new Visual C# Test Project. Note By default, the type of test project shown under Add to Test Project is the type that is currently set as the default in the Options dialog box, which is available on the Tools menu. 3. In the Templates pane, select the type of test that you want to add. 4. Click OK. The New Test Project dialog box is displayed. 5. Enter a name for the new test project, or accept the default name, and then click Create. 6. Depending on the type of test that you chose, you can either hand-code or generate the newly added test. 7. The test project and the test it contains are added to your solution.

77

LAB 2: Create a test project as you generate unit tests

To create a test project as you generate unit tests 1. Open a source-code file. 2. Right-click a namespace, class, or method. 3. Click Create Unit Tests. The Create Unit Tests dialog box is displayed. 4. (Optional) Change the selection of members for which you want to generate unit tests. 5. Under Output project, choose one of the options for creating a new test project. For example, click Create a new Visual C# Test Project. 6. Click Generate. The New Test Project dialog box is displayed. 7. Type a name for the new test project, or accept the default name, and then click Create. The test project is added to your solution together with the new unit test.

78

LAB 3: Create a test project by using the Add New Project dialog box
To create a test project by using the Add New Project dialog box 1. In Solution Explorer, right-click the solution. 2. Point to Add and then click New Project. The Add New Project dialog box is displayed. The Project types pane contains a Test Projects node and nodes for specific programming languages. 3. If you want your new test project to have a specific programming language, expand the node of that language. If not, expand Test Projects. Note Test projects are not available in every language. 4. If you expanded a language node, click Test. If you expanded the Test Projects node, click Test Documents. 5. Under Templates, click Test Project. 6. (Optional) Change the name or location of the project that you are about to add. 7. Click OK. The test project is added to your solution.

79

LAB 4: Configure Test Project Creation

1. On the Tools menu, click Options. The Options dialog box is displayed. 2. Expand Test Tools in the left pane and then click Test Project. 3. (Optional) Under Default test project language, choose a programming language for new test projects. 4. (Optional) You might want to make sure that your test project and the project that contains your code under test use the same language. If you do want them to use the same language, select the check box When automatically creating a test project, disregard the default language setting and use the language of the production-code project instead. 5. (Optional) You can choose to enable a feature that will cause your Test List and Test View windows to update automatically when you make changes to methods instead of having to click Refresh manually. Doing this has been known to cause some performance problems and other issues with VSMDI files. To enable this feature, clear the Disable background discovery of test methods check box. 6. Under Select the files that will be added to each new test project, by default , choose, by language, the type of test project that you want to configure. 7. Select the tests that you want added to test projects of the selected language as those projects are created. Note You can select zero tests. If you do select zero tests, each new test project is empty. This means that it contains, for example, a property page and a reference to a namespace, but no tests or source-code files. You can add tests later through the Add New Test dialog box. 8. (Optional) Repeat steps 4 and 5 for test projects in different languages. 9. Click OK. The settings are stored for use the next time that you create a test project. You can change these preferences at any time, before you create additional test projects.

80

81

Chapter 5
Tracking Software Quality

82

As a Tester you build out dashboards to show our progress on our software test projects, we quickly enjoy the luxury of Dashboard Views that utilize Windows SharePoint 2010 technology. This new web part connects to a URL on the TFS application site tier called tsfredirect.aspx that then will load a report from the report site. A tester can use Visual Studio Test Professional 2010 or Visual Studio 2010 Ultimate to help you track the quality of the applications that you are testing using Microsoft Test Manager. You can check which builds have bug fixes, new features, or new requirements or user stories added. You can also view recommendations about which tests might have to be rerun, based on the effect of code changes that have been made to your application if you are using test impact analysis when you run your tests. You can track bugs found in your application under test. You can assess your software quality, based on these bugs. You can triage these bugs to assess priorities, assign the bugs and determine whether the bugs can be fixed. It is also possible that the bugs need more information, or the bug is actually intended behavior for the application. In addition, you can view reports on the readiness of your test cases, view the progress that you are making in your test plan, and run queries to review specific data for your team project that relates to requirements or user stories, test cases and bugs. If your team uses a project portal, you can view the predefined reports on the Test Dashboard. You can access the project portal from the Track view in the Testing Center for Microsoft Test Manager. For more information about the Test Dashboard, see Test Dashboard (Agile).

Test Dashboard (Agile)

By using the Test dashboard, you can monitor test activities, report on progress, find gaps in test coverage, and identify test areas that may require additional investigation. This dashboard displays five reports that provide information about testing that has occurred in the most recent four weeks. Note You access dashboards through your team project portal. You can access the Test dashboard only if that portal has been enabled and is provisioned to use Microsoft Office SharePoint Server 2007 or 2010. For more information, see Dashboards (Agile) or Access a Team Project Portal and Process Guidance

83

SharePoint 2010 Developers Dashboards

Developer Dashboard is a new feature in SharePoint 2010 that is designed to provide additional performance and tracing information that can be used to find and debug more issues. Developer dashboard is turned off by default. This can be enabled using STSADM utility or through the object model or SharePoint Power Shell. Developer dashboard is one of the coolest features in SharePoint 2010 and loved by the developer community. Developer dashboard can help in diagnosing various issues in a custom code. There are many types of dashboard such as: My Dashboard (Agile) Progress Dashboard (Agile) Bugs Dashboard Build Dashboard (Agile) Quality Dashboard (Agile) Test Dashboard (Agile) Project Dashboard (Agile)

Agile teams can quickly find important information about their team projects by using dashboards. Dashboards show project data, support investigation, and help teams perform common tasks more quickly.

Counters in developer dashboard

Per-Thread counters Gives the following information of current thread with actual values Execution Time SPRequests Allocated Number of SQL queries performed Duration of SQL query Query Text URL Current User Start Time Pre-Process counters Global Heap Size Total size of all native heaps Native heap count Active native heap count Count of SharePoint 2010 operation in progress

84

Client Counters Page Size System Counter Machine's available memory Hard Page faults Processor Utilization Heap size of managed memory

How to use Monitor in Code?

Monitoring code is easy very easy with help of developer dashboard. First thing identify the portion of the code need real monitoring and wrap that code SPMonitorScope.

using (new SPMonitoredScope("MyTestScope")) { //Code which needs to be monitored goes here } Another approach implement ISPScopedPerformanceMonitor Interface in your custom class and use it. Once the developer dash board is turned on the information regarding execution of queries, function calls used in rendering the page; to enable the developer dashboard through the object model the Tester must create a new application to trigger the following code: SPWebService cs = SPWebService.ContentService; cs.DeveloperDashboardSettings.DisplayLevel = SPDeveloperDashboardLevel.On; cs.DeveloperDashboardSettings.Update();

85

Developer Dash Board in a custom Page

<Sharepoint:DeveloperDashboardLauncher ID="DeveloperDashboardLauncher" NavigateUrl="javascript:ToggleDeveloperDashboard()" runat="server" ImageUrl="/_layouts/images/fgimg.png" Text="<%$Resources:wss,multipages_launchdevdashalt_text%>" OffsetX=0 OffsetY=78 Height=16 Width=16 /> < div id="DeveloperDashboard" class="ms-developerdashboard" /> < SharePoint:DeveloperDashboard runat="server" />

SPDeveloperDashboardLevel has three set of dashboard levels by default: 1. ON - Developer dash board will always be active on the SharePoint farm

2. OFF - Dashboard will remain Inactive throughout the SharePoint farm

3. ON Demand - these options lets the end user activate and deactivate the developer dash board based on the choice of the developer/tester/end-user.

The last one is the most preferred option as usually people would not like the developer dashboard taking extra screen space on the SharePoint 2010 pages. When the On Demand option is set through the code then a small icon appears across the right corner of the SharePoint site pages and using that the Developer Dashboard can be set to on or off.

STSADM Utility
The following code example shows how a Testing Developer might enable the Developer Dashboard through STSADM Utility: Stsadm o setproperty pn developer-dashboard pv ondemand (or on or off)

86

Developer Dashboard is an excellent feature for Automated UI Test in SharePoint 2010 Development. Though Developer Dashboards cannot trace the events of a custom Sandboxed solution but still provides ample information to boost the performance of SharePoint pages.

PowerShell Commands
PowerShell Commands for Dashboards in SharePoint 2010 Dashboard Developer. $DevDashboardSetting = [Microsoft.SharePoint.Administration.SPWebservice}::ContentService.DeveloperDashbo ardSettings; $DevDashboardSettings.DisplayLevel = OnDemand; $DevDashboardSettings.RequiredPermissions = EmptyMask; $DevDashboardSettings.TraceEnable = $true; $DevDashboardSettings.Update()

87

LAB 1: Create the Portal in SharePoint and then fix up the missing documents
EXERCISE 1: Create new portal site
1. 2. Go to the SharePoint site of your project collection)

3. Click on the Site Actions at the left side of the screen and choose the option Site Settings

4. In the site settings, choose the Sites and workspaces option 5. Create a new site 6. Enter the values for the Title, the description, the site address. And choose for the TFS2010 Agile Dashboard as template. 7. Create the site, by clicking on the Create button

88

Integrate portal site with team project

1. Open Visual Studio 2010 2. Open the Team Explorer (View -> Team Explorer) 3. Select in the Team Explorer tool window the Team Project for which you are create a new portal 4. Open the Project Portal Settings (Team -> Team Project Settings -> Portal Setings...)

5. Check the Enable team project portal checkbox 6. Click on Configure URL.. . 7. You will get a new dialog as below 8. Enter the url to the TFS server in the web application ComboBox 9. And specify the relative site path: sites/<project collection>/<site name>

89

10. Confirm with OK 11. Check in the Project Portal Settings dialog the checkbox "Reports and dashboards refer to data for this team project" 12. Confirm the settings with OK (this takes a while...) 13. When you now browse to the portal, you will see that the dashboards are now showing up with the data for the current team project. We will fix that in the next step.

Download process template

1. To get a copy of the documents that are default in a team project, we need to have a fresh set of files that are not attached to a team project yet. You can do that with the following steps. 2. Start the Process Template Manager (Team -> Team Project Collection Settings -> Process Template Manager...) 3. Choose the Agile process template and click on download 4. Choose a folder to download

Add Product and Iteration backlog

90

1. Go to the Team Explorer in Visual Studio 2. Make sure the team project is in the list of team projects, and expand the team project 3. Right click the Documents node, and choose New Document Library

4. Enter "Shared Documents", and click on Add

5. Right click the Shared Documents node and choose Upload Document 6. Go the the file location where you stored the process template from step 3 and then navigate to the subdirectory "Agile Process Template 5.0\MSF for Agile Software Development v5.0\Windows SharePoint Services\Shared Documents\Project Management" 7. Select in the Open Dialog the files "Iteration Backlog" and "Product Backlog", and click Open

Bind Iteration backlog workbook to the team project

1. Right click on the "Iteration Backlog" file and select Edit, and confirm any warning messages 2. Place your cursor in cell A1 of the Iteration backlog worksheet 3. Switch to the Team ribbon and click New List. 4. Select your Team Project and click Connect 5. From the New List dialog, select the Iteration Backlog query in the Workbook Queries folder. 6. The final step is to add a set of document properties that allow the workbook to communicate with the TFS reporting warehouse. Before we create the properties we need to collect some information about your project. The first piece of information

91

comes from the table created in the previous step. As you collect these properties, copy them into notepad so they can be used in later steps.

Property [Table name] [Project GUID]

[Team Project name] [TFS server name]

How to retrieve the value? Switch to the Design ribbon and select the Table Name value in the Properties portion of the ribbon In the Visual Studio Team Explorer, right click your Team Project and select Properties. Select the URL value and copy the GUID (long value with lots of characters) at the end of the URL In the Properties dialog, select the Name field and copy the value In the Properties dialog, select the Server Name field and copy the value [UPDATE] I have found that this is not correct: you need to specify the instance of your SQL Server. The value is used to create a connection to the TFS cube.

7. Switch back to the Iteration Backlog workbook. 8. Click the Office button and select Prepare Properties. 9. Click the Document Properties Server drop down and select Advanced Properties. 10. Switch to the Custom tab and add the following properties using the values you collected above. Variable name [Table name]_ASServerName [Table name]_ASDatabase [Table name]_TeamProjectName [Table name]_TeamProjectId 11. Click OK to close the properties dialog. 12. It is possible that the Estimated Work (Hours) is showing the #REF! value. To resolve that change the formula with: =SUMIFS([Table name][Original Estimate]; [Table name][Iteration Path];CurrentIteration&"*";[Table name][Area Path];AreaPath&"*";[Table name][Work Item Type]; "Task") For example Value [TFS server name] tfs_warehouse [Team Project name] [Project GUID]

92

=SUMIFS(VSTS_ab392b55_6647_439a_bae4_8c66e908bc0d[Original Estimate]; VSTS_ab392b55_6647_439a_bae4_8c66e908bc0d[Iteration Path];CurrentIteration&"*";VSTS_ab392b55_6647_439a_bae4_8c66e908bc0d[Area Path];AreaPath&"*";VSTS_ab392b55_6647_439a_bae4_8c66e908bc0d[Work Item Type]; "Task") 13. Also the Total Remaining Work in the Individual Capacity table may contain #REF! values. To resolve that change the formula with: =SUMIFS([Table name][Remaining Work]; [Table name][Iteration Path];CurrentIteration&"*";[Table name][Area Path];AreaPath&"*";[Table name][Assigned To];[Team Member];[Table name][Work Item Type]; "Task") For example =SUMIFS(VSTS_ab392b55_6647_439a_bae4_8c66e908bc0d[Remaining Work]; VSTS_ab392b55_6647_439a_bae4_8c66e908bc0d[Iteration Path];CurrentIteration&"*";VSTS_ab392b55_6647_439a_bae4_8c66e908bc0d[Area Path];AreaPath&"*";VSTS_ab392b55_6647_439a_bae4_8c66e908bc0d[Assigned To];[Team Member];VSTS_ab392b55_6647_439a_bae4_8c66e908bc0d[Work Item Type]; "Task") 14. Save and close the workbook.

Bind Product backlog workbook to the team project

1. Repeat the steps for binding the Iteration backlog for thiw workbook too. 2. In the worksheet Capacity, the formula of the Storypoints might be missing. You can resolve it with: =IF([Iteration]="";"";SUMIFS([Table name][Story Points];[Table name][Iteration Path];[Iteration]&"*")) Example =IF([Iteration]="";"";SUMIFS(VSTS_487f1e4c_db30_4302_b5e8_bd80195bc2ec[Story Points];VSTS_487f1e4c_db30_4302_b5e8_bd80195bc2ec[Iteration Path];[Iteration]&"*"))

93

Chapter 6
Create a Coded UI Test

94

Introduction
Using Visual Studio Premium or Visual Studio Ultimate, you can create a coded UI test that can test that the user interface for an application functions correctly. The coded UI test performs actions on the user interface controls for an application and verifies that the correct controls are displayed with the correct values. For more information about which platforms and configurations are supported by coded UI tests, see Supported Configurations and Platforms for Coded UI Tests and Action Recordings. Creating a coded UI test generates a UIMap object that is specific to your test and represents the windows, controls, parameters, and assertions that are in the UI or that you created during the test recording. You can then perform actions on these UI objects to automate your user interface. For example, you can have your test method click a hyperlink in a Web application, type a value in a text box, or branch off and take different testing actions based on a value in a field. Note You can add multiple coded UI tests and multiple UI map objects and files to facilitate testing a large application. For more information, see Testing a Large Application with Multiple UI Maps. A coded UI test class is identified by a CodedUITestAttribute applied to the class. Each coded UI test is a test method in a coded UI test class. You can add multiple test methods to each coded UI test class and identify each coded UI test method by using the TestMethodAttribute. Your test method can also add validation code for a UI test control to obtain the value of a property of a UI test control. The test method can use an Assert statement to compare the actual value of the property to an expected value. The result of this comparison determines the outcome of the test. Every time that you run a coded UI test, you can analyze the test result and if the test fails, you can see or store the details of which assertion failed. When you create a coded UI test, these files are added to your test project: File CodedUITest1.cs UIMap.uitest Description Contains the coded UI test class, test methods and assertions. Contains the XML model for the UIMap class, including all windows, controls, properties, methods, parameters, actions, and assertions. UIMap.Designer.cs Contains the code representation of the XML contained in the UIMap.uitest file. Do not edit this file. UIMap.cs Contains more of the code for the UIMap class. You can put any customizations for the UI map in this file.

95

These assemblies are added as references to your test project:

Microsoft.VisualStudio.QualityTools.CodedUITestFramework Microsoft.VisualStudio.QualityTools.UnitTestFramework Microsoft.VisualStudio.TestTools.UITest.Common Microsoft.VisualStudio.TestTools.UITest.Extension Microsoft.VisualStudio.TestTools.UITesting

96

LAB: Create a coded UI test

1. Perform any one of the following tasks: a. In Solution Explorer, right-click a test project, point to Add, and then click Coded UI Test. b. In the Test List Editor or the Test View window, right-click the window, and then click New Test. In the Add New Test dialog box, click Coded UI Test and then click OK. c. On the Test menu, click New Test. In the Add New Test dialog box, click Coded UI Test and then click OK. The Generate Code dialog box appears. 2. From this dialog box, you can select the method that you want to use to create the UI test controls in your coded UI test: Methods Action Next steps Record actions in your Click Record actions, How to: Generate a Coded UI application under test, edit UI map or add Test by Recording the modify the UI map or add assertions Application Under Test assertions Use an existing action Click Use an existing How to: Generate a Coded UI recording action recording Test from an Action Recording 3. All these methods create a coded UI test class, which has a [CodedUITest] attribute, in your test project and opens the file for this class. The class is populated with a test method that has a [TestMethod] attribute. Solution Explorer displays the new test file in your test project. Note If you click Cancel, a coded UI test class is also created. You can manually add code to this class to create your coded UI tests. Or, you can use the other methods that are listed in the table by following the steps in the procedures in the Next Steps column.

97

Coding UI Test
The subsequent code example illustrates a coded UI test class and test method that is assigned a test category value of Priority1. The test method is for a simple calculator application that adds two numbers and verifies that they are added together correctly for this test to pass. The code for the two UI map methods, AddTwoNumbers() and AssertForAdd(), are in the noneditable portion of the UI map and cannot be edited directly. However, you can copy the generated code from the UI map to modify it and create alternate or additional methods in the MyCodedUITest class. See module on Best Activities for Coded UI Tests covered later in this book. After you create your coded UI test with specific data, you might want to run your coded UI test several times with different sets of data to test different conditions. To do this you can add parameters from a data source to your coded UI test to create a data-driven coded UI test. This topic will be covered in a later module Creating a Data-Driven Coded UI Test [CodedUITest] public class MyCodedUITest { public CodedUITest() { } [TestCategory("Priority1"), TestMethod] public void CodedUITestMethod1() { // To generate code for this test, select "Generate Code" from // the shortcut menu and select one of the menu items. this.UIMap.AddTwoNumbers(); this.UIMap.AssertForAdd(); } }

98

LAB: Generating a coded UI test by recording the application under test

1. In Solution Explorer, right-click a test project, point to Add, and then click Coded UI Test. - or In the Test View window, right-click the surface of the window and then click New Test. In the Add New Test dialog box, click Coded UI Test and then click OK. - or On the Test menu, click New Test. In the Add New Test dialog box, click Coded UI Test and then click OK. The New Test Project dialog box appears. 2. Type a name for the new coded UI test and click Create. 3. Click Record actions, edit UI map or add assertions. The Coded UI Test Builder dialog box appears. 4. To start recording, click the Record icon. Perform the actions that you want to record in your application including starting the application if required. Note By installing Microsoft Visual Studio 2010 Feature Pack 2, you can use Windows Internet Explorer 7 (or later versions) to record UI actions on a website or a Web-based application and then play back the tests using the Mozilla Firefox browser version 3.5 or 3.6. To download the feature pack, you must have either Visual Studio 2010 Ultimate, Visual Studio 2010 Premium or Test Professional 2010 with an MSDN subscription, Microsoft BizSpark, or MSDN Academic Alliance. For more information, see Recording Tests Using Windows Internet Explorer and Playing Back Using Mozilla Firefox and Microsoft Visual Studio 2010 Feature Pack 2. 5. To view the actions, click the Show Recorded Steps icon. The actions are displayed in the Coded UI Test Builder - Recorded Actions dialog box. Note If you do not want to record the actions to start your application under test, you must start your application before you click the Record icon. 6. To finish recording, click the Generate Code icon. Type a name for your coded UI test method in Method Name, and then click Add and Generate. This generates code as follows if the name that you entered is, for example, AddTwoNumbers:

99

Adds the controls and to your UI map (UIMap.uitest) Adds a method called AddTwoNumbers to your UI map. You can view the method in the UIMap.Designer.cs file. This method performs the actions that you recorded when you run the test. public void AddTwoNumbers() { #region Variable Declarations WinEdit textInput1Edit = this.DemoCalculatorWindowWindow.InputNumber2Window.TextInput1Edit; WinEdit textInput2Edit = this.DemoCalculatorWindowWindow.TextInput2Window.TextInput2Edit; WinButton addButton = this.DemoCalculatorWindowWindow.AddWindow.AddButton; #endregion // Launch '%USERPROFILE%\Desktop\SimpleWinformsCalculator.exe' ApplicationUnderTest demoCalculatorWindowWindow = ApplicationUnderTest.Launch( this.AddTwoNumbersParams.DemoCalculatorWindowWindowExePath, this.AddTwoNumbersParams.DemoCalculatorWindowWindowAlternateExePath); // Type '3' in 'textInput1' text box textInput1Edit.Text = this.AddTwoNumbersParams.TextInput1EditText; // Type '4' in 'textInput2' text box textInput2Edit.Text = this.AddTwoNumbersParams.TextInput2EditText; // Click 'Add' button Mouse.Click(addButton, new Point(83, 18));

} Adds a test method to your coded UI test file that calls your AddTwoNumbers method [TestMethod] public void CodedUITestMethod1() { // To generate code for this test, select "Generate Code" // from the shortcut menu and select one of the menu items. this.UIMap.AddTwoNumbers(); } When you choose a name for the recorded method, choose a name that is descriptive for the actions that you recorded. 7. If the actions that you recorded were not what you want to keep, you can click the Show Recorded Steps icon. Select the actions that you do not want to keep and click the Delete icon.

100

8. To generate assertions for your UI controls, click the crosshairs icon and drag it to the control that you want to verify is correct. The Coded UI Test Builder - Add Assertions dialog box is displayed. 9. Click the vertical bar to view the UI map. The UI control that you want to verify should be highlighted. 10. Right-click the property for the UI control that you want to verify, and then point to Add Assertion. The Add assertion dialog box is displayed. 11. Select the Comparator for your assertion. 12. Type the value for your assertion in Comparison Value. 13. To add the assertion, click OK. 14. When you have added all your assertions for your test, close the Coded UI Test Builder Add Assertions dialog box. 15. To generate the code for your assertions, click the Generate Code icon. The Coded UI Test Builder - Generate Code dialog box is displayed. 16. Type a name for your coded UI test method in Method Name, and then click Add and Generate. This generates code as follows if the name you entered is, for example, AssertForAddTwoNumbers:

Adds a method called AssertForAddTwoNumbers to your UI map (UIMap.uitest). You can view the method in the UIMap.Designer.cs file. This method performs the assert statements that you added. public void AssertForAddTwoNumbers() { #region Variable Declarations WinEdit textAnswerEdit = this.DemoCalculatorWindowWindow.AnswerWindow.TextAnswerEdit; #endregion // Verify that the 'textAnswer' text box's Text property // is '40' Assert.AreEqual( this.AssertForAddTwoNumbersExpectedValues.TextAnswerEditText, textAnswerEdit.Text);

101

} Adds a call to the assert method AssertForAddTwoNumbers to the test method in your coded UI test file [TestMethod] public void CodedUITestMethod1() { // To generate code for this test, select "Generate Code" // from the shortcut menu and select one of the menu items. this.UIMap.AddTwoNumbers(); this.UIMap.AssertForAddTwoNumbers(); }

When you choose a name for the method that has your assert statements, choose a name that is descriptive for these assertions that you created. 17. Click the Close icon to close the Coded UI Test Builder. 18. (Optional) To add code to start your application when the coded UI test runs, record the startup sequence for your application and save that to a method. You can call that method at the start of your test. Note You can add a test initialize method, identified by a [TestInitialize] attribute, which runs code at the start of each test method. For example, the method to start the application could be called from the TestInitialize method. 19. (Optional) To add code to close your application when the coded UI test runs, record the closing sequence for your application and save that to a method. If you do not close the browser or application, it will remain open after your test is finished. Note You can add a test cleanup method, identified by a [TestCleanup] attribute, which runs code at the end of each test method. For example, the method to close the application could be called from the TestCleanup method. 20. To run the test, right-click in the test method, and then click Run Tests. For more information about how to run coded UI tests, see Running Automated Tests. Note After you create your coded UI test with specific data, you might want to run your coded UI test several times with different sets of data to test different conditions. To do this you can add parameters from a data source to your coded UI test to create a datadriven coded UI test. For more information, see How to: Create a Data-Driven Coded UI Test.

102

21. You can now optionally add additional controls and validate them using the UI Test Builder. For more information, see How to: Add UI Controls and Validation Code Using the Coded UI Test Builder.

103

Chapter 7
UI Controls and Validation Code Using the Coded UI Test Builder

104

You can use the Coded UI Test Builder to add a user interface (UI) control to the UIMap for your test, or to generate code for a validation method that uses an assertion for a UI control. Add UI Controls to an Existing Coded UI Test You can add a UI control for the application under test to an existing UI map and generate code to add the control to your coded UI test. To add UI a control to a coded UI Test 1. In Solution Explorer, open a coded UI test source file from a test project in your solution. The file name is CodedUITest1.cs). This is the same as the test name. 2. To start the Coded UI Test Builder, right-click in the CodedUITestMethod1() method, select the Generate Code for Coded UI Test command. In the shortcut menu, select the Use Coded UI Test Builder command. The Coded UI Test Builder dialog box appears. 3. Start your application under test and navigate to the UI control that you want to add. 4. Drag the crosshairs button in the Coded UI Test Builder to the UI control in your application that you want to test. When the box outlines your control, release the mouse. The control class code is immediately created in the UIMap.Designer.cs file.

The properties for this control are now listed in the Coded UI Test Builder - Add Assertions dialog box. To see the UI controls in the map, click the arrow (<<) to expand the view for the UI control map. To find a parent, sibling, or child control, you can click on the map and use the arrow keys to move around the list of controls.

105

Note You can also locate a UI control by pressing the Windows logo key + I. This is useful for testing a shortcut menu where the control is dismissed when you change the focus to the Coded UI Test Builder. For more information, see How to: Select a Control Using the Keyboard in Coded UI Tests. 5. Close the Coded UI Test Builder or go to the next procedure to add validation assertions. 6. To run the test, right-click in the test method, and then click Run Tests. For more information about how to use UI control actions and set properties, see How to: Use UI Control Actions and Properties in Your Coded UI Tests. Verify UI Controls in a Coded UI Test After you add a UI control for the application under test to the UI map in your coded UI test, then you can use the Coded UI Test Builder to create a validation method to validate properties of the added UI control. To add and verify controls in coded UI Test 1. In Solution Explorer, open a coded UI test source file from a test project in your solution. 2. To start the Coded UI Test Builder, right-click in the CodedUITestMethod1() method, select the Generate Code for Coded UI Test command. In the shortcut menu, select the Use Coded UI Test Builder command. The Coded UI Test Builder dialog box appears. 3. Start your application under test and navigate to the UI control that you want to add.

106

4. To add a UI control to the UI map, drag the crosshairs to the UI control in your application that you want to test. When the box outlines your control, release the mouse. The properties for this control are now listed in Coded UI Test Builder - Add Assertions dialog box. To locate the UI control in the map, click the arrow to view the UI control map. Note You can also locate a UI control by pressing the Windows key + I. This is useful for testing a shortcut menu where the control is dismissed when you change the focus to the Coded UI Test Builder. To find a parent, sibling, or child control, you can use the arrows in the dialog box. 5. Click the Add control to UI Control Map button in the toolbar to add this control to the UI map. 6. Right-click the property that you want to verify, and then right-click and select the Add Assertion command. The Add Assertion dialog box appears.

7. Select the Comparator for your assertion: AreEqual. 8. Type the value for your assertion in Comparison Value and click OK to add it . 9. When you have added all your assertions for your test, close the Coded UI Test Builder Add Assertions dialog box. 10. To generate the code for your assertions and add the control to the UI map, click the Generate Code icon. The Coded UI Test Builder - Generate Code dialog box appears. 11. Type a name for your coded UI test method in Method Name, then click Generate. For more information about the anatomy of the generated code, see Anatomy of a Coded UI Test. 12. Click the Close icon to close the Coded UI Test Builder.

107

13. To run the test, right-click in the test method, and then click Run Tests. .Note After you create your coded UI test with specific data, you might want to run your coded UI test several times with different sets of data to test different conditions. To do this you can add parameters from a data source to your coded UI test to create a datadriven coded UI test. For more information, see How to: Create a Data-Driven Coded UI Test.

108

Chapter 8
Data-Driven Coded UI Test

109

Using Visual Studio Premium or Visual Studio Ultimate, you can create a coded UI test that tests whether the user interface (UI) for an application functions correctly. The coded UI test performs actions on the user interface controls for an application and verifies that specified controls display the correct values. For more information about how to create a coded UI test, see How to: Create a Coded UI Test. After you create your coded UI test with specified data, you might want to run your coded UI test multiple times with different sets of data to test different conditions. To do this you can add parameter values from a data source to create a data-driven coded UI test. Each row in the data source is an iteration of the coded UI test. The overall result of the test will be based on the outcome for all the iterations. For example, if one test iteration fails, the overall test result is failure. When you generate methods or assertions for your coded UI test by recording them, all constants referred to in the recorded methods are parameterized into parameter classes. For example, if you add a method named AddTwoNumbers, the coded UI testing framework generates another class named AddTwoNumbersParams and adds a field to this class for each constant value that you used when recording the method. If you generate an assertion method named AssertForAdd, the framework generates a class named AssertforAddExpectedValues for the expected values. These classes are generated automatically and are used in the following steps. All changes to your test to enable it to use a data source for the parameters will be made to your test's source code file (i.e. CodedUITest1.cs). You cannot modify the code in the UIMap.Designer.cs file. After you have created a coded UI test, you can use the steps in the following procedure to add your data source and parameters to your test. This example assumes that you have these code elements in your solution:

A coded UI test class named CodedUITest1. A test method named CodedUITestMethod1().

The test method is for a simple calculator application that adds two numbers and verifies that they are added together correctly for this test to pass. The data source is a .csv file that contains the following data: Input1 3 5 1200 Input2 4 6 24000 ExpectedResult 7 11 25200

110

To test the functionality of any application, we test it with different identified sets of data to make sure the functionality does not break while iterating through all these data sets. Coded UI supports 3 different datasources that can be used for providing the test data. 1. 2. 3. CSV files Xml files Databases

Configuring the UI Test DataSource

Please refer to my previous article data driven approach in coded UI to see the steps to perform before configuring the DataSource. Configuring the XML file as the DataSource 1. Select the XML File as the DataSource and click on the Next button.

2. Choose the XML file from the next screen. 3. You will see the data present in XML file as below:

111

4. Clicking Next button will show up a screen with different data tables.

If you have a look at the xml file which I used as the source of test data, it contains two different data tables named Search and Results as shown below:

5. Click on the Finish button to complete the configuration. Clicking on Finish will show a pop up to copy the database file into the current project.

112

6. Click on Yes button and it will copy the XML file in the current project.

Next all steps will be same as discussed in my pervious article. I am reiterating the steps again. 6. Open the CodedUITestMethod1 and notice the attribute on top of the method: [DeploymentItem("SampleCodedUIProject\\XMLFile1.xml"), DataSource("Microsoft.VisualStudio.TestTools.DataSource.XML", "|DataDirectory|\\XMLFile1.xml", "Search", DataAccessMethod.Sequential), TestMethod] public void CodedUITestMethod1() { string testData = TestContext.DataRow["SearchData"].ToString(); this.UIMap.RecordedMethod1(testData);

113

} The information related to the DataSource get added on top of the method. 7. Now, modify the CodedUITestMethod1 a bit to parameterize the test using TestContext. Add the following line in the test method: string testData = TestContext.DataRow ["SearchData"].ToString(); this.UIMap.RecordedMethod1 (testData);

8. As we have changed RecordedMethod1 to take one input parameter testData, we will change its implementation a bit. Go to the method definition and supply the testData where we were taking the Visual Studio 2010 as a hardcoded input. See the below highlighted line, we will change this line to take testData as the value. public void RecordedMethod1(string testData) { #region Variable Declarations HtmlEdit uIEnteryoursearchtermEdit = this.UIExercise3DataDrivenDWindow.UIBingDocument.UIEnteryoursearchtermEdit; HtmlDiv uIItemPane = this.UIExercise3DataDrivenDWindow.UIBingDocument.UISb_formCustom.UIItemPa n; HtmlInputButton uISearchButton = this.UIExercise3DataDrivenDWindow.UIBingDocument.UISearchButton; WinButton uICloseButton = this.UIExercise3DataDrivenDWindow.UIVisualstudio2010BingTitleBar.UICloseButton; #endregion this.UIExercise3DataDrivenDWindow.LaunchUrl(new System.Uri(this.RecordedMethod3Params.UIExercise3DataDrivenDWindowUrl)); Playback.PlaybackSettings.ContinueOnError = true; // Mouse hover 'Enter your search term' text box at (213, 5) Mouse.Hover(uIEnteryoursearchtermEdit, new Point(213, 5)); Playback.PlaybackSettings.ContinueOnError = false; // Click 'Unknown Name' pane Mouse.Click(uIItemPane, new Point(151, 34));

114

// Type 'visual studio 2010' in 'Unknown Name' pane Keyboard.SendKeys(uIItemPane, this.RecordedMethod3Params.UIItemPaneSendKeys, ModifierKeys.None); uIEnteryoursearchtermEdit.Text = testData; Mouse.Click(uISearchButton, new Point(21, 20)); // Click 'Search' button Mouse.Click(uICloseButton, new Point(18, 8)); // Click 'Close' button } Change the highlighted line as follows: uIEnteryoursearchtermEdit.Text = testData; 9. Now we are ready to run our test with different testData values as supplied from the XML file for the Search table. Run your test case, the test will run as many times the data present in XML file. See the result of the sample run below. The test run for two iterations as we have provided two values in the XML file for the Search table.

115

LAB: Create the file and add it to your test project.

create a data-driven coded UI test 1. In Solution Explorer, open the solution that contains your test project. 2. Click the Test menu, point to Windows, and then click Test View. The Test View window that shows your coded UI test opens. 3. In the Test View window, right-click the coded UI test method, and click Properties. The Properties window opens. 4. To select a data source to use for your coded UI test, click the ellipsis () to the right of Data Connection String in the Properties window. The New Test Data Source Wizard opens. 5. Select the type of data source that you want to use. For this example, select the .csv data source type. Click Next and enter the path and file name for your .csv file, then click Finish. The data source is now available for you to use in this test method. You use TestContext.DataRow to access the current row of data from your data source. You can access each column using the following syntax: Copy string paramVal = TestContext.DataRow["Input1"] 6. In Solution Explorer, open the CodedUITest1.cs file. Make the following changes to the CodedUITestMethod1() method: a. Add the following two lines of code before the call to the AddTwoNumbers method to provide values for the numbers to add. Copy this.UIMap.AddTwoNumbersParams.TextInput1EditText = TestContext.DataRow["Input1"].ToString(); this.UIMap.AddTwoNumbersParams.TextInput2EditText = TestContext.DataRow["Input2"].ToString(); b. Add the following line of code before the call to the AssertforAdd method to provide the value for the assert method. this.UIMap.AssertforAddExpectedValues.TextAnswerEditText = TestContext.DataRow["ExpectedResult"].ToString(); This is how the coded UI test method should look with the parameters and the data source added to it: [DeploymentItem("DataDriven.csv"), DataSource("Microsoft.VisualStudio.TestTools.DataSource.CSV", "|DataDirectory|\\DataDriven.csv", "DataDriven#csv",

116

DataAccessMethod.Sequential), TestMethod] public void CodedUITestMethod1() { this.UIMap.AddTwoNumbersParams.TextInput1EditText = TestContext.DataRow["Input1"].ToString(); this.UIMap.AddTwoNumbersParams.TextInput2EditText = TestContext.DataRow["Input2"].ToString(); this.UIMap.AddTwoNumbers(); this.UIMap.AssertforAddExpectedValues.TextAnswerEditText = TestContext.DataRow["ExpectedResult"].ToString(); this.UIMap.AssertforAdd(); } 7. Save the changes to the CodedUITest1.cs source code file. 8. To run your coded UI test, right-click the coded UI test in the Test View window and click Run Selection. After the tests have run, the overall test result for all iterations of the test displays in the Test Results window. To see the details of each iteration, double-click the test in the Test Results window.

Action Recording
Using an existing action recording is an efficient way to create coded UI tests. For more information about how to create action recordings, see Recording and Playing Back Manual Tests.

117

LAB: Generate a Coded UI Test from an Action Recording

To generate a coded UI test from an action recording 1. In Solution Explorer, right-click a test project, point to Add, and then click Coded UI Test. - or In the Test List Editor or the Test View window, right-click the surface of the window and then click New Test. In the Add New Test dialog box, click Coded UI Test and then click OK. - or On the Test menu, click New Test. In the Add New Test dialog box, click Coded UI Test and then click OK. The Coded UI Test dialog box appears. 2. Click Use an existing action recording. The Connect to Team Project dialog box appears. 3. Find the work item with an attached action recording: a. Select your project in the Project drop-down list and click Connect. b. Choose a method to find a work item with an attached action recording: Execute a saved query, query by ID, or search by the title of the work item. c. Click Find to execute the query. 4. Select the work item with the action recording you want to use to create a coded UI test, and then click OK. This adds the UI controls to the UI map, and actions to your coded UI test. A method is created for each step in the test case for the action recording. 5. To generate assertions for your UI controls, right-click in the coded UI test source file, point to Generate Code and then point to Use Coded UI Test Builder. The Coded UI Test Builder dialog box appears. 6. Start your application under test and navigate to the UI control that you want to add. 7. Click the crosshairs icon in the Coded UI Test Builder and drag it to the control that you want to verify is correct. The Coded UI Test Builder - Add Assertions dialog box is displayed. 8. Click the vertical bar to view the UI map. The UI control that you want to verify should be highlighted. 9. Right-click the property for the UI control that you want to verify, and then point to Add Assertion. The Add assertion dialog box is displayed. 10. Select the Comparator for your assertion. 11. Type the value for your assertion in Comparison Value. 12. To add the assertion, click OK. 13. When you have added all your assertions for your test, close the Coded UI Test Builder Add Assertions dialog box. 14. To generate the code for your assertions, click the Generate Code icon.

118

The Coded UI Test Builder - Generate Code dialog box is displayed. 15. Type a name for your coded UI test method in Method Name, and then click Add and Generate. This generates code as follows if the name that you entered is, for example, AssertForAddTwoNumbers: Adds a method called AssertForAddTwoNumbers to the UIMap.Designer.cs file. This method performs the assert statements that you added. Copy public void AssertForAddTwoNumbers() { #region Variable Declarations WinEdit textAnswerEdit = this.DemoCalculatorWindowWindow.AnswerWindow.TextAnswerEdit; #endregion // Verify that 'textAnswer' text box's Text property = '40' Assert.AreEqual( this.AssertForAddTwoNumbersExpectedValues.TextAnswerEditText, textAnswerEdit.Text); }

Adds a call to the assert method AssertForAddTwoNumbers to the test method in your coded UI test file Copy [TestMethod] public void CodedUITestMethod1() { // To generate code for this test, select "Generate Code" // from the shortcut menu and select one of the menu items. this.UIMap.AddTwoNumbers(); this.UIMap.AssertForAddTwoNumbers(); }

When you choose a name for the method that has your assert statements, choose a name that is descriptive for these assertions that you created. 1. Click the Close icon to close the Coded UI Test Builder. 2. (Optional) To add code to start your application when the coded UI test runs, record the startup sequence for your application and save that to a method. You can call that method at the start of your test. Note You can add a test initialize method, identified by a [TestInitialize] attribute, which runs code at the start of each test method. For example, the method to start the application could be called from the TestInitialize method. 3. (Optional) To add code to close your application when the coded UI test runs, record the closing sequence for your application and save that to a method.

119

If you do not close the browser or application, it will remain open after your test is finished. Note You can add a test cleanup method, identified by a [TestCleanup] attribute, which runs code at the end of each test method. For example, the method to close the application could be called from the TestCleanup method. 4. To run the test, right-click in the test method, and then click Run Tests. For more information about how to run coded UI tests, see Running Automated Tests. Note After you create your coded UI test with specific data, you might want to run your coded UI test multiple times with different sets of data to test different conditions. To do this you can add parameters from a data source to your coded UI test to create a data-driven coded UI test. For more information, see How to: Create a Data-Driven Coded UI Test.

120

Chapter 9
Actions and Properties in a Coded UI

121

When you work with UI test controls in coded UI tests they are separated into two parts: actions and properties.

The first part consists of actions that you can perform on UI test controls. For example, coded UI tests can simulate mouse button clicks on a UI test control, or simulate keys typed on the keyboard to affect a UI test control. The second part consists of enabling you to get and set properties on a UI test control. For example, coded UI tests can get the count of items in a ListBox, or set a CheckBox to the checked state. Note The Coded UI Test Editor lets you easily modify your coded UI tests. Using the Coded UI Test Editor, you can locate, view, and edit your test methods. You can also edit UI actions and their associated controls in the UI control map. The Coded UI Test Editor is included in the Microsoft Visual Studio 2010 Feature Pack 2. To download the feature pack, you must have either Visual Studio 2010 Ultimate, Visual Studio 2010 Premium or Test Professional 2010 with an MSDN subscription, Microsoft BizSpark, or MSDN Academic Alliance. For more information, see Editing Coded UI Tests Using the Coded UI Test Editor and Microsoft Visual Studio 2010 Feature Pack 2.

Accessing Actions of UI Test Control

To perform actions on UI test controls, such as mouse clicks or keyboard actions, use the methods in the Mouse and Keyboard classes. To access UI test control actions by simulating the mouse

To perform a mouse-oriented action, such as a mouse click, on a UI test control, use Click. Mouse.Click(buttonCancel);

To access UI test control actions by simulating the keyboard

To perform a keyboard-oriented action, such as typing into an edit control, use SendKeys. Keyboard.SendKeys(textBoxDestination, @"C:\Temp\Output.txt");

Accessing Properties of UI Test Control To get and set UI control specific property values, you can directly get or set the values the properties of a control, or you can use the UITestControl.GetProperty and UITestControl.SetProperty methods with the name of the specific property that you want you get or set.

122

GetProperty returns an object which can be cast to the appropriate Type. SetProperty accepts an object for the value of the property. To get or set properties from UI test controls directly

With controls that derive from T:Microsoft.VisualStudio.TestTools.UITesting.UITestControl, such as T:Microsoft.VisualStudio.TestTools.UITesting.HtmlControls.HtmlList or T:Microsoft.VisualStudio.TestTools.UITesting.WinControls.WinComboBox, you can get or set their property values directly as follows: Copy int i = myHtmlList.ItemCount; myWinCheckBox.Checked = true;

To get properties from UI test controls

To get a property value from a control, use GetProperty. To specify the property of the control to get, use the appropriate string from the PropertyNames class in each control as the parameter to GetProperty. GetProperty returns the appropriate data type, but this return value is cast as an Object. The return Object must then be cast to the appropriate type. Example: int i = (int)GetProperty(myHtmlList.PropertyNames.ItemCount);

To set properties for UI test controls

To set a property in a control, use SetProperty. To specify the property of the control to set, use the appropriate string from the PropertyNames class as the first parameter to SetProperty, with the property value as the second parameter. Example: SetProperty(myWinCheckBox.PropertyNames.Checked, true);

123

Best Activities
Use the following guidelines to create a flexible coded UI test.

Use the Coded UI Test Builder whenever possible. Do not modify the UIMap.designer.cs file directly. If you do this, the changes to the file will be overwritten. Create your test as a sequence of recorded methods. For more information about how to record a method, see How to: Create a Coded UI Test. Each recorded method should act on a single page, form, or dialog box. Create a new test method for each new page, form, or dialog box. When you create a method, use a meaningful method name instead of the default name. A meaningful name helps identify the purpose of the method. When possible, limit each recorded method to fewer than 10 actions. This modular approach makes it easier to replace a method if the UI changes. Create each assertion using the Coded UI Test Builder, which automatically adds an assertion method to the UIMap.Designer.cs file. If the user interface (UI) changes, re-record the test methods, or the assertion methods, or re-record the affected sections of an existing test method. Create a separate UIMap file for each MODULE in your application under test. For more information, see Testing a Large Application with Multiple UI Maps. In the application under test, use meaningful names when you create the UI controls. This gives more meaning and usability to the automatically generated control names. If you are creating assertions by coding with the API, create a method for each assertion in the part of the UIMap class that is in the UIMap.cs file. Call this method from your test method to execute the assertion. If you are directly coding with the API, use the properties and methods in the classes generated in the UIMap.Designer.cs file in your code as much as you can. These classes will make your work easier, more reliable, and will help you be more productive.

Coded UI tests automatically adapt to many changes in the user interface. If, for example, a UI element has changed position or color, most of the time the coded UI test will still find the correct element. During a test run, the UI controls are located by the testing framework by using a set of search properties which are applied to each control class in the definitions created by the Coded UI Test Builder in the UIMap.Designer.cs file. The search properties contain name-value pairs of property names and property values that can be used to identify the control, such as the FriendlyName, Name, and ControlType properties of the control. If the search properties are unchanged, the coded UI test will successfully find the control in the UI. If the search properties are changed, coded UI tests have a smart match algorithm which applies heuristics to find controls and windows in the UI. When the UI has changed, you might be able to modify the search properties of previously identified elements to make sure that they are found.

124

Chapter 10
What to do if your UI Changes

125

User interfaces frequently change during development. Here are some ways to reduce the effect of these changes:

Find the recorded method which references this control and use the Coded UI Test Builder to re-record the actions for this method. You can use the same name for the method to overwrite the existing actions. If a control has an assertion that is no longer valid:

Delete the method that contains the assertion. Remove the call to this method from the test method. Add a new assertion by dragging the cross-hair button onto the UI control, open the UI map, and add the new assertion.

For more information about how to record coded UI tests, see How to: Generate a Coded UI Test by Recording the Application Under Test or How to: Create a Coded UI Test. What to do if a background process needs to complete before the test can continue? You might have to wait until a process finishes before you can continue with the next UI action. To do this you can use WaitForReadyLevel to wait before the test continues as in the following sample.

// Set the playback to wait for all threads to finish Playback.PlaybackSettings.WaitForReadyLevel = WaitForReadyLevel.AllThreads; // Press the submit button this.UIMap.ClickSubmit(); // Reset the playback to wait only for the UI thread to finish Playback.PlaybackSettings.WaitForReadyLevel = WaitForReadyLevel.UIThreadOnly;

LAB: Creating, Editing and Maintaining a Coded UI Test

In this walkthrough, you will create a simple Windows Presentation Foundation (WPF) application to demonstrate how to create, edit, and maintain a coded UI test. The walkthrough provides solutions for correcting tests that have been broken by various timing issues and control refactoring. Prerequisites

126

For this walkthrough you will need:

Visual Studio 2010 Ultimate or Visual Studio 2010 Premium.

Create a Simple WPF Application 1. On the File menu, point to New, and then select Project. The New Project dialog box appears. 2. In the Installed Templates pane, expand Visual C#, and then select Windows. 3. Above the middle pane, verify that the target framework drop-down list is set to .NET Framework 4. 4. In the middle pane, select the WPF Application template. 5. In the Name text box, type SimpleWPFApp. 6. Choose a folder where you will save the project. In the Location text box, type the name of the folder. 7. Click OK. The WPF Designer for Visual Studio opens and displays MainWindow of the project. 8. If the toolbox is not currently open, open it. Click the View menu, and then click Toolbox. 9. Under the All WPF Controls section, drag a Button, CheckBox and ProgressBar control onto the MainWindow in the design surface. 10. Select the Button control. In the Properties window, change the value for the Content property from Button to Start. 11. Select the ProgressBar control. In the Properties window, change the value for Maximum property from 100 to 1000. 12. Select the Checkbox control. In the Properties window, clear the IsEnabled property.

13. Double-click the button control to add a Click event.

127

The MainWindow.xmal.cs is displayed in the Code Editor with the cursor in the new button1_Click event. 14. At the top of the MainWindow class, add a delegate. The delegate will be used for the progress bar. To add the delegate, add the following code: Copy private delegate void ProgressBarDelegate( System.Windows.DependencyProperty dp, Object value); 15. In the button1_Click method, add the following code: Copy double progress = 0; ProgressBarDelegate updatePbDelegate = new ProgressBarDelegate(progressBar1.SetValue); do { progress ++; Dispatcher.Invoke(updatePbDelegate, System.Windows.Threading.DispatcherPriority.Background, new object[] { ProgressBar.ValueProperty, progress }); progressBar1.Value = progress; } while (progressBar1.Value != progressBar1.Maximum); checkBox1.IsEnabled = true; Verify the WPF Application Runs Correctly 1. On the Debug menu, select Start Debugging or press F5. 2. Click Start. In a few seconds, the progress bar should be 100% complete. The check box control is now enabled. 3. Close SimpleWPFApp. Create and Run a Coded UI Test for SimpleWPFApp 1. In Solution Explorer, right-click the solution, click Add and then select New Project. The Add New Project dialog box appears. 2. In the Installed Templates pane, expand Visual C#, and then select Test. 3. In the middle pane, select the Test Project template.

128

4. Click OK. In Solution Explorer, the new test project named TestProject1 is added to your solution and the UnitTest1.cs file appears in the Code Editor. You can close the UnitTest1.cs file because it is not used in this walkthrough. 5. In Solution Explorer, right-click TestProject1, click Add and then select Coded UI test. The Generate Code for Coded UI Test dialog box appears. 6. Select the Record actions, edit UI map or add assertions option and click OK. The UIMap Coded UI Test Builder appears. For more information about the options in the dialog box, see How to: Create a Coded UI Test. 7. Locate and run the SimpleWPFApp application that you created earlier. By default, the application will be located at C:\Users\<username>\Documents\Visual Studio 2010\Projects\SimpleWPFApp\SimpleWPFApp\bin\Debug\SimpleWPFApp.exe 8. Create a desktop shortcut to the SimpleWPFApp application. Right-click SimpleWPFApp.exe and choose Copy. On your desktop, right-click and choose Paste shortcut. Tip A shortcut to the application makes it easier to add or modify Coded UI tests for your application because it lets you start the application quickly. You do not have to navigate to it. You will have to run the application again in this walkthrough. 9. Click Start Recording on the UIMap Coded UI Test Builder. In a few seconds, the Coded UI Test Builder will be ready. 10. Run SimpleWPFApp.exe using the desktop shortcut. 11. On the SimpleWPFApp, click Start. In a few seconds, the progress bar should be 100 complete. The check box control is now enabled. 12. Select the box for the CheckBox control. 13. Close the SimpleWPFApp application. 14. On the UIMap - Coded UI Test Builder, click Generate Code. 15. In the Method Name type SimpleAppTest and click Add and Generate. In a few seconds, the Coded UI test appears and is added to the Solution. 16. Close the UIMap Coded UI Test Builder. The CodedUITest1.cs file appears in the Code Editor. Run the Coded UI Test

In the CodedUITest1.cs file, locate the CodedUITestMethod method, right-click and select Run Tests. While the coded UI test runs, the SimpleWPFApp is visible. It conducts the steps that you did in the previous procedure. However, when the test tries to select the check box for the CheckBox control, the Test Results window shows that the test failed. This is because the test tries to select the check box but is not aware that the CheckBox control is disabled until the progress bar is 100% complete. You can correct this and similar issues by using the various UITestControl.WaitForControlXXX() methods that are available for coded UI testing. The next procedure will demonstrate using the

129

WaitForControlEnabled() method to correct the issue that caused this test to fail. For more information, see Making Coded UI Tests Wait For Specific Events During Playback. Edit and Rerun the Coded UI Test 1. In the Test Results window, right-click the failed test and select View Test Results Details. The CodedUITestMethod1[Results] is displayed. 2. In the Error Stack Trace section, click the first link next to TestProject1.UIMap.SimpleAppTest(). The UIMap.Designer.cs file opens with the point of error highlighted in the code: Copy // Select 'CheckBox' check box uICheckBoxCheckBox.Checked = this.SimpleAppTestParams.UICheckBoxCheckBoxChecked; 3. To correct this problem, you can make the coded UI test wait for the CheckBox control to be enabled before continuing on to this line using the WaitForControlEnabled() method. Caution Do not modify the UIMap.Designer.cs file. Any code changes you make in the UIMapDesigner.cs file will be overwritten every time you generate code using the UIMap - Coded UI Test Builder. If you have to modify a recorded method, you must copy it to UIMap.cs file and rename it. The UIMap.cs file can be used to override methods and properties in the UIMapDesigner.cs file. You must remove the reference to the original method in the Coded UITest.cs file and replace it with the renamed method name. 4. In the UIMapDesinger.cs file, select all the code for the entire SimpleAppTest method and copy it. 5. In Solution Explorer, open the UIMap.cs file. 6. Paste the SimpleAppTest method code into the UIMap partial class. 7. Rename the method from SimpleAppTest() to ModifiedSimpleAppTest() 8. Add the following using statement to the file: Copy using Microsoft.VisualStudio.TestTools.UITesting.WpfControls; 9. Add the following WaitForControlEnabled() method before the offending line of code identified previously: Copy uICheckBoxCheckBox.WaitForControlEnabled();

130

// Select 'CheckBox' check box uICheckBoxCheckBox.Checked = this.SimpleAppTestParams.UICheckBoxCheckBoxChecked; 10. In the CodedUITest1.cs file, locate the CodedUITestMethod method and either comment out or rename the reference to the original SimpleAppTest() method and then replace it with the new ModifiedSimpleAppTest(): [TestMethod] public void CodedUITestMethod1() { // To generate code for this test, select "Generate Code for Coded UI Test" from the shortcut menu and select one of the menu items. // For more information on generated code, see http://go.microsoft.com/fwlink/?LinkId=179463 //this.UIMap.SimpleAppTest(); this.UIMap.ModifiedSimpleAppTest(); } 11. On the Build menu, click Build Solution. 12. Right-click the CodedUITestMethod method and select Run Tests. 13. This time the coded UI test successfully completes all the steps in the test and Passed is displayed in the Test Results window. Refactor a Control in the SimpleWPFApp 1. In the MainWindow.xaml file, in the Designer, select the button control. 2. At the top of the Properties window, select the value button1 next to Button and change the value to buttonA. 3. On the Build menu, click Build Solution. 4. On the Test menu, select Windows and then click Test View. 5. In Test View, select CodedUITestMethod1 under the Test Name column and then click Run Selection in the toolbar. The test fails because the coded UI test cannot locate the button control that was originally mapped in the UIMap as button1. Refactoring can impact coded UI tests in this manner. 6. In the Test Results window, right-click the failed test and select View Test Results Details. The CodedUITestMethod1[Results] appears. 7. In the Error Stack Trace section, click the first link next to TestProject1.UIMpa.SimpleAppTest(). The UIMap.Designer.cs file opens. The point of error is highlighted in the code:

// Click 'Start' button

131

Mouse.Click(uIStartButton, new Point(27, 10)); Notice that the line of code earlier in this procedure is using UiStartButton, which is the UIMap name before it was refactored. To correct the issue, you can add the refactored control to the UIMap by using the Coded UI Test Builder. You can update the tests code to use the code, as demonstrated in the next procedure. Map Refactored Control and Edit and Rerun the Coded UI Test 1. In the CodedUITest1.cs file, in the CodedUITestMethod1() method, right-click, select Generate Code for Coded UI Test and then click Use Coded UI Test Builder. The UIMap Coded UI Test Builder appears. 2. Using the desktop shortcut you created earlier, Run the SimpleWPFApp application that you created earlier. 3. On the UIMap Coded UI Test Builder, drag the crosshair tool to the Start button on the SimpleWPFApp. The Start button is enclosed in a blue box and the Coded UI Test Builder takes a few seconds to process the data for the selected control and displays the controls properties. Notice that the AutomationUId is named buttonA. 4. In the properties for the control, click the arrow at the upper-left corner to expand the UI Control Map. Notice that UIStartButton1 is selected. 5. In the toolbar, click the Add control to UI Control Map. The status at the bottom of the window verifies the action by displaying Selected control has been added to the UI control map. 6. On the UIMap Coded UI Test Builder, click Generate Code. The Coded UI Test Builder Generate Code appears with a note indicating that no new method is required and that code will only be generated for the changes to the UI control map. 7. Click Generate. 8. Close SimpleWPFApp.exe. 9. Close UIMap Coded UI Test Builder. The UIMap Coded UI Test Builder takes a few seconds to process the UI control map changes. 10. In Solution Explorer, open the UIMap.Designer.cs file. 11. In the UIMap.Designer.cs file, below the constructor in the generated code class UIMainWindow, expand the Properties region. Notice that the public WpfButton UIStartButton1 property has been added. 12. In the UIStartButton1 property, expand the Search Criteria region. Notice the SearchProperties is set to "buttonA":

public WpfButton UIStartButton1 {

132

get { if ((this.mUIStartButton1 == null)) { this.mUIStartButton1 = new WpfButton(this); #region Search Criteria this.mUIStartButton1.SearchProperties[WpfButton.PropertyNames.AutomationId] = "buttonA"; this.mUIStartButton1.WindowTitles.Add("MainWindow"); #endregion } return this.mUIStartButton1; } } Now you can modify the coded UI test to use the newly mapped control. As pointed out in the previous procedure if you want to override any methods or properties in the coded UI test, you must do so in the UIMap.cs file. 13. In the UIMap.cs file, add a constructor and specify the SearchProperties property of the UIStartButton property to use the AutomationID property with a value of "buttonA": Copy public UIMap() { this.UIMainWindowWindow.UIStartButton.SearchProperties[WpfButton.PropertyName s.AutomationId] = "buttonA"; } 14. On the Build menu, click Build Solution. 15. In Test View, select CodedUITestMethod1 under the Test Name column, In the toolbar, click Run Selection. This time, the coded UI test successfully completes all the steps in the test. In the Test Results Window, you will see a status of Passed.

133

134

Chapter 11
Extending Coded UI Tests and Action Recordings to Support Microsoft Excel

135

The testing framework for coded UI tests and action recordings does not support every possible user interface. It might not support the specific UI that you want to test. For example, you cannot immediately create a coded UI test or an action recording for a Microsoft Excel spreadsheet. However, you can create your own extension to the coded UI test framework that will support your specific UI by taking advantage of the extensibility of the coded UI test framework. The following topic gives an example of how to extend the framework to support the creation of coded UI tests and action recordings for Microsoft Excel. For more information about the platforms that are supported, see Supported Configurations and Platforms for Coded UI Tests and Action Recordings. This section presents a coded UI test extension that can record and play back tests of Excel Worksheets. Each part of the extension is explained in this section and in the code comments for developers who want to create just such an extension.

Architecture Overview

136

SCENARIO:
A customer Im working with has placed a heavy investment in Waiting testing over the years and with a recent move to TFS2010 they also wanted to take advantage of the new Microsoft Test Manager (MTM) feature and the ability to associate automated tests to test cases in MTM. Heres a quick how-to for those of you wanting to do the same thing.

Create A Test Case

First up, lets create a test case:

Pretty simple open browser and check the url in the address bar. What I have also done with this test is use parameters to supply the data allowing testers to decide what values they want to test with. This is better than the devs doing this, plus it makes for a much nicer UI for data driven tests than excel or CSV files do.

Create The CodedUI Watin Test

Next we need to create the automation for this test. Add a Coded UI test class to a test project but dont go create any tests via the wizard that appears. Just press cancel and then go to your test code and add a normal Watin test as per usual. Oh, if youre a person who normally deletes all that TestContext stuff, then youll need to leave it in place this time around dont delete it. CODE SAMPLE:

137

using Microsoft.VisualStudio.TestTools.UnitTesting; using WatiN.Core; namespace CodedUITesting { [CodedUITest] public class CodedUITest1 { ... [TestMethod] [DataSource("Microsoft.VisualStudio.TestTools.DataSource.TestCase", "http://tfs2008-vm:8080/tfs/TemplateTrials;Scrum v5", "101", DataAccessMethod.Sequential)] public void RunWatinTestParameterized() { var url = TestContext.DataRow["url"].ToString(); var result = TestContext.DataRow["result"].ToString(); using (IE ie = new IE(url)) { Assert.AreEqual(result, ie.Url); } } } The things to note in this test are that the data source attribute is used to pull data from the TFS Test Case work item specifically the URL of the team project is included, as it the test case id. In the test itself we pull data from the parameters in the test case using the TestContext.DataRow[parameter'NameHere].ToString() calls, and the rest is just normal Watin code. If you have any issues with missing references make sure you reference Watin and Interop.ShDocVw and that both get copied to the output folder as part of the build. Attach the Automation The only thing we need to do once we have this test is to link it to the test case as its automation method. Open the test case in Visual Studio, navigate to the Associated Automation tab and click the [] button to select the method. You should see something like this once its done:

138

Run The Test Assuming you have a lab environment with test agents installed you should then be able to trigger a new build and when its complete start an automated test run for that build and see everything working as you would expect. Heres a result from a run on my local TFS server:

Testing Using Virtual Environments

139

Using Visual Studio Lab Management you can run manual or automated tests from a test plan using your virtual environments. This lets you run your tests on an environment that uses a known state for all the virtual machines in the environment. Now, testers no longer need to create clean machines in a known state manually, which can make it much quicker to set up an environment to use for testing. You can also deploy your application to your virtual environment that uses Lab Management. For more information about how to deploy your application, see How to: Deploy an Application on a Virtual Environment. Note If you will be performing manual or automated testing in purely physical environments, or in physical environments composed of third-party virtual machines, you do not need to do any of the following tasks: use Hyper-V, configure SCVMM, or configure lab management. Instead, you can simply install a test agent on the physical machine, register a test controller with your team project collection, and configure the test agent to use that test controller. After you have done this, use the Lab Center in Microsoft Test Manager to create a new physical environment. For the steps to create a physical environment and run tests, see Creating a Physical Environment to Use for Testing. When you run one or more of these tests from your test plan, you can save your test results into the team project for your Team Foundation Server. You can now view the progress of both your automated and manual tests together from your test plan. To run manual tests using a virtual environment, you must use Microsoft Test Manager to add the virtual environment and test settings for manual tests to your test plan and then run your tests from a test plan as shown in the following illustration. Then you can view your test results from your test plan.

140

Note You can also select Run with options from the Run Tests view to select the virtual environment and test settings to use when you run your tests. If you find a bug in your application when you run your manual tests, you can take a snapshot of your virtual environment that a developer can use to connect to the actual state of your machines in your environment when you found the bug. This helps create easily reproducible bugs. If you want to run automated tests using your virtual environments, you must associate your automated tests with test cases using Microsoft Visual Studio 2010, create test settings and add your virtual environment for your automated tests to your test plan, and then run them using Microsoft Test Manager. If you want to schedule your tests to run as part of your build process using Visual Studio Lab Management you can create a workflow using a build definition for Team Foundation Build that enables you to build your application, deploy it to your virtual environment and then run automated tests.

Running Tests
If you have Visual Studio Test Professional 2010 or Visual Studio 2010 Ultimate, you can run manual tests and automated tests from a test plan by using Microsoft Test Manager. When you run any of these tests from your test plan, you can save your test results into the team project for your Team Foundation Server. You can now view the progress of both your automated and manual tests from your test plan. If you have Visual Studio 2010 Ultimate ,Visual Studio 2010 Premium, or Visual Studio Professional you can run automated tests from Microsoft Visual Studio 2010 or from the command line. You can also run tests by using Team Foundation Build. Test results are created every time that you run a group of tests. To run manual tests, you must use Microsoft Test Manager to run your tests from a test plan, as shown in the following illustration. Then you can view your test results from your test plan.

141

After you have created any of the following types of automated tests by using Microsoft Visual Studio 2010, you can use Microsoft Test Manager or Microsoft Visual Studio 2010 to run the tests: Unit tests. Coded UI tests. Database unit tests. Load tests. Generic tests.

For example, if you want to run your automated tests and see the results immediately, you can just run your tests from Microsoft Visual Studio 2010 and view the test results, as shown in the following illustration.

142

If you want to run your automated tests and have the results added to a test plan, you can associate your automated tests with test cases using Microsoft Visual Studio 2010, and then run them using Microsoft Test Manager, as shown in the following illustration.

143

Database Unit Tests

Based on the following ways that you can run the database unit tests, you may have to install one of these SKUs on additional machines:

If you run database unit tests locally by using Visual Studio, you must have one of these SKUs installed locally. If you run database unit tests by using a test controller and test agents from Visual Studio, you must install one of these SKUs on any test agent computer that can be used to run these tests. If you associate your database unit tests with a test case and run these tests from Microsoft Test Manager, you must install one of these SKUs on any test agent in your environment that is included in the role to run tests. You must also install one of these SKUs on the test controller for this environment.

144

Chapter 12
Running Automated Tests

145

After you have created any one of the following types of automated tests by using Microsoft Visual Studio 2010, you can then run the tests.

Unit tests. Coded UI tests. Database unit tests. Load tests. Generic tests.

Many methods of running your automated tests are available, depending on how you want to run tests and view the results. If you run your automated tests by using a test plan, you can view your testing progress and easily rerun your tests as required. To run your automated tests by using a test plan, you associate your automated tests with test cases and run these test cases by using Microsoft Test Manager. To run your automated tests in this manner, you must create a physical or virtual environment to use when you run your tests. For more information about how to create virtual environments to use to run your tests, see Using a Virtual Lab for Your Application Lifecycle. Note: It is not a good idea to run load tests by using Microsoft Test Manager because you cannot view the test as it runs. It is useful to be able to do this for load tests. The environment enables you to run tests, gather data, or perform system actions on machines for each specific role that you add to the environment. A role specifies the purpose of a machine in the environment. For example, a specific role could be called "Web Site for Customer Data Store." A machine can be a physical computer or a virtual machine. You select which machines to use in an environment for each role. Scenario 1: A tester could run your tests on one machine and gather system information about a machine that runs the Web server for your application. Alternatively, you could run your tests on an environment that uses multiple machines and collect test impact data on those machines. In addition, you can also perform network emulation on the machine that runs the Web server for your application. The following illustration shows three examples of scenarios for how you can set your test settings to run your tests by using environments from Microsoft Test Manager.

146

This method of running automated tests lets you view the overall status of any tests in your test plan. You can view the results of both manual and automated tests together, if you want. You can also run these test cases by using associated automation from the command line. You can also run your tests directly from Microsoft Visual Studio 2010 or the command line without being part of a test plan or without using an environment to run them remotely. In addition, if you add your automated tests to a test category or a test list, your automated tests can be run automatically as part of the build process. Note To take advantage of the multiple processors in a multiprocessor machine, you must run your tests by using test agent controllers and test agents. Otherwise, if you run your tests locally, you will not use this additional processing power. For more information, see Setting Up Test Machines to Run Tests or Collect Data Based on the following ways that you can run database unit tests, you may have to install one of these SKUs on additional machines:

If you run database unit tests locally by using Visual Studio, you must install one of these SKUs locally. If you run database unit tests by using a test controller and test agents from Visual Studio, you must install one of these SKUs on any test agent computer that can be used to run these tests.

147

If you associate your database unit tests with a test case and run these tests from Microsoft Test Manager, you must install one of these SKUs on any test agent in your environment that is included in the role to run tests. You must also install one of these SKUs on the test controller for this environment.

148

149

Chapter 13
Introduction to Debugging

150

Introduction
In the software development life cycle, testing and defect fixing take more time than actually code writing. In general, debugging is a process of finding out defects in the program and fixing them. Defect fixing comes after the debugging, or you can say they are co-related. When you have some defects in your code, first of all you need to identify the root cause of the defect, which is called the debugging. When you have the root cause, you can fix the defect to make the program behavior as expected. Now how to debug the code? Visual Studio IDE gives us a lot of tools to debug our application. Sometimes debugging activity takes a very long time to identify the root cause. But VS IDE provides a lot of handy tools which help to debug code in a better way. Debugger features include error listing, adding breakpoints, visualize the program flow, control the flow of execution, data tips, watch variables and many more. Many of them are very common for many developers and many are not. In this article, I have discussed all the important features of VS IDE for debugging like Breakpoint, labeling and saving breakpoints, putting conditions and filter on breakpoints, DataTips, Watch windows, Multithreaded debugging, Thread window, overview of parallel debugging and overview of IntelliTrace Debugging. I hope this will be very helpful for beginners to start up with and for becoming an expert on debugging. Please note, targeted Visual Studio version is Visual Studio 2010. Many things are common in older versions, but many features such as Labeling breakpoint, Pinned DataTip, Multithreaded Debugging, Parallel debugging and IntelliTrace are added in VS 2010. Please provide your valuable suggestions and feedback to improve my article.

Beginning with Debugging

You can start debugging from the Debug menu of VS IDE. From the Debug Menu, you can select "Start Debugging" or just press F5 to start the program. If you have placed breakpoints in your code, then execution will begin automatically.

151

There is another way to start the debugging by "Attach Process". Attach process will start a debug session for the application. Mainly we are very much familiar with the attach process debugging for ASP.NET Web Application. I have published two different articles on the same on CodeProject. You may have a look into this. Debug Your ASP.NET Application that Hosted on IIS Remote IIS Debugging: Debug your ASP.NET Application which is hosted on "Remote IIS Server" We generally start debugging any application just by putting breakpoint on code where we think the problem may occur. So, lets start with breakpoints.

Breakpoints
Breakpoint is used to notify debugger where and when to pause the execution of program. You can put a breakpoint in code by clicking on the side bar of code or by just pressing F9 at the front of the line. So before keeping a breakpoint, you should know what is going wrong in your code and where it has to be stopped. When the debugger reaches the breakpoint, you can check out what's going wrong within the code by using a different debugging tool.

Debugging with Breakpoints

You have already set a breakpoint in your code where you want to pause the execution. And now start the program by pressing "F5". When the program reaches the breakpoint, execution will automatically pause. Now you have several options to check your code. After hitting the breakpoint, breakpoint line will show as yellow color which indicates that this is the line which will execute next. Now you have several commands available in break mode, using which you can proceed for further debugging.

152

Step Over
After debugger hits the breakpoint, you may need to execute the code line by line. "Step Over" [ F10 ] command is used to execute the code line by line. This will execute the currently highlighted line and then pause. If you select F10 while a method call statement is highlighted, the execution will stop after the next line of the calling statement. Step Over will execute the entire method at a time.

Step Into
This is similar to Step Over. The only difference is, if the current highlighted section is any methods call, the debugger will go inside the method. Shortcut key for Step Into is "F11".

Step Out
This is related when you are debugging inside a method. If you press the Shift - F11 within the current method, then the execution will complete the execution of the method and will pause at the next statement from where it called.

Continue
It's like run your application again. It will continue the program flow unless it reaches the next breakpoint. The shortcut key for continue is "F5".

Set Next Statement

This is quite an interesting feature. Set Next Statement allows you to change the path of execution of program while debugging. If your program paused in a particular line and you want to change the execution path, go to the particular line, Right click on the line and select "Set Next Statement" from the context menu. You will see, execution comes to that line without executing the previous lines of code. This is quite useful when you found some line of code may causing breaking your application and you dont want to break at that time. Shortcut key for Set Next Statement is Ctrl + Shift + F10.

153

Show Next Statement [ Ctrl+* ]

This line is marked as a yellow arrow. These lines indicate that it will be executed next when we continue the program.

Labeling in Break Point

This is the new feature in VS 2010. This is used for better managing breakpoints. It enables us to better group and filter breakpoints. It's kind of categorization of breakpoints. If you are having different types of breakpoints which are related with a particular functionality, you can give their name and can enable, disable, filter based on the requirements. To understand the whole functionality, let's assume that you have the below code block which you want to debug. class Program { static void Main(string[] args) { string[] strNames = { "Name1", "Name2", "Name3", "Name4", "Name5", "Name6" }; foreach (string name in strNames) { Console.WriteLine(name); // BreakPoint } int temp = 4; for (int i = 1; i <= 10; i++) { if (i > 6) temp = 5; } }

154

public static void Method1() { Console.WriteLine("Break Point in Method1"); // BreakPoint } public static void Method2() { Console.WriteLine("Break Point in Method2"); // BreakPoint Console.WriteLine("Break Point in Method2"); // BreakPoint } public static void Method3() { Console.WriteLine("Break Point in Method3"); // Breakpoint } } If you run the program, execution will pause on the first breakpoint. Now see the below picture, where you have the list of breakpoints.

In the given picture label column in blank. Now, see how you can set the label on break point and what the use of it is. To set label for any breakpoint, you just need to right click on the breakpoint symbol on the particular line or you can set it directly from breakpoint window.

Conditional Breakpoint
Suppose you are iterating through a large amount of data and you want to debug a few of them. It means you want to pause your program on some specific condition. Visual Studio Breakpoints allow you to put conditional breakpoint. So if and only if that condition is satisfied, the debugger will pause the execution. To do this, first of all you need to put the breakpoint on a particular line where you want to pause execution. Then just "Right Click" on the "Red" breakpoint icon. From there you just click on "Condition"

155

Let's assume that you have the following code block: class Program { static void Main(string[] args) { string [] strNames = { "Name1","Name2", "Name3", "Name4", "Name5", "Name6"}; foreach(string name in strNames) { Console.WriteLine(name); // Breakpoint is here } } } You have a breakpoint on Console.WriteLine() statement. On running of the program, execution will stop every time inside that for-each statement. Now if you want your code to break only when name="Name3". What needs to be done? This is very simple, you need to give the condition like name.Equals("Name3").

156

Immediate Window
Immediate window is very much common and a favorite with all developers. It's very much helpful in debug mode of the application if you want to change the variable values or execute some statement without impacting your current debugging steps. You can open the Immediate window from menu Debug > Window > Immediate Window { Ctrl + D, I / Alt + Ctrl - I }. Immediate window has a set of commands which can be executed any time during debugging. It also supports Intellisense. During Debug mode, you can execute any command or execute any code statement from here.

These are very much common features for all the developers, so I am not going into details of each and every command of Immediate window.

Call Stack
These features also improve the productivity during debugging. If you have multiple method calling or nested calling all over your application and during debugging, you want to check from where this method has invoked, "Call Stack" comes into the picture. The Call Stack Window shows that current method call nesting. In Call Stack window if you clicked on any of the rows, it will point you to the actual code of line of Visual Studio Code Editor. You can also customize the call stack row view by selecting different types of columns. To customize, Right Click on the "Call Stack" window, and from the context menu, you can select or deselect the option.

157

Call stack is very much important when you have multiple methods call all across the application and one particular method throwing an exception on some particular case. At that time, you can use call stack to see from where this method is getting invoked, based on that you can fix the defect.

Debugging Multithreaded Program

As of now, what I have discussed is all about fundamentals of debugging, knowing debugging tools and their uses. Now let's have a look into the multithreaded scenarios. Here you will see how to work with multithreaded program debugging, where is your current thread, what is the thread execution sequence, what is the state of thread. Before continuing with the demo, let's consider you have the following piece of code which you want to debug. class ThreadTest { static void Main() { Thread t = new Thread(new ThreadStart(Go)); t.Name = "Thread 1"; Thread t1 = new Thread(new ThreadStart(Go)); t1.Name = "Thread 2"; t.Start(); t1.Start(); Go(); } static void Go() { Console.WriteLine("hello!"); } }

158

In the sample code, you have three different threads - Main Thread, Thread 1, Thread 2. I have given a thread name to make you understand better. Now set a breakpoint inside "Go()" and run the application. When debugger hits the breakpoint, Press Ctrl+D,T or Navigate through Debug > Window > Threads. Threads window will appeared on the screen.

Exploring Threads Window

After selecting the thread window from debug menu, the following screen will come:

By default thread window having ID, Managed ID, Category, Name, Location and Priority column. At the start, execution pauses at "Main Thread". "Yellow Arrow" indicates the current executable thread. Category column indicates the category of threads, like main thread or worker thread. If you check the thread location, it is nothing but Namespace > Class > Method name. In the diagram, it is showing that the Main Thread will be executed next. Now to explore the next step by just pressing "F5" and see what are the changes in thread window.

So after pressing F5, it jumped to the next step to thread 1. you can also check the current location for Main Thread. It says "Sleep/ Wait / Join" , means waiting for something to complete. Similarly the next step will move you to thread 2. From the Thread window, you can understand how easy it is to monitor your threads using this debugger tool. There is another great feature available within the thread window. You can expand/collapse the Thread Location and can see what is next. For example, if you expand the location for "Main Thread", it will look like the diagram given below:

159

Flag Just My Code

The sample code which I have explained for the thread debugging is very simple. What will happen if you have a huge code block with multiple number of threads. Then it will be very difficult for you to identify which thread is part of your code or which ones are not related. Thread window gives you very easy features to set the "Flag" for all the threads which are part of your code. For that, you need to just flag your thread by option " Flag Just My Code".

Break Point Filter - Multithread Debugging

While discussing about breakpoint filter in breakpoint section, I said that breakpoint filter is very much helpful for Multithreaded debugging mode. Now this is the time to explore it. In our current example, we have three threads Main Thread, Thread1 and Thread 2. Now what if you want breakpoint to hit only for "Thread 2". What will you do ? Here is the use of breakpoint filter. Right click on the breakpoint, select "Filter" from the context menu. Now in breakpoint filter window, you need to fill the filter criteria. As per your requirement, you need to specify "ThreadName="Thread 2" .

160

Here ThreadName was one of the criteria by which you can filter, but you can filter on multiple clauses like ThreadID, ProcessName, ProcessID, etc. After setting the breakpoint filter, run the application and open the "Threads" window.

You will find your program execution has only paused during the execution of "Thread 2" . This is all about the debugging with multithreaded application. Hope you have learned something from it. Let's start with another most important topic "Parallel Debugging".

Debugging Parallel Program

This is another great feature added to Visual Studio 2010 to debug parallel program. Parallel programming is the new feature coming with .NET 4.0. If you want to learn more about parallel programming, please check here.

161

Now Debugging the parallel program is also a big topic. Here I will give you a basic overview to know about the debugging of parallel program. To discuss about it, let's consider you have the following piece of code: using System; using System.Collections.Generic; using System.Linq; using System.Text; using System.Threading; using System.Threading.Tasks; namespace ParalleTaskDebugging { class Program { static void Main(string[] args) { var task_a = Task.Factory.StartNew(() => DoSomeWork(10000)); var task_b = Task.Factory.StartNew(() => DoSomeWork(5000)); var task_c = Task.Factory.StartNew(() => DoSomeWork(1000)); Task.WaitAll(task_a, task_b, task_c); } static void DoSomeWork(int time) { Thread.Sleep(time); } } } To understand the parallel program debugging, we need to be aware about two window options: 1. Parallel Tasks 2. Parallel Stacks

Parallel Task and Parallel Stacks

Before continuing with parallel tasks and parallel stacks, you have to know about Threads Window which I have already covered. In the given code, you have three different tasks which are doing something and after sometime, all the tasks are put on hold. This is done intentionally

162

to check the status of each task. To test, put a breakpoint on DoSomeWork() method and run the application. You will see your program execution paused on the breakpoint. After the program break, you can go to Debug > Window > Open Parallel Tasks and Parallel Stacks window. I asked to open both at the same time only because you can visualize what is going on.

Parallel Task window will show you what are the different tasks that have been created for the program and what is their current status. On the other hand, Parallel Stacks will show you the graphical view of all thread creation, containing tasks, how they are related. If you click on the thread from the Parallel Stacks, it will show you the code line related with the thread (as shown in the picture with a Green Arrow). To move ahead, press F5. Let's see what comes next.

163

In the above diagram, you can find one of the tasks has been executed and the other two are remaining. Current execution point is set to AsyncMethod_1, so if you continue, this method will execute first and next time the others. When you are working with parallel programming, there are many scenarios which will come like Deadlock, Dependency problem, etc. These topics is very interesting and long to discuss. Please check further study section of the article to know more details.

164

Chapter 14
Create Test Cases from Automated Tests Using TCM.exe

165

Introduction
If you have created automated tests using Visual Studio, you might want to run these tests as part of a test plan. By adding them to a test plan, you can measure your testing progress. You can view the results from these automated tests and any other tests in your test plan together and track the progress that you are making. You can also link these test cases to requirements, so that you can view the status of automated tests for the requirement. For more information about how to do this, see Link Existing Test Cases to a Requirement. To run these tests as part of a test plan, you can use a command line tool tcm.exe to create test cases for all your test methods in an assembly or some of your test methods based on test method properties or the test category. If you want to add a test to an existing test case, you can associate a test method directly with that test case. For more information, see How to: Associate an Automated Test with a Test Case. If you have existing automated tests from earlier versions of Microsoft Visual Studio 2010, you can associate these tests with a test case after you upgrade the test project that contains the automated tests to Visual Studio 2010. After you have imported your test methods and created test cases, you have to add these test cases into your test plan. To run these test cases from your test plans, you must use a physical or a virtual environment. You cannot run automated tests using Microsoft Test Manager without an environment. You must check in the test project that contains the automated test methods and make sure that the test project is part of your build definition. You must make sure that a build created by using this build definition is selected in your test plan. The information from the build is used by Microsoft Test Manager to locate the correct test method assembly and use this to run the automated tests. For more information about how to check in your test project and team build, see Add Files to Version Control and Building the Application. You can also run automated tests from the command line using test settings and a test environment. For more information, see How to: Run Test Cases with Automation from the Command Line Using Tcm. Use the following procedures to create test cases from an assembly of automated tests that you can then run as part of your test plan:

Create the test cases from your assembly of automated tests and add them to your plan Set up your test plan to use your team build Create your test settings and environment to run your tests

166

Run the automated test using Microsoft Test Manager View and update the test results

Common Parameters for the Procedures

Parameter /collection:CollectionURL Description Specifies the URI of the team project collection. The format for the URI is as follows: http://ServerName:Port/VirtualDirectoryName/CollectionName If no virtual directory is used, then the format for the URI is the following: http://ServerName:Port/CollectionName If you do not know the correct path, contact your Team Foundation system administrator The team project that you want to use. This team project must be defined in the project collection that is specified by the /collection parameter. Optional. Specifies the name and password of a user who is logged on to the application-tier server for Team Foundation Server and who has permissions to run the command. You would use this option if your Windows credentials do not have the appropriate permissions, or you are using basic authentication, or you are not connected to a domain. Displays help about the command in the Command Prompt window.

/teamproject:

TeamProjectName

/login: UserName,Password

/? or help

167

LAB: Create Test Cases from an Assembly of Automated Tests

Open a Visual Studio command prompt. To do this, click Start, point to All Programs, point to Microsoft Visual Studio 2010, point to Visual Studio Tools, and then click Visual Studio 10.0 Command Prompt. The command prompt opens to the folder: <drive>:\Program Files\Microsoft Visual Studio 10.0\VC 1. Change directory to the folder that contains the assembly built from your test project. To do this, first change directory to your solution folder. Then change directory to the folder for your test project by typing the following command at the command prompt: cd <project name> \bin\Debug 2. To import all the test methods from your assembly, type the following at the command prompt: tcm testcase /collection:<CollectionURL> /teamproject: <Team Project name> /import /storage: <name of your assembly> .dll You will see results similar to these: Modification Added <Name of test method> Added <Name of test method> Title Summary Added 2 Updated 0 Total 2

Note: The test cases that are created will have the same names as the test methods that were imported. 3. (Optional) You can limit the test methods that are imported from the assembly based on test categories. For more information about test categories, see Defining Test Categories to Group Your Tests. For example, the following command shows you how to import tests that are in both the ShoppingCart and SmokeTest categories. tcm testcase /collection:<CollectionURL> /teamproject: <Team Project name> /import /storage: <name of your assembly> .dll /category:"ShoppingCart&SmokeTest" 4. (Optional) You can limit the test methods that are imported from the assembly based on a test method property. For example, the following command shows how to import all test methods that have priority set to 0 or 1.

168

tcm testcase /collection:<CollectionURL> /teamproject: <Team Project name> /import /storage: <name of your assembly> .dll/maxpriority:1 5. Open Microsoft Test Manager. Note: To display the Microsoft Test Manager window, click Start, and then click All Programs. Point to Microsoft Visual Studio 2010 and then click Microsoft Test Manager 2010. 6. To select a test plan, click the down-arrow on the center group switcher and then click Testing Center. 7. On the center group menu bar, click Plan. 8. (Optional) To select a different test plan, click the test plan name hyperlink in the upperright corner, or click the home icon. 9. You can add the test cases to a new test suite, or an existing test suite. To add the test cases into your plan, click Contents and then select the test suite in the test suite hierarchy or add a new test suite. Then click Add in the test suite details pane. The Add Test Cases to Suite dialog box is displayed. Note For more information about test suites in test plans, see Organizing Test Cases Using Test Suites. 10. Select the test cases that were created. These test cases will have the same names as the test methods in the test assembly. Then click Add Test Cases. The test cases are shown in the list of tests for the test suite that you selected. Note: You can also import your test cases directly into a static test suite. For more information about how to do this, see tcm: Importing Automated Tests into Test Cases.

169

To set up your test plan to run the automated test that you have created, you must choose the correct build definition used to build your automated test methods. You must do this so that the automated test assembly can be found in the share location for your build definition and then it can be run from Microsoft Test Manager.

EXERCISE:
1. Open Microsoft Test Manager. Note To display the Microsoft Test Manager window, click Start, and then click All Programs. Point to Microsoft Visual Studio 2010 and then click Microsoft Test Manager 2010. 2. To select a test plan, click the down-arrow on the center group switcher and then click Testing Center. 3. On the center group menu bar, click Plan. 4. To set up your test plan to run the automated tests, click Properties and then click Filter for builds. The Filter completed builds dialog box is displayed. 5. To select the build definition that is used to build your automated tests, click Build definition. 6. Each build can be given a specific value to reflect the quality of the build. To select the quality of the builds you want to be able to view, click Build quality. Note For more information about build definitions and build quality, see Define Your Build Process 7. To save your changes, click Set build filter. 8. To select the most recent build to use with this test plan that includes the latest changes to the automated test, you must first click Save to save the plan and then click Modify. The Assign Build activity is displayed. You can compare your current build with a build you plan to take. The associated items list shows the changes to work items between the builds. You can then assign the latest build to use for testing with this plan. For more information about how to select a build, see Determining Which Builds Have Bug Fixes, New Features, or Requirements. 9. To close the Assign Build activity and return to the test plan properties, click the Close icon.

170

To save these changes for this test plan, click Save in the toolbar.

EXERCISE:
To run your automated tests, you must use a physical or virtual environment. You cannot run automated tests using Microsoft Test Manager without an environment. You must create an environment that contains the roles in your test settings and then use this environment in your test plan. For more information about how to create your environment and roles and test settings, see Setting Up Test Machines to Run Tests or Collect Data. Note If you want to run automated tests that interact with the desktop, you must set up your agent to run as a process instead of a service. For more information, see How to: Set Up Your Test Agent to Run Tests that Interact with the Desktop. To create your test settings and environment to run your tests 1. To create a physical environment, follow the steps in this topic: Creating a Physical Environment to Use for Testing. 2. If you are using Visual Studio Lab Management, you can create a virtual environment. To create a virtual environment, follow the steps in this topic: How to: Create an Environment from Virtual Machines or Templates. If you want more information about how to create virtual environments, see Virtual Environments Concepts and Guidelines. 3. To create your test settings, follow the steps in this topic: Create Test Settings for Automated Tests as Part of a Test Plan.

EXERCISE:
EXERCISE: To run the automated test using Microsoft Test Manager 1. Open Microsoft Test Manager. Note To display the Microsoft Test Manager window, click Start, and then click All Programs. Point to Microsoft Visual Studio 2010 and then click Microsoft Test Manager 2010. 2. To run the automated test, click the down-arrow on the center group switcher and then click Testing Center. 3. In the center group menu bar, click Test.

171

4. (Optional) To override the build, the test settings or the environment to use for running the automated tests that you select in this test plan, right-click the test and then click Run with options. For example, if you want to run on a staging environment instead of your standard testing environment then you might choose a different environment. From the Run options dialog box, you can change these settings, and then click Run to run the selected test. Note If you select a different environment, it must have the same roles as the environment that was created in the test settings that you use. 5. To run the automated test without changing any options, right-click the test and then click Run. The Analyze Test Runs activity is displayed that shows the progress of the test run that contains this test. Note You can run multiple automated tests by selecting multiple tests, or you can select to run a whole suite of tests. To run a suite, right-click the test suite and then click Run.

172

EXERCISE:
To view and update the test results 1. Open Microsoft Test Manager. Note To display the Microsoft Test Manager window, click Start, and then click All Programs. Point to Microsoft Visual Studio 2010 and then click Microsoft Test Manager 2010. 2. To view the test results, click the down-arrow on the center group switcher and then click Testing Center. 3. On the center group menu bar, click Test and then click Analyze Test Runs. The Analyze Test Runs activity is displayed. It shows any test runs for this test plan 4. Double-click a test run to open it and view the details. The test run details are displayed. 5. (Optional) To update the title of your test run to be more meaningful, type the new name in Title. 6. (Optional) If your test failed, you can update the reason for the failure. Click Resolution and select the reason for the failure from the list. 7. (Optional) To add comments to the test result, click the Comments icon. Type your comments and then click Save comments. 8. (Optional) To view the details of an individual test, double-click the test. The test result is displayed. It shows the details from the test run, the attachments for data collected for this test result, and the test results history for that test. You can close this view to return to the test run. Note If you determine that there is a bug from your analysis, you can create a bug from this view. 9. To save these changes for this test run, click Save in the toolbar.

173

174

INDEX
Application Lifecycle Management, 4, 18, 19, 23, 25 Application Tier, 4, 21 Automated software testing, 4, 10, 11 Automatic Compiling, 4, 22 Client Tier, 4, 21 collect data remotely, 7, 63 Create a test plan, 6, 57 Creating and managing tests, 7, 63 Data Tier, 4, 21 Host, 5, 31, 32 Lab Center, 5, 33, 129 Library server, 5, 32 Library share, 5, 32 Manual software testing, 4, 10 Microsoft Test Manager, 4, 23, 24, 26, 27, 28, 33, 35, 51, 54, 56, 59, 61, 62, 63, 66, 67, 68, 69, 76, 129, 130, 131, 132, 133, 134, 135, 137, 138, 140, 141, 142, 144 Physical environment, 5, 33 Planning your testing effort, 6, 63 Running Your Tests, 7, 63 Setting up how to run tests, 7, 63 Snapshot, 5, 32, 34 Submitting bugs, 7, 63 tcm.exe, 8, 59, 137 Team project library, 5, 33 Template, 5, 32, 83, 84 Tracking software quality, 7, 63 UI Automation, 4, 11, 12, 13, 14, 15, 16 Virtual environment, 5, 33 Virtual machine, 5, 31 Visual Studio Ultimate, 4, 19, 23, 24, 49, 56, 62, 63, 89, 102

175

176

177

178