SOA Testing SOA Framework is flexible set of design principles used during the phases of system development and

integration in computing. A System based on SOA will provide a loosely coupled unit of Services that can be used within multiple separate systems from several business domains.SOA defines the interface in terms of protocols and functionality. Basic Components of SOA Service provider Service consumer Service registry

The service provider creates a service and in some cases publishes its interface and access information to a service registry. The service registry is responsible for making the service interface and implementation access information available to service consumers. The service consumer locates entries in the service registry and then binds to the service provider in order to invoke the defined service. Each component can also act as one of the two other components. For instance, if a service provider needs additional information that it can only acquire from another service, it acts as a service consumer. Web Services are used to implement SOA By using Web services, an application can publish its function or message to the rest of the world. Web services use XML to code and to decode data, and SOAP to transport it (using open protocols) Web Services have three basic platform elements SOAP: Simple Object Access Protocol It is an XML based protocol to let applications exchange information over HTTP. SOAP is a protocol for accessing a Web Service WSDL: Web Services Description Language It is an XML based language for locating and describing a Web Service UDDI: Universal Description, Discovery and Integration It is directory Service where companies can register and Search for Web Services Quality of service requirements for web services is: 1. 2. 3. 4. 5. 6. 7. Availability (is it running) Accessibility (can I run it now) Integrity/reliability (will it crash while I run/how often) Throughput (how many simultaneous requests can I run) Latency (response time) Regulatory (conformance to standards) Security (confidentiality, authentication)

SOA Test Approach Since Web Services are composed of 'loosely' coupled distributed over networks we must test the application o End to End o Service by Service o Interface by Interface And that is why a bottom up test approach is recommended. The levels reflect different viewpoints of testing on different levels. Test individual functions within a service Test an individual service Test a composite set of services through to testing an integrated process and Test a complete business system. SOA Testing can be categorized into following Phases Service Component Level Testing Service Level Testing Integration-Level Testing System-Level Testing Test phases: 1. Service- Component-Level Testing Service-component-level testing or Unit testing is normally performed by the developers to test that the code not only successfully compiles, but the basic functionality of the components and functions within a service are working as specified. Each Component is tested separately before integrating it into a service or services 2. Service-level testing Service testing will be the most important test level/phase within our SOA Test approach. Functional, performance and security regression suites to be executed against each Web service. 3. Integration-level testing The integration test phase will focus on service interfaces. This test phase aims to determine if interface behavior and information sharing between the services, are working as specified. 4. System-Level Testing This test phase will test that the SOA technical solution has delivered the defined business requirements and has met the defined business acceptance criteria.

Test Phases and Test Types The following table summarizes test types that are required in the different test phases. Test Phase Service-Component-Level Service-Level Integration-Level System-Level Functional Testing Yes Yes Yes Yes Performance Testing Yes Yes No Yes

Functional Testing: Functional or Black Box testing will determine if a component or a service or the whole system is operating to specification Web Service Testing typically includes the following tasks: 1. 2. 3. 4. Import WSDL file Define Test Data Invoke Web services by sending arguments to the Web method that is to be tested Verify actual Response is similar to that of Expected Response

Performance Testing: Application needs to be performance tested in the following manner: End to end from the requester perspective At the unit level during development (by provider) At service level (generally by requester and provider) Interface validation (generally by requester and provider) To ensure functionality under boundary load conditions Performance testing will determine Response Time, Throughput, and Error Percentage of a Service Request Test Tools: 1. Web Service Explorer (Integrated with Eclipse IDE) For Service-Level Functional Testing 2. JMeter (Web Service (SOAP) Request Sampler) For Functional and Performance Testing Other Open Source Tools 3. SOAP UI LOADUI For Functional and Performance Testing

Regression Testing Strategy: Regression testing is also known as validation testing and provides a consistent, repeatable validation of each change to services under development or being modified. Each time a defect is fixed, the potential exists to inadvertently introduce new errors, problems, and defects. An element of uncertainty is introduced about ability of the service to repeat everything that went right up to the point of failure. Regression testing is the selective retesting of a service or SOA system that has been modified to ensure that no previously working services fail as a result of the repairs. Test Schedule

Module New Service Connections Disconnection & Dismantlement Collections Metering Billing Energy Auditing CSC Accounting

Screens

Est. Effort (Hours)

Start Date

End Date

Scenarios for Load Testing What to test? Occasionally, we encounter customers who have only a single scenario to test. For example, one client developed an application in which the only scenario of interest involved a user registering him/herself with the system and scheduling an appointment. When this happens, the tester may devote their efforts to the accurate simulation, testing and analysis of this single scenario. You are unlikely to be so lucky. In any moderately complex system, there are dozens or even hundreds of scenarios that are candidates for load testing. You are not likely to have the time or resources to test them all. As a result, you must make some tough decisions about which scenarios will be included in the test plans. There are a number of key factors that will raise or lower the importance of each scenario: * Criticality How critical is the scenario to the overall operation of the application? * Frequency How often is the scenario expected to be performed? * Difficulty How difficult is the scenario to simulate? * Verifiability Can correct operation of the scenario be easily verified?

The first two factors, criticality and frequency, should be the keys in deciding IF the scenario is tested. Scenarios that are both critical and frequent must be load tested. Those that are neither will typically be left out of the testing process entirely. The remaining scenarios, those that are somewhat critical or somewhat frequent or a little of both should be tested as the schedule allows. We use the second two, difficulty and verifiability, to determine WHEN the scenario will be tested. This is particularly true at the beginning of the testing effort when the tester is not yet experienced with the application, tools and/or testing process. The first scenarios to be tested should be those that are easy to simulate and easy to verify. Unfortunately, at this point the inexperienced tester is not likely to know which scenarios will be difficult to simulate. You will have to substitute complexity as a measure of simulation difficulty until that knowledge is gained. As an example, in a CRM system, a scenario that consists of logging into the system and sending an e-mail is both relatively simple and easy to verify. You may be asking Why does the ease of testing or verification matter? Shouldnt we always be testing the most critical and frequent testcases first? The answer is a resounding No! If we lived in a world with no schedules, I might be inclined to agree with you. But this is the real world. Besides the realities of scheduling, there are two other important facts to take into account. First, bugs cost less to fix earlier in the development/testing cycle. Second, a large percentage of performance problems are related to system configuration, framework limitations or other system-wide factors. In these cases, exercising nearly any scenario will uncover the problem. To put that in plain english: testing 5 less-critical or less-frequent scenarios this week is better than testing 2 critical and frequent scenarios next month. If your organization does not have significant experience with load testing, this effect is amplified, since the testing schedule will be further extended as a result of the learning curve. Our Preferred Procedure Summarizing from above, we generally follow the procedure: 1. Pick one or two simple, easily verifiable scenarios and test them. We use this phase to ensure that the testing infrastructure is in place and operating properly. 2. Add scenarios that are easy to simulate and either frequent or critical to the test suite. 3. Add the most critical and most frequent scenarios, regardless of difficulty, to the test suite. I hope this helps you plan your load testing more effectively.