Académique Documents
Professionnel Documents
Culture Documents
http://www.agiledata.org/essays/databaseTesting.html
| Contact Me
Relational database management systems (RDBMSs) often persist mission-critical data which is updated by many applications and potentially thousands if not millions of end users. Furthermore, they implement important functionality in the form of database methods (stored procedures, stored functions, and/or triggers) and database objects (e.g. Java or C# instances). The best way to ensure the continuing quality of these assets, at least from a technical point of view, you should have a full regression test suite which you can run on a regular basis. In this article I argue for a fully automated, continuous regression testing based approach to database testing. Just as agile software developers take this approach to their application code, see Agile Testing and Quality Strategies, we should also do the same for our databases.
Ads by Google
Table of Contents
1. Why test an RDBMS? 2. What should we test? 3. When should we test? 4. How should we test? Database sandboxes Writing database tests Setting up database tests Database testing tools 5. Who should test? 6. Introducing database testing into your organization 7. Database testing and data inspection 8. Best practices
1 of 8
11/29/2011 12:26 AM
http://www.agiledata.org/essays/databaseTesting.html
professionals seem to think that testing is something that other people do, particularly test/quality assurance professionals, do. This reflects a penchant for over-specialization and a serial approach towards development by traditionalists, two ideas which have also been shown to be questionable organizational approaches at best.
Table 1. What to test in an RDBMS. Black-Box Testing at the Interface White/Clear-Box Testing Internally Within the Database Scaffolding code (e.g. triggers or updateable views) which support refactorings Typical unit tests for your stored procedures, functions, and triggers Existence tests for database schema elements (tables, procedures, ...) View definitions Referential integrity (RI) rules Default values for a column Data invariants for a single column Data invariants involving several columns
O/R mappings (including the meta data) Incoming data values Outgoing data values (from queries, stored functions, views ...)
2 of 8
11/29/2011 12:26 AM
http://www.agiledata.org/essays/databaseTesting.html
Test-driven development (TDD) is an evolutionary approach to development which combines test-first development and refactoring. When an agile software developer goes to implement a new feature, the first question they ask themselves is "Is this the best design possible which enables me to add this feature?" If the answer is yes, then they do the work to add the feature. If the answer is no then they refactor the design to make it the best possible then they continue with a TFD approach. This strategy is applicable to developing both your application code and your database schema, two things that you would work on in parallel. When you first start following a TDD approach to development you quickly discover that to make it successful you need to automate as much of the process as possible? Do you really want to manually run the same build script(s) and the same testing script(s) over and over again? Of course not. So, agile developers have created OSS tools such as ANT, Maven, and Cruise Control (to name a few) which enable them to automate these tasks. More importantly, it enables them to automate their database testing script into the build procedure itself.
Agile developers realize that testing is so important to their success that it is something they do every day, not just at the end of the lifecycle, following agile testing strategies. They test as often and early as possible, and better yet they test first. As you can see with the agile system development lifecycle (SDLC) of Figure 3 testing is in fact something that occurs during the development and release cycles, not just during release. Furthermore, many agile software developers realize that you can test more than just your code, you can in fact validate every work product created on a software development project if you choose to. This philosophy is exemplified by the Full Lifecycle Object-Oriented Testing (FLOOT) Methodology.
3 of 8
11/29/2011 12:26 AM
http://www.agiledata.org/essays/databaseTesting.html
4. How to Test
Although you want to keep your database testing efforts as simple as possible, at first you will discover that you have a fair bit of both learning and set up to do. In this section I discuss the need for various database sandboxes in which people will test: in short, if you want to do database testing then you're going to need test databases (sandboxes) to work in. I then overview how to write a database test and more importantly describe setup strategies for database tests. Finally, I overview several database testing tools which you may want to consider.
4 of 8
11/29/2011 12:26 AM
http://www.agiledata.org/essays/databaseTesting.html
1. Setup the test. You need to put your database into a known state before running tests against it. There are several strategies for doing so. 2. Run the test. Using a database regression testing tool, run your database tests just like you would run your application tests. 3. Check the results. You'll need to be able to do "table dumps" to obtain the current values in the database so that you can compare them against the results which you expected. The article What To Test in an RDBMS goes into greater detail.
I believe that there are several critical features which you need to successfully test RDBMSs. First, as Figure 1 implies you need two categories of database testing tools, one for interface tests and one for internal database tests. Second, these testing tools should support the language that you're developing in. For example, for internal database testing if you're a Microsoft SQL Server developer, your T-SQL procedures should likely be tested using some form of T-SQL framework. Similarly, Oracle DBAs should have a PL-SQL-based unit testing framework. Third, you need tools which help you to put your database into a known state, which implies the need not only for test data generation but also for managing that data (like other critical development assets, test data should be under configuration management control).
To make a long story short, although we're starting to see a glimmer of hope when it comes to database testing tools, as you can see in Table 2, but we still have a long way to go. Luckily there are some good tools being developed by the open source software (OSS) community and there are some commercial tools available as well. Having said that, IMHO there is still significant opportunity for tool vendors to improve their database testing offerings. Table 2. Some database testing tools. Category Data Privacy Tools Description Data privacy, or more generally information privacy, is a critical issue for many organizations. Many organizations must safeguard data by law due to regulatory compliance concerns. Examples IBM Optim Data Privacy tools
Empirix Mercury Interactive Tools simulate high usage loads on your database, enabling you Testing tools to determine whether your system's architecture will stand up to for load testing your true production needs. RadView Rational Suite Test Studio Web Performance
5 of 8
11/29/2011 12:26 AM
http://www.agiledata.org/essays/databaseTesting.html
Data Factory Test Data Generator Developers need test data against which to validate their systems. Test data generators can be particularly useful when you need large amounts of data, perhaps for stress and load testing. Datatect DTM Data Generator Turbo Data Your test data needs to be managed. It should be defined, either manually or automatically (or both), and then maintained under version control. You need to define expected results of tests and then automatically compare that with the actual results. You may even want to retain the results of previous test runs (perhaps due to regulatory compliance concerns).
AnyDbTest DBFit DBUnit NDbUnit Unit testing tools OUnit for Oracle (being replaced soon by Qute) SQLUnit TSQLUnit (for testing T-SQL in MS SQL Server) Visual Studio Team Edition for Database Professionals includes testing capabilities XTUnit
6 of 8
11/29/2011 12:26 AM
http://www.agiledata.org/essays/databaseTesting.html
8. Best Practices
I'd like to conclude this article by sharing a few database testing "best practices" with you: 1. Use an in-memory database for regression testing. You can dramatically speed up your database tests by running them, or at least portions of them, against an in-memory database such as HSQLDB. The challenge with this approach is that because database methods are implemented differently across database vendors that any method tests will still need to run against the actual database server. 2. Start fresh each major test run. To ensure a clean database, a common strategy is that at the beginning of each test run you drop the database, then rebuild it from scratch taking into account all database refactorings and transformations to that point, then reload the test data, and then run your tests. Of course, you wouldn't do this to your production database. ;-) 3. Take a continuous approach to regression testing. I can't say this enough, a TDD approach to development is an incredibly effective way to work. 4. Train people in testing. Many developers and DBAs have not been trained in testing skills, and they almost certainly haven't been trained in database testing skills. Invest in your people, and give them the training and education they need to do their jobs. 5. Pair with novices with people that have database testing experience. One of the easiest ways to gain database testing skills is to pair program with someone who already has them.
This book describes, in detail, how to refactor a database schema to improve its design. The first section of the book overviews the fundamentals evolutionary database techniques in general and of database refactoring in detail. More importantly it presents strategies for implementing and deploying database refactorings, in the context of both "simple" single application databases and in "complex" multi-application databases. The second section, the majority of the book, is a database refactoring reference catalog. It describes over 60 database refactorings, presenting data models overviewing each refactoring and the code to implement it.
This book describes the philosophies and skills required for developers and database administrators to work together effectively on project teams following evolutionary software processes such as Extreme Programming (XP), the Rational Unified Process (RUP), the Agile Unified Process (AUP), Feature Driven Development (FDD), Dynamic System Development Method (DSDM), or The Enterprise Unified Process (EUP). In March 2004 it won a Jolt Productivity award.
7 of 8
11/29/2011 12:26 AM
http://www.agiledata.org/essays/databaseTesting.html
This book presents a full-lifecycle, agile model driven development (AMDD) approach to software development. It is one of the few books which covers both object-oriented and data-oriented development in a comprehensive and coherent manner. Techniques the book covers include Agile Modeling (AM), Full Lifecycle Object-Oriented Testing (FLOOT), over 30 modeling techniques, agile database techniques, refactoring, and test driven development (TDD). If you want to gain the skills required to build mission-critical applications in an agile manner, this is the book for you.
Let Me Help
I actively work with clients around the world to improve their information technology (IT) practices as both a mentor/coach and trainer. A full description of what I do, and how to contact me, can be found here.
Copyright 2006-2010 Scott W. Ambler This site owned by Ambysoft Inc. Agile Modeling (AM) | Agile Unified Process (AUP) | Enterprise Unified Process (EUP) | My Writings | IT Surveys
8 of 8
11/29/2011 12:26 AM