New T-SQL programmability features in SQL Server 2008

Praveen Srivatsa Director, AsthraSoft Consulting MSDN Regional Director, Bangalore

Session Objectives And Takeaways

Introduce top T-SQL programmability enhancements in SQL Server 2008 Learn how to develop powerful & efficient database applications Learn how to control any type of data Learn how to deliver optimized solutions

Agenda
Data Types SQL Query Language Procedural Language CLR Integration Globalization Beyond Relational Other

Separate date & time data types

Larger fractional seconds precision Larger range of dates

Time zone aware date/time data types

New Date and Time Data Types

DEFAULT - 100 nanoseconds (10-millionth of a second) Optional type parameter

Can be an integer from 0 to 7(100 nanoseconds) Precision/Scale metadata is becoming meaningful
Column Type DATE TIME
DATETIMEOFFSET DATETIME2 DATETIME2 (0)

Default string literal format YYYY-MM-DD HH:MI:SS [.nnnnnnn]

YYYY-MM-DD HH:MI:SS[.nnnnnnn] [+|-]HH:MI YYYY-MM-DD HH:MI:SS[.nnnnnnn] YYYY-MM-DD HH:MI:SS

Precision 10 16
34 27 19

Scale 0 7
7 7 0

Full SNAC (ODBC and OLEDB) support in Katmai Full SqlClient/ADO.net support in Orcas
SQL DATE ODBC SQL_TYPE_DATE/SQL_DATE OLEDB DBTYPE_DBDATE DateTime TIME SQL_TIME/SQL_SS_TIME2 DBTYPE_DBTIME/DBTYPE _DBTIME2 DBTYPE_DBTIMESTAMPOF FSET DBTYPE_DBTIMESTAMP TimeSpan ADO.NET

DATETIMEOFFSET

SQL_SS_TIMESTAMPOFFSET

DateTimeOffset

DATETIME2

SQL_TYPE_TIMESTAMP SQL_TIMESTAMP

DateTime

Code example (New Date/Time)

CREATE TABLE t1 (c1 DATE, c2 TIME(3), c3 DATETIME2(7) NOT NULL DEFAULT GETDATE(), c4 DATETIMEOFFSET CHECK (c4 < CAST(GETDATE() AS DATETIMEOFFSET(0))) );

INSERT INTO t1 VALUES ('0001-01-01', '23:59:59', '0001-12-21 23:59:59.1234567', '0001-10-21 23:59:59.1234567 -07:00');
INSERT INTO t1 VALUES ('9999-12-31', '23:59:59', '9999-12-31 23:59:59.1234567', '1111-10-21 23:59:59.1234567 -07:00'); SELECT c4, DATEPART(TZOFFSET, c4), DATEPART(ISO_WEEK, c4), DATEPART(MICROSECOND, c4) FROM t1;

Large UDTs
SQL Server 2005
User-Defined Types (UDTs) limited to 8K bytes Fine for most objects, but some spatial objects could be quite large

SQL Server 2008

UDTs can be of max line-of-business (LOB) size conceptually identical to varbinary(max) The size of a UDT is defined on the type Converted to varbinary(max) or image for downlevel clients

Can I not repeat this code ..

DECLARE @NewCustomer TABLE ( [CustomerID] int NULL, [FirstName] varchar(50) NOT NULL, [LastName] varchar(50) NOT NULL, [CompanyName] [nvarchar](128) NULL )

Introducing Table Types

User-defined Table Types
A new user defined type Aligned with inlined table definition for table variables Can be used for declaring table variables Can define indexes and constraints New Catalog view for table types Sys.table_types

Benefits
Usability, Type Matching, Precise Typing CREATE TYPE myTAS table (a int, b varchar(100))

Code example (New Table Type)

CREATE TYPE myTableTypeAS TABLE ( STOCK VARCHAR(10) PRIMARY KEY CLUSTERED, TradeDate DATE CHECK (TradeDate< convert(date,SYSDATETIME())), Quantity INT); DECLARE @stockTradesmyTableType; INSERT INTO @stockTrades VALUES ('MSFT',convert(date,SYSDATETIME()),1) INSERT INTO @stockTrades SELECT Stock, MAX(CONVERT(DATE, TradeTime)), SUM(Quantity) FROM DailyTrades GROUP BY Stock SELECT * FROM @stockTrades WHERE stock = 'MSFT' UPDATE@stockTrades SET stock = 'microsoft' WHERE stock = 'MSFT'

Agenda
Data Types SQL Query Language Procedural Language CLR Integration Globalization Beyond Relational Other

MERGE
New DML statement that combines multiple DML operations
Building block for more efficient ETL SQL-2006 compliant implementation
Source

XXXXX XXX XXX XXXX XXX XXXXXXXXXX X XXX XXXX XX XX XXXX XXXXX XXX XX

Source can be any table or query

MERGE
New DML statement that combines multiple DML operations
Building block for more efficient ETL SQL-2006 compliant implementation
Source Target

XXXXX XXX XXX XXXX XXX XXXXXXXXXX X XXX XXXX XX XX XXXX XXXXX XXX XX

XXXXX X XXXX

Target can be any table or updateable view

XXX XXX

MERGE
New DML statement that combines multiple DML operations
Building block for more efficient ETL SQL-2006 compliant implementation
Source Target

XXXXX XXX XXX XXXX XXX XXXXXXXXXX X XXX XXXX XX XX XXXX XXXXX XXX XX

XXXXX XXX XXX XXXX XXX

If source matches target, UPDATE

XXX XXX

MERGE
New DML statement that combines multiple DML operations
Building block for more efficient ETL SQL-2006 compliant implementation
Source Target XXXXX XXX XXX XXXX XXX XXXXXXXXXX X XXX XXXX XX XX XXXX XXXXX XXX XX XXX XXX

XXXXX XXX XXX XXXX XXX XXXXXXXXXX X XXX XXXX XX XX XXXX XXXXX XXX XX

If no match, INSERT

MERGE
New DML statement that combines multiple DML operations
Building block for more efficient ETL SQL-2006 compliant implementation
Source Target XXXXX XXX XXX XXXX XXX

XXXXX XXX XXX XXXX XXX XXXXXXXXXX X XXX XXXX XX XX XXXX XXXXX XXX XX

If source not matched, DELETE

XXXXXXXXXX X XXX XXXX XX XX XXXX XXXXX XXX XX XXX XXX

Code example (MERGE)

CREATE TABLE Source (id INT, name NVARCHAR(100), qty INT); CREATE TABLE Target (id INT, name NVARCHAR(100), qty INT); -- Synchronize source data with target MERGE Target AS t USING Source AS s ON t.id = s.id WHEN MATCHED AND (t.name != s.name OR t.qty!= s.qty) THEN -- Row exists and data is different UPDATE SET t.name = s.name, t.qty = s.qty WHEN NOT MATCHED THEN -- Row exists in source but not in target INSERT VALUES (s.id, s.name, s.qty) WHEN SOURCE NOT MATCHED THEN -- Row exists in target but not in source DELETE OUTPUT$action, inserted.id, deleted.id;

Grouping Sets
Extension to the GROUP BY clause Ability to define multiple groupings in the same query Produces a single result set that is equivalent to a UNION ALL of differently grouped rows Makes aggregation querying and reporting easier and faster

Code example (GROUPING SETS)

-- Use UNION ALL on dual SELECT statements
SELECT customerType,Null as TerritoryID,MAX(ModifiedDate) FROM Sales.Customer GROUP BY customerType UNION ALL SELECT Null as customerType,TerritoryID,MAX(ModifiedDate) FROM Sales.Customer GROUP BY TerritoryID order by TerritoryID

-- Use GROUPING SETS on single SELECT statement

SELECT customerType,TerritoryID,MAX(ModifiedDate) FROM Sales.Customer GROUP BY GROUPING SETS ((customerType), (TerritoryID)) order by customerType

Agenda
Data Types SQL Query Language Procedural Language CLR Integration Globalization Beyond Relational Other

How to work on tabular data

Using local temporary tables
Increasing the disk I/O Being prone to locking and blocking Needing manually dropping the temporary table Leading to frequent stored procedures recompilations

Using multiple parameters

Multiple round trips Stored procedure needs to be executed multiple times Inefficient code

Table-valued Parameters (TVP)

Parameters of type Table Type Input parameters on SPs/Functions New ReadOnly keyword is needed. TVPs are scoped within the SP/function body Optimized to scale and perform better for large data Behaves like BCP inside server

a simple programming model. Strongly typed. Reduce client/server round trips Do not cause a statement to recompile.

Code example (TVP)

-- Create a user TABLE type CREATE TYPE myTableType AS TABLE (id INT, name NVARCHAR(100), qty INT); -- Create a stored procedure that accepts a table-variable -- of type TABLE as a parameter CREATE PROCEDURE myProc (@tvpmyTableType READONLY) AS UPDATE Inventory SET qty += s.qty FROM Inventory AS i INNER JOIN @tvpAS tvp ON i.id = tvp.id GO -- Declare & populate variable of the TABLE type DELCARE @list AS myTableType; INSERT INTO @list VALUES (1, Bicycle, 10), (2, Roller blades, 5), (3, Soccer ball, 25); -- Execute the stored procedure with TVP EXEC myProc @list;

Table-valued Parameters
Client Stack Support
Fully supported in ADO.Net 3
New Parameter type: SqlDbType.Structured Parameters can be passed in multiple ways
DataTable Ienumerable<SqlDataRecord> (fully streamed) DbDataReader

Supported in ODBC/OLEDB stacks

New Parameter Type SQL_SS_Table Familiar Parameter Binding: SQLBindParameter

Efficient implementation of TDS layer

Table-valued Parameters
ADO.Net Example using DataTable
Using (MyConnection){ //Create a data table DataTabledt = new DataTable(TVPOrdersDataTable); dt.Columns.Add(ProductType, typeof(string)); dt.Columns.Add(Quantity, typeof(int)); // Add rows dt.Rows.Add(Canon Digital Camera, 20); dt.Rows.Add(June, 10); dt.Rows.Add(Xbox-360, 8); // Create a command and bind parameter SqlCommandtvp_cmd = new SqlCommand(sp_UpdataInventory, MyConnection); SqlParametertvpParam = tvp_cmd.Parameters.AddWithValue( @OrdersTvp, dt); //Execute command tvp_cmd.ExecuteNonQuery();

T-SQL Delighters (1)

Insert multiple rows through VALUE clause of a single INSERT statement
INSERT INTO contacts VALUES ('John Doe', '425-333-5321'), ('Jane Doe', '206-123-4567'), ('John Smith', '650-434-7869');

Assignment operators: +=, -=, *=, /=

UPDATE Inventory SET quantity += s.quantity FROM Inventory AS i INNER JOIN Sales AS s ON i.id = s.id

Variable initialization during declaration

DECLAER @v int = 5; DECLARE @v1 varchar(10) = xxxxx;

T-SQL Delighters (2)

Extending the CONVERT function to provide new conversion styles between binary hex (hex) data and character data:
select convert(char(4), col1_of_type_binary,1), .... from t1 ...... select ...... from t1, t2 where convert(char(4), t1.col1_of_type_binary,1) = t2.col1_of_type_char ;

Object Dependencies (1)

Reliable discovery of dependencies between objects
Stored procedures, tables, views, functions, triggers, UDTs, etc Schema-bound and non-schema-bound objects

Scenarios
Discover all objects that a given object depends on Discover all objects that depend on a given object Discover all objects that depend on another database Discover all objects performing distributed queries using four-part names

Object Dependencies (2)

sys.sql_expression_dependencies

New catalog view; replaces sys.sql_dependencies Tracks both schema-bound and non-schema-bound dependencies Tracks cross-database and cross-server references (by name)
sys.dm_sql_referenced_entities

New dynamic management function; replaces sp_depends Returns a row for each entity referenced by a given entity For example, show me all objects referenced in stored procedure p1
sys.dm_sql_referencing_entities

New dynamic management function; replaces sp_depends Returns a row for each entity that references a given entity For example, show me all objects that would be broken if I drop table t1

CREATE PROCEDURE p1 @a INT, @b myUDT OUTPUT AS DECLARE @x INT, @y INT; EXEC p2; GO SELECT <see column list below> FROM sys.sql_expression_dependencies WHERE referencing_id = OBJECT_ID(p1); SET @b = CAST (@x, @y) AS myUDT;

Object Dependencies (3)

SELECT a, @x = s.foo(b), @y = MAX(c) FROM t1 WHERE a = @a;

Object Dependencies (4)

USE db1 CREATE PROCEDURE dbo.p2 SELECT * FROM t1; CREATE PROCEDURE p3 UPDATE dbo.t1 CREATE VIEW v1 WITH SCHEMABINDING SELECT t1.*, t2.* FROM dbo.t1 INNER JOIN dbo.t2 CREATE FUNCTION s.foo (@x INT) RETURNS TABLE AS BEGIN SELECT * FROM t1 WHERE a < @x; END USE db2 CREATE PROCEDURE p4 -- cross db dependency doesnt -- show up as a referencing entity SELECT * FROM db1..t1; SELECT referencing_schema_name, referencing_entity_name, referencing_id, is_caller_dependent FROM sys.dm_sql_referencing_entities(dbo.t1);

DDL Trigger Enhancements

Enhanced (Data Definition Language) DDL events to include all DDL operations
Stored procedures like sp_rename Language DDL like full-text, security DDL

Persisting event groups

No expansion of groups at creation

Exposing (XML Schema Definition) XSD for event schema

Change Data Capture

Mechanism to easily track changes on a table
Changes captured from the log asynchronously Information on what changed at the source

Table-Valued Functions (TVF) to query change data

Easily consumable from Integration Services

XXXXX XXX XXX XXXX XXX XXXXXXXXXX X XXX XXXX XX XX XXXX XXXXX XXX XX

XXXXX XXX XXX XXXX XXX XXXXXXXXXX X XXX XXXX XX

Transaction Log

Capture Process

Source Table

XXX XXX XXXXXX XXX

CDC Functions

Change Table

Change Data Capture

Mechanism to easily track changes on a table
Changes captured from the log asynchronously Information on what changed at the source

Table-Valued Functions (TVF) to query change data

Easily consumable from Integration Services

Capture XXXXXXXXXX Process sys.sp_cdc_enable_db_change_data_capturesy Transaction Log X XXX XXXX XX

XXXXX XXX XXX XXXX XXX XXXXXXXXXX X XXX XXXX XX

XXXXX XXX XXX XXXX XXX

s.sp_cdc_enable_table_change_data_capture
Source Table
XXX XXX XXXXXX XXX

XX XXXX XXXXX XXX XX

cdc.fn_cdc_get_all_changes_<instance> cdc.fn_cdc_get_net_changes_<instance> CDC

Functions

Change Table

Agenda
Data Types SQL Query Language Procedural Language CLR Integration Globalization Beyond Relational Other

SQL/CLR Enhancements
Large aggregates
Store more than 8K bytes in state

Multi-input aggregates Specifying sort order and uniqueness for table-valued functions (TVFs)
Better performance

Easy access to Microsoft .NET Framework functionality

Ability to register static methods as user-defined functions (UDFs)

Agenda
Data Types SQL Query Language Procedural Language CLR Integration Globalization Beyond Relational Other

New Collations
Align with Windows Vista collations Adding Windows new collations in SQL Server 2008 Adding new versions to existing Windows collations (*_100_*) Adding new versions to existing Windows collations with SIGNIFICANT CHANGES
Chinese_Taiwan_Stroke_100 and Chinese_Taiwan_Bopomofo_100 will now assign culturecorrect weight for each character, specifically the Ext. A + B characters

Agenda
Data Types SQL Query Language Procedural Language CLR Integration Globalization Beyond Relational Other

Spatial Goes Mainstream

Extends SQL Server with types, operations, and indexing to enable working with spatial geometry Simplifies storage of location data Improves SQL Server as platform for geospatial independent software vendors (ISVs) Standards-based data and programming model Based on large UDTs

Semi-Structured Data
Various enhancements to make SQL Server a better store for semi-structured data
XML: Improved schema and data model support. Hierarchy Id: compact encoding and representation of hierarchical data Sparse Columns: efficient storage of columns with sparsely populated values Large number of columns per table / index Filtered Indexes: indexing subsets of rows

HierarchyID
A system data type with variable length
CLR UDT Microsoft.SqlServer.Types Varbinary encoding ( < 900 bytes)

To represent position in a hierarchy Logically encodes information about a single node in a hierarchy tree by encoding the path from the root of the tree to the node Rich built-in methods for manipulating hierarchies Simplifies storage and querying of hierarchical data Comparison is in depth-first order, a<b means a comes before b in a depth first traversal of the tree Support for arbitrary insertions and deletions

Sparse Columns ...

Sparse as a storage attribute on a column
Storage Optimization: 0 bytes stored for a NULL value Co-location of data: performance benefits NULL Compression in the (Tabular Data Stream) TDS layer No change in Query/DML behavior

// Sparse as a storage attibute in Create/Alter table statements Create Table Products(Id int, Type nvarchar(16), Resolution int SPARSE, ZoomLengthint SPARSE);
// No Change in Query/DML Behavior Select Id, Type, Resolution, ZoomLength from Products; Update Products set Resolution=3, ZoomLength = 105 where Id = 101;

Filtered Indexes (1)

Filtered Indexes and Statistics
Indexing a portion of the data in a table Filtered/co-related statistics creation and usage Query/DML Optimization to use Filtered indexes and Statistics

Restrictions
Simple limited grammar for the predicate Only on non-clustered indexes

Benefits
Lower storage and maintenance costs for large number of indexes Query/DML Performance Benefits: IO only for qualifying rows

Filtered Indexes (2)

ID Type Brand Zoo m Resolution WaistSize Inseam Price ..

1 2
3 4 5 6 7

Camera Pant
Camera Camera Pant Pant ..

Canon Dockers
Nikon Pentax Polo Dockers

3x

5 Mb 38 34

300 45
600 195 30 40 32 32 65 45

5x 3x

10mb 3mb

// Create a Filtered Indexes Create Index ZoomIdx on Products(ZoomLength) where Type = Camera; Create Index PlusSizeIdx on Products(WaistSize) where Type = Pant and WaistSize> 36 // Optimizer will pick the filtered index when query predicates match Select ProductId, Type, Resolution, ZoomLength where Type = Camera Select ProductId, Type, WaistLength, Inseam where Type = Pant and WaistLength> 38 // DML operations need to maintain indexes only when needed. Insert into Products(ProductId, Type, waistLength, inseam) values (201, Pant, 30, 32);

Agenda
Data Types SQL Query Language Procedural Language CLR Integration Globalization Beyond Relational Other

Deprecation
Getting more serious about deprecation policy Deprecation announcements since SQL 7.0
Formalized policy in SQL Server 2005

Deprecation stages
Announcement/Warning Final support/Removal Future deprecation

SQL Server 2008 implements the deprecation policy

Perf counters for all deprecated features Trace events to track deprecated feature usage
DEPRECATION_ANNOUNCEMENT DEPRECATATION_FINALSUPPORT

SQL Server Resources

Microsoft SQL Server www.microsoft.com/sql/default.mspx

Microsoft SQL Server Books Online http://msdn2.microsoft.com/en-us/library/ms130214.aspx

SQL Server Developer Center http://msdn2.microsoft.com/en-us/sql/default.aspx T-SQL Programmability Resources http://msdn2.microsoft.com/en-us/sql/aa336296.aspx

Microsoft SQL Server 2008 June CTP www.microsoft.com/sql/prodinfo/futureversion/default.mspx

Microsoft SQL Server 2008 MSDN forums http://forums.microsoft.com/MSDN/default.aspx?ForumGroupID=428&SiteID=1

More Resources
Technical Communities, Webcasts, Blogs, Chats, and User Groups www.microsoft.com/communities/default.mspx

Microsoft Learning and Certification www.microsoft.com/learning/default.mspx

MSDN, the Microsoft Developer Network, and TechNet http://microsoft.com/msdn http://microsoft.com/technet

Trial Software and Virtual Labs www.microsoft.com/technet/downloads/trials/default.mspx

Announcements

 2007 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.