Vous êtes sur la page 1sur 8

efficient paging of recordsets: sql server 2000

here's the most efficient way that i can think of to return rows @a through @b
from a table or query, when each row is ordered and can be uniquely identified by
a set of columns.

in this example, we have a table called �data�, and we wish to sort by transdate.
the primary key of the table is �id� (yep, an identity !) so we include that as
the secondary sort to ensure all rows can be uniquely identified by the sort
columns.

the plan: set the rowcount to the starting row number (in this case, @a), and get
the value of the �starting points� for each sort column. then, set the rowcount
to the number of rows to return (@b-@a), set up a where clause so that we start
at the starting point, and then return the results.

this avoids temp tables and always processes as few rows as possible -- at most,
@b rows. so it will get slower as you go. but, it works pretty well so give it a
shot. if your ordering needs to be dynamically assigned, then this technique may
not do much for you but it may help give you some ideas.

update #1 (3/22/04) : i have slightly edited the code to fix the "first row"
error, as specified in the comments.

update #2 (4/8/07): if you are using sql server 2005, it is much easier ... see
this.

--------------------------------------------------------------------------------

-- for each column in your sort, you need a variable to hold


-- the "starting values". in our case, we need two:

declare @startingdate datetime;

declare @startingid int;

-- again, we want to returns results from row @a to row @b:

declare @a int;

declare @b int;

set @a = 200 -- start at row 200

set @b = 250 -- end at row 250

-- get the starting date and starting id to return results:

set rowcount @a
select @startingdate = transdate, @startingid = id

from data

order by transdate asc,id asc

-- find out how many rows to return, and set the rowcount:

set @b = 1 + @b - @a

set rowcount @b

-- now return the results:

select * from data

where

transdate > @startingdate or

(transdate = @startingdate and id >= @startingid)

order by transdate asc, id asc

-- clean up:

set rowcount 0

--------------------------------------------------------------------------------

the where clause in the final sql statement is the key, and it is good to be
familiar with this type of clause even if you don't use this exact paging
technique. look at it closely, and you will see it expresses logic to return rows
within a certain range when that range is defined by more than 1 column.

for example, if you have a column called �yearmonth� with values in a yyyy-mm
format, you can easily say:

where yearmonth >= '2003-04'

but if you have two separate columns, one for month and one for year, you would
express that same where clause this way:

where year > 2003 or (year = 2003 and month >= 04)

print | posted on monday, december 22, 2003 12:31 pm


feedback
# re: efficient paging of recordsets with t-sql
good job! i posted my own approach (linked to my name here) in response. it is
more database independent and reusable, but moves the record-skipping logic to
.net (there are advantages and disadvantages).
12/22/2003 3:48 pm | richard tallent
# re: efficient paging of recordsets with t-sql
what about order desc and using top 1??
12/23/2003 6:53 pm | valterborges
# re: efficient paging of recordsets with t-sql
this code has a bug in it that means it cannot return the very first row. the
example starts from row 200 but actually returns the next row, ie 201 (since the
first select gets the item at row 200, and the second selects gets the next value
> than this value).

this means that if you want row 1, you should specify row 0, ie to get the next
row. however if you do that, the first select will do a rowcount 0 and set the
starting row to the last row of the select. (and if you specify row 1, it will
return from row 2.)

my fix is to surround the second select with "if @@rowcount >= @nstartrow" so it
only does it if the first select returned more rows that the row we want to start
from. and change the where clause to "transdate = @startingdate and id >=
@startingid".

it now includes the start row, so the example would return from 200 rather than
201. and so it allows row 1 to be returned.
12/29/2003 5:32 am | thekode
# re: efficient paging of recordsets with t-sql
thanks for the fix! sorry i missed that.

valter -- not sure what you mean -- can you elaborate ?


12/30/2003 2:07 pm | jeff
# efficient and dynamic server-side paging with t-sql
3/22/2004 10:29 pm | jeff's blog
# re: efficient paging of recordsets with t-sql
fyi -- i have altered the code slightly to handle the "skipped first row" issue.
it should work quite well now. check out my dynamic version of this technique in
my latest blog ....
3/24/2004 3:22 pm | jeff
# paging a recordset - sql server side.
3/27/2004 4:14 pm | dim blog as new thoughtstream(me
# paging dr. recordset, dr. recordset, line 1.
3/29/2004 9:00 pm | dim blog as new thoughtstream(me
# paging a recordset - sql server side.
3/29/2004 9:02 pm | dim blog as new thoughtstream(me
# creating effecient paging with t-sql and the identity function
creating effecient paging with t-sql and the
identity function

now we all had the problem of creating a grid

with next and back buttons , as it retreieves

all the data from the server and filters it

on the client site .that is a poor technique


if you have a huge table ,few users will hang

your machine and result in memory consumption

sql server lacks a way to limit the number

of rows returned to the client in ranges ,and

you have to write a very complex query to

achieve this result ,like getting rows from 5

to 10 ,and you will depend on a primary key

for sure .lets first discuss how mysql and

oracle solves this problem ...

mysql has a very pretty code to solve this

problem you can say :

select * from foo limit 5,10

this will retrieve the rows starting row

number 5 till row number 15 .very pretty code

cause u just drop this { limit n,m } at the

end of your most complex query and you are

done .

oracle has a similar approach ,it has a

psuedocolumn always availiable to you when

you issue a query , this column have a name

of rownum .
if you do something like this
select rownum ,* from foo you will get

rownum col1 col2


1 data data
2 data data
3 data data

now to page just say where rownum between n

and m
pretty clean code just like the mysql code

now back to my only love sqlserver ,which is


the best engine i worked on despite this

issue ,there is no direct way to return a

range ,you can only return top ,but no range

here is a little piece of code that you can

compile as a stored procedure

set rowcount 10 -- set this to your desired

page size ,this is for optimization ,the

between clause will be enough

select identity(int, 1,1) as 'rownum' ,*

into #temptable from foo


select * from #temptable will yield this

rownum col1 col2


1 data data
2 data data
3 data data

which is similar to the oracle approach

,and the code will be even compatible with

oracle as i can say


select * from #temptable where rownum between
n and m

your code will be easy to read and understand


amr salah
6/7/2004 8:04 am | prince of egypt
# re: efficient paging of recordsets with t-sql
this thread is old, but interesting, so i thought i'd take a poke...

if you know beforehand that your always going to need rows 200 to 250, this would
work:

select *
from (select top 50 *
from (select top 250 *
from data
order by transdate asc
, id asc) a
order by transdate desc
, id desc) b
order by transdate asc
, id asc

if the range is to be specified dynamically, then you can make it into dynamic
sql...
declare @sql varchar (8000)
, @a int
, @b int

select @a = 200
, @b = 250

select @sql = 'select *'


+ ' from (select top ' + convert (varchar, @a) + ' *'
+ ' from (select top ' + convert (varchar, @b) + ' *'
+ ' from data'
+ ' order by transdate asc'
+ ', id asc) a'
+ ' order by transdate desc'
+ ', id desc) b'
+ ' order by transdate asc'
+ ', id asc'
exec (@sql)

better? worse? i don't know if this approach is more or less efficient. but i
think it meets the specification and it's arguably a simpler approach.
6/8/2004 10:53 am | lee dise
# re: efficient paging of recordsets with t-sql
the last solution (reversing the results) doesn't work on the last page. it will
always return the number of rows in a page.

7/22/2004 2:31 pm | kevin


# paging by t-sql
9/2/2004 1:27 am | a runner on microsoft.net
# efficient paging of recordsets with t-sql
efficient paging of recordsets with t-sql
9/22/2004 9:06 am | mahalakshmi natarajan
# re: efficient paging of recordsets with t-sql
there's something i don't like in this 'rowcount' setting. if any error occurs
between the 'set rowcount x' statement and 'set rowcount 0' statement, many
queries will return wrong results.

the rowcount property stays modified for the 'link' between the client and the
server and if something wrong occurs -> sh.t happens.

this might also lead to security issues (which i already had to handle because of
that rowcount conflict).

so, when using that property, be really sure your request cannot fail.
9/22/2004 1:03 pm | antoine
# re: efficient paging of recordsets with t-sql
i want to ask how can i get the record of one page in a eperate array. and so on..

actually i m using a function which returns records in an array.


9/13/2005 12:58 am | aatif chaudhry
# re: efficient paging of recordsets with t-sql
any way to get a record count out of this?
9/14/2005 8:25 pm | dave
# re: efficient paging of recordsets with t-sql
perfect! other solutions i have found only work if you wanted to select groups of
records using a unique identifier only.
the query makes my asp.net custom paging really efficient, it brings back the bare
minumum of data required, whilst not not requiring cursors,temp tables, or even
sql2005.
10/26/2005 7:02 pm | jason epstein
# re: efficient paging of recordsets with t-sql
great! thank you . i was using a paging solution using temp tables and it reduced
the time with at least 25% (recorded with a select on a table with 230.000+
records with joins and left joins with 5-6 other tables). can't wait to see what
happens with a table with over 2 million records and 5-6 joined tables.

best regards,
lucian chiriac
12/28/2005 11:59 am | lucian chiriac
# re: efficient paging of recordsets with t-sql
i enjoyed! very good solution!!!
2/22/2006 11:37 am | nataly
# re: efficient paging of recordsets with t-sql
nice job! thanks!
4/14/2006 7:09 am | jenny
# re: efficient paging of recordsets with t-sql
hi "am lee dise"
what u do is nice
but when ig i have 5 million or more rows!
then what will i do?
bring 5 million rows and then take 50 of them that slow :(
5/11/2006 7:02 am | peleg
# re: efficient paging of recordsets with t-sql
please can anyone tell me how to use next and back buttons in the query results of
large data in sql 2005
5/29/2006 3:35 pm | uncle sam
# re: efficient paging of recordsets with t-sql
this simply won't work:
set rowcount @a

select @startingdate = transdate, @startingid = id

from data

order by transdate asc,id asc

does not matter what you assign to the rowcount, the select will always return the
transdate and id of the first row in the table.
8/28/2006 1:18 pm | annon
# re: efficient paging of recordsets with t-sql
annon -- did you try it?
8/28/2006 1:55 pm | jeff
# re: efficient paging of recordsets: sql server 2000
this is a good article. i've also read a few on the topic at the following
locations:

http://aspnet.4guysfromrolla.com/articles/031506-1.aspx
http://www.4guysfromrolla.com/webtech/042606-1.shtml

the only thing i'm trying to figure out is how to use this to both sort my result
set desc and implement search functionality..

p.processid >= @firstid and


p.isdeleted = 0 and
p.processid = isnull(@processid, p.processid) and
i.vendorid = isnull(@vendorid, i.vendorid) and
etc.

jb
5/21/2007 2:53 pm | john bonds
# re: efficient paging of recordsets: sql server 2000
jb -- keep your sorting and your filtering separate; you can use a derived table
to help you.

i.e.,

select x.*
from
( your filtered select here .... )x
where
... your paging condition here ....
order by
....

5/21/2007 2:58 pm | jeff


# re: efficient paging of recordsets: sql server 2000
thanks, i tryed this aproach, but the grid view custom paging also requires the
number of rows and i couldn't find a way to get that because of rowcount and
making the count before was even more expensive than the temp table.
by the way, i think you avoid using the temp table 'cause the table you use has a
consecutive id, my table does'nt have that consecutive id, so i think it's
impossible to avoid the temp table unless we had something like oracle's
rownumber.
any way, we all should be in sql svr 2005 by now. i f we are into this yet, is
becaue our it departments