LATTICE POINT PROBLEMS AND DISTRIBUTION OF VALUES OF QUADRATIC FORMS

V. Bentkus1 F. Gotze1

University of Bielefeld July 1997

Abstract. For d-dimensional irrational ellipsoids E with d 9 we show that the number of lattice points in rE is approximated by the volume of rE , as r tends to in nity, up to an error of order o(rd?2 ). The estimate re nes an earlier authors' bound of order O(rd?2 ) which holds for arbitrary ellipsoids, and is optimal for rational ellipsoids. As an application we prove a conjecture of Davenport and Lewis that the gaps between successive values, say s < n(s), s; n(s) 2 Q Zd], of a positive de nite irrational quadratic form Q x]; x 2 Rd , are shrinking, i.e., that n(s) ? s ! 0 as s ! 1, for d 9. For comparison note that sups (n(s) ? s) < 1 and inf s (n(s) ? s) > 0, for rational Q x] and d 5. As a corollary we derive Oppenheim's conjecture for inde nite irrational quadratic forms, i.e., the set Q Zd] is dense in R, for d 9, which was proved for d 3 by G. Margulis in 1986 using other methods. Finally, we provide explicit bounds for errors in terms of certain characteristics of trigonometric sums.

Let R d , 1 d < 1, denote a real d-dimensional Euclidean space with scalar product ; and norm

1. Introduction and results

jxj2 = x; x = x2 1+
P

+ x2 d;

for x = (x1; : : :; xd ) 2 R d :

We shall use as well the norms jxj1 = d j =1 jxj j and jxj1 = max jxj j : 1 j d . d Let Z be the standard lattice of points with integer coordinates in R d . For a (measurable) set B R d , let vol B denote the Lebesgue measure of B , and let volZ B denote the lattice volume of B , that is the number of points in B \ Zd.
tive and inde nite quadratic forms, distribution of values of quadratic forms, Oppenheim conjecture, Davenport{Lewis conjecture. Typeset by AMS-TEX 1
1 Research supported by the SFB 343 in Bielefeld. 1991 Mathematics Subject Classi cation. 11P21. Key words and phrases. lattice points, ellipsoids, rational and irrational quadratic forms, posi-

V. BENTKUS

F. GOTZE

Consider a quadratic form, where Q : R d ! R d denotes a symmetric linear operator with non-zero eigenvalues, say q1 ; : : :; qd . Write q0 = 1min jq j; q = 1max jq j: (1.1) j d j j d j

Q x] def = Q x; x ;

for x 2 R d ;

We assume that the form in non-degenerate, that is, that q0 > 0. Thus, without loss of generality we can and shall assume throughout that q0 = 1, and hence q 1. De ne the sets

If the operator Q is positive de nite (henceforth called brie y positive), that is, Q x] > 0, for x 6= 0, then Es is an ellipsoid. Recall that a quadratic form Q x] with a non-zero matrix Q = (qij ), 1 i; j d, is rational if there exists an M 2 R , M 6= 0, such that the matrix M Q has integer entries only; otherwise it is called irrational. We identify the matrix of Q x] with the operator Q . Our main result|Theorems 2.1 and 2.2 yields the following Theorems 1.1, 1.3 and 1.5 and Corollaries 1.2, 1.4, 1.6 and 1.7, proofs of which we provide in Section 2. Theorem 1.1. Assume that Q is positive and d 9. Then (1.2) sup (s; Q ; a) def = sup volZ (Es + a) ? vol Es = o(s?1 );
as s ! 1 if and only if Q is irrational. The estimate of Theorem 1.1 re nes an explicit bound of order O(s?1) obtained by the authors (henceforth called BG) (1994) for arbitrary ellipsoids. That result has been proved using probabilistic notions and a version of the basic inequality (see (3.12) below) for trigonometric sums. Some methods of that proof will be used again in this paper. An alternative proof using more extensively the method of large sieves appeared as BG (1997). In the case of rational ellipsoids the bound O(s?1 ) is optimal. For arbitrary ellipsoids Landau (1915) obtained the estimate O(s?1+1=(1+d) ), d 1. This result has been extended by Hlawka (1950) to convex bodies with smooth boundary and strictly positive Gaussian curvature. Hlawka's estimate has been improved by Kratzel and Nowak (1991, 1992) to O(s?1+ ), where = 5=(6 d + 2), for d 8, and = 12=(14 d + 8), for 3 d 7. For special ellipsoids a number of particular results is available. For example, the error bound O(s?1 ) holds for d 5 and rational Q (see Wal sz 1924, d 9, and Landau 1924, d 5). Jarnik (1928) proved the same bound for diagonal Q with arbitrary (non zero) real entries. For a discussion see the monograph Wal sz (1957).
a2Rd a2Rd vol Es

Es = x 2 R d :

Q x]

s ;

for s 2 R:

LATTICE POINTS

Theorem 1.1 is applicable to irrational ellipsoids with arbitrary center for d 9. It extends the bound of order o(s?1) of Jarnik and Wal sz (1930) for diagonal irrational Q of dimension d 5. They showed that o(s?1) is optimal, that is, for any function such that (s) ! 1, as s ! 1, there exists an irrational diagonal form Q x] such that lim sup s (s) (s; Q ; 0) = 1:
s!1

See Theorem 1.3 for an explicit estimate of the remainder term in (1.2).
Gaps between values of positive quadratic forms. Let s; n(s) 2 Q Zd ], s < n(s), denote successive values of Q x]. Davenport and Lewis (1972) conjectured that the distance between successive values of the quadratic form Q x] converges to zero as s ! 1, provided that the dimension d 5 and Q is irrational. Theorem 1.1 combined with Theorem 1.1 of BG (1997) provides a complete solution of this problem for d 9. Introduce the maximal gap d( ; Q ; a) = sup n(s) ? s : s between values Q x ? a] in the interval ; 1). Corollary 1.2. Assume that d 9 and Q x] is positive de nite. If the quadratic form is irrational then sup d( ; Q ; a) ! 0, as ! 1. If Q is rational then sup d( ; Q ; a) < 1. 0 a2Rd ? If both Q and a are rational then inf n(s) ? s > 0. s

Answering a question of T. Esterman whether gaps must tend to zero for large dimensional positive forms, Davenport and Lewis (1972) proved the following: Assume that d d0 with some su ciently large d0 . Let " > 0. Suppose that y 2 Zd has a su ciently large norm jyj1. Then there exist x 2 Zd such that

Q y + x] ? Q x]

< ":

(1.3)

Of course (1.3) does not rule out the possibility of arbitrary large gaps between possible clusters of values Q x], x 2 Zd. The result of Davenport and Lewis was improved by Cook and Raghavan (1984). They obtained the estimate d0 995 and provided a lower bound for the number of solutions x 2 Zd of the inequality (1.3). See the reviews of Lewis (1973) and Margulis (1997). In order to formulate explicit bounds concerning gaps between values of positive quadratic forms and lattice point approximations for ellipsoids we need additional notation. Introduce the trigonometric sum
? 'a (t; s) = 2 ps ] + 1 ?3d P

exp it Q x1 + x2 + x3 ? a] ;

(1.4)

where the sum is taken over all x1; x2; x3 2 Zd such that jxj j1 s, for j = 1; 2; 3. Notice that the trigonometric sum (1.4) is normalized so that 'a (t; s) 'a (0; s) = 1.

V. BENTKUS

F. GOTZE

Simple selection arguments show that (1.5) yields that for irrational Q there exist sequences T (s) " 1, T (s) 1, and 0 (s) # 0 such that
s!1 a2Rd

Theorem 6.1 shows that for irrational Q and any xed 0 < 0 T < 1 the trigonometric sum 'a satis es lim sup sup 'a (t; s) = 0: (1.5) s!1
a2Rd
0

t T

lim sup

(s) t T (s)

sup

'a (t; s) = 0:

(1.6)

The relation (6.5) shows that for any non-degenerate Q . Hence, the irrationality of Q is equivalent to the following condition: there exist T (s) " 1 such that lim s; T (s) = 0; s!1
1 (s) = s1? + T ( s) +
? ?

s!1 a2Rd s?1=2 t

lim sup

sup

(s)

'a (t; s) = 0;

where

s; T = supd

Finally, given d 9, 0 < " < 1 ? 8=d and Q , introduce the quantity

a2R s?1=2 t T

sup

'a (t; s):

(1.7)

s; T (s)

1?8=d?" " T (s);

def 1 h d ? 1 i = 2 2 ;

(1.8)

on which our estimates will depend. Without loss of generality we can assume that T (s) in (1.7) and (1.8) are chosen so that for irrational Q . Indeed, if (1.9) does not hold, we can replace T (s) in (1.7) by min T (s);
n ? o s; T (s) ?(1?8=d?")=(2") :

s!1

lim (s) = 0;

(1.9)

We shall write A d B if there exist a constant cd depending on d only and such that A cd B. Theorem 1.3. Assume that the operator Q is positive, d 9 and 0 < " < 1 ? 8=d. Then we have volZ(Es + a) ? vol Es
d;" (s + 1)d=2 q d (s) s?1 :

(1.10)

Theorem 1.1 is an immediate corollary of Theorem 1.3.

LATTICE POINTS

If we choose T (s) = 1 in (1.8) and use

s; T (s)

1, then (1.10) yields

volZ(Es + a) ? vol Es

d (s + 1)d=2 q d s?1 :

This slightly improves the bound (s + 1)d=2 qd+2 s?1 given in BG (1997). An inspection of proofs shows that Theorem 1.3 holds for any < d=4. Moreover, the main result|Theorems 2.1 and 2.2|can be proved for any p < d=2. The assumption p 2 N is made for technical convenience only. Hence, assuming that T (s) ? and s; T (s) can be chosen so that the two terms in (1.8) depending on them are su ciently small, the right hand side of (1.10) is of order O(sd=4+ ), > 0. For a class of diagonal forms, Jarnik (1928a) proved bounds of this type and has shown that in general this order is optimal. Write 0 (s) = sup ( ):

Corollary 1.4. Assume that the operator Q is positive and d 9. Then

volZ (E + + a) n (E + a) = 1 + R; vol(E + n E )
?

for

s 1; > 0;

(1.11)

where R satis es jRj satis es

d(s; Q ; a) d;" q3d=2 0(s); for s 1: (1.12) The relation (1.11) gives an estimate of the number of values of a positive quadratic form in an interval (s; s + ], counting these values according to their multiplicities. More precisely, a value, say = Q x ? a], is counted volZ z : = Q z ? a] times. In the case of irrational Q the approximation (1.11) may be applied for intervals of shrinking size = (s) ! 0 as s ! 1. The approximation error in (1.11) still satis es R ! 0 for shrinking intervals such that = 0(s) ! 1 as s ! 1. The estimate (1.12) provides an upper bound for the maximal gap d(s; Q ; a) between values to the right of a value s 1, for positive Q . In particular, we get d(s; Q ; a) ! 0, as s ! 1, for irrational Q uniformly with respect to a.
The Oppenheim conjecture. Write

d;" q 3d=2 0 (s)= . In particular, the maximal gap d(s;

Q ; a)

Oppenheim (1929, 1931) conjectured that m(Q ) = 0, for d 5 and irrational inde nite Q . This conjecture has been extensively studied, see the review of Margulis (1997). A stronger version was nally proved by Margulis (1989): m(Q ) = 0, for d 3 and irrational inde nite Q . In 1952 A. Oppenheim proved that such a result is equivalent to the following: for irrational Q and d 3, the set Q Zd ] is dense in R d . In particular, d( ; Q ; 0) 0, for all , which is impossible for positive forms.

m(Q ) = inf

Q x]

: x 6= 0; x 2 Zd :
o

V. BENTKUS

F. GOTZE

The quantitative version of Oppenheim's conjecture was developed by Dani and Margulis (1984) and Eskin, Margulis and Mozes (1997). Let M : R d ! 0; 1) be any continuous function such that M (tx) = jtj M (x), for all t 2 R and x 2 R d , and such that M (x) = 0 if and only if x = 0. The function M is the Minkowski functional of the set = x 2 R d : M (x) 1 : (1.13) In particular, the set is a star-shaped closed bounded set with the non-empty interior containing zero. For an interval I = ( ; ] de ne the set W = x 2 R d : Q x ? a] 2 I . Assuming that d 5 and that the quadratic form Q x] is irrational and inde nite, Eskin, Margulis and Mozes (1997) showed that ? volZ? W \ (R ) as R ! 1: (1.14) vol W \ (R ) = 1 + o(1); Furthermore, with some = (Q ; ) 6= 0. Eskin, Margulis and Mozes (1997) provided as well re nements and extensions of (1.14) to lower dimensions. Introduce the box B (r) = x 2 R d : jxj1 r : (1.15) Let c0 = c(d; ") denote a positive constant. Consider the set vol W \ (R ) = ( ? ) Rd?2 + o(Rd?2 );
?

as R ! 1;

maximal gap between the successive values as

\ ?c0 r2; c0 r2] of values of Q x ? a] lying in the interval ?c0 r2 ; c0 r2], for x 2 B (r=c0). De ne the
d(r) def = max min v ? u : v > u; v 2 V : u2V
(1.16)

V def =

Q x ? a] : x 2 B(r=c0)

Theorem 1.5. Let Q x] be an inde nite quadratic form, d 9 and " > 0. Assume that the constant c0 = c(d; ") is su ciently small and that jaj c0 q?1=2 r. Then the
maximal gap satis es
1 3d=2 d(r) d;" q3d=2 (r2); for r2 c? 0 q ; with de ned by (1:8). In Section 2 we shall provide as well an explicit bound (see Theorem 2.6) for the remainder term in the quantative version (1.14) of the Oppenheim conjecture, for d 9. This bound is more complicated than the bound of Theorem 1.5 since it depends on the modulus of continuity of the Minkowski functional of the set . In this section we shall mention the following rough Corollaries 1.6 and 1.7 of Theorem 2.6 only.

LATTICE POINTS

Corollary 1.6. Assume that the form Q x] is inde nite and d 9. Then, for any > 0,
(1 ? ) vol W \ (C r )
provided that
?

there exist (su ciently large) constants C = C ( ; q; ; d) and C0 = C0 ( ; q; ; d) such that

volZ W \ (C r )

(1 + ) vol W \ (C r ) ; (1.17)

Corollary 1.6 is applicable to rational and irrational Q . For irrational Q the approximations can be improved. Corollary 1.7. Assume that the quadratic form Q x] of dimension d 9 is irrational and inde nite. Let R = rT 1=4. Then there exist T = T (r2) ! 1 such that
volZ? W \ (R ) vol W \ (R )
?

r C0 ;

C0 ;

jaj r;

j j + j j r2 :

(1.18)

?1

d;m;q g (r) + h(r)=(

? ) ! 0;

as r ! 1;

(1.19)

with some functions

g(r) = g(r; q; ; d) and h(r) = h(r; Q ; ; d)

such that g(r); h(r) ! 0. The convergence in (1:19) holds uniformly in the region jaj r and j j + j j r2. We have (s) ! 0, as s ! 1 (see (1.9)), for irrational Q . Thus, Theorem 1.5 gives an explicit upper bound for the maximal gap in Oppenheim's conjecture. The bound of Theorem 1.5 is constructive in the sense that in simple cases one might hope to estimate the quantity (s) explicitly using Diophantine approximation results. In general the estimation of remains an p open question. Corollary 1.7 is applicable for h(r). The bound of Theorem 2.6 is much more shrinking intervals, e.g., for ? precise than those of Corollaries 1.6 and 1.7. Nevertheless, in order to derive from Theorem 2.6 simple, sharp and precise bounds one needs explicit bounds for T , (s; T ) and the modulus of continuity of the functional M . Remark 6.2 shows that the results are uniform over compact sets of irrational matrices Q such that the spectrum of Q is uniformly bounded and is uniformly separated from zero. The basic steps of the proof consist of: (1) introduction of a general approximation problem for the distribution functions of lattice point measures by distribution functions of measures which are absolutely continuous with respect to the Lebesgue measure; both the elliptic as well as hyperbolic cases are obtained as specializations of this general scheme; (2) an application to the distribution functions of Fourier-Stieltjes transforms, reducing the problem to expansions and integration of Fourier type transforms of

V. BENTKUS

F. GOTZE

the measures (in particular, of certain trigonometric sums) with respect to a one dimensional frequency, say t; (3) integration in t using a basic inequality (BG 1994, 1997; see (3.12) in this paper), which leads to bounds depending on maximal values (see (1.7)) of the trigonometric sum; (4) showing that tends to zero if and only if the quadratic form is irrational. Bounds for rates of convergence in the multivariate Central Limit Theorem (CLT) for conic sections (respectively, for bivariate degenerate U -statistics) seem to correspond to bounds in the lattice point problems. The p \stochastic" diameter (standard deviation) of a sum of N random vectors is of order N , which corresponds to the size of the box of lattice points. In the elliptic case this fact was mentioned by Esseen (1945), who proved the rate O(N ?1+1=(1+d) ) for balls around the origin and random vectors with identity covariance, a result similar to the result of Landau (1915). For sums taking values in a lattice and special ellipsoids the relation of these error bounds for the lattice point problem and the CLT has been made explicit in Yarnold (1972). Esseen's result was extended to convex bodies by Matthes (1970), a result similar to that of Hlawka (1950). The bound O(N ?1) in the CLT, for d 5, of BG (1996) for ellipsoids with diagonal Q and random vectors with independent components (and with arbitrary distribution) is comparable to the results of Jarnik (1928). The bound O(N ?1 ), for d 9, for arbitrary ellipsoids and random vectors|an analogue of the results BG (1997)|is obtained in BG (1997a). This result is extended to the case of U -statistics in BG (1997b). Proofs of these probabilistic results are considerably more involved since one has to deal with a more general class of distributions compared to the class of uniform bounded lattice distributions in number theory. A probabilistic counterpart of the results of the present paper remains to be done. The paper is organized as follows. In Section 2 we formulate the main result| Theorems 2.1 and 2.2, derive its corollaries and prove the results stated in the introduction. Section 3 is devoted to the proof of Theorems 2.1 and 2.2, using auxiliary results of Sections 4{7. In Section 4 we prove an asymptotic expansion for the Fourier{Stieltjes transforms of the distribution functions and describe some properties of the terms of the expansion. Section 5 contains an integration procedure, which allows to integrate trigonometric sums satisfying the basic inequality (3.12). In Section 6 we obtain a criteria for Q x] to be irrational in terms of certain trigonometric sums. In Section 7 we investigate the terms of the asymptotic expansions in Theorems 2.1 and 2.2. In Section 8 we obtain auxiliary bounds for the volume of bodies related to inde nite quadratic forms. We shall use the following notation. By c with or without indices we shall denote generic absolute constants. We shall write A B instead of A cB. If a constant depends on a parameter, say d, then we write cd or c(d) and use A d B instead of A cd B. By B ] we denote the integer part of a real number B . We shall write r = r] + 1=2, for r 0. Thus r r, and for r 1 the reverse

LATTICE POINTS

inequality holds, r r. The set of natural numbers is denoted as N = f1; 2; : : : g, the set of integer numbers as Z = f0; 1; 2; : : : g, and N 0 = f0g N . P We write B (r) = x 2 R d : jxj1 r and jxj1 = 1max j x j , j x j = jxj j. j 1 j d 1 j d The region R of integration ed only in cases when it di ers from the whole R R is speci R space. Hence, R = and Rd = . We use the notation The Fourier{Stieltjes transforms of functions, say F : R ! R , of bounded variation are denoted as R b (t) = e ts dF (x): F Throughout I A denotes the indicator function of event A, that is, I A = 1 if A occurs, and I A = 0 otherwise. For s > 0, de ne the function e t = exp it ;

i = ?1:

M(t; s) = jtj s

?1 I jtj

s?1=2 + jtj I jtj > s?1=2 :

(1.20)

For a multi-index = ( 1; : : :; d ), we write ! = 1 ! : : : d !. Partial derivatives of functions f : R d ! C we denote by

@ 1 : : : @ d f (x): @ f (x) = @x f (x) = (@x (@xd ) d 1) 1

Sometimes we shall use notation related to Frechet derivatives: for we write

= ( 1; : : :; n), (1.21)

f (j j1) (x)h1 1 : : : hnn = @t11 : : : @tnn f (x + t1 h1 +

+ tn hn ) t = =t =0: 1 n

tion to the close relation between the quantative Oppenheim conjecture and the lattice point remainder problem and helpful discussions. Furthermore, we would like to thank A.Yu. Zaitsev for a careful reading of the manuscript and useful comments.
2. The Main result; Proofs of Theorems of the Introduction

Acknowledgment. We would like to thank G. Margulis for drawing our atten-

For the formulation of the main result|Theorem 2.1 we need some simple notions related to measures on R d . We shall consider signed measures, that is, -additive set functions : Bd ! R , where Bd denotes the -algebra of Borel subsets of R d . If a signed measure is non-negative and normalized, that is, (R d ) = 1 and (C ) 0, for C 2 Bd ,

10

V. BENTKUS

F. GOTZE
R

then we shall call measure. We shall write f (x) (dx) for the (Lebesgue) integral d over R d of a measurable function R f : R ! C with respect toda signed measure , and denote as usual by (C ) = (C ? x) (dx), for C 2 B , the convolution of the signed measures and . Equivalently, is de ned as the signed measure such that
R

f (x)

(dx) =

RR

f (x + y) (dx) (dy);

(2.1)

for any integrable function f . Let px 2 R , x 2 Zd, be a system of weights. Using signed measures, weighted trigonometric sums, say,
P

x2Z

e t Q x] px = e t Q x] d
R

(dx);

efvg = expfivg;

can be represented as an integral with respect to the signed measure concentrated on ? d the lattice Z such that fxg = px , for x 2 Zd. The uniform lattice measure ( ; r) concentrated on the lattice points in the cube B (r) = x 2 R d : jxj1 r is de ned by
\ B(r) (C ; r) = volZvolC B Z (r )
?

for C 2 Bd :

(2.2)

In other words, the measure ( ; r) assigns equal weights (fxg; r) = (2 r)?d to lattice points in the cube B (r), where r = r] + 1=2. Notice as well that ( ; r) = ( ; r). We de ne the uniform measure ( ; r) in B (r) by
C \ B (r) (C ; r) = vol vol B (r)
?

;
k ( ; r);

for C 2 Bd :

(2.3) (2.4)

For a number R > 0 write = ( ; R) and = ( ; R), and introduce the measures =
k ( ; r);

The distribution function, say G, of a quadratic form signed measure, say , on R d is de ned as

Q x ? a] with respect to a
s (dx);

k 2 N:

where I A denotes the indicator function of event A. The function G : R ! R is right continuous and satis es G(?1) = 0, G(1) = (R d ). If is a measure (i.e., nonnegative and normalized) then we have in addition: G : R ! 0; 1] is non-decreasing and G(1) = 1. We shall obtain an asymptotic expansion of the distribution function, say F , of Q x ? a] with respect to the measure de ned by (2.4). The rst term of this expansion will be the distribution function, say F0 , of Q x ? a] with respect to the

G(s) =

x 2 Rd :

Q x ? a]

s = I

Q x ? a]

(2.5)

LATTICE POINTS

11

measure de ned by (2.4). Other terms of this expansion will be distribution functions Fj , j 2 2 N , of certain signed measures related to the measure (or, in other words, to certain Lebesgue type volumes). A description of Fj will be given after Theorem 2.2. Introduce the function (cf. (1.4))

'a (t; r2) =

and, for a number T
?

e t Q x ? a]

3 (dx; r) ;

(2.6) (2.7)

1, de ne (cf. (1.6) and (1.7))

r2 ; T def = sup d

a2R r?1 t T

sup 'a (t; r2):

Our main result is the following theorem. Theorem 2.1. Assume that

d 9; p 2 N ; 2 p < d=2; k 2 p + 2; 0 r R; T F (s) = F0 (s) +

P

1: (2.8)

Then the distribution function F allows the following asymptotic expansion

j 22 N; j<p

Fj (s) + R

with a remainder term R satisfying

jRj

d;k;" r2 T + r2p 1 + r

qd=2

Rp

jaj p qp+d=2 + 1?8=d?"? r2; T T " qd=2 ; r2

(2.9)

for any " > 0. Notice, that the estimate (2.9) is uniform in s. The measure (or its support B (R) \ Zd) represents the main box of size R from which lattice points are taken. The convolution of with k ( ; r) is a somewhat smoother lattice measure than . Note though that the weights assigned by to the lattice points near the boundary of the box B (R) become smaller when the points approach the boundary of the box B (R + kr). The weights assigned to lattice points in B (R ? kr) remain unchanged. Later on we shall choose the size r of the smoothing measure ( ; r) smaller in comparison with R, that is, we shall assume that R ckr with a su ciently large constant c. This smoothing near the boundary simpli es the derivation of approximations and helps to avoid extra logarithmic factors in the estimates of errors. The corresponding measure is the continuous counterpart of with the dominating counting measure on Zd replaced by the Lebesgue measure on R d . Theorem 2.1 allows a generalization. The measure can be replaced by an arbitrary uniform lattice measure with support in a cube of size R, see Theorem 2.2 below. Theorem 2.1 is a partial case of Theorem 2.2. We shall prove Theorem 2.2 in Section 3. In order to formulate that result, we extend our notation.

12

V. BENTKUS

F. GOTZE

We de ne measures and as in (2.4), and denote distribution functions of Q x ? a] with respect to and as F and F0 respectively. Notice that the measure has the density d ( ; r) k ; d = d (2.12) D(x) def = dx dx dx where f g denotes the convolution of functions f and g, R f g(x) = f (x ? y) g(y) dy: Using Fourier transform, it is easy to verify that the density D admits continuous bounded partial derivatives j@ D(x)j d;k r ?d?j j1 , for j j1 k ? 2 (see Lemma 7.1). Theorem 2.2. Theorem 2:1 holds with and de ned by (2:10) and (2:11) respectively. Let us now de ne the functions Fj , for j 2 2 N .PLet = ( 1 ; : : :; m) denote a multi-index with entries 1 ; : : :; m 2 N . Write for the sum which extends : j j1 =j over all possible representations of the even number j as a sum j = 1 + + m of even 1 ; : : : ; m 2, for all possible m 1. For example, for j = 6, we have 6 = 6, 6 = 4 + 2, 6 = 2 + 4 and 6 = 2 + 2 + 2. Introduce the functions P Dj (x) = Dj (x) (2.13)
(k+1) (dul ); m Dj (x) = (?1) (2.14) D(j) (x)u11 : : : um ! l=1 where the density D is de ned by (2.12), and where we use the notation (1.21) for the Frechet derivatives. For example, we have 1 R 00 D (x)u2 (k+1)(du); D2 (x) = ? 2 and D4 (x) = D44 (x) + D422 (x) with 1 R (4) D44 (x) = ? 24 D (x)u4 (k+1)(du); RR (4) 2 (k+1) (du1 ) (k+1)(du2 ): D (x)u2 D422 (x) = 1 1 u2 4
R

We shall denote = ( ; 1=2). The measure has the density d = I jxj1 1=2 dx with respect to the Lebesgue measure in R d , so that (dx) = I jxj1 1=2 dx. The measure ( ; r) has the density (2 r)?d I jxj1 r . Notice as well that ( ; r) = ( ; r). Henceforth will denote a measure on R d such that (A) = 1, for some subset A B (R) \ Zd and ? fxg = 1= card A; for all x 2 A: (2.10) We do not impose restrictions on the structure of A except that A B (R) \ Zd and A 6= ?. Write = . It is easy to see that has the density d = 1 P I jx ? y j (2.11) 1 1=2 : dx card A
y2A

with

: j j1 =j

m R

m Q

LATTICE POINTS

13

Let j denote the signed measure on R d with density Dj . We de ne the function Fj , for j 2 2 N , as the distribution function of Q x ? a] with respect to the signed measure j , that is, R Fj (s) = j x 2 R d : Q x ? a] s = I Q x ? a] s Dj (x) dx: (2.15) The function Fj : R ! R is a function of bounded variation, Fj (?1) = Fj (1) = 0 and Rj 1 + jaj j q j +d=2 ; for j < d=2; (2.16) sup F (s) see Lemma 7.4.
s j j;d r2j

In the elliptic case the choice of is immaterial as long as the support of contains a su ciently massive box of lattice points. Thus we shall simply choose and as in (2.4). The same choice of is appropriate for the estimation of maximal gaps (cf. Theorem 1.5) in the hyperbolic case. The choice of a general as possible is appropriate for proving re nements of (1.14). We shall restrict ourselves to the following special generated by a star-shaped closed bounded set (see (1.13)) with the non-empty interior containing zero. De ne ? volZ C \ (R ) (2.17) (C ) = volZ (R )

and let in accordance with (2.10) the set A be given by Zd. The measure n A = (R ) \ o ? d is again de ned by (2.11). In order to guaranty that x 2 Z : fxg > 0 B (R), we shall assume throughout that B (1); this is not a restriction of generality. Hence, for the Minkowski functional of the set we have (2.18) jxj1 M (x) m jxj1 ; for all x 2 R d ; with some m 1. The inequalities (2.18) are equivalent to B (1=m) B (1). The modulus of continuity !( ) = sup M (x + y) ? M (x) (2.19) of M satis es lim !( ) = 0. For and in (2.4) we have = B (1), M (x) = jxj1 !0 and !( ) = . Let (@ ) def = @ + B ( ) be an -neighborhood of the boundary @ of . Then, introducing the weight p0 = 1= volZ(R ), writing for a while = kr=R and assuming that is de ned by (2.17), we have ? (C ) = p0 volZ C; for C R n (@ ) ; ? 0 (C ) p0 volZ C \ (R ) ; for C Rd ; (2.20) d (C ) = 0; for C R n (R ):
jyj1 ; jxj1 =1

14

V. BENTKUS

F. GOTZE

Notice that p0 = (2R)?d for de ned after (2.4). In order to prove (2.20), it su ces to consider the case when the set C is a one point set, and to use elementary properties of convolutions. Similarly, for measurable C R d , we have (C ) = p0 dx; 0 (C ) p0 (C ) = 0;
C
R R

for

C R

C \(R )

dx;

for C for C

Rd ; Rd n (R

n (@ ) ;
(2.21) ); (2.22)

where now = (kr + 1)=R. Using (2.19), it is easy to see that

R(@ )

x 2 R d : 1 ? !( ) M (x=R) 1 + !( ) ; for any > 0:

We conclude the section by deriving all results of the introduction as corollaries of Theorem 2.1; the re nement of (1.14)|Theorem 2.6|is implied by Theorem 2.2. The elliptic case. Let us start with the following corollary of Theorem 2.1. Corollary 2.3. Assume that the operator Q is positive and T 1. Then we have (2.23) for d 9, 2 p < d=2, p 2 N and any " > 0. The quantity (s; T ) is de ned in (1:7) (cf. (2:6) and (2:7)). Proof. The bound (2.23) obviously holds for s 1. Therefore proving (2.23) we shall assume that s > 1. We assume as well that jaj1 1. This assumption does not restrict generality since and in (2.23) we can replace a by a ? m with some m 2 Zd such that ja ? mj1 1. Choose the measure as in (2.4), and volZ(Es + a) = volZ(Es + a ? m); for any m 2 Zd; (2.24) volZ(Es + a) ? vol Es
? 1 1 d;" (s + 1)d=2 q p+d=2 sp=2 + sT + 1?8=d?" s; T

T" ; s

Obviously R k s + 1, for s > 1. Therefore the bound (2.9) of Theorem 2.1 implies (2.23) provided that we verify that our choices yield

k = 2 p + 2;

r = s;

R = 2 kr:

F (s) = (2 R)?d volZ(Es + a); F0 (s) = (2 R)?d vol Es; Fj (s) = 0; for j 6= 0: (2.25)
Let us prove the rst equality in (2.25). The ellipsoid E1 is contained in the unit ball, that is, E1 jxj 1 B (1), since the modulus of the minimal eigenvalue q0

LATTICE POINTS
?

is 1. Therefore p we have Es B s . Due to our choice of R, r and k ? 6,pwe have R ? kr 6 s. Thus, the inequality jaj1 1, the relations Es + a B 1 + s and F (s) = (Es + a) together with (2.20) imply the rst equality in (2.25). Let us prove the second equality in (2.25). Using (2.12) and (2.21) we see that ? the density D is equal to zero outside the set B R + kr + 1 , and D(x) (2 R)?d , for x 2 B (R ? kr ? 1). The ellipsoid Es + a is a subset of B (R ? kr ? 1), that yields the second equality in (2.25). For the proof of Fj (s) = 0 notice that the derivatives of D vanish in B (R ? kr ? 1), hence in the ellipsoid Es + a as well.
Proof of Theorem 1:3. This theorem is implied by Corollary 2.3. Indeed, the estimate (1.10) is obvious for s 1. For s > 1, the estimate (1.10) is implied by (2.23) estimating qp qd=2 , choosing p = 2 and T = T (s) as in the condition of Theorem 1.3. Proof of Corollary 1:4. We have to prove (1.11) and (1.12). The proof of (1.12) reduces to proving that volZ(E + + a) ? volZ(E + a) > 0; for c(d; ") q3d=2 0(s) with a su ciently large constant c(d; "). Using (1.11) it su ces to verify that jRj 1=2, which is obviously ful lled. Let us prove (1.11). Consider an interval ( ; + ] with s 1. We shall apply the bound of Theorem 1.3 which for s 1 yields volZ(Es + a) ? vol Es d;" qd s?1+d=2 0(s): (2.26) We get ? ? volZ (E + + a) n (E + a) ? vol E + n E d;" q d ( + )d=2?1 ( 0 ( + )+ 0 ( )): (2.27) The estimate (2.27) implies (1.11). Just note that ( ) ( s ), for s , divide both 0 0 ? sides of (2.27) by vol E + n E and use ? ? vol E + n E = ( + )d=2 ? d=2 vol E1;
+ R ( + )d=2 ? d=2 d u?1+d=2 du d ( + =2)?1+d=2 d ( + )?1+d=2 ;

15

vol E1 = vol x 2 R d :

+ =2

Q x]

vol x 2 R d : jxj 1=pq = cd q?d=2 :

Proof of Corollary 1:2. It su ces to use (1.9) and (1.12). Proof of Theorem 1:1. Assuming the irrationality of Q , the bound o(s?1) is implied by Theorem 1.3 and (1.9) since vol Es d q?d=2 sd=2. The bound o(s?1) in (1.2) implies d( ; Q ; 0) ! 0, as ! 1, which is impossible for rational Q .

16

V. BENTKUS

F. GOTZE

The hyperbolic case. For an interval I = ( ; ] R we write F (I ) = F ( ) ? F ( ); Fj (I ) = Fj ( ) ? Fj ( ): (2.28) Notice that F (I ) = F (s) in the case I = (?1; s]. Theorem 2.2 has the following obvious
corollary.

Corollary 2.4. Under the conditions of Theorem 2:2 we have P F (I ) = F0 (I ) + Fj (I ) + R

j 22 N; j<p

(2.29)

with remainder term R which satis es (2:9). Lemma 2.5. Let Q x] be an inde nite quadratic form of dimension d 9. Let p, k and " satisfy the conditions of Theorem 2:1. Assume that M (x) = jxj1 and = B (1). Let c1 = c1(d; ") denote a su ciently small positive constant. Finally, assume that

R r
Then
F (I ) F0 (I )

c1 ;
p;k;d;"

; 2 ?c1 R2; c1 R2];

q3d=2 Rd + qp+d R2 r2 rd ?

r R
1

c1 ; k

pq jaj c R: 1

(2.30)

2p 1?8=d?" ? r2 ; T T " : + R + 3 p r r2 (2.31) Proof. The result follows from Corollary 2.4 dividing (2.29) by F0 (I ) and using the estimates

?1

r2 T

Fj (I ) k;d ( ? ) qd?2 r?2?d Rd?2 ; j 22 N; j<p F0 (I ) k;d ( ? ) q?d=2 R?2 : M (x) = jxj1 ; R = R + k r; and m = = 1;
k;d r ?j ?d k;d (
R

(2.32) (2.33)

Let us prove (2.32). Using (2.15), (2.28), (2.30), (7.1), applying the estimate (8.9) of Lemma 8.2 with using the bound ja0j pq jaj, we obtain

Fj (I )

? ) q(d?2)=2 r ?j?d 1 + k;d ( ? ) q d?2 r?j ?d Rd?2 :

I jxj1 R + kr I

R+kr

pq jaj d?2

Q x ? a] 2 I

dx
(2.34)

(R + k r)d?2

In the proof of (2.34) the condition R r 1=c1 allowed us to replace R and r by R and r respectively. Summation in j , 2 j < p, of the inequalities (2.34) yields (2.32).

LATTICE POINTS

17

Let us prove (2.33). Using (2.12), (2.21), the lower bound (8.10) of Lemma 8.2 and conditions (2.30), we have

F0(I ) d R?d I jxj1 R ? kr I

Q x ? a] 2 I

dx d ( ? ) q?d=2 R?2 :

Proof of Theorem 1:5. We shall apply Lemma 2.5 choosing T = T (r2), R = rk=c1 , the maximal p and minimal k such that the conditions of Theorem 2.1 are ful lled. In this particular case we can rewrite (2.31) as
F (I ) F0 (I )

?1

d;" q 3d=2 r?2 +

qd+p

(r2):

(2.35)

In order to estimate the maximal gap, it su ces to show that any interval I = ( ; ] contains at least one value of the quadratic form (i.e., F (I ) > 0) provided that and satisfy ? d;" q d+p (r2). The inequality F (I ) > 0 holds if the right hand side of (2.35) is bounded from above by a su ciently small constant which can depend on d and ". Next we formulate and prove some re nements of (1.14). Recall that the set satis es B (1=m) B (1) (see (2.18)), and that ! denotes the modulus of continuity of the Minkowski functional M of the set (see (1.13) and (2.19)). Write

W = x 2 R d : Q x ? a] 2 I ; ? Z W \ (R ) ? = vol vol W \ (R ) ? 1 ;
and

I = ( ; ];

(2.36) (2.37)

"2 = (j j + j j)=R2; "0 = r=R; "1 = pq jaj=R; (2.38) p "3 = !("0 =c2); "4 = !("1 q=c2 ); "5 = !("2pq=c2 ); where c2 = c2 (d; m) denotes a positive constant. Theorem 2.6. Assume that the form Q x] is inde nite and d 9. Let r 1=c2, 0 < " < 1 ? 8=d, T 1 and "j c2 ; for j = 0; 1; 2; 3;
Then we have
1 d;";m q d?1 1 + r2 "d ("3 + "4 + "5 ) 0 3d=2 1 1 q 1?8=d?" ? r2 ; T + + + ? 2 d +2 ? 3+ d= 2 "0 T r "0

"j c2 q?1=2 ; for j = 4; 5:

(2.39)

T" "2 0

(2.40)

18

V. BENTKUS

F. GOTZE

provided that the constant c2 is su ciently small. Proof of Corollary 1:6. Let us apply Theorem 2.6 choosing c2 = c2 ( ; q; d; m) su ciently small depending on and q as well. Choose R = C r. Then we have "0 = C ?1 and the last two inequalities in (1.18) guaranty conditions (2.39). In particular, we have "j c2 , for j = 3; 4; 5. Hence, we can apply the bound (2.40). Choosing T = 1, " = (1 ? 8=d)=2 and estimating 1, we obtain (2.41) q;d;m c2 (1 + r?2 C d ) + (C 2 + r?1 C d+2 )=( ? ); for r 1. The estimates (1.17) follow if the right hand side of (2.41) is bounded from above by . But this holds in view of the rst two inequalities in (1.18) and our choices of C , C0 and c2. ? Proof of Corollary 1:7. During the proof we shall write T = T (r2) and = r2; T . Choose " = (1 ? 8=d)=2. Since R = rT 1=4 with T ! 1, we have "0 = T ?1=4 and the conditions (2.39) are ful lled for su ciently large r. Moreover, "j ! 0, for j = 3; 4; 5. Hence, the bound (2.40) yields (1.19) with g(r) = (1 + r?2 T d=4 ) ("3 + "4 + "5 ); h(r) = T ?1=2 + r?1 T (d+2)=4 + (1?8=d)=2 T 1?4=d : If g(r) 9 0 or h(r) 9 0, we can choose (if necessary) T = T (r2 ) ! 1 growing somewhat slower such that g(r) ! 0 or h(r) ! 0 (cf. a similar rede nition of T (s) in the case of (1.9)). For the proof of Theorem 2.6 we shall need the following Lemma 2.7, which allows to estimate using Theorem 2.2. Write ? ? = volZ W \ (R ) ? vol W \ (R ) : (2.42) k ( ; r) and R = k ( ; r), where = By R = , we shall denote the measures and emphasizing the dependence on the parameter R which enters in the de nition (2.17) of . Lemma 2.7. Let 0 = (kr + 1)=R. Assume that !(2 0) < 1. Write ? ? s1 = R= 1 + !(2 0) ; s2 = R= 1 ? !(2 0) ; p2 = 1= volZ(s2 ): (2.43) Then we have 1 max j j + v; j j p? (2.44) 2 s= s ;s s where and
1 2

v = vol W \ s1 1 ? !(2 0)
s=
R

M (x) s2 1 + !(2 0)
R

;
(2.45) (2.46)

If !(2 0) < 1=4 then v vol W \ 1 ? 3 !(2 0) M (x=R) 1 + 3 !(2 0) : Proof. We omit the elementary proof of (2.46).

I x2W

s (dx) ?

I x2W

s (dx):

LATTICE POINTS

19

Using s1 = R= 1 + !(2 0) it is easy to see that

R=2 and the fact that ! is a non-decreasing function,

1=

s1 1 + !( 1)

R s2 1 ? !( 2) ; with

Let us prove (2.44). Assume rst that 0. Applying (2.20){(2.22) to s and s and using (2.47), we have s2 (C ) = p2 volZ C; for C R ; ? ; for any C: (2.48) s2 (C ) p2 vol C \ s2 (1 + ! (2 0 )) Using (2.48), we have ? 1 ? 1 R I x 2 W s (dx); volZ W \ (R ) = p? p? 2 2 s2 W \ (R ) 2 R ? ? 1 p2 I x 2 W s2 (dx) vol W \ (R ) + v; (2.49) proving the lemma in the case 0. Assume now that < 0. Write p1 = 1= volZ(s1 ). Relations (2.20){(2.22) together with (2.47) yield ? ? vol W \ (R ) vol W \ s1(1 ? !(2 0)) + v; ? R 1 I x 2 W s (dx); vol W \ (s1(1 ? !(2 0)) ) p? 1 1 ? R ? 1 volZ W \ (R ) p1 I x 2 W s1 (dx); and using p2 p1 we obtain (2.44). Proof of Theorem 2:6. The result follows from the bound
d+p R2 + R2p+2 + 1?8=d?" ? r2 ; T T " R2 (2.50) + q? r2 T r3p r2 choosing the maximal p < d=2, k = 2 p + 2 and using the notation (2.38). Recall, that we assume that the constant c2 in (2.38) is su ciently small. In particular, this assumption guarantees that !(2 0) is as small as it is required in the auxiliary lemmas used below. ? Let us prove (2.50). Dividing the bound (2.44) of Lemma 2.7 by v1 def = vol W \(R ) , we obtain v : max j j + j j p21v1 s= s v1 s1 ;s2

kr + 1 ; s1

2=

kr + 1 : s2

(2.47)

1 Rd d;";m;p;k q d?1 1 + r2 rd

!(2 0) + "4 + "5 )

Hence, in order to prove (2.50) it su ces to show that 1 d p? 2 = volZ(s2 ) d R ; v1 d;m ( ? ) q?d=2 Rd?2 ; ? v d;m ( ? ) !(2 0) + "4 + "5 q(d?2)=2 Rd?2 ;

(2.51) (2.52) (2.53)

20

V. BENTKUS

F. GOTZE s;1j + j s;2j

and, for s = s1 ; s2,

j sj j

(2.54) (2.55) (2.56)

with s;1 and s;2 de ned below by (2.57) such that

j j

s;1j

s;2j

? R2p qd=2 qd=2 d;k;" r2 T + r3p q p+d=2 + 1?8=d?" r2 ; T T " r2 : ? d;m ( ? ) ! (2 0 ) + "4 + "5 q (d?2)=2 r?4 (R=r)d?2:

To prove (2.51) it su ces to notice that s2 2 R since B (1) and we assume that !(2 0) is su ciently small. The estimate (2.52) follows from (1.13) and (8.10) of Lemma 8.2 choosing = 1 p and using the estimate ja0j=R q jaj=R = "1 1=(2m) which is guaranteed by the assumption that the constant c2 is su ciently small. The bound (2.53) follows from (2.46) and Lemma 8.3 with = 3 !(2 0) 1=4 which is ful lled since c2 is small. Applying Lemma 3.8 we use the estimate "1;0 "1 . Let us prove (2.54). Let F (I ; R) and Fj (I ; R) denote the functions F (I ) and Fj (I ) de ned by (2.28) with the underlying parameter R which enters into the de nitions of the measures = R and = R . Then (2.54) holds with
s;1 = F (I ; s) ? F0 (I ; s) ? s;2 ; s;2 =
P

j 22 N; j<p

Fj (I ; s):

(2.57)

For the estimation of s;1 we shall apply Theorem 2.2. Our choice of the constant c2 yields !(2 0) 1=2. Therefore s = s1; s2 is asymptotically equivalent to R. The functions s ! F (I ; s) and s ! Fj (I ; s) are di erences of the corresponding distribution functions. Furthermore, jaj R. Hence, Theorem 2.2 yields (2.55). It remains to prove (2.56). Let us estimate Fj (I ; s). Using s R, r r and (7.4), we have Fj (I ; s) d;p v2 r ?j?d ; where o n v2 = vol W \ x 2 R d : M (x=s) ? 1 !(ck r=R) : By Lemma 8.3 and a suitable choice of c2 , the volume v2 allows the same upper bound as the volume v in (2.53) since s R. Hence, summing these bounds over j , we get (2.56).
3. Proof of Theorems 2.1 and 2.2

We shall use the following approximate (see, e.g., relation (8.4) in BG 1997) and precise (see, e.g., Chung 1974) formulas for the Fourier{Stieltjes inversion. For any T > 0

LATTICE POINTS

21

and any distribution function F of a (normalized non-negative) measure with the Fourier{Stieltjes transform
b (t) = e tx dF (x); F R

t 2 R;
(3.1)

we have

T R 1 b (t) dt + R e + 2i V: P: e ?xt F F (x) = 2 t ?T T 1 R T ?T

e such that with remainder term R e jRj b (t)j dt: jF

Here V: P: f (t) dt = lim !0

f (t) dt denotes the Principal Value of the integral. Furthermore, for any function F : R ! R of bounded variation such that F (?1) = 0 and 2 F (x) = F (x+) + F (x?), for all x 2 R , we have
jtj>
1 F (x) = 2 F (1) + 2i Mlim V: P: !1
R

jtj M

b (t) dt : ef?xtg F t

(3.2)

The formula is well-known for distribution functions. To functions of bounded variation it extends by linearity arguments. Theorem 2.1 is implied by Theorem 2.2 and we have to prove Theorem 2.2 only. The expansion (2.8) yields

jRj = F ?

j 22 N0 ; j<p

Fj :

(3.3)

Let us prove Theorem 2.2 assuming that r 1. Using (3.3), Lemma 7.4 to bound jFj j, the obvious estimates jF j 1 and jF0 j 1, we obtain

jRj

d 1+

Rj 1 + jaj j q j +d=2 2j r 1 j<p r

P

d r2p 1 + r

Rp

jaj p qp+d=2

(3.4)

since q 1, 1 R=r, 1=r 1 and j < p < d=2. The estimate (3.4) implies the theorem in the case r 1. Therefore in the remaining part of the proof we can and shall assume that r 1. Using the representation (3.3), representing F by the approximate Fourier{Stieltjes inversion (3.1) an Fj by the inversion formula (3.2), splitting the intervals of integration,

22 1

V. BENTKUS
R b (t) dt F

F. GOTZE
R b (t) dt ; F jtj

using the triangle inequality and the obvious estimate

T 1=r jtj T
1=r jtj T
P

we obtain with

jRj I1 + I2 + I3 +
I1 =
R

j 22 N0 ; j<p

Ij+4

(3.5)

1 I2 = T

jtj r 1

b (t) ? F R

j 22 N0 ; j<p

bj (t) dt ; F jtj

jtj r 1
R R

b (t) dt; F b (t) dt ; F jtj

I3 =

jtj r 1; jtj T jtj r 1

Ij+4 =
1 j +4

bj (t) dt ; F jtj

j 2 2 N 0 ; j < p:
0

The estimate (3.5) shows that in order to prove the theorem it su ces to prove that Rp 1 + jaj p q p+d=2 ; I ;I j 2 2 N ; j < p; (3.6)

I2 d q r2 T ; d=2 ? I3 d;" 1?8=d?" r2; T T " qr2 ; for any " > 0; Let us estimate I3 . Changing variables, we have R b (t) = e t Q x ? a] F (dx): Splitting (k?3) ( ; r); = 3 ( ; r) with = we obtain R 0; 0 def I3 I3 I3 = supd 'a (t; r2) jdt tj ;
a2R r jtj 1; jtj T

d=2

d;k r2p

(3.7) (3.8)

(3.9) (3.10) (3.11)

where 'a is given by

'a (t; r2) =

e t Q x ? a]

3 (dx; r) ;

(cf. (2.6)). The function 'a satis es the following inequalities (see Theorem 5.1 in BG 1997) 'a (t; r2) 'a (t + ; r2) d qd=2 Md=2 ( ; r2); (3.12) 'a (t; r2) d qd=2 Md=2(t; r2); t; 2 R (3.13)

LATTICE POINTS

23

involving the function M(t; r2) introduced in (1.20). The inequality (3.12) allows to apply Theorem 5.1 of the present paper. Choosing in this theorem = cd qd=2 ;

{ = d=2;
r2 ; T qd=2

s = r2 ;

= ?1;

using ln x " x", for x 1 and " > 0, we obtain

I0

d;"

1?8=d?"

d=2 (1 + ln T ) qr2 :

(3.14)

Estimating in (3.14) 1 + ln T " T ", for T 1, and q 1, and using (3.10), we derive the desired bound (3.8) for I3. Let us estimate I2 . Similarly to the estimation of I3 we obtain

I2

0 I2 T ;

0 def I2 = sup d

a2R r jtj 1

'a (t; r2) dt:

(3.15)

The bound (3.13) for 'a and the de nition (1.20) of the function M(t; r2) together with the inequality 'a 1 yields

'a (t; r2) d qd=2 min 1; (r2 jtj)?d=2 ;

Hence, we have
0 d q d=2 R min 1; (r2 jtj)?d=2 dt = qd=2 I2 r2
R
R

for r jtj 1:
d=2

(3.16)

min 1; jtj?d=2 dt d qr2 ; R

which together with (3.15) yields the bound (3.7) for I2 . Let us prove (3.6) for I1 . Applying the bound (4.10) to jRj de ned by (3.3), bounding by 'a the function in (4.10) and using (3.16), we obtain
p+d=2 p j p I0 I1 d;k q r2p R 1 + ja 1 r

with

0= I1

r jtj 1

jtj r2

(p)

? + jtj r2 p min 1; (jtj r2)?d=2 jdt tj :

(3.17)

By change of variables tr2 = we get

0 I1 1 R
0

( (p) + p) min 1; ?d=2 d

d;p 1;

(3.18)

since 0 < (p) p < d=2, see (4.7). The bounds (3.17) and (3.18) yield (3.6) for I1.

24

V. BENTKUS

F. GOTZE

Let us prove (3.6) for Ij+4 . Using (7.13) we have j jaj j qj+d=2 ; Ij+4 d rjR +d=2 1 + r and an application of the inequalities r 1; R=r 1; j < p < d=2; q 1 implies (3.6) for Ij+4, thus concluding the proof of Theorem 2.2.
b 4. An expansion for the Fourier{Stieltjes transform F

By change of variables we can write the Fourier{Stieltjes transforms of the distribution functions F and F0 as R R b (t) = e t Q x ? a] b0 (t) = e t Q x ? a] F (dx); F (dx): (4.1) Similarly, for the Fourier{Stieltjes transforms of the functions of bounded variation Fj (see (2.13){(2.15)) we have P bj (t) = bj (t); F F (4.2) where the sum
bj (t) = F P

: j j1 =j (?1)m R R !

is de ned as in (2.13), and

m dx e t Q x ? a] D(j) (x)u11 : : : um

: j j1=j

m Q l=1

(k+1) (dul ):

(4.3) (4.4) (4.5) (4.6)

In the proof of (4.6) we used that the function D has compact support and the derivatives @ D are continuous functions, for j j1 k ? 2. Write (s) = s=2; for even s; and (s) = (s + 1)=2; for odd s; (4.7) and R k=(p+1)] (dx; r) ; (t) = sup e t Q x] + a; x (4.8)
0 (t) =

Introducing the notation g(x) = exp h(x) ; h(x) = it Q x ? a]; integrating by parts and using D(x) dx = (dx), we have R R b (t) = g (x) (dx); b0 (t) = g (x) (dx) F F and m m R R (j ) 1 m (dx) Q (k+1) (dul ): bj (t) = (?1) g ( x ) u : : :u F m 1 !
l=1

a2Rd

a2Rd

sup

e t Q x] + a; x

k=(p+1)] (dx; r) :

(4.9)

LATTICE POINTS

25

Lemma 4.1. Let 2 p k ? 1 and q 1. Then, for R r 1, we have P b (t) ? F b0 (t)? bj (t) F F
j 22 N; j<p jaj p (t) Rp qp 1 + d;k 2 p r r
0 (t) Rj qj r2j
?

jtj r2

(p)

? + jtj r2 p

and, for j 2 2 N 0 , j < p,

bj (t) F

(4.10) (4.11)

d;k

Proof. Let g : R d ! C denote a su ciently smooth complex valued function. Assume that x; u1; : : :; up 2 R d . We shall use the following decomposition g(x) = g(x + u1) + g1 + + gp; (4.12) where P m ; for 1 j < p; gj = c( ) g(j)(x + um+1)u11 : : :um (4.13)
: j j1 =j
P

j j ? jtj r2 (j) + ? jtj r2 j : 1 + ja r

gp =
and

: j j1 =p

md ; cp ( ) (1 ? ) m g(p)(x + um)u11 : : :um

1 R 0

(4.14)

1) (?1) c( ) = 1(!? ; c ( ) = : (4.15) p : : : m! 1 ! : : : m?1 !( m ? 1)! P Here the sum is taken over all representations of j as a sum j = 1 + + m , : j j1 =j m j , of integers 1 ; : : :; m 2 N . For example, in the case j = 3 we have the following four representations: 3 = 3, 3 = 2 + 1, 3 = 1 + 2, 3 = 1 + 1 + 1. In particular, if p = 1 then 1 R g1 = ? (1 ? ) g0(x + u1)u1 d ;

and if p = 2 then

g1 = ?g0(x + u2)u1 ; g2 = (1 ? ) g00 (x + u2)u1 u2 d ? (1 ? )2 g00 (x + u1 )u2 1d :

In order to prove (4.12) it su ces to iteratively apply Taylor expansions. In the rst step we use the Taylor expansion of the function g:
0 0 1 pP ?1 1 ( ) 1 R 1 (x)u 1 ? g (1 ? )p g(p)(x + 1 ! ( p ? 1)! 1 0 1 =1

1 R

1 R

g(x) = g(x + u1 ) ?

u1 )up 1 d : (4.16)

In the second step we apply expansions of type (4.16) to the functions x ! g( 1) (x)u11 , for 1 1 < p, using u2 instead of u1 . After p such steps we arrive at (4.12).

26

V. BENTKUS

F. GOTZE

For the derivatives of the composite function g(x) (see (4.4)), we shall use the following formula, which follows from a general formula (see Theorem 2.5 in Averbuh and Smoljanov 1967). Let v1 ; : : :; vs 2 R d . Let A = A1 ; : : :; A denote a partition of the set v1 ; : : :; vs = A1 A into disjoint subsets Aj such that 1 card Aj 2, for all j . Notice that (s) s, where (s) is de ned by (4.7). For a subset A v1 ; : : :; vs , say A = vi1 ; : : :; vil , introduce the dedef rivative @A h(x) = h(l) (x)vi1 : : : vil . Then

g(s)(x)v1 : : : vs = g(x) @A1 h(x) : : :@A h(x);

P

(4.17)

where the sum extends over all partitions A with properties speci ed above. One A can easily prove (4.17) using h000 (x) 0 and induction in s. The following identities ( ; r) = ( ; r )
R

;
R

(k+1) ;

(4.18)

are obvious (see (2.2), (2.3) and (2.4) for the de nitions of measures appear in (4.18)). For example, for any integrable function u : R d ! R , we have

u(x) (dx; r) =

y2Zd\B(r)

(2 r)?d u(x + y) (dx) =

RR

u(x + y) (dx) (dy; r);

proving the rst identity in (4.18). Integrating both sides of the identity (4.12) with respect to the measures (dx);
(k+1) (du1 );

:::;

(k+1) (dup )

and using (4.5), (4.6), (4.13), (4.14), (4.18), we obtain

b (t) = F b0 (t) + f1 (t) + F b and F b0 de ned by (4.5), and with F

+ fp?1 (t) + fp (t);

m Q s=1
(k+1) (dus );

(4.19) (4.20)

fj (t) =

: j j1 =j

c( )

m (dx) g(j)(x)u11 : : : um

for 1 j < p, and

fp (t) = fp (t;

0 : j j1 =p R R ) def = g(p) (x +

cp ( ) (1 ? ) m fp (t; ) d ;
m (dx) um )u11 : : : um

1 R

(4.21)
m Q l=1
(k+1) (dul ):

LATTICE POINTS

27

The sums in (4.20) and (4.21) are the same as in (4.13) and (4.14) respectively. However, bj , notice that fj = 0, for odd j < p since the measure is symmetric. Moreover, fj = F for even j < p, since all terms in the sum (4.20) vanish unless all 1 ; : : :; m are even, due to the symmetry of . Hence, (4.19) yields P b (t) ? F b0 (t) ? bj (t) = fp (t) : F F (4.22) Now we can return to the proof of (4.10). The equality (4.22) shows that it su ces to verify that fp(t) is bounded from above by the right hand side of (4.10). Using (4.21), we have to verify that any of the suprema sup fp (t; ) is bounded by the right 0 1 hand side of (4.10), for all allowable 1 ; : : :; s. De ne v1 ; : : :; vp repeating u1 1 times, followed by u2 2 times, etc. For example, vj = u1 , for all 1 j 1 . Using (4.17) and (4.21) with m = p, we obtain m (k+1) R R Q sup I (dul ); (4.23) sup fp (t; ) p sup max A A = um ; (4.24) IA = g(x + z) @A1 h(x + z) : : : @A h(x + z) (dx) ; z def and (p) p. Fix a partition A = A1; : : :; A of type used in (4.17) and (4.23). Let denote the number of 1-point sets in this partition. Without loss of generality we can assume that A1; : : : ; A are 1-point sets, and that A +1; : : : ; A are 2-point sets. Then @Aj h(x + z) = 2 it x ? a + z; Q wj ; for 1 j ; (4.25) and (4.26) @Aj h(x + z) = 2 it wj ; Q wj ; for < j ; with some wj ; wj 2 u1 ; : : :; um . Notice that jwj j1 1 and jwj j1 1. Furthermore, using (4.25), (4.26) and (4.24), substituting x ? a + z; Q wj = x; Q wj ? a ? z; Q wj ; multiplying, applying the triangle inequality and re-enumerating wj if necessary, we obtain IA p jtj 0max I (4.27) where R Q x; Q wj B (dx) ; (4.28) I = g(x + z) and Using jQ wj q jwj jB j since jzj1 = jum j1
j =1 j 22 N; j<p

where

jzj1 1

l=1

B def =

d q , we have ? d;p q ? jz j ? + jaj ? 1 (see (4.24)) and jwj j1

j = +1

z ? a; Q wj

j = +1

w j ; Q wj :

(4.29) (4.30)

? p q ? 1 + jaj ? 1, jwj j1 1.

28

V. BENTKUS

F. GOTZE

Let us split the measure =

k ( ; r) as follows

= 0 (p+1) ; ( ; r) (k?(p+1) k=(p+1)]); 0= where u] denotes the integer part of u. Then we have
R R

k=(p+1)] ( ; r); (4.31)

U (x) (dx) = U (x0 + + xp+1 ) 0 (dx0) (dx1) : : : (dxp+1 ); for any integrable function U . Applying (4.32) to (4.28), writing x = x0 + and using pP +1 pP +1 Q xj1 ; Q w1 : : : xj ; Q w ; x; Q wj =
we obtain with
j =1 j1 =0 j =0

(4.32) + xp+1

I
R R

max p+1 p 0 j1 ;:::;j

I j;
pQ +1 s=1

(4.33) (dxs) 0 (dx0): (4.34)

Ij=

g(x + z) xj1 ; Q w1 : : : xj ; Q w B

Given the variables xj1 ; : : :; xj , p, we nd among x1 ; : : :; xp+1 at least one variable, say xl such that l 2 = j1 ; : : :; j . Without loss of generality we can assume that l = 1. Then (4.34) yields

Ij
with

Q x j ; w1
1

: : : Q xj ; w

jB j J 0 (dx0)

pQ +1 s=2

(dxs )

(4.35)

x = x0 + J = g(x + z) (dx1 ) ; Recall (see (4.4)) that g(x) = e t Q x ? a] . Therefore J

L2Rd

+ xp+1 : (4.36) (4.37)

pQ +1 s=2

sup

e t Q x] + L; x

k=(p+1)] (dx; r) = (t)

with de ned by (4.8). Hence, the bound (4.35) combined with (4.30) and (4.36) yields
? I j d;p (t) q ? 1 + jaj ? J j

with We have

Jj=

Q x j ; w1
1

: : : Q xj ; w

0 (dx0 )

(dxs ):

(4.38)

since we assume that R

Q x j ; w1
1

: : : Q xj ; w (4.39) kq R r, and since the variables xs in the integral (4.38) sat-

LATTICE POINTS

29

isfy jxsj1 k r + R

R. Combining (4.37){(4.39), we get

?

I j k (t) R q 1 + jaj ? : The estimate (4.40) combined with (4.33) and (4.27) yields IA
? ? (t) jtj r2 q r?2 0max R 1 + jaj ? :

(4.40) (4.41) , we obtain

Using the condition R r 1 and + 2( ? ) = 2 ? = p,

? r?2 0max R 1 + jaj ?

j ? r?2 0max R r ? 1 + ja r jaj : R p r?p r 1 + r

(4.42)

The inequalities p, q 1 and (p) p combined with (4.40){(4.42) and (4.21){ (4.23) yield (4.10). The proof of (4.11) repeats the proof of (4.10) starting from (4.22) since now we have bj j = jfj j (see (4.20)) instead of jfp j. In this proof we have to use (4.20) to estimate jF instead of (4.21) and to replace everywhere p by j . Furthermore, instead of (4.31) we have to use a similar splitting of of the form = 0 (p+1) with
0=

( ; r) (k?(p+1) k=(p+1)]) and

R

k=(p+1)] ( ; r):

(4.43)

In particular, the splitting (4.43) yields that the integral corresponding to J in (4.36) now satis es

L2R

supd

e t Q x] + L; x

k=(p+1)] (dx; r) = 0 (t);

which leads to the factor 0 in (4.11).

5. The integration procedure for large jtj

Recall that

M(t; s) = ? jtj s
?

?1 I jtj

s?1=2 + jtj I jtj > s?1=2 :

(5.1)

where s > 0 will be a positive large parameter. For a number T 1 and a family of functions '( ) = '( ; s) : R ! R introduce =
o n s; T def = sup '(t) : s?1=2 t T :

(5.2)

The following Theorem 5.1 sharpens Theorem 6.1 of BG (1997) in cases where < 1.

30

V. BENTKUS

F. GOTZE

Theorem 5.1. Let '(t), t 0 denote a continuous function such that '(0) = 1 and 0 ' 1. Assume that, for some { > 4 and 1,
'(t) '(t + ")
Let T

M{ ("; s);
if ? 1 <

for all

t 0 and " 0: (5.3)

1. Assume that the number de ned by (5:2) satis es

> 4{=({?4) s?{=4;

1 + ln 1
Then the integral

{ =({ ?4)

0;

> 4{=({?4) s?{=4 (1 + ln s){=({?4); if = ?1: J=

T R s?1=2

(5.4)

'(t) t dt

can be bounded as follows

J
and

;{
1?4={

1?4={

+1

s;

for ? 1 < for

0; = ?1:
for

(5.5)

;{

1 + ln

(1 + ln T ) s ;

If (5:4) is not ful lled then

(1 + ln s)(1 + ln T ) ; for

= ?1;

;{

+1 ;

> ?1: (5.6)

Proof. Evaluating the integral J using the method of Lebesgue integration by partitioning the range of ' in intervals 2?l?1 ; 2?l ], we have to estimate the Lebesgue measure of the corresponding sets Bl = ft : 2?l?1 '(t) 2?l g \ s?1=2; T ]. Using inequality (5.3), we shall show that two points in Bl are either very 'close' or far apart. This means that the set Bl consists of 'small' clusters of size Ol (s?1) separated by 'large' gaps of size Ol (1). These constraints on the structure of Bl su ce to bound the measure of Bl well enough in order to estimate the size of J for { > 4 as claimed in Theorem 5.1. For trigonometric sums ' this condition translates to to the assumption that the dimension d satis es d > 8. Throughout the proof we shall write instead of ;{ . To prove (5.6) it su ces to use '(t) and to notice that
dt s?1=2 t
T R

(1 + ln s)(1 + ln T );

T R s?1=2

t dt

+1 ;

for

> ?1:

LATTICE POINTS

31

Let us prove (5.5). The inequality (5.3) implies that (set t = 0, use '(0) = 1 and note that 1) '(t) M{ (t; s) and '(t) '(t + ") 2 M{ ("; s): (5.7) Starting the proof of (5.5) with (5.7) we may assume without loss of generality that = 1, that is, that '(t) M{ (t; s) and '(t) '(t + ") M{ ("; s): (5.8) Indeed, we may replace ' in (5.7) (resp. ) by '= (resp. by = ), and we may integrate over '= instead of '. Notice, that now '(0) 1 and the case '(0) < 1 is not excluded. Thus assuming (5.8) we have to prove that
T R

with F = T +1 , for ?1 < 0, and F?1 = (1+ln T ) 1+ln 1 . While proving (5.9) we may assume that 1 s. Otherwise (5.9) obviously holds since ' 1. Let l denote the smallest integer such that 2?l . For the integers l l , introduce the sets Bl = s?1=2; T ] \ t : 2?l?1 '(t) 2?l ; Dl = s?1=2; T ] \ t : '(t) 2?l?1 : Since the function ' satis es 0 '(t) , for s?1=2 t T , the sets Bl and Dl m S are closed and Dm Bl = s?1=2; T ]. Furthermore, (5.8) implies that '(t) t{ ,
1 ; T ], where Ll = 2(l+1)={ . for t s?1=2 , whence Bl L? l Recall that '(t) 2?l , for t 2 Bl , and '(t) 2?m?1 , for t 2 Dm . Therefore the relation Dm s?1=2; T ] yields

s?1=2

'(t) t dt

1?4={ F

s ;

for

{ > 4;

(5.9)

l=l

T R

where G = ln T + ln s, for = ?1, and G = T for ?1 < 0. We shall choose m such that 1?4={ F s?1 ; 2?m G for ? 1 0: (5.11) More precisely, we choose the minimal m such that sG for ? 1 0: (5.12) m ln12 ln 1? 4={ F ; Using (5.10), (5.11), we see that the estimate (5.9) follows provided that we show that m P 1?4={ F ; ?l R t dt: Il where I = 2 (5.13) l s
+1 ,

s?1=2

'(t) t dt

Dm

'(t) t dt +

m R P

l=l Bl

'(t) t dt

2?m G +

m P

l=l

2?l t dt; (5.10)

Bl

Below we shall prove the inequalities Il (l + ln T ) s?1 2?l+4l={ ; Il T +1 s?1 2?l+4l={ ;

l=l

Bl

for = ?1; for ? 1 <

0;

(5.14)

32

V. BENTKUS

F. GOTZE

for l m. These inequalities imply (5.13). Indeed, in both cases = ?1 and > ?1 we can apply the bound
1 P
l=l

2?l+4l={

(2?l )1?4={

1?4={

since { > 4 ensures the convergence of the series, and, according to the de nitions of and l , we have 2?l ?1 . In the case = ?1 one needs in addition the following estimates. For l 1, we have
1 P
l=l

l 2?l+4l={
1 R
l ?1

1 P
l=0

l 2?l+4l={

(2?l )1?4={

1?4={ ;

and, for l
1 P
l=l

c{ with a su ciently large constant c{ , we obtain l 2?l+4l={ x 2?x+4x={ dx l (2?l )1?4={

1?4={

1 + ln 1 :

It remains to prove the inequalities (5.14). For the estimation of Il we need a description of the structure of the sets Bl with l m. Let t; t0 2 Bl denote points such that t0 > t. The inequality (5.8) and the de nition of Bl imply 4?l?1 M{ (t0 ? t; s): If t0 ? t s?1=2 then by (5.15) and the de nition of M("; s) we get (5.15) = s?14(l+1)={ : = 4?(l+1)={ : (5.16) (5.17) (5.18)

t0 ? t t0 ? t

; ;

where where 1 note that

If t0 ? t s?1=2 then by (5.15) and the de nition of M("; s) we have For { > 4 and su ciently large s

< ;

provided l m:

Indeed, using the de nitions of and , we see that the inequality (5.18) follows from the inequality 4 s?{=4 < 2?m , which is implied by (5.12), the assumption (5.4) and s 1. The estimate (5.18) implies that either t0 ? t or t0 ? t . Therefore it follows from (5.16){(5.18) that

t 2 Bl =) Bl \ (t + ; t + ) = ?;

(5.19)

that is, that in the interval (t + ; t + ) the function ' takes values lying outside of the interval 2?l?1 ; 2?l ].

LATTICE POINTS

33

Let us return to the proof of (5.14). If the set Bl is empty then (5.14) is obviously ful lled. If Bl is nonempty then de ne e1 = min t : t 2 Bl . Choosing t = e1 and using (5.19) we see that the interval (e1 + ; e1 + ) does not intersect Bl . Similarly, let e2 denote the smallest t e1 + such that t 2 Bl . Then the interval (e2 + ; e2 + ) does not intersect Bl . Repeating this procedure we construct a 1 e <e < sequence L? < ek T such that 1 2 l

Bl

k S j =1

ej ; ej + ]

and

ej+1 ej + :

(5.20)

The sequence e1 <

< ek can not be in nite. Indeed, due to (5.20) we have ek e1 + (k ? 1)

1 L? l + (k ? 1)

k ;
= ?1. Us-

and therefore k T= . Using (5.20) we can nally prove (5.14). We start with the case ing ln(1 + x) x, for x 0, we have

Il

n k k ejR+ dt P P ? l ? l =2 ln 1 + ej 2 j =1 j =1 ej t

2?l

j =1 ej

k P

(l + ln T ) 2?l+4l={ s?1

1 since e1 L? l

, k T= , and
j =1 e1 + (j ? 1) k P 1 k 1 1 P j =1 j ln T + ln ?1

j =1 ej

k P

(l + ln T ) 4l={ :

Finally, let us prove (5.14) for ?1 <

0. We have
2l j =1 k P

Il

k 1 P 2l j =1

ejR+ ej

t dt

k 1 P 2l j =1

ej

T +1
2l

s?1 2?l+4 l={ T

+1 :

6. Trigonometric sums of irrational quadratic forms

In this section we introduce a criterion for the rationality of a quadratic form in terms of the behavior of an associated sequence of trigonometric sums. Recall that a quadratic form Q x] = Q x; x , x 2 R d , with a non-zero symmetric matrix Q = (qij ), 1 i; j d, is rational if there exists an M 2 R , M 6= 0, such that the matrix M Q has integer entries; otherwise it is irrational.

34

V. BENTKUS

F. GOTZE

For a 2 R d , d 1, consider the polynomial

P (x) = Q x] + a; x ;

x 2 Rd :

(6.1)

Throughout this section we shall denote by the uniform lattice measure in the box B (r) = jxj1 r (cf. with the notation ( ; r) used ?in other sections). In other words, the measure is non-negative normalized (R d ) = Zd \ B (r) = 1 and assigns ? equal weights x def = (fxg) = 2 r] + 1 ?d to points x 2 Zd \ B (r). For k 2 N and t 2 R , introduce the trigonometric sum

f (t) = f (t; r) =

x2Zd

e tP (x)

(2k+1) ; x

(2k+1) def = x

(2k+1) (fxg);

where k denotes of , R the k -fold convolution of . Let e denote the symmetrization k k and using that is, e(C ) = (C +x) (dx), for C 2 Bd . Writing (2k+1) = the symmetrization inequality (see Lemma 6.5 below), we obtain 0 f (t) 2 '(t) with

'(t) = '(t; r) def =

For a family of functions g = g( ; r) : R ! C parameterized by r following condition: for any 0 < 0

x2Zd y2Zd

e 2 t Q x; y

ex eyk :

(6.2) 0, consider the (6.3)

< 1;

r!1

lim

sup
jtj

g(t; r) = 0:

We shall apply condition (6.3) to the trigonometric sums f and '. See (6.4) and (6.6) for equivalent formulations of (6.3). The following characterization result holds without assumptions on the eigenvalues of Q . Theorem 6.1. Let k 1. The quadratic form Q x] is irrational if and only if ' satis es condition (6:3). If Q is irrational then f satis es (6:3). If a and Q are rational (that is, there exists M 6= 0 such that M a and M Q have rational coordinates, resp. entries) then f does not satisfy (6:3). The proof of Theorem 6.1 will be given later. It is based on an application of the theory of successive minima (see Cassels (1959), Davenport (1958)) and techniques in BG (1994). For a given family of functions, the condition (6.3) allows the following equivalent reformulation: the exist functions 0 (r) # 0 and (r) " 1 such that
r!1

lim

(r) jtj (r)

sup

g(t; r) = 0:

(6.4)

Applying a double large sieve type bound (see the estimate (5.22) in BG 1997) we obtain '(t) d;k q2d Md (t; r2) with M de ned by (1.20). Hence, assuming 0 (r) r?1 ,

LATTICE POINTS d (r), for r?1 t we have '(t) d;k;Q 0

0 (r), and

35

r!1 a2Rd r?1 t

lim sup

sup

(r)

'(t) = 0

(6.5)

provided that the eigenvalues of Q are non-zero. Combining (6.4) and (6.5) we see that the irrationality of Q is equivalent to the following condition: there exist (r) " 1 such that (6.6) rlim !1 sup ?1 sup '(t) = 0
a2Rd r

provided that the eigenvalues of and (6.6) hold for f as well.

t (r)

are non-zero. Due to jf j2

', relations (6.5)

Remark 6:2. An inspection of the proof shows that the condition (6.3) for the trigonometric sums f and ' holds uniformly over compact sets, say Q, of irrational matrices, that is, lim sup sup '(t; r) = 0: r!1
Q2Q
0

jtj

Let us recall some facts of the theory of successive minima in the geometry of numbers (see Davenport 1958). Let F : R d ! 0; 1) be a norm, that is, F ( x) = j j F (x), for 2 R , and F (x + y) F (x) + F (y). The successive minima M1 Md of F d with respect to the lattice Z are de ned as follows: M1 = inf F (x) : x 6= 0; x 2 Zd , and Mk is de ned as the lower bound of > 0 such that the set x 2 Zd : F (x) < contains k linearly independent vectors. It is easy to see that there exist linearly independent vectors a1 ; : : :; ad 2 Zd such that F (aj ) = Mj .
linear forms in d such that jk = kj . Assume that P

Lemma 6.3 (Davenport 1958, Lemma 3). Let Lj (x) = Pd k=1

R 1 and let k k denote the distance of the number to the nearest integer. Then the number of x = (x1; : : : ; xd) 2 Zd such that kLj (x)k < P ?1; jxj1 < P; for all 1 j d;

jk xk , 1

d, be

Md are the rst d of is bounded from above by M1 cd M , where M1 d the 2 d successive minima M1 M2 d of the norm F : R 2 d ! 0; 1) de ned 2 d d for y = (x; m) 2 R with x; m 2 R as F (y) def = max P L1(x) ? m1 ; : : :; P Ld (x) ? md ; P ?1 jxj1 :
Furthermore, M1
n o

(6.7)

P ?1 .

36

V. BENTKUS

F. GOTZE

Lemma 6.4 (BG 1994, Lemma 5.4). For z = (z1 ; : : :; zd ) 2 R d ,

g(z) def =
where
P

y2Zd

e z; y

eyk

d;k

m2Zd

h(m); s 2 Z:
(6.8)

h(m) =

d Q j =1

h0 (zj ? 2 mj );

? h0 (s) = 1 + r2 s2 ?k ;

Proof. We have g(z) = vk (z) with

v(z) =

y2Zd

e z; y

ey

y2Zd

e z; y

d 2 Q u(zj ); y = j =1

(6.9)

? where u(x) = D2r] (x)= 2 r 2 and 2 r = 2 r] + 1 with the Dirichlet kernel

D r] (x) =

sin(xr) exp ixn = sin( x=2) ; n=? r]

o

r] P

for x 2 R : , we obtain (6.10)

Using j sin yj min 1; jyj and x2

sin2(x=2), with y = xr and jxj

1

u(x)

min x21r2 ; 1
P

1 + r 2 x2

;
P

for jxj
1

The function u(x) is even and 2 -periodic. Hence (6.10) yields

uk (x) k

m2Z

jx ? 2 mj 1 + r2 (x ? 2 m)2
I

k m2Z 1 + r2 (x ? 2 m)2
?

(6.11)

where IfAg denotes the indicator function of the event A. Combining g(z) = vk (z) and (6.9), (6.11) we conclude the proof. Proof of Theorem 6:1. Let us show that for rational Q the trigonometric sum ' does not satisfy (6.3). Assume for simplicity that Q has integer entries. Let denote the product measure of measures and . Clearly, Q x; y 2 Z, for x; y 2 Zd, and the function

'(t) =

m2Z

pm e 2 tm ;
P

pm def =e

k ? (x; y ) 2 2d :

Q x; y

=m ;

with weights pm such that pm = 1, is a -periodic function and '( ) = 1. Similar m2Z arguments show that for rational Q and a the trigonometric sum f does not satisfy (6.3).

LATTICE POINTS

37

To complete the proof of the theorem, it remains to prove that if f or ' does not satisfy condition (6.3) then Q is rational. The inequality jf j2 ' shows that we have to consider the case of ' only. Since (6.3) is not ful lled, we have for some 0 < 0 < 1; lim sup sup '( t; r) > 0:

r!1 0 jtj Hence, there exist " > 0 and sequences tn ! t0 , 0 jtn j , and rn " 1 such that '( tn; rn ) " > 0; for all n 2 : (6.12)

We shall prove that (6.12) implies the rationality of Q . Henceforth we shall write t and r instead of tn and rn . Without loss of generality we shall assume throughout the proof that r = rn is su ciently large, that is, that r c with some su ciently large constant c = c(d; k; Q ; ; "). Furthermore, we shall write instead of d;k;Q; ;" . The measure e is concentrated in the cube B (2 r) such that P P P G(x) ex = G(x) ex ; ex = 1; 0 ex r?d ; (6.13) for any summable function G. Using the representation (6.2) of ', Lemma 6.4, the relations (6.12) and (6.13) for the function h(m) de ned by (6.8) with z = 2 t Q x, we have P P " r?d h(m): (6.14) Obviously we may replace h(m) in (6.14) by
x2B(2 r)\Zd m2Zd x2Zd x2B(2 r)\Zd x2B(2 r)\Zd

h(m) def =

d Q

since this yields in the inequality (6.14) an extra factor depending on d and k only. For vectors s = (s1; : : :; sd) 2 Zd such that sj 0 and = ( 1; : : :; d) 2 Zd such that j = 1, for all j , introduce the class Hs; of vectors (x; m) 2 Z2d such that ? sj jz (x) ? m j < (sj + 1) ; x 2 B (2 r); sign zj (x) ? mj = j ; j j 4r 4r for all 1 j d. Here zj (x) denote the j th coordinate of z(x) = t Q x. Moreover, consider the class H of vectors (x; m) 2 Z2d such that (6.16) x 2 B (4 r); kzj (x)k < 41r ; for all 1 j d; where k k denote the distance from the number to the nearest integer. Then S Hs; = (x; m) 2 Zd : x 2 B (2 r) : Furthermore, it is easy to see that (x; m); (x; m) 2 Hs; implies that (x ? x; m ? m) 2 H . Hence, card Hs; card H . Using (6.14) and the de nitions introduced, we obtain P " r?d card Hs; max h(m) : (x; m) 2 Hs;
s; s;

j =1

h0 (zj ? mj );

? h0 (s) = 1 + r2 s2 ?k ;

z = z(x) = t Q x;

(6.15)

r?d card H

d ? P Q s j =1

?k 1 + s2 j

r?d card H;

(6.17)

38

V. BENTKUS

F. GOTZE

for su ciently large r. Lemma 6.3 shows that the cardinality of the class H de ned by (6.16) is bounded c d from above by M1 Md , where Mj are the successive minima of the norm F de ned by (6.7) with P = 4 r and

Lj (x) = zj (x) = (t Q x)j =

d P i=1

tqji xi :

(6.18)

Combining (6.17) with card H (M1 Md )?1 , we obtain rd M1 Md 1. Hence, using P = 4 r and P ?1 M1 Md , we have Mj r?1 , for all ? 1 j d. 2 d Let (as; bs) 2 Z denote linearly independent vectors such that F (as; bs) = Ms, for ? all 1 s 2 d. The equalities F (as; bs) = Ms together with (6.7) and relations Ms r?1 P ?1 imply that

jasj1 1;
?

jbsj1 1;

1 s d:

(6.19)

Combining the relations F (as; bs) = Ms with P = 4 r and (6.7), (6.18), we get n) (n) (recall that t = tn and r = rn ; therefore as = a( s and bs = bs also depend on n)
d P k=1 n) (n) tn qjk a( sk ? bsj
2 ; rn

1 j d; 1 s d:

(6.20)

n) (n) in Zd The inequalities (6.19) guaranty that the sequence of systems a( 1 ; : : :; ad is repeating in nitely often. Choosing an appropriate subsequence, we may assume n) (n) that a( s = as are independent of n. Similarly we may assume that bs = bs are independent of n. Passing to the limit as n ! 1 in (6.20) along the subsequence and using tn ! t0 , t0 6= 0, we see that the numbers t0 qj1 ; : : :; t0 qjd satisfy d P i=1

asi t0 qji = bsj ;

1 s d;

(6.21)

To conclude the proof we have to show that the vectors a1 ; : : :; ad 2 Zd are linearly independent in Zd (or, equivalently, in R d ). If a1; : : : ; ad are linearly dependent then d P there exist vs 2 R not all equal to zero such that jvsj < 1 and vs as = 0. Let s=1 us prove that there exist integers u1 ; : : :; ud not all equal to zero such that jusj 1 d P and x def = us as = 0. By the multivariate Dirichlet approximation (see, for example,
s=1

for all 1 j d. Below we shall prove that the vectors a1 ; : : :; ad are linearly independent. Therefore the system (6.21) has the unique solution t0 qj1 ; : : :; t0 qjd , which obviously has to be rational by Cramer's rule, for all 1 j d.

LATTICE POINTS

39

Cassels 1959, Section V.10), for any Q 2 N there exist uj 2 Z and an integer 0 < q Q such that for all 1 s d: (6.22) vs ? uqs < q Q11=d ; The inequalities jvs j < 1 and (6.22) yield jusj 2 Q. Since the vectors as have ind P teger coordinates and jasj1 1, the equation vs as = 0 together with (6.22) implies x =
d P

For the vector (x; m) with m def =

s=1

us as = 0, for su ciently large Q

d P s=1

s=1

1. Hence jus j
1

1.

us bs we have

F (x; m)
since jusj 1 and F (as; bs)
?

d P

1=r. Using (6.7), we have 4r

d P s=1

s=1

jusj F ? (as; bs)

(6.23) (6.24)

F (x; m)
s=1

us bsj ;

for all 1 j d;
s=1

where bs = (bs1; : : :; bsd). Combining (6.23) and (6.24), using that r is su ciently large d d P P and that us bsj are integers, we conclude that us bsj = 0, for all 1 j d. In other words, m = us bs = 0, which together with the assumption x = us as = 0 s=1 s=1 means that the vectors (as; bs) 2 Z2d, 1 s d, are linearly dependent, a contradiction. A symmetrization inequality. The following symmetrization inequality is proved in BG (1997) the symmetrization e of a measure is de ned R as Lemma 7.1. Recall that d by e(C ) = (C + x) (dx), for C 2 B . Lemma 6.5. Let Q : R d ! R d be a linear symmetric operator, L 2 R d and C 2 R . Let 1; 2; 3; denote arbitrary measures on R d . De ne a real valued polynomial of second order by P (x) = Q x; x + L; x + C; for x 2 R d : Then the integral 2 R J = e tP (x) 1 2 3 (dx) satis es 2 J J1 + J2 , where RR J1 = e 2 t Q x; y e1 (dx) e2(dy); RR J2 = e 2 t Q x; z e1 (dx) e3(dz): In particular, if 2 = 3 then J J1 .
d P d P

40

V. BENTKUS

F. GOTZE

7. Properties of the distribution functions Fj and the signed measures j We start by establishing some properties of Fj and j (see (2.13){(2.15) for de nitions) using the Fourier transforms of the measures j . Recall that we denote W = x 2 d : x ? a] 2 I ; I = ( ; ]; Fj (I ) = Fj ( ) ? Fj ( ):

Let var Fj denote the variation of Fj . Write

R Q

Lemma 7.1. The density D : R d ! R de ned by (2:12) has continuous bounded partial derivatives @ D, for j j1 k ? 2. Assume that j k ? 2. Then we have Dj (x) k;d r ?j?d I jxj1 R + kr ; (7.1)
and If the measure and

V = x 2 R d : M (x=R) ? 1
n

!( 0) ;

0 = (kr + 1)=R;

r = r] + 1=2:

var Fj k;d r ?j (R=r)d ; for R r: is de ned by (2:17) then we have in addition Dj (x) k;d r ?j?d I x 2 V ;

Fj (s)

(7.2) (7.3) (7.4)

Proof. The Fourier transforms of complex valued functions f : R d ! C are denoted by

R

Fj (I )

k;d r ?j ?d vol(W

\ V ):

e(y ) = e y; x f (x) dx: f Let u(x) = I jxj1 1=2 , x 2 R d , denote density of the measure . The mea? = (2 r) ?d u x=(2 r) . Write U = d sure ( ; r) has density ur (x) def dx for density of the k measure de ned by (2.11). Then has density D = U ur (see (2.12)), where k and e =U e u denotes the convolution of functions. Therefore D er

@ D(x)
e j 1. Hence, using since jU

d d Q

e (y ) dy jy j D

k jy j u er (y ) dy;

(7.5)

u e(y ) =
we obtain

sin(yj =2) j =1 yj =2

e(2 ry ); u er (y ) = u

@ D(x) k;d r ?j j1?d ; for j j1 k ? 2; which proves the assertion about @ D of the lemma.

(7.6)

LATTICE POINTS

41

Let us prove (7.1). Using the de nition (2.13){(2.14) of Dj , well-known properties of the Fourier transforms, the equality j = 1 + + m , estimating in (2.14) jus j d 1 and applying (7.5), (7.6), we have

Dj (x)

j :max j j1 =j j;d
R

m D(j) (x)u11 : : : um
1

m Q
m

l=1

(k+1) (dul )

e (y ) y; u1 D e (y ) dy D

y; um

dy

m Q l=1

(k+1) (dul )

k;d jy jj

j;d r ?j ?d :

Now (7.1) follows since D(x) = 0, for jxj1 > R + kr. Let us prove (7.2). Using (2.15) and (7.1), we obtain

Dj (x) dx k;d r ?j?d (R + kr)d k;d r ?j (R=r)d : Let us prove (7.3). By (2.21) and (2.22) the density D is constant outside the set V . Thus, outside V the derivatives of D vanish and Dj (x) = 0, for x 2 Rd n V . Thus (7.1) yields (7.3). Similarly, (7.4) follows from (7.1) and (7.3) and Fj (s) F (I )
R n

I x 2 W \ V Dj (x) dx:

For further investigation of Fj we shall use the following double large sieve type bound (Lemma 7.2 below) which is closely related to Lemma 2.4 in Bombieri and Iwaniec (1986). It follows from Corollary 5.3 in BG (1997) replacing in that corollary q2 by q. Lemma 7.2. Assume that functions g; h : R d ! C satisfy g(x) 1 and h(x) 1. Let and denote arbitrary measures on R d such that

x 2 R d : jxj1 T J=
R R

= 1 and

x 2 R d : jxj1 S
2

= 1;

for some T > 0 and S > 0. Write

g(x) h(y) e t Q x; y

(dx) (dy) ;

t 2 R:

Then there exists a positive constant cd depending on the dimension d only such that
? d B sup x + cd B ; J d qd 1 + S T jtj d supd x + c j t j S x2Rd jtj T x2R where B = x 2 R d : jxj1 1 .

42

V. BENTKUS
R

F. GOTZE m (dx; r) ;

Corollary 7.3. For an integer m 2, the function

0 (t) =

a2Rd

sup

e t Q x] + a; x

satis es

0 (t)

d q d=2 min 1; (r2 jtj)?d=2

(7.7) 1. Thus we can assume

Proof. We shall derive the result using Lemma 7.2. If 1 r2 jtj then (7.7) is obviously ful lled since 0(t) that r2 jtj > 1. Using the obvious identity of the type
R

the fact that the form Q x] is quadratic and Holder's inequality, we obtain
2 0 (t)

f (x)

3 (dx) =

f (x + y + z) 1 (dx) 2(dy) 3(dz);

a2R

R supd J (m?2) (dz; r)

(7.8)
2

with

g(x) h(y) e 2 t Q x; y (dx; r) (dy; r) ; J= where the functions jgj 1 and jhj 1 may depend on z and a. Choose S = T = r. The measure ( ; r) has nonnegative density (with respect to the Lebesgue measure on R d ) bounded from above by (2 r) ?d . Therefore (A; r) (2 r) ?d vol A, for any measurable set A R d . Thus Lemma 7.2 and the assumption r2 jtj > 1 yield
R R ? ? qd ; J d qd 1 + r2 jtj d r?2d jtj r ?2d d r2d jtjd whence, using (7.8), we derive (7.7). Lemma 7.4. The distribution function s ! F0 (s) has a bounded continuous derivative for d 3. The functions s ! Fj (s) are functions of bounded variation for 2 j < d=2,

j j qj+d=2 ; 1 + ja r s Moreover, each of the functions s ! Fj (s), j 2 2 N 0 , has (d ? 1)=2] ? j bounded continuous derivatives. Proof. Changing the variables t = r?2 , we have

and

sup Fj (s)

j;d r2j

Rj

R?

R R

jtj r2 min 1; ? jtj r2

n

? ?

o o

dt
jtj
dt

R?

jtj r2 min 1; jtj r2

1 ; r2 ; 1;

for ? 1 < < ? 1; (7.9) for 0 < < : (7.10)

LATTICE POINTS

43

bj (t)j, the estimate of Corollary 7.3 for 0 (t) Using (j ) j , the bound (4.11) for F and (7.9), (7.10), choosing = d=2 and = (j ), = j respectively, we obtain R Rj 1 + jaj j q j +d=2 ; b (t)j dt F (7.11)

for j 2 2 N 0 such that j < ?1 + d=2. Similarly to (7.11), we obtain R Rj 1 + jaj j q j +d=2 ; bj (t)j dt F j;d jtj r2j r R for j 2 2 N such that j < d=2. Finally we have R Rj jaj j qj+d=2 ; bj (t)j dt 1 + F jtj j;d rj+d=2 r r jtj 1 for r 1 and j 2 2 N 0 such that j < d=2, and R bj (t)j dt < 1; jtj F for 0
R

j;d r2j+2

(7.12) (7.13) (7.14)

< ?j ? 1 + d=2:

To conclude the proof it su ces to apply the Fourier and Fourier{Stieltjes inversion formulas and to use the bounds (7.11){(7.14).
8. On the Lebesgue volumes of certain bodies related to indefinite quadratic forms

We shall describe the asymptotic behaviour of the volume of the set A = x 2 R d : M (x) 2 R I0 ; Q x ? a] 2 I ; R ! 1; (8.1) where M is the Minkowski functional of the set de ned by (1.13) and I0 and I = ; ] are nite intervals. Throughout the section we assume that Q x] is an inde nite quadratic form in R d , d 3. Re-enumerating the eigenvalues of Q , choosing an appropriate orthonormal basis of R d and denoting the coordinates of x 2 R d in this basis as x1 ; : : :; xd , we may assume that 2 Q x] = q1 x2 1 + + q d xd with q1 ; : : :; qn > 0, qn+1 ; : : :; qd < 0 and some 1 n d=2. Indeed, if n > d=2, we may replace in (8.1) the matrix Q and the interval I by ?Q and ?I respectively. Let S = S n?1 S d?n?1 denote a direct product of the unit (n ? 1)- and (d ? n ? 1)dimensional spheres 2 2 def 2 r2 def = x2 = x2 1 + : : : xn = 1 and n+1 + : : :xd = 1 with area elements d 1 and d 2 respectively, and d def = d 1 d 2 . Write p p ? M0 (x) = M x1 = jq1 j; : : :; xd = jqd j :

44

V. BENTKUS

F. GOTZE

Lemma 8.1 (cf. Lemma 3.8 in Eskin, Margulis and Mozes (1997)). The volume of the
set A de ned by (8:1) satis es
1 ?d+2 vol A = j det Q j?1=2 ? R ud?3 lim R 2 R!1
0
R

I M0(u 1 + u 2 ) 2 I0 d du:
(8.2) (8.3) (8.4) (8.5)

Proof. We shall use the following representation vol A = 2?1 Rd?2 j det Q j?1=2 J; where
2 1 u R R 2 '(r; ) I R2 v J=R 0 ?1 R '(r; ) = I M0(x + a0=R) 2 I0 S

rn?1 d?n?2 dv du: d; x=r 1+

2;

and a0 = jq1 j a1 ; : : :; jqd j ad , p r = r(u; v) = u; = (u; v) = u2 ? v; 0 u < 1; ?1 < v u2: (8.6) For the proof of (8.3) it su ces to write R vol A = I M (x + a) 2 R I0 I Q x] dx; p to change the variables xj = Ryj = jqj j, 1 j d, to use the polar coordinates r and and to perform the change of variables (8.6). Let us prove (8.2). Assuming that R ja0j, it is easy to see that I M0 (x + a0 =R) 2 I0 = 0 unless 0 r c; 0 c; with some su ciently large c = c(M0; I0). Consequently, we have '(r; ) = 0 unless 0 u c; jvj c2 . Therefore, for su ciently large R, we can write

?p

J = f (u) du
with
0

c R

(8.7)

f (u) = rn?1 R2

2 =R R

The function '(r; ) is a continuous function of the variables u; v and a0=R. Therefore, for any u > 0, we obtain d?3 ( ? ) R I M0 (u 1 + u 2 ) 2 I0 d : lim f ( u ) = u R!1 The estimate f (u) theorem, and (8.7) yields
? d cd?2 j

=R2

'(r; ) I ?1 v u2 d?n?2 dv:

j + j j allows to apply the dominated convergence

R

which combined with (8.3) concludes the proof of (8.2).

lim J = ( ? ) ud?3 I M0 (u 1 + u 2 ) 2 I0 d du; R!1

0

1 R

LATTICE POINTS

45

(dq)?1=2 jxj M0 (x) m jxj: (8.8) Lemma 8.2. Let I0 = 0; ] and = + ja0j=R, = =m ? ja0j=R. Then the volume of the set A de ned by (8:1) satis es vol A d ( ? ) q(d?2)=2 d?2 Rd?2 : (8.9) If > 0 and j j + j j 2 R2=5 then vol A d ( ? ) q?d=2 d?2 Rd?2 : (8.10)

The relation (2.18) yields

Proof. We shall use the representation (8.3){(8.6). Estimating 1 j det Q j q d , we see that it su ces to prove that J d ( ? ) q(d?2)=2 d?2 ; (8.11) d ? 2 J d( ? ) : (8.12) ? Let us prove (8.11). Using (8.8), we have M0 (x + a0=R) (dq)?1=2 jxj ? ja0j=R and R '(r; ) I jxj2 dq 2 d d I u2 + 2 dq 2

since r = u and jxj2 = r2 + 2. Hence, we get

1 R d R2 q (d?3)=2 d?3 0

2 =R R 2

proving (8.11). ? Let us prove (8.12). Using (8.8), we have M0 (x + a0=R) m jxj + ja0j=R and R '(r; ) I jxj2 2 d d I 2 u2 + jvj 2 since r = u and we obtain
S 2 = u2 ? v

=R

I u2 dq

dv du d ( ? ) q(d?2)=2 d?2 ;

u2 + jvj. Hence, using the condition j j + j j

2 u R

2 R2 =5,

proving (8.12). The de nition (2.19) of the modulus of continuity ! of M yields ? M0(x + y) ? M0(x) jxj ! d pq jyj=jxj : Write 0 j ; " = j j + j j ; " = ! (2 d " pq ); " = ! (30 d " pq ): "1;0 = ja 5 2 2 4 1;0 R R2 R2

1n 2 R u2 d R2 I 4 0 R n 2 d ? 3 2 R I 4 d

2o

u2

?1
3

I jvj I

2o R

2o

vR2

rn?1 d?n?2 dv du

vR2

dv du d ( ? ) d?2 ;
(8.13) (8.14)

46

V. BENTKUS

F. GOTZE

Lemma 8.3. Let I0 = 1 ? ; 1 + ], 0

"1;0 c; "2 c;

1=4. Assume that

"4 c q?1=2 ; "5 c q?1=2 (8.15) with some su ciently small positive constant c = c(d; m) depending on d and m only. Then the volume of the set A de ned by (8:1) satis es
vol A d;m ( ? ) ( + "4 + "5 ) Rd?2 q(d?2)=2 :
Proof. We shall write inequality j det Q j?1=2

(8.16)

instead of d;m . Using the representation (8.3){(8.6) and the 1, we reduce the proof of (8.16) to the veri cation of

J ( ? ) ( + "4 + "5 ) q(d?2)=2: (8.17) Using jxj2 = r2 + 2, applying the inequalities (8.8) and the triangle inequality, we see that '(r; ) = 0 unless 1 ? ? "1;0 jxj pq (1 ? ) + "1;0 since now I0 = 1 ? ; 1 + ] with 0 1=4. Hence the assumptions (8.15) and q 1 imply that '(r; ) = 0 unless 1=2 jxj pq: (8.18) The variable v in (8.4) satis es jvj "2 c 1=8. Thus, (8.18) shows that we canp asq, sume that r = u in (8.4) satis es u 1=4. Using (8.18) we can estimate r; jxj and we have 2 1 =R R R ( d ? 3) = 2 2 J q R '(r; ) dv du: (8.19)
Using (8.13) and (8.18), we obtain
1=4

=R2

0 M0 x + a R ? M0 (x)

pq ! d pq ja0j

Let us write x = r 1 + r 2 + ( ? r) 2. The condition jxj 1=2 yields r + 1=2. Therefore, using u = r 1=4 and repeating the arguments used for the proof of (8.20), we obtain j ? rj = jvj M (x) ? uM ( + ) pq " : (8.21) 8 jvj 8 " ;
r r (r + )
2 0 0 1 2 5

R jxj

pq " :
4

(8.20)

Using (8.20) and (8.21), we can replace the indicator function in (8.5) by

I 1 ? ? c0 pq "4 ? c0 pq "5 u M0( 1 + 2 ) 1 + + c0 pq "4 + c0 pq "5 ; (8.22)

where the constant c0 = c0 (d; m) depends on d and m only. Now using (8.19), integrating the indicator function (8.22) in the variable u, estimating M0( 1 + 2 ) 1 and integrating in v, we obtain (8.17).

LATTICE POINTS

47

References
Averbuh, V.I. and Smoljanov, O.G., Di erentiation theory in linear topological spaces, Russian Math. Surveys 22 (1967), no. 6, 201{258. Bentkus, V. and Gotze, F., On the lattice point problem for ellipsoids, Preprint 94{111 SFB 343, Universitat Bielefeld (1994). Bentkus. V. and Gotze, F., On the lattice point problem for ellipsoids, Russian Acad. Sc. Doklady 343 (1995), no. 4, 439{440. Bentkus, V. and Gotze, F., Optimal rates of convergence in the CLT for quadratic forms, Ann. Prob. (1996), no. 1. Bentkus, V. and Gotze, F., On the lattice point problem for ellipsoids, Acta Arithmetica LXXX.2 (1997), 101{125. Bentkus, V. and Gotze, F., Uniform rates of convergence in the CLT for quadratic forms in multidimensional spaces, Probab. Theory Relat. Fields 109 (1997a), 367{416. Bentkus, V. and Gotze, F., Optimal bounds in non-Gaussian limit theorems for U -statistics, Preprint 97{077 SFB 343, Universitat Bielefeld (1997b). Bombieri, E. and Iwaniec, H., On the order of (1=2 + it), Annali Scuola Normale Superiore{Pisa (4) 13 (1986), 449{472. Cassels, J.W.S., An introduction to the geometry of numbers, Springer, Berlin { Gottingen { Heidelberg, 1959. Chung, K.L., A course in probability theory, Academic Press, New York and London, 1974. Cook, R.J. and Raghavan, S., Inde nite quadratic polynomials of small signature, Monatsh. Math. 97 (1984), no. 3, 169{176. Dani, S.G. and Margulis, G.A., On orbits of unipotent ows on homogeneous spaces, Ergod. Theor. Dynam. Syst. 4 (1984), 25{34. Davenport, H., Inde nite quadratic forms in many variables (II ), Proc. London Math. Soc. 8 (1958), no. 3, 109{126. Davenport, H. and Lewis, D.J., Gaps between values of positive de nite quadratic forms, Acta Arithmetica 22 (1972), 87{105. Eskin, A., Margulis, G.A. and Mozes, S., Upper bounds and asymptotics in a quantative version of the Oppenheim conjecture, Preprint (to appear in Ann. Math.) (1997), 1{57. Esseen, C.G., Fourier analysis of distribution functions, Acta Math. 77 (1945), 1{125. Hlawka, E., Uber Integrale auf konvexen Korpern I; II , Monatshefte fur Math. 54 (1950), 1{36, 81{99. Jarnik, V., Sur le points a coordonnees entieres dans les ellipsoides a plusieurs dimensions, Bull. internat. de l'acad. des sciences de Boh^ eme (1928). Jarnik, V., Uber Gitterpunkte in Mehrdimensionalen Ellipsoiden., Math. Ann. 100 (1928a), 699{721. Jarnik, V. and Wal sz, A., Uber Gitterpunkte in Mehrdimensionalen Ellipsoiden., Math. Zeitschrift 32 (1930), 152{160. Kratzel, E. and Nowak, G., Lattice points in large convex bodies, Mh. Math. 112 (1991), 61{72. Kratzel, E. and Nowak, G., Lattice points in large convex bodies, II, Acta Arithmetica LXII.3 (1992), 285{295. Landau, E., Zur analytischen Zahlentheorie der de niten quadratischen Formen (Uber die Gitterpunkte in einem mehrdimensionalen Ellipsoid), Sitzber. Preuss. Akad. Wiss. 31 (1915), 458{476. Landau, E., Uber Gitterpunkte in mehrdimensionalen Ellipsoiden, Mathem. Zeitschrift 21 (1924), 126{132. Lewis, D.J., The distribution of the values of real quadratic forms at integer points, Analytic number theory (Proc. Sympos. Pure Math., Vol. XXIV, St. Louis Univ., 1972), Providence, 1973, pp. 159{ 174. Margulis, G.A., Discrete subgroups and ergodic theory, Number theory, trace formulas and discrete groups (Oslo, 1987), Academic Press, Boston, 1989, pp. 377{398. Margulis, G.A., Oppenheim conjecture, Preprint (1997), 1{49.

48

V. BENTKUS

F. GOTZE

Matthes, T.K., The multivariate Central Limit Theorem for regular convex sets, Ann. Prob. 3 (1970), 503{515. Oppenheim, A., The minima of inde nite quaternary quadratic forms, Proc. Nat. Acad. Sci. USA 15 (1929), 724{727. Oppenheim, A., The minima of inde nite quaternary quadratic forms, Ann. Math. 32 (1931), 271{298. Wal sz, A., Uber Gitterpunkte in mehrdimensionalen Ellipsoiden., Math. Zeitschrift 19 (1924), 300{ 307. Wal sz, A., Gitterpunkte in mehrdimensionalen Kugeln, Warszawa, 1957. Yarnold, J., Asymptotic approximations for the probability that a sum of lattice random vectors lies in a convex set, Ann. Math. Stat. 43 (1972), 1566{1580.

LATTICE POINTS
Vidmantas Bentkus Fakultat fur Mathematik Universitat Bielefeld Postfach 100131 33501 Bielefeld 1 Germany Friedrich Gotze Fakultat fur Mathematik Universitat Bielefeld Postfach 100131 33501 Bielefeld 1 Germany

E-mail address : bentkus@mathematik.uni-bielefeld.de

E-mail address : goetze@mathematik.uni-bielefeld.de