IEEE COMMUNICATIONS LETTERS, VOL. 20, NO.

1, JANUARY 2016

125

Lattice-Reduction-Aided Sphere Decoding for MIMO

Detection Achieving ML Performance
Jinzhu Liu, Song Xing, and Lianfeng Shen, Senior Member, IEEE

AbstractWe propose a lattice-reduction-aided sphere

decoding (SD) algorithm for MIMO detection achieving exact
maximum likelihood (ML) detection performance with very low
computational complexity in the mid and high signal-to-noise
ratio (SNR) regions. A simple criterion is presented to determine
if the ML detection result is obtained by a primary search. If
not, a further search will be carried out to find the ML detection
result eventually. Simulation results show that in the mid and high
SNR environments, the computational complexity of the proposed
algorithm is considerably lower than that of the conventional SD.
Index TermsLattice reduction, maximum likelihood detection,
MIMO, sphere decoding.

I. I NTRODUCTION

ATTICE reduction (LR) has been introduced as a promising technique to improve the performance of the low complexity suboptimal MIMO detectors such as zero-forcing (ZF),
minimum mean-squared error (MMSE), and ordered successive
interference cancellation (OSIC) detection [1][3]. However,
it has been shown that although these LR-aided suboptimal
detectors can achieve a full diversity order of the MIMO fading channels, the performance of them still generally presents
an obvious gap with that of the optimal maximum likelihood
(ML) detector [2], [3].
The sphere decoding (SD) [4][6], on the other hand, can
achieve exact ML detection performance, but it requires considerable amount of computations compared with various suboptimal detection schemes. It is naturally expected that the lattice
reduction technique can be combined with the SD to reduce its
computational complexity. However, little attention has been
paid to this research direction, mainly owing to the negative
views on it in the literature. For example, in [5], the lattice
reduction is considered not useful for the SD for MIMO detection, which is a closest lattice vector problem (CVP) over a
(finite) subset of the infinite lattice. The reason claimed is that
the lattice transforming matrix often destroys the properties of
the subset, and the result of the LR-aided SD is not necessarily
the ML solution, which might be referred to as the out-ofbound symbol effects in the LR-aided SD. The authors of [6]

Manuscript received May 14, 2015; revised October 4, 2015 and November
17, 2015; accepted November 21, 2015. Date of publication November 26,
2015; date of current version January 7, 2016. This work was supported in part
by the China Important National Science and Technology Specific Projects (no.
2012ZX03004005-003) and in part by the National Natural Science Foundation
of China under Grant 61171081. The associate editor coordinating the review
of this paper and approving it for publication was A. C. Iossifides.
J. Liu is with the School of Electronic and Information Engineering, Nanjing
University of Information Science and Technology, Nanjing 210044, China
(e-mail: jzliu21@ nuist.edu.cn).
S. Xing is with the Department of Information Systems, California
State University, Los Angeles, Los Angeles, CA 90032 USA (e-mail:
sxing@calstatela.edu).
L. Shen is with the National Mobile Communications Research Laboratory,
Southeast University, Nanjing 210096, China (e-mail: lfshen@seu.edu.cn).
Digital Object Identifier 10.1109/LCOMM.2015.2504094

believe that although the LR-aided SD can derive the ML solution, it can not offer the complexity reduction. The theoretical
analysis in [7] confirms that the LR-aided SD can reduce the
computational complexity significantly, but the authors admit
that it has a performance gap with the ML detection due to its
out-of-bound symbol effects. Some improved fixed-complexity
SD schemes have been presented [8], in which the lattice reduction is intuitively adopted as a preprocessing to reduce the
complexity of the searching stage. However, the inherent outof-bound symbol effect problem is not taken into consideration
or only treated by using hard limiting.
In this letter, we propose a LR-aided SD algorithm for MIMO
detection achieving exact ML performance with very low computational complexity in the mid and high signal to noise
ratio (SNR) regions (roughly defined as the regions such that
Pe (S N R) < 102 , where Pe (S N R) is the symbol error rate
of the ML detection), which usually cover the practical SNR
range. A simple criterion is presented to determine if the ML
detection result is obtained by a primary search in a hypersphere. If not, a further search will be carried out to find the ML
detection result eventually. In addition, reordering the symbols
of the symbol vector, and the Schnorr-Euchner (SE) enumeration strategy are adopted to further increase the computational
efficiency.
II. S YSTEM M ODEL AND P RELIMINARIES
Consider a MIMO wireless communication system with
Nt transmitters and Nr (Nr Nt ) receivers. The relationship
between the Nt -dimensional transmitted complex-valued symbol vector sc (i.e. QAM symbol vector) and the Nr -dimensional
received vector xc is determined by xc = Ac sc + wc , where
Ac C Nr Nt presents a flat-fading channel matrix and has
complex-valued independent and identically distributed (i.i.d.)
Gaussian entries, and wc is the complex-valued additive white
Gaussian noise vector. In sphere decoding, the complex number
signal model needs to be reformulated to a real number signal
one as
x = As + w,

(1)

where x R M , s Z N , w R M , and A R MN with N =

2Nt and M = 2Nr . Now s is an N -dimensional PAM symbol
vector and belongs to a finite subset of the infinite integer vector
space Z N , denoted by s  Z N .
Let L (A) = Az, z Z N denote the lattice with A as its
basis, the closest lattice vector problem is to find the closest
lattice point to an arbitrary vector point x, formulated as
z = arg min x Az2 .

(2)

zZ N

Then the ML MIMO detection is to find the closest lattice

point to the received signal x over the finite subset , i.e.

1558-2558 2015 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.
See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.

126

IEEE COMMUNICATIONS LETTERS, VOL. 20, NO. 1, JANUARY 2016

sML = arg min x Az2 .

(3)

zZ N

In order to reduce the computational complexity of ML

detection, the search for sML can be limited within a hypersphere of a certain radius , which is referred to as sphere
decoding, expressed as


(4)
sML = arg min x Az2 2 .
zZ N

III. LR-A IDED S PHERE D ECODING FOR MIMO

D ETECTION
A. Basic Idea and Primary Search
By applying lattice reduction approaches such as the LenstraLenstra-Lovsz (LLL) algorithm [2], [3] on the basis A of
and a unimodular matrix U can
lattice L (A), a reduced basis A

has the almost orthogonal

be found such that A = AU, and A
columns. The unimodular matrix U has integer components and
so its inverse. Therefore, (1) can be transformed as
+ w,
x = AUU1 s + w = Ad

(5)

where d = U1 s is the transformed symbol vector. As aforementioned, s  Z N , so the range of d is the image of 

under U1 , denoted by d .
The basic idea of the LR-aided SD for MIMO detection is
as follows. In the range of transformed symbol vector d, i.e.
+w
the closest lattice point to the received signal x = Ad
,
is searched within a sphere of a certain radius , which is
described as


2


d ML = arg min x Az
(6)
 2 ,
N

zZ

and the inverse transformed symbol vector sML = Ud ML is

intuitively the exact ML detection result.
However, direct search for d ML outlined in (6) is impractical
is no longer of the simple
because, in general, the subset 
hypercube structure as that of . Therefore, controlling the
is a very complicated problem. To avoid
search range z 
the boundary control in the search, a expanded search range
 ,
is utilized in pracwithout the boundary control, z 
tice. Then, the primary search of the LR-aided SD for MIMO
detection will be expressed as


2


(7)
d = arg min x Az
 2 .

z

Nevertheless, the resulting d is not necessarily equal d ML and

thus s = Ud is not necessarily the ML detection result sML .
In the primary search, the SD algorithm with decreasing
radius search [6] is employed, in which the initial radius is set
as to ensure there exist lattice points in the hypersphere.
B. Further Search for Exact ML Detection Result
To obtain the exact ML detection result sML , firstly we need
a criterion to determine if the primary search result s = Ud is
sML or not.

Hence from (6) and (7), it can be

 .
Notice that 
2 
2



d ML 
d 
affirmed that x A
 x A
 , which is equivalent
2 

2
to x As x AsML  .
According to the definition of sML in (3), the following
proposition obviously holds.
Proposition 1: In lattice L (A) = Az, z Z N , suppose
AsML is the closest lattice point to a certain vector point
 x
among the lattice points in the subset Az : z  Z N . If
there exists a lattice point As which is closer to x than AsML ,
2 

2
/ .
i.e., x As < x AsML  , then s
Thus, the criterion to determine if s = Ud is the ML solution
sML can be concluded as follows: s = Ud must be sML if s .
Conversely, s must not be sML if s
/ .
If s = Ud
/ , further search for sML is required. The further search adopts the SD algorithm with increasing radius
search [5]. It proceeds as follows. Set a series of sphere radii
1 < < n in advance. Perform the SD to obtain all lattice
points in the sphere with the radius 1 , denoted by d 1 , d m .
According to Proposition 1, find the particular d i among them,
which is the closest point to x that satisfies si = Ud i  at the
same time. Therefore, si must be the ML detection result sML .
If sML is not found, increase the sphere radius as 2 and conduct the search again in the new sphere. The above procedure
repeats until sML is found.

C. Implementation of LR-Aided SD for MIMO Detection

The proposed LR-aided SD algorithm for MIMO detection
can be implemented as follows.
1) Main Routine: Input: x, A. Output: sML .
U (A
= AU).
1: Execute LR algorithm to obtain A,

2: Execute the QR decomposition of A to obtain the upper

= QR). The columns of A
is
triangular matrix R (A
reordered according to V-BLAST detection ordering [9].
x (A
is the Moore3: Compute the decision statistic y = A

Penrose inverse of A).

4: Execute primary search subroutine to obtain d.

5: Compute s = Ud. If s , sML = s, stop. Else, execute

further search subroutine to obtain sML .
y, R. Output: d.

2) Primary Search Subroutine: Input: x, A,

1: Set 02 = (a large enough value in practice).
2: Execute the basic SD search [5], [6] to find one point d in
the sphere with radius 0 (or no point is found later).
2


d 
3: If one point d is found, set 2 = x A
 (decreasing
0

radius), go to 2. Else, stop.

y, R, 2 =
3) Further Search Subroutine: Input: x, A,
0
2




x Ad . Output: sML .

1: Determine the radius series 1 < < n . Set
j = 1.
2: Execute the basic SD search to find all points in the sphere
with radius j , and sort them in ascending order of their
Euclidean distances to x, and denote them by d 1 , d m
2
2




d 1 
d m 
such that x A
 < < x A
 .
3: Compute si = Ud i , i = 1, , m. If the first si  is
found, sML = si , stop. Else, j j + 1, go to 2.

LIU et al.: LATTICE-REDUCTION-AIDED SPHERE DECODING FOR MIMO DETECTION

127

In the further search subroutine, the radius series 1 < n

is determined as follows [5]. Let w2 be the variance of the
entries of w, the noise term in x. Notice that = w2 /w2 =

2


x Ad
 /w2 is a 2 random variable with M degrees of
freedom. Hence, the radius 1 may be chosen in such a way
that with a high probability, the sML is found in the sphere, i.e.

12 w2


1 M 21 2
2M/ 2 M 2
/
e / d = 1

(8)

where the integrand is the probability density function of the

random variable , and the probability 1 is set to a value
close to 1. 2 , 3 , are determined in the same way, but with
the probability set as 1 2 , 1 3 , instead. If any entry
in the above obtained radius series is smaller than 0 , it must
be deleted since the corresponding sphere has been searched in
the primary SD. As for the choice of the parameter
, it
can be

optimized by solving the equation C fur , M, N , w2 =
0 to minimize the complexity, where C fur denotes the average
complexity of the further search step, which can be expressed as
C fur = C (1 ) + C (2 ) + 2 C (3 ) + ,

(9)

Fig. 1. Symbol error rate of various MIMO detectors in a Nt Nr uncoded

MIMO system using QAM. (a) Nt Nr = 8 8, 16QAM. (b)Nt Nr =
16 16, 64QAM.

The LR-aided SD described above is based on ZF, while the

LR-aided SD based on MMSE (MMSE-SD-LR) can be easily
obtained by replacing the signal x and the channel A with their
MMSE extended version respectively.

where
C ( ) = K

N
k=1


 

k/ 2  k 2 k

(10)

is the average total number of points


visited
 by SD in the
sphere of radius . In (10), k / 2  k 2 k is the volume
of the k-dimensional sphere of radius , which is proportional
to the expected number of lattice points inside the sphere, with
K denoting the coeffcient of proportionality [10].
The following is a rough argument to explain the lower
complexity of the proposed LR-aided SD compared to the conventional one. In the case of the ZF based SD (ZF-SD), noticing
that the first point found is the Babai point sB (also the ZF based
OSIC, or ZF-OSIC detection result), due to the SE enumeration
strategy being adopted [6], its complexity can be expressed as
C av = Pa C (a ) + (1 Pa ) C (b )
(11)


Pr () denoting the
wherePa = Pr sB = sML with 
 probability,


a = x AsML  w, b = A s sB + w, C (a ) and
C (b ) are computed according to (10).
The complexity of the LR-aided ZF-SD (ZF-SD-LR) is
 





C fur
C av
= Pa C (a ) + 1 Pa C b + 1 Pa 1 Pok
(12)




where Pa = Pr dB = dML (dB denotes the Babai point.),



 =P d
ML (d is the primary search result.),  =
Pok
r =d
b

 




A d dB + w, and Cfur denotes the complexity of the

further search, expressed as (9). When

the SNR increases to
a certain extent, say S N R >Pe1 102 , the following statements will hold: 1) C (a ) is roughly equal to the complexity of
the ZF-OSIC; 2) Pa  Pa , then 1 Pa  1 Pa (due to the

3)   b results in C   C (b )
better condition of A);
b
b
is reduced lattice base; C ( ) decreases exponentially with
(A
 are close to 1, thus 1 P  , 1 P  are close
); 4) Pa , Pok
a
ok
to 0. Taking all these into account, it can be concluded that

C av  C av .

IV. S IMULATION AND E XTENSION D ISCUSSIONS

A. Performance and Complexity Comparison
For performance and complexity evaluation and comparison, the numerical simulations are performed for various
MIMO detectors including the proposed ZF-SD-LR, ZF-SD
(conventional SD), ZF-OSIC-LR, ZF-OSIC, and their MMSE
counterparts.
The conventional SD scheme is chosen as the SD with
the decreasing radius search and V-BLAST detection symbol
ordering. In all relevant detectors, the LLL is chosen as the lattice reduction algorithm, and the parameter is set to 0.99 to
achieve its best performance [2].
1) Performance Comparison: Fig. 1 shows the symbol
error rate (SER) of various MIMO detectors in different systems. From Fig. 1 (a) it is observed that the proposed ZF-SD-LR
achieves exact ML performance since its performance curve
overlaps that of the ZF-SD perfectly. The MMSE-based SD
can not achieve exact ML performance, but the difference is
negligible. It is difficult to obtain the accurate performance
curves of the ZF-SD and the MMSE-SD in the case of 16 16,
64QAM, because the extremely high computational costs lead
to too long program runtime. The performance results of the
suboptimal detectors ZF-OSIC, ZF-OSIC-LR, MMSE-OSIC
and MMSE-OSIC-LR, are given also to show the significant
performance gain of the SD-LR.
2) Complexity Comparison: The computational complexity is measured in terms of the average number of floating-point
operations (FLOPs) used. Fig. 2 (a) illustrates that the complexity of the ZF-SD-LR decreases rapidly as the SNR increases,
and it is considerably lower than that of the ZF-SD when SNR
becomes higher.
For a larger MIMO system, 16 16 with 64QAM for
instance, the complexity of the ZF-SD-LR and the MMSE-SDLR gets higher, but it is still acceptable in the mid and high
SNR regions (see Fig. 2(b)). The complexity of the ZF-SD and
the MMSE-SD becomes extremely higher for this large system

128

IEEE COMMUNICATIONS LETTERS, VOL. 20, NO. 1, JANUARY 2016

TABLE II
N UMBER OF FLOP S U SED OF LR-A IDED F IXED C OMPLEXITY T REE
S EARCHING D ETECTION ( p = p = 2, M = 36 [8]) IN 8 8 MIMO
S YSTEMS . D ETECTOR 1: H ARD L IMITING IS U SED IN ML S TEP.
D ETECTOR 2: P ROPOSITION 1 P ROPOSED IN T HIS L ETTER IS U TILIZED IN
ML S TEP

Fig. 2. Average number of FLOPs used of conventional and LR-aided SD

MIMO detector in a Nt Nr uncoded MIMO system using QAM. (a) Nt
Nr = 8 8, 16QAM. (b) Nt Nr = 16 16, 64QAM.
TABLE I
SNR R EQUIREMENTS AND AVERAGE N UMBER OF FLOP S U SED OF
LR-A IDED SD W ITH T REE P RUNING (TP) AND C ONVENTIONAL SD
W ITH TP IN A 8 8, 16QAM MIMO S YSTEM

3) Soft-Output LR-Aided SD: The soft-output SD with

repeated tree search strategy [13] is separated into two stages.
The first stage is no other than the hard-output SD for obtaining
the ML solution. According to [13], in a soft-output SD with
modified repeated tree search strategy, the computational costs
of the two stages are comparable. Thus, the LR-aided SD can be
used instead of the conventional one in the first stage to reduce
the complexity.
V. C ONCLUSION
We have demonstrated that the LR-aided sphere decoding
for MIMO detection can achieve exact ML performance, and
can be implemented with very low computational complexity
in the mid and high SNR regions. The proposed method can
also be effectively used in conjunction with tree pruning methods, fixed LR-aided SD schemes or soft-output SD methods for
better complexity/performance tradeoff.

and cannot be illustrated in Fig. 2 (b) because too long program

runtime is needed to obtain the accurate results.
The MMSE-based SD algorithms with or without LR-aided
are always more computationally efficient than their ZF counterparts, almost without performance degradation.

B. Discussions on Some Extensions

1) LR-Aided SD with Tree Pruning: The tree pruning
(TP) methods for conventional SD can also be utilized in
the LR-aided SD to achieve near-ML performance while the
complexity can be further reduced. For example, when the
probabilistic tree pruning introduced in [11] is used in the primary search of the LR-aided SD, and the improved increasing
radius method presented in [12] is used in the further search,
the simulation results are showed in table I.
2) Fixed-Complicity LR-Aided SD: In the LR-aided fixedcomplexity tree searching MIMO detection proposed in [8], if
the proposition presented in this letter is utilized instead of the
hard limiting method in its ML test step, the out-of-bound candidates will be deleted, and the corresponding computations of
the tests saved, thus the computational costs of symbol detection will be reduced significantly, as the simulation results in
table II show. The complexity statistical method in table II
is slightly different from that in [8], noting that the computational costs of the LR and the QR decomposition of the channel
matrix should not be simply added to the detection complexity
statistics, since they are executed only when the channel matrix
changes.
On the other hand, in the MMSE-SD-LR detector with tree
pruning, if a complexity limitation is set for an acceptable
performance degradation, it then becomes a fixed-complexity
(complexity constraint) SD detector with more efficient performance/complexity tradeoff.

R EFERENCES
[1] H. Yao and G. Wornell, Lattice-reduction-aided detectors for MIMO
communication systems, in Proc. IEEE Conf. Global Commun., Nov.
2002, pp. 424428.
[2] Y. H. Gan, C. Ling, and H. M. Mow, Complex lattice reduction algorithm for low-complexity full-diversity MIMO detection, IEEE Trans.
Signal Process., vol. 57, no. 7, pp. 27012710, Jul. 2009.
[3] D. Wbben, D. Seethaler, J. Jaldn, and G. Marz, Lattice reduction:
A survey with applications in wireless communications, IEEE Signal
Process. Mag., vol. 28, no. 3, pp. 7091, May 2011.
[4] U. Fincke and M. Pohst, Improved methods for calculating vectors of
short length in a lattice, including a complexity analysis, Math. Comput.,
vol. 44, pp. 463471, Apr. 1985.
[5] B. Hassibi and H. Vikalo, On the expected complexity of sphere
decoding I. Theory, IEEE Trans. Signal Process., vol. 53, no. 8,
pp. 28062818, Aug. 2005.
[6] M. O. Damen, H. E. Gamal, and G. Caire, On maximum-likelihood
detection and the search for the closest lattice point, IEEE Trans. Inf.
Theory, vol. 49, no. 10, pp. 23892401, Oct. 2003.
[7] A. K. Singh, P. Elia, and J. Jaldn, Achieving a vanishing SNR gap to
exact lattice decoding at a subexponential complexity, IEEE Trans. Inf.
Theory, vol. 58, no. 6, pp. 36923707, Jun. 2012.
[8] H. Kim et al., Near-ML MIMO detection algorithm with LR-aided fixedcomplexity tree searching, IEEE Comm. Lett., vol. 18, no. 12, pp. 2221
2224. Dec. 2014.
[9] G. D. Golden et al., Detection algorithm and initial laboratory results
using V-BLAST space-time communication architecture, Electron. Lett.,
vol. 35, no. 1, pp. 1416, Jan. 1999.
[10] J. Conway and N. Sloane, Sphere Packings, Lattices and Graphs. Berlin,
Germany: Springer-Verlag, 1993.
[11] B. Shim and I. Kang, On further reduction of complexity in tree pruning
based sphere search, IEEE Trans. Comm., vol. 58, no. 2, pp. 417422,
Feb. 2010.
[12] W. Zhao and G. B. Giannakis, Sphere decoding algorithms with
improved radius search, IEEE Trans. Comm., vol. 53, no. 7, pp. 1104
1109, Jul. 2005.
[13] S-L. Shieh et al., Low-complexity soft-output sphere decoding with
modified repeated tree search strategy, IEEE Comm. Lett., vol. 17, no. 1,
pp. 5154. Jan. 2013.