Electronic copy available at: http://ssrn.

com/abstract=1933936
A
All
N
ass
For
A com
In pa
l the work co
No substantia
sessments for
appropria
recast
hig
mpari
artial fulfi
M
ontained with
al part(s) of t
r accredited c
ate reduction
ting t
h-fre
ison o
Su
Ol
Su
Professor
llment of
MSc in Fi
hin is my ow
guidel
he work subm
courses of st
n in the mark
Septe

the FT
equen
of rea


ubmitted b
leg Komar
upervised
r Valentin
the requir
inance & E
wn unaided ef
lines on plag
mitted here h
tudy, and I ac
I might othe
ember 22,
TSE
ncy da
alized
by
rov
by
na Corradi
rements fo
Economic
ffort and con
giarism.
has also been
cknowledge
erwise have r
2011
100 w
ata:
d mea
or the degr
s
nforms with t
n submitted b
that if this h
received will

with
asures
ree of
the Universit
by me in oth
has been done
l be made.
s
tys
her
e an
Electronic copy available at: http://ssrn.com/abstract=1933936
Forecasting the FTSE 100 with high-frequency data:
A comparison of realized measures
Oleg Komarov

Warwick Business School, Claycroft 2 077, Coventry Warwickshire, CV4 7ES, UK.
E-mail: oleg.komarov.10@mail.wbs.ac.uk
Last update: September 22, 2011
Summary In this study I apply the recent advances in volatility estimation and forecasting to
the price series of the FTSE 100, retrieved from the TRTH database. I use the recently introduced
threshold bipower variation and the realized semivariance together with the usual realized variance
and the simple bipower variation. I partially conrm with one-day-ahead in-sample forecasts the
general ndings. Realized variance is easily outperformed by bipower variation and its threshold
integrated version but only the realized semivariance consistently highlights the role of jumps.
Moreover, I systematically combine the estimators with: the pre-averaging heuristic, two sampling
spaces (calendar and business time) and several sampling frequencies (1 sec, 5 sec, 1 min, 5 min,
1 hrs and 1 day). I nd that the estimators with higher forecasting power were pre-averaged in
business time at 1 min intervals. Finally, the role of jumps should be interpreted in light of the
general trends.
Keywords: Realized variance, Volatility forecasting, Forecast comparison, FTSE, TRTH
1. INTRODUCTION
In the past decade the technological evolution of nancial markets has moved towards ever
more rapid execution of trades. Nowadays, the speed at which bid-ask quotes are matched in
the electronic books and prices are recorded has increased to the extent that even millisecond
granularity is not enough to ensure uniqueness of the timestamp. Hence, such copious and
complete availability of high-frequency data (HFD) has strongly encouraged the development of
new non-parametric methods for the estimation, modeling and forecasting of log-price variability.
The new estimation methods take advantage of intraday data to compute realized quantities
(better known as realized measures, RM) over a longer horizon, e.g. a day. The objective of these
measures, depending on the underlying assumptions of the data generating process (DGP), is to
estimate the quadratic variation (QV) or its continuous component, the integrated volatility (IV
t
).
The popularity of realized measures lies in their ease of implementation (ex-post, volatility can be
The whole MATLAB code can be found on
http://www.mathworks.co.uk/matlabcentral/fileexchange/32922
2 Oleg Komarov
treated as observed), in their improved accuracy and increased forecasting power with respect to,
e.g. quantities based on parametric models such as GARCH, ex-post squared returns or volatilities
implied in option prices. For instance, Andersen et al. (2003), Becker and Clements (2008), and
Andersen et al. (2007) nd that forecasts based on realized measures respectively dominate those
based on GARCH, VIX, and longer-horizon returns.
The best known and rst measure to be introduced by Andersen et al. (2001) is the realized
variance or volatility (RV
t
) which simply cumulates frequently sampled intraday squared returns.
In a purely continuous-time diffusive world and under ideally innitesimal sampling frequency,
the RV
t
consistently estimates the QV which coincides, in this context, with the IV
t
(integral of
instantaneous volatility, see Section 2). However, observed price series are discrete in time and in
increments, i.e. there is only a nite amount of observations per day and the price can only change
by a multiple of a tick.
1
Another feature that introduces a signicant challenge to estimation are
the relevant shifts, i.e. jumps, which occur in the series at random arrival times.
2
Therefore, in
a more realistic scenario the DGP cannot be simply diffusive but has to take the discontinuous
component into account. Then, the QV of such a process will aggregate both sources of risk. See
Back (1991) for an economic perspective on special semi-martingales and the interpretation of
jumps as indicators of idiosyncratic ows of information. For these specic reasons, the realized
volatility is not the optimal choice for the estimation of the sole continuous component implicit in
the QV.
Over the years more sophisticated estimators have been proposed in order to better describe
the latent log-price processes and in order to deal with market frictions, usually referred to as mi-
crostructure noise. Barndorff-Nielsen and Shephard (2007) give a well-rounded general overview.
To deal with the former case, Barndorff-Nielsen and Shephard (2006) and Barndorff-Nielsen et al.
(2005) have shown a way to decompose the QV in a practical implementation. They introduced
the so-called realized multipower variation (BPV
t
), a more general class of realized measures
which enables to consistently estimate the integrated volatility even in the presence of jumps in
the series. On one hand, this approach has proven to be empirically valid for testing, detecting
and estimating the jump component in the data, but on the other hand it has provided contrasting
results in forecasting applications, e.g. Andersen et al. (2007) nd that the discontinuous compo-
nent bears no predicting power even though signicant jumps are often followed by increased
trading activity and dispersion in prices. Corsi et al. (2010) attribute this puzzling effect to the
small-sample bias affecting BPV
t
and derive a corrected estimator which integrates the realized
bipower variation with a threshold function (TBPV
t
). A different justication for the puzzle is
given by Barndorff-Nielsen et al. (2010). They point out that the derivation of the discontinuous
component based on measures of multipower variation does not discriminate among jumps of
opposite sign, i.e. both negative and positive shifts are summed up together and their effect on
1
A tick is the minimum movement by which the price of a security, option, or index changes. The size of the tick
depends on market regulations.
2
I will use the terms "jump", "discontinuity" and "shift" interchangeably.
Forecasting the FTSE 100 with high-frequency data 3
volatility forecasts is neutralized. Therefore, they decompose the realized volatility into two
complementary components, the positive and negative realized semivariances (RSV
t
). Each of
them captures the dispersion due to either the positive or negative returns (an empirical analysis
with supporting results can be found in Patton and Sheppard, 2011).
Another aspect of realized measures estimation which has received much attention is microstruc-
ture noise, a structural feature of nancial markets which pollutes the latent price process. Several
authors have proposed elaborated estimators specically built to limit the impact of this type of
noise, whereas others have adopted broader methods which are easily extendable to a variety of
measures. I specically consider the implementation of some of the solutions belonging to the
latter case. For instance, I use the pre-averaging heuristic of At-Sahalia et al. (2005) to deal with
discretization of prices. Also, against the effects of bid-ask bounce and irregular trading I sample
the data at different frequencies both in calendar and business time.
3
In this work I propose an empirical analysis of the predictive power of several realized measures
in one-period-ahead in-sample volatility forecasts of the FTSE 100. My objective is to apply
the recent advancement in volatility estimation: I assume that jumps characterize the log-price
process and I include the RSV
t
and TBPV
t
measures along with the well-known RV
t
and BPV
t
.
Alternative methods, e.g. out-of-sample or multiple-periods forecasts, are out of the scope of the
investigation. Moreover, I nd that the appealing method for direct ranking of realized estimators
used by Patton (2011a) is not applicable in my case due to the relevance of the discontinuous
component in the QV of the FTSE 100.
4
My contribution lies in the treatment of a relatively new
dataset, the Thomson Reuters Tick History (TRTH), and the systematic application of 4 realized
measures declined for: two sampling spaces, several sampling frequencies, with and without
pre-averaging, for a total of 56 estimators (similar in fashion to the work of Patton and Sheppard,
2009).
5
The remainder of the paper unfolds as follows. Section 2 summarizes the methodology of the
realized measures, the subsampling heuristic and the forecasting models. A presentation of the
dataset along with the cleaning procedure is given in Section 3. The empirical analysis is carried
out in Section 4 and Section 5 concludes.
3
For a discussion of the optimal sampling space see Oomen (2006), and Grifn and Oomen (2008). Also, a formal
selection method of the optimal frequency and an informal one with signature plots can be found respectively in Bandi
and Russell (2008) and Andersen et al. (2000).
4
The average contributions of jumps to the QV of the FTSE 100 are available upon request.
5
Data supplied by Securities Industry Research Centre of Asia-Pacic (SIRCA) on behalf of Thomson Reuters Tick
History.
http://sirca.org.au/
4 Oleg Komarov
2. THE GENERAL FRAMEWORK
2.1. Realized measures
Let the log-price p
t
of an economic variable, e.g. an index, satisfy the following assumption
ASSUMPTION 2.1. The data generating process (DGP) for {p
t
, t 0} is a stochastic process
which consists of a continuous-time component plus a pure jump component
p
t
=
_
t
0

s
ds +
_
t
0

s
dW
s
+ J
t
, (2.1)
where
s
is a locally bound and predictable drift,
s
is a diffusive cdlg process, W
s
is a
standard Brownian motion and J
t
is a nite-activity jump process
J
t

0<st
p
s
,
where p
s
= p
s
p
s
is the size of the jump.
The latent volatility of the process in Eq. (2.1) is not directly observable but we can take the
ex-post variation into consideration by means of the quadratic variation (QV)
[ p ]
[2]
t
=
_
t
0

2
s
ds +

0<st
(p
s
)
2
. (2.2)
Note that in the absence of jumps the QV is equal to its continuous component, also termed
integrated volatility, IV
t

_
t
0

2
s
ds.
Let percentage returns be expressed as r
t
(p
t
p
t
) 100, the standard choice for estimating
[ p ]
[2]
t
is the realized volatility introduced by Andersen et al. (2001)
RV
()
t

1/

j=1
r
2
t1+j
, (2.3)
where 1/ is assumed to be a positive integer indicating the actual number of intervals in which
the day is parted and is the sampling frequency.
6
It has been shown that RV
t
converges in
probability to [ p ]
[2]
t
as 0, i.e. as the number of sampled intraday observations goes to innity.
For clarity of exposition I will henceforth suppress the day referencing subscript t in the RHS of
the equations.
Ideally, we would like to get the best estimate of the IV
t
and leave out the discontinuous
component, but the QV unavoidably aggregates both types of risk in the presence of jumps in the
DGP. Barndorff-Nielsen et al. (2005) and Barndorff-Nielsen and Shephard (2006) have shown
how to disentangle the continuous component and proposed a broader class of realized measures
6
In practice, the length of a trading day is variable and a common approach is to x the number of intervals which
correspond to a given sampling frequency on average.
Forecasting the FTSE 100 with high-frequency data 5
based on bipower variation which allows to estimate
_
t
0

2
s
ds robustly to jumps
BPV
()
t

1/

j=2
|r
(j1)
| |r
j
|, (2.4)
and
BPV
()
t
p
[ p ]
[1,1]
t
=
2
1
_
t
0

2
s
ds, (2.5)
implying that
2
1
BPV
t
= IV
t
with
1

_
2/ 0.79788 and Z is distributed as a standard
normal.
7
Although very popular, the BPV
t
is affected by small sample (upward) bias in the presence of
discontinuities. The intuition is simple, let us suppose that a given return contains a jump, then for
> 0 its effect will not vanish when multiplied by the preceding and following absolute return,
and the estimator will not converge to the IV
t
. To avoid this outcome, Corsi et al. (2010) propose
the realized threshold bipower variation which substitutes excessive returns with zeros
TBPV
()
t

1/

j=2
|r
(j1)
| |r
j
| 1
{r
2
(j1)

(j1)
r
2
j

j
}
, (2.6)
where 1
{}
is the indicator function and
j
is a strictly positive stochastic threshold function
satisfying certain conditions

j
= c
2

V
j
, (2.7)
where c
2

is a threshold calibrating constant and

V
j
is an iterative non-parametric lter used to
estimate the local variance (see Eq. A.5). The convergence in probability of the TBPV
t
follows
the behavior and the implication outlined by Eq. (2.5).
Both BPV
t
and TBPV
t
feasibly estimate the
_
t
0

2
s
ds even in the presence of jumps. However,
the evidence of the impact of discontinuities on volatility forecasts is contrasting. Andersen et al.
(2007) nd that they do not contribute to future volatility while Corsi et al. (2010) document an
expected positive inuence and attributes the puzzle to the small sample bias of BPV
t
. One of
the explanations to the puzzle is that positive jumps are averaged out by negative ones, therefore
Barndorff-Nielsen et al. (2010) further decompose the RV
t
in two complementary components to
capture and separate the sign effect of returns
RSV
()
t

1/

j=1
r
2
j
1
{r
2
j
0}
, (2.8a)
RSV
+()
t

1/

j=1
r
2
j
1
{r
2
j
0}
, (2.8b)
7
From the asymptotic results for multi power variation processes
p
2
p/2

p
2
+
1
2

1
2

= E[|Z|
p
] where
Z is a standard normal random variable.
6 Oleg Komarov
and
RSV
()
t
p

1
2
_
t
0

2
s
ds +

0<st
(p
s
)
2
1
{r
2
j
0}
, (2.9a)
RSV
+()
t
p

1
2
_
t
0

2
s
ds +

0<st
(p
s
)
2
1
{r
2
j
0}
, (2.9b)
implying that the decomposition RV
t
= RSV

t
+ RSV
+
t
holds for any .
2.2. Sampling and pre-averaging
The objective of the realized quantities is to estimate the QV or specically its continuous
component, the IV
t
. Although it is known that the higher the frequency of the data the closer is
the approximation of the practical counterpart, we usually compute the realized measure with n
equispaced intraday returns taken from the observed price series. Thus, not every observation is
considered but just a sample whose size depends on , the sampling frequency. This procedure
is necessary because the unobserved price process is increasingly contaminated by the so-called
microstructure noise as 0. Typical features of the real price series that induce this type of
noise are discreteness of the increments, bid/ask bounce, irregular trading etc. The scope of my
investigation is not devoted to microstructure issues of high frequency data but nevertheless I
am not ignoring their existence. Sampling methods and frequencies are chosen to consider some
feasible corrections.
Firstly, I sample returns both in calendar time (CTS) and in business time (BTS, also known
as tick time or trade time). The former is self-explanatory, a grid of evenly spaced intraday
times is constructed and the price with the closest timestamp to the gridpoint is chosen using
nearest interpolation.
8
The latter is simply achieved by skipping every x trades. To clarify, let us
suppose that the trading day starts at 8:00am and ends at 4:30pm. I can choose a , e.g. 5 min,
and count how many gridpoints occur in the range at the given frequency, in my example 102.
Then, the rst and the last price of the day are automatically included in the sample and the
remaining 101 prices are chosen to be evenly spaced among the two extremes by skipping every
x = (total daily trades)/(102 + 1) observations. Oomen (2006) and Grifn and Oomen (2008)
show that BTS is superior to the simple CTS since the evenly spaced grid in event time is variably
spaced in calendar time. Therefore, the noise associated with irregular trading is reduced because
higher periods of activity are sampled more frequently and vice versa in the opposite case.
Secondly, alongside with the standard implementation of the estimators, I also pre-average in
the sense of At-Sahalia et al. (2005).
9
This approach has been shown in Andersen et al. (2011)
to perform at par with other more sophisticated estimators specically designed to mitigate
8
Nearest interpolation can be summarized in the following way: rst I pinpoint the two observations which immediately
precede and follow a given gridpoint, then I choose whichever is the closest in absolute terms. The point is interpolated
(carried over) even if it is located farther apart than other surrounding gridpoints.
9
I will use the terms "subsample" and "pre-average" interchangeably.
Forecasting the FTSE 100 with high-frequency data 7
microstructure noise. Formally, let RM be any of the estimators in Subsection 2.1, I will denote
its subsampled version by appending the sufx "-ss"
RMss
()
t
=
1
N
N1

j=0
RM
t+j/(N1)
, (2.10)
where N is the number of intermediate estimators. This approach reduces the estimated variation
and counters the effect of discretization in prices. Even so, pre-averaging is feasible when the
sampling frequency is sufciently low, i.e. when the number of observations between each
gridpoint is at least as large as N 1. The main difference between the plain RM and the
intermediate estimators that form the pre-averaged version, lies in the starting point of the
sampling grid. In other words, the rst intermediate estimator in Eq. (2.10) is the plain realized
measure itself while the second one is computed by shifting onwards the previously used sampling
grid by a fraction of the sampling interval ( /N ). The latter step is repeated for the remaining
addends. In the end, I have N realised measures of the same type which I average to obtain the
subsampled version. In the empirical analysis I always use N = 10.
2.3. The forecasting model
For notation purposes consider a generic realized measure RM (or one of the quantities in Eqs. 2.12
and 2.13), where the superscript indicating the frequency is suppressed for clarity. I dene the
h-lagged average of the estimator as
RM
h

1
h
h

j=1
RM
tj
.
A simple example can clarify, suppose h = 5, then RM
5
is the average of the estimators computed
for the past ve days. Note that RM
0
= RM
t
(todays quantity).
The in-sample forecasts are based on the standard HAR model of Corsi (2009) which explains
the current daily realized volatility with a linear combination of RV
h
at different, usually longer,
horizons h. I also use the extended HAR-CJ proposed by Andersen et al. (2007) to include the
jump component estimated via BPV
t
and its adapted HAR-TCJ version which Corsi et al. (2010)
elaborate for their TBPV
t
. Finally, as in Patton and Sheppard (2011) the HAR-CJ is designed
to capture the sign effect of jumps with the help of the RSV
t
through a linear combination of
8 Oleg Komarov
continuous, discontinuous and lagged components. In total I have 4 forecasting equations
HAR : RV
t
=
0
+
d
RV
1
+
w
RV
5
+
m
RV
22
+
t
, (2.11a)
HAR-CJ : RV
t
=
0
+
d
C
1
+
w
C
5
+
m
C
22
+
dJ
J
1
+
wJ
J
5
+
mJ
J
22
+
t
, (2.11b)
HAR-TCJ : RV
t
=
0
+
d
TC
1
+
w
TC
5
+
m
TC
22
+
dJ
TJ
1
+
wJ
TJ
5
+
mJ
TJ
22
+
t
, (2.11c)
HAR-CJ : RV
t
=
0
+
dJ
J
1
+
d

2
1
BPV
1
+
w
RV
5
+
m
RV
22
+
t
. (2.11d)
The respective estimated daily jump components at any time t are dened by

J
t
I
{Z
t
>

}
max{0, RV
t

2
1
BPV
t
}, (2.12a)

TJ
t
I
{TZ
t
>

}
max{0, RV
t

2
1
TBPV
t
}, (2.12b)

J
t
RSV
+
t
RSV

t
=

0<st
(p
s
)
2
1
{r
2
j
0}

0<st
(p
s
)
2
1
{r
2
j
0}
, (2.12c)
where Z
t
and TZ
t
are given in the Appendix A by Eq. (A.1) and

is the cumulative distribution

function of a standard normal at the condence level (throughout the empirical analysis I
always use = 99.9%).
10
The rst two expressions capture signicant jumps by setting a certain
threshold value through the indicator function. The last one follows directly from Eq. (2.9) and is
termed in Patton and Sheppard (2011) as the "signed jump variation". Since we are dealing with
estimated quantities, to ensure that the jump and the continuous component sum up to the realized
variance (estimated quadratic variation), the latter is dened residually

C
t
RV
t


J
t
, (2.13a)

TC
t
RV
t


TJ
t
. (2.13b)
The continuous component of the HAR-CJ model is estimated with the realized bipower
variation and it does not have to add up to the realized variance with

J
t
.
To evaluate the forecasting performance of the models in Eq. (2.11) I use: (a) the R
2
from the
Mincer - Zarnowitz regression, (b) the Mean Squared Error (MSE) and (c) the Quasi-Likelihood
10
From Eqs. 2.2 through 2.5 follows that RV
t

2
1
BPV
t
p

0<st
(p
s
)
2
but the LHS is not ensured to
be positive because the quantities are estimated, thus the max{0, x} for the rst expression in Eq. (2.6). A similar
consideration applies to equation containing TBPV
t
(the second one).
Forecasting the FTSE 100 with high-frequency data 9
(QLIKE) loss functions
R
2
Corr(RV
t
,

RV
t
)
2
, (2.14a)
MSE
1
T
T

t=1
(RV
t


RV
t
)
2
, (2.14b)
QLIKE ln(

RV
t
) +
RV
t

RV
t
, (2.14c)
where

RV
t
is the one-day-ahead forecasted realized variance.
11
Better forecast will have higher
R
2
and MSE close to zero. A note of caution is needed for the QLIKE measure which can be
negative, but like the mean squared error it approaches zero as the forecast improves.
3. THE DATASET
3.1. The TRTH dataset
The empirical analysis is based on the FTSE 100 price index (generally referred to as trade data)
retrieved from the TRTH database.
12
The series covers the period from the 2
nd
of January 1996 to
the 12
th
of July 2011 for a total of 3,892 days (almost 31 million timestamps).
The upper pane of Figure 1 presents the daily counts of intraday observations and outlines two
main features of the dataset.
13
First, the series can be divided into three main periods
14
:
I. January 2, 1996 to February 20, 1998.
II. February 23, 1998 to November 27, 2009.
III. November 30, 2009 to July 12, 2011.
Second, the rst two periods mostly exhibit constant daily counts as opposed to the third one
whose number of intraday observations varies signicantly over time. Thus, in order to highlight
the same behavior at a ner level, the bottom pane (Figure 1) breaks the counts of three specic
days taken from the three main periods into half-hours. As expected, the only day which takes the
canonical U-shape form belongs to the last period while the other two remain at. At this point it
is trivial to infer that up until the 20
th
of February 1998 the series was pre-averaged at 1 minute
intervals while for the next period the frequency was set to 15 seconds. The series is tick-by-tick
from November 30, 2009 to July 12, 2011.
15
11
The MSE and QLIKE loss functions are robust in the sense of Patton (2011b) while rankings based on R
2
depend on
the distribution of the proxy in place for the QV or IV.
12
See Note 5.
13
According to rule R1, 1 does not include timestamps outside the trading hours window (8:00am to 4:30pm).
14
Two minor breaks occur during the second period but they are marginal with respect to the difference in counts which
distinguishes the main ones.
15
A tick-by-tick series is comprehensive of all price changes that occurred in a given interval. The frequency of the data
is not xed and the interval is likely to vary between each subsequent trade.
10 Oleg Komarov
# trades: 480
date: 20 Feb 98
# trades: 1 920
date: 23 Feb 98
# trades: 2 039
date: 27 Nov 09
# trades: 46 762
date: 30 Nov 09
Number of intraday observations
Dec 97 Nov 99 Oct 01 Sep 03 Sep 05 Aug 07 Jul 09 Jul 11
10
1
10
2
10
3
10
4
10
5
Sep 05, 1997
Apr 15, 2002
May 05, 2010
Halfhour breakdown of intraday counts
08:30 10:22 12:15 14:22 16:30
Figure 1. The top pane plots the number of intraday observations for each day. Days circled in red precede or follow the
breaks in the counts. The bottom pane shows the half-hour breakdown for three specic days belonging to the three main
period observed in the upper pane.
3.2. The cleaning procedure
While the availability of high-frequency data (HFD) on one hand has given the opportunity to
develop new approaches to volatility estimation, providing more accurate and rened estimators,
on the other hand it imposes additional steps on data management (for a thorough description of
the tasks see Brownlees and Gallo, 2006). The necessity of these steps is driven by the sensibility
of RMs to outliers. Some authors described some relevant aspects in HFD management: e.g.
Falkenberry (2002) gives a good overview of the general issues that affect HFD, why they exist
and how to deal with them; Brownlees and Gallo (2006) outline the effects of bad data on nancial
time series analysis and present a lter rule whose similar versions have been widely used since;
Barndorff-Nielsen et al. (2009) provide a step-by-step cleaning procedure and discuss their lter
rules, comparing them to similar solutions adopted in the literature.
Forecasting the FTSE 100 with high-frequency data 11
Dec 97 Nov 99 Oct 01 Oct 03 Sep 05 Aug 07 Aug 09 Jul 11
00:00
08:00
16:30
23:59
Figure 2. The red lines delimit the normal trading session window. The green crosses are data corresponding to out-of-
range timestamps.
The following list summarizes the selection rules I applied to the dataset
16
:
R1. Out-of-range timestamps
R2. Late start days
R3. Insufcient intraday data
R4. Multiple prices per timestamp
R5. Outliers
Figure 2 plots in green a total of 7,525 timestamps outside the 8:00 16:30 window, the normal
trading session of the London Stock Exchange (denoted with horizontal red lines).
17
I drop those
observations in accordance with rule R1. Nevertheless, it can be observed that the majority of the
datapoints lie within 5 minutes off the end of the accepted interval and correspond to the price of
the closing auction.
18
The remaining abnormal timestamps do not represent changes in the series
but merely keep track of corrections to the volume of trades in the constituents of the index.
Rule R2 completely removes the days whose rst timestamp occurred after 9:30am. The impact
of this exclusion is limited to 10 days and the rationale lies in the concentrated activity at the
beginning of a trading day. In fact, the number of transactions that usually occur during the
rst hour is very high compared to the rest of the day and leaving that moment out would most
probably deliver an unnecessarily biased estimate of the true daily volatility.
16
In addition to the listed rules, I checked for null prices and found none.
17
The ofcial opening time of the London Stock Exchange has been altered several times. Until July, 20 1998 the opening
time was 8:30am, then it changed to 9:00am and since September, 20 1999 it is at 8:00am.
18
The price formation process of the closing auction differs substantially from the continuous trading and usually the
resulting observation is set too far apart from the previous trade in terms of price change and time interval. For these
reasons, it is not desirable to include such datapoints in the dataset.
12 Oleg Komarov
The last hour of trading is a similarly active period but early closings are usual at specied
dates of the calendar year and a trivial exclusion rule as R2 would not discriminate between
legit closings and breaks in the data feed. Thus, I implement rule R3 which also proxies for the
minimum number that a day should have in order to be included in the analysis. I did not impose a
xed minimum because such a method is not consistent with the high variability of daily volumes
of the 3
rd
period. Instead, I use a minimum range of 4 hours, i.e. I exclude days with the last
timestamp occurring in less than four hours from the rst one. I discard 7 days which potentially
have a break in the feed or simply do not meet the required minimum amount of observations for
a feasible estimation of the realized quantities.
Rule R4 highlights a very common issue of HFD, multiple prices/entries per timestamp. It
occurs when the precision of the recorded time of transaction is not sufciently high to ensure
uniqueness, and it happens even with an approximation to the millisecond. In my case, only the
last period is affected. I nd 2,890,614 timestamps with multiple entries and I consolidate them
into median prices (as suggested in Brownlees and Gallo, 2006).
Falkenberry (2002) shows that even under fully automated trading, prices may be misreported
and outliers can appear in the series. Unfortunately, there is no easy way to detect such occurrences,
unless they are system-generated and appropriately coded. R5 tries, with a certain degree of
discretion, to identify and eliminate potential outliers. I use a lter which is both a modication
of the original rule formulated by Brownlees and Gallo (2006) and an implementation of some
principles by Barndorff-Nielsen et al. (2009). So, let { p
j
} be an intraday time series of prices,
then an observation is considered bad data if
|p
j
p
j
(k
L
)| > max{4MD
j
(k), n}
|p
j
p
j
(k
R
)| > max{4MD
j
(k), n},
(3.15)
where k is the bandwidth, and p
j
(k
L
) and p
j
(k
R
) denote the sample medians of the k/2 observa-
tions respectively before and after j (also simply the left and right side of the series) and MD
j
(k)
is the mean absolute deviation from the median of the whole neighborhood.
19
Finally, is the
intraday mean of the absolute price changes, n is its multiplier and together they represent the
granularity parameter. The main innovation of this rule lies in the separate comparison of the i
th
trade against the left and right neighbors while the measure of dispersion is calculated on the
whole bunch of k trades. This approach is specically designed to avoid detecting jumps as false
outliers.
20
Usually, the RHS is an additive expression and the granularity parameter is set ad hoc
so that at portions of the series will not result in a null threshold. I avoid this approach for two
reasons: the series is too long to choose a single value for n and it is impractical and discretionary
to choose several ones to accommodate for different periods; since the scope of the granularity
parameter is to ensure a positive threshold I make this behavior explicit and independent of the
19
Another principle is to use the -trimmed sample mean as measure of central tendency in place of the median. However,
this method is not feasible for long horizons and requires recurrent calibration.
20
See Falkenberry (2002) for an example of false detection of a jump.
Forecasting the FTSE 100 with high-frequency data 13
Table 1. Total number of outliers detected with Eq. (3.15) and daily averages. The statistics are grouped by period and are
reported along with the coefcient k for the number of neighbors and the multiplier n for the granularity threshold.
Jan 2, 1996 Feb 20, 1998 Feb 23, 1998 Nov 27, 2009 Nov 30, 2009 Jul 12, 2011
k = 10 n = 10 k = 20 n = 10 k = 50 k = 5
Total outliers 47 1967 4555
Days 525 2945 406
Daily average 0.0895 0.6679 11.2192
mean absolute deviation, i.e. I use it as a complementary threshold. The lter is re-initialized
every day to circumvent scrubbing off initial observations due to the possible overnight gap.
Table 1 reports, grouped by period, the total number of outliers and the daily averages together
with the bandwidth k and the multiplier n. The parameters were chosen to ensure that the
neighborhood of observations would not stretch too far away from the i
th
, and the threshold to be
reasonable. The number of trades detected as outliers reects the parsimonious action of the lter.
4. EMPIRICAL ANALYSIS
The analysis is carried out in two steps. First, I compute the measures in Subsection 2.1 and then
I forecast the realized variance with the models in Eq. (2.11). The accuracy of the forecasts is
assessed with the R
2
of Mincer-Zarnowitz regression and with the MSE and QLIKE loss functions
(see Eq. 2.14).
As outlined in Section 3 only the 3
rd
period can be sampled both in CTS and BTS, thus I
differentiate sampling methods and frequencies when considering the whole horizon and the
period from November 30, 2009 to July 12, 2011.
21
For the analysis on the whole series, I sample each of the 4 estimators every 1 and 5 min,
and the sole RV
t
also at intervals of 56.1 min (henceforth 1 hrs) and 1 day (taking rst and last
intraday prices) for a total of 12 combinations. I do not subsample as in Eq. (2.10) with 1, 5 min
frequencies since it is not feasible due to the lack of in-between observations, while intervals of
1 hrs and 1 day are not expected to improve upon such treatment due to the vanishing effect of
microstructure noise as frequency falls.
In the last period I use both CTS and BTS combining them with the subsampled versions of
the estimators. In general, each measure has been implemented with 1 and 5 min intervals, and
declined across the two sampling spaces and further combined with the subsampling heuristic.
22
I
obtain a total of 44 combinations. To distinguish between estimators sampled in business time and
calendar time I add respectively the subscripts bus and cal. I follow Patton and Sheppard (2009)
in the choice of the frequencies.
21
From January 2, 1996 to November 27, 2009 the price series is de facto already sampled in calendar time and switching
to tick time is impossible unless the two spaces coincide from the beginning.
22
The only exception is RV
t
which additionally has high and low frequency versions: 1 sec, 5 sec, 1 hrs and 1 day.
14 Oleg Komarov
FTSE 100 price index
Dec 97 Nov 99 Oct 01 Oct 03 Sep 05 Aug 07 Aug 09 Jul 11
3000
4000
5000
6000
7000
Annualized realized volatility
Dec 97 Nov 99 Oct 01 Oct 03 Sep 05 Aug 07 Aug 09 Jul 11
0
40
80
120
160
Figure 3. the upper pane plots the FTSE 100 index and the bottom pane shows the annualized deviation of the index
estimated with

RV
(5 min)
t
252.
4.1. General trends
Figure 3 outlines the trends in the FTSE 100 index and in its volatility through the RV
(5 min)
t
estimator. The price follows the well known pattern of the Russian crisis in August 1998, the
dot-com bubble peaking in March 2000 and the burst which drags until the end of 2002. Next
comes a steady growth due to the speculations in the American housing market which culminates
with the sub-prime crisis in the late 2007 and the Lehman Brothes bankruptcy in September
2008. Afterwards, the index recovers with a brief slowdown in 2010. Moderate-to-high volatility
usually comes with the bubbles and peaks in proximity of the mentioned critical events. It is worth
noticing that on October 20, 1997 the London Stock Exchange introduced the Stock Exchange
Electronic Trading System (SETS) which Chelley-Steeley (2005) describes as a structural break
in volatility. The sudden change is clearly visible in the plot and justies the second highest peak
together with the previous period of low variability.
Table 2 reports the summary statistics of the realized quantities calculated on the whole horizon.
A general effect of time scaling can be noted from the median, mean and standard deviation
which increase with the interval, regardless of the RM. In a cross-measure comparison the RV
t
has the highest rst and second central moments at any sampling frequency, which suggests
Forecasting the FTSE 100 with high-frequency data 15
Table 2. Summary statistics of the realized measures on the whole horizon (3982 days) January 2, 1996 to July 12, 2011.
The sampling interval is reported in parentheses. The realized threshold bipower variations as in Eq. (2.6) were calculated
assuming c

= 3.
Median Mean St. deviation Skewness Kurtosis Min Max
RV
(1 min)
0.503 0.925 2.419 21.896 742.259 0.002 98.1326
RV
(5 min)
0.594 1.131 2.795 19.488 596.139 0.005 105.2084
RV
(1 hrs)
0.602 1.269 3.231 22.530 818.855 0.007 135.5169
RV
(1 day)
0.310 1.276 3.380 9.698 153.547 0.000 77.8612
RSV
(1 min)
0.224 0.477 1.731 34.106 1530.446 0.001 84.8272
RSV
+ (1 min)
0.241 0.448 1.013 12.979 251.137 0.001 28.0938
RSV
(5 min)
0.264 0.587 1.947 29.335 1183.557 0.001 88.3241
RSV
+ (5 min)
0.277 0.544 1.165 10.402 159.984 0.003 25.3313
BPV
(1 min)
0.381 0.705 1.463 12.031 225.329 0.002 38.8383
BPV
(5 min)
0.491 0.925 2.022 12.870 252.570 0.005 51.4057
TBPV
(1 min)
0.217 0.449 0.931 10.000 160.202 0.002 21.0059
TBPV
(5 min)
0.320 0.634 1.183 7.711 91.882 0.001 20.9445
that the discontinuous component of the QV should be relevant, since the other estimators are
expected to be robust to jumps. For example, the RV
(1 min)
t
has respectively mean and standard
deviation of 0.925 and 2.419 against the values of the TBPV
(1 min)
t
, 0.449 and 0.932 (second
lowest and lowest). The TBPV
(1 min)
t
has also very low skewness, kurtosis and maximum and its
5 min version scores the absolute lowest values in the same statistics, which is indicative of the
ltering effect of the threshold. More interesting behavior can be observed in the distribution of
the positive and negative realized semivariances. While medians and means are very similar, the
difference in the standard deviations, e.g. RSV
(1 min)
t
with a value of 1.731 against the 1.013
of RSV
+ (1 min)
t
, shows that negative returns account for a bigger portion in the volatility of the
FTSE 100. If I consider for the same measures the even stronger difference in skewness, 32.106
versus 12.979, and kurtosis, 1530.446 against 251.137, it will not be too hazardous to consider
that excessive negative returns are jumps which trigger volatility.
Summary statistics for the third period are listed in Table 3. While the time scaling property
is still generally valid under comparisons of the same type of measure and sampling space, the
behavior of realized semivariances changes. Their means do not differ substantially and the
standard deviation of the negative semivariance is almost always larger than the positive one,
which supports the hypothesis that the distribution of negative returns tend to be wider. However,
the counterintuitive statistics come from the skewness and kurtosis. This time, the relationship
highlighted for the whole horizon seems reversed, i.e. the distribution of the negative realized
semivariance has lower 3
rd
and 4
th
moments than the positive one. This evidence can be justied
bywith positive jumps which are more likely to occur in a period with an overall increasing
trend as can be seen on Figure 3. Apart from these specic results it is interesting to note that
subsampling generally reduces the rst two moments (as already documented in previous studies),
with the exception of the TBPV
t
which remains mostly unaffected, and has a mixed effect on
16 Oleg Komarov
Table 3. Summary statistics of the realized measures on the last period (406 days) November 30, 2009 to July 12, 2011.
The sampling interval is reported in parentheses. The sufx "-ss" stands for subsampled/pre-averaged measure as in
Eq. (2.10) and with N = 10. The "cal" and "bus" subscripts indicate respectively sampling in calendar time and business
time. The realized threshold bipower variations as in Eq. (2.6) were calculated assuming c

= 3.
Median Mean St. deviation Skewness Kurtosis Min Max
RV
(1 sec)
cal
0.180 0.239 0.224 5.656 49.393 0.037 2.5688
RV
(5 sec)
cal
0.300 0.400 0.390 5.297 45.503 0.044 4.5719
RV
(1 min)
cal
0.646 0.911 0.937 4.548 32.909 0.048 8.8954
RVss
(1 min)
cal
0.524 0.678 0.646 4.996 40.174 0.034 7.1413
RV
(1 min)
bus
0.561 0.821 0.885 4.575 32.974 0.049 8.6864
RVss
(1 min)
bus
0.429 0.581 0.576 5.320 45.432 0.030 6.6851
RV
(5 min)
cal
0.679 0.954 0.903 3.823 25.071 0.016 8.3058
RVss
(5 min)
cal
0.541 0.715 0.663 4.892 41.372 0.037 7.7436
RV
(5 min)
bus
0.711 0.971 0.904 3.664 22.681 0.044 7.9245
RVss
(5 min)
bus
0.586 0.773 0.691 4.308 30.842 0.031 6.8761
RV
(1 hrs)
cal
0.691 1.021 1.395 6.726 67.953 0.021 17.3735
RV
(1 day)
cal
0.364 0.940 1.750 5.645 54.740 0.000 21.7119
RSV
(1 min)
cal
0.264 0.453 0.645 5.689 50.371 0.023 7.6640
RSV
+(1 min)
cal
0.317 0.458 0.516 6.483 73.191 0.018 7.1226
RSVss
(1 min)
cal
0.234 0.344 0.356 4.562 36.076 0.015 3.9595
RSVss
+(1 min)
cal
0.252 0.334 0.348 7.499 84.971 0.019 4.8619
RSV
(1 min)
bus
0.238 0.408 0.612 6.278 59.720 0.021 7.6375
RSV
+(1 min)
bus
0.276 0.414 0.468 5.647 55.501 0.028 5.9837
RSVss
(1 min)
bus
0.212 0.295 0.298 4.892 42.420 0.013 3.5500
RSVss
+(1 min)
bus
0.214 0.287 0.309 7.193 75.044 0.017 4.0258
RSV
(5 min)
cal
0.280 0.475 0.618 4.637 34.728 0.013 6.5681
RSV
+(5 min)
cal
0.338 0.478 0.487 4.008 28.564 0.003 5.0857
RSVss
(5 min)
cal
0.257 0.368 0.379 4.505 36.893 0.011 4.3484
RSVss
+(5 min)
cal
0.268 0.347 0.339 5.846 52.348 0.026 3.8730
RSV
(5 min)
bus
0.290 0.477 0.609 4.491 32.125 0.025 6.2372
RSV
+(5 min)
bus
0.336 0.494 0.506 4.532 37.272 0.019 5.8285
RSVss
(5 min)
bus
0.265 0.387 0.395 3.718 24.172 0.015 3.8886
RSVss
+(5 min)
bus
0.290 0.386 0.380 6.553 68.952 0.016 5.0704
BPV
(1 min)
cal
0.435 0.586 0.554 5.556 53.196 0.036 6.9746
BPVss
(1 min)
cal
0.416 0.557 0.540 5.579 52.083 0.032 6.6963
BPV
(1 min)
bus
0.446 0.582 0.542 5.372 48.138 0.040 6.5568
BPVss
(1 min)
bus
0.414 0.539 0.528 5.490 49.647 0.027 6.4188
BPV
(5 min)
cal
0.499 0.667 0.633 5.177 44.396 0.014 7.3399
BPVss
(5 min)
cal
0.479 0.631 0.593 5.453 52.308 0.020 7.4897
BPV
(5 min)
bus
0.527 0.690 0.679 5.480 46.392 0.049 7.6921
BPVss
(5 min)
bus
0.502 0.660 0.622 5.029 40.625 0.032 6.8720
TBPV
(1 min)
cal
0.297 0.388 0.400 5.446 49.094 0.015 4.8000
TBPVss
(1 min)
cal
0.293 0.390 0.401 5.222 43.567 0.014 4.4069
TBPV
(1 min)
bus
0.362 0.470 0.452 4.901 39.429 0.021 4.8008
TBPVss
(1 min)
bus
0.359 0.470 0.455 5.187 44.448 0.016 5.2038
TBPV
(5 min)
cal
0.360 0.478 0.446 4.373 32.828 0.014 4.4601
TBPVss
(5 min)
cal
0.385 0.497 0.479 5.408 50.050 0.007 5.8660
TBPV
(5 min)
bus
0.440 0.573 0.557 5.746 54.199 0.017 6.7531
TBPVss
(5 min)
bus
0.450 0.575 0.553 5.676 52.980 0.015 6.6035
Forecasting the FTSE 100 with high-frequency data 17
Table 4. OLS regression for one-day-ahead volatility forecasts for the FTSE 100 index on the last period November 30,
2009 to July 12, 2011 (384 observations). The interval indicates the frequency () of the realized measure. Reported in
parentheses are the t-statistics based on the Newey-West correction with 5 lags. The accuracy of forecasts is assessed with
the R
2
of Mincer-Zarnowitz regression, and through the MSE and QLIKE loss functions. The realized threshold bipower
variations as in Eq. (2.6) were calculated assuming c

= 3.
HAR : RV
t
=
0
+
d
RV
1
+
w
RV
5
+
m
RV
22
+
t
HAR-CJ : RV
t
=
0
+
d
C
1
+
w
C
5
+
m
C
22
+
dJ
J
1
+
wJ
J
5
+
mJ
J
22
+
t
HAR-TCJ : RV
t
=
0
+
d
TC
1
+
w
TC
5
+
m
TC
22
+
dJ
TJ
1
+
wJ
TJ
5
+
mJ
TJ
22
+
t
HAR-CJ : RV
t
=
0
+
dJ
J
1
+
d

2
1
BPV
1
+
w
RV
5
+
m
RV
22
+
t
HAR HAR-CJ HAR-TCJ HAR-CJ
1 min 5 min 1 hrs 1 day 1 min 5min 1 min 5min 1 min 5min

0
0.182 0.220 0.293 0.278 0.103 0.163 0.182 0.184 0.162 0.217
(3.417) (3.652) (3.084) (2.794) (1.413) (2.136) (4.260) (2.923) (3.234) (3.536)

d
0.150 0.138 0.104 -0.055 0.305 0.236 0.683 0.662 0.345 0.197
(3.391) (2.374) (1.353) (-1.208) (2.006) (2.203) (2.771) (5.262) (3.469) (2.066)

w
0.411 0.419 0.283 0.464 0.900 0.537 1.147 0.597 0.355 0.402
(2.671) (2.725) (2.016) (3.664) (1.415) (1.732) (1.218) (1.622) (2.853) (3.210)

m
0.245 0.250 0.384 0.375 0.070 0.332 -0.412 -0.014 0.207 0.243
(2.741) (2.893) (3.559) (4.099) (0.259) (2.644) (-0.912) (-0.075) (2.460) (2.805)

dJ
-0.036 -0.023 0.013 -0.012
(-0.372) (-0.331) (0.506) (-0.241)

wJ
-0.156 0.254 0.008 0.286
(-0.363) (1.293) (0.041) (2.247)

mJ
-0.157 -0.417 0.208 0.051
(-0.531) (-0.916) (1.671) (0.275)

J
-0.088 -0.125
(-1.885) (-2.298)
R
2
0.272 0.271 0.177 0.179 0.304 0.284 0.329 0.306 0.290 0.282
MSE 4.284 5.725 8.637 9.434 4.093 5.621 3.945 5.448 4.177 5.641
QLIKE 0.472 0.683 0.862 0.887 0.457 0.665 0.491 0.664 0.459 0.677
skewness and kurtosis. The effect of BTS is not of easy interpretation and it is specic for each
measure.
4.2. Results of forecasts
In order to avoid confusion, the forecasts were estimated with one-day-ahead OLS regressions,
thus any mention of frequency or interval, if not differently specied, is referred to the sampling
method of the realized measures.
Table 4 summarizes the forecasting results and their accuracy on the whole horizon. It is
noteworthy that the quantities in Eq. (2.14) consistently rank within the same model (with a slight
discrepancy for the HAR) and the ranking worsens as the sampling interval increases. Overall,
the worst model is the 1 day HAR and the best one is not univocally identied but both the
Mincer-Zarnowitz R
2
and the MSE point at the 1 min HAR-TCJ while the QLIKE settles down
on the HAR-CJ at the same frequency. Thus, high-frequency data seems to add value to the
forecasting power of realized measures, backing up the results obtained by Andersen et al. (2007).
18 Oleg Komarov
Table 5. OLS regression for one-day-ahead HAR volatility forecasts for the FTSE 100 index on the last period November
30, 2009 to July 12, 2011 (384 observations). The interval indicates the frequency () of the realized measure and the
"bus" header stands for business time sampling, if absent calendar time sampling is used. The subsampled/pre-averaged
RVs (N = 10) are used in the forecasts with the "ss" header. Reported in parentheses are the t-statistics based on the
Newey-West correction with 5 lags. The accuracy of forecasts is assessed with the R
2
of Mincer-Zarnowitz regression,
and through the MSE and QLIKE loss functions.
HAR : RV
t
=
0
+
d
RV
1
+
w
RV
5
+
m
RV
22
+
t
1 sec 5 sec 1 min 1 min 1 min 1 min 5 min 5 min 5 min 5 min 1 hrs 1 day
bus bus bus bus
ss ss ss ss

0
0.060 0.099 0.189 0.129 0.166 0.113 0.214 0.148 0.201 0.154 0.303 0.448
(3.184) (3.076) (2.998) (2.935) (2.871) (2.914) (2.810) (2.963) (2.828) (2.839) (2.739) (4.021)

d
0.306 0.342 0.419 0.567 0.426 0.563 0.338 0.487 0.417 0.545 0.414 -0.058
(2.049) (3.156) (1.907) (4.339) (1.976) (5.627) (2.245) (6.031) (2.632) (4.794) (2.665) (-0.799)

w
0.340 0.318 0.202 0.145 0.229 0.155 0.259 0.190 0.214 0.126 -0.023 0.455
(1.809) (2.180) (1.088) (1.187) (1.167) (1.494) (1.789) (1.830) (1.424) (1.269) (-0.197) (2.455)

m
0.106 0.098 0.178 0.106 0.151 0.095 0.185 0.123 0.168 0.136 0.322 0.132
(0.790) (0.821) (1.451) (1.021) (1.270) (0.908) (1.345) (1.015) (1.409) (1.198) (1.996) (0.677)
R
2
0.310 0.334 0.375 0.488 0.399 0.485 0.325 0.421 0.381 0.452 0.232 0.059
MSE 0.036 0.107 0.576 0.224 0.493 0.178 0.576 0.265 0.530 0.274 1.571 2.981
QLIKE -0.541 -0.026 0.793 0.479 0.668 0.312 0.860 0.543 0.873 0.634 0.937 0.881
However, the two best forecasts estimate the jump components as statistically insignicant.
While such a result is expected for the model containing the BPV
t
, it is indeed counterintuitive
for the HAR-TCJ. In fact, my ndings disagree with Corsi et al. (2010) but neither deny them
since I use an expanded model which additionally includes the lagged jump components at
the weekly and monthly level. In spite of this, the 5 min version of the same model exhibits a
positive weekly discontinuous component and attains the best forecast among those with 5 min
sampled RMs. A possible motivation could be adduced to the sampling frequency, i.e. the jump
component in Eq. (2.1) tends to vanish as 0, whereas at higher intervals, e.g. 5 min, the effect
of discontinuities becomes noticeable and signicant.
Moreover, it is interesting to note that the second best forecast according to the QLIKE is the
1 min HAR-CJ. Coherently with Patton and Sheppard (2011) the coefcient of the signed jump
variation (J) is statistically negative, i.e. whenever negative jumps dominate the day, the future
variance is expected to be higher; the effect becomes stronger at the 5 min frequency.
The empirical results for the period going from November 30, 2009 to July 12, 2011 are
summarized by Tables 58, each referring to separate models in the order outlined in Eq. (2.11). I
can notice a robust result, the best intra-model forecast always uses realized measures subsampled
in business time at 1 min intervals. The HAR model congures a minor exception where the MSE
performs better for realized volatilities sampled at higher frequencies, 1 and 5 sec. However, the
Forecasting the FTSE 100 with high-frequency data 19
Table 6. OLS regression for one-day-ahead HAR-CJ volatility forecasts for the FTSE 100 index on the last period
November 30, 2009 to July 12, 2011 (384 observations). The interval indicates the frequency () of the realized measure
and the "bus" header stands for business time sampling, if absent calendar time sampling is used. The subsampled/pre-
averaged RMs (N = 10) are used in the forecasts with the "ss" header. Reported in parentheses are the t-statistics based on
the Newey-West correction with 5 lags. The accuracy of forecasts is assessed with the R
2
of Mincer-Zarnowitz regression,
and through the MSE and QLIKE loss functions.
HAR-CJ : RV
t
=
0
+
d
C
1
+
w
C
5
+
m
C
22
+
dJ
J
1
+
wJ
J
5
+
mJ
J
22
+
t
1 min 1 min 1 min 1 min 5 min 5 min 5 min 5 min
bus bus bus bus
ss ss ss ss

0
-0.007 0.086 -0.153 0.050 0.181 0.143 0.179 0.135
(-0.093) (2.125) (-1.271) (1.166) (2.436) (2.937) (2.485) (2.584)

d
0.639 0.615 0.662 0.560 0.496 0.549 0.581 0.613
(1.941) (4.751) (2.297) (5.374) (3.482) (6.554) (3.503) (4.770)

w
0.848 0.422 0.614 0.428 0.219 0.199 0.130 0.126
(1.993) (3.112) (1.492) (3.522) (0.941) (1.903) (0.583) (0.974)

m
0.308 0.017 0.709 0.033 0.760 -0.020 0.674 0.264
(1.014) (0.116) (1.956) (0.313) (2.079) (-0.143) (2.150) (1.328)

dJ
0.067 -0.156 0.083 -0.688 0.064 -0.320 0.092 -0.039
(0.674) (-1.069) (0.814) (-3.134) (0.672) (-1.880) (1.232) (-0.194)

wJ
-0.795 -0.827 -0.660 -1.688 -0.329 0.182 -0.188 -0.097
(-2.136) (-2.017) (-1.655) (-2.364) (-1.445) (0.241) (-0.856) (-0.235)

mJ
0.287 0.886 -0.259 1.000 -0.538 1.947 -0.539 -0.133
(0.839) (1.433) (-0.649) (0.682) (-1.013) (2.075) (-1.077) (-0.180)
R
2
0.493 0.546 0.504 0.549 0.394 0.448 0.452 0.485
MSE 0.467 0.198 0.406 0.156 0.517 0.253 0.469 0.258
QLIKE 0.783 0.472 0.685 0.375 0.853 0.536 0.863 0.629
R
2
remains low compared to the 1, 5 min versions of the HAR. Also, although close to zero, the
QLIKE is negative and simple comparisons of values are not feasible.
23
Concentrating on the 1 min business-time subsampled versions, the list from best to worst
model, in terms of forecasting performance, reads: HAR-CJ, HAR-TCJ, HAR-CJ and HAR.
Proceeding in this order, the rst model conrms the ndings for the whole horizon on the signed
jump variation which is again statistically negative to conrm that days dominated by negative
jumps will likely lead to higher volatility in the future. Second, the forecast based on TBPV
t
does not assign predictive power to the discontinuous components (as opposed to the estimates
in Table 4). This result adds to the already existing puzzle raised by forecasts based on realized
bipower variation (see Andersen et al., 2007). The coefcients of the discontinuities at the daily
and weekly level are even negative in the third model, implying a reduction of future volatility if
jumps occurred in the past days. Nevertheless, I should stress that the effects described for the
HAR-CJ and the HAR-TCJ models should be interpreted in light of the circumstances, i.e. the late
bullish trend of the market. In fact, during this period, I do not expect negative jumps to dominate
23
For three models out of four the R
2
selects the forecast based on 1 min measures subsampled in calendar time rather
than in business time. Nevertheless, the difference in the explained variance is always smaller than 1%, therefore I prefer
to ignore this evidence especially considering that MSE and QLIKE agree on business time sampling.
20 Oleg Komarov
Table 7. OLS regression for one-day-ahead HAR-TCJ volatility forecasts for the FTSE 100 index on the last period
November 30, 2009 to July 12, 2011 (384 observations). The interval indicates the frequency () of the realized measure
and the "bus" header stands for business time sampling, if absent calendar time sampling is used. The subsampled/pre-
averaged RMs (N = 10) are used in the forecasts with the "ss" header. Reported in parentheses are the t-statistics based on
the Newey-West correction with 5 lags. The accuracy of forecasts is assessed with the R
2
of Mincer-Zarnowitz regression,
and through the MSE and QLIKE loss functions. The realized threshold bipower variations as in Eq. (2.6) were calculated
assuming c

= 3.
HAR-TCJ : RV
t
=
0
+
d
TC
1
+
w
TC
5
+
m
TC
22
+
dJ
TJ
1
+
wJ
TJ
5
+
mJ
TJ
22
+
t
1 min 1 min 1 min 1 min 5 min 5 min 5 min 5 min
bus bus bus bus
ss ss ss ss

0
0.093 0.116 -0.028 0.085 0.174 0.132 0.092 0.124
(1.440) (2.473) (-0.368) (2.172) (2.030) (2.837) (1.199) (2.321)

d
0.799 0.843 0.618 0.784 0.439 0.682 0.556 0.636
(1.617) (2.465) (1.300) (3.513) (1.440) (5.051) (2.403) (4.605)

w
1.458 0.659 1.101 0.232 1.214 0.423 0.637 0.234
(1.905) (2.091) (1.764) (1.588) (2.270) (2.025) (1.989) (1.520)

m
0.015 -0.386 0.345 -0.052 0.116 -0.468 0.516 0.129
(0.031) (-1.479) (0.834) (-0.457) (0.257) (-1.451) (1.280) (0.571)

dJ
0.115 0.082 0.146 -0.134 0.167 -0.010 0.167 0.168
(1.044) (0.767) (1.095) (-1.031) (1.482) (-0.085) (2.004) (1.346)

wJ
-0.758 -0.583 -0.712 -0.192 -0.629 -0.437 -0.472 -0.178
(-2.147) (-1.925) (-2.053) (-0.611) (-1.844) (-0.939) (-1.530) (-0.499)

mJ
0.519 0.948 0.185 0.659 0.318 1.705 0.041 0.386
(1.551) (2.173) (0.453) (0.872) (0.867) (2.227) (0.106) (0.851)
R
2
0.507 0.556 0.509 0.552 0.415 0.463 0.459 0.485
MSE 0.455 0.194 0.402 0.155 0.499 0.246 0.463 0.258
QLIKE 0.784 0.469 0.758 0.304 0.851 0.534 0.860 0.625
over the positive ones and speculation would see the former only to be responsible for increased
future volatility. Thus, insignicant or negative coefcients on jumps components can indicate
that positive jumps do not increase volatility but on the contrary, they reduce it. The last model,
the standard HAR, is used as a simple benchmark since it incorporates both types of risk. No
specic comments are needed except that future volatility is determined by the sole previous-day
realized variance.
In a broader comparison across different methods for the estimation of realized measures I
observe improved forecasting performance whenever the plain estimator is enriched with pre-
averaging or business time sampling and even more when both are combined. Nevertheless,
their action/interaction should be read with caution since the coefcients tend to change sign
or to become statistically insignicant depending on which model/RM we inspect. Thus, the
interpretation is of nontrivial nature. Closer and specic investigation is still required.
5. CONCLUSION
This study analyzes the variability of the FTSE 100 index and the role of jumps in one-day-ahead
in-sample forecasts performed with the HAR model and its modied versions. The series is
Forecasting the FTSE 100 with high-frequency data 21
Table 8. OLS regression for one-day-ahead HAR-CJ volatility forecasts for the FTSE 100 index on the last period
November 30, 2009 to July 12, 2011 (384 observations). The interval indicates the frequency () of the realized measure
and the "bus" header stands for business time sampling, if absent calendar time sampling is used. The subsampled/pre-
averaged RMs (N = 10) are used in the forecasts with the "ss" header. Reported in parentheses are the t-statistics based on
the Newey-West correction with 5 lags. The accuracy of forecasts is assessed with the R
2
of Mincer-Zarnowitz regression,
and through the MSE and QLIKE loss functions.
HAR-CJ : RV
t
=
0
+
dJ
J
1
+
d

2
1
BPV
1
+
w
RV
5
+
m
RV
22
+
t
1 min 1 min 1 min 1 min 5 min 5 min 5 min 5 min
bus bus bus bus
ss ss ss ss

0
0.079 0.114 0.047 0.098 0.167 0.142 0.163 0.151
(1.493) (3.328) (0.947) (3.243) (2.185) (3.230) (2.366) (3.252)

d
0.930 0.706 0.858 0.618 0.627 0.500 0.657 0.604
(4.024) (5.251) (4.203) (5.372) (5.480) (5.477) (4.890) (4.689)

w
0.108 0.171 0.142 0.220 0.150 0.266 0.145 0.172
(1.039) (1.915) (1.115) (2.380) (1.337) (2.443) (1.117) (1.808)

m
0.213 0.085 0.199 0.037 0.243 0.092 0.228 0.122
(2.112) (0.828) (2.001) (0.350) (1.923) (0.800) (2.033) (1.087)

J
0.004 -0.191 -0.057 -0.418 -0.063 -0.268 -0.132 -0.329
(0.075) (-1.694) (-1.001) (-3.139) (-1.165) (-1.703) (-2.710) (-3.065)
R
2
0.462 0.543 0.472 0.535 0.379 0.450 0.452 0.513
MSE 0.496 0.200 0.433 0.161 0.529 0.252 0.469 0.244
QLIKE 0.776 0.471 0.660 0.306 0.845 0.536 0.853 0.625
retrieved from a relatively new source, the TRTH database. The empirical exercise covers the
horizon from January 2, 1996 to July 12, 2011 and additionally is repeated on the period ranging
from November 30, 2009 to July 12, 2011. I use a variety of estimators of the quadratic variation or
integrated volatility and I decline them under several sampling frequencies and spaces, additionally
combining them with the pre-averaging heuristic.
The results on the whole horizon basically conrm the general ndings and the last advance-
ments in the literature. The forecasting power of the realized variance, assessed with the Mincer-
Zarnowitz R
2
, the MSE and QLIKE loss functions, is easily outperformed by bipower variation
which however does not recognize the role of jumps as signicant. The threshold bipower variation
can correct this effect but provides contrasting evidence depending on the sampling frequency.
Nevertheless, the HAR-CJ model exploits the signed jump variation estimated with the realized
semivariance and highlights the effect of negative jumps as triggering events of increased future
volatility.
Results on the sub-period, rank the HAR-CJ rst while the intra-model forecasts show
that realized measures pre-averaged in business time at 1 min intervals, consistently detain the
highest forecasting power. Focusing on the methods used to estimate the RMs, pre-averaging and
business time sampling usually adds to the forecasting power. However, their action combines
with the model/measure considered and their effect on the coefcients may vary unexpectedly. It
is important to keep in mind that such ndings may depend on the circumstances and specically
on the bullish trend which covers the last period. Thus, addressing the role of negative jumps
22 Oleg Komarov
which are expected to trigger higher future volatility may prove to be the wrong direction. In fact,
in periods of rising prices it is more likely that positive jumps dominate the negative and that the
former have a positive impact on future beliefs which reduces volatility.
ACKNOWLEDGEMENTS
I wish to thank my supervisor Professor Valentina Corradi for her guidance and invaluable advice. I
acknowledge Diccon Close from the Securities Industry Research Centre of Asia-Pacic (SIRCA)
for providing the access to the Thomson Reuters Tick History database and Giovanni Secondin
for his useful comments and suggestion. I am also very grateful to Julia Ewert for her excellent
proofreading. Finally, I would like to dedicate this work to my family. All errors and omissions
remain my own and sole responsibility.
APPENDIX A: JUMP DETECTION STATISTICS AND LOCAL VARIANCE
The statistics for Eq. (2.12) are dened as follows
Z
t

1
2
_

2
1
BPV
()
t
RV
()
t
1
__
max
_
1, TriPV
()
t
_
_
BPV
()
t
_
2
_
(A.1)
TriPV
()
t

3
4
3

1
1/

j=3
3

k=1
|r
(j1+k)
|
4
3
(A.2)
TZ
t

1
2
_

2
1
TBPV
()
t
RV
()
t
1
__
max
_
1, TTriPV
()
t
_
_
TBPV
()
t
_
2
_
(A.3)
TTriPV
()
t

3
4
3

1
1/

j=3
3

k=1
|r
(j1+k)
|
4
3
1
{r
2
(j1+k)

(j1+k)
}
(A.4)
where ( /2)
2
+ 5.
The local variance is estimated by iterating in z a non-parametric lter

V
z
j
=

j
K
_
j
L
_
r
2
j
1
{r
2
j
c
2

V
z1
j
}

j
K
_
j
L
_
1
{r
2
j
c
2

V
z1
j
}
(A.5)
j = {x Z : x = [L, L] x = [1, 1]} and z = 1, 2, . . .
where K() is a Gaussian kernel
K(y) =
1

2
exp
_

y
2
2
_
for details refer to Corsi et al. (2010).
Forecasting the FTSE 100 with high-frequency data 23
REFERENCES
At-Sahalia, Y., P. A. Mykland, and L. Zhang (2005). How often to sample a continuous-time
process in the presence of market microstructure noise. Review of Financial Studies 18(2),
351416.
Andersen, T. G., T. Bollerslev, and F. X. Diebold (2007). Roughing it up: Including jump
components in the measurement, modeling, and forecasting of return volatility. Review of
Economics & Statistics 89(4), 701720.
Andersen, T. G., T. Bollerslev, F. X. Diebold, and P. Labys (2000). Great realizations. Risk
Magazine 13, 105108.
Andersen, T. G., T. Bollerslev, F. X. Diebold, and P. Labys (2001). The distribution of realized
exchange rate volatility. Journal of the American Statistical Association 96(453), 4255.
Andersen, T. G., T. Bollerslev, F. X. Diebold, and P. Labys (2003). Modeling and forecasting
realized volatility. Econometrica 71(2), 579625.
Andersen, T. G., T. Bollerslev, and N. Meddahi (2011). Realized volatility forecasting and market
microstructure noise. Journal of Econometrics 160(1), 220234.
Back, K. (1991). Asset pricing for general processes. Journal of Mathematical Economics 20(4),
371395.
Bandi, F. M. and J. R. Russell (2008). Microstructure noise, realized variance, and optimal
sampling. Review of Economic Studies 75(2), 339369.
Barndorff-Nielsen, O. E., P. R. Hansen, A. Lunde, and N. Shephard (2009). Realized kernels in
practice: Trades and quotes. Econometrics Journal 12(3), C1C32.
Barndorff-Nielsen, O. E., S. Kinnebrock, and N. Shephard (2010). Measuring downside risk:
Realised semivariance. In T. Bollerslev, J. Russell, and M. Watson (Eds.), Volatility and Time
Series Econometrics: Essays in Honor of Robert F. Engle. Oxford University Press.
Barndorff-Nielsen, O. E. and N. Shephard (2006). Econometrics of testing for jumps in nancial
economics using bipower variation. Journal of Financial Econometrics 4(1), 130.
Barndorff-Nielsen, O. E. and N. Shephard (2007). Variation, jumps and high frequency data
in nancial econometrics. In R. Blundell, T. Persson, and W. K. Newey (Eds.), Advances in
Economics and Econometrics. Theory and Applications, Ninth World Congress, Econometric
Society Monographs, pp. 328372. Cambridge University Press.
Barndorff-Nielsen, O. E., N. Shephard, and M. Winkel (2005). Limit theorems for multipower
variation in the presence of jumps. Working paper.
Becker, R. and A. E. Clements (2008). Are combination forecasts of S&P 500 volatility statistically
superior? International Journal of Forecasting 24(1), 122133.
Brownlees, C. and G. Gallo (2006). Financial econometric analysis at ultra-high frequency: Data
handling concerns. Computational Statistics & Data Analysis 51(4), 22322245.
Chelley-Steeley, P. (2005). Noise and the trading mechanism: The case of SETS. European
Financial Management 11(3), 387424.
Corsi, F. (2009). A simple approximate long-memory model of realized volatility. Journal of
Financial Econometrics 7(2), 174196.
24 Oleg Komarov
Corsi, F., D. Pirino, and R. Ren (2010). Threshold bipower variation and the impact of jumps on
volatility forecasting. Journal of Econometrics 159(2), 276288.
Falkenberry, T. N. (2002). High frequency data ltering. Technical report, Data Tick.
Grifn, J. E. and R. C. A. Oomen (2008). Sampling returns for realized variance calculations:
Tick time or transaction time? Econometric Reviews 27(1-3), 230253.
Oomen, R. C. A. (2006). Properties of realized variance under alternative sampling schemes.
Journal of Business & Economic Statistics 24(2), 219237.
Patton, A. J. (2011a). Data-based ranking of realised volatility estimators. Journal of Economet-
rics 161(2), 284303.
Patton, A. J. (2011b). Volatility forecast comparison using imperfect volatility proxies. Journal of
Econometrics 160(1), 246256.
Patton, A. J. and K. Sheppard (2009). Optimal combinations of realised volatility estimators.
International Journal of Forecasting 25(2), 218238.
Patton, A. J. and K. Sheppard (2011). Good volatility, bad volatility: Signed jumps and the
persistence of volatility. Working paper.