Académique Documents
Professionnel Documents
Culture Documents
ABSTRACT
When no model coincides with a nonconstant data generation process, forecast failure occurs, and
noncausal statistical devices may provide the best available forecasts: examples include intercept
corrections and differenced-data VARs. However, such models are not a reliable basis for economic
policy analyses and may even have no policy implications. Indeed, a “paradox” can result if their
forecasts induce policy changes, which in turn alter the data outcome. This suggests correcting
statistical forecasts by using the econometric model’s estimate of the “scenario” change, and doing
so is shown to yield reduced forecast-error biases.
1. INTRODUCTION
As the title of this volume indicates, the majority of Tom Rothenberg’s con-
tributions to econometrics and statistics were in the fields of identification and
inference. However, we know that Tom has an interest in all areas of econo-
metrics, even though his awareness of the numerous difficulties to be overcome
in undertaking applied work has so far led him to concentrate on theoretical
research topics. Therefore, we are delighted to offer our chapter on the use of
econometric models in forecasting and economic policy analysis as an indica-
tion of some more problems – as well as an analysis of how several of these
might be overcome.
A personal anecdote will explain our interest. When the first author reported
in Hendry (1981) to the UK Parliamentary Select Committee on the Treasury
and Civil Service Enquiry into Monetary Policy, he complained about the pro-
liferation of theoretical models of money and the dearth of reliable empirical
evidence; a decade later in Hendry (1991), he complained to the same commit-
tee (now on Official Economic Forecasting) about the abundance of empirical
evidence on forecasting – and the absence of theoretical models thereof! We
hope to fill a part of that second gap.
In Hendry and Mizon (2000), we investigated three aspects of the rela-
tionship between statistical forecasting devices and econometric models in the
context of economic policy analysis. First, whether there were grounds for bas-
ing economic policy analysis on the “best” forecasting system. Second, whether
Downloaded from https://www.cambridge.org/core. IP address: 114.4.82.245, on 14 Sep 2017 at 15:44:28, subject to the Cambridge Core
Cambridge Books Online © https://doi.org/10.1017/CBO9780511614491.021
terms of use, available at https://www.cambridge.org/core/terms. Cambridge University Press, 2010
Forecasting in the Presence of Structural Breaks 481
forecast failure in an econometric model precluded its use for economic policy
analysis. Finally, whether in the presence of policy change, improved forecasts
could be obtained by using “scenario” changes derived from the econometric
model, to modify an initial statistical forecast. To resolve these issues, we an-
alyzed the problems arising when forecasting took place immediately after a
structural break (i.e., a change in the parameters of the econometric system),
but before a regime shift (i.e., a change in the behavior of nonmodeled, often
policy, variables), perhaps in response to the break (Hendry and Mizon (1998)
for discussion of this distinction). No forecasting system can be immune to
unmodeled breaks that occur after forecasts are announced, whereas some de-
vices are robust to breaks that occur prior to forecasting. Such robust devices
are particularly relevant in real time when a structural break has occurred but,
perhaps, is not yet known to have occurred. These three dichotomies, between
econometric and statistical models, structural breaks and regime shifts, and
pre- and post-forecasting events, remain central to our present results. Further
particular concerns in the present chapter are working with open models and
considering the effects of structural breaks in cointegrating vectors.
Statistical forecasting devices are taken to have no basis of economic theory
(in contrast to econometric models for which this is central); so even when
combined into a system, such devices will rarely have implications for eco-
nomic policy analysis – and may not even entail links between target variables
and policy instruments. This feature will be true of the forecasting devices we
consider. Consequently, being the “best” available forecasting device is insuffi-
cient to ensure any value for policy analysis. Some “forecasting models,” such
as vector autoregressions (VARs), may also have policy implications, and we
comment on such VARs later, but that does not vitiate the previous statement.1
The converse is more relevant: Does the existence of a dominating fore-
casting procedure invalidate the use of an econometric model for policy? In
Hendry and Mizon (2000), our answer was almost the opposite of the Lucas
(1976) critique: when forecast failure results from factors unrelated to policy
changes – as Stock and Watson (1996) and Clements and Hendry (2001) show
often occurs – an econometric model can continue to accurately characterize the
responses of the economy to the policy, despite its forecast of those variables
being inaccurate. However, forecast failure could also derive from incorrectly
modeled policy reactions. Conversely, when forecast failure results from an
in-sample structural break, forecasts from a statistical model that is robust to
that structural break may be improved by combining them with the predicted
response from an econometric model of a known out-of-sample policy change.
The structure of the chapter is as follows. In the next section, we summa-
rize the relevant forecasting and economic policy concepts and issues to mo-
tivate the analysis, illustrated in Section 2.2 by an example of forecasting and
policy in the presence of regime shifts. Section 3 formalizes the data generation
1 Sims (1986) regards both classes of model distinguished here as “forecasting” models, but argues
that nevertheless both can have policy implications.
Downloaded from https://www.cambridge.org/core. IP address: 114.4.82.245, on 14 Sep 2017 at 15:44:28, subject to the Cambridge Core
Cambridge Books Online © https://doi.org/10.1017/CBO9780511614491.021
terms of use, available at https://www.cambridge.org/core/terms. Cambridge University Press, 2010
482 Hendry and Mizon
2. BACKGROUND
The literature on economic policy is vast, even restricting attention to its imple-
mentation using econometric models: related analyses include Bryant, Hooper,
and Mann (1993) on evaluating policy regimes; Budd (1998) on conducting
economic policy with and without forecasts; Britton (1989) on the more gen-
eral topic of policy making with macroeconomic models; Sims (1982), Turner,
Wallis, and Whitley (1989), and Banerjee, Hendry, and Mizon (1996) on the
econometric analysis of economic policy; and Burns (2001) on the costs of
forecast errors in an economic policy context. However, the present focus on
combining forecasts from models in the face of both structural breaks (specif-
ically, ones which take the form of location shifts) and regime shifts is not
prominent in that literature.
The specific rationale for our analysis is as follows. Using the taxonomy of
forecast errors in Clements and Hendry (1995), Hendry and Doornik (1997)
establish that shifts in the coefficients of deterministic variables (inducing loca-
tion shifts) are the primary source of systematic forecast failure in econometric
models. Deterministic variables include intercepts and linear trends – variables
whose future values are known with certainty, but the parameters of which could
change. Location shifts are viewed as any change in the unconditional expec-
tation of the nonintegrated transformations of the variables. In the simplest
location model of a variable yt ,
yt = α × 1 + u t , (2.1)
∗
with α = 0 where {u t } is I(0), a location shift to α from T onwards induces:
yT +h = α ∗ × 1 + u T +h ≡ α × µ + u T +h , (2.2)
∗
where µ = α /α = 1 is the shifted intercept for h > 0. Thus, shifts in pa-
rameters of deterministic terms are equivalent to location shifts, as are other
factors that mimic deterministic shifts, such as misestimating or misspecify-
ing deterministic components in models. The simulation evidence in Hendry
(2000) confirms their pernicious effects on forecasts, as well as highlights the
difficulty of detecting other forms of break (i.e., those associated with mean
zero changes, namely those that have no effect on unconditional means). More-
over, Barrell (2001) illustrates the frequency of such breaks by considering six
major episodes of change during the 1990s alone.
Downloaded from https://www.cambridge.org/core. IP address: 114.4.82.245, on 14 Sep 2017 at 15:44:28, subject to the Cambridge Core
Cambridge Books Online © https://doi.org/10.1017/CBO9780511614491.021
terms of use, available at https://www.cambridge.org/core/terms. Cambridge University Press, 2010
Forecasting in the Presence of Structural Breaks 483
Despite location shifts being easier to detect than mean zero changes in
general, they too may be overlooked if they occur at, or very near, the end of
the sample period. However, there exist devices which can robustify forecast-
ing models against breaks that have occurred prior to forecasting (see, e.g.,
Clements and Hendry 1996, and Hendry and Clements 2000): examples in-
clude imposing an additional unit root, or adding a specific form of intercept
correction. Such “tricks” can help mitigate forecast failures, but they do not
change a model’s policy implications. This result nullifies the argument for us-
ing a forecast-based criterion to choose models in order to predict the effects of
known policy changes: whether or not a given model forecasts well after a break
may depend on its robustness to location shifts as much as on its “closeness”
to the data generation process.
Importantly, no methods are robust to unanticipated breaks that occur after
forecasts are announced, and Clements and Hendry (1999b) show that those
same “robustifying” devices do not offset post-forecasting breaks. But location-
shift policy changes that occur after forecasting will induce breaks in all models
that do not embody the appropriate policy links: such models, even if unaffected
by an earlier structural break, lose their robustness to post-forecasting regime
shifts. Thus, the existence of a procedure that, in the presence of in-sample
structural breaks, systematically produces better forecasts need not invalidate
basing policy on another model. Indeed, despite possibly experiencing forecast
failures from pre-forecasting breaks, econometric systems that do embody the
relevant policy effects need not experience a post-forecasting break induced
by an out-of-sample policy regime shift. Consequently, when both in-sample
structural breaks and forecast-period regime shifts occur, neither class of model
alone is adequate: this suggests investigating whether, and if so how, they should
be combined.
Given the limited knowledge about the data generation process (DGP) that
is available to empirical investigators, we adopt a framework in which the DGP
is unknown and nonstationary (due to both unit roots and structural breaks),
and the econometric model is misspecified for that DGP, if only by failing to
correctly model breaks at the forecast origin. These features seem descriptive
of operational economic forecasting and provide a rationale for using “intercept
corrections” and differencing transformations. A key consequence of these re-
sults is that the best available forecasting model need not be based on the “causal
determinants” of the actual economic process, and as the example in Section 2.2
shows, may be based on “noncausal” variables, that is, variables which do not
enter the DGP.
The fact that a purely statistical device may provide the best available fore-
casts induces an apparent paradox when policy change is feasible. In a world
characterized by the framework we adopt, when forecasting after a structural
break, forecasts based on the currently best econometric model may be beaten
by statistical devices. Assume for the moment that the statistical forecasting
model does not depend on any policy variables and, hence, neither has policy
implications nor produces any revisions to its forecasts following any policy
Downloaded from https://www.cambridge.org/core. IP address: 114.4.82.245, on 14 Sep 2017 at 15:44:28, subject to the Cambridge Core
Cambridge Books Online © https://doi.org/10.1017/CBO9780511614491.021
terms of use, available at https://www.cambridge.org/core/terms. Cambridge University Press, 2010
484 Hendry and Mizon
changes. These “best” forecasts for some future period are presented to the
finance minister of a given country, who thereupon decides that a major policy
initiative is essential, and implements it. That the statistical forecasts are not
then revised would be greeted with incredulity. More pertinently, providing the
policy model did not fall foul of the critique in Granger and Deutsch (1992), so
that changes to policy instruments in fact correctly altered target variables; then
a better forecast seems likely by adding the policy change effects predicted by
the econometric model to the previous forecasts. But this contradicts any claim
to the effect that the statistical device produced the best forecasts in a world of
structural change and policy regime shifts.
The resolution depends on distinguishing between unknown, or unantic-
ipated, breaks that have occurred – where, for example, differenced models
may deliver the best achievable forecast – and subsequent known changes, the
consequences of which are partly measurable. An implication is that combin-
ing robustified statistical forecasts with the scenario changes from econometric
systems subject to policy interventions may provide improved forecasts. This
is the subject of Section 7. First, we establish our terminology and notation to
clarify the analysis later.
2.1. Terminology
Consider the following illustrative static bivariate system:
2
yt = α + βz t + vt vt 0 σv 0
with ∼ IN2 , ,
z t = µz + ε z,t ε z,t 0 0 σ 2z
(2.3)
where α, β, µz , σ 2v , σ 2z ∈ R3 × R2+ . The first equation relating yt to z t is
deemed to be causal here:
∂ yt
= β,
∂z t
with z t strongly exogenous for α, β, σ 2v ; whereas the second equation for z t
is a policy process. However, the specification in (2.3) does not directly entail
how changes in any subset of parameters might affect the remainder, and so
additional assumptions are required. Note that (2.3) can be solved to yield
2
yt = µ y + ε y,t ε y,t 0 σy ρ
with ∼ IN2 , ,
z t = µz + ε z,t ε z,t 0 ρ σ 2z
(2.4)
where µ y = α + βµz , ρ = βσ 2z , and σ 2y = σ 2v + βρ.
A structural break occurs when the parameters of the causal model change
(here either ∇β = 0 or ∇α = 0, or both, using ∇ to denote a parameter change).
A policy regime shift occurs when the parameters of the policy process change
(here either ∇µz = 0 or ∇σ 2z = 0, or both). Structural invariance occurs when
Downloaded from https://www.cambridge.org/core. IP address: 114.4.82.245, on 14 Sep 2017 at 15:44:28, subject to the Cambridge Core
Cambridge Books Online © https://doi.org/10.1017/CBO9780511614491.021
terms of use, available at https://www.cambridge.org/core/terms. Cambridge University Press, 2010
Forecasting in the Presence of Structural Breaks 485
Downloaded from https://www.cambridge.org/core. IP address: 114.4.82.245, on 14 Sep 2017 at 15:44:28, subject to the Cambridge Core
Cambridge Books Online © https://doi.org/10.1017/CBO9780511614491.021
terms of use, available at https://www.cambridge.org/core/terms. Cambridge University Press, 2010
486 Hendry and Mizon
Downloaded from https://www.cambridge.org/core. IP address: 114.4.82.245, on 14 Sep 2017 at 15:44:28, subject to the Cambridge Core
Cambridge Books Online © https://doi.org/10.1017/CBO9780511614491.021
terms of use, available at https://www.cambridge.org/core/terms. Cambridge University Press, 2010
Forecasting in the Presence of Structural Breaks 487
inside the unit circle. For t < T , the I(0) variables xt and β xt are stationary
in sample, so let
E [xt ] = γ and E β xt = µ ∀ t < T. (3.2)
Taking expectations in (3.1) yields
E [xt ] = τ + ΓE [xt ] + αE β xt ,
so that
τ = (In − Γ) γ − αµ, (3.3)
where E β xt = β γ = 0. Substituting from (3.3) into (3.1),
xt − γ = Γ (xt−1 − γ) + α β xt−1 − µ + t .
Factorizing (3.1) gives the in-sample open VEqCM for the I(1) variables yt ,
conditional on zt , representing the behavior of the private sector, which we take
to be a causal relation:
yt − γ y = Π zt − γ z + Λ (xt−1 − γ)
+ λ β xt−1 − µ + ut , (3.4)
−1
where Π = Σ yz Σzz , Λ = Γ y − ΠΓz , λ = α y − Παz , with
Σ yy Σ yz Γy Γ yy Γ yz
Σ= , Γ= = ,
Σzy Σzz Γz Γzy Γzz
αy γy
α= , γ= , (3.5)
αz γz
and
ut 0 Ω 0
∼ INn , , (3.6)
z,t 0 0 Σzz
in which Ω = (Σ yy − ΠΣzz Π ). The in-sample marginal model of the policy
variables zt is
zt − γ z = Γz (xt−1 − γ) + αz β xt−1 − µ + z,t . (3.7)
The relevant policy instruments (e.g., interest rates and tax rates) are elements of
zt that are assumed to be under the control of the policy agency (e.g., a central
bank, or Treasury). The fact that in modeling zt it is treated as I(1) neither
implies that it is intrinsically I(1) nor that it cannot be controlled by the policy
agency. Further, although (3.7) is the representation in the econometric model
of the evolution of zt , the marginal process may differ for particular policy rules
in operation from time to time (i.e., because of regime shifts); see, for example,
Johansen and Juselius (2000).
In a policy setting, zt is unlikely to be weakly exogenous for the parameters
of the conditional model, as policy reactions are often based on private sector
disequilibria such as β xt−1 − µ , so that αz is usually nonzero. Moreover,
Downloaded from https://www.cambridge.org/core. IP address: 114.4.82.245, on 14 Sep 2017 at 15:44:28, subject to the Cambridge Core
Cambridge Books Online © https://doi.org/10.1017/CBO9780511614491.021
terms of use, available at https://www.cambridge.org/core/terms. Cambridge University Press, 2010
488 Hendry and Mizon
the growth rates are linked by the cointegration vectors as β γ = 0, even when
αz = 0. The key to understanding the impact of policy changes in such for-
mulations depends on the remaining (implicit) parameter links and invariances,
particularly co-breaking relations: we deal with that case here and return briefly
to the absence of appropriate co-breaking (a variant of the Lucas critique) in
Section 7.3. As discussed by Hendry and Mizon (2000), if all the parameters
in (3.4) and (3.7) were unconnected, the system would manifest “policy inef-
fectiveness,” in that only deviations of zt from γ z would have an impact, so
that only impulse responses would be of interest. However, shifts in γ z must
alter γ y through the feedback relations (unless r = 0), and so the impact and
long-run responses differ.
which therefore does not have any policy implications. Clements and Hendry
(1999b) show that these predictors have the same forecast biases for breaks that
occur after forecasts are announced, but that the DDV is more robust to breaks
in deterministic terms that have occurred before forecasting: importantly, that
robustness does not require knowledge that breaks have occurred. Section 4
draws on their approach, extending it to open models and to forecasts of growth
rates (rather than levels). In terms of the example in Section 7.4, forecasts are
made after a regime shift resulting in a change in the appropriate measurement
of the opportunity cost of holding money – which induces a structural break
in models not incorporating that change – but before a further known policy
regime shift. Since the open VEqCM has some response to the new policy, but
the DDV does not, such comparisons yield insights into the effects of using
robustified forecasting methods and then exploiting policy change information
via an econometric system.
In the present context, a VAR is simply the unrestricted version of the
econometric model and so is subject to the same drawbacks, namely, a lack
of robustness to location shifts, exacerbated by the ill-determination of the
estimated intercepts, which compound (small) growth rates with (potentially
large) equilibrium means; see Equation (3.3). A VAR in first differences is mis-
specified by omitting the cointegration relations, often of central concern in
policy, but thereby gains robustness to previous equilibrium mean shifts (see,
Downloaded from https://www.cambridge.org/core. IP address: 114.4.82.245, on 14 Sep 2017 at 15:44:28, subject to the Cambridge Core
Cambridge Books Online © https://doi.org/10.1017/CBO9780511614491.021
terms of use, available at https://www.cambridge.org/core/terms. Cambridge University Press, 2010
Forecasting in the Presence of Structural Breaks 489
e.g., Clements and Hendry 1999a). Although both these models are sometimes
viewed as “statistical forecasting devices” with possible policy implications,
they lie intermediate between the econometric system in (3.1) and the DDV, and
so introducing them adds little to the understanding of the analysis; Clements
and Hendry (1999a) analyze their susceptibility to location shifts before and
after forecasting.
a policy regime shift in the DGP could also result in a structural break in
(3.4) unless the parameters of the latter were invariant, an issue addressed in
Section 7.3.
In this section, we consider the impacts of these parameter changes in the
VEqCM DGP over the forecast period on the preexisting forecasting model,
written as
yt = γ 0 + Πzt + Λz zt−1 + Λ y yt−1 + λ β xt−1 − µ + ut ,
(4.1)
where
γ 0 = In 1 − Λ y γ y − (Π + Λz ) γ z .
Following the parameter changes noted, (4.1) for h ≥ 1 becomes
y∗T +h = γ ∗y + Π z∗T +h − γ ∗z + Λz z∗T +h−1 − γ ∗z
+ Λ y y∗T +h−1 − γ ∗y + λ∗ β ∗ x∗T +h−1 − µ∗ + uT +h . (4.2)
It is assumed that the forecaster, though aware of the possibility of parameter
changes, does not know that they have actually changed. As a result, the open
VEqCM (4.1) estimated using data up to t = T , and indeed for t = T + h for
h/T small, will suffer forecast failure because of the shift in µ to µ∗ and any
Downloaded from https://www.cambridge.org/core. IP address: 114.4.82.245, on 14 Sep 2017 at 15:44:28, subject to the Cambridge Core
Cambridge Books Online © https://doi.org/10.1017/CBO9780511614491.021
terms of use, available at https://www.cambridge.org/core/terms. Cambridge University Press, 2010
490 Hendry and Mizon
I(0) Cases
This case includes the change from β to β ∗ simply being a linear transformation
(rotation), so that there exists an r × r matrix D of rank r , such that β ∗ = Dβ
with a corresponding change in the equilibrium mean to µ∗ = Dµ. Such a
situation is essentially one of no change, merely a reparameterization as
λ∗ β ∗ x∗T +h−1 − µ∗ = λ∗ Dβ x∗T +h−1 − Dµ
= λ β x∗T +h−1 − µ (4.3)
as λ∗ D = λ.
A second class of changes in β that have only I(0) effects is when new
cointegration relations are added. Here r changes to r ∗ (say), and so both λ and
β are changed in dimension, and the outcome is the addition of a mean zero
∗
term, which
has
minimal effects
on
forecasts. Since r ∗
> r , let λ = λ, λ† ,
β ∗ = β, β † , and µ∗ = µ, µ† , so that the new and the old cointegration
vectors are linked by
λ∗ β ∗ x∗T +h−1 − µ∗ = λ β x∗T +h−1 − µ + λ† β † x∗T +h−1 − µ† ,
and the model error over the forecast period becomes
u∗T +h = uT +h + λ† β † x∗T +h−1 − µ† ,
which remains I(0) with a zero mean, albeit perhaps autocorrelated. Potentially,
however, there is an “initial conditions” problem, since the relevant variables
that enter β † x∗T +h−1 were not previously
cointegrated, and so could have drifted
apart such that at T + 1, β † x∗T − µ† is sufficiently “large” to induce one-step
forecast failure in models that exclude it (see, e.g., Hendry 2000).
In the third case, when fewer cointegration relations occur over the forecast
period, the outcome is less clear, as some existing elements of β are eliminated.
∗ ∗ ∗ ∗
case r <r , and so partitioning λ , β, and µ into (λa , λb ) =
This is the
(λa , 0), β a , β b , and µa , µb , respectively, the new and the old cointegration
vectors are linked by
λ∗ β x∗T +h−1 − µ = λa∗ β a x∗T +h−1 − µa
= λ β x∗T +h−1 − µ − λb β b x∗T +h−1 − µb ,
and so the model error over the forecast period becomes
u∗T +h = uT +h − λb β b x∗T +h−1 − µb .
Downloaded from https://www.cambridge.org/core. IP address: 114.4.82.245, on 14 Sep 2017 at 15:44:28, subject to the Cambridge Core
Cambridge Books Online © https://doi.org/10.1017/CBO9780511614491.021
terms of use, available at https://www.cambridge.org/core/terms. Cambridge University Press, 2010
Forecasting in the Presence of Structural Breaks 491
This is only I(0) provided that (β b x∗T +h−1 − µb ) remains I(0), in which case,
u∗T +h is still an I(0) error with a zero mean. However, such an outcome seems
unlikely: rather, although λb has become zero in the DGP, the forecasting
model retains linear combinations of I(1) variables that are now I(1), and µb
has no meaning. Nevertheless, the simulation results in Hendry 2000 suggest
that such changes are difficult to detect empirically from moderate sample
sizes.
A final case is when there is a set of k additional I(1) variables, say wt , that are
self-cointegrated within the DGP but have not been included in theforecasting
model in any way, and so the I(0) combinations β + wT +h−1 − δ have been
omitted from the model when E[β + wT +h ] = δ for h > 0. None of these types
of I(0) changes in β is liable to induce systematic forecast failure, and so they
are not considered here.
I(1) Cases
An important type of break in the cointegration relations that induces I(1) ef-
fects arises in the last case when β x∗T +h and β + wT +h cease to be I(0) individ-
ually. Now a system that excludes wT +h will be seriously misspecified:
both
β xT +h − µ and u∗T +h will be I(1). As a result, β xT +h − µ could become
large, in which case forecast failure should manifest itself, forcing a revision
to the model specification. This case comes close to the situation considered in
the example in Section 7.4, in which the additional I(1) variable is Ro , the then
newly legalized own interest rate on retail sight deposits.
The last case arises when the rank remains the same, but now cointe-
gration becomes defined by new coefficients β ∗ , so that the error in (4.3)
becomes
u∗T +h = uT +h + λ ∇β ∗ x∗T +h−1 − ∇µ∗ ,
with ∇β ∗ = β ∗ − β and ∇µ∗ = (µ∗ − µ). Although (β ∗ x∗T +h−1 − µ∗ ) is
∗
I(0), β xT +h−1 − µ will be I(1) after time T , and so the unmodified system
will suffer consequences similar to the I(1) mistake case just noted. Even if
the equilibrium mean did not change, so that ∇µ∗ = 0, the same problems
would arise since u∗T +h must be I(1) via the non-cointegrating combinations
∇β ∗ x∗T +h−1 . However, we note that even in the most favorable case for avoid-
ing forecast failure (i.e., when µ = µ∗ = 0), the simulation results in Hendry
(2000) show quite high power in detecting changes in β, at least in moderately
large samples. Conversely, the situation where β ∗ = β but µ∗ = µ is a pure
location shift, adding −λ∇µ∗ to the forecast error, and is readily detected when
breaks exceed the equation error standard deviations.
Downloaded from https://www.cambridge.org/core. IP address: 114.4.82.245, on 14 Sep 2017 at 15:44:28, subject to the Cambridge Core
Cambridge Books Online © https://doi.org/10.1017/CBO9780511614491.021
terms of use, available at https://www.cambridge.org/core/terms. Cambridge University Press, 2010
492 Hendry and Mizon
Downloaded from https://www.cambridge.org/core. IP address: 114.4.82.245, on 14 Sep 2017 at 15:44:28, subject to the Cambridge Core
Cambridge Books Online © https://doi.org/10.1017/CBO9780511614491.021
terms of use, available at https://www.cambridge.org/core/terms. Cambridge University Press, 2010
Forecasting in the Presence of Structural Breaks 493
3 This assumes that the VEqCM is the DGP. If there are omitted variables from the former, then
the error variance can actually decrease on differencing (Hendry 2004).
Downloaded from https://www.cambridge.org/core. IP address: 114.4.82.245, on 14 Sep 2017 at 15:44:28, subject to the Cambridge Core
Cambridge Books Online © https://doi.org/10.1017/CBO9780511614491.021
terms of use, available at https://www.cambridge.org/core/terms. Cambridge University Press, 2010
494 Hendry and Mizon
a further shift in the equilibrium mean from µ∗ to µ p and the growth rates to
γ p , but does not affect the cointegration vectors, so that β ∗ γ p = 0. After the
policy regime shift, the zt process takes the form (considering the special case
that αz = 0 for simplicity)
zT +h+1 = γ zp + Γz x∗T +h − γ ∗ + z,T +h+1
p
p p
zT +h+ j = γ zp + Γz xT +h+ j−1 − γ p + z,T +h+ j for j > 1,
(6.1)
p p p
and results in (γ 0 = In 1 − Λ y γ y − (Π + Λz ) γ z ):
p
yT +h+1 = γ y − Λ y γ ∗y − Πγ zp − Λz γ ∗z + ΠzT +h+1
p p
+ Λ y y∗T +h + Λz z∗T +h
+ λ β ∗ x∗T +h − µ∗ + uT +h+1
p p p p (6.2)
yT +h+ j = γ 0 + ΠzT +h+ j + Λ y yT +h+ j−1
+ Λz zT +h+ j−1 + λ β ∗ xT +h+ j−1 − µ p
p p
Downloaded from https://www.cambridge.org/core. IP address: 114.4.82.245, on 14 Sep 2017 at 15:44:28, subject to the Cambridge Core
Cambridge Books Online © https://doi.org/10.1017/CBO9780511614491.021
terms of use, available at https://www.cambridge.org/core/terms. Cambridge University Press, 2010
Forecasting in the Presence of Structural Breaks 495
and so the DDV forecast is unaltered, implying that its forecast error changes
one for one with the impact of the policy change. Thus, the forecast error is
uT +h+1|T +h = yT +h+1 − y∗T +h , and so denoting by
p p
1
γ h = γ yp − Λ y γ ∗y − Πγ zp − Λz γ ∗z ,
p
then
p
1
uT +h+1|T +h
= γ h + ΠzT +h+1 + Λ y y∗T +h + Λz z∗T +h
p p
+ λ β ∗ x∗T +h − µ∗ + uT +h+1 − γ ∗0 − Πz∗T +h
− Λ y y∗T +h−1 − Λz z∗T +h−1 − λ β ∗ x∗T +h−1 − µ∗ − uT +h
p p
= uT +h+1 + γ h − γ ∗0 + Π zT +h+1 − z∗T +h
+ 2 x∗T +h + λβ ∗ x∗T +h .
Consequently, the expected DDV unconditional forecast error is
p
uT +h+1|T +h = γ h − γ ∗0 + Π E zT +h+1 − E z∗T +h
p p
E 1
p
= γ h − γ ∗0 + Π γ zp − γ ∗z = γ yp − γ ∗y . (7.1)
As anticipated, a forecast bias results because the policy shift occurs after the
DDV forecast is announced.
Forecasts from the open VEqCM, on the other hand, are now
∗
3p T +h+1|T +h = γ 0 + Πz p
y ∗
T +h+1 + ΛxT +h + λ β xT +h − µ .
(7.2)
p p
Hence, the VEqCM has a forecast error of )
uT +h+1|T +h = yT +h+1 −
3p T +h+1|T +h :
y
p
)
uT +h+1|T +h
= γ h + ΠzT +h+1 + Λ y y∗T +h + Λz z∗T +h
p p
+ λ β ∗ x∗T +h − µ∗ + uT +h+1
− γ 0 + ΠzT +h+1 + Λx∗T +h + λ β x∗T +h − µ
p
p
= γ h − γ 0 + λ∇β ∗ x∗T +h − λ∇µ∗ + uT +h+1 , (7.3)
with a conditional expectation of
p
E yT +h+1 − y9 Tp +h+1|T +h | x∗T +h
p
= γ h − γ 0 + λ∇β ∗ x∗T +h − λ∇µ∗ . (7.4)
It is easy to envisage conditions where (7.4) is much larger than (7.1).
Downloaded from https://www.cambridge.org/core. IP address: 114.4.82.245, on 14 Sep 2017 at 15:44:28, subject to the Cambridge Core
Cambridge Books Online © https://doi.org/10.1017/CBO9780511614491.021
terms of use, available at https://www.cambridge.org/core/terms. Cambridge University Press, 2010
496 Hendry and Mizon
Nevertheless, comparing (4.4) and (7.2), the VEqCM forecasts are revised
from their pre–policy change values by the difference
p
y 9 ∗T +h+1|T +h = Π z p
9 T +h+1|T +h − y ∗
T +h+1 − zT +h+1 .
(7.5)
which might avoid much of the structural break, yet capture some, and possibly
all, of the policy effect.
The unconditional bias of yT +h+1|T +h from (7.6) is
p
E yT +h+1 −yT +h+1|T +h
p
= E yT +h+1 − y 0 Tp +h+1|T +h
p
− E y 9 ∗T +h+1|T +h
9 T +h+1|T +h − y
= γ yp −γ ∗y − Π γ zp − γ ∗z . (7.7)
Because the growth rates are linked by the cointegration relationships, co-
breaking of the form γ y = Πγ zp and γ ∗y = Πγ ∗z cannot hold, but there will nev-
p
Downloaded from https://www.cambridge.org/core. IP address: 114.4.82.245, on 14 Sep 2017 at 15:44:28, subject to the Cambridge Core
Cambridge Books Online © https://doi.org/10.1017/CBO9780511614491.021
terms of use, available at https://www.cambridge.org/core/terms. Cambridge University Press, 2010
Forecasting in the Presence of Structural Breaks 497
while recognizing that other choices of weighting factors are also used. The un-
conditional bias (where it exists) in the post–policy VEqCM forecast is given by
uT +h+1|T +h = γ h − γ 0 + λE ∇β ∗ x∗T +h − λ∇µ∗
p p
E ) (7.9)
Hence, most of the break will be corrected, just as with the scenario correction in
(7.7). The only additional component unconditionally is λβ ∗ γ, which will be
small on average (e.g., if β ∗ = β, it would be precisely zero). Again, there are
variance consequences (the forecast error variance is doubled for innovation
errors), but these have been small relative to the sizes of structural breaks
Downloaded from https://www.cambridge.org/core. IP address: 114.4.82.245, on 14 Sep 2017 at 15:44:28, subject to the Cambridge Core
Cambridge Books Online © https://doi.org/10.1017/CBO9780511614491.021
terms of use, available at https://www.cambridge.org/core/terms. Cambridge University Press, 2010
498 Hendry and Mizon
7.4. Example
A potential example is one in which yt includes real money holdings (m − p)
(M1), inflation p, and real total final expenditure f, whereas zt includes the
variables determining the opportunity cost of holding money. The effect of the
UK Banking Act of 1984, which made the payment of interest on checking
accounts legal, is represented in this framework by a change in the opportunity
cost of holding money from Ra , the return from an alternative asset to money
(e.g., the three-month local authority interest rate) to Rn = Ra − Ro , where Ro
is the own interest rate paid on checking accounts (zero till 1984 3rd quarter).
Money demand equations based on Ra experienced major forecast failure post
1985 as (m − p) increased dramatically, although identical equations using Rn
remained constant. Models of this aspect of the UK economy have been ex-
tensively analyzed by, inter alia, Hendry and Ericsson (1991), Boswijk (1992),
Johansen (1992), Hendry and Mizon (1993), Harris (1995), Paruolo (1996),
Rahbek, Kongsted, and Jørgensen (1999), and Doornik, Hendry, and Nielsen
(1998).
Forecasting models of the DDV form, such as
3
(m − p)T +1 = (m − p)T ,
did not suffer forecast failure from 1985 onwards; equally intercept correc-
tions worked well from that date. A government that was concerned about the
inflationary consequences of the resulting large increase in M1 might have re-
sponded by raising interest rates or income taxes, creating the combined event
of interest in our analysis.
Some representative orders of magnitude may help to indicate when, and
why, the DDV might dominate the VEqCM in forecasting when shifts occur
in deterministic and related terms. In many estimated econometric equations,
residual standard deviations lie between 0.5% and 1.5% of the levels of the
Downloaded from https://www.cambridge.org/core. IP address: 114.4.82.245, on 14 Sep 2017 at 15:44:28, subject to the Cambridge Core
Cambridge Books Online © https://doi.org/10.1017/CBO9780511614491.021
terms of use, available at https://www.cambridge.org/core/terms. Cambridge University Press, 2010
Forecasting in the Presence of Structural Breaks 499
dependent variables (albeit that some fall outside this region). Any deterministic
shift in excess of 2% will, therefore, favor the DDV in short-term forecasts. The
example of the 1984 Banking Act introducing interest-bearing sight deposits
led to a more than 40% increase in holdings of M1 over a couple of years; the
1986 Building Societies Act (the UK analog of Savings and Loans Associations
in the United States), which permitted borrowing on wholesale as well as retail
money markets, induced a doubling in mortgage lending over four years. Such
massive shifts swamp any uncertainty effects.
The UK M1 example also highlights the key practical problems confronting
policy makers, namely, whether a structural break has occurred, and if so
whether it necessitates a policy shift. With hindsight, the fact that the VEqCM
is unaltered once the correct measure of opportunity cost is used shows that the
increase in M1 was primarily a portfolio response (shifting into an asset with a
greatly increased own yield). A policy response would have been counterpro-
ductive and, if based on raising interest rates, probably ineffective in changing
M1 demands as these were based on the differential Rn . At the time, the Bank
of England acted exactly that way and did not seek to intervene: inflation in fact
fell considerably over the next few years, corroborating their judgement. In an
important sense, however, there was a major policy change: instead of raising
Ra as a policy response to M1 growing rapidly, the bank did not do so, in effect
lowering their reaction parameter.
A similar situation more recently was the continued fall in inflation despite
much lower unemployment rates, either occasioned by a structural shift in the
NAIRU or a drop in the impact of any given departure from the NAIRU on
inflation. Such settings correspond to those modeled above.
8. CONCLUSION
We consider the problem of forecasting when models are misspecified for a
nonstationary data generation process, and policy may react to forecast changes.
In such a setting, shifts in the coefficients of deterministic terms in the process
relative to any models thereof (location shifts) induce forecast failure. Moreover,
noncausal statistical devices that are robust to such location shifts – usually by
imposing unnecessary additional unit roots – may provide the best available
forecasts, but need have no policy implications. Since intercept corrections act
like differencing and improve forecasts without altering policy conclusions,
the best forecasting model is not necessarily a good basis for economic policy
analysis.
Conversely, forecast failure in an econometric model need not preclude its
use in policy analysis, since location shifts need not alter policy reactions. If so,
such a model’s implications from changes in policy regimes may be a useful
guide to the outturn. Unfortunately, such “scenario changes” are not helpful in
practice when the model is misforecasting badly.
Consequently, we analyze the impacts of a range of structural breaks and
policy regime shifts on both econometric models and robustified forecasting
Downloaded from https://www.cambridge.org/core. IP address: 114.4.82.245, on 14 Sep 2017 at 15:44:28, subject to the Cambridge Core
Cambridge Books Online © https://doi.org/10.1017/CBO9780511614491.021
terms of use, available at https://www.cambridge.org/core/terms. Cambridge University Press, 2010
500 Hendry and Mizon
devices. The results reveal that neither is immune to both changes: the econo-
metric model fails for location shifts that occurred prior to forecasting, whereas
the robust device does not; the robust device produces biased forecasts when the
policy change then occurs, whereas the econometric model is no worse. Thus,
we investigate correcting the statistical forecast using the econometric model’s
estimate of the “scenario” change resulting from the regime shift. The outcome
is potentially an improvement over either forecast alone, and perhaps even over
intercept correction, and dominates pooling the two forecasts by averaging.
The analysis also highlights the importance of the assumed links between
the parameters of the policy equations and those of the private sector when the
latter are subject to breaks, as well as the role of timing of breaks. Further,
the assumptions that the econometric model coincides with the in-sample DGP
and that the structural break is known are strong, and so practical applications
will not attain the precise offsets found here. For example, an empirical policy
model may be invalid because it embodies the wrong causal attributions; its
target-instrument links are not autonomous; or its parameters are not invariant
to the policy changes under analysis. While some of these problems may not
be revealed in sample, the failure of a policy to produce the anticipated re-
sults would certainly do so – at a cost to society only partly mitigated by the
benefits of improved knowledge of the economy. Nevertheless, even allowing
for in-sample misspecification, the outcome seems likely to be worse than us-
ing either model alone only if the responses to policy are exceptionally poorly
modeled.
Finally, although the timing and form of structural breaks are rarely known,
and hence the appropriate policy reaction is unclear in real time, the components
of our analysis indicate how to proceed, that is, combining econometric with
robust forecasts and calculated scenario effects.
ACKNOWLEDGMENTS
Financial support from the UK Economic and Social Research Council under
grant L138251009, and from the EUI Research Council, is gratefully acknowl-
edged. We are indebted to Anindya Banerjee, Mike Clements, Jurgen Doornik,
Neil Ericsson, John Muellbauer, Neil Shephard, Jim Stock, Ian Walker, and
Ken Wallis for helpful comments.
References
Banerjee, A., D. F. Hendry, and G. E. Mizon (1996), “The Econometric Analysis of
Economic Policy,” Oxford Bulletin of Economics and Statistics, 58, 573–600.
Barrell, R. (2001), “Forecasting the World Economy,” in Understanding Economic
Forecasts (ed. by D. F. Hendry and N. R. Ericsson), Cambridge: MIT Press,
149–69.
Bates, J. M., and C. W. J. Granger (1969), “The Combination of Forecasts,” Operations
Research Quarterly, 20, 451–468. [Reprinted in T.C. Mills (Ed.) (1999), Economic
Forecasting, Cheltenham, UK: Edward Elgar.
Downloaded from https://www.cambridge.org/core. IP address: 114.4.82.245, on 14 Sep 2017 at 15:44:28, subject to the Cambridge Core
Cambridge Books Online © https://doi.org/10.1017/CBO9780511614491.021
terms of use, available at https://www.cambridge.org/core/terms. Cambridge University Press, 2010
Forecasting in the Presence of Structural Breaks 501
Downloaded from https://www.cambridge.org/core. IP address: 114.4.82.245, on 14 Sep 2017 at 15:44:28, subject to the Cambridge Core
Cambridge Books Online © https://doi.org/10.1017/CBO9780511614491.021
terms of use, available at https://www.cambridge.org/core/terms. Cambridge University Press, 2010
502 Hendry and Mizon
Downloaded from https://www.cambridge.org/core. IP address: 114.4.82.245, on 14 Sep 2017 at 15:44:28, subject to the Cambridge Core
Cambridge Books Online © https://doi.org/10.1017/CBO9780511614491.021
terms of use, available at https://www.cambridge.org/core/terms. Cambridge University Press, 2010