Vous êtes sur la page 1sur 2

THE SIZE OF THE TEST WINDOW

A simulation of a trading strategy is performed on a segment of historical


data of some length or another. For example, a historical simulation of a
moving average system is constructed on S&P 500 futures historical price
data from 1/1/1995 through 12/31/2006.
Definition: The test window is the length of the historical price data on
which a trading strategy is evaluated by historical simulation.
Two main considerations must be satisfied when deciding the size of
the test window: statistical soundness and relevance to the trading system
and to the market.
These two requirements do not stipulate the size of a particular test
window in days, weeks, or months. Instead, they specify a set of guidelines
that can be followed to determine the correct window size for a particular
trading strategy and market. One size does not fit all when it comes to size
of the test window.
The size of the test window will have a significant impact upon the
outcome and reliability of the historical simulation. Its size will influence
parameter selection and trading pace. It will also go a long way toward
determining the statistical reliability, or lack thereof, of the simulation.

Statistical Requirements

The test window must be large enough to generate statistically sound results
and also include a broad sample of data conditions. Statistically
sound means two things. There must be a sufficiently large number of
trades so as to be able to draw meaningful conclusions. The test window
must also be large enough to allow enough degrees of freedom for
the number and length of the variables employed by the trading strategy.
If these guidelines are not followed, the results of the historical simulation
are likely to be deficient in statistical robustness, and are therefore
suspect.

Sample Size and Statistical Error


The standard error is a mathematical concept used in statistical analysis.
We can use an application of this statistic to provide us with some helpful
insight regarding the impact of the trade sample size produced by our
historical simulation on the robustness and precision of the resulting performance
statistics. A large standard error would indicate that the data
points are far from the average and a small standard error indicates that
they are clustered closely around the average. The smaller the standard
error the less an individual winning trade will vary from the average winning
trade.
Standard Error = Standard Deviation/Square Root of the Sample Size
We are going to calculate three standard errors of the average winning
trade based on three different numbers of winning trades.
Let us specify the values to be used in our application of this formula
to calculate the standard error of the mean or average win:
AWt = Average Win
StDev = Standard Deviation
SqRt = Square Root
Nwt = Number of Winning Trades
StandardError = StDev(AWt)/SqRt(Nwt)
Standard error will provide us a measure of reliability of our average
win as a function of the number of winning trades, that is, the sample size.
For example, if the average win is $200 and has a standard error of $50,
then the typical win will be within a range of $150 to $250 ($200 +/ $50.)
The wise strategist will always err to the side of conservatism, so he will
assume that the average win is likely to be $150 (the pessimistic side of the

range of expected wins).


To get an idea of how this plays out with different sample sizes, consider
three examples of standard error based on different trade sample
sizes of 10, 30, and 100. We will assume a standard deviation for our winning
trades of $100.
When our number of wins is 10, the standard error is:
Standard Error = 100/SqRt(10)
Standard Error = 100/3.16
Standard Error = 31.65
With a sample of 10 trades, the standard error is 31.65 rounded to
$32. Plugging this value into our formula, the range of wins is $200 +/
$32 or $168 to $232. With a sample of 30 winning trades, the standard error
is $18. ($18.25 rounded). The expected range then of wins is $200 +/
$18 or $182 to $218. Finally, with a sample of 100 wining trades, the standard
error is $10. The expected range then of wins is $200 +/ $10 or $190
to $210.
From these examples, it is clear that the larger the trade sample size,
the lower the standard error or variance of winning trades. Whereas we
selected the average winning trader for our analysis, this relationship of
larger sample size to smaller standard error will hold true for all performance
statistics produced by a historical simulation. The larger the trade
sample, the smaller the standard error.