Ch-21

Ch-15
Tests

1) OLS - Ordinary least squares (OLS) regression is a statistical method of analysis that estimates the
relationship between one or more independent variables and a dependent variable; the method estimates
the relationship by minimizing the sum of the squares in the difference between the observed and predicted
values of the dependent variable configured as a straight line.
2) CLT - The Central Limit Theorem states that the sampling distribution of the sample means approaches a
normal distribution as the sample size gets larger — no matter what the shape of the population distribution.
3) GLS - Another approach is based on generalized or weighted least squares which is an modification of
ordinary least squares which takes into account the inequality of variance in the observations. Weighted
least squares play an important role in the parameter estimation for generalized linear models.
4) Breusch-Pagan (BP) Test - The Breusch-Pagan-Godfrey Test (sometimes shorted to the Breusch-Pagan test) is
a test for heteroscedasticity of errors in regression. Heteroscedasticity means “differently scattered”; this is
opposite to homoscedastic, which means “same scatter.” Homoscedasticity in regression is an important
assumption; if the assumption is violated, you won’t be able to use regression analysis. Note that the Breush-
Pagan test measures how errors increase across the explanatory variable, Y. The test assumes the error
variances are due to a linear function of one or more explanatory variables in the model.
That means heteroskedasticity could still be present in your regression model, but those errors (if present)
are not correlated with the Y-values.
5) White’s Test of Heteroscedasticity - White’s test is used to test for heteroscedastic (“differently dispersed”)
errors in regression analysis. It is a special case of the (simpler) Breusch-Pagan test.
6) 2SLS - Two-Stage least squares (2SLS) regression analysis is a statistical technique that is used in the analysis
of structural equations. This technique is the extension of the OLS method. It is used when the dependent
variable's error terms are correlated with the independent variables. The first stage of the two-stage least
squares algorithm regresses these variables on the exogenous and instrument variables. The predicted
values of the endogenous variables are then used in the second stage as predictors of the dependent
variable along with any exogenous variables.
7) Maximum Likelihood Estimation - Maximum likelihood, also called the maximum likelihood method, is the
procedure of finding the value of one or more parameters for a given statistic which makes the known
likelihood distribution a maximum. The maximum likelihood estimate for a parameter is denoted .
8) Durbin Watson d-statistic - The Durbin Watson statistic is a number that tests for autocorrelation in the
residuals from a statistical regression analysis. The Durbin-Watson statistic is always between 0 and 4. A
value of 2 means that there is no autocorrelation in the sample. Values from 0 to less than 2 indicate positive
autocorrelation and values from more than 2 to 4 indicate negative autocorrelation. Autocorrelation can be
a significant problem in analyzing historical data if one does not know to look out for it. For instance, since
stock prices tend not to change too radically from one day to another, the prices from one day to the next
could potentially be highly correlated, even though there is little useful information in this observation. In
order to avoid autocorrelation issues, the easiest solution in finance is to simply convert a series of historical
prices into a series of percentage-price changes from day to day.
9) T-statistic - The T Statistic is used in a T test when you are deciding if you should support or reject the null
hypothesis. In statistics, the t-statistic is the ratio of the departure of the estimated value of a parameter
from its hypothesized value to its standard error. It is used in hypothesis testing via Student's t-test. For
example, it is used in estimating the population mean from a sampling distribution of sample means if the
population standard deviation is unknown.
10) P-value - When you run a hypothesis test, you use the T statistic with a p value. The p-value tells you what
the odds are that your results could have happened by chance. Let’s say you and a group of friends score an
average of 205 on a bowling game. You know the average bowler scores 79.7. Should you and your friends
consider professional bowling? Or are those scores a fluke? Finding the t statistic and the probability value
will give you a good idea. More technically, finding those values will give you evidence of a significant
difference between your team’s mean and the population mean (i.e. everyone).

The greater the T, the more evidence you have that your team’s scores are significantly different from
average. A smaller T value is evidence that your team’s score is not significantly different from average. It’s
pretty obvious that your team’s score (205) is significantly different from 79.7, so you’d want to take a look
at the probability value. If the p-value is larger than 5%, the odds are your team getting those scores are due
to chance. Very small (under 5%), you’re onto something: think about going professional.
11) F-statistic - The F-test of overall significance indicates whether your linear regression model provides a better
fit to the data than a model that contains no independent variables. In this post, I look at how the F-test of
overall significance fits in with other regression statistics, such as R-squared. R-squared tells you how well
your model fits the data, and the F-test is related to it.
An F-test is a type of statistical test that is very flexible. You can use them in a wide variety of settings. F-tests
can evaluate multiple model terms simultaneously, which allows them to compare the fits of different linear
models. In contrast, t-tests can evaluate just one term at a time.
12) Chi-square - a statistical method assessing the goodness of fit between a set of observed values and those
expected theoretically.
13) Breusch-Godfrey Test – for testing autocorrelation
14) Autoregressive model - An autoregressive model is when a value from a time series is regressed on previous
values from that same time series. ... The order of an autoregression is the number of immediately preceding
values in the series that are used to predict the value at the present time.
15) Unit root test - In statistics, a unit root test tests whether a time series variable is non-stationary and
possesses a unit root. The null hypothesis is generally defined as the presence of a unit root and the
alternative hypothesis is either stationarity, trend stationarity or explosive root depending on the test used.
16) Dickey-Fuller test - In statistics, the Dickey–Fuller test tests the null hypothesis that a unit root is present in
an autoregressive model.
17) Augmented Dickey-Fuller test - The Augmented Dickey Fuller Test (ADF) is unit root test for stationarity. Unit
roots can cause unpredictable results in your time series analysis.
The Augmented Dickey-Fuller test can be used with serial correlation. The ADF test can handle more complex
models than the Dickey-Fuller test, and it is also more powerful. That said, it should be used with caution
because—like most unit root tests—it has a relatively high Type I error rate.
18) Cointegration - Cointegration naturally arises in economics and finance. In economics, cointegration is most
often associated with economic theories that imply equilibrium relationships between time series variables.
The permanent income model implies cointegration between consumption and income, with consumption
being the common trend. Money demand models imply cointegration between money, income, prices and
interest rates. Growth theory models imply cointegration between income, consumption and investment,
with productivity being the common trend. Purchasing power parity implies cointegration between the
nominal exchange rate and foreign and domestic prices. Covered interest rate parity implies cointegration
between forward and spot exchange rates. The Fisher equation implies cointegration between nominal
interest rates and inflation. The expectations hypothesis of the term structure implies cointegration between
nominal interest rates at different maturities. The equilibrium relationships implied by these economic
 theories are referred to as long-run equilibrium relationships, because the economic forces that act in
response to deviations from equilibriium may take a long time to restore equilibrium. As a result,
cointegration is modeled using long spans of low frequency time series data measured monthly, quarterly or
annually.
19) Hausman test - The Hausman test can be also used to differentiate between fixed effects model and random
effects model in panel data. In this case, Random effects (RE) is preferred under the null hypothesis due to
higher efficiency, while under the alternative Fixed effects (FE) is at least as consistent and thus preferred.