Batch38 CSE7315c ChiSquare FDist ANOVA Cor Cov Reg

Inspire…Educate…Transform.
Stat Skills
Chi-Square Distribution, F
Distribution, ANOVA, Correlation
and Covariance
Dr. Anand Jayaraman

Anand.jayaraman@insofe.edu.in
Jan 7, 2018
Thanks to Dr.Sridhar Pappu for the material
The best place for students to learn Applied Engineering http://www.insofe.edu.in
What do I remember from yesterday? Do you mean
other than my way back here?
The best place for students to learn Applied Engineering 2 http://www.insofe.edu.in

Review
• Student t-distribution
– Small samples or 𝜎 unknown
– Depends on degrees of freedom (sample size)
• Confidence Intervals
– Margin of Error
• Hypothesis Testing
– Null Hypothesis and Alternate Hypothesis
– Compute relevant statistic and check if it lies in the
critical region
– Types of Errors in Hypothesis testing

Review
• Two sample t-test for the mean
– Dependent samples – paired t-test
– Unrelated samples – unpaired t-test
• Connection between Hypothesis testing formalism
and confidence intervals

𝟐
𝝌 DISTRIBUTION

Suppose you modeled a situation using a probability distribution
and have an expectation of how things will shape up in the long
run.
But what if, the frequency of observed data is different from

expected data? We are not just comparing the mean of observed
vs expected. We are talking about the entire distribution.
How would you know if the difference is due to normal

fluctuations or if your model was incorrect?
Chi Square Distribution
• Let X ~ N(0,1)
• How does a distribution of X2 look like?
• How does the distribution of sum of squares of

two random picks looks like? i.e X12 + X22

𝝌𝟐 distribution
𝑋−𝜇
𝑅𝑒𝑐𝑎𝑙𝑙 𝑧 =
𝜎
2
(𝑋 − 𝜇)
𝑧2 =
𝜎2
2
𝑧 2 = 𝜒(1)
𝜒 2 distribution is a distribution of the

squared deviates.
The shape depends on number of squared

deviates added together.

𝝌𝟐 distribution
2
𝑋 2 ~𝜒(𝜈) , where 𝜈 represents the degrees
of freedom.
When 𝜈 is greater than 2, the shape of

the distribution is skewed positively
gradually becoming approximately
normal for large 𝜈.

Properties of 𝑿𝟐 random variable
• A 𝑋 2 random variable takes values between 0 and ∞.

• Mean of a 𝜒 2 distribution is 𝜈.
• Variance of a 𝜒 2 distribution is 2𝜈.
• Mode of a 𝜒 2 distribution is 𝜈 − 2.
• The shape of the distribution is skewed to the right.

• As 𝜈 increases, Mean gets larger and the distribution spreads
wider.
• As 𝜈 increases, distribution tends to normal.

Let us say you are running a casino and the slot machines are
causing you headaches. You had designed them with the
following expected probability distribution, with X being the net
gain from each game played.
x -2 23 48 73 98
P(X=x) 0.977 0.008 0.008 0.006 0.001
You collected some statistics and found the following frequency of

peoples’ winnings.
x -2 23 48 73 98
Frequency 965 10 9 9 7

You want to compare the actual frequency with the expected
frequency.
x -2 23 48 73 98
P(X=x) 0.977 0.008 0.008 0.006 0.001
x Observed Expected
Frequency Frequency
-2 965 977
23 10 8
48 9 8
73 9 6
98 7 1
Are these differences significant and if they are, is it just pure

chance?

𝝌𝟐 test to the rescue
𝜒 2 distribution uses a test statistic to look at the difference

between the expected and the actual, and then returns a
probability of getting observed frequencies as extreme.
(𝑂−𝐸)2
𝑋2 = σ ,
where O is the observed frequency and E the
𝐸
expected frequency.

x Observed Expected
Frequency Frequency
-2 965 977
23 10 8
48 9 8
73 9 6
98 7 1
𝑋 2 = 38.272
Is this high?
To find this, we need to look at the 𝜒 2 distribution.

x Observed Expected
Frequency Frequency
-2 965 977
23 10 8
48 9 8
73 9 6
98 7 1
In the above case, we had 5 frequencies to calculate. However,

since the TOTAL expected frequency has to be equal to the TOTAL
observed frequency (RESTRICTION), calculating 4 would give the
5th. Therefore, there are 5-1=4 degrees of freedom.
𝝂 = (number of classes) – (number of restrictions), or
𝝂 = (number of classes) – 1 – (number of parameters being
estimated from sample data)

How do we know the Significance of the difference?
One-tailed test using the upper tail of the
distribution as the critical region.
A test at significance level 𝛼 is written as

𝜒𝛼2 (𝜈). The critical region is to its right. Critical
Region
Higher the value of the test statistic, the 𝝌𝟐𝜶 (𝝂)

bigger the difference between observed
and expected frequencies.

Using 𝝌𝟐 probability tables

Uses of 𝝌𝟐 distribution
• To test goodness of fit.

• To test independence of two variables.

Steps to test Goodness-of-fit
You want to see if there is sufficient evidence at the 5%

significance level to say the slot machines have been rigged.
What are the null and alternate hypotheses?
H0: The slot machine winnings per game follow the described
probability distribution, i.e., they are not rigged.
H1: The slot machine winnings per game do not follow this
distribution.

What are the expected frequencies and degrees of freedom?
x Observed Expected
Frequency Frequency
-2 965 977
23 10 8
48 9 8
73 9 6
98 7 1
ν=4

What is the critical region?
2
𝜒5% 4 = 9.488. This means the critical region is X2 > 9.488.

Is the test statistic inside or outside the critical region?
Since X2 = 38.27 and the critical region is X2 > 9.488, this means X2
is inside the critical region.
Will you accept or reject the null hypothesis?
Reject. There is sufficient evidence to reject the hypothesis that
the slot machine winnings follow the described probability
distribution.
This sort of hypothesis test is called a goodness of fit test. This

test is used whenever you have a set of values that should fit a
distribution, and you want to test whether the data actually does.

x Observed Expected
Frequency Probability
-2 965 0.977
23 10 0.008
48 9 0.008
73 9 0.006
98 7 0.001
Reject the null hypothesis, since p value is less than 5% threshold.

𝝌𝟐 goodness of fit works for any probability
distribution
Distribution Condition 𝜈
You know p (probability of success or the 𝜈 =𝑛−1
proportion of successes in a population)
Binomial
You don’t know p and have to estimate it from 𝜈 =𝑛−2
observed frequencies
You know λ 𝜈 =𝑛−1
Poisson
You don’t know λ, and have to estimate it from 𝜈 =𝑛−2
observed frequencies
You know μ and σ2 𝜈 =𝑛−1
Normal
You don’t know μ and σ2, and have to estimate 𝜈 =𝑛−3
them from observed frequencies

The 108 Medical Emergency Service received calls during 150 5-minute
intervals as follows. Is the distribution Poisson at α=0.01?
# of calls per 5-min interval Frequency
0 18
1 28
2 47
3 21
4 16
5 11
6 or more 9

Step 1: Decide H0 and H1
H0: The frequency distribution is Poisson.
H1: The frequency distribution is not Poisson.

Step 2: Find expected frequencies and degrees of freedom
# of calls Observed Total Calls = # Expected frequency = Probability * Total
per 5-min Frequency of
interval calls/interval X
Frequency 𝑒 −𝜆 𝜆𝑟
𝑃 𝑋=𝑟 =
0 18 0 𝑟!
1 28 28
2 47 94 𝜆 = 𝐴𝑣𝑔 𝑛𝑜. 𝑜𝑓 𝑐𝑎𝑙𝑙𝑠 𝑝𝑒𝑟 5 min 𝑖𝑛𝑡𝑒𝑟𝑣𝑎𝑙
3 21 63 𝑇𝑜𝑡𝑎𝑙 # 𝑜𝑓 𝑐𝑎𝑙𝑙𝑠
=
4 16 64 𝑇𝑜𝑡𝑎𝑙 # 𝑜𝑓 𝑖𝑛𝑡𝑒𝑟𝑣𝑎𝑙𝑠
5 11 55
6 or more 9 54
TOTAL 150 358
358
𝜆= = 2.39
150

Step 2: Find expected frequencies and degrees of freedom
Expected frequencies are obtained by multiplying expected probabilities by
𝑒 −𝜆 𝜆𝑟
the total frequency. 𝑃 𝑋 = 𝑟 = , where λ = 2.39 ν = 7-2 = 5
𝑟!
# of calls per 5- Expected Expected Observed Frequency
min interval Probability Frequency
0 0.0916 13.74 18
1 0.2190 32.85 28
2 0.2617 39.25 47
3 0.2085 31.27 21
4 0.1246 18.69 16
5 0.0595 8.93 11
6 or more 0.0351 5.26 9
TOTAL 150.00 150
How do you find this value?

Step 3: Determine the critical region
2

Step 4: Calculate the test statistic X2
# of calls per 5-min Observed Expected (𝑶 − 𝑬)𝟐
interval Frequency Frequency 𝑬
0 18 13.74 1.32
1 28 32.85 0.72
2 47 39.25 1.53
3 21 31.27 3.37
4 16 18.69 0.39
5 11 8.93 0.48
6 or more 9 5.26 2.66
TOTAL 150 150 10.47
X2 = 10.47

Step 5: See whether the test statistic is in the critical region
X2 = 10.46, which is less than the critical value of 15.086. It is NOT in the
critical region.
Critical
Region
𝟏𝟓. 𝟎𝟖𝟔

Step 6: Make your decision
There is not enough evidence to reject the null hypothesis that the
distribution is Poisson.
Business Implication: Now that 108 services management knows that

the distribution is Poisson, it can plan the staffing of the call centre more
efficiently.

Another Example: 𝝌2 Goodness-of-fit Test
USDA intends to update the nutritional guideline for ideal diet. 90 adults
are shown 3 different diagrams (D1, D2, D3) explaining the proper
proportion of food groups, and asked to identify the easiest to follow.
The group’s picks are given below.
Are there any significant differences in the acceptability of the diagrams

at 5% significance level?
D1: 23
D2: 39
D3: 28

H0: All Diagrams are likely to be equally selected.
H1: There is significant difference in the likelihood of different diagrams

being selected.

Step 2: Find Expected Frequencies and Degrees of Freedom
Leaflet Observed Frequency Expected Frequency (𝑶 − 𝑬)𝟐
𝑬
D1 23 30 1.63
D2 39 30 2.70
D3 28 30 0.13
TOTAL 90 90 4.46
𝜈=2

2
The observed value is 4.46. This is outside the critical region

and so null hypothesis cannot be rejected.

What is the business implication/decision?
Leaflet Observed Frequency Expected Frequency
D1 23 30
D2 39 30
D3 28 30
While the results are not statistically significant, with a large

enough sample of 90 Adults, it can be clearly seen that D2 shows a
lot more preference over the other two.
Statistical insignificance doesn’t mean there is no difference. The

experiment simply may not have the power to detect this. Further
thought is warranted when such frustrating results are obtained.

𝟐
𝝌 independence test
• Two random variables x and y are called independent if the

probability distribution of one variable is not affected by the
presence of another.

𝝌𝟐 independence test
Your casino is facing another issue. You think you are losing
more money from one of the croupiers on the blackjack tables.
You want to test if the outcome of the game is dependent on
which croupier is leading the game.
Croupier A Croupier B Croupier C

Possible Win 43 49 22 Observed
Outcomes Draw 8 2 5 Results
Lose 47 44 30

The process is the same as before. The null hypothesis
assumes that choice of croupier is independent of the
outcome, and is rejected if there is sufficient evidence against
it.
However, a contingency table has to be drawn to find the

expected frequencies using probability.

Croupier A Croupier B Croupier C Total
Win 43 49 22 114
Draw 8 2 5 15
Lose 47 44 30 121
Total 98 95 57 250
𝑇𝑜𝑡𝑎𝑙 𝑊𝑖𝑛𝑠 114

𝑃 𝑊𝑖𝑛 = =
𝐺𝑟𝑎𝑛𝑑 𝑇𝑜𝑡𝑎𝑙 250
𝑇𝑜𝑡𝑎𝑙 𝐴 98
𝑃 𝐴 = =
𝐺𝑟𝑎𝑛𝑑 𝑇𝑜𝑡𝑎𝑙 250
If croupier and the outcome are independent,
𝑇𝑜𝑡𝑎𝑙 𝑊𝑖𝑛𝑠 𝑇𝑜𝑡𝑎𝑙 𝐴
𝑃 𝑊𝑖𝑛 𝑎𝑛𝑑 𝐴 = 𝑋
𝐺𝑟𝑎𝑛𝑑 𝑇𝑜𝑡𝑎𝑙 𝐺𝑟𝑎𝑛𝑑 𝑇𝑜𝑡𝑎𝑙

Win 43 49 22 114
Draw 8 2 5 15
Lose 47 44 30 121
Total 98 95 57 250
𝐸𝑥𝑝𝑒𝑐𝑡𝑒𝑑 𝐹𝑟𝑒𝑞𝑢𝑒𝑛𝑐𝑦 𝑜𝑓 𝑊𝑖𝑛 𝑎𝑛𝑑 𝐴
𝑇𝑜𝑡𝑎𝑙 𝑊𝑖𝑛𝑠 𝑇𝑜𝑡𝑎𝑙 𝐴 𝑇𝑜𝑡𝑎𝑙 𝑊𝑖𝑛𝑠 𝑋 𝑇𝑜𝑡𝑎𝑙 𝐴

= 𝐺𝑟𝑎𝑛𝑑 𝑇𝑜𝑡𝑎𝑙 𝑋 𝑋 =
𝐺𝑟𝑎𝑛𝑑 𝑇𝑜𝑡𝑎𝑙 𝐺𝑟𝑎𝑛𝑑 𝑇𝑜𝑡𝑎𝑙 𝐺𝑟𝑎𝑛𝑑 𝑇𝑜𝑡𝑎𝑙
𝑅𝑜𝑤 𝑇𝑜𝑡𝑎𝑙 𝑋 𝐶𝑜𝑙𝑢𝑚𝑛 𝑇𝑜𝑡𝑎𝑙

=
𝐺𝑟𝑎𝑛𝑑 𝑇𝑜𝑡𝑎𝑙

𝝌𝟐 independence test – Finding expected frequencies
Win 43 49 22 114
Draw 8 2 5 15
Lose 47 44 30 121
Total 98 95 57 250

Win (114*98)/250 (114*95)/250 (114*57)/250
Draw (15*98)/250 (15*95)/250 (15*57)/250
Lose (121*98)/250 (121*95)/250 (121*57)/250

𝝌𝟐 independence test – Calculating X2
Observed Expected (𝑶 − 𝑬)𝟐
𝑬
43 44.688 0.0638
A 8 5.88 0.7644
47 47.432 0.0039
49 43.32 0.7447
B 2 5.7 2.4018
44 45.98 0.0853
22 25.992 0.6131
C 5 3.42 0.7299
30 27.588 0.2109
(𝑂 − 𝐸)2
෍ 𝑂 = 250 ෍ 𝐸 = 250 ෍ = 5.618
𝐸

𝝌𝟐 independence test – Calculating 𝝂
Win
Draw
Lose
We calculated 9 but really need to calculate 4 and figure out the

rest using the total frequency of each row and column. In
general, the degrees of freedom will be (m-1)(n-1) where m is the
number of columns and n the number of rows.

𝝌𝟐 independence test – Determine critical region
Let us say we need 1% significance level to see if the outcome is
independent of the croupier.
2

𝝌𝟐 independence test – Decision
Since calculated X2 = 5.618, it is outside the critical region, and
hence we accept the null hypothesis.
Critical
Region
𝟓. 𝟔𝟏𝟖 𝟏𝟑. 𝟐𝟕𝟕

𝝌𝟐 independence test – using R
Win 43 49 22
Draw 8 2 5
Lose 47 44 30
Since the p-value is greater than the threshold of 1%, we accept the null hypothesis
that the Croupiers are independent.

There is widespread abuse of prescription
drugs for a number of reasons like getting
high, reducing appetite, relieving tension,
feeding an addiction, etc.

The National Council on Alcoholism and Drug Dependence wants to
understand if there is dependence of the type of prescription drug
abuse on the age of the patient. A random poll of 309 patients is
taken as shown below. At α = 0.01, are the two variables
independent? Pain Tranquilizers Stimulants TOTAL
relievers /Sedatives
21-34 26 95 18 139
35-55 41 40 20 101
>55 24 13 32 69
TOTAL 91 148 70 309

H0: Type of prescription drug abuse is independent of age.
H1: Type of prescription drug abuse is not independent of age.

𝐒𝐭𝐞𝐩 𝟐: Find expected frequencies and degrees of freedom
OBSERVED
Pain Tranquilizers/ Stimulants TOTAL
relievers Sedatives
21-34 26 95 18 139
35-55 41 40 20 101
>55 24 13 32 69
TOTAL 91 148 70 309
EXPECTED
Pain Tranquilizers/ Stimulants TOTAL
ν=4
relievers Sedatives
21-34 40.94 66.58 31.49 139
35-55 29.74 48.38 22.88 101
>55 20.32 33.05 15.63 69
TOTAL 91 148 70 309

2

Step 4: Calculate the test statistic X2
Observed Expected (𝑶 − 𝑬)𝟐
𝑬
26 40.94
Pain relievers 41 29.74
24 20.32
95 66.58
Tranquilizers/ 40 48.38
Sedatives 13 33.05
18 31.49
Stimulants 20 22.88
32 15.63
(𝑂 − 𝐸)2
෍ 𝑂 = 309 ෍ 𝐸 = 309 ෍ = 59.41
𝐸

Step 5: See whether the test statistic is in the critical region
X2 = 59.41, which is greater than the critical value of 13.277. It is in the

critical region.
Critical
Region
𝟏𝟑. 𝟐𝟕𝟕

Step 6: Make your decision
There is enough evidence to reject the null hypothesis that the type of
prescription drug abuse and age are independent.
So, age does have an influence on the type of drug abuse.

Testing Hypotheses about a Variance
Sample estimate of population variance is given by
σ (𝑥 − 𝑥)
ҧ 2
𝑖
𝑠2 =
𝑛−1
Multiplying the variance estimate by n-1 gives the sum of squares.
Dividing by population variance gives a random variable distributed as
chi-squared with n-1 degrees of freedom.
2
(𝑛 − 1)𝑠
∴ 𝜒2 =
𝜎2
σ(𝑥𝑖 −𝑥)ҧ 2
Recall 𝜒2 =
𝜎2
See https://onlinecourses.science.psu.edu/stat414/node/174 for proof

A manufacturing company produces bearings of 2.65 cm in diameter. A
major customer requires that the variance in diameter be no more than
0.001 cm2. The manufacturer tests 20 bearings using a precise
instrument and gets the below values. Assuming the diameters are
normally distributed, can the population of these bearings be rejected
due to high variance at 1% significance level?
Data: 2.69, 2.66, 2.64, 2.59, 2.62, 2.63, 2.69, 2.66, 2.63, 2.65, 2.57, 2.63,
2.70, 2.71, 2.64, 2.65, 2.59, 2.66, 2.62, 2.57

What are null and alternate hypotheses?
𝐻0 : 𝜎 2 ≤ 0.001; 𝐻1 : 𝜎 2 > 0.001
How many degrees of freedom?
Since n=20, df=19.

What is the critical region?
2
𝜒0.01,19 = 36.191

What is the observed 𝜒 2 value?
(𝑛 − 1)𝑠 2 19 ∗ 0.001621
2
𝜒 = = = 30.8
𝜎 2 0.001
2 Critical
Is it in critical region? 𝜒0.01,19 = 36.191
Region
No.
𝟑𝟔. 𝟏𝟗𝟏
Will you reject or fail to reject the null hypothesis?
Fail to reject.
What is the business decision?
The population variance is within specification limits required by the
customer and hence the bearings can be shipped.

F DISTRIBUTION

F distribution
• 𝜒 2 was useful in testing hypotheses about a single population
variance.
• Sometimes we want to test hypotheses about difference in
variances of two populations:
– Is the variance of 2 stocks the same?
– Do parts manufactured in 2 shifts or on 2 different machines or in 2
batches have the same variance or not?
– Is the powder mix for tablet granulations homogeneous?
– Is there variability in assayed drug blood levels in a bioavailability study?
– Is there variability in the clinical response to drug therapy of two samples?

F distribution
𝑠12 𝑒𝑠𝑡.𝜎12
• Ratio of 2 variance estimates: 𝐹 = =
𝑠22 𝑒𝑠𝑡.𝜎22
• Ideally, this ratio should be about 1 if 2 samples come from
the same population or from 2 populations with same
variance, but sampling errors cause variation.
(𝑛−1)𝑠2
• Recall = 𝜒2 .
So, F is also a ratio of 2 chi-squares,
𝜎2
each divided by its degrees of freedom, i.e.,
𝜒2
𝜈1
𝜈1
𝐹= 𝜒2𝜈2
𝜈2

F distribution

Hypothesis test for 2 sample variances
A machine produces metal sheets with 22mm thickness. There is variability in
thickness due to machines, operators, manufacturing environment, raw
material, etc. The company wants to know the consistency of two machines
and randomly samples 10 sheets from machine 1 and 12 sheets from machine
2. Thickness measurements are taken. Assume sheet thickness is normally
distributed in the population.
The company wants to know if the variance from each sample comes from the
same population variance (population variances are equal) or from different
population variances (population variances are unequal).
How do you test this?

Data Machine 1 Machine 2
22.3 21.9 22.0 21.7
21.8 22.4 22.1 21.9
22.3 22.5 21.8 22.0
21.6 22.2 21.9 22.1
21.8 21.6 22.2 21.9
22.0 22.1
𝑠12 = 0.11378 𝑛 = 10 𝑠22 = 0.02023 𝑛 = 12
𝑠12 0.11378
Ratio of sample variances, 𝐹 = = = 5.62
𝑠22 0.02023

What are null and alternate hypotheses?
𝐻0 : 𝜎12 = 𝜎22 ; 𝐻1 : 𝜎12 ≠ 𝜎22
Is it a one-tailed test or a two-tailed test?
Two-tailed.
What are numerator and denominator degrees of freedom?
𝜈1 = 10 − 1 = 9; 𝜈2 = 12 − 1 = 11

Reading an F-table.
1
𝐹0.025,9,11 = 3.5879; 𝐹0.975,9,11 = = 0.2787
𝐹0.025,9,11

1
𝐹0.025,9,11 = 3.5879; 𝐹0.975,9,11 = = 0.2787; 𝐹𝑜𝑏𝑠𝑒𝑟𝑣𝑒𝑑 = 5.62
𝐹0.025,9,11
Will you reject the null hypothesis or not?
Reject. Population variances are not equal.

What are the business implications?
Variance in machine 1 is higher than in machine 2. Machine 1 needs to

be inspected for any issues.

Applications of F Distribution
• Test for equality of variances.
• Test for differences of means in ANOVA.
• Test for regression models (slopes relating one continuous
variable to another, e.g., Entrance exam scores and GPA)

Relations among Distributions – Children of the Normal
• 𝜒 2 is drawn from the normal – N(0,1) deviates squared and

summed.
• F is the ratio of 2 chi-squares, each divided by its df.
• A 𝜒 2 divided by its df is a variance estimate, i.e., a sum of
squares divided by the degrees of freedom.
• F=t2. If you square t, you get an F with 1 df in the
2
numerator, i.e., 𝑡(𝜈) = 𝐹(1,𝑣)
(see http://www-ist.massey.ac.nz/dstirlin/CAST/CAST/HsimpleAnova/simpleAnova3.html for
an example using this relationship)

ANOVA

The purpose of ANOVA (Analysis of Variance) is to test for
significant differences between means of different groups.
Let us say 3 groups of students were given 3 different memory

pills and their scores in an exam recorded. We want to
understand if the differences are due to within group differences
or between group differences.

Group 1 Group 2 Group 3
3 5 5
2 3 6
1 4 7
𝑋ത1 = 2 𝑋ത2 = 4 𝑋ത3 = 6
3+2+1+5+3+4+5+6+7
𝑋ധ = =4
9
𝑇𝑜𝑡𝑎𝑙 𝑆𝑢𝑚 𝑜𝑓 𝑆𝑞𝑢𝑎𝑟𝑒𝑠, 𝑆𝑆𝑇

= (3 − 4)2 +(2 − 4)2 +(1 − 4)2 +(5 − 4)2 +(3 − 4)2 +(4 − 4)2 +(5 − 4)2 +(6 − 4)2 +(7 − 4)2 = 𝟑𝟎
(Do you see any similarities with the formula for variance?)
When there are m groups and n members in each group, the degrees of freedom are 𝒎𝒏 − 𝟏,
since we can calculate one member knowing the overall mean.
How much of this variation is coming from within the groups and how much from between the
groups?

3 5 5
2 3 6
1 4 7
𝑋ത1 = 2 𝑋ത2 = 4 𝑋ത3 = 6
3+2+1+5+3+4+5+6+7
𝑋ധ = =4
9
𝑇𝑜𝑡𝑎𝑙 𝑆𝑢𝑚 𝑜𝑓 𝑆𝑞𝑢𝑎𝑟𝑒𝑠 𝑊𝑖𝑡ℎ𝑖𝑛, 𝑆𝑆𝑊

= (3 − 2)2 +(2 − 2)2 +(1 − 2)2 +(5 − 4)2 +(3 − 4)2 +(4 − 4)2 +(5 − 6)2 +(6 − 6)2 +(7 − 6)2 = 𝟔
When there are m groups and n members in each group, the degrees of freedom are 𝒎(𝒏 −
𝟏), since we can calculate one member knowing the group mean.
𝑇𝑜𝑡𝑎𝑙 𝑆𝑢𝑚 𝑜𝑓 𝑆𝑞𝑢𝑎𝑟𝑒𝑠 𝐵𝑒𝑡𝑤𝑒𝑒𝑛, 𝑆𝑆𝐵 = 3(2 − 4)2 + 3(4 − 4)2 +3(6 − 4)2 = 𝟐𝟒
When there are m groups, the degrees of freedom are 𝒎 − 𝟏.
SST = SSW + SSB

Also, for degrees of freedom, 𝒎𝒏 − 𝟏 = 𝒎 𝒏 − 𝟏 + (𝒎 − 𝟏)

3 5 5
2 3 6
1 4 7
𝑋ത1 = 2 𝑋ത2 = 4 𝑋ത3 = 6
3+2+1+5+3+4+5+6+7
𝑋ധ = =4
9
Given that mean of group 3 is highest and that of group 1 lowest, can we conclude that the
pills given to group 3 had a larger impact or is it just variation within the group?
Let us have a null hypothesis that the population means of the 3 groups from which the
samples were taken have the same mean, i.e., the pills do not have an impact on the
performance in the exam. 𝜇1 = 𝜇2 = 𝜇3 . Let us also have a significance level, 𝛼 = 0.10.
What is the alternate hypothesis?
The pills have an impact on performance.

3 5 5
2 3 6
1 4 7
𝑋ത1 = 2 𝑋ത2 = 4 𝑋ത3 = 6
3+2+1+5+3+4+5+6+7
𝑋ധ = =4
9
The test statistic used is F-statistic.

𝑆𝑆𝐵 24
𝑑𝑓
𝐹 − 𝑠𝑡𝑎𝑡𝑖𝑠𝑡𝑖𝑐 = 𝑆𝑆𝐵 = 2 = 12
𝑆𝑆𝑊 6
𝑑𝑓𝑆𝑆𝑊 6
If numerator is much bigger than the denominator, it means variation between means has
bigger impact than variation within, thus rejecting the null hypothesis.

The df are 2 for numerator and 6 for denominator.
Fc, the critical F-statistic, therefore, is 3.46330. 12 is way higher than this and hence we reject
the null hypothesis. That means the pills do have an impact on the performance.

ANOTHER EXAMPLE
A wind turbine manufacturer is testing 3 different designs of the

turbines. It picks 3 different sites in the same district to install each
model of the turbine. The mean power output (MW) over the day is
measured for 9 consecutive days in each of the sites.
We want to understand if the differences are due to within-group

differences or between-group differences.

Model 1 Model 2 Model 3
3 4 3 3 5 7 5 5 5
2 5 5 6 7 6 6 5 7
4 3 3 4 4 8 7 6 6
𝑋ത1 = 3.56 𝑀𝑊 𝑋ത2 = 5.56 𝑀𝑊 𝑋ത3 = 5.78 𝑀𝑊
134
𝑋ധ = = 4.96 𝑀𝑊
27
𝑇𝑜𝑡𝑎𝑙 𝑆𝑢𝑚 𝑜𝑓 𝑆𝑞𝑢𝑎𝑟𝑒𝑠, 𝑆𝑆𝑇

= (2 − 4.96)2 +5 ∗ 3 − 4.96 2 + 4 ∗ 4 − 4.96 2 + 7 ∗ 5 − 4.96 2 + 5 ∗ 6 − 4.96 2
+ 4 ∗ 7 − 4.96 2 + 8 − 4.96 2 = 𝟔𝟐. 𝟗𝟔
𝑇𝑜𝑡𝑎𝑙 𝑆𝑢𝑚 𝑜𝑓 𝑆𝑞𝑢𝑎𝑟𝑒𝑠 𝑊𝑖𝑡ℎ𝑖𝑛, 𝑆𝑆𝑊

= (2 − 3.56)2 +4 ∗ (3 − 3.56)2 + 2 ∗ 4 − 3.56 2 + 2 ∗ 5 − 3.56 2 + 3 − 5.56 2 + 2 ∗ 4 − 5.56 2
+ 5 − 5.56 2 + 2 ∗ 6 − 5.56 2 + 2 ∗ 7 − 5.56 2 + 8 − 5.56 2 + 4 ∗ 5 − 5.78 2 + 3 ∗ 6 − 5.78 2
+ 2 ∗ 7 − 5.78 2 = 𝟑𝟔. 𝟎𝟎
𝑇𝑜𝑡𝑎𝑙 𝑆𝑢𝑚 𝑜𝑓 𝑆𝑞𝑢𝑎𝑟𝑒𝑠 𝐵𝑒𝑡𝑤𝑒𝑒𝑛, 𝑆𝑆𝐵

= 9 ∗ (3.56 − 4.96)2 + 9 ∗ 5.56 − 4.96 2 + 9 ∗ 5.78 − 4.96 2 = 𝟐𝟔. 𝟗𝟔

What is the null hypothesis?
All 3 sites from which the samples were taken have the same
population-mean, i.e., the turbine design does not have an impact
on the power production. That is 𝜇1 = 𝜇2 = 𝜇3 .
Let us also specify a significance level, 𝛼 = 0.10.
What is the alternate hypothesis?

The turbine design does impact the power output.

Compute the statistic
𝑆𝑆𝐵 26.96
𝑑𝑓𝑆𝑆𝐵
𝐹 − 𝑠𝑡𝑎𝑡𝑖𝑠𝑡𝑖𝑐 = = 2 = 8.99
𝑆𝑆𝑊 36
𝑑𝑓𝑆𝑆𝑊 24
If numerator is much bigger than the denominator, it means

variation between means has bigger impact than variation within,
thus rejecting the null hypothesis.

The df are 2 for numerator and 24 for
denominator.
Fc, the critical F-statistic, therefore, is

2.53833.
Our F=8.99 is way higher than this and

hence we reject the null hypothesis.
That means the turbine design does
have an impact on the power
production.

Anova: Single Factor
SUMMARY
Groups Count Sum Average Variance
Group1 9 32 3.55556 1.027778
Group2 9 50 5.55556 2.777778
Group3 9 52 5.77778 0.694444
ANOVA
Source of Variation SS df MS F P-value F crit
Between Groups 26.963 2 13.4815 8.987654 0.0012 2.5383
Within Groups 36 24 1.5
Total 62.963 26
Tree Diagram Taxonomy of Inferential Techniques
Confidence Intervals Hypothesis Tests
(Estimation)
Variable
1 Independent
Two-way
ANOVA
Means
One-way
ANOVA
z for p1-p2 𝜒2
𝜒2 z for p1-p2
z z
F for 𝜎1 2 − 𝜎2 2
z t
z t Paired t-test
t for Diff
z for µ1-µ2 t for µ1-µ2 z for µ1-µ2 t for µ1-µ2

SIMPLE LINEAR REGRESSION

Speed vs Stopping distance
The “cars” dataset in R contains 50 pairs of datapoints

for Speed(mph) vs stopping distance(ft), that were collected in 1920
See: https://stat.ethz.ch/R-manual/R-devel/library/datasets/html/cars.html

• Independent variable (explanatory) – Speed (mph) – Plotted on X-axis
• Dependent variable (response) – Stopping distance(ft) – Plotted on Y-axis

• Another car with the same speed, might not
have the same stopping distance
• x is known (No uncertainty)
• y has uncertainty (It’s a sample picked from
some unknown distribution)

We need to minimize errors.
y=a+bx
Line of best fit is the line that
Actual Values minimizes all the distances
(residuals) between the Actual and
Actual and Estimated values
Estimated
values of y for Predicted values
the same x based on line of
best fit
We could do that by minimizing σ(𝑦𝑖 − 𝑦ෝ𝑖 ), where 𝑦𝑖 is the actual

value and 𝑦ෝ𝑖 its estimate. (𝑦𝑖 − 𝑦ෝ𝑖 ) is also known as the residual.

We need to minimize errors.
Just as we did when finding variance, we find the sum of squared
errors or SSE. Note in variance calculations, we subtract mean, 𝑦,
ത not 𝑦ෝ𝑖 .
𝑆𝑆𝐸 = ෍(𝑦𝑖 − 𝑦ෝ𝑖 )2
The value of b, the slope, that minimizes the SSE is given by

σ( 𝑥𝑖 − 𝑥ҧ 𝑦𝑖 − 𝑦ത )
𝑏=
σ(𝑥𝑖 − 𝑥)ҧ 2

The value of b, the slope, that minimizes the SSE is given by
σ( 𝑥𝑖 − 𝑥ҧ 𝑦𝑖 − 𝑦ത )
𝑏=
σ(𝑥𝑖 − 𝑥)ҧ 2
The intercept a can be computed by requiring the fitted line pass
through (𝑥,ҧ 𝑦).
ത Substituting in the equation y = a + bx, we can find
a.
This method of fitting the line of best fit is called least squares
regression.

Speed vs Stopping distance
y = 3.93 x – 17.58

Covariance
σ( 𝑥 − 𝑥ҧ 𝑦 − 𝑦ത )
𝑏=
σ(𝑥 − 𝑥)ҧ 2
σ(𝑥−𝑥)(𝑦−
ҧ ത
𝑦)
Covariance 𝑠𝑥𝑦 =
𝑛−1
Slope Coefficient can be expressed in terms of covariance & Standard

deviation 𝑠𝑥
𝑠𝑥𝑦
𝑏= 2
𝑠𝑥

Covariance
σ(𝑥 − 𝑥)(𝑦
ҧ − 𝑦)
ത
𝑠𝑥𝑦 =
𝑛−1
• If both x and y are large distance away from their respective

means, the resulting covariance will be even larger.
– The value will be positive if both are below the mean or both are above.
– If one is above and the other below, the covariance will be negative.
• If even one of them is very close to the mean, the covariance will
be small.
• Cov(x,x)=Var(x)
Covariance: Speed vs Stopping distance
Covariance and Correlation
σ(𝑥 − 𝑥)(𝑦
ҧ − 𝑦)
ത
𝑠𝑥𝑦 =
𝑛−1
• The value of covariance itself doesn’t say much. It only shows
whether the variables are moving together (positive value) or
opposite to each other (negative value).
• To know the strength of how the variables move together,
covariance is standardized to the dimensionless quantity,
correlation.
𝑠𝑥𝑦
𝑟=
𝑠𝑥 𝑠𝑦
Correlation Coefficient
Correlation coefficient, r, is a number between -1 and 1 and tells us
how well a regression line fits the data.
𝑏𝑠𝑥
𝑟=
𝑠𝑦
It gives the strength and direction of the relationship between two

variables.
Img source: https://www.mathsisfun.com/data/correlation.html
Access date: 8/1/2017
Correlation Coefficient
𝑏𝑠𝑥
𝑟=
𝑠𝑦
where b is the slope of the line of best fit, 𝑠𝑥 is the standard

deviation of the x values in the sample, and 𝑠𝑦 is the standard
deviation of the y values in the sample.
σ(𝑥−𝑥)ҧ 2 ത 2
σ(𝑦−𝑦)
𝑠𝑥 = and 𝑠𝑦 =
𝑛−1 𝑛−1
Correlation: Speed vs Stopping distance
y = 3.93 x – 17.58
THE STORY
We started by looking at various types of data
(Categorical/Numerical) and we sought methods to describe the
central tendency of the data. We discussed Mean, Median and
Mode.
We realized to understand the data better, along with the central-

tendency, we need to understand the spread. We looked at
Range, Interquartile Range, Variance and Standard Deviation.
We discussed Box and Whisker plots as a way to represent both

central-tendency and the spread in the data.
We moved on to basic Probability theory and described rules of
probabilities using Venn Diagrams. We talked about conditional
probabilities and that led to Bayes Theorem.
We then looked at some of the commonly occurring Probability

Distributions and their properties, and looked at the expected
values, their variance and the probabilities of various possible
outcomes.
Probability Distributions
From: http://blog.cloudera.com/blog/2015/12/common-probability-distributions-the-data-scientists-crib-sheet/
From: http://www.math.wm.edu/~leemis/chart/UDR/UDR.html
Then we saw how the Sampling Distributions of Means tend to
normal distribution irrespective of how the population is
distributed and learned how to describe populations based on
available sample data.
We then looked at Confidence Intervals to properly describe the

conclusions about populations based on samples.
Then we studied Hypothesis Tests as an alternative inferential

technique to prove our claims. Of course, there are errors in these
tests too.
We then studied various statistical tests to test hypotheses.
We looked at how to analyze results and find differences between

what we expect and what we get, through 𝜒 2 Distributions
(goodness-of-fit).
We studied ANOVA, 2-sample t-tests and F test as a means of

understanding significant differences between means and
variances.
HYDERABAD BENGALURU
2nd Floor, Jyothi Imperial, Vamsiram Builders, Old L77, 15th Cross Road, 3rd Main Road, Sector 6,
Mumbai Highway, Gachibowli, Hyderabad - 500 032 HSR Layout, Bengaluru – 560 102
+91-9701685511 (Individuals) +91-9502334561 (Individuals)
+91-9618483483 (Corporates) +91-9502799088 (Corporates)
Social Media
Web: http://www.insofe.edu.in
Facebook: https://www.facebook.com/insofe
Twitter: https://twitter.com/Insofeedu
YouTube: http://www.youtube.com/InsofeVideos
SlideShare: http://www.slideshare.net/INSOFE
LinkedIn: http://www.linkedin.com/company/international-school-of-engineering
This presentation may contain references to findings of various reports available in the public domain. INSOFE makes no representation as to their accuracy or that the organization
subscribes to those findings.

Batch38 CSE7315c ChiSquare FDist ANOVA Cor Cov Reg

Transféré par

Informations du document

Description originale:

Titre original

Copyright

Formats disponibles

Partager ce document

Partager ou intégrer le document

Options de partage

Avez-vous trouvé ce document utile ?

Ce contenu est-il inapproprié ?

Droits d'auteur :

Formats disponibles

Batch38 CSE7315c ChiSquare FDist ANOVA Cor Cov Reg

Transféré par

Droits d'auteur :

Formats disponibles

Inspire…Educate…Transform.

Dr. Anand Jayaraman

The best place for students to learn Applied Engineering 2 http://www.insofe.edu.in

The best place for students to learn Applied Engineering 3 http://www.insofe.edu.in

The best place for students to learn Applied Engineering 4 http://www.insofe.edu.in

The best place for students to learn Applied Engineering 5 http://www.insofe.edu.in

But what if, the frequency of observed data is different from

How would you know if the difference is due to normal

• How does a distribution of X2 look like?

• How does the distribution of sum of squares of

The best place for students to learn Applied Engineering 7 http://www.insofe.edu.in

𝜒 2 distribution is a distribution of the

The shape depends on number of squared

The best place for students to learn Applied Engineering 9 http://www.insofe.edu.in

When 𝜈 is greater than 2, the shape of

The best place for students to learn Applied Engineering 10 http://www.insofe.edu.in

• A 𝑋 2 random variable takes values between 0 and ∞.

• The shape of the distribution is skewed to the right.

The best place for students to learn Applied Engineering 11 http://www.insofe.edu.in

You collected some statistics and found the following frequency of

The best place for students to learn Applied Engineering 12 http://www.insofe.edu.in

Are these differences significant and if they are, is it just pure

The best place for students to learn Applied Engineering 13 http://www.insofe.edu.in

𝜒 2 distribution uses a test statistic to look at the difference

The best place for students to learn Applied Engineering 14 http://www.insofe.edu.in

The best place for students to learn Applied Engineering 15 http://www.insofe.edu.in

In the above case, we had 5 frequencies to calculate. However,

The best place for students to learn Applied Engineering 16 http://www.insofe.edu.in

A test at significance level 𝛼 is written as

Higher the value of the test statistic, the 𝝌𝟐𝜶 (𝝂)

The best place for students to learn Applied Engineering 17 http://www.insofe.edu.in

The best place for students to learn Applied Engineering 18 http://www.insofe.edu.in

• To test goodness of fit.

The best place for students to learn Applied Engineering 19 http://www.insofe.edu.in

You want to see if there is sufficient evidence at the 5%

The best place for students to learn Applied Engineering 20 http://www.insofe.edu.in

The best place for students to learn Applied Engineering 21 http://www.insofe.edu.in

The best place for students to learn Applied Engineering 22 http://www.insofe.edu.in

This sort of hypothesis test is called a goodness of fit test. This

The best place for students to learn Applied Engineering 23 http://www.insofe.edu.in

Reject the null hypothesis, since p value is less than 5% threshold.

The best place for students to learn Applied Engineering 24 http://www.insofe.edu.in

The best place for students to learn Applied Engineering 25 http://www.insofe.edu.in

The best place for students to learn Applied Engineering 26 http://www.insofe.edu.in

The best place for students to learn Applied Engineering 27 http://www.insofe.edu.in

The best place for students to learn Applied Engineering 28 http://www.insofe.edu.in

How do you find this value?

The best place for students to learn Applied Engineering 29 http://www.insofe.edu.in

The best place for students to learn Applied Engineering 30 http://www.insofe.edu.in

The best place for students to learn Applied Engineering 31 http://www.insofe.edu.in

The best place for students to learn Applied Engineering 32 http://www.insofe.edu.in

Business Implication: Now that 108 services management knows that

The best place for students to learn Applied Engineering 33 http://www.insofe.edu.in

Are there any significant differences in the acceptability of the diagrams

The best place for students to learn Applied Engineering 34 http://www.insofe.edu.in

H1: There is significant difference in the likelihood of different diagrams

The best place for students to learn Applied Engineering 35 http://www.insofe.edu.in

The best place for students to learn Applied Engineering 36 http://www.insofe.edu.in

The observed value is 4.46. This is outside the critical region

The best place for students to learn Applied Engineering 37 http://www.insofe.edu.in

While the results are not statistically significant, with a large

Statistical insignificance doesn’t mean there is no difference. The