Analysis of variance

ANOVA

Group Members….

Robin a/l Ganaselvam M20151000353

Prabakaran Subramaniam M20151000356

ANOVA is used to test hypotheses about differences between more than two means.

When there are more than two means, it is possible to compare each mean with each other mean using t-tests.

ANOVA can be used to test differences among several means for significance without increasing the Type I error rate.

The ANOVA Procedure:This is the ten step procedure for analysis of variance:

1.Description of data2.Assumption: Along with the assumptions, we represent the model for each design we discuss.3. Hypothesis4.Test statistic5.Distribution of test statistic6.Decision rule7.Calculation of test statistic: The results of the arithmetic calculations will be summarized in a table called the analysis of variance (ANOVA) table. The entries in the table make it easy to evaluate the results of the analysis.8.Statistical decision9.Conclusion10.Determination of p value

ONE-WAY ANOVA-Completely Randomized Design (CRD)

One-way ANOVA: It is the simplest type of ANOVA, in which

only one source of variation, or factor, is investigated.

It is an extension to three or more samples of the t test procedure for use with two independent samples

In another way t test for use with two independent samples is a special case of one-way analysis of variance.

Advantages: Very simple: Reduce the experimental error to a great extent. We can reduce or increase some treatments. Suitable for laboratory experiment.Disadvantages: Design is not suitable if the experimental

units are not homogeneous. Design is not so much efficient and sensitive as compared

to others. Local control is completely neglected. Not suitable for field experiment.

• ANOVA is comparison of means. Each possible value of a factor or combination of factor is a treatment.

• The ANOVA is a powerful and common statistical procedure in the social sciences. It can handle a variety of situations.

Why ANOVA instead of multiple t-tests? If you are comparing means between more than

two groups, why not just do several two sample t-tests to compare the mean from one group with the mean from each of the other groups?

Before ANOVA, this was the only option available to compare means between more than two groups.

The problem with the multiple t-tests approach is that as the number of groups increases, the number of two sample t-tests also increases.

As the number of tests increases the probability of making a Type I error also increases.

ANOVA HypothesesThe Null hypothesis for ANOVA is that the means for

all groups are equal:

The Alternative hypothesis for ANOVA is that at least two of the means are not equal.

The test statistic for ANOVA is the ANOVA F-statistic.

koH ....: 321

SST = SSW + SSB This partitioned relationship is also true for the squared

differences: The variability between each observation and the overall (or

grand) mean is measured by the ‘sum of squares total’ (SST) The variability within groups is measured by the ‘sum of

squares within’ (SSW). ▪ MSW = SSW/(n-k)

The variability between groups is measured by the ‘sum of squares between’ (SSB).

▪ MSB = SSB/(k-1)

One Way ANOVA

25 mg (X1) 50mg (X2) 100mg (X3)

3 5 6

5 5 10

3 1 10

1 5 6

Mean= 12/4 = 3 Mean= 16/4 = 4 Mean= 32/4 = 8

Three different dosages drug

X1 x² ss X2 x² Ss X3 x² ss

3 9 (3-3) ² = 0 5 25 (4-5) ² = 1 6 36 (8-6) ² =4

5 25 (3-5) ² = 4 5 25 (4-5) ² = 1 10 100 (8-10) ² = 4

3 9 (3-3) ² = 0 1 1 (4-1) ² = 9 10 100 (8-10) ² = 4

1 1 (3-1) ² = 4 5 25 (4-5) ² = 1 6 36 (8-6) ² = 4

Σx² = 44 ss= 8 Σx²= 76 Ss = 12 Σx²= 272 Ss=16

Σx² = 44 + 76 + 272 = 392K = 3N = 12G = 60Σx² = 392

df Between = K- 1 3 – 1= 2

df Within = N – K 12 – 3 = 9

Degree of freedom (df)

SSW : 8 + 12 + 16 = 36

Sum of square within (sswithin)

SSt : Σx² - G² = 392 - 60 = 92 N 12

Sum of square total (sstotal)

SSb :SS total – SS within92 – 36 = 56

Sum of square between (ssbetween)

Square SS Df s² F-ratio

Between 56 2 28 7

Within 36 9 4

Total 92

÷

÷

÷

Construct a source table……

Two-way ANOVA• The two-way ANOVA compares the mean differences

between groups that have been split on two independent variables (called factors). The primary purpose of a two-way ANOVA is to understand if there is an interaction between the two independent variables on the dependent variable.

When to use what method• When you in addition to the main observation have some observations that can be used to pair or block objects, and want to compare groups, and assumption of normally distributed independent errors is OK: – For two groups, use paired-data t-test– For three or more groups, we can use two-way

ANOVA

Two-way ANOVA (without interaction)• In two-way ANOVA, data fall into categories in two different ways: Each observation can be placed in a table.

• Example: Both doctor and type of treatment should influence outcome.

• Sometimes we are interested in studying both categories, sometimes the second category is used only to reduce unexplained variance. Then it is called a blocking variable

Sums of squares for two-way ANOVA• Assume K categories, H blocks, and assume one observation xij for each category i and each block j block, so we have n=KH observations. – Mean for category i: – Mean for block j: – Overall mean:

ix

jxx

Sums of squares for two-way ANOVA2

1

( )K

ii

SSG H x x

2

1

( )H

jj

SSB K x x

2

1 1

( )K H

ij i ji j

SSE x x x x

2

1 1

( )K H

iji j

SST x x

SSG SSB SSE SST

ANOVA table for two-way dataSource of variation

Sums of squares

Deg. of freedom

Mean squares F ratio

Between groups SSG K-1 MSG= SSG/(K-1) MSG/MSE

Between blocks SSB H-1 MSB= SSB/(H-1) MSB/MSE

Error SSE (K-1)(H-1) MSE= SSE/(K-1)(H-1)

Total SST n-1

Test for between groups effect: compare to

Test for between blocks effect: compare to

MSGMSEMSBMSE

1,( 1)( 1)K K HF

1,( 1)( 1)H K HF

Two-way ANOVA (with interaction)• The setup above assumes that the blocking variable influences outcomes in the same way in all categories (and vice versa)

• We can check if there is interaction between the blocking variable and the categories by extending the model with an interaction term

Sums of squares for two-way ANOVA (with interaction)• Assume K categories, H blocks, and assume L observations xij1, xij2, …,xijL for each category i and each block j block, so we have n=KHL observations. – Mean for category i: – Mean for block j:– Mean for cell ij: – Overall mean:

ix

jx

xijx

Sums of squares for two-way ANOVA (with interaction)

2

1

( )K

ii

SSG HL x x

2

1

( )H

jj

SSB KL x x

2

1 1

( )K H

ij i ji j

SSI L x x x x

2

1 1 1

( )K H L

ijli j l

SST x x

SSG SSB SSI SSE SST

2

1 1 1

( )K H L

ijl iji j l

SSE x x

ANOVA table for two-way data (with interaction)

Source of variation

Sums of squares

Deg. of freedom

Mean squares F ratio

Between groups SSG K-1 MSG= SSG/(K-1) MSG/MSE

Between blocks SSB H-1 MSB= SSB/(H-1) MSB/MSE

Interaction SSI (K-1)(H-1) MSI= SSI/(K-1)(H-1)

MSI/MSE

Error SSE KH(L-1) MSE= SSE/KH(L-1)

Total SST n-1

Test for interaction: compare MSI/MSE with

Test for block effect: compare MSB/MSE with

Test for group effect: compare MSG/MSE with 1, ( 1)K KH LF

1, ( 1)H KH LF

( 1)( 1), ( 1)K H KH LF

Notes on ANOVA• All analysis of variance (ANOVA) methods are based on the assumptions of normally distributed and independent errors

• The same problems can be described using the regression framework. We get exactly the same tests and results!

• There are many extensions beyond those mentioned

THANK YOU FOR THE

ATTENTION GIVEN