SMHS ANOVA

Scientific Methods for Health Sciences - Analysis of Variance (ANOVA)

Overview

Analysis of Variance (ANOVA) is the common method applied to analyze the differences between group means. In ANOVA, we divide the observed variance into components attributed to different sources of variation. It is widely used statistical technique which provides a statistical test of whether or not the means of several groups are equal, that is ANOVA can be thought as a generalized t-test for more than 2 groups (ANOVA results in the case of 2 groups coincide with the corresponding results of a 2-sample independent t-test). Here we introduce the ANOVA method, specifically one-way ANOVA and two-way ANOVA, with examples.

Motivation

In the previous two-sample inference, we applied the independent t-test to compare two independent group means. What if we want to compare k (k>2) independent samples? In this case, we will need to decompose the entire variation into components allowing us to analyze the variance of the entire dataset. Suppose 5 varieties of products are tested for further study. A filed was divided into 20 plots, with each variety planted in four plots. The measurements are shown in the table below:

A	B	C	D	E
26.2	29.2	29.1	21.3	20.1
24.3	28.1	30.8	22.4	19.3
21.8	27.3	33.9	24.3	19.9
28.1	31.2	32.8	21.8	22.1

A	26.2,24.3,21.8,28.1
B	29.2,28.1,27.3,31.2
C	29.1,30.8,33.9,32.8
D	21.3,22.4,24.3,21.8
E	20.1,19.3,19.9,22.1

Using ANOVA, the data are regarded as random samples from k populations. Suppose the population means of the sample are denoted as $\mu_{1},\mu_{2},\mu_{3},\mu_{4},\mu_{5}$and their population standard deviation are denoted as $\sigma_{1},\sigma_{2},\sigma_{3},\sigma_{4},\sigma_{5}$. An obvious method is to do $\binom{5}{2}=10$ separate t-tests and compare all independent pairs of groups. In this case, ANOVA would be much easier and powerful.

Theory

One-way ANOVA

One-way ANOVA expands our inference methods to study and compare k independent samples. In this case, we will be decomposing the entire variation in the data into independent components.

• Notations first: $y_{ij}$ is the measurement from group $i$, observation index $j$; $k$ is the number of groups; $n_{i}$ is the number of observations in group $i$; $n$ is the total number of observations and $n=n_{1}+n_{2}+⋯+n_{k}$. The group mean for group $i$ is $\bar y_{l}$=$\frac{\sum_{j=1}^{n_{i}} y_{ij}} {n_{i}}$, the grand mean is $\bar y =\bar y_{..}=$ $\frac{\sum_{i=1}^{k}\sum_{j=1}^{n}_{i}y_{ij}}{n}$

• Difference between the means (compare each group mean to the grand mean): total variance $SST(total)=\sum_{i=1}^{k}\sum_{j=1}^{n_i}(y_{ij}-(\bar y_{..})^{2}$, degrees of freedom $df(total)=n-1$; difference between each group mean and grand mean: $SST(between)$=$\sum_{i=1}^{k} \sum_{j=1}^{n_i}(y_{ij}-\bar y_{..})^{2}$, degrees of freedom $df(between)=k-1$; Sum square due to error (combination of variation within group):$SSE=\sum_{i=1}^{k} n_{i}(\bar y_{l.}- \bar y_{..})^2$, degrees of freedom $df(within)=n-k$. With ANOVA decomposition, we have $\sum_{i=1}^{k} {\sum_{j=1}^{n_{i}} {(y_{ij}- \bar y_{..})^2 }} = \sum_{i=1}^{k} {n_{i} (y_{l.}-\bar y_{..})^2} + \sum_{i=1}^{k} {\sum_{j=1}^{n_i} {(y_{ij}-\bar y_{l.})^2}},$ that is $SST(total)$=$SST(between)$+$SSE(within)$ and $df(total)$=$df(between)$+$df(within).$

• Calculations:

Variance Source	Degrees of Freedom (df)	Sum of Squares (SS)	Mean Sum of Squares (MS)	F-Statistics	P-value
Treatment Effect (Between Group)	k-1	\(\sum_{i=1}^{k}{n_i(\bar{y}_{i,.}-\bar{y})^2}\)	\(MST(Between)={SST(Between)\over df(Between)}\)	\(F_o = {MST(Between)\over MSE(Within)}\)	\(P(F_{(df(Between), df(Within))} > F_o)\)
Error (Within Group)	n-k	\(\sum_{i=1}^{k}{\sum_{j=1}^{n_i}{(y_{i,j}-\bar{y}_{i,.})^2}}\)	\(MSE(Within)={SSE(Within)\over df(Within)}\)		F-Distribution Calculator
Total	n-1	\(\sum_{i=1}^{k}{\sum_{j=1}^{n_i}{(y_{i,j} - \bar{y})^2}}\)			ANOVA Activity

• ANOVA hypotheses (general form): $H_{\sigma}:\mu_{1}=\mu_{2}=⋯=\mu_{k}$; $H_{a}:\mu_{I}≠\mu_{j}$ for some $i≠j$. The test statistics: $F_{0}=\frac{MST(between)}{MSE(within)}$ , if $F_{0}$ is large, then there is a lot between group variation, relative to the within group variation. Therefore, the discrepancies between the group means are large compared to the variability within the groups (error). That is large $F_{0}$ provides strong evidence against $H_{0}$.

• Examples: given the following data from a hands-on study.

	Groups
Index	A	B	C
1	0	1	4
2	1	0	5
3		2
$n_{i}$	2	3	2
$s$	1	3	9
$\bar y_{l}$	0.5	1	4.5

Using this data, we have the following ANOVA table:

Variance Source	Degrees of Freedom (df)	Sum of Squares (SS)	Mean Sum of Squares (MS)	F-Statistics	P-value
Treatment Effect (Between Group)	3-1	\(\sum_{i=1}^{k}{n_i(\bar{y}_{i,.}-\bar{y})^2}=19.86\)	\({SST(Between)\over df(Between)}={19.86\over 2}\)	\(F_o = {MST(Between)\over MSE(Within)}=13.24\)	\(P(F_{(df(Between), df(Within))} > F_o)=0.017\)
Error (Within Group)	7-3	\(\sum_{i=1}^{k}{\sum_{j=1}^{n_i}{(y_{i,j}-\bar{y}_{i,.})^2}}=3\)	\({SSE(Within)\over df(Within)}={3\over 4}\)		F-Distribution Calculator
Total	7-1	\(\sum_{i=1}^{k}{\sum_{j=1}^{n_i}{(y_{i,j} - \bar{y})^2}}=22.86\)			Anova Activity

Based on the ANOVA table above, we can reject the null hypothesis at $\alpha=0.05.$

• ANOVA conditions: valid if (1) design conditions: all groups of observations represent random samples from their population respectively. Plus, all the observations within each group are independent of each other; (2) population conditions: the k population distributions must be approximately normal. If sample size is large, the normality condition is less crucial. Plus, the standard deviations of all populations are equal, which can be slightly relaxed when $0.5≤\frac{\sigma_{i}}{\sigma_{j}}≤2,$ for all $i$ and $j$, none of the population variance is twice larger than any of the other ones.

Two-way ANOVA

Two-way ANOVA decomposes the variance of a dataset into independent (orthogonal) components when we have two grouping factors. Notations first: two-way model:$y_{ijk}=\mu+\tau_{i}+\beta_{j}+γ_{ij}+\varepsilon_{ijk},$ for all $1≤i≤a,1≤j≤b$ and $1≤k≤r.$ $y_{ijk}$ is the A-factor level $i$, and B-factor level $j$, observation-index $k$ measurement; $k$ is the number of replications; $a_{i}$ is the number of A-factor observations at level $i,a=a_{1}+⋯+a_{I}$; $b_{j}$ is the number of B-factor observations at level $j$, $b=b_{1}+⋯+b_{J}$; $N$ is the total number of observations and $N=a*a*b$. Here $\mu$ is the overall mean response, $\tau_{i}$ is the effect due to the $i^{th}$ level of factor A, $\beta_{j}$ is the effect due to the $j^{th}$ level of factor B, and $\gamma_{ij}$ is the effect due to any interaction between the $i^{th}$ level of factor A and $j^{th}$ level of factor B. The mean for A-factor group mean at level $I$ and B-factor at level $j$ is $\bar{y}_{ij.}=\frac{\sum_{k=1}^{r} {y_{ijk}}} {r},$ the grand mean is $\bar {y} =\bar{y}_{...} = \frac{\sum_{k=1}^{r} {\sum_{i=1}^{a} {\sum_{j=1}^{b} {y_{ijk}}}}} {n}$, and we have we have $SST(total)=SS(A)+SS(B)+SS(AB)+SSE$.

• Hypotheses:
  • Null hypotheses: (1) the population means of the first factor are equal, which is like the one-way ANOVA for the row factor; (2) the population means of the second factor are equal, which is like the one-way ANOVA for the column factor; (3) there is no interaction between the two factors, which is similar to performing a test for independence with contingency tables.
  • Factors: factor A and factor B are independent variables in two-way ANOVA.
  • Treatment groups: formed by making all possible combinations of two factors. For example, if the factor A has 3 levels and factor B has 5 levels, then there will be 3*5=15 different treatment groups.
  • Main effect: involves the dependent variable one at a time. The interaction is ignored for this part.
  • Interaction effect: the effect that one factor has on the other factor. The degree of freedom is the product of the two degrees of freedom of each factor.

• Calculations:

It is assumed that main effect A has a levels (and df(A) = a-1), main effect B has b levels (and (df(B) = b-1), r is the sample size of each treatment, and \(N = a\times b\times n\) is the total sample size. Notice the overall degree of freedom is once again one less than the total sample size.

Variance Source	Degrees of Freedom (df)	Sum of Squares (SS)	Mean Sum of Squares (MS)	F-Statistics	P-value
Main Effect A	df(A)=a-1	\(SS(A)=r\times b\times\sum_{i=1}^{a}{(\bar{y}_{i,.,.}-\bar{y})^2}\)	\({SS(A)\over df(A)}\)	\(F_o = {MS(A)\over MSE}\)	\(P(F_{(df(A), df(E))} > F_o)\)
Main Effect B	df(B)=b-1	\(SS(B)=r\times a\times\sum_{j=1}^{b}{(\bar{y}_{., j,.}-\bar{y})^2}\)	\({SS(B)\over df(B)}\)	\(F_o = {MS(B)\over MSE}\)	\(P(F_{(df(B), df(E))} > F_o)\)
A vs.B Interaction	df(AB)=(a-1)(b-1)	\(SS(AB)=r\times \sum_{i=1}^{a}{\sum_{j=1}^{b}{((\bar{y}_{i, j,.}-\bar{y}_{i, .,.})+(\bar{y}_{., j,.}-\bar{y}))^2}}\)	\({SS(AB)\over df(AB)}\)	\(F_o = {MS(AB)\over MSE}\)	\(P(F_{(df(AB), df(E))} > F_o)\)
Error	\(N-a\times b\)	\(SSE=\sum_{k=1}^r{\sum_{i=1}^{a}{\sum_{j=1}^{b}{(\bar{y}_{i, j,k}-\bar{y}_{i, j,.})^2}}}\)	\({SSE\over df(Error)}\)
Total	N-1	\(SST=\sum_{k=1}^r{\sum_{i=1}^{a}{\sum_{j=1}^{b}{(\bar{y}_{i, j,k}-\bar{y}_{., .,.})^2}}}\)			ANOVA Activity

• Two-way ANOVA is valid if:

(1) the population from which the samples were obtained must be normally or approximately normally distributed;

(2) the samples must be independent;

(3) the variances of the populations must be equal; (4) the groups must have the same sample size.

• Example: clinical example of knee pain study

Applications

• This activity presents the Box and Whisker Chart, which is often used in exploratory data analyses. It demonstrates the range, standard deviation, mean and quartiles of the values and is especially useful in comparing statistical data. This article illustrated the implementation of the chart in SOCR with comprehensive introduction. It also included the application of this method in different areas.

• The SOCR Two-Way ANOVA Java Applet includes examples of two-way analysis of variance using SOCR tools. It illustrated the application of two-way ANOVA with examples applied in the SOCR. It also expanded the two-way ANOVA in softwares like R and SAS.

• Ther SOCR Snails Sexual Dimorphism Activity shows an application of ANOVA. This activity recreates part of the design of a classification method for the Cocholotoma septemspirale snail. By observing multiple traits of the shells, the original researchers were able to decide on a series of dimorphisms (difference in forms) between male and female snails. This article presents a comprehensive illustration of the example.

Software

• SOCR Analyses Java Applets
• One-Way ANOVA Activity
• R:

# fit a  model 
# one-way ANOVA with completely randomized design
fit <- aov(y ~ A, data = mydata)
# randomized block design (B as the blocking factor)
fit <- aov(y ~ A + B, data = mydata)
# two-way factorial design
fit <- aov(y ~ A + B + A*B, data = mydata)
# to check out the model fitted with type I ANOVA table
summary(fit)
# type III SS and F test
drop1(fit, ~., test=’F’)

Problems

• Tom was shopping for a ping pong table that could be taken apart quickly and easily. For some reason, the salesman happened to have a table of the assembly times (sec) for the three tables. Using ANOVA, do you think there is a difference in the average time of assembly for the three brands of ping pong tables?

Assembly_time_(sec)	Brand
93.0	1
67.0	1
77.0	1
92.0	1
97.0	1
62.0	1
136.0	2
120.0	2
115.0	2
104.0	2
115.0	2
121.0	2
102.0	2
130.0	2
198.0	3
217.0	3
209.0	3
221.0	3
190.0	3

(a) We can say that there is no reason to reject the null that the average assembly times are the same

(b) We should reject the null that the average assembly times are the same

• Based on the data in the previous problem, what is the value for R square:

(a) 0.342

(b) 0.143

(c) 0.832

(d) 0.943

• Tom is curious to see if two-door vehicles drive faster on average than four-door vehicles. He parks behind a bush so as not to be seen, and records the car type and the speed reading. Here are the results (1 means two-door, and 2 means four-door):

Speed_(MPH)	Vehicle_Type
45	2
45	2
40	2
69	1
72	1
40	1
75	2
19	2
62	1
43	2
75	1
42	2
58	1
58	1
47	2
48	2
49	2
45	2
54	2

At the 1% significance level, should we reject the null hypothesis that that average speed is the same for both types of vehicles?

(a) Yes, we should reject the null hypothesis.

(b) No, we should not reject the null hypothesis.

(c) There is not enough information.

• Based on data above, what is the value for R square?

(a) 0.432

(b) 0.983

(c) 0.308

(d) 0.231

• In a two-way ANOVA test, which of the following is not the typical null hypothesizes?

(a) The population means of the first factor are equal.

(b) The population means of the first and second factor are equal.

(c) The population means of the second factor are equal.

(d) There is no interaction between the two factors.

• Suppose that two factors, A and B, is thought to affect the top speed of a car. We will use two-way ANOVA analysis. Are the population means of factor A equal?

Top_Speed	A	B
93.0	1	1
136.0	1	2
198.0	1	3
88.0	2	1
148.0	2	2
279.0	2	3

(a) Yes, they are equal.

(b) No, they are not equal.

• Use the data above and apply the two-way ANOVA analysis, are the population means of factor B equal?

(a) Yes, they are equal.

(b) No, they are not equal.

• Use data from problem 6.6 and apply the two-way ANOVA analysis, is there an interaction effect between the two factors

(a) Yes, they are equal.

(b) No, they are not equal.

References

• SOCR
• ANOVA Wikipedia

---

• SOCR Home page: http://www.socr.umich.edu

---

---

Translate this page:

(default)	Deutsch	Español	Français	Italiano	Português	日本語	България	الامارات العربية المتحدة	Suomi	इस भाषा में	Norge
한국어	中文	繁体中文	Русский	Nederlands	Ελληνικά	Hrvatska	Česká republika	Danmark	Polska	România	Sverige