SMHS FixedRandomMixedModels - Revision history

Clgalla: /* Theory */

2014-10-16T15:52:53Z

‎Theory

Clgalla: /* Theory */

2014-10-16T15:47:09Z

‎Theory

Clgalla: /* Theory */

2014-10-16T13:51:55Z

‎Theory

Clgalla: /* Software */

2014-10-16T13:50:49Z

‎Software

Clgalla: /* Software */

2014-10-16T13:50:11Z

‎Software

Clgalla: /* References */

2014-10-16T13:49:18Z

‎References

Clgalla at 13:41, 16 October 2014

2014-10-16T13:41:16Z

Clgalla: /* Theory */

2014-10-16T13:30:51Z

‎Theory

Show changes

Clgalla at 21:03, 15 October 2014

2014-10-15T21:03:07Z

Clgalla: /* Theory */

2014-10-15T20:58:42Z

‎Theory

@@ Line 13: / Line 13: @@
 *The fixed effect assumption: the individual specific effect is correlated with the independent variables. This is the priority criteria in using the fixed effect model.
 *The fixed effect model: consider the linear unobserved effects model for $N$ observations and $T$ time periods: $y_{it}=X_{it}\beta + \alpha_{i} + \mu_{it}, for t = 1, 2, \cdots, T$ and $i = 1, 2, \cdots, N$, where $y_{it}$ is the dependent variable observed for individual $i$ and time $t$, $X_{it}$ is the time variant $1xk$ regressor matrix, $\alpha_{i}$ is the unobserved time invariant individual effect and $u_{it}$ is the error term. $\alpha_{i}$ can’t be observed by the econometrician, common examples for time invariant effects $\alpha_{i}$ are innate ability for individuals or institutional factors for countries. The model allows $\alpha_{i}$ to be correlated with the regressor matrix, $x_{it}.$
-*$\alpha_{i}$ is not observable and cannot be directly controlled for. The fixed effect model eliminates $\alpha_{i}$ by demeaning the variables using the within transformation: $y_{it} - \bar{y_{i}} = (X_{it} - \bar{X_{i}})\beta + (\alpha_{i} - \bar{\alpha_{i}}) + (\mu_{it} - \bar{mu_{it}}) \Rightarrow \ddot{y_{it}}=\ddot{X_{it}}\beta+\ddot{\mu_{it}}$, where $\bar{X_{i}}=\frac{1}{T}\sum_{t=1}^{T}X_{it}$, and $\bar{\mu_{i}}=\frac{1}{T} \sum_{t=1}^{T}\mu_{it}$, since $\alpha_{i}$ is constant, $\bar{\alpha_{i}}=\alpha_{i}$, hence the effect is eliminated. The fixed effect estimator $\hat{\beta_{FE}}$ is then obtained by an OLS regression of $\ddot{y}$ on $\ddot{X}.$
+*$\alpha_{i}$ is not observable and cannot be directly controlled for. The fixed effect model eliminates $\alpha_{i}$ by demeaning the variables using the within transformation: $y_{it} - \bar{y_{i}} = (X_{it} - \bar{X_{i}})\beta + (\alpha_{i} - \bar{\alpha_{i}}) + (\mu_{it} - \bar{\mu_{it}}) \Rightarrow \ddot{y_{it}}=\ddot{X_{it}}\beta+\ddot{\mu_{it}}$, where $\bar{X_{i}}=\frac{1}{T}\sum_{t=1}^{T}X_{it}$, and $\bar{\mu_{i}}=\frac{1}{T} \sum_{t=1}^{T}\mu_{it}$, since $\alpha_{i}$ is constant, $\bar{\alpha_{i}}=\alpha_{i}$, hence the effect is eliminated. The fixed effect estimator $\hat{\beta_{FE}}$ is then obtained by an OLS regression of $\ddot{y}$ on $\ddot{X}.$
-*Equality of Fixed Effects (FE) and First Differences (FD) estimator with the special case of T=2. That is the FE estimator effectively doubles the data set used in the FD estimator. $FE_{T=2}=[(x_{i1}-\bar{x_{i}})(x_{i1}-\bar{x_{i}})'+(x_{i2}-\bar{x_{i}})(x_{i2}-\bar{x_{i}})']^{-1}[(x_{i1}-\bar{x_{i}})(y_{i1}-\bar{y_{i}})+(x_{i2}-\bar{x_{i}})(y_{i2}-\bar{y_{i}})],$ since each $(x_{i1}-\bar{x_{i}})$ can be rewritten as $(x_{i1}-x_{i2})/2,$
+*Equality of Fixed Effects (FE) and First Differences (FD) estimator with the special case of T=2. That is the FE estimator effectively doubles the data set used in the FD estimator. $FE_{T=2}=[(x_{i1}-\bar{x_{i}})(x_{i1}-\bar{x_{i}})'+(x_{i2}-\bar{x_{i}})(x_{i2}-\bar{x_{i}})']^{-1}[(x_{i1}-\bar{x_{i}})(y_{i1}-\bar{y_{i}})+(x_{i2}-\bar{x_{i}})(y_{i2}-\bar{y_{i}})],$ since each $(x_{i1}-\bar{x_{i}})$ can be rewritten as $\frac{x_{i1}-x_{i2}}{2},$
 so $FE_{T=2}=[\sum_{i=1}^{N}\frac{x_{i1}-x_{i2}}{2}\frac{x_{i1}-x_{i2}'}{2}+\frac{x_{i2}-x_{i1}}{2}\frac{x_{i2}-x_{i1}'}{2}]^{-1}[\sum_{i=1}^{N}\frac{x_{i1}-x_{i2}}{2}\frac{y_{i1}-y_{i2}}{2}+\frac{x_{i2}-x_{i1}}{2}\frac{y_{i2}-y_{i1}}{2}]$

@@ Line 12: / Line 12: @@
 *The fixed effect model can be used to control for unobserved heterogeneity when this heterogeneity is constant over time and correlated with independent variables. The constant can be removed by differencing.
 *The fixed effect assumption: the individual specific effect is correlated with the independent variables. This is the priority criteria in using the fixed effect model.
-*The fixed effect model: consider the linear unobserved effects model for $N$ observations and $T$ time periods: $y_{it}=X_{it}\beta + \alpha_{i} + u_{it}, for t = 1, 2, \cdots, T$ and $i = 1, 2, \cdots, N$, where $y_{it}$ is the dependent variable observed for individual $i$ and time $t$, $X_{it}$ is the time variant $1xk$ regressor matrix, $\alpha_{i}$ is the unobserved time invariant individual effect and $u_{it}$ is the error term. $\alpha_{i}$ can’t be observed by the econometrician, common examples for time invariant effects $\alpha_{i}$ are innate ability for individuals or institutional factors for countries. The model allows $\alpha_{i}$ to be correlated with the regressor matrix, $x_{it}.$
+*The fixed effect model: consider the linear unobserved effects model for $N$ observations and $T$ time periods: $y_{it}=X_{it}\beta + \alpha_{i} + \mu_{it}, for t = 1, 2, \cdots, T$ and $i = 1, 2, \cdots, N$, where $y_{it}$ is the dependent variable observed for individual $i$ and time $t$, $X_{it}$ is the time variant $1xk$ regressor matrix, $\alpha_{i}$ is the unobserved time invariant individual effect and $u_{it}$ is the error term. $\alpha_{i}$ can’t be observed by the econometrician, common examples for time invariant effects $\alpha_{i}$ are innate ability for individuals or institutional factors for countries. The model allows $\alpha_{i}$ to be correlated with the regressor matrix, $x_{it}.$
-*$\alpha_{i}$ is not observable and cannot be directly controlled for. The fixed effect model eliminates $\alpha_{i}$ by demeaning the variables using the within transformation: $y_{it} - \bar{y_{i}} = (X_{it} - \bar{X_{i}})\beta + (\alpha_{i} - \bar{\alpha_{i}}) + (u_{it} - \bar{u_{it}}) \Rightarrow \ddot{y_{it}}=\ddot{X_{it}}\beta+\ddot{u_{it}}$, where $\bar{X_{i}}=\frac{1}{T}\sum_{t=1}^{T}X_{it}$, and $\bar{u_{i}}=\frac{1}{T} \sum_{t=1}^{T}u_{it}$, since $\alpha_{i}$ is constant, $\bar{\alpha_{i}}=\alpha_{i}$, hence the effect is eliminated. The fixed effect estimator $\hat{\beta_{FE}}$ is then obtained by an OLS regression of $\ddot{y}$ on $\ddot{X}.$
+*$\alpha_{i}$ is not observable and cannot be directly controlled for. The fixed effect model eliminates $\alpha_{i}$ by demeaning the variables using the within transformation: $y_{it} - \bar{y_{i}} = (X_{it} - \bar{X_{i}})\beta + (\alpha_{i} - \bar{\alpha_{i}}) + (\mu_{it} - \bar{mu_{it}}) \Rightarrow \ddot{y_{it}}=\ddot{X_{it}}\beta+\ddot{\mu_{it}}$, where $\bar{X_{i}}=\frac{1}{T}\sum_{t=1}^{T}X_{it}$, and $\bar{\mu_{i}}=\frac{1}{T} \sum_{t=1}^{T}\mu_{it}$, since $\alpha_{i}$ is constant, $\bar{\alpha_{i}}=\alpha_{i}$, hence the effect is eliminated. The fixed effect estimator $\hat{\beta_{FE}}$ is then obtained by an OLS regression of $\ddot{y}$ on $\ddot{X}.$
 *Equality of Fixed Effects (FE) and First Differences (FD) estimator with the special case of T=2. That is the FE estimator effectively doubles the data set used in the FD estimator. $FE_{T=2}=[(x_{i1}-\bar{x_{i}})(x_{i1}-\bar{x_{i}})'+(x_{i2}-\bar{x_{i}})(x_{i2}-\bar{x_{i}})']^{-1}[(x_{i1}-\bar{x_{i}})(y_{i1}-\bar{y_{i}})+(x_{i2}-\bar{x_{i}})(y_{i2}-\bar{y_{i}})],$ since each $(x_{i1}-\bar{x_{i}})$ can be rewritten as $(x_{i1}-x_{i2})/2,$
-so$FE_{T=2}=[\sum_{i=1}^{N}\frac{x_{i1}-x_{i2}}{2}\frac{x_{i1}-x_{i2}'}{2}+\frac{x_{i2}-x_{i1}}{2}\frac{x_{i2}-x_{i1}'}{2}]^{-1}[\sum_{i=1}^{N}\frac{x_{i1}-x_{i2}}{2}\frac{y_{i1}-y_{i2}}{2}+\frac{x_{i2}-x_{i1}}{2}\frac{y_{i2}-y_{i1}}{2}]$
+so $FE_{T=2}=[\sum_{i=1}^{N}\frac{x_{i1}-x_{i2}}{2}\frac{x_{i1}-x_{i2}'}{2}+\frac{x_{i2}-x_{i1}}{2}\frac{x_{i2}-x_{i1}'}{2}]^{-1}[\sum_{i=1}^{N}\frac{x_{i1}-x_{i2}}{2}\frac{y_{i1}-y_{i2}}{2}+\frac{x_{i2}-x_{i1}}{2}\frac{y_{i2}-y_{i1}}{2}]$
 $=[\sum_{i=1}^{N}2\frac{x_{i1}-x_{i2}}{2}\frac{x_{i2}-x_{i1}'}{2}]^{-1}[\sum_{i=1}^{N}2\frac{x_{i2}-x_{i1}}{2}\frac{y_{i2}-y_{i1}}{2}]$
@@ Line 40: / Line 40: @@
 *The random effects model can be applied to control for unobserved heterogeneity when this heterogeneity is constant over time and correlated with independent variables. The constant can be removed from the data through differencing.
 *The random effect assumption: individual specific effects are uncorrelated with the independent variables. The random effects model is efficient and consistent only when this assumption is met.
-*Suppose $m$ schools are chosen randomly from the united states to study on the math score of the 7th grade students. Suppose that $n$ students of the same age are randomly chosen from each selected school and their math scores are recorded. Let $Y_{ij}$ be the score of the $j^{th}$ student at the $i^{th}$ school. A simple model can be fitted $Y_{ij} = \mu + U_{i} + W_{ij}$, where $\mu$ is the average test score for the entire population. In this model, $U_{i}$ is the school-specific random effect, which measures the difference between the average math score at school $i$ and the average math score in the entire country. It is considered to be random because the school are randomly chosen form the contry. $W_{ij}$ is the individual-specific error. This is also random since the students within the school are random chosen. We can augment the model further by adding in additional explanatory variables, say teacher to student ratio.
+*Suppose $m$ schools are chosen randomly from the united states to study on the math score of the 7th grade students. Suppose that $n$ students of the same age are randomly chosen from each selected school and their math scores are recorded. Let $Y_{ij}$ be the score of the $j^{th}$ student at the $i^{th}$ school. A simple model can be fitted $Y_{ij} = \mu + \mu_{i} + W_{ij}$, where $\mu$ is the average test score for the entire population. In this model, $\mu_{i}$ is the school-specific random effect, which measures the difference between the average math score at school $i$ and the average math score in the entire country. It is considered to be random because the school are randomly chosen form the contry. $W_{ij}$ is the individual-specific error. This is also random since the students within the school are random chosen. We can augment the model further by adding in additional explanatory variables, say teacher to student ratio.
 *Variance components: the variance of $Y_{ij}$ is the sum of the variance $\tau^2$ and $\sigma^2$ of $U_{i}$ and $W_{ij}$ respectively. Let $\bar{Y_{i\cdot}}=\frac{1}{n} \sum_{j=1}^{n} Y_{ij}$ be the average of the $i^{th}$ school that are included in the random sample. Let $\bar{Y_{\cdot\cdot}} = \frac{1}{mn}\sum_{i=1}^{m}\sum_{j=1}{n}Y_{ij}$ be the grand average. Let $SSW = \sum_{i=1}^{m}\sum_{j=1}^{n}(Y_{ij}-\bar{Y_{i\cdot}})^2, SSB = n\sum_{i=1}^{m}(\bar{Y_{i\cdot}}-\bar{Y_{\cdot\cdot}})^2 $ are the sum of square due to differences within groups and the sum of squares due to difference between groups respectively. $\frac{1}{m(n-1)}E(SSW)=\sigma^2, \frac{1}{(m-1)n}E(SSB)=\frac{\sigma^2}{n+\tau^2}.$ This can be used to estimate the variance components $\sigma^2$ and $\tau^2$.
 *Comments on random effects model: in general the random effects model are efficient and should be used if the assumptions underlying are satisfied. For random effect to work in the school examples, the school-specific effects should be uncorrelated to other covariates of the model. A Hausman specification test can be used to test on this assumption as described in the previous Fixed Effects Model.
 '''3) Mixed models:''' a statistical model containing both fixed and random effects, which is widely used in varieties of disciplines, particularly useful for situations where repeated measurements are made on the same statistical units (longitudinal study), or where measurements are made on clusters of related statistical units.
-*Mixed model formula (in matrix form): $y = X\beta + Zu+ \epsilon$, where $y$ is a vector of observations with mean $E(y) = X\beta$, $\beta$ is a vector of fixed effects, $u$ is a vector of random effects with mean $E(u)=0$ and variance-covariance matrix $var(u)=G$, $\epsilon$ is a vector of i.i.d. random error terms with mean $E(\epsilon)=0$ and variance $var(\epsilon)=R,$ $X$ and $Z$ are matrices of regressors relating the observations $y$ to $\beta$ and $u$, respectively.
+*Mixed model formula (in matrix form): $y = X\beta + Zu+ \epsilon$, where $y$ is a vector of observations with mean $E(y) = X\beta$, $\beta$ is a vector of fixed effects, $\mu$ is a vector of random effects with mean $E(\mu)=0$ and variance-covariance matrix $var(\mu)=G$, $\epsilon$ is a vector of i.i.d. random error terms with mean $E(\epsilon)=0$ and variance $var(\epsilon)=R,$ $X$ and $Z$ are matrices of regressors relating the observations $y$ to $\beta$ and $\mu$, respectively.
-*Estimation of Henderson’s mixed model equations (MME): $\begin{pmatrix} X'R^{-1}Z&X'R^{-1}Z\\ Z'R^{-1}X&Z'R^{-1}Z+G^{-1} \end{pmatrix} \bigl(\begin{smallmatrix} X'R^{-1}y\\Z'R^{-1}y \end{smallmatrix}\bigr).$ The solutions to the MME, $\tilde{\beta}$ and $\tilde{u}$ are the best linear unbiased estimates (BLUE) and predictors for $\beta$ and $u$, respectively. When the conditional variance is known, the inverse variance weighed least squares estimate is BLUE, however, the conditional variance is rarely known. So it is desirable to jointly estimate the variance and weighted parameter estimates when solving MMEs. One method to fit such mixed models is that of EM algorithm, which can be implemented by lme in the nlme library in the R package.
+*Estimation of Henderson’s mixed model equations (MME): $\begin{pmatrix} X'R^{-1}Z&X'R^{-1}Z\\ Z'R^{-1}X&Z'R^{-1}Z+G^{-1} \end{pmatrix} \bigl(\begin{smallmatrix} X'R^{-1}y\\Z'R^{-1}y \end{smallmatrix}\bigr).$ The solutions to the MME, $\tilde{\beta}$ and $\tilde{
 '''4) Comparing fixed effects model and random effects model:'''

@@ Line 9: / Line 9: @@
 ===Theory===
-) '''Fixed effects model:''' a statistical model that represents the observed quantities in terms of explanatory variables treated as if they are non-random. The fixed effects estimator is used to refer to an estimator for the coefficients in the regression model and we impose time independent effects for each entity when we assume fixed effects for our model.
+'''1) Fixed effects model:''' a statistical model that represents the observed quantities in terms of explanatory variables treated as if they are non-random. The fixed effects estimator is used to refer to an estimator for the coefficients in the regression model and we impose time independent effects for each entity when we assume fixed effects for our model.
 *The fixed effect model can be used to control for unobserved heterogeneity when this heterogeneity is constant over time and correlated with independent variables. The constant can be removed by differencing.
 *The fixed effect assumption: the individual specific effect is correlated with the independent variables. This is the priority criteria in using the fixed effect model.
@@ Line 37: / Line 37: @@
 **Conclude as to whether treatment effect significantly affects the variable of interest.
-'''Random effects model:'''a statistical model in which the dataset being analyzed consist of a hierarchy of different population whose differences relate to that hierarchy. It is in contrast of the fixed effects model where the data being studied on consist only of non-random variables.
+'''2) Random effects model:'''a statistical model in which the dataset being analyzed consist of a hierarchy of different population whose differences relate to that hierarchy. It is in contrast of the fixed effects model where the data being studied on consist only of non-random variables.
 *The random effects model can be applied to control for unobserved heterogeneity when this heterogeneity is constant over time and correlated with independent variables. The constant can be removed from the data through differencing.
 *The random effect assumption: individual specific effects are uncorrelated with the independent variables. The random effects model is efficient and consistent only when this assumption is met.
@@ Line 44: / Line 44: @@
 *Comments on random effects model: in general the random effects model are efficient and should be used if the assumptions underlying are satisfied. For random effect to work in the school examples, the school-specific effects should be uncorrelated to other covariates of the model. A Hausman specification test can be used to test on this assumption as described in the previous Fixed Effects Model.
-) '''Mixed models:''' a statistical model containing both fixed and random effects, which is widely used in varieties of disciplines, particularly useful for situations where repeated measurements are made on the same statistical units (longitudinal study), or where measurements are made on clusters of related statistical units.
+'''3) Mixed models:''' a statistical model containing both fixed and random effects, which is widely used in varieties of disciplines, particularly useful for situations where repeated measurements are made on the same statistical units (longitudinal study), or where measurements are made on clusters of related statistical units.
 *Mixed model formula (in matrix form): $y = X\beta + Zu+ \epsilon$, where $y$ is a vector of observations with mean $E(y) = X\beta$, $\beta$ is a vector of fixed effects, $u$ is a vector of random effects with mean $E(u)=0$ and variance-covariance matrix $var(u)=G$, $\epsilon$ is a vector of i.i.d. random error terms with mean $E(\epsilon)=0$ and variance $var(\epsilon)=R,$ $X$ and $Z$ are matrices of regressors relating the observations $y$ to $\beta$ and $u$, respectively.
 *Estimation of Henderson’s mixed model equations (MME): $\begin{pmatrix} X'R^{-1}Z&X'R^{-1}Z\\ Z'R^{-1}X&Z'R^{-1}Z+G^{-1} \end{pmatrix} \bigl(\begin{smallmatrix} X'R^{-1}y\\Z'R^{-1}y \end{smallmatrix}\bigr).$ The solutions to the MME, $\tilde{\beta}$ and $\tilde{u}$ are the best linear unbiased estimates (BLUE) and predictors for $\beta$ and $u$, respectively. When the conditional variance is known, the inverse variance weighed least squares estimate is BLUE, however, the conditional variance is rarely known. So it is desirable to jointly estimate the variance and weighted parameter estimates when solving MMEs. One method to fit such mixed models is that of EM algorithm, which can be implemented by lme in the nlme library in the R package.
-) '''Comparing fixed effects model and random effects model:'''
+'''4) Comparing fixed effects model and random effects model:'''
 *Estimating the summary effect: under the fixed effect model, it is assumed that the true effect size for all studies is identical and the only reason the effect size varies between studies is sampling error. In situations like this, assigning equal weights to different studies largely ignore the information in smaller studies since more information are known about the same effect size in larger studies. In contrast, the random effect model aims to estimate the mean of a distribution of effects. The estimate provided by that study may be imprecise, but it is information about an effect that no other study has estimated. We cannot discount a small study by giving it a small weight, not can we give too much weight to a large study. Hence, in fixed-effect model, there is a wide range of weights while the weights in the random effects model fall in a relatively narrow range.  Study weights are more balanced under the random effect model than the fixed effect model.
 *A fixed effect meta analysis estimates a single effect that is assumed to be common to every study while a random effects meta analysis estimates the mean of a distribution of effects.

@@ Line 63: / Line 63: @@
 ===Software===
-[http://cran.r-project.org/doc/contrib/Fox-Companion/appendix-mixed-models.pdf
+[http://cran.r-project.org/doc/contrib/Fox-Companion/appendix-mixed-models.pdf]
-lme (linear mixed effects) function in the nlme library]
+lme (linear mixed effects) function in the nlme library
   RCODE:

@@ Line 70: / Line 70: @@
   library(lattice) ## for Trellis graphics
   data(MathAchieve)
-  MathAchieve[1:10,]   # first 10 students; relates to 7185 students; SES:the socioeconomic status   of the student’s family, centered to the overall mean of 0; MathAch: the student’s score on a math- achievement test; Sector: factor coded ‘Catholic’or ‘Public’; MEANSES: mean SES for student in each school.
+  MathAchieve[1:10,]   # first 10 students; relates to 7185 students; SES:the socioeconomic status
   Grouped Data: MathAch ~ SES | School

← Older revision		Revision as of 13:49, 16 October 2014
Line 120:		Line 120:

	===References===		===References===
−	http://mirlyn.lib.umich.edu/Record/004199238	+	[http://mirlyn.lib.umich.edu/Record/004199238 Statistical inference / George Casella, Roger L. Berger]
−	~~http://mirlyn.lib.umich.edu/Record/004232056~~
−	~~http:~~/~~/mirlyn~~.~~lib.umich.edu/Record/004133572~~

		+	[http://mirlyn.lib.umich.edu/Record/004232056 Sampling / Steven K. Thompson]
		+
		+	[http://mirlyn.lib.umich.edu/Record/004133572 Sampling theory and methods / S. Sampath]

@@ Line 58: / Line 58: @@
 ===Applications===
-[http://www.tandfonline.com/doi/abs/10.1080/01621459.1984.10477102#.U-2PdhZTWdA   This article] proposed approximations for standard errors of estimators of fixed, and random effects in mixed linear models. Best linear unbiased estimators of the fixed and random effects of mixed linear models are available when the true values of the variance ratios are known. If the true values are replaced by estimated values, the mean squared errors of the estimators of the fixed and random effects increase in size. The magnitude of this increase is investigated, and a general approximation is proposed. The performance of this approximation is investigated in the context of (a) the estimation of the effects of the balanced one-way random model and (b) the estimation of treatment contrasts for balanced incomplete block designs.
+) [http://amstat.tandfonline.com/doi/abs/10.1080/01621459.1955.10501298#.U-2P1xZTWdA This article] discussed about the fixed, mixed and random models. Some explicit questions are raised regarding the adequacy of assumed linear models as a basis for the interpretation of the analysis of variance of randomized experiments. A generally applicable method for the derivation of a linear statistical model, based on the experimental situation and the design of the experiment, is exemplified. The central features of the method are the notion of “experimental unit,” the concept of “true response,” and the use of randomization in the design. A model is derived for the case where two factors having A and B levels respectively, are to be examined with respect to a population of P experimental units, where selection of levels of the factors to be tested, selection of experimental units to be used, and the allocation of selected treatment combinations to units is at random. First a linear population model is given, based on the structure of the experimental situation, whose components are (unknown) parameters of the population of (conceptual) “true” responses. Then the conditions of the design are imposed to obtain a linear statistical model whose components involve the parameters of the population model and some defined random variables reflecting the experimental procedure and design. The derived statistical model is then used to obtain expected mean squares in the analyses of variance. A second illustration of the general methodology is given for a more complex example originated by Vaurio and Daniel and discussed statistically by Scheffé. Some differences from Scheffé's results are noted.
 ===Software===
@@ Line 132: / Line 70: @@
   library(lattice) ## for Trellis graphics
   data(MathAchieve)
-  MathAchieve[1:10,]   # first 10 students; relates to 7185 students; SES:
+  MathAchieve[1:10,]   # first 10 students; relates to 7185 students; SES:the socioeconomic status   of the student’s family, centered to the overall mean of 0; MathAch: the student’s score on a math- achievement test; Sector: factor coded ‘Catholic’or ‘Public’; MEANSES: mean SES for student in each school.
- the socioeconomic status   of the student’s family, centered to the overall mean of 0;
   Grouped Data: MathAch ~ SES | School
      School Minority    Sex    SES MathAch MEANSES
@@ Line 169: / Line 105: @@
   mses[as.character(MathAchSchool$School[1:10])]   # for first 10 schools
-…..
 Check the R example in the attached file at http://cran.r-project.org/doc/contrib/Fox-Companion/appendix-mixed-models.pdf .

@@ Line 13: / Line 13: @@
 *The fixed effect assumption: the individual specific effect is correlated with the independent variables. This is the priority criteria in using the fixed effect model.
 *The fixed effect model: consider the linear unobserved effects model for $N$ observations and $T$ time periods: $y_{it}=X_{it}\beta + \alpha_{i} + u_{it}, for t = 1, 2, \cdots, T$ and $i = 1, 2, \cdots, N$, where $y_{it}$ is the dependent variable observed for individual $i$ and time $t$, $X_{it}$ is the time variant $1xk$ regressor matrix, $\alpha_{i}$ is the unobserved time invariant individual effect and $u_{it}$ is the error term. $\alpha_{i}$ can’t be observed by the econometrician, common examples for time invariant effects $\alpha_{i}$ are innate ability for individuals or institutional factors for countries. The model allows $\alpha_{i}$ to be correlated with the regressor matrix, $x_{it}.$
-*$\alpha_{i}$ is not observable and cannot be directly controlled for. The fixed effect model eliminates $\alpha_{i}$ by demeaning the variables using the within transformation: $y_{it} - \bar{y_{i}} = (X_{it} - \bar{X_{i}})\beta + \alpha_{i} - \bar{\alpha_{i}}) + (u_{it} - \bar{u_{it}}) \Rightarrow \ddot{y_{it}}=\ddot{X_{it}}\beta+\ddot{u_{it}}$, where $\bar{X_{i}}=1/T \sum_{t=1}^{T}X_{it}$, and $\bar{u_{i}}=1/T \sum_{t=1}^{T}u_{it}$, since $\alpha_{i}$ is constant, $\bar{\alpha_{i}}=\alpha_{i}$, hence the effect is eliminated. The fixed effect estimator $\hat{\beta_{FE}}$ is then obtained by an OLS regression of $\ddot{y}$ on $\ddot{X}.$
+*$\alpha_{i}$ is not observable and cannot be directly controlled for. The fixed effect model eliminates $\alpha_{i}$ by demeaning the variables using the within transformation: $y_{it} - \bar{y_{i}} = (X_{it} - \bar{X_{i}})\beta + (\alpha_{i} - \bar{\alpha_{i}}) + (u_{it} - \bar{u_{it}}) \Rightarrow \ddot{y_{it}}=\ddot{X_{it}}\beta+\ddot{u_{it}}$, where $\bar{X_{i}}=1/T \sum_{t=1}^{T}X_{it}$, and $\bar{u_{i}}=1/T \sum_{t=1}^{T}u_{it}$, since $\alpha_{i}$ is constant, $\bar{\alpha_{i}}=\alpha_{i}$, hence the effect is eliminated. The fixed effect estimator $\hat{\beta_{FE}}$ is then obtained by an OLS regression of $\ddot{y}$ on $\ddot{X}.$
-*Equality of Fixed Effects (FE) and First Differences (FD) estimator with the special case of T=2. That is the FE estimator effectively doubles the data set used in the FD estimator. $FE_{T=2}=[(x_{i1}-\bar{x_{i}})(x_{i1}-\bar{x_{i}})'+(x_{i2}-\bar{x_{i}})(x_{i2}-\bar{x_{i}})']^{-1}[(x_{i1}-\bar{x_{i}})(y_{i1}-\bar{y_{i}})+(x_{i2}-\bar{x_{i}})(y_{i2}-\bar{y_{i}})],$ since each $(x_{i1}-\bar{x_{i}})$ can be rewritten as $(x_{i1}-x_{i2})/2,$ so $FE_{T=2}=[\sum_{i=1}^{N}\frac{x_{i1}-x_{i2}}{2}\frac{x_{i1}-x_{i2}'}{2}+\frac{x_{i2}-x_{i1}}{2}\frac{x_{i2}-x_{i1}'}{2}]^{-1}[\sum_{i=1}^{N}\frac{x_{i1}-x_{i2}}{2}\frac{y_{i1}-y_{i2}}{2}+\frac{x_{i2}-x_{i1}}{2}\frac{y_{i2}-y_{i1}}{2}]$
+*Equality of Fixed Effects (FE) and First Differences (FD) estimator with the special case of T=2. That is the FE estimator effectively doubles the data set used in the FD estimator. $FE_{T=2}=[(x_{i1}-\bar{x_{i}})(x_{i1}-\bar{x_{i}})'+(x_{i2}-\bar{x_{i}})(x_{i2}-\bar{x_{i}})']^{-1}[(x_{i1}-\bar{x_{i}})(y_{i1}-\bar{y_{i}})+(x_{i2}-\bar{x_{i}})(y_{i2}-\bar{y_{i}})],$ since each $(x_{i1}-\bar{x_{i}})$ can be rewritten as $(x_{i1}-x_{i2})/2,$
-=[\sum_{i=1}^{N}2\frac{x_{i1}-x_{i2}}{2}\frac{x_{i2}-x_{i1}'}{2}]^{-1}[\sum_{i=1}^{N}2\frac{x_{i2}-x_{i1}}{2}\frac{y_{i2}-y_{i1}}{2}]=[\sum_{i=1}^{N}(x_{i2}-x_{i1}x_{i2}-x_{i1}']^{-1}\sum_{i=1}^{N}(x_{i2}-x_{i1})(y_{i2}-y_{i1})=FD_{T=2}.$
+so$FE_{T=2}=[\sum_{i=1}^{N}\frac{x_{i1}-x_{i2}}{2}\frac{x_{i1}-x_{i2}'}{2}+\frac{x_{i2}-x_{i1}}{2}\frac{x_{i2}-x_{i1}'}{2}]^{-1}[\sum_{i=1}^{N}\frac{x_{i1}-x_{i2}}{2}\frac{y_{i1}-y_{i2}}{2}+\frac{x_{i2}-x_{i1}}{2}\frac{y_{i2}-y_{i1}}{2}]=[\sum_{i=1}^{N}2\frac{x_{i1}-x_{i2}}{2}\frac{x_{i2}-x_{i1}'}{2}]^{-1}[\sum_{i=1}^{N}2\frac{x_{i2}-x_{i1}}{2}\frac{y_{i2}-y_{i1}}{2}]
-*Hausman-Taylor method: need more than one time-variant regressor $(X)$ and time invariant regressor $(Z)$ and at least one $X$ and one $Z$ that are uncorrelated with $/alpha_{i}$, partition the $X$ and $Z$ variables such that  $X=[X_{1it}\vdots X_{2it}], Z=[Z_{1it}\vdots Z_{2it}]$ where $X_{1}$ and $Z_{1}$ are uncorrelated with $\alpha_{i}.$ Need $K1 > G2.$ Estimating $\gamma via OLS on \hat{di} = Z_{i}\gamma + \varphi_{it},$ using $X_{1}$ and $Z_{1}$ as instruments yields a consistent estimate.
+=[\sum_{i=1}^{N}(x_{i2}-x_{i1}x_{i2}-x_{i1}']^{-1}\sum_{i=1}^{N}(x_{i2}-x_{i1})(y_{i2}-y_{i1})=FD_{T=2}.$
 *Testing FE vs. RE with a Hausman test: $H_{0}: \alpha_{i}$ independent from $X_{it}$, $Z_{it}$ vs. $H_{a}: \alpha_{i}$ not independent $X_{it}$, $Z_{it}$. If $H_{0}$ is true, both $\hat{\beta_{RE}}$ and $\hat{\beta_{FE}}$ are consistent, but only $\hat{\beta_{RE}}$ is efficient. If $H_{a}$ is true, $\hat{\beta_{FE}}$ is consistent and $\hat{\beta_{RE}}$ not. $\hat{Q}$ = $\hat{\beta_{RE}}-\hat{\beta_{FE}}$, $\hat{HT}= T\hat{Q’}[Var(\hat{\beta_{FE}})-var(\hat{\beta_{RE}})]^{-1}\hat{Q} \sim \chi_{k}^2,$ where $K = dim(Q)$. The Hausman test is a specification test so a large test statistic might be indication that there might be errors in variables or the model is misspecified. If the fixed effect assumption is true, $\hat{\beta_{LD}} \approx \hat{\beta_{FD}} \approx \hat{\beta_{FE}}.$