SMHS LinearModeling - Revision history

Pineaumi: /* Scientific Methods for Health Sciences - Linear Modeling */

2016-05-19T12:47:03Z

‎Scientific Methods for Health Sciences - Linear Modeling

Dinov: /* Linear mixed effects analyses */

2016-02-01T16:01:56Z

‎Linear mixed effects analyses

Dinov at 15:02, 1 February 2016

2016-02-01T15:02:31Z

Dinov at 14:55, 1 February 2016

2016-02-01T14:55:15Z

Show changes

Dinov at 14:29, 1 February 2016

2016-02-01T14:29:08Z

Show changes

Imoubara at 14:13, 1 February 2016

2016-02-01T14:13:01Z

Imoubara at 13:24, 1 February 2016

2016-02-01T13:24:50Z

Imoubara at 13:22, 1 February 2016

2016-02-01T13:22:43Z

Imoubara at 13:13, 1 February 2016

2016-02-01T13:13:27Z

Dinov: Why adding hte 'translate' line at the bottom breaks the Wiki page? http://wiki.socr.umich.edu/index.php/SMHS_LinearModeling

2016-01-26T16:00:24Z

Why adding hte 'translate' line at the bottom breaks the Wiki page? http://wiki.socr.umich.edu/index.php/SMHS_LinearModeling

@@ Line 17: / Line 17: @@
 ===[[SMHS_LinearModeling_MachineLearning|Machine Learning Algorithms]]===
 Data modeling, training	, testing, forecasting, prediction, and simulation.
 <hr>
 * SOCR Home page: http://www.socr.umich.edu
 {{translate|pageName=http://wiki.socr.umich.edu/index.php?title=SMHS_LinearModeling}}

@@ Line 13: / Line 13: @@
 ===[[SMHS_LinearModeling_LMM |Linear mixed effects analyses]]===
-Scientific inference base don fixed and random effect models, assumptions, and mixed effects logistic regression.
+Scientific inference based on fixed and random effect models, assumptions, and mixed effects logistic regression.
 ===[[SMHS_LinearModeling_MachineLearning|Machine Learning Algorithms]]===

@@ Line 12: / Line 12: @@
 Review and demonstration of computing and visualizing the regression-model coefficients (effect-sizes), (fixed-effect) linear model assumptions, examination of residual plots, and independence.
-===[[SMHS_LinearModeling_MLR |Linear mixed effects analyses]]===
+===[[SMHS_LinearModeling_LMM |Linear mixed effects analyses]]===
 Scientific inference base don fixed and random effect models, assumptions, and mixed effects logistic regression.

← Older revision		Revision as of 14:13, 1 February 2016
Line 694:		Line 694:
	*If residual plots include stripes, then there may be categorical variables playing role, which may necessitate alternative class of models, such as logistic or multinomial models.		*If residual plots include stripes, then there may be categorical variables playing role, which may necessitate alternative class of models, such as logistic or multinomial models.

		+	Lack of collinearity: A pair of (linearly) correlated predictors are also called collinear. Suppose Age and Height are correlated (which is certainly to be expected!), then including both as predictors of Weight presents a collinearity problem. In the presence of significant variable collinearity, the interpretation of the model becomes challenging. Depending on which correlated predictors are included in the model, the fixed effects may (incorrectly) become significant or insignificant. Significant findings for these correlated or collinear fixed effects is difficult to interpret as there may be trading-off between the “explanatory power” of the pair of covariates. Multiple predictors that are very similar (linearly correlated) inhibit our ability to decide what plays a big role and what has only marginal impact on the response.

		+	Collinearity may be avoided early during the study-design/data-collection phase of the study to collect fewer fixed effects known to not be linearly correlated. Alternatively, we can selectively include/exclude predictors (e.g., only include in model the most meaningful independent variable and drop the others). Dimensionality reduction methods (e.g., Principal/Independent Component Analyses) also identify (fuse) linearly correlated variables into factors (representing linear combinations transforming several correlated variables into one component variable, which can be used as new fixed effect).

		+	Heteroskedasticity (lack of homoscedasticity). If the variance of the data is (approximately) stable across the range of the predicted values, then the process is homoscedastic (equal variance assumption). Otherwise, when the homoscedasticity criterion is violated, the process is called heteroskedastic (unequal variances).

		+	Satisfying the homoscedasticity assumption requires the model residuals to roughly have a similar amount of deviation from the predicted values. This can be checked by examining the residual plot.

		+	# lm.2 = lm(Weight ~ Age, df.2)
		+	lm.2.res = resid(lm.2)
		+	plot(Age, lm.2.res, ylab="(Weight) Residuals", xlab="Age", main="MLB (lm.2) model residuals")
		+	abline(0, 0)
		+
		+	[[Image:SMHS_LinearModeling_Fig21.png\|500px]]
		+
		+	Overall, these data are mostly homoscedastic. A good residual plot essentially looks random blob cluster. For instance, using real (simulated) random data, we can see the expectation for the residuals under the linear model assumptions:
		+
		+	plot(rnorm(1000), rbeta(1000, 1,1)-0.5)
		+	abline(0, 0)
		+
		+	[[Image:SMHS_LinearModeling_Fig22.png\|500px]]
		+
		+	This creates two sets of 1,000 random numbers (x and y axes representing Normal(mean=0,SD=1) and Beta(1,1) distributions, respectively ). Redoing this simulation multiple times and recreating the residual plots will not change the appearance of the scatter!
		+
		+	However there are many situations where the residual plots may show clear heteroskedasticity:
		+
		+	set.seed(11) # see the random generator with some integer
		+	n <- 256 # define sample size
		+	X <- (1:n)/n # The set of the predictor X values (uniform steps of 1/n from 0 to 1)
		+	E <- rnorm(n, sd=1) # A set of normally distributed random noise E values
		+	i <- order(runif(n, max=dnorm(E))) # Reorder E, by putting larger errors at the end, on average
		+	Y <- 1 + 5 * X + E[rev(i)] # Simulate new (observed responses) Y values, X plus "error" `E`.
		+	lm.5 <- lm(Y ~ X) # Regress `Y` against `X`.
		+	par(mfrow=c(1,3)) # Set up 1 row of 3 plots for drawing the graphs
		+	plot(X,Y, main="Simulated Noisy Data (Y) across time/index", cex=0.8)
		+	abline(coef(lm.5), col="Red")
		+	hist(residuals(lm.5), main="Linear Model Residuals")
		+	plot(predict(lm.5), residuals(lm.5), cex=0.8, main="Residuals vs. Predicted (Heteroskedasticity)")
		+
		+	[[Image:SMHS_LinearModeling_Fig23.png\|500px]]
		+
		+	In this example, larger fitted values correspond with larger residuals (the opposite may be true as well). Transforming the data often helps resolve such Heteroskedasticity.
		+
		+	Normality of residuals. The normality of residuals assumption (parametric assumptions) is also important. Linear models may be robust and gracefully handle certain some violations of the normality assumption.
		+
		+	# lm.2 = lm(Weight ~ Age, df.2)
		+	par(mfrow=c(1,2))
		+	hist(residuals(lm.2))
		+	qqnorm(residuals(lm.2))
		+	qqline(lm.2.res, col="red")
		+
		+	[[Image:SMHS_LinearModeling_Fig24.png\|500px]]
		+
		+	The histogram (left) is relatively bell-shaped and the Q-Q normal probability plot (right) indicates the residuals are mostly on a straight line (suggesting errors (observed-predicted) are similar to a normal distribution). Thus, there is no strong evidence suggesting possible violation of the normality assumption.
		+
		+	Outliers (influential data points). Data should not include extreme influential points, otherwise the linear model may be heavily biased. Outliers (influential data points) can drastically change the interpretation of model results and inference, similarly to variable collinearity.
		+
		+	The R function dfbeta() computes some of the regression (leave-one-out deletion) diagnostics for linear models and allows us to check for outliers.

← Older revision		Revision as of 13:24, 1 February 2016
Line 684:		Line 684:
	plot(fitted(lm.2),residuals(lm.2))		plot(fitted(lm.2),residuals(lm.2))

		+	[[Image:SMHS_LinearModeling_Fig20.png\|500px]]

		+
		+	Protocols for rectifying residual plots indicating nonlinearity:
		+
		+	Model may be excluding important fixed effects that interact with fixed effects already accounted for in the model. If new fixed effects are added, the pattern in the residual plot may disappear.
		+	*Variable transformation – Perform a nonlinear transformation of the response, e.g., log, or reciprocal transform. See this SOCR Activity .
		+	*Perform a nonlinear transformation to explanatory variables. For instance, if Age is related to Weight in a U‐shaped way (e.g., quadratic relation), then the model could include Age and Age2 as predictors.
		+	*If residual plots include stripes, then there may be categorical variables playing role, which may necessitate alternative class of models, such as logistic or multinomial models.

← Older revision		Revision as of 13:22, 1 February 2016
Line 637:		Line 637:

	Linearity: The response variable (Y=Weight, in our case) can be expressed via a linear combination formula of the independent variable (e.g., Height, Age, etc.) If it assumption is invalid, the residual plot may include patterns (e.g., quadratic curve or non-linear trend) indicating another type of model may be appropriate. For instance, the residuals may include a pair of lines when we have dichotomous categorical data. Similarly, the QQ Normal Probability Plot may not be perfectly linear (e.g., S-shaped), which again indicates the distribution of the residuals (observed minus fitted (predicted) values) may not be IID Normal (Gaussian distributed).		Linearity: The response variable (Y=Weight, in our case) can be expressed via a linear combination formula of the independent variable (e.g., Height, Age, etc.) If it assumption is invalid, the residual plot may include patterns (e.g., quadratic curve or non-linear trend) indicating another type of model may be appropriate. For instance, the residuals may include a pair of lines when we have dichotomous categorical data. Similarly, the QQ Normal Probability Plot may not be perfectly linear (e.g., S-shaped), which again indicates the distribution of the residuals (observed minus fitted (predicted) values) may not be IID Normal (Gaussian distributed).
		+
		+	This plot shows the age/pitch relationship along with the depiction of the residuals:
		+	# Weight ~ Age + Height + Position + Team + ε
		+	# lm.4 = lm(Weight ~ Age.centered + Height+ Position + Team, df.2)
		+
		+	lm.4.res = resid(lm.4)
		+	plot(Height, lm.4.res, ylab="(Weight) Residuals", xlab="Height", main="MLB (lm.4) model residuals")
		+	abline(0, 0)
		+
		+
		+	[[Image:SMHS_LinearModeling_Fig17.png\|500px]]
		+
		+	# Weight ~ Age + Height + Position + Team + ε
		+	# lm.4 = lm(Weight ~ Age.centered + Height+ Position + Team, df.2)
		+	# lm.4.res = resid(lm.4)
		+	qqnorm(lm.4.res) # A quantile normal plot - good for checking normality
		+	qqline(lm.4.res)
		+
		+	[[Image:SMHS_LinearModeling_Fig18.png\|500px]]
		+
		+	Mind that now, for illustrative purposes, we change the model to (lm.2: Weight ~ Age)!
		+
		+	# lm.2 = lm(Weight ~ Age, df.2)
		+	# calculate residuals and predicted values
		+	lm.2.res = resid(lm.2)
		+	residuals <- signif(lm.2.res, 5)
		+	predicted <- predict(lm.2)
		+
		+	plot(Age, Weight, main="Visualization of the Residuals of Model: Weight ~ Age", xlab="Age", ylab="Weight", pch=19)
		+	abline(lm(Weight ~ Age), col="red")
		+	# plot distances between points and the regression line
		+	segments(Age, Weight, Age, predicted, col="blue")
		+
		+	# add residual-value labels to (Age) points
		+	# install.packages("calibrate")
		+	# library("calibrate", lib.loc="~/R/win-library/3.1")
		+	library(calibrate)
		+	textxy(Age, Weight, residuals, cx=0.7)
		+
		+	[[Image:SMHS_LinearModeling_Fig19.png\|500px]]
		+
		+	The <U>blue</U> lines indicate the (magnitude of the) residuals and represent the deviations of the observed data points from the model-predicted values (fitted values). The sum of all residuals (positive and negative) is zero, ensured by the least-squares method for estimating the linear model parameters (intercept and slope).
		+
		+	An alternative view is to plot the fitted values (predicted means) on the horizontal line and the residuals, as deviations, on the vertical line.
		+
		+	plot(fitted(lm.2),residuals(lm.2))
		+
		+
		+
		+
		+
		+
		+

@@ Line 629: / Line 629: @@
   coefplot(lm.3a, vertical=FALSE, add=TRUE, col.pts="black", offset=0.4)
   coefplot(lm.4, vertical=FALSE, add=TRUE, col.pts="blue", offset=0.6)
 <hr>
 * SOCR Home page: http://www.socr.umich.edu

← Older revision		Revision as of 16:00, 26 January 2016
Line 633:		Line 633:
	<hr>		<hr>
	* SOCR Home page: http://www.socr.umich.edu		* SOCR Home page: http://www.socr.umich.edu
−	~~{{translate\|pageName=http://wiki.socr.umich.edu/index.php?title=SMHS_LinearModeling}}~~