SMHS LinearModeling MLR - Revision history

Pineaumi: /* SMHS Linear Modeling - Multiple Linear Regression */

2016-05-23T13:15:12Z

‎SMHS Linear Modeling - Multiple Linear Regression

Imoubara at 12:55, 23 May 2016

2016-05-23T12:55:19Z

Imoubara at 12:52, 23 May 2016

2016-05-23T12:52:49Z

Pineaumi: /* SMHS Linear Modeling - Multiple Linear Regression */

2016-05-20T13:01:54Z

‎SMHS Linear Modeling - Multiple Linear Regression

Show changes

Pineaumi: /* SMHS Linear Modeling - Multiple Linear Regression */

2016-05-20T12:33:11Z

‎SMHS Linear Modeling - Multiple Linear Regression

Pineaumi: /* SMHS Linear Modeling - Multiple Linear Regression */

2016-05-19T19:59:52Z

‎SMHS Linear Modeling - Multiple Linear Regression

Pineaumi: /* SMHS Linear Modeling - Multiple Linear Regression */

2016-05-19T17:46:36Z

‎SMHS Linear Modeling - Multiple Linear Regression

Pineaumi: /* SMHS Linear Modeling - Multiple Linear Regression */

2016-05-19T17:26:49Z

‎SMHS Linear Modeling - Multiple Linear Regression

Pineaumi: /* SMHS Linear Modeling - Multiple Linear Regression */

2016-05-19T13:44:54Z

‎SMHS Linear Modeling - Multiple Linear Regression

Pineaumi: /* SMHS Linear Modeling - Multiple Linear Regression */

2016-05-18T17:30:19Z

‎SMHS Linear Modeling - Multiple Linear Regression

@@ Line 338: / Line 338: @@
 These examples illustrate "multiple linear regression" as a linear modeling technique involving 1 response (dependent) variable expressed as a (affine) function of multiple predictor (independent) variables.
-<sup>1</sup>http://www.statmethods.net/input/datatypes.html <br>  <sup>2</sup>http://www.statmethods.net/advstats/glm.html <br>
+* <sup>1</sup> http://www.statmethods.net/input/datatypes.html
-<sup>3</sup>http://www.statmethods.net/management/subset.html
+* <sup>2</sup> http://www.statmethods.net/advstats/glm.html
 ===See next===

@@ Line 104: / Line 104: @@
 The report shows:
-<li>The model analytical formula specified by the lm() call. </li>
+*The model analytical formula specified by the lm() call.
-<li>The residuals (errors, discrepancies between observed and model-predicted outcomes). </li>
+*The residuals (errors, discrepancies between observed and model-predicted outcomes).
-<li>The coefficients of the fixed effects (predictors, explanatory variables). </li>
+*The coefficients of the fixed effects (predictors, explanatory variables).
-<li>And finally, the overall model quality (describing the ability of the model to describe the linear relations in these data).  "Multiple R-squared" refers to the R<sup>2</sup> statistic that measures the "variance explained by the model" or "variance accounted for by the model". 0≤R<sup>2</sup>≤1, in our result, R<sup>2</sup> = 0.4293, which is good, but not great. In essence, 42.93% of the data variability may be explained by this specific (best linear fit) model. In this case, the model solely relies on "Treatment" (fixed effect) to explain Obs (outcome). So, the R<sup>2</sup> reflects how much of the Obs variance is accounted for by different Treatments (A or B).</li>
+*And finally, the overall model quality (describing the ability of the model to describe the linear relations in these data).  "Multiple R-squared" refers to the R<sup>2</sup> statistic that measures the "variance explained by the model" or "variance accounted for by the model". 0≤R<sup>2</sup>≤1, in our result, R<sup>2</sup> = 0.4293, which is good, but not great. In essence, 42.93% of the data variability may be explained by this specific (best linear fit) model. In this case, the model solely relies on "Treatment" (fixed effect) to explain Obs (outcome). So, the R<sup>2</sup> reflects how much of the Obs variance is accounted for by different Treatments (A or B).
 Models with high R<sup>2</sup> values are preferred, subject to 2 conditions:
-<li>What is considered a high R<sup>2</sup> value is relative and depends on study/application.</li>
+*What is considered a high R<sup>2</sup> value is relative and depends on study/application.
-<li>When the study phenomenon is highly deterministic, R<sup>2</sup> values can approach 1. </li>
+*When the study phenomenon is highly deterministic, R<sup>2</sup> values can approach 1.
 Higher number of explanatory variable and higher model-complexity tend to yield higher R<sup>2</sup> values, but are more difficult to interpret.
@@ Line 119: / Line 119: @@
 The statistical test of "model significance" is quantified by the output "F-statistic: 3.008 on 1 and 4 DF,  p-value: 0.1579". Under a null hypothesis where the model captures little or no information about the process, the probability value quantifies the data-driven evidence to reject the null and accept an alternative stating that this model does capture useful patterns in the data. Specifically, the p-value represents the conditional probability under the condition that the null hypothesis is true. In this case,
-<li>The null hypothesis is Ho: "Treatment has no effect on Obs". </li>
+*The null hypothesis is Ho: "Treatment has no effect on Obs".
-<li>Alternative research Hypothesis is H1: "Treatment has effect on Obs".</li>
+*Alternative research Hypothesis is H1: "Treatment has effect on Obs".
 This p-value is 0.1579, and the linear model fails to reject the null hypothesis. This leaves open the possibility that Treatment may have no effect on Obs (just based on using these 6 data points). <BR>
@@ Line 333: / Line 333: @@
 Notice the linear model coefficients, the corresponding p-values, and the overall model quality, at the bottom of the output change. Are these changed between the 2 models?
-<LI>new model (Weight ~ Age + Height + Position + Team + e.) vs. </LI>
+*new model (Weight ~ Age + Height + Position + Team + e.) vs.
-<LI>old model   (Weight ~ Age + Height + e)? </LI>
+*old model   (Weight ~ Age + Height + e)?
 These examples illustrate "multiple linear regression" as a linear modeling technique involving 1 response (dependent) variable expressed as a (affine) function of multiple predictor (independent) variables.

@@ Line 4: / Line 4: @@
 '''Questions''':
-<li>Are there (linear) associations between predictors and a response variable(s)?</li>
+*Are there (linear) associations between predictors and a response variable(s)?
-<li>When to look for linear relations and what assumptions play role?</li>
+*When to look for linear relations and what assumptions play role?
 Let’s use some of the data included in the Appendix. Snapshots of the first few rows in the data are shown below:

@@ Line 109: / Line 109: @@
 <li>The residuals (errors, discrepancies between observed and model-predicted outcomes). </li>
 <li>The coefficients of the fixed effects (predictors, explanatory variables). </li>
-<li>And finally, the overall model quality (describing the ability of the model to describe the linear relations in these data).  "Multiple R-squared" refers to the R<sup>2</sup> statistic that measures the "variance explained by the model" or "variance accounted for by the model". 0=R<sup>2</sup>=1, in our result, R<sup>2</sup> = 0.4293, which is good, but not great. In essence, 42.93% of the data variability may be explained by this specific (best linear fit) model. In this case, the model solely relies on "Treatment" (fixed effect) to explain Obs (outcome). So, the R<sup>2</sup> reflects how much of the Obs variance is accounted for by different Treatments (A or B).</li>
+<li>And finally, the overall model quality (describing the ability of the model to describe the linear relations in these data).  "Multiple R-squared" refers to the R<sup>2</sup> statistic that measures the "variance explained by the model" or "variance accounted for by the model". 0≤R<sup>2</sup>≤1, in our result, R<sup>2</sup> = 0.4293, which is good, but not great. In essence, 42.93% of the data variability may be explained by this specific (best linear fit) model. In this case, the model solely relies on "Treatment" (fixed effect) to explain Obs (outcome). So, the R<sup>2</sup> reflects how much of the Obs variance is accounted for by different Treatments (A or B).</li>
 Models with high R<sup>2</sup> values are preferred, subject to 2 conditions:

@@ Line 124: / Line 124: @@
 <li>Alternative research Hypothesis is H1: "Treatment has effect on Obs".</li>
-This p-value is 0.1579, and the linear model fails to reject the null hypothesis. This leaves open the possibility that Treatment may have no effect on Obs (just based on using these 6 data points).
+This p-value is 0.1579, and the linear model fails to reject the null hypothesis. This leaves open the possibility that Treatment may have no effect on Obs (just based on using these 6 data points). <BR>
+However, if the p-value was much smaller, and assuming Ho were true, then this data would be less likely to be observed in reality (hence we would interpret small p-values as showing that the alternative hypothesis "Treatment affects Obs" as more likely and the model result is "statistically significant"). <BR>
 There is a distinction between the overall "model-significance" (as quantified by the p-value at the very bottom of the output, which considers all effects together) and the p-value of individual effects (coefficients table including significance for each predictor).
-The model F-statistic and the degrees of freedom are in 1-1 correspondence with the p-value – the latter is computed as the right tail probability for an F(df1,df2) distribution corresponding to the F-statistics (critical value). To report the overall model inference in this case, we can state:
+The model F-statistic and the degrees of freedom are in 1-1 correspondence with the p-value – the latter is computed as the right tail probability for an F(df1,df2) distribution corresponding to the F-statistics (critical value). To report the overall model inference in this case, we can state:<BR>
+<I>"Using a simple linear model of Obs as a function of Treatment, the model was not significant (F(1,4)=3.001, p>0.15), which indicates that "Treatment" may not be a critical explanatory factor describing the "Obs" outcome."</I><BR>
 To examine the coefficient table for (lm.1 model).
@@ Line 161: / Line 157: @@
 <U>7.108776</U> - 0.5327 = 6.57611,
-which is the mean of the "TreatmentB" cohort. Thus,
+which is the mean of the "TreatmentB" cohort. Thus, <BR>
-the estimate for "(Intercept)" represents for the "TreatmentA" category, and
+the estimate for "(Intercept)" represents for the "TreatmentA" category, and <BR>
 the estimate for "TreatmentB" represents the difference between the "Treatment" "A" and "B" categories.
@@ Line 200: / Line 196: @@
   summary(lm.2)
+ Call:
-Call:<BR>
+ lm(formula = Weight ~ Age, data = df.2)
 lm(formula = Weight ~ Age, data = df.2)
+ Residuals:
-Residuals:<BR>
+ Min      1Q  Median      3Q     Max
-Min      1Q  Median      3Q     Max <BR>
+ -52.479 -14.489  -0.814  13.400  89.915
 -52.479 -14.489  -0.814  13.400  89.915
+ Coefficients:<BR>
-Coefficients:<BR>
+ Estimate Std. Error t value Pr(>|t|)
-Estimate Std. Error t value Pr(>|t|)<BR>
+ (Intercept) 179.6684     4.3418  41.381  < 2e-16 ***
-(Intercept) 179.6684     4.3418  41.381  < 2e-16 ***<BR>
+ Age           0.7672     0.1494   5.135 <MARK>3.37e-07</MARK> ***
-Age           0.7672     0.1494   5.135 <MARK>3.37e-07</MARK> ***<BR>
+ ---<BR>
----<BR>
+ Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
-Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1<BR>
+ Residual standard error: 20.75 on 1032 degrees of freedom
-Residual standard error: 20.75 on 1032 degrees of freedom<BR>
+ Multiple R-squared:  <MARK>0.02492</MARK>,	Adjusted R-squared:  0.02397
-Multiple R-squared:  <MARK>0.02492</MARK>,	Adjusted R-squared:  0.02397<BR>
+ F-statistic: 26.37 on 1 and 1032 DF,  p-value: <MARK>3.368e-07</MARK>
 The significance of the intercept is not very interesting, as it just specifies the baseline.  However, the p-value in each (predictor) row evaluates whether the corresponding coefficient is significantly non-trivial. Trivial coefficients can be removed from the model. The intercept (179.6684) represents the predicted Weight for a player at Age 0, which doesn’t make sense for Baseball players, but it captures the constant impact of Age on Weight.

@@ Line 230: / Line 230: @@
 <CENTER>[[Image:SMHS_LinearModeling_Fig15.png|500px]]</CENTER>
-Interpreting Model Intercepts
+'''Interpreting Model Intercepts'''<BR>
 The model estimates of the intercept may not always be useful. For instance, if we subtract the mean Age from each player's Age:

@@ Line 37: / Line 37: @@
 This "~" (tilde) notation implies "Weight predicted by Race" or "Observation as a linear function of Treatment". The "dependent variable" (a measurable response) on the left is predicted by the factor on the right acting as an "independent variable", co-variate, predictor, "explanatory variable", or "fixed effect".
-Often times inter-dependencies may not be perfect, deterministic, or rigid (like in the case of predicting the Area of a disk knowing its radius, or predicting the 3D spatial location of a planet having a precise date and time). Weight may not completely and uniquely determined by Race alone, as many different factors play role (genetics, environment, aging, etc.) Even if we can measure all factors we can identify as potentially influencing Weight, there will still be intrinsic random variability into the observed Weight measures, which can’t be control for. This intrinsic random variation may be captured and accounted for using "random" term at the end.
+Often times inter-dependencies may not be perfect, deterministic, or rigid (like in the case of predicting the Area of a disk knowing its radius, or predicting the 3D spatial location of a planet having a precise date and time). Weight may not be completely and uniquely determined by Race alone, as many different factors play role (genetics, environment, aging, etc.) Even if we can measure all factors we can identify as potentially influencing Weight, there will still be intrinsic random variability into the observed Weight measures, which can’t be control for. This intrinsic random variation may be captured and accounted for using "random" term at the end.
-Weight ~ Race + e ~ D(m=0, s).
+Weight ~ Race + ε~ D(m=0, s).
-Epsilon "e" represents the error term of predicting Weight by Gender alone and summarizes the aggregate impact of all factors aside from Race that impact individual’s Weight, which are experimentally uncontrollable or random. This formula is a schematic analytic representation of a linear model that we’re going to estimate, quantify, and use for prediction and inference. The right hand side splits the knowledge representation of Weight into 2 complementary components - a "fixed effect" for Race, which we understand and expect, and a "random effect" ("e") what we don’t know well. (W ~ R represents the "structural" or "systematic" part of the linear model and "e" stands for the "random" or "probabilistic" part of the model.
+Epsilon "ε" represents the error term of predicting Weight by Gender alone and summarizes the aggregate impact of all factors aside from Race that impact individual’s Weight, which are experimentally uncontrollable or random. This formula is a schematic analytic representation of a linear model that we’re going to estimate, quantify, and use for prediction and inference. The right hand side splits the knowledge representation of Weight into 2 complementary components - a "fixed effect" for Race, which we understand and expect, and a "random effect" ("ε") what we don’t know well. (W ~ R represents the "structural" or "systematic" part of the linear model and "ε" stands for the "random" or "probabilistic" part of the model.
 <B>R Experiment (See Appendix)</B>
@@ Line 75: / Line 75: @@
 </center>
-Using the linear model Obs ~ Treatment + e, we can invoke the linear modeling function
+Using the linear model Obs ~ Treatment + ε, we can invoke the linear modeling function
-lm()<sup>2</sup> . The "e" term is implicit in all models so it need not be specified.
+lm()<sup>2</sup> . The "ε" term is implicit in all models so it need not be specified.
   lm.1 <- lm(Obs ~ Treatment, mydata1)
@@ Line 109: / Line 109: @@
 <li>The residuals (errors, discrepancies between observed and model-predicted outcomes). </li>
 <li>The coefficients of the fixed effects (predictors, explanatory variables). </li>
-<li>And finally, the overall model quality (describing the ability of the model to describe the linear relations in these data).  "Multiple R-squared" refers to the R2 statistic that measures the "variance explained by the model" or "variance accounted for by the model". 0=R2=1, in our result, R2 = 0.4293, which is good, but not great. In essence, 42.93% of the data variability may be explained by this specific (best linear fit) model. In this case, the model solely relies on "Treatment" (fixed effect) to explain Obs (outcome). So, the R2 reflects how much of the Obs variance is accounted for by different Treatments (A or B).</li>
+<li>And finally, the overall model quality (describing the ability of the model to describe the linear relations in these data).  "Multiple R-squared" refers to the R<sup>2</sup> statistic that measures the "variance explained by the model" or "variance accounted for by the model". 0=R<sup>2</sup>=1, in our result, R<sup>2</sup> = 0.4293, which is good, but not great. In essence, 42.93% of the data variability may be explained by this specific (best linear fit) model. In this case, the model solely relies on "Treatment" (fixed effect) to explain Obs (outcome). So, the R<sup>2</sup> reflects how much of the Obs variance is accounted for by different Treatments (A or B).</li>
 Models with high R<sup>2</sup> values are preferred, subject to 2 conditions:
 <li>What is considered a high R<sup>2</sup> value is relative and depends on study/application.</li>
-<li>When the study phenomenon is highly deterministic, R<sup>2</sup> values can be approach 1. </li>
+<li>When the study phenomenon is highly deterministic, R<sup>2</sup> values can approach 1. </li>
 Higher number of explanatory variable and higher model-complexity tend to yield higher R<sup>2</sup> values, but are more difficult to interpret.
-The "Adjusted R-squared" value is a modified R<sup>2</sup> value that normalizes the total variance "explained" by the model by the number of fixed effects included in the model. Larger number predictors will lower the "Adjusted R-squared". The adjusted R2adj = 0.2866, but in general, R2adj  can be much lower when the model includes many fixed effects.
+The "Adjusted R-squared" value is a modified R<sup>2</sup> value that normalizes the total variance "explained" by the model by the number of fixed effects included in the model. Larger number predictors will lower the "Adjusted R-squared". The adjusted R<sup>2</sup>adj = 0.2866, but in general, R<sup>2</sup>adj  can be much lower when the model includes many fixed effects.
 The statistical test of "model significance" is quantified by the output "F-statistic: 3.008 on 1 and 4 DF,  p-value: 0.1579". Under a null hypothesis where the model captures little or no information about the process, the probability value quantifies the data-driven evidence to reject the null and accept an alternative stating that this model does capture useful patterns in the data. Specifically, the p-value represents the conditional probability under the condition that the null hypothesis is true. In this case,

@@ Line 51: / Line 51: @@
+  )
-We construct an R frame object<sup>1</sup>concatenating 3 data-elements for 6 subjects, and saving it into "mydata1."
+We construct an R frame object<sup>1</sup> concatenating 3 data-elements for 6 subjects, and saving it into "mydata1."
 <center>
@@ Line 92: / Line 92: @@
 Residuals:<BR>
         2        3        4        5        6<BR>
 -0.10342  0.44100 -0.37540  0.03782 -0.27881  0.27881<BR>
 Coefficients:<BR>
-            Estimate Std. Error t value Pr(>|t|)<BR>
+Estimate Std. Error t value Pr(>|t|)<BR>
 (Intercept)   7.1088     0.2507  28.350 9.21e-06 ***<BR>
 TreatmentB   -0.5327     0.3071  -1.734    0.158<BR>
@@ Line 151: / Line 151: @@
   mean(mydata1[mydata1$\$$Treatment=="B",]$\$$Obs)
   > 6.57611
-  # in general, subsetting   can be accomplished by:
+  # in general, subsetting<sup>3</sup> can be accomplished by:
-  subset_data <- mydata1[ which(mydata1$\$$Treatment=='A' & mydata1$\$$Day=='Day2'),]
+  subset_data <- mydata1[ which(mydata1$\$$Treatment=='A' &
 The mean of the Obs values for {Treatment=A} is the same as the "Intercept" term.
@@ Line 180: / Line 181: @@
 The linear model represents the difference between Treatments "A" and "B" as a slope. Going From Treatment "A" to "B" we go down -0.5327 (in terms of the units measuring the "Obs" variable), which is exactly the "TreatmentB" coefficient, relative to Treatment "A".
 Treatment "A" acts as baseline (center of the local coordinate system) and is represented by the "(Intercept)". The difference between Treatments "A" and "B" is expressed as a slope heading down from 7.108776 by 0.5327. The p-values in the coefficient table correspond to the significance that each coefficient (intercept or Treatment) is "non-trivial".
@@ Line 261: / Line 261: @@
 Residuals:<BR>
-    Min      1Q  Median      3Q     Max<BR>
+Min      1Q  Median      3Q     Max<BR>
 -50.818 -12.148  -0.344  10.720  74.175<BR>
@@ Line 344: / Line 344: @@
 These examples illustrate "multiple linear regression" as a linear modeling technique involving 1 response (dependent) variable expressed as a (affine) function of multiple predictor (independent) variables.
 ===See next===