SMHS PowerSensitivitySpecificity - Revision history

Pineaumi: /* Experimental Designs */

2016-05-13T20:36:30Z

‎Experimental Designs

Pineaumi: /* Experimental Designs */

2016-05-13T20:29:17Z

‎Experimental Designs

Pineaumi: /* Power Analysis/Psychometrics */

2016-05-13T20:24:04Z

‎Power Analysis/Psychometrics

Pineaumi: /* R power analysis */

2016-05-13T20:20:28Z

‎R power analysis

Pineaumi: /* Power Analysis/Psychometrics */

2016-05-13T20:18:24Z

‎Power Analysis/Psychometrics

Pineaumi: /* Power Analysis/Psychometrics */

2016-05-13T20:16:33Z

‎Power Analysis/Psychometrics

Pineaumi: /* Power Analysis/Psychometrics */

2016-05-13T20:16:01Z

‎Power Analysis/Psychometrics

Pineaumi: /* Power Analysis/Psychometrics */

2016-05-13T20:15:20Z

‎Power Analysis/Psychometrics

Pineaumi: /* Creating Power or Sample Size Plots */

2016-05-13T20:11:34Z

‎Creating Power or Sample Size Plots

Pineaumi: /* Psychometrics */

2016-05-13T20:09:52Z

‎Psychometrics

@@ Line 269: / Line 269: @@
 *Cohen’s protocol classifies <i>f</i>2 values of 0.02, 0.15, and 0.35 as small, medium, and large effect sizes.
 <sup>5</sup>http://wiki.socr.umich.edu/index.php/SMHS_MLR
 <b>Unequal n's

@@ Line 175: / Line 175: @@
 ====Experimental Designs====
-<b>T-tests
+<b>T-tests <sup>3</sup>
 &#35;  install.packages("pwr"); library("pwr")
@@ Line 194: / Line 194: @@
 *<b>type</b> 	specifies two-sample t-test, one-sample t-test or paired t-test.
 <b>Unequal sample-sizes
@@ Line 213: / Line 215: @@
 Where $μ_1$ and $μ_2$ are the group 1 and 2 means, and $σ^2$ is the common error variance.
-<b>1-Way ANOVA
+<b>1-Way ANOVA<sup>4</sup>
 &#35; pwr.anova.test(k = , n = , f = , sig.level = , power =
@@ Line 230: / Line 232: @@
 *In the Cohen’s protocol, effect-sizes (<i>f</i>) values of <i>0.1, 0.25</i>, and <i>0.4</i> represent small, medium, and large effects.
 <b>Correlations
@@ Line 241: / Line 245: @@
 Again, the Cohen’s protocol classifies <i>r</i> values of <i>0.1, 0.3</i>, and <i>0.5</i> as small, medium, and large correlations.
-<b>Linear Models</b>
+<b>Linear Models<sup>5</sup></b>
 *For all linear models (e.g., multiple regression) we can use:
@@ Line 264: / Line 268: @@
 *Cohen’s protocol classifies <i>f</i>2 values of 0.02, 0.15, and 0.35 as small, medium, and large effect sizes.
 <b>Unequal n's

@@ Line 87: / Line 87: @@
 Questions:
-*Why are sample-sizes important and how do they influence our ability to design experiments tailored to detect effect of certain magnitude? Recall the first (LLN) and the second (CLT) fundamental laws of probability theory.
+*Why are sample-sizes important and how do they influence our ability to design experiments tailored to detect effect of certain magnitude? Recall the first (LLN) and the second (CLT) fundamental laws of probability theory<sup>1</sup>.
 *How to assess the reliability and validity of newly introduced psychometric instruments?
-<b>Statistical Power Analysis</b>
+<sup>1</sup>http://wiki.socr.umich.edu/index.php/EBook#Chapter_VI:_Relations_Between_Distributions
 Power analysis is a technique employed in many experimental designs to determine the relation between sample size of the study, specific effect sizes, and ability to detect these effects from sample data given its size and a degree of confidence. In a nutshell, the statistical power is a proxy for the probability of detecting an effect (signal, discrepancy, misalignment, etc.) of a given size, provided we start with a given level of confidence and some sample size constraints. If this probability is very low (e.g., p<0.5), the study may be under-powered which may require a redesign or reformulation of the experiment. If the power is very high (e.g., p>0.999), then we can tradeoff (e.g., lower the sample-size, but preserve higher power to detect effects) various parameters in the power calculations.
@@ Line 128: / Line 130: @@
 This is because    $power=1- β =1-\frac{FN}{FN+TP}=\frac{FN+TP}{FN+TP}-\frac{FN}{FN+TP}=\frac{TP}{FN+TP}=sensitivity$
 ====R power analysis====

@@ Line 136: / Line 136: @@
 {| class="wikitable" style="text-align:left; " border="1"
 |-
-|Function||Purpose
+|<b>Function</b>||<b>Purpose</b>
 |-
 |pwr.2p.test||Two proportions (equal n)

@@ Line 116: / Line 116: @@
-The table below gives an example of calculating specificity, sensitivity, False positive rate α, False Negative Rate, β, and power given the information about <i>TN</i> and <i>FN</i>.
+The table below gives an example of calculating specificity, sensitivity, False positive rate $α$, False Negative Rate, $β$, and power given the information about <i>TN</i> and <i>FN</i>.
 <mark>!!!!INSERT TABLE!!!!</mark>
-*Specificity = TN/(TN+FP), Sensitivity = TP/(TP+FN), α= FP/(TN+FP), β = FN/(TP+FN), ''power''=1-β.
+*Specificity = TN/(TN+FP), Sensitivity = TP/(TP+FN), $α$= FP/(TN+FP), $β$ = FN/(TP+FN), ''power''=1-$β$.
-Note that both (<i>Type I (α)</i> and <i>Type II (β)</i>) errors are proportions in the range [0,1], so they represent ''error-rates''. The reason they are listed in the corresponding cells is that they are directly proportional to the numerical values of the FP and FN, respectively.
+Note that both (<i>Type I ($α$)</i> and <i>Type II ($β$)</i>) errors are proportions in the range [0,1], so they represent ''error-rates''. The reason they are listed in the corresponding cells is that they are directly proportional to the numerical values of the FP and FN, respectively.
 *Note that the two alternative definitions of power are equivalent:

@@ Line 101: / Line 101: @@
 . The <b>effect size</b> (must come outside of the study, can’t use the same data to estimate it. E.g., prior pubs, previous reports of expected effects, etc.)
-. <b>Significance level</b> is the α=P(<i>Type I error</i>)= probability of finding a (spurious) effect that is not there in reality, but is due to random chance alone
+. <b>Significance level</b> is the $α$=P(<i>Type I error</i>)= probability of finding a (spurious) effect that is not there in reality, but is due to random chance alone
-. The <b>statistical power</b> β= 1 - P(<i>Type II error</i>) = probability of finding an effect that is there. In some specific experimental designs (but not always), given any three of these components, we can determine the fourth. For instance, we can estimate the (minimum) sample-size that yields a power lower-bound by, say, β=0.8.
+. The <b>statistical power</b> $β$= 1 - P(<i>Type II error</i>) = probability of finding an effect that is there. In some specific experimental designs (but not always), given any three of these components, we can determine the fourth. For instance, we can estimate the (minimum) sample-size that yields a power lower-bound by, say, $β$=0.8.
 *<b>Type I error</b>: the false positive (Type I) error of rejecting the null hypothesis given that it is actually true; e.g., the purses are detected to containing the radioactive material while they actually do not.

@@ Line 95: / Line 95: @@
 Power analysis is a technique employed in many experimental designs to determine the relation between sample size of the study, specific effect sizes, and ability to detect these effects from sample data given its size and a degree of confidence. In a nutshell, the statistical power is a proxy for the probability of detecting an effect (signal, discrepancy, misalignment, etc.) of a given size, provided we start with a given level of confidence and some sample size constraints. If this probability is very low (e.g., p<0.5), the study may be under-powered which may require a redesign or reformulation of the experiment. If the power is very high (e.g., p>0.999), then we can tradeoff (e.g., lower the sample-size, but preserve higher power to detect effects) various parameters in the power calculations.
-Important components of power calculations:
+Important components of power calculations<sup>2</sup>:
 . The <b>sample size</b> (either investigator controlled, or estimated)

@@ Line 308: / Line 308: @@
 ===Creating Power or Sample Size Plots===
-The functions in the pwr package can be used to generate power and sample size graphs.
+The functions in the <b>pwr</b> package can be used to generate power and sample size graphs.
   &#35; Plot sample size curves for detecting correlations of

← Older revision		Revision as of 20:09, 13 May 2016
Line 359:		Line 359:

	*Deployment of procedures and analytic strategies for interpreting the data generated by the instruments		*Deployment of procedures and analytic strategies for interpreting the data generated by the instruments
		+

	Reliability and validity are core psychometric components. '''Reliable measures''' are consistently across time, individuals, and situations, whereas '''valid measures''' assess the intended process (instead of tracking a tangential characteristics). Validity implies reliability (necessary, but not sufficient, for validity), but not vice-versa. '''Cronbach's α''' is the most commonly used index of reliability along with the '''intra-class correlation,''' which is the ratio of variance of measurements of a given target to the variance of all targets.		Reliability and validity are core psychometric components. '''Reliable measures''' are consistently across time, individuals, and situations, whereas '''valid measures''' assess the intended process (instead of tracking a tangential characteristics). Validity implies reliability (necessary, but not sufficient, for validity), but not vice-versa. '''Cronbach's α''' is the most commonly used index of reliability along with the '''intra-class correlation,''' which is the ratio of variance of measurements of a given target to the variance of all targets.