SMHS CIs - Revision history

Dinov: /* Maximum likelihood estimation (MLE) */

2014-11-10T13:01:03Z

‎Maximum likelihood estimation (MLE)

Dinov: /* Theory */

2014-10-26T17:45:55Z

‎Theory

Dinov at 16:50, 26 October 2014

2014-10-26T16:50:05Z

Dinov: /* Hands-on Activity */

2014-09-03T15:07:47Z

‎Hands-on Activity

Dinov: /* Estimating a population proportion */

2014-09-03T14:31:42Z

‎Estimating a population proportion

Dinov: /* Maximum likelihood estimation (MLE) */

2014-09-03T14:19:15Z

‎Maximum likelihood estimation (MLE)

Dinov: /* Method of Moments (MOM) Estimation */

2014-09-03T14:18:50Z

‎Method of Moments (MOM) Estimation

Dinov: /* Method of Moments (MOM) Estimation */

2014-09-03T14:18:22Z

‎Method of Moments (MOM) Estimation

Dinov: /* Method of Moments (MOM) Estimation */

2014-09-03T14:18:03Z

‎Method of Moments (MOM) Estimation

Dinov: /* Method of Moments (MOM) Estimation */

2014-09-03T14:17:34Z

‎Method of Moments (MOM) Estimation

@@ Line 51: / Line 51: @@
 * Note: The MLE may not be unique and is not guaranteed to exist.
-* Example: Consider the coin flipping example above in which we observed the number of head, and use this to infer the true probability of p(Head).
+* Example: Consider the coin flipping example above in which we observed the number of heads, and use this to infer the true probability of p(Head).
 : Likelihood function: $L(\theta)=f(x│\theta=p)={10 \choose 4} p^4 (1-p)^6$
 : Log-likelihood function: $L^* (\theta)=\ln{10 \choose 4} + 4\ln{p} + 6\ln{(1-p)}$.

@@ Line 56: / Line 56: @@
 : Maximize the log-likelihood function by setting its first derivative to zero:
 $$ 0=\frac{d(\ln{10 \choose 4} + 4\ln{p} + 6\ln{(1-p))}}{dp} =4/p-6/(1-p), p=2/5.$$
 ====MOM vs. MLE====

@@ Line 58: / Line 58: @@
 ====MOM vs. MLE====
 * The MOM is inferior to Fisher’s MLE method because MLE has a higher probability of being close to the quantities to be estimated.
 * MLE may be intractable in some situations, whereas the MOM estimates can be quickly and easily calculated by hand or using a computer.
@@ Line 64: / Line 65: @@
 * MOM estimates are not necessarily sufficient statistics, i.e., they sometimes fail to take into account all relevant information in the sample.
 * MOM may be preferred to MLE for estimating some structural parameters when appropriate probability distributions are unknown.
 ===Student’s T Distribution===

@@ Line 232: / Line 232: @@
 : <math>0.0564=\sqrt{15\times 0.0053 \over 24.9958} \leq \sigma \leq \sqrt{15\times 0.0053 \over 7.261}=0.10464</math>
-** [[AP_Statistics_Curriculum_2007_Estim_Var#More_Examples|See more examples here]].
+* [[AP_Statistics_Curriculum_2007_Estim_Var#More_Examples|See more examples here]].
 ===Applications===

@@ Line 174: / Line 174: @@
 * Example: Suppose a researcher is interested in studying the effect of aspirin in reducing heart attacks. He randomly recruits 500 subjects with evidence of early heart disease and has them take one aspirin daily for two years.  At the end of the two years, he finds that during the study only 17 subjects had a heart attack. Calculate a 95% (<math>\alpha=0.05</math>) confidence interval for the true (unknown) proportion of subjects with early heart disease that have a heart attack while taking aspirin daily. Note that [[AP_Statistics_Curriculum_2007_Normal_Critical | <math>z_{\alpha \over 2} = z_{0.025}=1.96</math>]]:
-:: <math>\hat{p} = {17\over 500}=0.034</math> ; <math>\tilde{p} = {17+0.5z_{0.025}^2\over 500+z_{0.025}^2}== {17+1.92\over 500+3.84}=0.038</math>
+:: <math>\hat{p} = {17\over 500}=0.034;</math>  <math>\tilde{p} = {17+0.5z_{0.025}^2\over 500+z_{0.025}^2}== {17+1.92\over 500+3.84}=0.038</math>
-:: <math>SE_{\hat{p}}= \sqrt{0.034(1-0.034)\over 500}=0.0036</math>; <math>SE_{\tilde{p}}= \sqrt{0.038(1-0.038)\over 500+3.84}=0.0085</math>
+:: <math>SE_{\hat{p}}= \sqrt{0.034(1-0.034)\over 500}=0.0036;</math> <math>SE_{\tilde{p}}= \sqrt{0.038(1-0.038)\over 500+3.84}=0.0085</math>
 ::And the corresponding confidence intervals are given by
 :: <math>\hat{p}\pm 1.96 SE_{\hat{p}}=[0.026944, 0.041056]</math>

@@ Line 47: / Line 47: @@
 The MLE of $\theta$ is the value of $\theta$ that maximizes $L(\theta)$: $\arg\max_\theta{L(\theta)}.$
-It is typically assumed that the observed data are independent and identically distributed (iid) with unknown parameter $\theta$. The likelihood can be written as a product of $n$ univariate probability densities: $L(\theta)=\prod_{i=1}^n {f_\theta (x_i |]theta)}$. Because maxima are unaffected by monotone transformations, one can take the logarithm of this expression and turn it into a sum: $L^* (θ)=\sum_{i=1}^n {\ln{f_θ (x_i |θ)}}$. The maximum of this expression can then be found numerically using various optimization algorithms.
+It is typically assumed that the observed data are independent and identically distributed (iid) with unknown parameter $\theta$. The likelihood can be written as a product of $n$ univariate probability densities: $L(\theta)=\prod_{i=1}^n {f_\theta (x_i |\theta)}$. Because maxima are unaffected by monotone transformations, one can take the logarithm of this expression and turn it into a sum: $L^* (θ)=\sum_{i=1}^n {\ln{f_θ (x_i |θ)}}$. The maximum of this expression can then be found numerically using various optimization algorithms.
 * Note: The MLE may not be unique and is not guaranteed to exist.

@@ Line 35: / Line 35: @@
 The beta distribution mean and variance are defined explicitly in terms of two parameters.
 * Mean: $\mu=\frac{\alpha}{\alpha+\beta}$,
-* Variance: $]sigma^2=\frac{\alpha \beta}{(\alpha+\beta)^2 (\alpha+\beta+1)}$.
+* Variance: $\sigma^2=\frac{\alpha \beta}{(\alpha+\beta)^2 (\alpha+\beta+1)}$.
 The sample mean and sample variance are $\bar{x} = 0.251$, and $s^2=0.6187$. Solve for $\alpha$ and $\beta$.

@@ Line 18: / Line 18: @@
 # Set the sample moments equal to the population moments and solve for a (linear or non-linear) system of $k$ equations with $k$ unknowns.
-* MOM proportion example: Consider the example of observing the clinical outcomes (survival or death) of 10 patients undergoing an experimental treatment for terminally ill cancer patients. We can record the outcomes as $S$ (1 month survival) and $D$ (death within 1 month) and we need to compute (infer) the true probability of $S$, i.e., $p=P(S)$. Suppose we observe the outcome $\{S,D,D,D,D,S,S,D,S,D\}$. In the MOM framework, this is a [[SMHS_ProbabilityDistributions#Binomial_distribution |Binomial experiment]] and the expected number of patients surviving 1 month would be $E[X]=np$, where $X=\sum_{i=1}^{10}{\{X_1+X_2+...+X_{10}\}}$, where
+* MOM proportion example: Consider the example of observing the clinical outcomes (survival or death) of 10 patients undergoing an experimental treatment for terminally ill cancer patients. We can record the outcomes as $S$ (1 month survival) and $D$ (death within 1 month) and we need to compute (infer) the true probability of $S$, i.e., $p=P(S)$. Suppose we observe the outcomes $\{S,D,D,D,D,S,S,D,S,D\}$. In the MOM framework, this is a [[SMHS_ProbabilityDistributions#Binomial_distribution |Binomial experiment]] and the expected number of patients surviving 1 month would be $E[X]=np$, where $X=\sum_{i=1}^{10}{\{X_1+X_2+...+X_{10}\}}$, where
 $$X_i=\begin{cases}
 , & \text{if } i^{th} \text{ subject is alive after 1 month of treatment} \\

@@ Line 18: / Line 18: @@
 # Set the sample moments equal to the population moments and solve for a (linear or non-linear) system of $k$ equations with $k$ unknowns.
-* MOM proportion example: Consider the example of observing the clinical outcomes (survival or death) of 10 patients undergoing an experimental treatment for terminally ill cancer patients. We can record the outcomes as $S$ (1 month survival) and $D$ (death within 1 month) and we need to compute (infer) the true probability of $S$ ($p=P(S)$). Suppose we observe the outcome $\{S,D,D,D,D,S,S,D,S,D\}$. In the MOM framework, this is a [[SMHS_ProbabilityDistributions#Binomial_distribution |Binomial experiment]] and the expected number of patients surviving 1 month would be $E[X]=np$, where $X=\sum_{i=1}^{10}{X_1+X_2+...+X_{10}}$, where $X_i=\begin{cases}
+* MOM proportion example: Consider the example of observing the clinical outcomes (survival or death) of 10 patients undergoing an experimental treatment for terminally ill cancer patients. We can record the outcomes as $S$ (1 month survival) and $D$ (death within 1 month) and we need to compute (infer) the true probability of $S$ ($p=P(S)$). Suppose we observe the outcome $\{S,D,D,D,D,S,S,D,S,D\}$. In the MOM framework, this is a [[SMHS_ProbabilityDistributions#Binomial_distribution |Binomial experiment]] and the expected number of patients surviving 1 month would be $E[X]=np$, where $X=\sum_{i=1}^{10}{\{X_1+X_2+...+X_{10}\}}$, where
-, & \text{if } i^{th} \text{subject is alive after 1 month of treatment} \\
+$$X_i=\begin{cases}
-, & \text{if } i^{th} \text{subject is dead (did not survive) after 1 month of treatment}
+, & \text{if } i^{th} \text{ subject is alive after 1 month of treatment} \\
-\end{cases}. Hence, $np=4$ (4 patients did survive for a month), and $MOM(p)=p = \frac{4}{10}$.
+, & \text{if } i^{th} \text{ subject is dead (did not survive) after 1 month of treatment}
 * MOM Beta distribution example: Suppose we have 10 observations we suspect came from a [http://www.distributome.org/js/calc/BetaCalculator.html Beta distribution].