SMHS MissingData - Revision history

Imoubara at 13:46, 23 May 2016

2016-05-23T13:46:06Z

Pineaumi: /* Multiple Imputation Protocol */

2016-05-19T19:42:55Z

‎Multiple Imputation Protocol

Pineaumi: /* Simulated Example */

2016-05-19T18:43:59Z

‎Simulated Example

Pineaumi: /* Multiple Imputation Protocol */

2016-05-19T18:41:58Z

‎Multiple Imputation Protocol

Pineaumi: /* Multiple Imputation Protocol */

2016-05-19T18:39:48Z

‎Multiple Imputation Protocol

Pineaumi: /* Multiple Imputation Protocol */

2016-05-19T18:38:26Z

‎Multiple Imputation Protocol

Pineaumi: /* Overview */

2016-05-19T18:35:51Z

‎Overview

Pineaumi: /* Overview */

2016-05-19T18:35:05Z

‎Overview

Pineaumi: /* Overview */

2016-05-19T18:34:49Z

‎Overview

Pineaumi: /* Overview */

2016-05-19T18:34:07Z

‎Overview

@@ Line 2: / Line 2: @@
 ===Questions===
-<li>Why is data usually incomplete?</li>
+*Why is data usually incomplete?
-<li>What are the best strategies for dealing with missing data- ignore cases, replace them by some population derived values, or impute them?</li>
+*What are the best strategies for dealing with missing data- ignore cases, replace them by some population derived values, or impute them?
-<li>What is the impact of data manipulations on the core scientific inference?</li>
+*What is the impact of data manipulations on the core scientific inference?
 ===Overview===
@@ Line 22: / Line 22: @@
 The type of missingness in the data is an important factor in the imputation process. The basic patterns of missingness include:
-•	Missing completely at random (MCAR) assumes that the missing data is not related to any factor, known or unknown, in the study.
+*Missing completely at random (MCAR) assumes that the missing data is not related to any factor, known or unknown, in the study.
-•	Missingness at random (MAR) assumes that the missingness depends only on observed quantities, which may include outcomes and/or predictors.
+*Missingness at random (MAR) assumes that the missingness depends only on observed quantities, which may include outcomes and/or predictors.
-•	Non-ignorable missingness occurs when the missing data depends on unobserved quantities.
+*Non-ignorable missingness occurs when the missing data depends on unobserved quantities.
 ===Multiple Imputation Protocol===

@@ Line 30: / Line 30: @@
 ===Multiple Imputation Protocol===
-<b>(1) Imputation</b>: Generate a set of $m > 1$ plausible values for $Z_{mis}=(Y_{mis},X_{mis})$.
+====(1) Imputation:====
 The imputation step relies upon assumptions regarding the cause of missingness in the dataset. The goal of the imputation is to account for the relationships between the unobserved and observed variables, while taking into account the uncertainty of the imputation. The commonly made MAR assumption for missing data is untestable without additional information. With MAR assumption we can generate imputations (Z<sub>{1}</sub>,Z<sub>{2}</sub>,...Z<sub>{m}</sub>) from the distribution $f(Z_{mis}|Z_{obs})$, since after conditioning on $Z_{obs}$ the missingness is assumed to be random.
@@ Line 48: / Line 50: @@
 The <b>propensity score</b> method uses an alternative model for imputation where the values are imputed from observations that are equally likely to be missing, by fitting a <u>logistic regression model for the missingness indicators</u>. In some situations, propensity score imputation may lead to serious bias for missing covariates. For discrete incomplete variables, discriminant analysis or dichotomous logistic regression may be employed to impute values based on the estimated probability that a missing observation takes on a certain value.
-<b>(2) Analysis</b>: Analyze the <i>m</i> datasets using complete-case methods. This second step in the protocol involves carrying out the analysis of interest for each of the m imputed complete-observation datasets and storing the parameter vectors (e.g., effect-size coefficients, β’s) and their standard error estimates.
+====(2) Analysis:====
-<b>(3) Combination</b>: Combine the results from the <i>m</i> analyses and calculate the estimates of the <i>within imputation and between imputation</i> variability. If the imputation model is correct (assumptions are valid), these statistics account for the variability of the imputations and provide consistent estimates of the parameters and their standard errors.
+Combine the results from the <i>m</i> analyses and calculate the estimates of the <i>within imputation and between imputation</i> variability. If the imputation model is correct (assumptions are valid), these statistics account for the variability of the imputations and provide consistent estimates of the parameters and their standard errors.
 ===Simulated Example===

@@ Line 62: / Line 62: @@
   if (length(pct.mis) == 1) {
   n.mis <- rep((n * (pct.mis/100)), J)
-−
+  }
   else {
@@ Line 68: / Line 68: @@
   stop("The length of missing does not equal to the column of the data")
   n.mis <- n * (pct.mis/100)
-−
+  }
      for (i in 1:ncol(data)) {

@@ Line 34: / Line 34: @@
 The imputation step relies upon assumptions regarding the cause of missingness in the dataset. The goal of the imputation is to account for the relationships between the unobserved and observed variables, while taking into account the uncertainty of the imputation. The commonly made MAR assumption for missing data is untestable without additional information. With MAR assumption we can generate imputations (Z<sub>{1}</sub>,Z<sub>{2}</sub>,...Z<sub>{m}</sub>) from the distribution $f(Z_{mis}|Z_{obs})$, since after conditioning on $Z_{obs}$ the missingness is assumed to be random.
-The missingness is <b>monotone</b> when the data matrix can be rearranged so that there is a hierarchy of missingness where observing a particular variable $Z_b$ for a subject implies that $Z_a$  is observed, for a < b. <i>In monotone condition settings many imputation methods may be employed including (for continuous variables) propensity methods, predictive mean matching, and (for discrete variables) discriminant analysis or logistic regression</i>. For non-monotonic missingness, Markov Chain Monte Carlo (MCMC) approaches may be used.
+The missingness is <b>monotone</b> when the data matrix can be rearranged so that there is a hierarchy of missingness where observing a particular variable $Z_b$ for a subject implies that $Z_a$  is observed, for $a < b$. <i>In monotone condition settings many imputation methods may be employed including (for continuous variables) propensity methods, predictive mean matching, and (for discrete variables) discriminant analysis or logistic regression</i>. For non-monotonic missingness, Markov Chain Monte Carlo (MCMC) approaches may be used.
-Each method has its own assumptions. For instance, predictive mean matching and MCMC approaches require multivariate normality. Predictive mean matching approach employs linear regression for the distribution of a partially observed variable, conditional on other factors. To impute the data using the <b>predictive mean matching approach</b> for a variable $Z_i$ with missing values, we fit a model using complete observations for Z<sub>1</sub>, . . . , Z<sub>(i-1)</sub> (not monotonicity assumption!):
+Each method has its own assumptions. For instance, predictive mean matching and MCMC approaches require multivariate normality. Predictive mean matching approach employs linear regression for the distribution of a partially observed variable, conditional on other factors. To impute the data using the <b>predictive mean matching approach</b> for a variable $Z_i$ with missing values, we fit a model using complete observations for $Z_1, . . . , Z_{i-1}$ (not monotonicity assumption!):
 $E[Z_i|φ]=φ_0+φ_1Z_1+φ_2Z_2+...+φ_{i-1}Z_{i-1}$

@@ Line 32: / Line 32: @@
 <b>(1) Imputation</b>: Generate a set of $m > 1$ plausible values for $Z_{mis}=(Y_{mis},X_{mis})$.
-The imputation step relies upon assumptions regarding the cause of missingness in the dataset. The goal of the imputation is to account for the relationships between the unobserved and observed variables, while taking into account the uncertainty of the imputation. The commonly made MAR assumption for missing data is untestable without additional information. With MAR assumption we can generate imputations (Z<sub>{1}</sub>,Z<sub>{2}</sub>,...Z<sub>{m}</sub>) from the distribution $f(Z_{mis}|Z_{obs})$, since after conditioning on $Z_{obs}$the missingness is assumed to be random.
+The imputation step relies upon assumptions regarding the cause of missingness in the dataset. The goal of the imputation is to account for the relationships between the unobserved and observed variables, while taking into account the uncertainty of the imputation. The commonly made MAR assumption for missing data is untestable without additional information. With MAR assumption we can generate imputations (Z<sub>{1}</sub>,Z<sub>{2}</sub>,...Z<sub>{m}</sub>) from the distribution $f(Z_{mis}|Z_{obs})$, since after conditioning on $Z_{obs}$ the missingness is assumed to be random.
 The missingness is <b>monotone</b> when the data matrix can be rearranged so that there is a hierarchy of missingness where observing a particular variable $Z_b$ for a subject implies that $Z_a$  is observed, for a < b. <i>In monotone condition settings many imputation methods may be employed including (for continuous variables) propensity methods, predictive mean matching, and (for discrete variables) discriminant analysis or logistic regression</i>. For non-monotonic missingness, Markov Chain Monte Carlo (MCMC) approaches may be used.

@@ Line 18: / Line 18: @@
 <b>3) Combine</b>: synthesize the results accounting for the uncertainty within each imputation round.
-In a general regression setting, let’s denote the scalar, or vector valued, outcomes by <i>Y</i>, and the corresponding vector of predictors by <i>X</i>. For a given case (e.g., subject, unit), these quantities are either observed (<i>obs</i>) or missing (<i>mis</i>). Thus, Y<sub>obs</sub> and X<sub>obs</sub> represent the observed component of the outcome and the predictors; and Y<sub>mis</sub> and X<sup>mis</sup> denote the unobserved components of the outcome and predictors, respectively. Imputation involves the estimation of the regression parameters β governing the conditional distribution of Y given X: f(Y|X,β). The efficiency, bias and precision of the estimates are important in this process.
+In a general regression setting, let’s denote the scalar, or vector valued, outcomes by <i>Y</i>, and the corresponding vector of predictors by <i>X</i>. For a given case (e.g., subject, unit), these quantities are either observed (<i>obs</i>) or missing (<i>mis</i>). Thus, Y<sub>obs</sub> and X<sub>obs</sub> represent the observed component of the outcome and the predictors; and Y<sub>mis</sub> and X<sub>mis</sub> denote the unobserved components of the outcome and predictors, respectively. Imputation involves the estimation of the regression parameters β governing the conditional distribution of Y given X: f(Y|X,β). The efficiency, bias and precision of the estimates are important in this process.
 The type of missingness in the data is an important factor in the imputation process. The basic patterns of missingness include:

@@ Line 12: / Line 12: @@
 Multiple imputation involves 3 steps:
-<b>1)Impute</b>: Create sets of plausible values for the missing observations that reflect uncertainty about the non-response model. Each of these sets of plausible values can be used to “fill-in” or complete the data-set.
+<b>1) Impute</b>: Create sets of plausible values for the missing observations that reflect uncertainty about the non-response model. Each of these sets of plausible values can be used to “fill-in” or complete the data-set.
-<b>2)Analyze</b>: process each of these imputed data-sets using complete-data methods.
+<b>2) Analyze</b>: process each of these imputed data-sets using complete-data methods.
-<b>3)Combine</b>: synthesize the results accounting for the uncertainty within each imputation round.
+<b>3) Combine</b>: synthesize the results accounting for the uncertainty within each imputation round.
 In a general regression setting, let’s denote the scalar, or vector valued, outcomes by <i>Y</i>, and the corresponding vector of predictors by <i>X</i>. For a given case (e.g., subject, unit), these quantities are either observed (<i>obs</i>) or missing (<i>mis</i>). Thus, Y<sub>obs</sub> and X<sub>obs</sub> represent the observed component of the outcome and the predictors; and Y<sub>mis</sub> and X<sup>mis</sup> denote the unobserved components of the outcome and predictors, respectively. Imputation involves the estimation of the regression parameters β governing the conditional distribution of Y given X: f(Y|X,β). The efficiency, bias and precision of the estimates are important in this process.

@@ Line 10: / Line 10: @@
 Many research studies encounter incomplete (missing) data that require special handling (e.g., processing, statistical analysis, visualization). There are a variety of methods (e.g., multiple imputation) to deal with missing data, detect missingness, impute the data, analyze the completed data-set and compare the characteristics of the raw and imputed data.
-Multiple imputation involves 3 steps.
+Multiple imputation involves 3 steps:
 )	<b>Impute</b>: Create sets of plausible values for the missing observations that reflect uncertainty about the non-response model. Each of these sets of plausible values can be used to “fill-in” or complete the data-set.