SOCR - User contributions [en]

SMHS NonParamInference

2014-08-14T18:38:26Z

Mwolvie: /* Problems */

==[[SMHS| Scientific Methods for Health Sciences]] - Non-Parametric Inference ==

===Overview===
Nonparametric inference is descriptive and inferential statistics made that are not based on parametrized families of probability distributions, which is the basis of the parametric inference we discussed in HS 550. That is, nonparametric inference made no assumptions about the probability distributions of the variables assessed and the model structure is not specified a priori but is determined from data instead. The term nonparametric is not strictly referring to models completely lack of parameters but that the number and nature of the parameters are flexible and not fixed in advance. In this lecture, we are going to introduce to the area of nonparametric inference and illustrate various nonparametric inference applications with examples.

===Motivation===
We have discussed about parametric inference where the inference is made based on the assumptions of the probability distributions of the variables being assessed. What if such an assumption is violated and variables studied cannot be categorized into a parametrized family of probability distribution? Distribution-free (nonparametric) statistical methods would be the answer to solving problems in such situation.

===Theory===

3.1) Differences of Median of Two Paired Samples: the sign test and the Wilcoxon signed rand test are the simplest nonparametric tests, which are also alternatives to the one-sample and paired t-test.

*Motivational Clinical Example: the relative risk of mortality from 16 studies of septic patients is reported, which measures whether the patients developed complications of acute renal failure. A relative risk of 1.0 means no effect and relative risk ≠1 suggests beneficial or detrimental effect of developing acute renal failure in sepsis. The main goal was to determine whether developing acute renal failure as a complication of sepsis impacts patient mortality from the cumulative evidence in these 16 studies and the data is recorded as below. The data is heavily skewed and not bell-shaped and apparently not normally distributed, so the traditional paired t-test fails here.
<center>
{|class="wikitable" style="text-align:center; width:75%" border="1"
|-
|Study||Relative Risk||Sign (Relative Risk -1)
|-
|1||0.75||-
|-
|2||2.03||+
|-
|3||2.29||+
|-
|4||2.11||+
|-
|5||0.80||-
|-
|6||1.50||+
|-
|7||0.79||-
|-
|8||1.01||+
|-
|9||1.23||+
|-
|10||1.48||+
|-
|11||2.45||+
|-
|12||1.02||+
|-
|13||1.03||+
|-
|14||1.30||+
|-
|15||1.54||+
|-
|16||1.27||+
|}
</center>

*The Sign Test: a nonparametric alternative to the One-Sample and Paired T-Test and doesn’t require the data to be normally distributed. It assigns a positive (+) or negative (-) sign to each observation according to whether it is greater or less than some hypothesized value. It measures the difference between the ± signs and how distinct the difference is from our expectations to observe by chance alone. For the motivational example, if there were no effects of developing acute renal failure on the outcome from sepsis, about half of the 16 studies above would be expected to have a relative risk less than 1.0 (a "-" sign) and the remaining 8 would be expected to have a relative risk greater than 1.0 (a "+" sign). In the actual data, 3 studies had "-" signs and the remaining 13 studies had "+" signs. Intuitively, this difference of 10 appears large to be simply due to random variation. If so, the effect of developing acute renal failure would be significant on the outcome from sepsis.
**Calculations: suppose N_+ is the number of “+” signs and the significance level α=0.05; the hypotheses: H_0:N_+=8;vs.H_a: N_+≠8 (the effect of developing acute renal failure is not significant on the outcome from sepsis vs. the effect of developing acute renal failure is significant on the outcome from sepsis). Define the following test statistics: B_S=max⁡(N_+,N_- ), where N_+ and N_- are the number of positive and negative sings respectively. For the example above, we have B_S=max⁡(N_+,N_- )=max⁡(13,3)=13 and the probability that such binomial variable exceeds 13 is P(Bin(16,0.5,13))=0.010635. Therefore, we can reject the null hypothesis H_0 and conclude that the significant effect of developing acute renal failure on the outcome from sepsis.
**Example of set of 12 identical twins given psychological tests to determine whether the first born of the set tends to be more aggressive than the second born. Each twin is scored according to aggressiveness; a higher score indicates greater aggressiveness. Because of nature pairing in a set of twins these data can be considered paired.
<center>
{|class="wikitable" style="text-align:center; width:75%" border="1"
|-
|Twin-Index||1st Born||2nd Born||Sign
|-
|1||86||88||-
|-
|2||71||77||-
|-
|3||77||76||+
|-
|4||68||64||+
|-
|5||91||96||-
|-
|6||72||72||0 (Drop)
|-
|7||77||65||+
|-
|8||91||90||+
|-
|9||70||65||+
|-
|10||71||80||-
|-
|11||88||81||+
|-
|12||87||72||+
|}
</center>

First plot the data using SOCR Linear Chart (http://wiki.socr.umich.edu/index.php/SOCR_EduMaterials_Activities_LineChart) and observe that there seems to be no strong effect of the order of birth on baby’s regression.
<center>
[[File:NonParamInference fig 1.png]]
</center>

Next, use the SOCR Sign Test Analysis to quantitatively evaluate the evidence to reject the null hypothesis that there is no birth-order effect on baby’s aggressiveness. (p-value of 0.274 => don’t reject the null hypothesis at 5% level of significance.)
<center>
[[File:NonParamInference fig 2.png]]
</center>

*The Wilcoxon Signed Rank Test: comparisons of differences between measurements and requires that the data are measured at an interval level of measurements but does not require assumptions about the form of the distribution of the measurements. It is used whenever the distributional assumption of the t-test is not satisfied.
**Motivational example: data on the central venous oxygen saturation (SvO2 (%)) from 10 consecutive patients at 2 time points, at admission and 6 hours after admission to the intensive care unit (ICU) are recorded. H_0: there is no effect of 6 hours of ICU treatment on SvO2, which states that the mean of the differences between SvO2 at admission and that 6 hours after admission should be zero. The data are recorded as below:
<center>
{|class="wikitable" style="text-align:center; width:75%" border="1"
|-
|Patient||On Admission||At 6 Hours||Difference||Rank
|-
|2||59.1||56.7||-2.4||1
|-
|7||58.2||60.7||2.5||2
|-
|9||56.0||59.5||3.5||3
|-
|10||65.3||59.8||-5.5||4
|-
|3||56.1||61.9||5.8||5
|-
|5||60.6||67.7||7.1||6
|-
|6||37.8||50.0||12.2||7
|-
|1||39.7||52.9||13.2||8
|-
|4||57.7||71.4||13.7||9
|-
|8||33.6||51.3||17.7||10
|}
</center>
<center>
[[File:NonParamInference fig 3.png]]
</center>

Result: the one-sided and two-sided alternative hypotheses p-values for the Wilcoxon Signed Rank Test reported by the SOCR Analysis are 0.011 and 0.022 respectively.
Result output and interpretation:

Variable 1 = At_Admission

Variable 2 = 6_Hrs_Later

Results of Two Paired Sample Wilcoxon Signed Rank Test:

Wilcoxon Signed-Rank Statistic = 5.000: [data-driven estimate of the Wilcoxon Signed-Rank Statistic].
E(W+), Wilcoxon Signed-Rank Score = 27.500: [Wilcoxon Signed-Rank Score = expectation of the Wilcoxon Signed-Rank Statistic].

Var(W+), Variance of Score = 96.250: [Variance of Score = variance of the Wilcoxon Signed-Rank Statistic].
Wilcoxon Signed-Rank Z-Score = -2.293 [Z-score=(W_stat-E(W_+))/√(Var(W_+))].

One-Sided P-Value = .011: [the one-sided (uni-directional) probability value expressing the strength of the evidence in the data to reject the null hypothesis that the two populations have the same medians (based on Gaussian, standard Normal, distribution)].

Two-Sided P-Value = .022: [the double-sided (non-directional) probability value expressing the strength of the evidence in the data to reject the null hypothesis that the two populations have the same medians (based on Gaussian, standard Normal, distribution)].

3.2) Difference of Medians of Two Independent Samples: the Wilcoxon-Mann-Whitney (WMW) test is nonparametric test of assessing whether two samples from the same distribution.

*Motivational Example: 9 observations of surface soil PH were made at two different independent locations. There is no pairing in this design though this is a balanced design with 9 observations in each group. The question is does the data suggest that the true mean soil PH values differs for the two locations. Testing using α=0.05 and be sure to check any necessary assumptions for the validity of the test.
<center>
{|class="wikitable" style="text-align:center; width:40%" border="1"
|-
|Location 1||Location 2
|-
|8.10||7.85
|-
|7.89||7.30
|-
|8.00||7.73
|-
|7.85||7.27
|-
|8.01||7.58
|-
|7.82||7.27
|-
|7.99||7.50
|-
|7.80||7.23
|-
|7.93||7.41
|}
</center>
Plot the data and we can see that the distributions may be different or not even symmetric, unimodal and bell-shaped. Therefore, the independent t-test fails to test a Null-hypothesis that the centers of the two distributions are identical.
<center>
[[File:NonParamInference fig 4.png]]
[[File:NonParamInference fig 5.png]]
</center>

The first figure shows the index plot of the pH levels for both samples. The second figure shows the sample histograms of these samples, which are clearly not Normal-like. Therefore, the independent t-test would not be appropriate to analyze these data.

Intuitively, we may consider these group differences significantly large, especially if we look at the Box-and-Whisker Plots, but this is a qualitative inference that demands a more quantitative statistical analysis that can back up our intuition.
<center>
[[File:NonParamInference fig 6.png]]
</center>

*The Wilcoxon-Mann-Whitney Test (WMW): a non-parametric test for assessing whether two samples come from the same distribution. The null hypothesis is that the two samples are drawn from a single population, and therefore that their probability distributions are equal. It requires that the two samples are independent, and that the observations are ordinal or continuous measurements.
*Calculations: The U statistic for the WMW test may be approximated for sample sizes above about 20 using the normal distribution. The U test is provided as part of SOCR analyses http://socr.umich.edu/html/ana/
**For small samples: (1) Choose the sample for which the ranks seem to be smaller. Call this Sample 1, and call the other sample Sample 2; (2) Taking each observation in Sample 2, count the number of observations in Sample 1 that are smaller than it (count 0.5 for any that are equal to it); (3) The total of these counts is U.
**For larger samples: (1) Arrange all the observations into a single ranked series. That is, rank all the observations without regard to which sample they come from; (2) Add up the ranks in Sample 1, The sum of ranks in Sample 2 follows by calculation, since the sum of all the ranks equals N (N + 1)/2, where N is the total number of observations; (3) U is given by the formula: U_1=R_1-(n_1 (n_1+1))/2, where n_1 is the two sample size for Sample 1, and R_1 is the sum of the ranks in Sample 1. there is no specification as to which sample is considered Sample 1. An equally valid formula for U is U_2=R_2-(n_2 (n_2+1))/2, the sum of the two values is given by U_1+U_2=R_1-(n_1 (n_1+1))/2+R_2-(n_2 (n_2+1))/2. Given that R_1+R_2=N(N+1)/2 and N=n_1+n_2, hence we have U_1+U_2=n_1 n_2. The maximum value of U is the product of the sample sizes.
*Application using SOCR analyses: it is much quicker to use SOCR analyses to compute the statistical significance of the sign test.
<center>
[[File:NonParamInference fig 7.png]]
</center>

Clearly the p value < 0.05 and therefore our data provides sufficient evidence to reject the null hypothesis. So we assume that there were significant PH differences between the two soil lots tested in this experiment.

One-sided P-value for Sample 2 < Sample 1 = 0.00040.

Two-sided P-value for Sample 1 not equal to Sample 2 = 0.00079.

*The WMW Tess vs. Independent T-test: Both types of tests answer the same question but they treat data differently.
**The WMW test uses rank ordering: Positive: Doesn’t depend on normality or population parameters Negative: Distribution free lacks power because it doesn't use all the info in the data
**The T-test uses the raw measurements: Positive: Incorporates all of the data into calculations Negative: Must meet normality assumption
**Neither test is uniformly superior. If the data are normally distributed we use the T-test. If the data are not normal use the WMW test.

3.3) Differences of Proportions of Two Samples: depending upon whether the samples are dependent or independent where different statistical tests are applied.

*Differences of proportions of two independent samples: If the samples are independent and we are interested in the differences in the proportions of subjects of the same trait (a characteristic of each observation, e.g., gender) we need to use the standard proportion tests. For small sample sizes, we use corrected proportion (p ̃) that and we use the raw sample proportion (p ̂) in large samples.
*Differences of proportions of two paired samples: If the samples are paired, then we can employ the McNemar's non-parametric test for differences in proportions in matched pair samples. It is most often used when the observed variable is a dichotomous variable (presence or absence of a trait/characteristic for each observation).
*Example: suppose a medical doctor is interested in determining the effect of a drug on a particular disease (D). Suppose the doctor conducts a study and records the frequencies of incidence of the disease (D+ and D− ) in a random population before the treatment with the new drug takes place. Then the doctor prescribes the treatment to all subjects and records the incidence of the disease in the rows following the treatment. The test requires the same subjects to be included in the before- and after-treatment measurements (matched pairs).
<center>
{|class="wikitable" style="text-align:center; width:40%" border="1"
|-
|colspan=2 rowspan=2| ||colspan=3|Before Treatment
|-
|D +||D −||Total
|-
|rowspan=3|After Treatment||D +||a=101||b=59||a+b=160
|-
|D −||c=121||d=33||c+d=154
|-
|Total||a+c=222||b+d=92||a+b+c+d=314
|}
</center>

**Marginal homogeneity occurs when the row totals equal to the column totals, a and d in each equation can be cancelled; leaving b equal to c: a+b=a+c, c+d=b+d. In this example, marginal homogeneity would mean there was no effect of the treatment.
**The mcNemar statistic is shown: χ_0^2=(b-c)^2/(b+c)~χ_(df=1)^2
**The marginal frequencies are not homogeneous if the χ_0^2 result is significant p < 0.05. If b and/or c are small (b + c < 20) then χ_0^2 is not approximated by the Chi-Square Distribution and a Sign Test should instead be used.
**An interesting observation when interpreting McNemar's test is that the elements of the main diagonal contribute no information whatsoever to the decision if (in the above example) pre- or post-treatment condition is more favorable.
*General McNemar test of marginal homogeneity for a single category: If we have observed measurements on a K-level categorical variable -- e.g., agreement between two evaluators summarized by a K×K classification table, where each row or column contains the number of individuals rated as part of this group by each evaluator. For instance, two instructors may evaluate students as 1=poor, 2=good and 3=excellent. There could be significant differences in the evaluations of the same students by the 2 instructors. Suppose we are interested in whether the proportions of students rated excellent by the 2 instructors are the same. Then we'll pool the poor and good categories together and form a 2x2 table that we can then use the 2x2 McNemar test statistics on, as shown below.

<center>
{|class="wikitable" style="text-align:center; width:40%" border="1"
|-
| || ||colspan=4|Evaluator 2
|-
| || ||Poor||Good||Excellent||Total
|-
|rowspan=4|Evaluator 1||Poor||5||15||4||24
|-
|Good||16||10||9||35
|-
|Excellent||11||17||13||41
|-
|Total||32||42||26||100
|}
</center>

<center>
{|class="wikitable" style="text-align:center; width:40%" border="1"
|-
| || ||colspan=3|Evaluator 2
|-
| || ||Poor||Good or Excellent||Total
|-
|rowspan=3|Evaluator 1||Poor||a=5||b=19||a+b=24
|-
|Good or Excellent||c=27||d=49||c+d=76
|-
|Total||a+c=32||b+d=68||a+b+c+d=100
|}
</center>

To test marginal homogeneity for one single category (in this case poor evaluation) means to test row/column marginal homogeneity for the first category (poor). This is achieved by collapsing all rows and columns corresponding to the other categories.

χ_o^2=(b-c)^2/(b+c)=(-8)^2/46=1.39 ~ χ_(df=1)^2
P( χ_(df=1)^2>1.39)=0.238405

Therefore, we don’t have sufficient evidence to reject the null hypothesis of two evaluators were consistent in their ratings of students.

<center>
[[File:NonParamInference fig 8.png]]
</center>

3.4) Differences of Means of Several Independent Samples: extend the multi-sample inference discussed in ANOVA to the situation where ANOVA assumptions are invalid.
*Motivational Example: Suppose four groups of students are randomly assigned to be taught with four different techniques, and their achievement test scores are recorded. Are the distributions of test scores the same, or do they differ in location? The data is presented in the table below. The small sample sizes and the lack of distribution information of each sample illustrate how ANOVA may not be appropriate for analyzing these types of data.
<center>
{|class="wikitable" style="text-align:center; width:40%" border="1"
|-
|colspan=5|Teaching Method
|-
| ||Method 1||Method 2||Method 3||Method 4
|-
|rowspan=4|Index||65||75||59||94
|-
|87||69||78||89
|-
|73||83||67||80
|-
|79||81||62||88
|}
</center>

*The Kruskal-Wallis Test: Kruskal-Wallis One-Way Analysis of Variance by ranks is a non-parametric method for testing equality of two or more population medians. Intuitively, it is identical to a One-way Analysis of Variance with the raw data (observed measurements) replaced by their ranks. Since it is a non-parametric method, the Kruskal-Wallis Test does not assume a normal population, unlike the analogous one-way ANOVA. However, the test does assume identically-shaped distributions for all groups, except for any difference in their centers (e.g., medians).
**Calculations: Let N be the total number of observations and N=∑_(i=1)^k▒n_i and R(X_ij) be the rank assigned to X_ij and let R_i be the sum of ranks assigned to the i^th sample and R_i=∑_(i=1)^(n_i)▒〖R(X_ij)〗,i=1,2,…,k.

The SOCR program computes R_i for each sample. The test statistic is defined for the following formulation of hypotheses:

H_o: All of the k population distribution functions are identical.

H_a: At least one of the populations tends to yield larger observations than at least one of the other populations.

Suppose {X_(i,1),X_(i,2),…,X_(i,n_i ) } represents the values of the i^th sample, where i≤i≤k.
Test statistics: T=(1/S^2)∑_(i=1)^k▒R_i^2 /n_i-N(N+1)^2/4, where S^2=(1/(N-1) ∑▒〖R(X_ij )^2 〗-N(N+1))^2/4.

Note: If there are no ties, then the test statistic is reduced to:

T=12/N(N+1) ∑_(i=1)^k▒(R_i^2)/n_i -3(N+1).

However, the SOCR implementation allows for the possibility of having ties; so it uses the non-simplified, exact method of computation. Multiple comparisons have to be done here. For each pair of groups, the following is computed and printed at the Result Panel.

|R_i/n_i -R_j/n_j |>t_(1-α/2) ((S^2 (N-1-T))/(N-K))^(1/2)/(1/n_i +1/n_j )^(1/2)

The SOCR computation employs the exact method instead of the approximate one (Conover 1980), since computation is easy and fast to implement and the exact method is somewhat more accurate.
*The Kruskal-Wallis Test Using SOCR Analyses: It is much quicker to use SOCR Analyses to compute the statistical significance of this test. This SOCR Kruskal-Wallis Test Activity may also be helpful in understanding how to use this test in SOCR. For the teaching-methods example above, we can easily compute the statistical significance of the differences between the group medians (centers):
<center>
[[File:NonParamInference fig 9.png]]
</center>

Clearly, there is only one significant group difference between medians, after the multiple testing correction, for the group1 vs. group4 comparison (see below):

Group Method1 vs. Group Method2: 1.0 < 5.2056

Group Method1 vs. Group Method3: 4.0 < 5.2056

'''Group Method1 vs. Group Method4: 6.0 > 5.2056'''

Group Method2 vs. Group Method3: 5.0 < 5.2056

Group Method2 vs. Group Method4: 5.0 < 5.2056

Group Method3 vs. Group Method4: 10.0 > 5.2056

3.5) Differences of Variances of Independent Samples (variance homogeneity): different tests for variance equality can be applied. It is frequently necessary to test if k samples have equal variances. Homogeneity of variances is often a reference to equal variances across samples. Some statistical tests, for example the analysis of variance, assume that variances are equal across groups or samples.
*Calculation: The (modified) Fligner-Killeen test (the median-centering Fligner-Killeen test) provides the means for studying the homogeneity of variances of k populations { X_ij, for 1≤i≤n_j and 1≤j≤k}. The test jointly ranks the absolute values of |X_ij-(X_j ) ̃| and assigns increasing scores a_Ni=Φ^(-1) ((1+i/(N+1)))/2, based on the ranks of all observations. In this test, (X_j ) ̃ is the sample median of the j^th population, and Φ(.) is the cumulative distribution function for Normal distribution.
**Fligner-Killeen test statistics: χ_o^2=(∑_(j=1)^k▒〖n_j ((A_j ) ̅-a ̅ )^2 〗)/V^2 , where (A_j ) ̅ is the mean score for the j^th sample, a ̅ is the overall mean score of all a_Nj and V^2 is the sample variance of all scores. We have N=∑_(j=1)^k▒n_j ,(A_j ) ̅=1/n_j ∑_(j=1)^(n_j)▒a_(Nm_i ) , where a_(Nm_i ) is the increasing rank score for the i^th observation in the j^th sample, a ̅=1/N ∑_(i=1)^N▒a_Ni ,V^2=1/(N-1) ∑_(i=1)^N▒(a_Ni-a ̅ )^2 .
**Fligner-Killeen probabilities: for large sample size, the modified Fligner-Killeen test statistic has an asymptotic chi-square distribution with k-1 degrees of freedom χ_o^2~χ_(k-1)^2.

===Applications===

4.1) This article (http://www.jstor.org/discover/10.2307/2958850?uid=3739728&uid=2&uid=4&uid=3739256&sid=21103957523251) presents a general introduction to nonparametric inference and the model involved and studied the existence of complete and sufficient statistics for this model. It gives an empirical process estimating the model and generalized the empirical cumulative hazard rate from survival analysis. Consistency and weak convergence results were given and tests for comparison of two counting processes, generalizing the two sample rank tests are defined and studied. Finally, it gives an application to a set of biological data.

4.2) This article (http://projecteuclid.org/euclid.aos/1033066215) presents a study relates a different solution to the challenge of constructing a sequence of functions, each based on only finite segments of the past, which together provide a strongly consistent estimator for the conditional probability of the next observation, given the infinite past. It also studied on some extensions to regression, pattern recognition and on-line forecasting.

===Software ===

http://wiki.stat.ucla.edu/socr/index.php/SOCR_EduMaterials_AnalysisActivities_TwoPairedSign

http://www.socr.ucla.edu/htmls/ana/TwoPairedSampleSign-Test_Analysis.html

===Problems===

6.1) Suppose 10 randomly selected rats were chosen to see if they could be trained to escape a maze. The rats were released and timed (seconds) before and after 2 weeks of training (N means the rat did not complete the maze-test). Do the data provide evidence to suggest that the escape time of rats is different after 2 weeks of training? Test using α = 0.05.
<center>
{|class="wikitable" style="text-align:center; width:40%" border="1"
|-
|Rat||Before||After||Sign
|-
|1||100||50||+
|-
|2||38||12||+
|-
|3||N||45||+
|-
|4||122||62||+
|-
|5||95||90||+
|-
|6||116||100||+
|-
|7||56||75||-
|-
|8||135||52||+
|-
|9||104||44||+
|-
|10||N||50||+
|}
</center>

6.2) Automated brain volume segmentation is an important step in many modern computational brain mapping studies. Suppose we have two separate and competing versions of automated brain parsing (segmentation) algorithms that automatically tessellate (partition) the brain into 57 separate regions of interest (ROI’s). An important question then is how consistent are these 2 different techniques, across the 57 ROIs. We can use the ROI volume as a measure of the resulting automated brain parcellation and compare the paired differences between the 2 methods across all ROIs. The image shows an example of a brain parcellated into these 57 regions and the table below contains the volumes of the 57 ROIs for the 2 different brain tessellation techniques. Use appropriate SOCR analyses and relevant SOCR charts to argue whether or not the 2 different methods are consistent and agree on their ROI labels.
<center>
{|class="wikitable" style="text-align:center; width:75%" border="1"
|-
|Index||Volume_Intensity||ROI_Name||Method1_Volume||Method2_Volume
|-
|1||0||Background||9236455||9241667
|-
|2||21||L_superior_frontal_gyrus||78874||78693
|-
|3||22||R_superior_frontal_gyrus||69575||74391
|-
|4||23||L_middle_frontal_gyrus||67336||68872
|-
|5||24||R_middle_frontal_gyrus||68344||67024
|-
|6||25||L_inferior_frontal_gyrus||31912||21479
|-
|7||26||R_inferior_frontal_gyrus||26264||29035
|-
|8||27||L_precentral_gyrus||28942||33584
|-
|9||28||R_precentral_gyrus||35192||30537
|-
|10||29||L_middle_orbitofrontal_gyrus||10141||11608
|-
|11||30||R_middle_orbitofrontal_gyrus||9142||11850
|-
|12||31||L_lateral_orbitofrontal_gyrus||7164||5382
|-
|13||32||R_lateral_orbitofrontal_gyrus||5964||4947
|-
|14||33||L_gyrus_rectus||3840||1995
|-
|15||34||R_gyrus_rectus||2672||2994
|-
|16||41||L_postcentral_gyrus||24586||27672
|-
|17||42||R_postcentral_gyrus||21736||28159
|-
|18||43||L_superior_parietal_gyrus||25791||27500
|-
|19||44||R_superior_parietal_gyrus||28850||32674
|-
|20||45||L_supramarginal_gyrus||16445||22373
|-
|21||46||R_supramarginal_gyrus||11893||11018
|-
|22||47||L_angular_gyrus||20740||22245
|-
|23||48||R_angular_gyrus||20247||17793
|-
|24||49||L_precuneus||14491||12983
|-
|25||50||R_precuneus||15589||16323
|-
|26||61||L_superior_occipital_gyrus||6842||6106
|-
|27||62||R_superior_occipital_gyrus||5673||6539
|-
|28||63||L_middle_occipital_gyrus||15011||19085
|-
|29||64||R_middle_occipital_gyrus||19063||25747
|-
|30||65||L_inferior_occipital_gyrus||10411||8675
|-
|31||66||R_inferior_occipital_gyrus||12142||12277
|-
|32||67||L_cuneus||6935||9700
|-
|33||68||R_cuneus||7491||11765
|-
|34||81||L_superior_temporal_gyrus||29962||34934
|-
|35||82||R_superior_temporal_gyrus||30630||28788
|-
|36||83||L_middle_temporal_gyrus||27558||19633
|-
|37||84||R_middle_temporal_gyrus||26314||25301
|-
|38||85||L_inferior_temporal_gyrus||24817||24885
|-
|39||86||R_inferior_temporal_gyrus||25088||20661
|-
|40||87||L_parahippocampal_gyrus||6761||6977
|-
|41||88||R_parahippocampal_gyrus||6529||7964
|-
|42||89||L_lingual_gyrus||16752||14748
|-
|43||90||R_lingual_gyrus||20914||18500
|-
|44||91||L_fusiform_gyrus||16565||15020
|-
|45||92||R_fusiform_gyrus||14409||17311
|-
|46||101||L_insular_cortex||10779||9814
|-
|47||102||R_insular_cortex||8222||5599
|-
|48||121||L_cingulate_gyrus||14662||12490
|-
|49||122||R_cingulate_gyrus||16595||14489
|-
|50||161||L_caudate||1906||1608
|-
|51||162||R_caudate||2353||1997
|-
|52||163||L_putamen||3015||2622
|-
|53||164||R_putamen||2177||3758
|-
|54||165||L_hippocampus||3791||4454
|-
|55||166||R_hippocampus||3596||4673
|-
|56||181||cerebellum||174045||158617
|-
|57||182||brainstem||32567||28225
|}
</center>

6.3) Urinary Fluoride Concentration in Cattle: The urinary fluoride concentration (ppm) was measured both for a sample of livestock grazing in an area previously exposed to fluoride pollution and also for a similar sample of livestock grazing in an unpolluted area.
<center>
{|class="wikitable" style="text-align:center; width:40%" border="1"
|-
|Polluted||Unpolluted
|-
|21.3||10.1
|-
|18.7||18.3
|-

|21.4||17.2
|-
|17.1||18.4
|-
|11.1||20.0
|-
|20.9||
|-
|19.7||
|}
</center>

6.4) To study whether the variances in certain time period (e.g. 1981 to 2006) of the consumer-price-indices (CPI) of several items were significantly different. Use the SOCR CPI dataset (http://wiki.socr.umich.edu/index.php/SOCR_Data_Dinov_021808_ConsumerPriceIndex) to answer this question for the Fuel, Oil, Bananas, Tomatoes, Orange juice, Beef and Gasoline items. (Apply the methods introduced in Differences of variances of independent samples).

===References===

Statistical inference / George Casella, Roger L. Berger. http://mirlyn.lib.umich.edu/Record/004199238

Sampling / Steven K. Thompson. http://mirlyn.lib.umich.edu/Record/004232056

Sampling theory and methods / S. Sampath. http://mirlyn.lib.umich.edu/Record/004133572

SMHS NonParamInference

2014-08-14T18:37:17Z

Mwolvie: /* Problems */

==[[SMHS| Scientific Methods for Health Sciences]] - Non-Parametric Inference ==

===Overview===
Nonparametric inference is descriptive and inferential statistics made that are not based on parametrized families of probability distributions, which is the basis of the parametric inference we discussed in HS 550. That is, nonparametric inference made no assumptions about the probability distributions of the variables assessed and the model structure is not specified a priori but is determined from data instead. The term nonparametric is not strictly referring to models completely lack of parameters but that the number and nature of the parameters are flexible and not fixed in advance. In this lecture, we are going to introduce to the area of nonparametric inference and illustrate various nonparametric inference applications with examples.

===Motivation===
We have discussed about parametric inference where the inference is made based on the assumptions of the probability distributions of the variables being assessed. What if such an assumption is violated and variables studied cannot be categorized into a parametrized family of probability distribution? Distribution-free (nonparametric) statistical methods would be the answer to solving problems in such situation.

===Theory===

3.1) Differences of Median of Two Paired Samples: the sign test and the Wilcoxon signed rand test are the simplest nonparametric tests, which are also alternatives to the one-sample and paired t-test.

*Motivational Clinical Example: the relative risk of mortality from 16 studies of septic patients is reported, which measures whether the patients developed complications of acute renal failure. A relative risk of 1.0 means no effect and relative risk ≠1 suggests beneficial or detrimental effect of developing acute renal failure in sepsis. The main goal was to determine whether developing acute renal failure as a complication of sepsis impacts patient mortality from the cumulative evidence in these 16 studies and the data is recorded as below. The data is heavily skewed and not bell-shaped and apparently not normally distributed, so the traditional paired t-test fails here.
<center>
{|class="wikitable" style="text-align:center; width:75%" border="1"
|-
|Study||Relative Risk||Sign (Relative Risk -1)
|-
|1||0.75||-
|-
|2||2.03||+
|-
|3||2.29||+
|-
|4||2.11||+
|-
|5||0.80||-
|-
|6||1.50||+
|-
|7||0.79||-
|-
|8||1.01||+
|-
|9||1.23||+
|-
|10||1.48||+
|-
|11||2.45||+
|-
|12||1.02||+
|-
|13||1.03||+
|-
|14||1.30||+
|-
|15||1.54||+
|-
|16||1.27||+
|}
</center>

*The Sign Test: a nonparametric alternative to the One-Sample and Paired T-Test and doesn’t require the data to be normally distributed. It assigns a positive (+) or negative (-) sign to each observation according to whether it is greater or less than some hypothesized value. It measures the difference between the ± signs and how distinct the difference is from our expectations to observe by chance alone. For the motivational example, if there were no effects of developing acute renal failure on the outcome from sepsis, about half of the 16 studies above would be expected to have a relative risk less than 1.0 (a "-" sign) and the remaining 8 would be expected to have a relative risk greater than 1.0 (a "+" sign). In the actual data, 3 studies had "-" signs and the remaining 13 studies had "+" signs. Intuitively, this difference of 10 appears large to be simply due to random variation. If so, the effect of developing acute renal failure would be significant on the outcome from sepsis.
**Calculations: suppose N_+ is the number of “+” signs and the significance level α=0.05; the hypotheses: H_0:N_+=8;vs.H_a: N_+≠8 (the effect of developing acute renal failure is not significant on the outcome from sepsis vs. the effect of developing acute renal failure is significant on the outcome from sepsis). Define the following test statistics: B_S=max⁡(N_+,N_- ), where N_+ and N_- are the number of positive and negative sings respectively. For the example above, we have B_S=max⁡(N_+,N_- )=max⁡(13,3)=13 and the probability that such binomial variable exceeds 13 is P(Bin(16,0.5,13))=0.010635. Therefore, we can reject the null hypothesis H_0 and conclude that the significant effect of developing acute renal failure on the outcome from sepsis.
**Example of set of 12 identical twins given psychological tests to determine whether the first born of the set tends to be more aggressive than the second born. Each twin is scored according to aggressiveness; a higher score indicates greater aggressiveness. Because of nature pairing in a set of twins these data can be considered paired.
<center>
{|class="wikitable" style="text-align:center; width:75%" border="1"
|-
|Twin-Index||1st Born||2nd Born||Sign
|-
|1||86||88||-
|-
|2||71||77||-
|-
|3||77||76||+
|-
|4||68||64||+
|-
|5||91||96||-
|-
|6||72||72||0 (Drop)
|-
|7||77||65||+
|-
|8||91||90||+
|-
|9||70||65||+
|-
|10||71||80||-
|-
|11||88||81||+
|-
|12||87||72||+
|}
</center>

First plot the data using SOCR Linear Chart (http://wiki.socr.umich.edu/index.php/SOCR_EduMaterials_Activities_LineChart) and observe that there seems to be no strong effect of the order of birth on baby’s regression.
<center>
[[File:NonParamInference fig 1.png]]
</center>

Next, use the SOCR Sign Test Analysis to quantitatively evaluate the evidence to reject the null hypothesis that there is no birth-order effect on baby’s aggressiveness. (p-value of 0.274 => don’t reject the null hypothesis at 5% level of significance.)
<center>
[[File:NonParamInference fig 2.png]]
</center>

*The Wilcoxon Signed Rank Test: comparisons of differences between measurements and requires that the data are measured at an interval level of measurements but does not require assumptions about the form of the distribution of the measurements. It is used whenever the distributional assumption of the t-test is not satisfied.
**Motivational example: data on the central venous oxygen saturation (SvO2 (%)) from 10 consecutive patients at 2 time points, at admission and 6 hours after admission to the intensive care unit (ICU) are recorded. H_0: there is no effect of 6 hours of ICU treatment on SvO2, which states that the mean of the differences between SvO2 at admission and that 6 hours after admission should be zero. The data are recorded as below:
<center>
{|class="wikitable" style="text-align:center; width:75%" border="1"
|-
|Patient||On Admission||At 6 Hours||Difference||Rank
|-
|2||59.1||56.7||-2.4||1
|-
|7||58.2||60.7||2.5||2
|-
|9||56.0||59.5||3.5||3
|-
|10||65.3||59.8||-5.5||4
|-
|3||56.1||61.9||5.8||5
|-
|5||60.6||67.7||7.1||6
|-
|6||37.8||50.0||12.2||7
|-
|1||39.7||52.9||13.2||8
|-
|4||57.7||71.4||13.7||9
|-
|8||33.6||51.3||17.7||10
|}
</center>
<center>
[[File:NonParamInference fig 3.png]]
</center>

Result: the one-sided and two-sided alternative hypotheses p-values for the Wilcoxon Signed Rank Test reported by the SOCR Analysis are 0.011 and 0.022 respectively.
Result output and interpretation:

Variable 1 = At_Admission

Variable 2 = 6_Hrs_Later

Results of Two Paired Sample Wilcoxon Signed Rank Test:

Wilcoxon Signed-Rank Statistic = 5.000: [data-driven estimate of the Wilcoxon Signed-Rank Statistic].
E(W+), Wilcoxon Signed-Rank Score = 27.500: [Wilcoxon Signed-Rank Score = expectation of the Wilcoxon Signed-Rank Statistic].

Var(W+), Variance of Score = 96.250: [Variance of Score = variance of the Wilcoxon Signed-Rank Statistic].
Wilcoxon Signed-Rank Z-Score = -2.293 [Z-score=(W_stat-E(W_+))/√(Var(W_+))].

One-Sided P-Value = .011: [the one-sided (uni-directional) probability value expressing the strength of the evidence in the data to reject the null hypothesis that the two populations have the same medians (based on Gaussian, standard Normal, distribution)].

Two-Sided P-Value = .022: [the double-sided (non-directional) probability value expressing the strength of the evidence in the data to reject the null hypothesis that the two populations have the same medians (based on Gaussian, standard Normal, distribution)].

3.2) Difference of Medians of Two Independent Samples: the Wilcoxon-Mann-Whitney (WMW) test is nonparametric test of assessing whether two samples from the same distribution.

*Motivational Example: 9 observations of surface soil PH were made at two different independent locations. There is no pairing in this design though this is a balanced design with 9 observations in each group. The question is does the data suggest that the true mean soil PH values differs for the two locations. Testing using α=0.05 and be sure to check any necessary assumptions for the validity of the test.
<center>
{|class="wikitable" style="text-align:center; width:40%" border="1"
|-
|Location 1||Location 2
|-
|8.10||7.85
|-
|7.89||7.30
|-
|8.00||7.73
|-
|7.85||7.27
|-
|8.01||7.58
|-
|7.82||7.27
|-
|7.99||7.50
|-
|7.80||7.23
|-
|7.93||7.41
|}
</center>
Plot the data and we can see that the distributions may be different or not even symmetric, unimodal and bell-shaped. Therefore, the independent t-test fails to test a Null-hypothesis that the centers of the two distributions are identical.
<center>
[[File:NonParamInference fig 4.png]]
[[File:NonParamInference fig 5.png]]
</center>

The first figure shows the index plot of the pH levels for both samples. The second figure shows the sample histograms of these samples, which are clearly not Normal-like. Therefore, the independent t-test would not be appropriate to analyze these data.

Intuitively, we may consider these group differences significantly large, especially if we look at the Box-and-Whisker Plots, but this is a qualitative inference that demands a more quantitative statistical analysis that can back up our intuition.
<center>
[[File:NonParamInference fig 6.png]]
</center>

*The Wilcoxon-Mann-Whitney Test (WMW): a non-parametric test for assessing whether two samples come from the same distribution. The null hypothesis is that the two samples are drawn from a single population, and therefore that their probability distributions are equal. It requires that the two samples are independent, and that the observations are ordinal or continuous measurements.
*Calculations: The U statistic for the WMW test may be approximated for sample sizes above about 20 using the normal distribution. The U test is provided as part of SOCR analyses http://socr.umich.edu/html/ana/
**For small samples: (1) Choose the sample for which the ranks seem to be smaller. Call this Sample 1, and call the other sample Sample 2; (2) Taking each observation in Sample 2, count the number of observations in Sample 1 that are smaller than it (count 0.5 for any that are equal to it); (3) The total of these counts is U.
**For larger samples: (1) Arrange all the observations into a single ranked series. That is, rank all the observations without regard to which sample they come from; (2) Add up the ranks in Sample 1, The sum of ranks in Sample 2 follows by calculation, since the sum of all the ranks equals N (N + 1)/2, where N is the total number of observations; (3) U is given by the formula: U_1=R_1-(n_1 (n_1+1))/2, where n_1 is the two sample size for Sample 1, and R_1 is the sum of the ranks in Sample 1. there is no specification as to which sample is considered Sample 1. An equally valid formula for U is U_2=R_2-(n_2 (n_2+1))/2, the sum of the two values is given by U_1+U_2=R_1-(n_1 (n_1+1))/2+R_2-(n_2 (n_2+1))/2. Given that R_1+R_2=N(N+1)/2 and N=n_1+n_2, hence we have U_1+U_2=n_1 n_2. The maximum value of U is the product of the sample sizes.
*Application using SOCR analyses: it is much quicker to use SOCR analyses to compute the statistical significance of the sign test.
<center>
[[File:NonParamInference fig 7.png]]
</center>

Clearly the p value < 0.05 and therefore our data provides sufficient evidence to reject the null hypothesis. So we assume that there were significant PH differences between the two soil lots tested in this experiment.

One-sided P-value for Sample 2 < Sample 1 = 0.00040.

Two-sided P-value for Sample 1 not equal to Sample 2 = 0.00079.

*The WMW Tess vs. Independent T-test: Both types of tests answer the same question but they treat data differently.
**The WMW test uses rank ordering: Positive: Doesn’t depend on normality or population parameters Negative: Distribution free lacks power because it doesn't use all the info in the data
**The T-test uses the raw measurements: Positive: Incorporates all of the data into calculations Negative: Must meet normality assumption
**Neither test is uniformly superior. If the data are normally distributed we use the T-test. If the data are not normal use the WMW test.

3.3) Differences of Proportions of Two Samples: depending upon whether the samples are dependent or independent where different statistical tests are applied.

*Differences of proportions of two independent samples: If the samples are independent and we are interested in the differences in the proportions of subjects of the same trait (a characteristic of each observation, e.g., gender) we need to use the standard proportion tests. For small sample sizes, we use corrected proportion (p ̃) that and we use the raw sample proportion (p ̂) in large samples.
*Differences of proportions of two paired samples: If the samples are paired, then we can employ the McNemar's non-parametric test for differences in proportions in matched pair samples. It is most often used when the observed variable is a dichotomous variable (presence or absence of a trait/characteristic for each observation).
*Example: suppose a medical doctor is interested in determining the effect of a drug on a particular disease (D). Suppose the doctor conducts a study and records the frequencies of incidence of the disease (D+ and D− ) in a random population before the treatment with the new drug takes place. Then the doctor prescribes the treatment to all subjects and records the incidence of the disease in the rows following the treatment. The test requires the same subjects to be included in the before- and after-treatment measurements (matched pairs).
<center>
{|class="wikitable" style="text-align:center; width:40%" border="1"
|-
|colspan=2 rowspan=2| ||colspan=3|Before Treatment
|-
|D +||D −||Total
|-
|rowspan=3|After Treatment||D +||a=101||b=59||a+b=160
|-
|D −||c=121||d=33||c+d=154
|-
|Total||a+c=222||b+d=92||a+b+c+d=314
|}
</center>

**Marginal homogeneity occurs when the row totals equal to the column totals, a and d in each equation can be cancelled; leaving b equal to c: a+b=a+c, c+d=b+d. In this example, marginal homogeneity would mean there was no effect of the treatment.
**The mcNemar statistic is shown: χ_0^2=(b-c)^2/(b+c)~χ_(df=1)^2
**The marginal frequencies are not homogeneous if the χ_0^2 result is significant p < 0.05. If b and/or c are small (b + c < 20) then χ_0^2 is not approximated by the Chi-Square Distribution and a Sign Test should instead be used.
**An interesting observation when interpreting McNemar's test is that the elements of the main diagonal contribute no information whatsoever to the decision if (in the above example) pre- or post-treatment condition is more favorable.
*General McNemar test of marginal homogeneity for a single category: If we have observed measurements on a K-level categorical variable -- e.g., agreement between two evaluators summarized by a K×K classification table, where each row or column contains the number of individuals rated as part of this group by each evaluator. For instance, two instructors may evaluate students as 1=poor, 2=good and 3=excellent. There could be significant differences in the evaluations of the same students by the 2 instructors. Suppose we are interested in whether the proportions of students rated excellent by the 2 instructors are the same. Then we'll pool the poor and good categories together and form a 2x2 table that we can then use the 2x2 McNemar test statistics on, as shown below.

<center>
{|class="wikitable" style="text-align:center; width:40%" border="1"
|-
| || ||colspan=4|Evaluator 2
|-
| || ||Poor||Good||Excellent||Total
|-
|rowspan=4|Evaluator 1||Poor||5||15||4||24
|-
|Good||16||10||9||35
|-
|Excellent||11||17||13||41
|-
|Total||32||42||26||100
|}
</center>

<center>
{|class="wikitable" style="text-align:center; width:40%" border="1"
|-
| || ||colspan=3|Evaluator 2
|-
| || ||Poor||Good or Excellent||Total
|-
|rowspan=3|Evaluator 1||Poor||a=5||b=19||a+b=24
|-
|Good or Excellent||c=27||d=49||c+d=76
|-
|Total||a+c=32||b+d=68||a+b+c+d=100
|}
</center>

To test marginal homogeneity for one single category (in this case poor evaluation) means to test row/column marginal homogeneity for the first category (poor). This is achieved by collapsing all rows and columns corresponding to the other categories.

χ_o^2=(b-c)^2/(b+c)=(-8)^2/46=1.39 ~ χ_(df=1)^2
P( χ_(df=1)^2>1.39)=0.238405

Therefore, we don’t have sufficient evidence to reject the null hypothesis of two evaluators were consistent in their ratings of students.

<center>
[[File:NonParamInference fig 8.png]]
</center>

3.4) Differences of Means of Several Independent Samples: extend the multi-sample inference discussed in ANOVA to the situation where ANOVA assumptions are invalid.
*Motivational Example: Suppose four groups of students are randomly assigned to be taught with four different techniques, and their achievement test scores are recorded. Are the distributions of test scores the same, or do they differ in location? The data is presented in the table below. The small sample sizes and the lack of distribution information of each sample illustrate how ANOVA may not be appropriate for analyzing these types of data.
<center>
{|class="wikitable" style="text-align:center; width:40%" border="1"
|-
|colspan=5|Teaching Method
|-
| ||Method 1||Method 2||Method 3||Method 4
|-
|rowspan=4|Index||65||75||59||94
|-
|87||69||78||89
|-
|73||83||67||80
|-
|79||81||62||88
|}
</center>

*The Kruskal-Wallis Test: Kruskal-Wallis One-Way Analysis of Variance by ranks is a non-parametric method for testing equality of two or more population medians. Intuitively, it is identical to a One-way Analysis of Variance with the raw data (observed measurements) replaced by their ranks. Since it is a non-parametric method, the Kruskal-Wallis Test does not assume a normal population, unlike the analogous one-way ANOVA. However, the test does assume identically-shaped distributions for all groups, except for any difference in their centers (e.g., medians).
**Calculations: Let N be the total number of observations and N=∑_(i=1)^k▒n_i and R(X_ij) be the rank assigned to X_ij and let R_i be the sum of ranks assigned to the i^th sample and R_i=∑_(i=1)^(n_i)▒〖R(X_ij)〗,i=1,2,…,k.

The SOCR program computes R_i for each sample. The test statistic is defined for the following formulation of hypotheses:

H_o: All of the k population distribution functions are identical.

H_a: At least one of the populations tends to yield larger observations than at least one of the other populations.

Suppose {X_(i,1),X_(i,2),…,X_(i,n_i ) } represents the values of the i^th sample, where i≤i≤k.
Test statistics: T=(1/S^2)∑_(i=1)^k▒R_i^2 /n_i-N(N+1)^2/4, where S^2=(1/(N-1) ∑▒〖R(X_ij )^2 〗-N(N+1))^2/4.

Note: If there are no ties, then the test statistic is reduced to:

T=12/N(N+1) ∑_(i=1)^k▒(R_i^2)/n_i -3(N+1).

However, the SOCR implementation allows for the possibility of having ties; so it uses the non-simplified, exact method of computation. Multiple comparisons have to be done here. For each pair of groups, the following is computed and printed at the Result Panel.

|R_i/n_i -R_j/n_j |>t_(1-α/2) ((S^2 (N-1-T))/(N-K))^(1/2)/(1/n_i +1/n_j )^(1/2)

The SOCR computation employs the exact method instead of the approximate one (Conover 1980), since computation is easy and fast to implement and the exact method is somewhat more accurate.
*The Kruskal-Wallis Test Using SOCR Analyses: It is much quicker to use SOCR Analyses to compute the statistical significance of this test. This SOCR Kruskal-Wallis Test Activity may also be helpful in understanding how to use this test in SOCR. For the teaching-methods example above, we can easily compute the statistical significance of the differences between the group medians (centers):
<center>
[[File:NonParamInference fig 9.png]]
</center>

Clearly, there is only one significant group difference between medians, after the multiple testing correction, for the group1 vs. group4 comparison (see below):

Group Method1 vs. Group Method2: 1.0 < 5.2056

Group Method1 vs. Group Method3: 4.0 < 5.2056

'''Group Method1 vs. Group Method4: 6.0 > 5.2056'''

Group Method2 vs. Group Method3: 5.0 < 5.2056

Group Method2 vs. Group Method4: 5.0 < 5.2056

Group Method3 vs. Group Method4: 10.0 > 5.2056

3.5) Differences of Variances of Independent Samples (variance homogeneity): different tests for variance equality can be applied. It is frequently necessary to test if k samples have equal variances. Homogeneity of variances is often a reference to equal variances across samples. Some statistical tests, for example the analysis of variance, assume that variances are equal across groups or samples.
*Calculation: The (modified) Fligner-Killeen test (the median-centering Fligner-Killeen test) provides the means for studying the homogeneity of variances of k populations { X_ij, for 1≤i≤n_j and 1≤j≤k}. The test jointly ranks the absolute values of |X_ij-(X_j ) ̃| and assigns increasing scores a_Ni=Φ^(-1) ((1+i/(N+1)))/2, based on the ranks of all observations. In this test, (X_j ) ̃ is the sample median of the j^th population, and Φ(.) is the cumulative distribution function for Normal distribution.
**Fligner-Killeen test statistics: χ_o^2=(∑_(j=1)^k▒〖n_j ((A_j ) ̅-a ̅ )^2 〗)/V^2 , where (A_j ) ̅ is the mean score for the j^th sample, a ̅ is the overall mean score of all a_Nj and V^2 is the sample variance of all scores. We have N=∑_(j=1)^k▒n_j ,(A_j ) ̅=1/n_j ∑_(j=1)^(n_j)▒a_(Nm_i ) , where a_(Nm_i ) is the increasing rank score for the i^th observation in the j^th sample, a ̅=1/N ∑_(i=1)^N▒a_Ni ,V^2=1/(N-1) ∑_(i=1)^N▒(a_Ni-a ̅ )^2 .
**Fligner-Killeen probabilities: for large sample size, the modified Fligner-Killeen test statistic has an asymptotic chi-square distribution with k-1 degrees of freedom χ_o^2~χ_(k-1)^2.

===Applications===

4.1) This article (http://www.jstor.org/discover/10.2307/2958850?uid=3739728&uid=2&uid=4&uid=3739256&sid=21103957523251) presents a general introduction to nonparametric inference and the model involved and studied the existence of complete and sufficient statistics for this model. It gives an empirical process estimating the model and generalized the empirical cumulative hazard rate from survival analysis. Consistency and weak convergence results were given and tests for comparison of two counting processes, generalizing the two sample rank tests are defined and studied. Finally, it gives an application to a set of biological data.

4.2) This article (http://projecteuclid.org/euclid.aos/1033066215) presents a study relates a different solution to the challenge of constructing a sequence of functions, each based on only finite segments of the past, which together provide a strongly consistent estimator for the conditional probability of the next observation, given the infinite past. It also studied on some extensions to regression, pattern recognition and on-line forecasting.

===Software ===

http://wiki.stat.ucla.edu/socr/index.php/SOCR_EduMaterials_AnalysisActivities_TwoPairedSign

http://www.socr.ucla.edu/htmls/ana/TwoPairedSampleSign-Test_Analysis.html

===Problems===

6.1) Suppose 10 randomly selected rats were chosen to see if they could be trained to escape a maze. The rats were released and timed (seconds) before and after 2 weeks of training (N means the rat did not complete the maze-test). Do the data provide evidence to suggest that the escape time of rats is different after 2 weeks of training? Test using α = 0.05.
<center>
{|class="wikitable" style="text-align:center; width:40%" border="1"
|-
|Rat||Before||After||Sign
|-
|1||100||50||+
|-
|2||38||12||+
|-
|3||N||45||+
|-
|4||122||62||+
|-
|5||95||90||+
|-
|6||116||100||+
|-
|7||56||75||-
|-
|8||135||52||+
|-
|9||104||44||+
|-
|10||N||50||+
|}
</center>

6.2) Automated brain volume segmentation is an important step in many modern computational brain mapping studies. Suppose we have two separate and competing versions of automated brain parsing (segmentation) algorithms that automatically tessellate (partition) the brain into 57 separate regions of interest (ROI’s). An important question then is how consistent are these 2 different techniques, across the 57 ROIs. We can use the ROI volume as a measure of the resulting automated brain parcellation and compare the paired differences between the 2 methods across all ROIs. The image shows an example of a brain parcellated into these 57 regions and the table below contains the volumes of the 57 ROIs for the 2 different brain tessellation techniques. Use appropriate SOCR analyses and relevant SOCR charts to argue whether or not the 2 different methods are consistent and agree on their ROI labels.
<center>
{|class="wikitable" style="text-align:center; width:75%" border="1"
|-
|Index||Volume_Intensity||ROI_Name||Method1_Volume||Method2_Volume
|-
|1||0||Background||9236455||9241667
|-
|2||21||L_superior_frontal_gyrus||78874||78693
|-
|3||22||R_superior_frontal_gyrus||69575||74391
|-
|4||23||L_middle_frontal_gyrus||67336||68872
|-
|5||24||R_middle_frontal_gyrus||68344||67024
|-
|6||25||L_inferior_frontal_gyrus||31912||21479
|-
|7||26||R_inferior_frontal_gyrus||26264||29035
|-
|8||27||L_precentral_gyrus||28942||33584
|-
|9||28||R_precentral_gyrus||35192||30537
|-
|10||29||L_middle_orbitofrontal_gyrus||10141||11608
|-
|11||30||R_middle_orbitofrontal_gyrus||9142||11850
|-
|12||31||L_lateral_orbitofrontal_gyrus||7164||5382
|-
|13||32||R_lateral_orbitofrontal_gyrus||5964||4947
|-
|14||33||L_gyrus_rectus||3840||1995
|-
|15||34||R_gyrus_rectus||2672||2994
|-
|16||41||L_postcentral_gyrus||24586||27672
|-
|17||42||R_postcentral_gyrus21736||28159
|-
|18||43||L_superior_parietal_gyrus||25791||27500
|-
|19||44||R_superior_parietal_gyrus||28850||32674
|-
|20||45||L_supramarginal_gyrus||16445||22373
|-
|21||46||R_supramarginal_gyrus||11893||11018
|-
|22||47||L_angular_gyrus||20740||22245
|-
|23||48||R_angular_gyrus||20247||17793
|-
|24||49||L_precuneus||14491||12983
|-
|25||50 R_precuneus||15589||16323
|-
|26||61||L_superior_occipital_gyrus||6842||6106
|-
|27||62||R_superior_occipital_gyrus||5673||6539
|-
|28||63||L_middle_occipital_gyrus||15011||19085
|-
|29||64||R_middle_occipital_gyrus||19063||25747
|-
|30||65||L_inferior_occipital_gyrus||10411||8675
|-
|31||66||R_inferior_occipital_gyrus||12142||12277
|-
|32||67||L_cuneus||6935||9700
|-
|33||68 R_cuneus||7491||11765
|-
|34||81||L_superior_temporal_gyrus||29962||34934
|-
|35||82||R_superior_temporal_gyrus||30630||28788
|-
|36||83||L_middle_temporal_gyrus||27558||19633
|-
|37||84||R_middle_temporal_gyrus||26314||25301
|-
|38||85||L_inferior_temporal_gyrus||24817||24885
|-
|39||86||R_inferior_temporal_gyrus||25088||20661
|-
|40||87||L_parahippocampal_gyrus||6761||6977
|-
|41||88||R_parahippocampal_gyrus||6529||7964
|-
|42||89||L_lingual_gyrus||16752||14748
|-
|43||90||R_lingual_gyrus||20914||18500
|-
|44||91||L_fusiform_gyrus||16565||15020
|-
|45||92||R_fusiform_gyrus||14409||17311
|-
|46||101||L_insular_cortex||10779||9814
|-
|47||102||R_insular_cortex||8222||5599
|-
|48||121||L_cingulate_gyrus||14662||12490
|-
|49||122||R_cingulate_gyrus||16595||14489
|-
|50||161||L_caudate||1906||1608
|-
|51||162||R_caudate||2353||1997
|-
|52||163||L_putamen||3015||2622
|-
|53||164||R_putamen||2177||3758
|-
|54||165||L_hippocampus||3791||4454
|-
|55||166||R_hippocampus||3596||4673
|-
|56||181||cerebellum||174045||158617
|-
|57||182||brainstem||32567||28225
|}
</center>

6.3) Urinary Fluoride Concentration in Cattle: The urinary fluoride concentration (ppm) was measured both for a sample of livestock grazing in an area previously exposed to fluoride pollution and also for a similar sample of livestock grazing in an unpolluted area.
<center>
{|class="wikitable" style="text-align:center; width:40%" border="1"
|-
|Polluted||Unpolluted
|-
|21.3||10.1
|-
|18.7||18.3
|-

|21.4||17.2
|-
|17.1||18.4
|-
|11.1||20.0
|-
|20.9||
|-
|19.7||
|}
</center>

6.4) To study whether the variances in certain time period (e.g. 1981 to 2006) of the consumer-price-indices (CPI) of several items were significantly different. Use the SOCR CPI dataset (http://wiki.socr.umich.edu/index.php/SOCR_Data_Dinov_021808_ConsumerPriceIndex) to answer this question for the Fuel, Oil, Bananas, Tomatoes, Orange juice, Beef and Gasoline items. (Apply the methods introduced in Differences of variances of independent samples).

===References===

Statistical inference / George Casella, Roger L. Berger. http://mirlyn.lib.umich.edu/Record/004199238

Sampling / Steven K. Thompson. http://mirlyn.lib.umich.edu/Record/004232056

Sampling theory and methods / S. Sampath. http://mirlyn.lib.umich.edu/Record/004133572

SMHS NonParamInference

2014-08-14T18:19:32Z

2014-08-14T17:59:47Z

Mwolvie:

File:ReliabilityValidity Fig 2.png

2014-08-14T14:14:33Z

Mwolvie:

File:ReliabilityValidity Fig 1.png

2014-08-14T14:14:10Z

Mwolvie:

SMHS ReliabilityValidity

2014-08-14T14:13:55Z

Mwolvie: /* Scientific Methods for Health Sciences - Measurement Reliability and Validity */

==[[SMHS| Scientific Methods for Health Sciences]] - Measurement Reliability and Validity ==

===Overview===
Reliability and validity are two of the most commonly used criteria in choosing the ideal measurement. Reliability is the overall consistency of a measure that is the ability to produce similar results under consistent conditions. Validity is the extent to which a measurement is accurately to reflect the real fact that is the extent to which the measurement measures what it claims to measure. In the perfect situation, we would expect the measurement to be reliable and valid, though it is not always achievable. In fact, in many cases, we need to strike a balance between reliability and validity based on our objectives of the study in choosing the ideal measurement. In this section, we are going to discuss about the measurement reliability and validity and illustrate their application with examples and we are going to focus on the application of these two criteria in the field of epidemiology as an example.

===Motivation===
In choosing an ideal measurement in the study, we are always encountered with the problem of whether it is capable of producing the similar results with consistent conditions and whether it is capable to measure what it claims to measure. Ideally, we would prefer to be able to measure the exact situation and to produce similar results when measured repeatedly in consistent conditions. In real studies, we would need to choose between measurements to strike a balance between these two aspects given the restrictions in real world. So, how do we choose between validity and reliability? How these two would influence the results of the test?

===Theory===

3.1) Measurement: refers to the systematic, replicable process through which the objects are quantified or classified with respect to a particular dimension and is usually achieved by assigning numerical values to the objects measured.
*There are four levels of measurements (the relationship among the values assigned to the attributes for a variable): (1) Nominal measure: the numerical values just ‘name’ the attributes uniquely and no ordering of the cases is implied; (2) Ordinal measure: where the attributes can be rank-ordered while the distances between attributes don’t have any meaning. For example, the education background of the participants are measured in a study where 0=less than high school; 1=some high school; 2=high school degree; 3=some college; 4=college degree; 5=post college. (3) Interval measure: where the distance is meaningful in the measurement. For example, the temperature of the participants. (4) Ratio measure: an absolute zero is meaningful meaning that you can construct a meaningful fraction with a ratio variable.
*Variation in a repeated measure can be caused by (1) pure chance or unsystematic events caused by subject, observer, situations, instrument or data processing; (2) systematic inconsistency; (3) actual change in the underlying event being measured.
*Validity of a measure is the extent to which the measurement can describe or quantify what it intends to measure; reliability of a measure is the extent to which a measure can be depended upon to secure consistent results in repeated application.

The following charts shows the three possible outcomes: (from left to right) valid not reliable, reliable not valid and valid and reliable.

[[File:ReliabilityValidity Fig 1.png]][[File:ReliabilityValidity Fig 2.png]][[File:ReliabilityValidity Fig 3.png]]

3.2) Validity: validity is the extent to which the assessment measures what it is supposed to measure while reliability is the ability to replicate results on same sample if test if repeated. Within validity, we don’t always expect the measurements to produce the similar results in repeated tests. Similarly, a measure may not be valid within reliability.
There are different types of validity:
Construct validity: refers to the extent to which the operation actually measures what the theory intends to. It involves the empirical and theoretical support for the interpretation of the measure. (1) Convergent validity: refers to the extent to which a measure is correlated with other measures that it is theoretically correlated to; (2) Discriminant validity: refers to whether the measurement is supposed to be unrelated are unrelated.
Content validity: refers to the non-statistical type of validity, which is to test the extent to which the content of the test matches the content associated with the construct. (1) Representation validity: the extent to which an abstract theoretical construct can be turned into a specific practical test; (2) Face validity: test whether the test appears to measure a certain criterion.
Criterion validity: involves the correlation between the test and a criterion variable taken as representative of the construct and compares the test with other measures or outcomes. (1) Concurrent validity: refers to the extent to which the operation correlates with other measures with the same construct measured at the same time; (2) Predictive validity: refers to the extent to which the operation can predict other measures of the same construct measured at the same time.
Experimental validity: validity of design of experimental research studies. (1) statistical conclusion validity: the extent to which conclusions about the relationship among variables based on the data are correct or reasonable, it involves ensuring the use of adequate sampling procedures, appropriate statistical tests and reliable measurement procedures; (2) internal validity: estimate the extent to which conclusions about causal relationships be made; (3) external validity: concerns the extent to which the results of the study can be held to be true in general case.

3.3) Reliability (repeatability) of tests: can the results be replicated if the test is redone? The results may be influenced by three factors: (1) Intrasubject variation: variation within individual subjects; (2) Intraobserver variation: variation in reading of results by the same reader; (3) Interobserver variation: variation between those reading results.
Types of Reliability: (1) Test-retest reliability: measure of reliability obtained by administering the same test twice over a period of time to a group of individuals; (2) Parallel forms reliability: measure of reliability obtained by administering different versions of an assessment tool to the same group of individuals; (3) Inter-rater reliability: measure of reliability used to assess the extent to which different judges or raters agree in their assessment decisions; (4) Internal consistency reliability: measure of reliability used to evaluate the extent to which different test items that probe the same construct produce similar results.

3.4) Kappa statistic: Answers the question of ‘How much better is the agreement between observers than would be expected by chance alone?’
Kappa=((% agreement observed)-(% agreement expected by chance alone))/(100%-(% agreement expected by chance alone))
Percent agreement=(number in cells that 'agree')/(Total number readings)*100
Calculation of Kappa:
Reader 1
Reader 2 Positive Negative
Positive 180 40
Negative 50 230

Percent agreement: (1) proportion of tests reader 1 rate as positive =(180+50)/500=46%; proportion of tests reader 1 rate as negative =54%; if the results from reader 1 and reader 2 are independent, then reader 1 should have 46% positives regardless of reader 2’s scores.
Expected agreement based on chance alone: for 220 times reader 2 is positive, we expect reader 1 will be positive 46% of the time; for 280 times reader 2 is negative, we expect reader 1 will be negative 54% of the time:

Reader 1
Reader 2 Positive Negative
Positive 101 119
Negative 129 151

Expected agreement by chance =(101+151)/500=50.4%.
Observed agreement = 82%; expected agreement based on chance = 50.4%. Kappa=((% agreement observed)-(% agreement expected by chance alone))/(100%-(% agreement expected by chance alone))=(82%-50.4%)/(100%-50.4%)=63.71%
Interpretation of Kappa: > 0.75 excellent; 0.4 – 0.75 intermediate to good; < 0.40 poor reliability.

4) Applications

4.1) This article (http://www.socialresearchmethods.net/kb/constval.php) presents a comprehensive introduction to measurements, validity and reliability and illustrate the concepts and application with examples. It is very well developed and would be a great introduction to the material we are going to cover in this lecture.

4.2) This article (http://www.egadconnection.org/Reliability%20and%20validity.pdf) presents a general introduction to reliability, validity and generalizability and studied on various problems with measurement. It gives comprehensive analysis of reliability and validity with definitions, different ways to measure reliability and validity as well as problems associated with these characteristics. This would be a great start to get to know measurement reliability and validity.

4.3) This article (http://psycnet.apa.org/psycinfo/1995-00092-001) assessed the reliability and validity of the Childhood Trauma Questionnaire (CTQ), a retrospective measure of child abuse and neglect. 286 drug- or alcohol-dependent patients (aged 24–68 years) were given the CTQ as part of a larger test battery, and 40 of these patients were given the questionnaire again after an interval of 2–6 months. 68 Ss were also given the Childhood Trauma Interview. Principal-components analysis of responses on the CTQ yielded 4 rotated orthogonal factors: physical and emotional abuse, emotional neglect, sexual abuse, and physical neglect. The CTQ demonstrated high internal consistency and good test-retest reliability over an interval of 2–6 months. The CTQ also demonstrated convergence with the Childhood Trauma Interview indicating that Ss' reports of child abuse and neglect based on the CTQ were highly stable, both over time and across types of instruments.

5) Software
none

6) Problems

6.1) In public health practice, optimizing the validity of tests is important in order to:
a. reduce health care costs
b. reduce unnecessary stress for patients
c. Be able to identify opportunities for intervention early in the course of disease
d. all of the above

6.2) The Kappa statistic is used to measure ___ of a test?
a. sensitivity
b. reliability
c. positive predictive value
d. specificity

6.3) Randomization of treatment groups ensures the study’s external validity.
a. True
b. False

6.4) In a study investigating whether a new serum-based screening test for pancreatic cancer allowed for earlier detection than traditional tests, the researchers found that those who enrolled in the study were more likely to have a family history of pancreatic cancer than those who did not. This characteristic of the study population affects the study’s:
a. Internal validity
b. External validity
c. Both
d. Neither

6.5) As sample size increases:
a. Sampling variability increases and the chance of selecting an unrepresentative sample increases
b. Sampling variability increases and the chance of selecting an unrepresentative sample decreases
c. Sampling variability decreases and the chance of selecting an unrepresentative sample decreases
d. Sampling variability decreases and the chance of selecting an unrepresentative sample increases

7) References
http://mirlyn.lib.umich.edu/Record/004199238
http://mirlyn.lib.umich.edu/Record/004232056
http://mirlyn.lib.umich.edu/Record/004133572

SMHS ROC

2014-08-14T14:08:58Z

Mwolvie: /* Problems */

==[[SMHS| Scientific Methods for Health Sciences]] - Receiver Operating Characteristic (ROC) Curve ==

===Overview===
Receiver operating characteristic (ROC curve) is a graphical plot, which illustrates the performance of a binary classifier system as its discrimination threshold varies. The ROC curve is created by plotting the fraction of true positive out of the total actual positives vs. the fraction of false positives out of the total actual negatives at various threshold settings. In this section, we are going to introduce the ROC curve and illustrate applications of this method with examples.

===Motivation===
We have talked about the cases with a binary classification where the outcomes are either absent or present and the test results are positive or negative. We have also discussed about sensitivity and specificity of a test and are familiar with the concepts of true positive and true negatives. With ROC curve, we are looking to demonstrate the following aspects:
*To show the tradeoff between sensitivity and specificity;
*The closer the curve follows the left-hand border and top border of ROC space, the more accurate is the test;
*The closer the curve comes to the 45-degree diagonal, the less accurate is the test;
*The slope of the tangent line at a cut-point gives the likelihood ratio for the value of the test;
*The area under the curve is a measure of text accuracy.
<center>
{|class="wikitable" style="text-align:center; width:75%" border="1"
|-
|colspan=2 rowspan=2| ||colspan=2|Actual condition
|-
|Absent (H_0 is true)||Present (H_1 is true)
|-
|rowspan=2|Test Result||Negative (fail to reject H_0)||Condition absent + Negative result = True (accurate) Negative (TN, 0.98505)||Condition present + Negative result = False (invalid) Negative (FN, 0.00025)Type II error (β)
|-
|Positive||(reject H_0) Condition absent + Positive result = False Positive (FP, 0.00995) Type I error (α)||Condition Present + Positive result = True Positive (TP, 0.00475)
|-
|Test Interpretation||Power = 1-FN=1-0.00025 = 0.99975||Specificity: TN/(TN+FP) =0.98505/(0.98505+0.00995) = 0.99||Sensitivity: TP/(TP+FN) =0.00475/(0.00475+ 0.00025)= 0.95
|}
</center>

===Theory===

3.1) Review of basic concepts in a binary classification:
<center>
{|class="wikitable" style="text-align:center; width:75%" border="1"
|-
|rowspan=2| ||rowspan=2| ||colspan=2|Disease Status||colspan=2|Metrics
|-
|Disease||No Disease||Prevalence=(∑▒〖Condition positive〗)/(∑▒〖Total population〗)| ||
|-
|rowspan=2|Screening Test||Positive||a (True positives)||b (False positives)||Positive predictive value (PPV)=(∑▒〖Ture positive〗)/(∑▒〖Test positives〗)||False discovery rate (FDR)==(∑▒〖False positive〗)/(∑▒〖Test positive〗)
|-
|Negative||c (False negatives)||d (True negatives)||False omission rate (FOR)=(∑▒〖False negative〗)/(∑▒〖Test negative〗)||Negative predictive value (NPV)=(∑▒〖True negative〗)/(∑▒〖Test negative〗)
|-
| ||Positive Likelihood Ratio=TPR/FBR||True positive rate (TPR)=(∑▒〖True positive〗)/(∑▒〖condition positive〗)||False positive rate (FPR)=(∑▒〖False positve〗)/(∑▒〖condition positive〗)||Accuracy(ACC)=(∑▒〖True positive〗+∑▒〖True negative〗)/(∑▒〖Total population 〗)| ||
|-
| ||Negative Likelihood Ratio=FNR/TNR||False negative rate (FNR)=(∑▒〖False negative〗)/(∑▒〖condition negative〗)||True negative rate (TNR)=(∑▒〖True negative〗)/(∑▒〖condition negative〗)||True negative rate (TNR)=(∑▒〖True negative〗)/(∑▒〖condition negative〗)| ||
|}
</center>

3.2) Introduction of ROC Curve:

Sensitivity and specificity are both characteristics of a test but they also depend on the definition of what constitutes an abnormal test. Consider a medical test on diagnostic tests where the cut-points would without doubt influence the test results. We can use the hypothyroidism data from the likelihood ratio to illustrate how these two characteristics change depending on the choice of T4 level that defines hypothyroidism. Recall the data where patients with suspected hypothyroidism are reported.
<center>
{|class="wikitable" style="text-align:center; width:75%" border="1"
|-
|T4||<1||1-2||2-3||3-4||4-5||5-6||6-7||7-8||8-9||9-10||10-11||11-12||>12
|-
|Hypothyroid||2||3||1||8||4||4||3||3||1||0||2||1||0
|-
|Euthyroid||0||0||0||0||1||6||11||19||17||20||11||4||4
|}
</center>

With the following cut-points, we have the data listed:
<center>
{|class="wikitable" style="text-align:center; width:75%" border="1"
|-
|T4 value||Hypothyroid||Euthyroid
|-
|5 or less||18||1
|-
|5.1 - 7||7||17
|-
|7.1 - 9||4||36
|-
|9 or more||3||39
|-
|Totals:||32||93
|}
</center>

Suppose that patients with T4 of 5 or less are considered to be hypothyroid, then the data would be displayed as the following and the sensitivity is 0.56 and specificity is 0.99 in this case.
<center>
{|class="wikitable" style="text-align:center; width:75%" border="1"
|-
|T4 value||Hypothyroid||Euthyroid
|-
|5 or less||18||1
|-
|More than 5||14||92
|-
|Totals:||32||93
|}
</center>
Now suppose, we decided to be less stringent on the disease and consider the patients with T4 values of 7 or less to be hypothyroid, then the data would be recorded as the following and the sensitivity in this case would be 0.78 and specificity 0.81:
<center>
{|class="wikitable" style="text-align:center; width:75%" border="1"
|-
|T4 value||Hypothyroid||Euthyroid
|-
|7 or less||25||18
|-
|More than 7||7||75
|-
|Totals:||32||93
|}
</center>
If we move the cut-point for hypothyroidism a little bit higher, say 9, we would have the sensitivity of 0.91 and specificity 0.42:
<center>
{|class="wikitable" style="text-align:center; width:75%" border="1"
|-
|T4 value||Hypothyroid||Euthyroid
|-
|9 or less||29||54
|-
|More than 9||3||39
|-
|Totals:||32||93
|}
</center>
To sum up, we have the pairs of sensitivity an specificity with corresponding cut-points in the following table:
<center>
{|class="wikitable" style="text-align:center; width:75%" border="1"
|-
|Cut points||Sensitivity||Specificity
|-
|5||0.56||0.99
|-
|7||0.78||0.81
|-
|9||0.91||0.42
|}
</center>
From the table above, we observe that the sensitivity improves with increasing cut-point T4 value while specificity increases with decreasing cut-point T4 value. That is a tradeoff between sensitivity and specificity. The table above can also be shown as TP and FP.
<center>
{|class="wikitable" style="text-align:center; width:75%" border="1"
|-
|Cut points||True positives||False positives
|-
|5||0.56||0.01
|-
|7||0.78||0.19
|-
|9||0.91||0.58
|}
</center>

Plot sensitivity vs. specificity, we have the ROC curve:

x<-c(0,0.01,0.19,0.58,1)

y<-c(0,0.56,0.78,0.91,1)

plot(x,y,type='o',main='ROC curve for T4',xlab='False positive rate (specificity)',ylab='True positive rate (sensitivity)')
<center>
[[File:ROC Fig 1.png]]
</center>
*Accuracy of a test is measured by the area under the ROC Curve and a rough guide for classifying the accuracy of a diagnostic test is traditional academic point system: 0.90-1:excellent;0.8-0.9: good (B);0.7-0.8:fair (C);0.6-0.7:poor (D);0.5-0.7:fail (F).
With our example above, the area under the T4 ROC curve is 0.86, which shows that the accuracy of the test is good in separating hypothyroid from euthyroid patients.

*How to calculate the area under the curve? The area measures discrimination, which is the ability of the test to correctly classify those with and without the disease. The area under the curve is the percentage of randomly drawn pairs for which this is true. Two methods are commonly used to calculate the area of the scope: (1) non-parametric method based on constructing trapezoids under the curve as approximation of area; (2) a parametric method using a MLE to fit a smooth curve to data points.

===Applications===

4.1) This article (http://pubs.rsna.org/doi/abs/10.1148/radiology.143.1.7063747) titled The Meaning And the Use Of The Area Under A Receiver Operating Characteristic (ROC) Curve presented a representation and interpretation of the area under a ROC curve obtained by the ‘rating’ method, or by mathematical predictions based on patient characteristics. It showed that that in such a setting the area represents the probability that a randomly chosen diseased subject is (correctly) rated or ranked with greater suspicion than a randomly chosen non-diseased subject.

4.2) This article (http://www.sciencedirect.com/science/article/pii/S0001299878800142) illustrated practical experimental techniques for measuring ROC curves and discussed about the issues of case selection and curve-fitting. It also talked about possible generalizations of conventional ROC analysis to account for decision performance in complex diagnostic tasks and showed ROC analysis related in direct and natural way to cost/benefit analysis of diagnostic decision making. This paper developed the concepts of ‘average diagnostic cost’ and ‘average net benefit’ to identify the optimal compromise among various kinds of diagnostic error and suggested ways in ROC analysis to optimize diagnostic strategies.

===Software===

"#"With the given example in R:

x<-c(0,0.01,0.19,0.58,1)

y<-c(0,0.56,0.78,0.91,1)

plot(x,y,type='o',main='ROC curve for T4',xlab='False positive rate (specificity)',ylab='True positive rate (sensitivity)')

===Problems===

6.1) Suppose that a new study is conducted on lung cancer and the following data is collected in identify between two types of lung cancers (say type a and type b). Conduct the ROC curve for this example by varying the cut-points from 2 to 10 by increasing 2 units each time. Calculate the area under the curve and interpret on the accuracy of the test.

<center>
{|class="wikitable" style="text-align:center; width:75%" border="1"
|-
|measurements||<1||1-2||2-3||3-4||4-5||5-6||6-7||7-8||8-9||9-10||10-11||11-12||>12
|-
|Type a||2||1||4||2||8||7||4||3||0||0||1||2||2
|-
|Type b||1||3||0||2||2||5||10||23||18||20||15||8||2
|}
</center>
6.2) When a serious disease can be treated if it is caught early, it is more important to have a test with high specificity than high sensitivity.

a. True

b. False

6.3) The positive predictive value of a test is calculated by dividing the number of:

(a) True positives in the population

(b) True negatives in the population

(c) People who test positive

(d) People who test negative

6.4) A new screening test has been developed for diabetes. The table below represents the results of the new test compared to the current gold standard.

<center>
{|class="wikitable" style="text-align:center; width:75%" border="1"
|-
| ||Condition positive||Condition negative||Total
|-
|Test positive||80||70||150
|-
|Test negative||10||240||250
|-
|Total||90||310||400
|}
</center>
What is the sensitivity of the test?

(a) 77%

(b) 89%

(c) 80%

(d) 53%

6.5) What is the specificity of the test?

(a) 77%

(b) 89%

(c) 80%

(d) 53%

6.6) What is the positive predictive value of the test?

(a) 77%

(b) 89%

(c) 80%

(d) 53%

7) References

Statistical inference / George Casella, Roger L. Berger. http://mirlyn.lib.umich.edu/Record/004199238

Sampling / Steven K. Thompson. http://mirlyn.lib.umich.edu/Record/004232056

Sampling theory and methods / S. Sampath. http://mirlyn.lib.umich.edu/Record/004133572

Introduction to ROC Curves http://gim.unmc.edu/dxtests/roc1.htm

SMHS ROC

2014-08-14T14:07:44Z

Mwolvie: /* Theory */

SMHS ROC

2014-08-14T14:05:42Z

Mwolvie: /* Scientific Methods for Health Sciences - Receiver Operating Characteristic (ROC) Curve */

==[[SMHS| Scientific Methods for Health Sciences]] - Receiver Operating Characteristic (ROC) Curve ==

===Overview===
Receiver operating characteristic (ROC curve) is a graphical plot, which illustrates the performance of a binary classifier system as its discrimination threshold varies. The ROC curve is created by plotting the fraction of true positive out of the total actual positives vs. the fraction of false positives out of the total actual negatives at various threshold settings. In this section, we are going to introduce the ROC curve and illustrate applications of this method with examples.

===Motivation===
We have talked about the cases with a binary classification where the outcomes are either absent or present and the test results are positive or negative. We have also discussed about sensitivity and specificity of a test and are familiar with the concepts of true positive and true negatives. With ROC curve, we are looking to demonstrate the following aspects:
*To show the tradeoff between sensitivity and specificity;
*The closer the curve follows the left-hand border and top border of ROC space, the more accurate is the test;
*The closer the curve comes to the 45-degree diagonal, the less accurate is the test;
*The slope of the tangent line at a cut-point gives the likelihood ratio for the value of the test;
*The area under the curve is a measure of text accuracy.
<center>
{|class="wikitable" style="text-align:center; width:75%" border="1"
|-
|colspan=2 rowspan=2| ||colspan=2|Actual condition
|-
|Absent (H_0 is true)||Present (H_1 is true)
|-
|rowspan=2|Test Result||Negative (fail to reject H_0)||Condition absent + Negative result = True (accurate) Negative (TN, 0.98505)||Condition present + Negative result = False (invalid) Negative (FN, 0.00025)Type II error (β)
|-
|Positive||(reject H_0) Condition absent + Positive result = False Positive (FP, 0.00995) Type I error (α)||Condition Present + Positive result = True Positive (TP, 0.00475)
|-
|Test Interpretation||Power = 1-FN=1-0.00025 = 0.99975||Specificity: TN/(TN+FP) =0.98505/(0.98505+0.00995) = 0.99||Sensitivity: TP/(TP+FN) =0.00475/(0.00475+ 0.00025)= 0.95
|}
</center>

===Theory===

3.1) Review of basic concepts in a binary classification:
<center>
{|class="wikitable" style="text-align:center; width:75%" border="1"
|-
|rowspan=2| ||rowspan=2| ||colspan=2|Disease Status||colspan=2|Metrics
|-
|Disease||No Disease||Prevalence=(∑▒〖Condition positive〗)/(∑▒〖Total population〗)| ||
|-
|rowspan=2|Screening Test||Positive||a (True positives)||b (False positives)||Positive predictive value (PPV)=(∑▒〖Ture positive〗)/(∑▒〖Test positives〗)||False discovery rate (FDR)==(∑▒〖False positive〗)/(∑▒〖Test positive〗)
|-
|Negative||c (False negatives)||d (True negatives)||False omission rate (FOR)=(∑▒〖False negative〗)/(∑▒〖Test negative〗)||Negative predictive value (NPV)=(∑▒〖True negative〗)/(∑▒〖Test negative〗)
|-
| ||Positive Likelihood Ratio=TPR/FBR||True positive rate (TPR)=(∑▒〖True positive〗)/(∑▒〖condition positive〗)||False positive rate (FPR)=(∑▒〖False positve〗)/(∑▒〖condition positive〗)||Accuracy(ACC)=(∑▒〖True positive〗+∑▒〖True negative〗)/(∑▒〖Total population 〗)| ||
|-
| ||Negative Likelihood Ratio=FNR/TNR||False negative rate (FNR)=(∑▒〖False negative〗)/(∑▒〖condition negative〗)||True negative rate (TNR)=(∑▒〖True negative〗)/(∑▒〖condition negative〗)| || | ||
|}
</center>

3.2) Introduction of ROC Curve:

Sensitivity and specificity are both characteristics of a test but they also depend on the definition of what constitutes an abnormal test. Consider a medical test on diagnostic tests where the cut-points would without doubt influence the test results. We can use the hypothyroidism data from the likelihood ratio to illustrate how these two characteristics change depending on the choice of T4 level that defines hypothyroidism. Recall the data where patients with suspected hypothyroidism are reported.
<center>
{|class="wikitable" style="text-align:center; width:75%" border="1"
|-
|T4||<1||1-2||2-3||3-4||4-5||5-6||6-7||7-8||8-9||9-10||10-11||11-12||>12
|-
|Hypothyroid||2||3||1||8||4||4||3||3||1||0||2||1||0
|-
|Euthyroid||0||0||0||0||1||6||11||19||17||20||11||4||4
|}
</center>

With the following cut-points, we have the data listed:
<center>
{|class="wikitable" style="text-align:center; width:75%" border="1"
|-
|T4 value||Hypothyroid||Euthyroid
|-
|5 or less||18||1
|-
|5.1 - 7||7||17
|-
|7.1 - 9||4||36
|-
|9 or more||3||39
|-
|Totals:||32||93
|}
</center>

Suppose that patients with T4 of 5 or less are considered to be hypothyroid, then the data would be displayed as the following and the sensitivity is 0.56 and specificity is 0.99 in this case.
<center>
{|class="wikitable" style="text-align:center; width:75%" border="1"
|-
|T4 value||Hypothyroid||Euthyroid
|-
|5 or less||18||1
|-
|More than 5||14||92
|-
|Totals:||32||93
|}
</center>
Now suppose, we decided to be less stringent on the disease and consider the patients with T4 values of 7 or less to be hypothyroid, then the data would be recorded as the following and the sensitivity in this case would be 0.78 and specificity 0.81:
<center>
{|class="wikitable" style="text-align:center; width:75%" border="1"
|-
|T4 value||Hypothyroid||Euthyroid
|-
|7 or less||25||18
|-
|More than 7||7||75
|-
|Totals:||32||93
|}
</center>
If we move the cut-point for hypothyroidism a little bit higher, say 9, we would have the sensitivity of 0.91 and specificity 0.42:
<center>
{|class="wikitable" style="text-align:center; width:75%" border="1"
|-
|T4 value||Hypothyroid||Euthyroid
|-
|9 or less||29||54
|-
|More than 9||3||39
|-
|Totals:||32||93
|}
</center>
To sum up, we have the pairs of sensitivity an specificity with corresponding cut-points in the following table:
<center>
{|class="wikitable" style="text-align:center; width:75%" border="1"
|-
|Cut points||Sensitivity||Specificity
|-
|5||0.56||0.99
|-
|7||0.78||0.81
|-
|9||0.91||0.42
|}
</center>
From the table above, we observe that the sensitivity improves with increasing cut-point T4 value while specificity increases with decreasing cut-point T4 value. That is a tradeoff between sensitivity and specificity. The table above can also be shown as TP and FP.
<center>
{|class="wikitable" style="text-align:center; width:75%" border="1"
|-
|Cut points||True positives||False positives
|-
|5||0.56||0.01
|-
|7||0.78||0.19
|-
|9||0.91||0.58
|}
</center>

Plot sensitivity vs. specificity, we have the ROC curve:

x<-c(0,0.01,0.19,0.58,1)

y<-c(0,0.56,0.78,0.91,1)

plot(x,y,type='o',main='ROC curve for T4',xlab='False positive rate (specificity)',ylab='True positive rate (sensitivity)')
<center>
[[File:ROC Fig 1.png]]
</center>
*Accuracy of a test is measured by the area under the ROC Curve and a rough guide for classifying the accuracy of a diagnostic test is traditional academic point system: 0.90-1:excellent;0.8-0.9: good (B);0.7-0.8:fair (C);0.6-0.7:poor (D);0.5-0.7:fail (F).
With our example above, the area under the T4 ROC curve is 0.86, which shows that the accuracy of the test is good in separating hypothyroid from euthyroid patients.

*How to calculate the area under the curve? The area measures discrimination, which is the ability of the test to correctly classify those with and without the disease. The area under the curve is the percentage of randomly drawn pairs for which this is true. Two methods are commonly used to calculate the area of the scope: (1) non-parametric method based on constructing trapezoids under the curve as approximation of area; (2) a parametric method using a MLE to fit a smooth curve to data points.

===Applications===

4.1) This article (http://pubs.rsna.org/doi/abs/10.1148/radiology.143.1.7063747) titled The Meaning And the Use Of The Area Under A Receiver Operating Characteristic (ROC) Curve presented a representation and interpretation of the area under a ROC curve obtained by the ‘rating’ method, or by mathematical predictions based on patient characteristics. It showed that that in such a setting the area represents the probability that a randomly chosen diseased subject is (correctly) rated or ranked with greater suspicion than a randomly chosen non-diseased subject.

4.2) This article (http://www.sciencedirect.com/science/article/pii/S0001299878800142) illustrated practical experimental techniques for measuring ROC curves and discussed about the issues of case selection and curve-fitting. It also talked about possible generalizations of conventional ROC analysis to account for decision performance in complex diagnostic tasks and showed ROC analysis related in direct and natural way to cost/benefit analysis of diagnostic decision making. This paper developed the concepts of ‘average diagnostic cost’ and ‘average net benefit’ to identify the optimal compromise among various kinds of diagnostic error and suggested ways in ROC analysis to optimize diagnostic strategies.

===Software===

"#"With the given example in R:

x<-c(0,0.01,0.19,0.58,1)

y<-c(0,0.56,0.78,0.91,1)

plot(x,y,type='o',main='ROC curve for T4',xlab='False positive rate (specificity)',ylab='True positive rate (sensitivity)')

===Problems===

6.1) Suppose that a new study is conducted on lung cancer and the following data is collected in identify between two types of lung cancers (say type a and type b). Conduct the ROC curve for this example by varying the cut-points from 2 to 10 by increasing 2 units each time. Calculate the area under the curve and interpret on the accuracy of the test.

<center>
{|class="wikitable" style="text-align:center; width:75%" border="1"
|-
|measurements||<1||1-2||2-3||3-4||4-5||5-6||6-7||7-8||8-9||9-10||10-11||11-12||>12
|-
|Type a||2||1||4||2||8||7||4||3||0||0||1||2||2
|-
|Type b||1||3||0||2||2||5||10||23||18||20||15||8||2
|}
</center>
6.2) When a serious disease can be treated if it is caught early, it is more important to have a test with high specificity than high sensitivity.

a. True

b. False

6.3) The positive predictive value of a test is calculated by dividing the number of:

(a) True positives in the population

(b) True negatives in the population

(c) People who test positive

(d) People who test negative

6.4) A new screening test has been developed for diabetes. The table below represents the results of the new test compared to the current gold standard.

<center>
{|class="wikitable" style="text-align:center; width:75%" border="1"
|-
| ||Condition positive||Condition negative||Total
|-
|Test positive||80||70||150
|-
|Test negative||10||240||250
|-
|Total||90||310||400
|}
</center>
What is the sensitivity of the test?

(a) 77%

(b) 89%

(c) 80%

(d) 53%

6.5) What is the specificity of the test?

(a) 77%

(b) 89%

(c) 80%

(d) 53%

6.6) What is the positive predictive value of the test?

(a) 77%

(b) 89%

(c) 80%

(d) 53%

7) References

Statistical inference / George Casella, Roger L. Berger. http://mirlyn.lib.umich.edu/Record/004199238

Sampling / Steven K. Thompson. http://mirlyn.lib.umich.edu/Record/004232056

Sampling theory and methods / S. Sampath. http://mirlyn.lib.umich.edu/Record/004133572

Introduction to ROC Curves http://gim.unmc.edu/dxtests/roc1.htm

Answers:
2 – 6 : F, c, b, a, d

2014-07-31T19:44:50Z

Mwolvie: /* References */

==[[SMHS| Scientific Methods for Health Sciences]] - Data Management ==

===Overview===
Data management comprised all the discipline related to managing data as a valuable resource and it is of significant importance in various fields. It is officially defined as the development and execution of architectures, policies, practices and procedures that properly manage the full data lifecycle needs of an enterprise. There are various ways to manage data. In this lecture, we are going to introduce the fundamental roles of data management in statistics and illustrate commonly used ways and steps in data managements through examples from different areas including tables, streams, cloud, warehouses, DBs, arrays, binary ASC II, handling and mechanics.

===Motivation===
The next step after getting data would be to make proper management of the data in hand. Data management is of course a vital step in data analysis and is crucial to the success and reproducibility of a statistical analysis. So how would we do a good data management and what are the commonly used ways of data management? In order to further make good use of the data, we are going to learn more about the area of data management and various ways to implement it. Selection of appropriate tools and efficient use of these tools can save the researcher numerous hours, and allow other researchers to leverage the products of their work.

===Theory===

3.1) Tables: one of the most commonly used ways to manage data. It is a means of arranging data in rows and columns. It is of pervasive use throughout all research and data analysis.
There are two basic types of tables:
Simple Table: consider the following example of a table summarizing the data from two groups. The table presents a general comparison of the two groups and shows us a clear picture of the measurements and the comparative characteristics between the two groups. What we learn from the table below: (1) these two groups with the same sample size have the same mean; (2) group 1 has a bigger range of the data values; (3) group 2 has a smaller standard deviation indicating a less variant dataset compared to group 1.
Minimum Maximum Mean Standard Deviation Size
Group 1 12 45 22 2.6 40
Group 2 15 30 22 1.5 40

Multi-dimensional table: consider the F distribution (http://socr.umich.edu/Applets/F_Table.html) where the first row and the column is the degree of freedoms and the data in the middle is the 99% quantiles of F(k,l). This is a two dimensional example and the coordinates or combinations of the basic headers give a unique value attached.

l \ k 1 2 3 4 5
1 4052. 4999.5 5403. 5625. 5764.
2 98.50 99.00 99.17 99.25 99.30
3 34.12 30.82 29.46 28.71 28.24
4 21.20 18.00 16.69 15.98 15.52
5 13.27 12.06 11.39 10.97 10.67

3.2) Streams: is also an easy way of data management and it is more visualized with pictures. It is a sequence of data elements made available over time and can be thought as a conveyor belt that allows items to be processed one at a time rather than in large batches. It is a sequence of digitally encoded coherent signals used to transmit or receive information that is in the process of being transmitted. In electronics and computer architecture, it determines for which time which data item is scheduled to enter or leave which port.

From the chart below, we have three apparent observations: (1) pre-dialysis kidney function is associated with patient survival; (2) pre-dialysis kidney function differs by type of dialysis treatment (it is associated with type of dialysis); (3) pre-dialysis kidney function is not in the causal pathway of type of dialysis and survival. And from these three conditions, we can conclude that pre-dialysis kidney function is a potential confounder.

[[File:DataManagementChart1.png]]

An illustrative example of streaming is wind map (http://hint.fm/wind/), which generates a vivid demonstration of the wind speed over the USA with data streamed live from different resources. We can have a clear picture of the wind range countrywide with streaming.

[[File:DataManagementFig1.png]]

3.3) Cloud Data Storage: Data cloud storage demands high availability, durability, and scalability from a few bytes to petabytes. Examples of data cloud storage services include Amazon’s S3, which promises a 99.9% monthly availability and 99.999999999% durability per year. This translates into less than an hour outage per month. For an example of durability, assume that a user stores 10,000 objects in a cloud storage, then, on average, the user would expect to lose one object every 10,000,000 years. S3, and other cloud service providers, achieve this reliability by storing data in multiple facilities with error checking and self-healing processes to detect and repair errors and device failures. This process is completely transparent to the user and requires no actions or knowledge of the underlying complexities. Global data-intense service providers like Google and Facebook have the expertise and scale to provide enormous storage and significant reliability and uptime in an efficient/economical way. Many Big Data research projects benefit from using cloud storage services (e.g., umich email, file-sharing, computational services etc.)

3.4) Warehouse: a system used for reporting and data analysis. Integrating data from one or more disparate sources creates a central repository of data, a data warehouse (DW). DW stores current and historical data and are used for creating trending reports for senior management reporting such as annual comparison.

[[File:DataManagementFig2.jpg]]

3.5) DBs: database is an organized collection of data where data are typically organized to model relevant aspects of reality in a way that supports processes requiring this information.

[[File:DataManagementFig3.png]]

A well-known example would be the pipeline, which utilizes data from the LONI Image Data Archive (IDA). The pipeline takes advantage of the cluster nodes to download files in parallel from the IDA database. (http://pipeline.loni.usc.edu/learn/user-guide/building-a-workflow/#IDA)
This article (https://ida.loni.usc.edu/login.jsp) provides a specific introduction to The LONI Image Data Archive (IDA), which offers an integrated environment for safely archiving, querying, visualizing and sharing neuroimaging data. It facilitates the de-identification and pooling of data from multiple institutions, protecting data from unauthorized access while providing the ability to share data among collaborative investigators. The archive provides flexibility in establishing project metadata, accommodating one or more research groups, sits and others. It’s simple, secure and requires nothing more than a computer with Internet connection and a web browser.

3.6) Arrays: a data structure consisting of a collection of elements, each identified by at least one array index or key.
Types of arrays:
One-dimensional array: a simple example: an array of 8 integer variables with indices 0 through 7 may be sored as 8 words at memory address: {200, 202, 204, 206, 208, 210, 212, 214} which can be memorized as 200+2i.
Multidimensional arrays: Data=[■(2&3&0@6&4&5@5&3&1)].

Example: R arrays and data-frames

DF <- data.frame(a = 1:3, b = letters[10:12],
c = seq(as.Date("2004-01-01"), by = "week", len = 3),
stringsAsFactors = TRUE)
> DF
a b c
1 1 j 2004-01-01
2 2 k 2004-01-08
3 3 l 2004-01-15

data.matrix(DF[1:2])
data.matrix(DF)

> data.matrix(DF)
a b c
[1,] 1 1 12418
[2,] 2 2 12425
[3,] 3 3 12432

> sleep # sleep dataset
extra group ID
1 0.7 1 1
2 -1.6 1 2
3 -0.2 1 3
4 -1.2 1 4
5 -0.1 1 5
6 3.4 1 6
7 3.7 1 7
8 0.8 1 8
9 0.0 1 9
10 2.0 1 10
11 1.9 2 1
12 0.8 2 2
13 1.1 2 3
14 0.1 2 4
15 -0.1 2 5
16 4.4 2 6
17 5.5 2 7
18 1.6 2 8
19 4.6 2 9
20 3.4 2 10

3.7) Binary ASCII: ASCII, which is short for American Standard Code for Information Interchange is a character-encoding scheme originally based on English alphabet that encodes 128 specified characters – the numbers 0 – 9, the letters a – z and A – Z, some basic punctuation symbols, some control codes that originated with teletype machines and a bland space – into the 7-bit binary integers. It represents text in computers, communications equipment, and other devices that use text. To convert the ASCII code to binary character (part):

Letter ASCII Code Binary Letter ASC II Code Binary
a 097 01100001 A 065 01000001
b 098 01100010 B 066 01000010
c 099 01100011 C 067 01000011
d 100 01100100 D 068 01000100
e 101 01100101 E 069 01000101

Dec Hex Binary
0 00 00000000
1 01 00000001
2 02 00000010
3 03 00000011
4 04 00000100
5 05 00000101
6 06 00000110
7 07 00000111
8 08 00001000
9 09 00001001
10 0A 00001010
11 0B 00001011
12 0C 00001100
13 0D 00001101
14 0E 00001110
15 0F 00001111
16 10 00010000

3.8) Handling: the process of ensuring that the research data is stored, archived or disposed off in a safe and secure manner during and after the conclusion of a research project. This includes the development of policies and procedures to manage data handled electronically as well as through non-electronic means.

Data handling is important in ensuring the integrity of research data since it addresses concerns related to confidentially, security, and preservation/retention of research data. Proper planning for data handling can also result in efficient and economical storage, retrieval, and disposal of data. In the case of data handled electronically, data integrity is a primary concern to ensure that recorded data is not altered, erased, lost or accessed by unauthorized users.

Data handling issues encompass both electronic as well as non-electronic systems, such as paper files, journals, and laboratory notebooks. Electronic systems include computer workstations and laptops, personal digital assistants (PDA), storage media such as videotape, diskette, CD, DVD, memory cards, and other electronic instrumentation. These systems may be used for storage, archival, sharing, and disposing off data, and therefore, require adequate planning at the start of a research project so that issues related to data integrity can be analyzed and addressed early on.
Issues needed to be considered in ensuring integrity of data handled:
Type of data handled and its impact on the environment (especially if it is on a toxic media);
Type of media containing data and its storage capacity, handling and storage requirements, reliability, longevity (in the case of degradable medium), retrieval effectiveness, and ease of upgrade to newer media;
Data handling responsibilities/privileges, that is, who can handle which portion of data, at what point during the project, for what purpose, etc;
Data handling procedures that describe how long the data should be kept, and when, how, and who should handle data for storage, sharing, archival, retrieval and disposal purposes.

===Applications===

4.1) This article (http://en.wikibooks.org/wiki/OpenClinica_User_Manual) presents a comprehensive introduction to EDC (electronic data capture) in clinical research. It contains a series of guides to help users learn how to use OpenClinica for clinical data management. This user manual serves a perfect introduction to OpenClinica and equips researches with background information, specific data management procedures, study construction and system maintenance for data management in clinical studies.

4.2) This article (http://pipeline.loni.usc.edu/learn/user-guide/building-a-workflow/#XNAT) talked about issues regarding building the workflow with LONI Pipeline. It introduced dragging and connecting in the modules and how data is managed with pipeline. It also illustrates usage of pipeline with IDA, NDAR, XNAT and the cloud storage it supported.

4.3) This article (http://onlinepubs.trb.org/onlinepubs/nchrp/cd-22/manual/v2chapter2.pdf) presents a general introduction to data management, analysis tools and analysis mechanics. It illustrated the purpose, steps, considerations and useful tools in data management and introduced the specific steps in data handling. It go through the major steps with examples and illustrated the database handling with consideration of the dataset size. This article offers a comprehensive analysis of data management and would be a good start to implement data management.

4.4) This article (http://www.sciencedirect.com/science/article/pii/S0167819102000947) presents two services that they believe are fundamental to any data grid: reliable, high-speed transport and replica management. Their high-speed transport service, GridFTP, extends the popular FTP protocol with new features required for Data Grid applications, such as striping and partial file access and their replica management service integrates a replica catalog with GridFTP transfers to provide for the creation, registration, location, and management of dataset replicas. The article also presents the design of both services and also preliminary performance results and implementations exploit security and other services provided by the Globus Toolkit.

===Software===
Data Import/Export in R: http://cran.r-project.org/doc/manuals/r-devel/R-data.html
Package ‘rmongodb’ in R (provides an interface to the NoSQL MongoDB database): http://cran.r-project.org/web/packages/rmongodb/ .

===Problems===

Example 1: Data Streaming:

> library("stream")
> dsd <- DSD_Gaussians(k=3, d=3, noise=0.05)
> dsd
Static Mixture of Gaussians Data Stream
With 3 clusters in 3 dimensions
> p <- get_points(dsd, n=5)
> p
V1 V2 V3
1 0.6930698 0.4082633 0.3857444
2 0.9086645 0.5545718 0.6942871
3 0.6031121 0.5288573 0.5795389
4 0.7590460 0.5485160 0.6484480
5 0.7249682 0.7327056 0.4891291
> p <- get_points(dsd, n=100, assignment=TRUE)
> attr(p, "assignment")
[1] 3 3 3 NA 2 2 2 3 1 2 1 2 2 3 1 1 1 3 3 2 3 1 1 2
[25] 3 1 3 3 1 1 3 2 3 3 2 NA 1 1 1 1 1 3 2 2 2 3 1 2
[49] 2 2 2 3 1 NA 1 3 2 2 3 3 3 2 1 2 2 2 1 3 1 1 1 2
[73] 1 3 2 2 3 1 NA 1 2 1 1 3 3 1 1 2 1 3 3 3 2 3 3 2
[97] NA 1 2 1
> plot(dsd, n=500)

[[File:DataManagementFig4.png]]

Example 2: Import of big data: zips (the data would be available at http://media.mongodb.org/zips.json) import using R/MongoDB:
Steps. Install the package of ‘rmongodb’ and import data zips, this data is included in the rmongodb package and can be loaded using the command data(zips):

install.package('rmonogodb')
library(rmongodb)
data(zips)
head(zips)

Output:

> install.packages('rmongodb')
trying URL 'http://cran.mtu.edu/bin/macosx/leopard/contrib/2.15/rmongodb_1.6.5.tgz'
Content type 'application/x-gzip' length 1291831 bytes (1.2 Mb)
opened URL
==================================================
downloaded 1.2 Mb

The downloaded binary packages are in
/var/folders/k6/3r5dstw5385_b4fzt679pmsr0000gn/T//RtmpWD2wiV/downloaded_packages

> head(zips)
city loc pop state _id
[1,] "ACMAR" Numeric,2 6055 "AL" "35004"
[2,] "ADAMSVILLE" Numeric,2 10616 "AL" "35005"
[3,] "ADGER" Numeric,2 3205 "AL" "35006"
[4,] "KEYSTONE" Numeric,2 14218 "AL" "35007"
[5,] "NEW SITE" Numeric,2 19942 "AL" "35010"
[6,] "ALPINE" Numeric,2 3062 "AL" "35014"

===References===

Statistical inference / George Casella, Roger L. Berger http://mirlyn.lib.umich.edu/Record/004199238

Sampling / Steven K. Thompson. http://mirlyn.lib.umich.edu/Record/004232056

Sampling theory and methods / S. Sampath. http://mirlyn.lib.umich.edu/Record/004133572

Data Handling http://ori.hhs.gov/education/products/n_illinois_u/datamanagement/dhtopic.html