Difference between revisions of "Scientific Methods for Health Sciences"

Latest revision as of 12:34, 2 January 2020

The Scientific Methods for Health Sciences EBook is still under active development. When the EBook is complete this banner will be removed.

SOCR Wiki: Scientific Methods for Health Sciences

Scientific Methods for Health Sciences EBook

Preface

The Scientific Methods for Health Sciences (SMHS) EBook (ISBN: 978-0-9829949-1-7) is designed to support a 4-course training curriculum emphasizing the fundamentals, applications and practice of scientific methods specifically for graduate students in the health sciences.

Format

Follow the instructions in this page to expand, revise or improve the materials in this EBook.

Learning and Instructional Usage

This section describes the means of traversing, searching, discovering and utilizing the SMHS EBook resources in both formal and informal learning setting.

Copyrights

The SMHS EBook is a freely and openly accessible electronic book developed by SOCR and the general health sciences community.

Chapter I: Fundamentals

Exploratory Data Analysis, Plots and Charts

Review of data types, exploratory data analyses and graphical representation of information.

Ubiquitous Variation

There are many ways to quantify variability, which is present in all natural processes.

Parametric Inference

Foundations of parametric (model-based) statistical inference.

Probability Theory

Random variables, stochastic processes, and events are the core concepts necessary to define likelihoods of certain outcomes or results to be observed. We define event manipulations and present the fundamental principles of probability theory including conditional probability, total and Bayesian probability laws, and various combinatorial ideas.

Odds Ratio/Relative Risk

The relative risk, RR, (a measure of dependence comparing two probabilities in terms of their ratio) and the odds ratio, OR, (the fraction of one probability and its complement) are widely applicable in many healthcare studies.

Centrality, Variability and Shape

Three main features of sample data are commonly reported as critical in understanding and interpreting the population, or process, the data represents. These include Center, Spread and Shape. The main measures of centrality are Mean, Median and Mode(s). Common measures of variability include the range, the variance, the standard deviation, and mean absolute deviation. The shape of a (sample or population) distribution is an important characterization of the process and its intrinsic properties.

Probability Distributions

Probability distributions are mathematical models for processes that we observe in nature. Although there are different types of distributions, they have common features and properties that make them useful in various scientific applications. This section presents the Bernoulli, Binomial, Multinomial, Geometric, Hypergeometric, Negative binomial, Negative multinomial distribution, Poisson distribution, and Normal distributions, as well as the concept of moment generating function.

Resampling and Simulation

Resampling is a technique for estimation of sample statistics (e.g., medians, percentiles) by using subsets of available data or by randomly drawing replacement data. Simulation is a computational technique addressing specific imitations of what’s happening in the real world or system over time without awaiting it to happen by chance.

Design of Experiments

Design of experiments (DOE) is a technique for systematic and rigorous problem solving that applies data collection principles to ensure the generation of valid, supportable and reproducible conclusions.

Intro to Epidemiology

Epidemiology is the study of the distribution and determinants of disease frequency in human populations. This section presents the basic epidemiology concepts. More advanced epidemiological methodologies are discussed in the next chapter. This section also presents the Positive and Negative Predictive Values (PPV/NPV).

Experiments vs. Observational Studies

Experimental and observational studies have different characteristics and are useful in complementary investigations of association and causality.

Estimation

Estimation is a method of using sample data to approximate the values of specific population parameters of interest like population mean, variability or 97^th percentile. Estimated parameters are expected to be interpretable, accurate and optimal, in some form.

Hypothesis Testing

Hypothesis testing is a quantitative decision-making technique for examining the characteristics (e.g., centrality, span) of populations or processes based on observed experimental data. In this section we discuss inference about a mean, mean differences (both small and large samples), a proportion or differences of proportions and differences of variances.

Statistical Power, Sensitivity and Specificity

The fundamental concepts of type I (false-positive) and type II (false-negative) errors lead to the important study-specific notions of statistical power, sample size, effect size, sensitivity and specificity.

Data Management

All modern data-driven scientific inquiries demand deep understanding of tabular, ASCII, binary, streaming, and cloud data management, processing and interpretation.

Bias and Precision

Bias and precision are two important and complementary characteristics of estimated parameters that quantify the accuracy and variability of approximated quantities.

Association and Causality

An association is a relationship between two, or more, measured quantities that renders them statistically dependent so that the occurrence of one does affect the probability of the other. A causal relation is a specific type of association between an event (the cause) and a second event (the effect) that is considered to be a consequence of the first event.

Rate-of-change

Rate of change is a technical indicator describing the rate in which one quantity changes in relation to another quantity.

Clinical vs. Statistical Significance

Statistical significance addresses the question of whether or not the results of a statistical test meet an accepted quantitative criterion, whereas clinical significance is answers the question of whether the observed difference between two treatments (e.g., new and old therapy) found in the study large enough to alter the clinical practice.

Correction for Multiple Testing

Multiple testing refers to analytical protocols involving testing of several (typically more then two) hypotheses. Multiple testing studies require correction for type I (false-positive rate), which can be done using Bonferroni's method, Tukey’s procedure, family-wise error rate (FWER), or false discovery rate (FDR).

Chapter II: Applied Inference

Epidemiology

This section expands the Epidemiology Introduction from the previous chapter. Here we will discuss numbers needed to treat and various likelihoods related to genetic association studies, including linkage and association, LOD scores and Hardy-Weinberg equilibrium.

Correlation and Regression (ρ and slope inference, 1-2 samples)

Studies of correlations between two, or more, variables and regression modeling are important in many scientific inquiries. The simplest situation such situation is exploring the association and correlation of bivariate data ($X$ and $Y$).

ROC Curve

The receiver operating characteristic (ROC) curve is a graphical tool for investigating the performance of a binary classifier system as its discrimination threshold varies. We also discuss the concepts of positive and negative predictive values.

ANOVA

Analysis of Variance (ANOVA) is a statistical method fpr examining the differences between group means. ANOVA is a generalization of the t-test for more than 2 groups. It splits the observed variance into components attributed to different sources of variation.

Non-parametric inference

Nonparametric inference involves a class of methods for descriptive and inferential statistics that are not based on parametrized families of probability distributions, which is the basis of the parametric inference we discussed earlier. This section presents the Sign test, Wilcoxon Signed Rank test, Wilcoxon-Mann-Whitney test, the McNemar test, the Kruskal-Wallis test, and the Fligner-Killeen test.

Instrument Performance Evaluation: Cronbach's α

Cronbach’s alpha (α) is a measure of internal consistency used to estimate the reliability of a cumulative psychometric test.

Measurement Reliability and Validity

Measures of Validity include: Construct validity (extent to which the operation actually measures what the theory intends to), Content validity (the extent to which the content of the test matches the content associated with the construct), Criterion validity (the correlation between the test and a variable representative of the construct), experimental validity (validity of design of experimental research studies). Similarly, there many alternate strategies to assess instrument Reliability (or repeatability) -- test-retest reliability, administering different versions of an assessment tool to the same group of individuals, inter-rater reliability, internal consistency reliability.

Survival Analysis

Survival analysis is used for analyzing longitudinal data on the occurrence of events (e.g., death, injury, onset of illness, recovery from illness). In this section we discuss data structure, survival/hazard functions, parametric versus semi-parametric regression techniques and introduction to Kaplan-Meier methods (non-parametric).

Decision Theory

Decision theory helps determining the optimal course of action among a number of alternatives, when consequences cannot be forecasted with certainty. There are different types of loss-functions and decision principles (e.g., frequentist vs. Bayesian).

CLT/LLNs – limiting results and misconceptions

The Law of Large Numbers (LLT) and the Central Limit Theorem (CLT) are the first and second fundamental laws of probability. CLT yields that the arithmetic mean of a sufficiently large number of iterates of independent random variables given certain conditions will be approximately normally distributed. LLT states that in performing the same experiment a large number of times, the average of the results obtained should be close to the expected value and tends to get closer to the expected value with increasing number of trials.

Association Tests

There are alternative methods to measure association two quantities (e.g., relative risk, risk ratio, efficacy, prevalence ratio). This section also includes details on Chi-square tests for association and goodness-of-fit, Fisher’s exact test, randomized controlled trials (RCT), and external and internal validity.

Bayesian Inference

Bayes’ rule connects the theories of conditional and compound probability and provides a way to update probability estimates for a hypothesis as additional evidence is observed.

PCA/ICA/Factor Analysis

Principal component analysis is a mathematical procedure that transforms a number of possibly correlated variables into a fewer number of uncorrelated variables through a process known as orthogonal transformation. Independent component analysis is a computational tool to separate a multivariate signal into additive subcomponents by assuming that the subcomponents are non-Gaussian signals and are statistically independent from each other. Factor analysis is a statistical method, which describes variability among observed correlated variables in terms of potentially lower number of unobserved variables.

Point/Interval Estimation (CI) – MoM, MLE

Estimation of population parameters is critical in many applications. In statistics, estimation is commonly accomplished in terms of point-estimates or interval-estimates for specific (unknown) population parameters of interest. The method of moments (MOM) and maximum likelihood estimation (MLE) techniques are used frequently in practice. In this section, we also lay the foundations for expectation maximization and Gaussian mixture modeling.

Study/Research Critiques

The scientific rigor in published literature, grant proposals and general reports needs to be assessed and scrutinized to minimize errors in data extraction and meta-analysis. Reporting biases present significant obstacles to collecting of relevant information on the effectiveness of an intervention, strength of relations between variables, or causal associations.

Common mistakes and misconceptions in using probability and statistics, identifying potential assumption violations, and avoiding them

Chapter III: Linear Modeling

Multiple Linear Regression (MLR)

Multiple Linear Regression encapsulated a family of statistical analyses for modeling single or multiple independent variables and one dependent variable. MLR computationally estimates all of the effects of each independent variable (coefficients) based on the data using least square fitting.

Generalized Linear Modeling (GLM)

Generalized Linear Modeling (GLM) is a flexible generalization of ordinary linear multivariate regression, which allows for response variables that have error distribution models other than a normal distribution. GLM unifies statistical models like linear regression, logistic regression and Poisson regression.

Analysis of Covariance (ANCOVA)

Analysis of Variance (ANOVA) is a common method applied to analyze the differences between group means. Analysis of Covariance (ANCOVA) is another method applied to blend ANOVA and regression and evaluate whether population means of a dependent variance are equal across levels of a categorical independent variable while statistically controlling for the effects of other continuous variables.

Multivariate Analysis of Variance (MANOVA)

A generalized form of ANOVA is the multivariate analysis of variance (MANOVA), which is a statistical procedure for comparing multivariate means of several groups.

Multivariate Analysis of Covariance (MANCOVA)

Similar to MANOVA, the multivariate analysis of covariance (MANOCVA) is an extension of ANCOVA that is designed for cases where there is more than one dependent variable and when a control of concomitant continuous independent variables is present.

Repeated measures Analysis of Variance (rANOVA)

Repeated measures are used in situations when the same objects/units/entities take part in all conditions of an experiment. Given there is multiple measures on the same subject, we have to control for correlation between multiple measures on the same subject. Repeated measures ANOVA (rANOVA) is the equivalent of the one-way ANOVA, but for related, not independent, groups. It is also referred to as within-subject ANOVA or ANOVA for correlated samples.

Partial Correlation

Partial correlation measures the degree of association between two random variables by measuring variances controlling for certain other factors or variables.

Time Series Analysis

Time series data is a sequence of data points measured at successive points in time. Time series analysis is a technique used in varieties of studies involving temporal measurements and tracking metrics.

Fixed, Randomized and Mixed Effect Models

Fixed effect models are statistical models that represent the observed quantities in terms of explanatory variables (covariates) treated as non-random, while random effect models assume that the dataset being analyzed consist of a hierarchy of different population whose differences relate to that hierarchy. Mixed effect models consist of both fixed effects and random effects. For random effects model and mixed models, either all or part of the explanatory variables are treated as if they rise from random causes.

Hierarchical Linear Models (HLM)

Hierarchical linear model (also called multilevel models) refer to statistical models of parameters that vary at more than one level. These are generalizations of linear models and are widely applied in various studies especially for research designs where data for participants are organized at more than one level.

Multi-Model Inference

Multi-Model Inference involves model selection of a relationship between $Y$ (response) and predictors $X_1, X_2, ..., X_n$ that is simple, effective and retains good predictive power, as measured by the SSE, AIC or BIC.

Mixture Modeling

Mixture modeling is a probabilistic modeling technique for representing the presence of sub-populations within overall population, without requiring that an observed data set identifies the sub-population to which an individual observation belongs.

Surveys

Survey methodologies involve data collection using questionnaires designed to improve the number of responses and the reliability of the responses in the surveys. The ultimate goal is to make statistical inferences about the population, which would depend strongly on the survey questions provided. The commonly used survey methods include polls, public health surveys, market research surveys, censuses and so on.

Longitudinal Data

Longitudinal data represent data collected from a population over a given time period where the same subjects are measured at multiple points in time. Longitudinal data analyses are widely used statistical techniques in many health science fields.

Generalized Estimating Equations (GEE) Models

Generalized estimating equation (GEE) is a method for parameter estimation when fitting generalized linear models with a possible unknown correlation between outcomes. It provides a general approach for analyzing discrete and continuous responses with marginal models and works as a popular alternative to maximum likelihood estimation (MLE).

Model Fitting and Model Quality (KS-test)

The Kolmogorov-Smirnov Test (K-S test) is a nonparametric test commonly applied to test for the equality of continuous, one-dimensional probability distributions. This test can be used to compare one sample against a reference probability distribution (one-sample K-S test) or to compare two samples (two-sample K-S test).

Chapter IV: Special Topics

Data Simulation

This section demonstrates the core principles of simulating multivariate datasets.

Linear Modeling

This section is a review of linear modeling.

Scientific Visualization

This section discusses how and why we should "look" at data.

Methods for Studying Heterogeneity of Treatment Effects, Case-Studies of Comparative Effectiveness Research

This section discusses methods for studying heterogeneity of treatment effects and case-studies of comparative effectiveness research.

Big-Data/Big-Science

This section discusses structural equation modeling and generalized estimated equation modeling. Furthermore, it discusses statistical validation, cross validation, classification, and prediction.

Missing data

Many research studies encounter incomplete (missing) data that require special handling (e.g., teleprocessing, statistical analysis, visualization). There are a variety of methods (e.g., multiple imputation) to deal with missing data, detect missingness, impute the data, analyze the completed dataset and compare the characteristics of the raw and imputed data.

Genotype-Environment-Phenotype associations

Medical imaging

Data Networks

Adaptive Clinical Trials

Databases/registries

Meta-analyses

Causality/Causal Inference, SEM

Classification methods

Time-Series Analysis

In this section, we will discuss Time Series Analysis, which represents a class of statistical methods applicable for series data aiming to extract meaningful information, trend and characterization of the process using observed longitudinal data.

Scientific Validation

Geographic Information Systems (GIS)

Rasch measurement model/analysis

MCMC sampling for Bayesian inference

Network Analysis

References

Translate this page:

(default)	Deutsch	Español	Français	Italiano	Português	日本語	България	الامارات العربية المتحدة	Suomi	इस भाषा में	Norge
한국어	中文	繁体中文	Русский	Nederlands	Ελληνικά	Hrvatska	Česká republika	Danmark	Polska	România	Sverige

@@ Line 1: / Line 1: @@
-'''The Scientific Methods for Health Sciences EBook is still under active development. It is expected to be complete by Sept 01, 2014, when this banner will be removed.'''
+{| cellspacing="5" cellpadding="0" style="margin:0em 0em 1em 0em; border:1px solid #1DA0E7; background:#B3DDF4;width:100%"
+| '''The Scientific Methods for Health Sciences EBook is still under active development. When the EBook is complete this banner will be removed.'''
+|}
 == [[Main_Page | SOCR Wiki]]: Scientific Methods for Health Sciences ==
 [[Image:SMHS_EBook.png|250px|thumbnail|right| Scientific Methods for Health Sciences EBook]]
-Electronic book (EBook) on Scientific Methods for Health Sciences (coming up ...)
 ==[[SMHS_Preface| Preface]]==
-The ''Scientific Methods for Health Sciences (SMHS) EBook'' is designed to support a 4-course training of scientific methods for graduate students in the health sciences.
+The ''Scientific Methods for Health Sciences (SMHS) EBook'' (ISBN: 978-0-9829949-1-7) is designed to support a [http://www.socr.umich.edu/people/dinov/SMHS_Courses.html 4-course training curriculum] emphasizing the fundamentals, applications and practice of scientific methods specifically for graduate students in the health sciences.
 ===[[SMHS_Format| Format]]===
@@ Line 17: / Line 16: @@
 ===[[SMHS_copyright | Copyrights]]===
-The SMHS EBook is a freely and openly accessible electronic book developed by SOCR and the general community.
+The SMHS EBook is a freely and openly accessible electronic book developed by [[SOCR]] and the general health sciences community.
 ==Chapter I: Fundamentals==
@@ Line 34: / Line 33: @@
 ===[[SMHS_OR_RR | Odds Ratio/Relative Risk]]===
 The relative risk, RR, (a measure of dependence comparing two probabilities in terms of their ratio) and the odds ratio, OR, (the fraction of one probability and its complement) are widely applicable in many healthcare studies.
+===[[SMHS_CenterSpreadShape | Centrality, Variability and Shape]]===
+Three main features of sample data are commonly reported as critical in understanding and interpreting the population, or process, the data represents. These include Center, Spread and Shape. The main measures of centrality are Mean, Median and Mode(s). Common measures of variability include the range, the variance, the standard deviation, and mean absolute deviation. The shape of a (sample or population) distribution is an important characterization of the process and its intrinsic properties.
 ===[[SMHS_ProbabilityDistributions | Probability Distributions]]===
-Probability distributions are mathematical models for processes that we observe in nature. Although there are different types of distributions, they have common features and properties that make them useful in various scientific applications.
+Probability distributions are mathematical models for processes that we observe in nature. Although there are different types of distributions, they have common features and properties that make them useful in various scientific applications. This section presents the Bernoulli, Binomial, Multinomial, Geometric, Hypergeometric, Negative binomial, Negative multinomial distribution, Poisson distribution, and Normal distributions, as well as the concept of moment generating function.
 ===[[SMHS_ResamplingSimulation | Resampling and Simulation]]===
@@ Line 45: / Line 47: @@
 ===[[SMHS_IntroEpi | Intro to Epidemiology]]===
-Epidemiology is the study of the distribution and determinants of disease frequency in human populations. This section presents the basic epidemiology concepts. More advanced epidemiological methodologies are discussed in [[SMHS_Epidemiology|the next chapter]].
+Epidemiology is the study of the distribution and determinants of disease frequency in human populations. This section presents the basic epidemiology concepts. More advanced epidemiological methodologies are discussed in [[SMHS_Epidemiology|the next chapter]]. This section also presents the Positive and Negative Predictive Values (PPV/NPV).
 ===[[SMHS_ExpObsStudies | Experiments vs. Observational Studies]]===
@@ Line 54: / Line 56: @@
 ===[[SMHS_HypothesisTesting | Hypothesis Testing]]===
-Hypothesis testing is a quantitative decision-making technique for examining the characteristics (e.g., centrality, span) of populations or processes based on observed experimental data.
+Hypothesis testing is a quantitative decision-making technique for examining the characteristics (e.g., centrality, span) of populations or processes based on observed experimental data. In this section we discuss inference about a mean, mean differences (both small and large samples), a proportion or differences of proportions and differences of variances.
 ===[[SMHS_PowerSensitivitySpecificity | Statistical Power, Sensitivity and Specificity]]===
@@ Line 73: / Line 75: @@
 ===[[SMHS_ClinicalStatSignificance | Clinical vs. Statistical Significance]]===
 Statistical significance addresses the question of whether or not the results of a statistical test meet an accepted quantitative criterion, whereas clinical significance is answers the question of whether the observed difference between two treatments (e.g., new and old therapy) found in the study large enough to alter the clinical practice.
-IV. HS 850: Fundamentals
-Multiple Testing Inference
-) Overview: Multiple testing refers to the cases where testing of several hypotheses are involved simultaneously. This is very common in empirical researches and additional methods besides the traditional rules needs to be applied in multiple testing in order to adjust for the multiple testing problems. In this lecture, we are going to introduce to the area of multiple testing: we will introduce to the basic concepts we are going to use in multiple testing and discuss about the general problems with multiple testing and ways to deal with the multiple testing problems efficiently including with Bonferroni, Tukey’s procedure, Family-Wise Error Rate (FWER), and FDR (false discovery rate).
-) Motivation: We have learned how to do the hypothesis testing with statistical tests. However, the multiple testing problems will occur when one considers a set of statistical inference simultaneously, or infers a subset of parameters selected based on the observed values. So what can we do to adjust for multiple testing? How can we keep the prescribed family wise error rate of α in an analysis involving more than one comparison? Apparently, the error rate for each comparison, it must be more stringent than α. Multiple testing correction would be the way to go and we are going to introduce some commonly used methods for adjusting for this type of error in multiple testing.
-) Theory
-.1) Family-Wise Error Rate (FWER): the probability of making the type I error among all the hypotheses when performing multiple hypothesis tests. FWER exerts a more stringent control over false discovery compared to false discovery rate controlling procedure. Suppose we did simultaneous tests on m hypotheses denoted by H_1,H_2,…,H_m with corresponding p-values p_1,p_2,…,p_m. Let I_0 be the subset of the true null hypotheses with m_0. Our aim is to achieve an overall type I error rate of α from this multiple testing.  FWER=Pr⁡(V≥1)=1-Pr⁡(V=0). By assuming FWER≤α, the probability of making even one type I error in the family is controlled at level α.
-	Null hypothesis is True	Alternative hypothesis is True	Total
-Declared significant	V (number of false positives)	S(number of true positives)	R
-Declared non-significant	U(number of true negatives)	T(number of false negatives)	m-R
-Total	m_0(number of true null hypotheses)	m-m_0(number of true alternatvies)	m
-	A procedure controls the FWER in the weak sense if the FWER control at level α is guaranteed only when all hypotheses are true.
-	A procedure controls the FWER in the strong sense if the FWER control at level α is guaranteed for any configuration of true and non-true null hypotheses.
-Controlling FWER:
-	Bonferrroni: states that rejecting all p_i≤α/m will control that FWER≤α which is proved through Boole’s Inequality: FWER=Pr⁡{⋃_(I_0)▒〖(p_i≤α/m)}〗 ∑_(I_0)▒Pr⁡(p_i≤α/m) }≤m_0  α/m≤m α/m=α. This is the simplest and most conservative method to control FWER though it can be somewhat conservative if there are a large number of tests and/or the test statistics are positively correlated. It controls the probability of false positives only.
-	Tukey’s procedure: only applicable for pairwise comparisons. It assumes independence of the observations being tested as well as equal variation across observations. The procedure calculates for each pair the standardized range statistics: (Y_A-Y_B)/SE where Y_A is the larger of the two means being compared and Y_B is the smaller one and SE is the standard error of the data.
-	The S ̌Ida ́k procedure: works for independent tests where each hypothesis test has α_SID=1-(1-α)^(1/m). This is a more powerful method than Bonferroni but the gain is small.
-	Holm’s step-down procedure: starts by ordering the p values from lowest to highest as  p_((1)),p_((2)),…,p_((m) ) with corresponding hypotheses  H_((1)),H_((2)),…,H_((m) ). Suppose R is the smallest k such that p_((k))>α/(m+1-k). Reject the null hypotheses H_((1)),H_((2)),…,H_((R-1) ). If R=1 then none of the hypotheses are rejected. This method is uniformly better than Bonferroni’s and it is based on Bonferroni with no restriction on the joint distribution of the test statistics.
-	Hochberg’s step-up procedure: starts by ordering the p values from lowest to highest as  p_((1)),p_((2)),…,p_((m) ) with corresponding hypotheses  H_((1)),H_((2)),…,H_((m) ). For a given α, let R be the largest k such that p_((k))≤α/(m+1-k). Reject the null hypotheses H_((1)),H_((2)),…,H_((R) ). It is more powerful than Holm’s, however, it is based on the Simes test so it holds only under independence (and also under some form of positive dependence).
-.4) FDR (false discovery rate): a statistical method used in multiple hypothesis testing to adjust for multiple comparisons. It is designed to control the expected proportion of incorrectly rejected null hypotheses. Compared to FWER, it exerts a less stringent control over false discovery and seeks to reduce the probability of even one false discovery as opposed to the expected proportion of false discoveries and enjoys greater power at the cost increased rate of type I errors.
-	Null hypothesis is True	Alternative hypothesis is True	Total
-Declared significant	V (number of false positives)	S(number of true positives)	R
-Declared non-significant	U(number of true negatives)	T(number of false negatives)	m-R
-Total	m_0(number of true null hypotheses)	m-m_0(number of true alternatives)	m
-Define Q as the proportion of false discoveries among the discoveries (Q=V/R) then FDR is defined as FDR=Q_e=E[Q]=E[V/(V+S)]=E[V/R],where V/R is defined to be 0 when R=0. Our aim is to keep FDR below the threshold α (or q). And q-value is defined as FDR analogue of the p-value, the q-value of individual hypothesis test is the minimum FDR at which the test may be called significant.
-Controlling procedures of FDR:
-With m null hypotheses H_1,H_2,…,H_m and p_1,p_2,…,p_m as their corresponding p-values. We order these p-values in increasing order and denote as p_((1)),p_((2)),…,p_((m) ).
-	Benjamini-Hochberg procedure: controls the false discovery (at least α). For a given α, find the largest k such that p_((k))≤k/m α; then reject all H_((i)) for i=1,…,k. This method works when the m tests are independent as well as with some cases of dependence. E(Q)≤m_0/m α≤α.
-	Benjamini-Hochberg-Yekutieli procedure: controls the FDR under positive dependence assumptions. It modifies the BH procedure: p_((k))≤k/(m*c(m)) α, if the tests are independent or positively correlated we choose c(m)=1 and choose c(m)=∑_(i=1)^m▒1/i with arbitrary dependent tests, when the tests are negatively correlated, c(m) can be approximated with ∑_(i=1)^m▒1/i≈ln⁡(m)+γ.
-Example: Suppose we have computed a vector of p-values (p_1,p_2…,p_n). Let’s compare the corrections using different strategies:
-# Given a set of p-values, returns p-values adjusted using one of several methods.
-# c("holm", "hochberg", "hommel", "bonferroni", "fdr", "BY",
-#   "fdr", "none")
-> p.adjust(c(0.05,0.05,0.1),"bonferroni")
-[1] 0.15 0.15 0.30
-> p.adjust(c(0.05,0.05,0.1),"fdr")
-[1] 0.075 0.075 0.100
-> p.adjust(c(0.05,0.05,0.1),"fdr")
-[1] 0.075 0.075 0.100
-> p.adjust(c(0.05,0.05,0.1),"holm")
-[1] 0.15 0.15 0.15
-) Applications
-.1) This article (http://wiki.stat.ucla.edu/socr/index.php/SOCR_EduMaterials_AnalysesCommandLineFDR_Correction) presents information on how to use the SOCR analyses library for the purpose of computing the False Discovery Rate (FDR) correction for multiple testing in volumetric and shape-based analyses. It provides the specific procedure to compute FDR using SOCR in multiple testing and illustrates with examples and supplementary information about FDR.
-.2) This article (http://home.uchicago.edu/amshaikh/webfiles/palgrave.pdf)  is a comprehensive introduction to multiple testing. It describes the problem of multiple testing more formally and discusses methods, which account for the multiplicity issue. In particular, the recent developments based on resampling results in an improved ability to reject false hypotheses compared to classical methods such as Bonferroni.
-) Software
-http://bioinformatics.oxfordjournals.org/content/21/12/2921.full
-http://socr.ucla.edu/htmls/SOCR_Analyses.html
-http://graphpad.com/quickcalcs/PValue1.cfm
-http://wiki.stat.ucla.edu/socr/index.php/SOCR_EduMaterials_AnalysesCommandLineFDR_Correction
-) Problems
-.1) Suppose the research is conducted to test a new drug and there are 10 hypotheses being tested simultaneously. Calculate the significance level of each individual test using Bonferroni correction if we want to maintain an overall type I error of 5% and the probability of observing at least one significant result when using the correction you chose?
-.2) Consider we are working with a study on test of a new drug for cancer where we have three treatments: the new medicine, the old medicine and the combination of the two. We are doing a pairwise test on these three treatments and want to maintain a Type I error rate of 5%. Consider the Tukey’s correction and describe how you are going to apply this method here.
-) References
-http://mirlyn.lib.umich.edu/Record/004199238
-http://mirlyn.lib.umich.edu/Record/004232056
-http://mirlyn.lib.umich.edu/Record/004133572
 ===[[SMHS_CorrectionMultipleTesting | Correction for Multiple Testing]]===
 Multiple testing refers to analytical protocols involving testing of several (typically more then two) hypotheses. Multiple testing studies require correction for type I (false-positive rate), which can be done using Bonferroni's method, Tukey’s procedure, family-wise error rate (FWER), or false discovery rate (FDR).
 ==Chapter II: Applied Inference==
 ===[[SMHS_Epidemiology| Epidemiology]]===
+This section expands the [[SMHS_IntroEpi|Epidemiology Introduction]] from the previous chapter. Here we will discuss numbers needed to treat and various likelihoods related to genetic association studies, including linkage and association, LOD scores and Hardy-Weinberg equilibrium.
 ===[[SMHS_SLR| Correlation and Regression (ρ and slope inference, 1-2 samples)]]===
+Studies of correlations between two, or more, variables and regression modeling are important in many scientific inquiries. The simplest situation such situation is exploring the association and correlation of bivariate data ($X$ and $Y$).
 ===[[SMHS_ROC| ROC Curve]]===
+The receiver operating characteristic (ROC) curve is a graphical tool for investigating the performance of a binary classifier system as its discrimination threshold varies. We also discuss the concepts of positive and negative predictive values.
 ===[[SMHS_ANOVA| ANOVA]]===
+Analysis of Variance (ANOVA) is a statistical method fpr examining the differences between group means. ANOVA is a generalization of the [[SMHS_HypothesisTesting|t-test]] for more than 2 groups. It splits the observed variance into components attributed to different sources of variation.
 ===[[SMHS_NonParamInference| Non-parametric inference]]===
+Nonparametric inference involves a class of methods for descriptive and inferential statistics that are not based on parametrized families of probability distributions, which is the basis of the [[SMHS_ParamInference|parametric inference we discussed earlier]]. This section presents the Sign test, Wilcoxon Signed Rank test, Wilcoxon-Mann-Whitney test, the McNemar test, the Kruskal-Wallis test, and the Fligner-Killeen test.
 ===[[SMHS_Cronbachs| Instrument Performance Evaluation: Cronbach's α]]===
+Cronbach’s alpha (α) is a measure of internal consistency used to estimate the reliability of a cumulative psychometric test.
 ===[[SMHS_ReliabilityValidity| Measurement Reliability and Validity]]===
+Measures of Validity include: Construct validity (extent to which the operation actually measures what the theory intends to), Content validity (the extent to which the content of the test matches the content associated with the construct), Criterion validity (the correlation between the test and a variable representative of the construct), experimental validity (validity of design of experimental research studies). Similarly, there many alternate strategies to assess instrument Reliability (or repeatability) -- test-retest reliability, administering different versions of an assessment tool to the same group of individuals, inter-rater reliability, internal consistency reliability.
 ===[[SMHS_SurvivalAnalysis| Survival Analysis]]===
+Survival analysis is used for analyzing longitudinal data on the occurrence of events (e.g., death, injury, onset of illness, recovery from illness). In this section we discuss data structure, survival/hazard functions, parametric versus semi-parametric regression techniques and introduction to Kaplan-Meier methods (non-parametric).
 ===[[SMHS_DecisionTheory| Decision Theory]]===
+Decision theory helps determining the optimal course of action among a number of alternatives, when consequences cannot be forecasted with certainty. There are different types of loss-functions and decision principles (e.g., frequentist vs. Bayesian).
 ===[[SMHS_CLT_LLN| CLT/LLNs – limiting results and misconceptions]]===
+The Law of Large Numbers (LLT) and the Central Limit Theorem (CLT) are the first and second fundamental laws of probability. CLT yields that the arithmetic mean of a sufficiently large number of iterates of independent random variables given certain conditions will be approximately normally distributed. LLT states that in performing the same experiment a large number of times, the average of the results obtained should be close to the expected value and tends to get closer to the expected value with increasing number of trials.
 ===[[SMHS_AssociationTests| Association Tests]]===
+There are alternative methods to measure association two quantities (e.g., relative risk, risk ratio, efficacy, prevalence ratio). This section also includes details on Chi-square tests for association and goodness-of-fit, Fisher’s exact test, randomized controlled trials (RCT), and external and internal validity.
 ===[[SMHS_BayesianInference| Bayesian Inference]]===
+Bayes’ rule connects the theories of conditional and compound probability and provides a way to update probability estimates for a hypothesis as additional evidence is observed.
 ===[[SMHS_PCA_ICA_FA| PCA/ICA/Factor Analysis]]===
+Principal component analysis is a mathematical procedure that transforms a number of possibly correlated variables into a fewer number of uncorrelated variables through a process known as orthogonal transformation. Independent component analysis is a computational tool to separate a multivariate signal into additive subcomponents by assuming that the subcomponents are non-Gaussian signals and are statistically independent from each other. Factor analysis is a statistical method, which describes variability among observed correlated variables in terms of potentially lower number of unobserved variables.
 ===[[SMHS_CIs| Point/Interval Estimation (CI) – MoM, MLE]]===
+Estimation of population parameters is critical in many applications. In statistics, estimation is commonly accomplished in terms of point-estimates or interval-estimates for specific (unknown) population parameters of interest. The method of moments (MOM) and maximum likelihood estimation (MLE) techniques are used frequently in practice. In this section, we also lay the foundations for expectation maximization and Gaussian mixture modeling.
 ===[[SMHS_ResearchCritiques| Study/Research Critiques]]===
+The scientific rigor in published literature, grant proposals and general reports needs to be assessed and scrutinized to minimize errors in data extraction and meta-analysis. Reporting biases present significant obstacles to collecting of relevant information on the effectiveness of an intervention, strength of relations between variables, or causal associations.
 ===[[SMHS_CommonMistakesMisconceptions| Common mistakes and misconceptions in using probability and statistics, identifying potential assumption violations, and avoiding them]]===
@@ Line 196: / Line 129: @@
 ==Chapter III: Linear Modeling==
 ===[[SMHS_MLR | Multiple Linear Regression (MLR)]]===
+Multiple Linear Regression encapsulated a family of statistical analyses for modeling single or multiple independent variables and one dependent variable. MLR computationally estimates all of the effects of each independent variable (coefficients) based on the data using least square fitting.
 ===[[SMHS_GLM| Generalized Linear Modeling (GLM)]]===
+Generalized Linear Modeling (GLM) is a flexible generalization of ordinary linear multivariate regression, which allows for response variables that have error distribution models other than a normal distribution. GLM unifies statistical models like linear regression, logistic regression and Poisson regression.
 ===[[SMHS_ANCOVA| Analysis of Covariance (ANCOVA)]]===
-First, see the [[SMHS_ANOVA|ANOVA]] section above.
+Analysis of Variance ([[SMHS_ANOVA|ANOVA]]) is a common method applied to analyze the differences between group means. Analysis of Covariance (ANCOVA) is another method applied to blend ANOVA and regression and evaluate whether population means of a dependent variance are equal across levels of a categorical independent variable while statistically controlling for the effects of other continuous variables.
 ===[[SMHS_MANOVA| Multivariate Analysis of Variance (MANOVA)]]===
+A generalized form of [[SMHS_ANOVA|ANOVA]] is the multivariate analysis of variance (MANOVA), which is a statistical procedure for comparing multivariate means of several groups.
 ===[[SMHS_MANCOVA| Multivariate Analysis of Covariance (MANCOVA)]]===
+Similar to [[SMHS_MANOVA|MANOVA]], the multivariate analysis of covariance (MANOCVA) is an extension of [[SMHS_ANCOVA|ANCOVA]] that is designed for cases where there is more than one dependent variable and when a control of concomitant continuous independent variables is present.
 ===[[SMHS_rANOVA| Repeated measures Analysis of Variance (rANOVA)]]===
+Repeated measures are used in situations when the same objects/units/entities take part in all conditions of an experiment. Given there is multiple measures on the same subject, we have to control for correlation between multiple measures on the same subject. Repeated measures ANOVA (rANOVA) is the equivalent of the one-way [[SMHS_ANOVA|ANOVA]], but for related, not independent, groups. It is also referred to as within-subject ANOVA or ANOVA for correlated samples.
 ===[[SMHS_PartialCorrelation| Partial Correlation]]===
+Partial correlation measures the degree of association between two random variables by measuring variances controlling for certain other factors or variables.
-===[[SMHS_TimeSeriese| Time Series Analysis]]===
+===[[SMHS_TimeSeries| Time Series Analysis]]===
+Time series data is a sequence of data points measured at successive points in time. Time series analysis is a technique used in varieties of studies involving temporal measurements and tracking metrics.
 ===[[SMHS_FixedRandomMixedModels|Fixed, Randomized and Mixed Effect Models]]===
+Fixed effect models are statistical models that represent the observed quantities in terms of explanatory variables (covariates) treated as non-random, while random effect models assume that the dataset being analyzed consist of a hierarchy of different population whose differences relate to that hierarchy. Mixed effect models consist of both fixed effects and random effects. For random effects model and mixed models, either all or part of the explanatory variables are treated as if they rise from random causes.
 ===[[SMHS_HLM| Hierarchical Linear Models (HLM)]]===
+Hierarchical linear model (also called multilevel models) refer to statistical models of parameters that vary at more than one level. These are generalizations of linear models and are widely applied in various studies especially for research designs where data for participants are organized at more than one level.
 ===[[SMHS_MultimodelInference|Multi-Model Inference]]===
+Multi-Model Inference involves model selection of a relationship between $Y$ (response) and predictors $X_1, X_2, ..., X_n$ that is simple, effective and retains good predictive power, as measured by the SSE, AIC or BIC.
 ===[[SMHS_MixtureModeling|Mixture Modeling]]===
+Mixture modeling is a probabilistic modeling technique for representing the presence of sub-populations within overall population, without requiring that an observed data set identifies the sub-population to which an individual observation belongs.
 ===[[SMHS_Surveys|Surveys]]===
+Survey methodologies involve data collection using questionnaires designed to improve the number of responses and the reliability of the responses in the surveys. The ultimate goal is to make statistical inferences about the population, which would depend strongly on the survey questions provided. The commonly used survey methods include polls, public health surveys, market research surveys, censuses and so on.
 ===[[SMHS_LongitudinalData|Longitudinal Data]]===
+Longitudinal data represent data collected from a population over a given time period where the same subjects are measured at multiple points in time. Longitudinal data analyses are widely used statistical techniques in many health science fields.
 ===[[SMHS_GEE| Generalized Estimating Equations (GEE) Models]]===
+Generalized estimating equation (GEE) is a method for parameter estimation when fitting [[SMHS_GLM|generalized linear models]] with a possible unknown correlation between outcomes. It provides a general approach for analyzing discrete and continuous responses with marginal models and works as a popular alternative to maximum likelihood estimation (MLE).
 ===[[SMHS_ModelFitting| Model Fitting and Model Quality (KS-test)]]===
+The Kolmogorov-Smirnov Test (K-S test) is a nonparametric test commonly applied to test for the equality of continuous, one-dimensional probability distributions. This test can be used to compare one sample against a reference probability distribution (one-sample K-S test) or to compare two samples (two-sample K-S test).
 ==Chapter IV: Special Topics==
-===Scientific Visualization===
+===[[SMHS_DataSimulation| Data Simulation ]]===
-===PCOR/CER methods Heterogeneity of Treatment Effects===
+This section demonstrates the core principles of simulating multivariate datasets.
-===Big-Data/Big-Science===
+===[[SMHS_LinearModeling| Linear Modeling ]]===
-===Missing data===
+This section is a review of linear modeling.
+===[[SMHS_SciVisualization| Scientific Visualization ]]===
+This section discusses how and why we should "look" at data.
+===[[SMHS_MethodsHeterogeneity| Methods for Studying Heterogeneity of Treatment Effects, Case-Studies of Comparative Effectiveness Research ]]===
+This section discusses methods for studying heterogeneity of treatment effects and case-studies of comparative effectiveness research.
+===[[SMHS_BigDataBigSci| Big-Data/Big-Science ]]===
+This section discusses structural equation modeling and generalized estimated equation modeling. Furthermore, it discusses statistical validation, cross validation, classification, and prediction.
+===[[SMHS_MissingData|Missing data]]===
+Many research studies encounter incomplete (missing) data that require special handling (e.g., teleprocessing, statistical analysis, visualization). There are a variety of methods (e.g., multiple imputation) to deal with missing data, detect missingness, impute the data, analyze the completed dataset and compare the characteristics of the raw and imputed data.
 ===Genotype-Environment-Phenotype associations===
 ===Medical imaging===
@@ Line 241: / Line 204: @@
 ===Causality/Causal Inference, SEM===
 ===Classification methods===
-===Time-series analysis===
+===[[SMHS_TimeSeriesAnalysis|Time-Series Analysis]]===
+In this section, we will discuss Time Series Analysis, which represents a class of statistical methods applicable for series data aiming to extract meaningful information, trend and characterization of the process using observed longitudinal data.
 ===Scientific Validation===
 ===Geographic Information Systems (GIS)===
@@ Line 248: / Line 214: @@
 ===Network Analysis===
+<hr>
+==References==
-<hr>
 * SOCR Home page: http://www.socr.umich.edu
+* [http://www.socr.umich.edu/people/dinov/SMHS_Courses.html Scientific Methods for Health Sciences (SMHS) Course Series]
+* [http://predictive.space/ Data Science and Predictive Analytics (DSPA)]
+* Dinov, ID. (2018) [http://www.springer.com/us/book/9783319723464 Data Science and Predictive Analytics: Biomedical and Health Applications using R, Springer (ISBN 978-3-319-72346-4)]
 {{translate|pageName=http://wiki.socr.umich.edu/index.php?title=Scientific_Methods_for_Health_Sciences}}