SMHS AssociationTests
Contents
Scientific Methods for Health Sciences - Association Tests
Overview
Measuring the association between two quantities is one of the most commonly applied tools researchers needed in studies. The term association implies on the possible correlation where two or more variables vary accordingly to some pattern. There are many statistical measures of association including relative ratio, odds ratio and absolute risk reduction. In this section, we are going to introduce measures of association in different studies.
Motivation
In many cases, we need to measure if two quantities are associated with each other -- that is if two or more variables vary together according to some pattern. There are many statistical tools we can apply to measure the association between variables. How can we decide what types of measures we need to use? How do we interpret the test results? What does the test results imply about the association between the variables we studied?
Theory
- Measures of Association:
(1) relative measures: $Relative\, risk=\frac{Cumulative\, incidence\, in\, exposed}{Cumulative\,incidence\, in \,unexposed}=ratio\,of\,risks =Risk\,Ratio;$ $Rate\,Ratio=\frac{Incidence\,rate\,in\,exposed} {Incidence\,rate\,in\,unexposed}$
(2) difference:
$Efficacy=\frac{Cumulative\,incidence\,in\,placebo\,-\,Cumulative\,incidence\,in\,the\,treatement} {Cumulative\,incidence\,rate\,in\,placebo\,group}.$
We are going to interpret the measurement results and conclude about the association between variables through examples in different types of trials.
Chi-square test
The Chi-square test is a non-parametric test of statistical significance of two variables. It tests if the measured factor is associated with the members in one of two samples with chi square. For example, the chi-square test tests whether there is statistical evidence that the measured factor is not randomly distributed in the cases compared to the controls in a case-control study. The test statistic is $X_{o}^{2}=\sum_{i=1}^{n}\frac{(O_i-E_i )^{2}}{E_{i}} \sim χ_{df}^{2}$, where $E_{i}$ is the expected frequency under the null hypothesis and $O_{i}$ is the observed frequency, n is the number of cells in the table and $ df=(\#\,rows-1)(\#\,columns-1)$, $E=\frac{row\,total*column\, total} {grand total}$. The null hypothesis is that there is no association between exposure group the disease studied.
- Conditions for validity of the $χ^{2}$test are:
- Design conditions
- for a good fit, it must be reasonable to regard the data as a random sample of categorical observations from a large population.
- for a contingency table, it must be appropriate to view the data in one of the following ways: as two or more independent random samples, observed with respect to a categorical variable; as one random sample, observed with respect to two categorical variables.
- for either type of test, the observations within a sample must be independent of one another.
- Sample conditions: critical values only work if each expected value > 5
- Design conditions
Example of association: a study on the association of a particular gene and the risk of late onset disease. The data is summarized in the data table below:
| Genotype | Cases | Controls | Total |
| Exposure group | 89 | 51 | 140 |
| Reference group | 119 | 134 | 253 |
| Total | 208 | 185 | 393 |
The Odds Ratio (OR) of a disease (D) in relative to exposure (E) is $OR = \frac{\frac{P \left( D \mid E \right)}{1 - P \left( D \mid E \right)}}{\frac{P \left( D \mid E' \right)}{1 - P \left( D \mid E' \right)}}$, where $E'$ is the complement of exposure (Reference group). Thus, $OR=((89/140)/(1-(89/140))/((119/253)/(1-(119/253)))) = 1.965068$, and the 95% CI(OR) is: $(1.29-3.00)$.
$χ^{2}=\frac{(89-74.10)^{2}}{74.10}$+$\frac{(119-133.90)^{2}}{133.90}$+$\frac{(51-65.90)^{2}}{65.90}$+$\frac{(134-119.10)^{2}}{119.10}=9.89$, $(p=0.0017)$.
Hence, we have significant evidence that we can reject the null hypothesis and conclude that there is statistically significant association between the exposure group and case and control status in the sample. Cases with AD are about 2 times as likely to carry the exposure group genotype as the controls.
Example of non-association: this study examined the same gene and disease and was repeated in a separate data-set with more cases and controls and obtain different results (no evidence of association).
| Genotype | Cases | Controls | Total |
| Exposure group | 209 | 97 | 306 |
| Reference group | 400 | 220 | 620 |
| Total | 609 | 317 | 926 |
$Odds\, ratio=1.19,(95\%\, CI:0.88-1.60)$,$χ^{2}=\frac{(209-201.25)^{2}}{201.25}$ +$\frac{(400-407.75)^{2}} {407.75}$ + $\frac{(97-104.75)^{2}} {104.75}$+$\frac{(220-212.25)^{2}} {212.25}$=$1.30,(p=0.25)$
The case frequencies were the same between the two studies but control frequencies were different. We fail to reject the null hypothesis of no association and failed to find association suggesting that the first study was false positive.
Fisher’s exact test
The Fisher’s exact test is a statistical significance test used in the analysis of contingency tables and is valid for all sample sizes especially when the sample size is small. Suppose we want to study if the proportion of dieting is higher among women than among men and the following data is collected:
| Men | Women | Row Total | |
| Dieting | 1 | 9 | 10 |
| Non-dieting | 11 | 3 | 14 |
| Column Total | 12 | 12 | 14 |
Fisher showed the probability of obtaining any such set of values was given by the hyper-geometric distribution:
$p=\frac{{a+b\choose a}{c+d\choose c}}{n\choose a+c}$ =$\frac{(a+b)!(c+d)!(a+c)!(b+d)!} {a! b! c! d! n!}$=$\frac{{10\choose1} {14\choose11}} {24\choose12}$ = $\frac{10!\,14!\,12!\,12!} {11!\,9!\,11!\,3!\,24!}\sim 0.00135.$
This is the exact hyper-geometric probability of observing this particular arrangement of the data assuming the given marginal totals on the null hypothesis that men and women are equally likely to be dieters. The smaller the p value, the greater the evidence to reject the null hypothesis, so we have significant evidence to reject the null hypothesis and conclude that women and men are not equally likely to be dieters.
See this Fisher's exact test example using the simulated HELP data.
Randomized Controlled Trials (RCT)
In Randomized Controlled Trials, the investigator assigns exposure at random to study participants, investigator then observes if there are differences in health outcomes between people who were (treatment group) and were not (comparison group) exposed to the factor. Special care is taken in ensuring that the follow-up is done in an identical way in both groups. The essence of good comparison between “treatments” is that the compared groups are the same except for the “treatment”.
- Steps of a RCT: hypothesis formed; study participant recruited based on specific criteria and their informed consent is sought; eligible and willing participants randomly allocated to receive assignment to a particular study group; study groups are monitored for outcome under study; rates of outcome in the various groups are compared:
External and internal validity
External validity: Generalization of study to larger source population. Influenced by factors like: demographic differences between eligible and ineligible subgroups; intervention mirror what will happen in the community or source population. Internal validity: Ability to reach correct conclusion in study. Influenced by factors like: ability of subjects to provide valid and reliable data; expected compliance with a regimen; low probability of dropping out.
Measures of Association and Effect in RCT: Ratio of two measures of disease incidence (relative measures) - Risk Ratio (Relative Risk), Rate Ratio. Difference between two measures of disease incidence: Risk difference, efficacy
| Disease Status | |||
| Disease | No Disease | ||
| Treatment | Drug A | a | b |
| Placebo | c | d | |
$Relative\,Risk\,=\frac{Cumulative\,Incidence\,in\,exposed}{Cumulative\,Incidence\,in\, unexposed}=ratio\,of\,risks=Risk\, Ratio=\frac{a/(a+b)}{c/(c+d)}=\frac{CI_{drugA}}{CI_{placebo}}$
$Rate\,Ratio=\frac{Incidence\,rate\,in\,exposed} {Incidence\,rate\,in\,unexposed}$
Interpretation: RR>1, The risk of X is RR times more likely to occur in group A than in group B; RR=1, Null value (no difference between groups); RR<1, Either calculate the reduction in risk ratios (100%-xx%) or invert (1/RR) to be interpreted as “less likely” risk.
$Efficacy=\frac{C.I.\,in\, placebo\,-\,C.I.\,Rate\,in\,the\,treatment} {C.I.\,Rate\,in\,placebo\,group}$
- Situations that favor the use of RCT: (1) Exposure of interest is a modifiable factor over which individuals are willing to relinquish control; (2) Legitimate uncertainty exists regarding the effect of interventions on outcome, but reasons exist to believe that the benefits of the intervention in question overweight the risks; (3) Effect of intervention on outcome is of sufficient importance to justify a large study.
Cohort Study: Population of exposed and unexposed individuals at risk of developing outcomes are followed over time to compare the development of disease in each group.
- Steps: Establish the study population. Identify a study population that is reflective of base population of interest and has a distribution of exposure; identify group of exposed and unexposed individuals. Study on the outcomes of exposed and not exposed groups.
- Types: Prospective (concurrent) and Retrospective Cohort Studies (non-concurrent) based on when is the data collected.
Retrospective has benefits: more cost effective; good for disease of long latency. Prospective has benefits: data quality presumably higher. Both designs need to be cautious of ascertainment biases if outcomes or exposure is known.
- Measures of Association in Cohort Study:
Ratio of two measures of disease incidence (relative measures): Risk Ratio (Relative Risk), Rate Ratio. Difference between two measures of disease incidence: Risk Difference, Rate Difference.
- Strengths and weakness of Cohort Design:
Strengths: (1) Maintain temporal sequence – can estimate incidence of disease; exposure precedes development of disease; also explore time-varying information. (2) Excellent for studying known adverse exposures or those that cannot practically be randomized. (3) Like RCT, excellent for studying rare exposures. (4) Multiple outcomes and sometimes multiple exposures can be studied. Disadvantages: (1) Long-term follow-up required and expensive; (2) Not effective at capturing rare outcomes and can be challenging to study disease that take a long time to develop; (3) Loss to follow-up can be a problem; (4) Changes over time in criteria and methods can lead to problems with inferences; (5) People self-select exposures so exposed and unexposed may differ with respect to important characteristics.
- Situations favor a Cohort Study: (1) When there is evidence of an association between the exposure and the disease from other studies; (2) When the exposure is rare but incidence of disease among the exposure is high; (3) When time between exposure and development of the disease is relatively short or historical data is available; (4) When good follow-up can be ensured.
Case Control Study: compare cases and controls to see which group has greater exposure to the disease.
- Measures of Association: Odds Ratio.
| Case | Control | ||
| Exposed | Yes | a | b |
| No | c | d |
$Odds\,Ratio\,=\frac{odds\,of\,a\,case\,being\,exposed} {odds\,of\,a\,control\,being\,exposed}=\frac{a/c} {b/d}=\frac {ad}{bc}.$
Interpretation: Odds of being exposed is OR times higher (if OR > 1) in the cases than the controls (1/OR times lower (if OR < 1) in the cases than the controls; No association – odds is the same in cases and controls (if OR = 1)).
- Strengths and weakness of Case Control Study:
Strengths: Case Control Study Design is efficient and can evaluate many risk factors for the same disease, so is good for diseases about which little is known; it is observational – we don’t ask people to change their behavior, we just collect information on events that happen “naturally”. Weakness: Inefficient for rare exposures; can study only one outcome at a time; cannot calculate incidence of disease but can only estimate the odds of being exposed in cases vs. controls; the number of cases and controls in study is artificial and does not represent the natural distribution of disease in the population.
- Avoiding Recall / Reporting Bias: adjusting timing so that the time between the event/illness and the study is as short as possible; use standardized questionnaires that obtain complete information; using existing information if/when possible (e.g. medical record); mask participants to study hypothesis.
- Conditions when an OR from a Case-Control Study can approximate a RR OR≈RR:
(1) when the cases are representative, with respect to their exposure status, of all people with the disease in the population from which the cases were drawn; (2) when the controls are representative, with respect to their exposure status, of all people without the disease in the population from which the cases are drawn; (3) when the disease being studied does not occur frequently.
Cross-Sectional Studies: observational study in which a subject’s exposure and disease data are measured at the same time; identify prevalent cases of the disease; exposure prevalence in relation to disease prevalence (no incidence cases; unable to determine temporality).
- Strengths and Limitations of Cross-Sectional Studies:
Strengths: (1) good for generating hypotheses; (2) easily sets up other analytic designs; (3) temporality is not a problem for time invariant exposures (genetic markers); (4) relatively low cost.
- Weakness: (1) temporality – exposure or disease which happened first; (2) prevalent cases may not be the same as incident cases; (3) not useful for rare disease; (4) subject to selection bias.
- Measures of Association in Cross-Sectional Studies:
| Disease Status | |||
| Disease | No Disease | ||
| Exposed | Yes | a | b |
| No | c | d | |
$Prevalence\,Ratio\,=\frac{Prevalence\,of\,disease\,in\,exposed} {Prevalence\,of\, disease\,in\,unexposed}=\frac{a/(a+b)}{c/(c+d)}$
$Odds\,Ratio\,=\frac{Odds\,of\,diseases\,in\,exposed}{Odds\,of\,disease\,in\,unexposed}=\frac{a/b}{c/d}=\frac{ad}{cd}$
Applications
This article presents the analyses example on Fisher’s Exact Test Activity in SOCR.
This article presents the application of Chi-square test in SOCR. It illustrated the chi-square test in SOCR Chi-square contingency-table calculations.
This article presents a meta-analysis on the correlation between the implicit association test and explicit self-report measures. Theoretically, low correlations between implicit and explicit measures can be due to (a) motivational biases in explicit self reports, (b) lack of introspective access to implicitly assessed representations, (c) factors influencing the retrieval of information from memory, (d) method-related characteristics of the two measures, or (e) complete independence of the underlying constructs. The present study addressed these questions from a meta-analytic perspective, investigating the correlation between the Implicit Association Test (IAT) and explicit self-report measures. Based on a sample of 126 studies, the mean effect size was .24, with approximately half of the variability across correlations attributable to moderator variables. Correlations systematically increased as a function of (a) increasing spontaneity of self-reports and (b) increasing conceptual correspondence between measures. These results suggest that implicit and explicit measures are generally related but that higher order inferences and lack of conceptual correspondence can reduce the influence of automatic associations on explicit self-reports.
Software
cor.test(x, y, …) ## test for association/correlation between paired samples
A Handbook of Statistical Analyses Using R
Problems
6.1) Relative risk (i.e. risk ratio) is a ratio of prevalence of disease in the exposed vs. prevalence of disease in the unexposed.
a. True
b. False
6.2) A case-control study was designed to evaluate the association between breast cancer and past use of hormonal replacement therapy (HRT) for at least 2 years consecutively among women aged 60-70 years. Assume the researchers obtained an odds ratio of 2.3 when HRT use was considered the exposure. Which is the best interpretation for this OR?
a. Women aged 60-70 years old with breast cancer were 2.3 times more likely to use HRT than women without breast cancer
b. In the study population women aged 60-70 years old that used HRT were 2.3 times more likely to develop breast cancer than women who did not use HRT.
c. Women aged 50-70 years old that used HRT developed breast cancer 2.3 times faster than women who did not use HRT in our study population.
d. In the study population women aged 60-70 years old with breast cancer were 2.3 times less likely to use HRT than women without breast cancer.
A researcher is interested in looking at the association between Medicaid status and the completion of the childhood vaccination series. She decides to use the Michigan Care Improvement Registry MCIR), an immunization registry containing immunization records for all children and adolescents in the state of Michigan. She takes a representative sample from the MCIR and obtains the following data:
6.3) Which of the following associations can you estimate in this study?
a. Risk ratio
b. Prevalence ratio
c. Rate ratio
d. All of the above
6.4) You are interested in evaluating the outcome of being fully immunized, comparing children on Medicaid to those not on Medicaid. Calculate the odds ratio for this association.
a. 4.75
b. 0.21
c. 2.56
d. 1.19
6.5) Calculate the prevalence ratio of being fully immunized, comparing children on Medicaid to children not on Medicaid
a. 0.95
b. 4.75
c. 1.19
d. 0.80
6.6) Which of the following are measures of association that can be used to compare two different treatments?
a. Percent difference (Efficacy)
b. Prevalence and incidence
c. Relative risk
d. Both a and c
e. Both a and b
6.7) Suppose 200 randomly selected cancer patients were asked if their primary diagnosis was Brain cancer and if they owned a cell phone before their diagnosis. The results are presented in the table below. Suppose the data is recorded as follow:
| Brain Cancer | ||||
| Yes | No | Total | ||
| Cell Phone Use | ||||
| Yes | 18 | 80 | 98 | |
| No | 7 | 95 | 102 | |
| Total | 25 | 175 | 200 | |
Does it seem like there is an association between brain cancer and cell phone use? Use a chi-square test at 5% level of significance. Include all the steps needed (include the hypothesis, test statistic and results).
References
http://mirlyn.lib.umich.edu/Record/004199238
http://mirlyn.lib.umich.edu/Record/004232056
http://mirlyn.lib.umich.edu/Record/004133572
- SOCR Home page: http://www.socr.umich.edu
Translate this page:
