SMHS ProbabilityDistributions

Scientific Methods for Health Sciences - Probability Distributions

Overview

Distributions are the fundamental basis of probability theory. There are two types of processes that we observe in nature – discrete and continuous distributions (there could also be mixtures, multidimensional or tensor distributions which are not discussed here). The type of distribution depends on the type of data. Discrete and continuous distributions represent discrete or continuous random variables, respectively. This section aims to introduce various kinds of discrete and continuous distributions and the relationships between distributions.

• Discrete distribution: [[AP_Statistics_Curriculum_2007_Distrib_Binomial|Bernoulli distribution, Binomial distribution, Multinomial distribution, Geometric distribution, Hypergeometric distribution, Negative binomial distribution, Negative multinomial distribution, Poisson distribution.

• Continuous distribution: Normal distribution, Multivariate normal distribution.

Motivation

We have talked about different types of data and the fundamentals of probability theory. In order to capture and estimate the patterns of data, we introduced the concept of distribution. A probability distribution assigns a probability to each measurable subset of the possible outcomes of a random experiment. It can either be univariate or multivariate. A univariate distribution gives the probability of a single random variable while the a multivariate distribution (a joint probability distribution) gives the probability of a random vector which is a set of two or more random variables taking on various combinations of values. Consider the coin tossing experiment, what would be the distribution of the outcome?

Theory

Random variables: a random variable is a function or a mapping from a sample space into the real numbers (most of the time). In other words, a random variable assigns real values to outcomes of experiments.

Probability density / mass and (cumulative) distribution functions The probability density or probability mass function (pdf), for a continuous or discrete random variable, is the function defined by the probability of the subset of the sample space $\{s\in S\}\subset S$. $p(x)=P(\{s\in S\} | X(s)=x)$, for all $x$. The cumulative distribution function (cdf) $F(x)$ of any random variable $X$ with probability mass or density function $p(x)$ is defined by the total probability of all $\{s\in S\}\subset S$, where $X(s) \leq x; F(x)=P(X\leq x)$, for all x.

Expectation and variance

• Expectation: The expected value, expectation or mean, of a discrete random variable $X$ is defined as $E[X]=\sum_i {x_i P(X=x_i)}$. The expected value of a continuous random variable $Y$ is defined as $E[Y]=\int_y{yP(y)dy}$, which is the integral over the domain of $Y$ and $P(y)$ is the probability density function of $Y$. An important property of expectation is that it is a linear functional, i.e., $E[aX+bY]=aE[X]+bE[Y]$.

• Variance: The variance of a discrete random variable $X$ is defined as $VAR[X]=\\sum_i {(x_i-E[X])^2 P(X=x_i)}$. Variance of a continuous random variable $Y$ is defined as $VAR[Y]=\int_y {(y-E[Y])^2 P(y)dy}$, which is the integral over the domain of $Y$ and $P(y)$ is the probability density function of $Y$. The second moment, variance, does not quite have the same linear functional properties as the expectation: $VAR[aX]= a^2 VAR[X]$ and $VAR[X+Y]=VAR[X]+VAR[Y]+2COV(X,Y)$.
• Covariance:$COV(X,Y)=E[(X-E(X))(Y-E[Y])]$.

Bernoulli distribution

A Bernoulli trial is an experiment whose dichotomous outcomes are random (e.g. ‘head vs. ‘tail’). $X(outcome)= \begin{cases} 0, & \text{s=head} \\ 1, & \text{s=tail} \end{cases}$. If p=P(head), then $E[X]=p$ and $VAR[X]=p(1-p)$.

Binomial distribution

Suppose we conduct an experiment observing n trial Bernoulli process. If we are interested in the RV $x$ = {Number of heads in the $n$ trials}, then $X$ is called a Binomial RV and its distribution is called Binomial distribution, $X \approx B(n,p)$,where $n$ is sample size, $p$ is the probability of head at one trial. $P(X=x)={n\choose x} p^x (1-p)^{n-x}$, for $x=0,1,…,n$, where ${n\choose x}=\frac {n!} {x!(n-x)!}$ is the binomial coefficient. $$E[X]=np,VAR[X]=np(1-p)$$

Multinomial distribution

The multinomial distribution is an extension of binomial where the experiment consists of $k$ repeated trials and each trial has a discrete number of possible outcomes; on any given trial, the probability that a particular outcome will occur is constant; the trials are independent.

$ p=P(X_1=r_1 \cap \cdots \cap X_k=r_{k}│r_1 + ⋯ +r_k=n)$ = ${n\choose r_1,…,r_k} p_1^{r_1} p_2^{r_2}…p_k^{r_k}$ for all (∀) $r_1+⋯+r_k=n$ where ${n\choose r_1,…,r_k}=\frac {n!}{r_1! \times … \times r_k!}$.

Geometric distribution

The probability distribution of number X of Bernoulli trials needed to get one success, supported on the set $\{1,2,3,…\}$. $P(X=x)=(1-p)^{x-1}p$, for $x = 1, 2, … $

$$(E[X]=\dfrac {1} {p},VAR[X]= \dfrac {1-p} {p^{2}}$$

Hypergeometric distribution

A discrete probability distribution that describes the number of successes in a sequence of n draws from a finite population without replacement. An experimental design for using Hypergeometric distribution is illustrated in this table: a shipment of N objects in which m are defective. The Hypergeometric Distribution describes the probability that in a sample of n distinctive objects drawn from the shipment exactly k objects are defective.

Type	Drawn	Not-Drawn	Total
Defective	k	m-k	m
Non-Defective	n-k	N+k-n-m	N-m
Total	n	N-n	N

$$ P(X=k)=\dfrac {{m \choose k}{N-m \choose n-k}} {N \choose n}$,$E[X]=\frac{nm}{N}$, $VAR[X]=\frac{\frac{nm}{N}(1-\frac{m}{N})(N-n)} {N-1}$$

Negative binomial distribution

Suppose X=trial index (n) of the $r^{th}$ success, or total number of experiments ($n$) to get $r$ successes. $P(X=n)={n-1 \choose r-1} p^r (1-p)^(n-r)$, for $n=r,r+1,r+2,…$, where n is the trial number of the $𝑟^𝑡ℎ$ success.

$$E[X]=\dfrac {r} {p},VAR[X]=\dfrac {r(1-p)} {p^{2}}$$

Suppose Y= Number of failures (k) to get r successes. $P(Y=k)={k+r-1 choose k} p^{r} (1-p)^{k}$, for $k=0,1,2,…,$ where k is the number of failures before the $ r^{th} $ success. $Y~NegBin(r,p)$, the probability of k failures and r successes in n = k+1 Bernoulli(p) trials with success on the last trial.

$E[Y]=\dfrac{r(1-p)}{p},VAR[Y]=\dfrac {r(1-p)} {p^{2}}$

NOTE: $X=Y+r,E[X]=E[Y]+r,VAR[X]=VAR[Y]$.

3.10) Negative multinomial distribution (NMD): a generalization of the two-parameter NB(r,p) to more than one outcomes. Suppose we have m possible outcomes where $m≥1,{X_{0},…,X_{m}}$ each with probability \({p_0,…,p_m }\) respectively where \(0<p_i<1\) and \(∑_(i=0)^m p_i =1\) Suppose the experiment generate independent outcomes until \({X_0,…,X_m }\) occur exactly \({k_0,…,k_m }\) times, then ${X_{0},…,X_{m}}$ s Negative Multinomial with parameter vector $(k_0,{p_{1},…,p_{m}})$. The degree of freedom is m. X is the total number of experiments (n) to get $k_{0}$ and $n-k_{o}$ outcomes of the other possible outcome $(X_{1})$. $X~NegativeMultinomial(k_{0},{p_[0},p_{1})$

$ P(k_{1},…,k_{m}|k_{0},\{p_{1},…,p_{m}\})$ = $\Bigg(\Sum_{i=0}^{m}k_{i}-1\Bigg)$!$\dfrac{p_{0}^{k}_{0}}{(k_{0}-1)}!$

REST OF PROBLEM HERE $∏_{i=1}^{m}\dfrac{p_{I}^{k}_{i}} {k_{i}}!$

3.11) Poisson distribution: a discrete distribution that expresses the probability of the number of events occurring in a fixed interval of time if these events occur with a known average rate and independently of the time since the last event.

Figure 1 PDF of Poisson Distribution with changing parameter.

This is the PDF of Poisson distribution with changing values for parameter λ, as we can see that the curve gets more and more sharp with increasing λ and when λ is large enough (say greater than 1) the distribution gives a skewed bell shape. The function is only defined at integer values of k so the connecting lines are only guides for the eye.

Figure 2 CDF of Poisson Distribution with changing parameters.

$P(X=k)=\frac{λ^{k}e^{-λ}}{k!},E[X]=λ,VAR[X]=λ.$

This is the CDF of Poisson distribution with changing values of λ, as we can see that the curve increases faster to 1 with decreasing λ. The CDF is discontinuous at the integers of k and flat everywhere else because the variable only takes on integer values. That is the CDF of Poisson distribution is left continuous but not right continuous. Also note, the CDF of Poisson distribution takes on the value of 0 with 0 occurrence and it is non-decreasing with increasing number of occurrence. And it increases and stays at 1 after certain number of occurrence.

3.12) Normal distribution: continuous distribution with the density $f(x)=\frac{{e^{x}^{2}-{2}}{√2\pi}}$, the cumulative distribution $Φ(y)=∫_{-∞}^{y}$

Applications

4.1) This article (http://www.mdm.com/articles/28757-the-case-for-proactive-inside-sales?v=preview) titled The Case for Proactive inside Sales examined how a proactive inside sales force can be critical to serving mid-market and small customers as part of a broader multichannel strategy and in included steps for initiating an effective program.

4.2) This article http://wiki.socr.umich.edu/index.php/SOCR_EduMaterials_Activities_NegativeBinomial) provides an example of Negative Binomial Experiment by SOCR. The goal of this experiment is to provide a simulation demonstrating properties of the Negative Binomial(k,p) distribution. The applet facilitates the calculations of the Negative Binomial mass/density function, the moments and cumulative distribution function. It gives the specific steps of the experiment in SOCR and it allows users to learn about the variation of the distribution with changing parameters.

5) Software

• SOCR Distributions
• Bivariate_Normal_Experiment

• Normal T Chi$^{2}$

• Multinomial_Distribution
• Activities Binomial Distributions

Problems

6.1) If sampling distributions of sample means are examined for samples of size 1, 5, 10, 16 and 50, you will notice that as sample size increases, the shape of the sampling distribution appears more like that of the:

(a) normal distribution

(b) uniform distribution

(c) population distribution

(d) binomial distribution

6.2) Which of the following statements best describes the effect on the Binomial Probability Model if the number of trials is held constant and the p( the probability of "success") increases?

(a) None of these statements are true

(b) The mean and the standard deviation both increase

(c) The mean decreases and the standard deviation increases

(d) The mean increases and the standard deviation decreases

(e) The mean and standard deviation both decrease

6.3) Suppose you draw one card from a standard deck three times, with replacement. What is the probability that you get spades all three times? Choose one answer.

(a) 0.002

(b) 0.321

(c) 0.015

(d) 0.021

6.4) Suppose the number of cars that enter a parking lot in an hour is a Poisson random variable, and suppose that P(X=0)=0.05. Determine the variance of X.

(a) 0.349

(b) 3.232

(c) 9.321

(d) 2.996

6.5) A researcher converts 100 lung capacity measurements to z-scores. The lung capacity measurements do not follow a normal distribution. What can we say about the standard deviation of the 100 z-scores?

(a) It depends on the standard deviation of the raw scores

(b) It equals 1

(c) It equals 100

(d) It must always be less than the standard deviation of the raw scores

(e) It depends on the shape of the raw score distribution

6.6) Among first year students at a certain university, scores on the verbal SAT follow the normal curve. The average is around 500 and the SD is about 100. Tatiana took the SAT, and placed at the 85% percentile. What was her verbal SAT score?

(a) 604

(b) 560

(c) 90

(d) 403

6.7) Consider a random sample 100 orc soldiers and found the mean and the standard deviation to be 200lbs and and 20lbs respectively. He can be 68% confident that the mean weight in the population of orc soldiers is between

(a) 196 to 204 lbs

(b) 198 to 202 lbs

(c) 194 to 206 lbs

(d) None of the above

6.8) The Rockwell hardness of certain metal pins is known to have a mean of 50 and a standard deviation of 1.5. If the distribution of all such pin hardness measurements is known to be normal, what is the probability that the average hardness for a random sample of nine pins is at least 50.5?

(a) Approximately 4

(b) 0.4

(c) Approximately 0.1587

(d) Approximately 0

6.9) You read that the heights of college women are nearly normal with a mean of 65 inches and a standard deviation of 2 inches. If Vanessa is at the 10th percentile (shortest 10% for women) in height for college women, then her height is closest to:

(a) 64.5 inches

(b) It cannot be determined from this information

(c) 60.5 inches

(d) 62.44 inches

6.10) The settlement (in cm) of a structure shown in the following figure may be evaluated from S = 0.3A + 0.2B + 0.1C

---

• SOCR Home page: http://www.socr.umich.edu

---

---

Translate this page:

(default)	Deutsch	Español	Français	Italiano	Português	日本語	България	الامارات العربية المتحدة	Suomi	इस भाषा में	Norge
한국어	中文	繁体中文	Русский	Nederlands	Ελληνικά	Hrvatska	Česká republika	Danmark	Polska	România	Sverige