SMHS BayesianInference - Revision history

Dinov at 13:16, 10 February 2026

2026-02-10T13:16:48Z

Dinov: /* Software */

2026-02-09T22:43:44Z

‎Software

Dinov: /* Advantages of Bayesian Methods */

2026-02-09T22:35:52Z

‎Advantages of Bayesian Methods

Dinov: /* Advantages of Bayesian Methods */

2026-02-09T22:33:34Z

‎Advantages of Bayesian Methods

Dinov: /* Advantages of Bayesian Methods */

2026-02-09T22:32:51Z

‎Advantages of Bayesian Methods

Dinov: /* Advantages of Bayesian methods */

2026-02-09T22:31:29Z

‎Advantages of Bayesian methods

Dinov: /* Advantages of Bayesian methods */

2026-02-09T22:27:31Z

‎Advantages of Bayesian methods

Dinov at 22:26, 9 February 2026

2026-02-09T22:26:01Z

Show changes

Dinov: /* References */

2017-07-21T20:46:36Z

‎References

Dinov: /* Multiplication rule */

2014-09-15T17:05:01Z

‎Multiplication rule

← Older revision		Revision as of 13:16, 10 February 2026
Line 1:		Line 1:
−	~~```mediawiki~~
	==[[SMHS\| Scientific Methods for Health Sciences]] - Bayesian Inference ==		==[[SMHS\| Scientific Methods for Health Sciences]] - Bayesian Inference ==

Line 270:		Line 269:

	{{translate\|pageName=https://wiki.socr.umich.edu/index.php?title=SMHS_BayesianInference}}		{{translate\|pageName=https://wiki.socr.umich.edu/index.php?title=SMHS_BayesianInference}}
−	~~```~~

← Older revision		Revision as of 22:43, 9 February 2026
Line 244:		Line 244:

	persp(a,b,post.prob.theta,xlab="a",ylab="b",zlab="p(theta > 0.8 \| y)",main="P(theta > 0.8 \| y)", theta=30,phi=20)		persp(a,b,post.prob.theta,xlab="a",ylab="b",zlab="p(theta > 0.8 \| y)",main="P(theta > 0.8 \| y)", theta=30,phi=20)
		+
		+	image(post.prob.theta)
	</pre>		</pre>

@@ Line 62: / Line 62: @@
    ** Model 1: Fixed number of trials (Binomial).
 Let <math>Y_1</math> be the number of successful treatments out of 12 patients, modeled as [[SMHS_ProbabilityDistributions#Binomial_distribution|Binomial(n=12, \theta)]]. The observed data is <math>Y_1 = 9</math>.
    ** Model 2: Stop when a fixed number of failures occur (Negative Binomial).
-  ** Let <math>Y_2</math> be the number of successful treatments observed before reaching 3 failures (sick patients), modeled as a [[SMHS_ProbabilityDistributions#Negative_binomial_distribution|Negative Binomial(r=3, \theta)]] (number of successes before r failures, with success probability <math>\theta</math>). The observed data is <math>Y_2 = 9</math>.
+The likelihood functions under both models are proportional (both involve <math>\theta^9 (1-\theta)^3</math> up to a constant), yet the frequentist p-values differ: approximately 0.073 for the binomial model and 0.033 for the negative binomial model. At a 5% significance level, the negative binomial model would reject <math>H_0</math>, while the binomial model would not, despite the data being the same.
 * '''Sequential Learning and Updating''': Bayes' theorem provides a coherent framework for updating beliefs with new data. If you have a posterior distribution <math>p(\theta \mid y_1, \dots, y_n)</math> from initial data <math>y_1, \dots, y_n</math>, it can serve as the prior for incorporating a new observation <math>y_{n+1}</math>, yielding an updated posterior <math>p(\theta \mid y_1, \cdots, y_{n+1})</math>. This enables rational, incremental learning without starting from scratch each time.

@@ Line 71: / Line 71: @@
    The likelihood functions under both models are proportional (both involve <math>\theta^9 (1-\theta)^3</math> up to a constant), yet the frequentist p-values differ: approximately 0.073 for the binomial model and 0.033 for the negative binomial model. At a 5% significance level, the negative binomial model would reject <math>H_0</math>, while the binomial model would not—despite the data being the same.
-* '''Sequential Learning and Updating''': Bayes' theorem provides a coherent framework for updating beliefs with new data. If you have a posterior distribution <math>p(\theta \mid y_1, \dots, y_n)</math> from initial data <math>y_1, \dots, y_n</math>, it can serve as the prior for incorporating a new observation <math>y_{n+1}</math>, yielding an updated posterior <math>p(\theta \mid y_1, \dots, y_{n+1>)</math>. This enables rational, incremental learning without starting from scratch each time.
+* '''Sequential Learning and Updating''': Bayes' theorem provides a coherent framework for updating beliefs with new data. If you have a posterior distribution <math>p(\theta \mid y_1, \dots, y_n)</math> from initial data <math>y_1, \dots, y_n</math>, it can serve as the prior for incorporating a new observation <math>y_{n+1}</math>, yielding an updated posterior <math>p(\theta \mid y_1, \cdots, y_{n+1})</math>. This enables rational, incremental learning without starting from scratch each time.
 * '''No Reliance on Large Samples''': Unlike many frequentist methods that depend on asymptotic approximations (e.g., normal approximations for valid inference), Bayesian inference is exact for any sample size. The posterior distribution is derived directly from the likelihood and prior, making it suitable for small-sample problems where frequentist methods may fail.

@@ Line 61: / Line 61: @@
 * '''Likelihood Principle''': Bayesian inference adheres to the likelihood principle, which asserts that if two different sampling models produce proportional likelihood functions for <math>\theta</math> given the observed data, then inferences about <math>\theta</math> should be identical under both models. Frequentist methods, however, can violate this principle, leading to different conclusions from the same data depending on the sampling design. For example, consider a clinical trial where we observe 9 patients cured and 3 still sick. We wish to test the null hypothesis <math>H_0: \theta = 1/2</math> against the alternative <math>H_a: \theta > 1/2</math>, where <math>\theta</math> is the probability of a successful cure (i.e., testing if the cure rate is better than 50/50).
-   ** Model 1: Fixed number of trials (Binomial).** Let <math>Y_1</math> be the number of successful treatments out of 12 patients, modeled as [[SMHS_ProbabilityDistributions#Binomial_distribution|Binomial(n=12, \theta)]]. The observed data is <math>Y_1 = 9</math>.
+   ** Model 1: Fixed number of trials (Binomial).
-   ** Model 2: Stop when a fixed number of failures occur (Negative Binomial).** Let <math>Y_2</math> be the number of successful treatments observed before reaching 3 failures (sick patients), modeled as a [[SMHS_ProbabilityDistributions#Negative_binomial_distribution|Negative Binomial(r=3, \theta)]] (number of successes before r failures, with success probability <math>\theta</math>). The observed data is <math>Y_2 = 9</math>.
+  ** Let <math>Y_1</math> be the number of successful treatments out of 12 patients, modeled as [[SMHS_ProbabilityDistributions#Binomial_distribution|Binomial(n=12, \theta)]]. The observed data is <math>Y_1 = 9</math>.
    The likelihood functions under both models are proportional (both involve <math>\theta^9 (1-\theta)^3</math> up to a constant), yet the frequentist p-values differ: approximately 0.073 for the binomial model and 0.033 for the negative binomial model. At a 5% significance level, the negative binomial model would reject <math>H_0</math>, while the binomial model would not—despite the data being the same.

@@ Line 53: / Line 53: @@
 Once data <math>y_1,...,y_n</math> have been collected, we specify a sampling model for the data <math>p(y_i|\theta)</math>, <math>i=1,\ldots,n</math>, which represents the probability of observing <math>y_i</math> if we knew <math>\theta</math>. In light of the data observed, using Bayes’ theorem we update our belief about <math>\theta</math> and derive the posterior distribution <math>p(\theta|y_1,...,y_n)=p(\theta|y)</math>, where <math>y=\{y_1,...,y_n\}</math>.
-====Advantages of Bayesian methods====
+==== Advantages of Bayesian Methods ====
-* ''Likelihood Principle'': Bayesian inference obeys the likelihood principle. The likelihood principle states that if two sampling models yield proportional likelihoods for <math>\theta</math>, then inference about <math>\theta</math> should be identical under the two models. Frequentist inference does not always obey the likelihood principle! For example: suppose we observe the clinical outcomes in experimental treatment applied on 12 patients and at the end of the trial we observe 3 patients cured and 9 still sick. We want to test the hypothesis that the probability of successful cure is better than 50/50, i.e., <math>H_o: \theta=1/2</math> vs. <math>H_a: \theta>1/2</math>. Suppose that we set up two sampling models for these data:
+Bayesian methods offer several key advantages over frequentist approaches in statistical inference. Below, we outline these benefits, highlighting how they address limitations in frequentist methods.
-: The two likelihoods of the 2 models are proportional but they lead to two different p-values.
+* '''Interpretation''': Bayesian inference provides a posterior distribution for the unknown parameter(s) <math>\theta</math>, which allows for direct probabilistic statements about <math>\theta</math>. For instance, suppose the parameter of interest is the population mean <math>\theta</math>. After collecting data, a Bayesian 95% credible interval directly indicates that there is a 95% posterior probability that <math>\theta</math> lies within the interval. In contrast, a frequentist 95% confidence interval, such as <math>\hat{y} \pm 1.96 \frac{s}{\sqrt{n}}</math>, does not represent the probability that <math>\theta</math> is in the interval after the data are observed—it either contains <math>\theta</math> or it does not. Instead, the 95% refers to the long-run coverage probability across repeated hypothetical samples.
-* Bayesian theorem provides a rational method for learning. Suppose you collect data <math>y_1,...,y_n</math> and compute your posterior distribution <math>p(\theta|y_1,...,y_n)</math>. If we later collect an additional observation <math>y_{n+1}</math>, then the posterior distribution <math>p(\theta|y_1,...,y_n)</math> could be used as a prior for the current Bayesian data analysis.
+* '''Likelihood Principle''': Bayesian inference adheres to the likelihood principle, which asserts that if two different sampling models produce proportional likelihood functions for <math>\theta</math> given the observed data, then inferences about <math>\theta</math> should be identical under both models. Frequentist methods, however, can violate this principle, leading to different conclusions from the same data depending on the sampling design. For example, consider a clinical trial where we observe 9 patients cured and 3 still sick. We wish to test the null hypothesis <math>H_0: \theta = 1/2</math> against the alternative <math>H_a: \theta > 1/2</math>, where <math>\theta</math> is the probability of a successful cure (i.e., testing if the cure rate is better than 50/50).
-* Bayesian inference does not require large sample theory. Bayesian methods do not require asymptotics for valid inference. Small sample Bayesian inference proceeds in the same way as if one had a large sample.
+  ** Model 1: Fixed number of trials (Binomial).** Let <math>Y_1</math> be the number of successful treatments out of 12 patients, modeled as [[SMHS_ProbabilityDistributions#Binomial_distribution|Binomial(n=12, \theta)]]. The observed data is <math>Y_1 = 9</math>.
-* Finally, Bayesian inference often includes frequentist inference as a special case. One can often obtain frequentist answers by choosing a uniform prior for the parameters. In this cases, frequentist answers can be obtained from the posterior distribution, where the MLE is then the posterior mode.
+  ** Model 2: Stop when a fixed number of failures occur (Negative Binomial).** Let <math>Y_2</math> be the number of successful treatments observed before reaching 3 failures (sick patients), modeled as a [[SMHS_ProbabilityDistributions#Negative_binomial_distribution|Negative Binomial(r=3, \theta)]] (number of successes before r failures, with success probability <math>\theta</math>). The observed data is <math>Y_2 = 9</math>.
-* The choice of the prior distribution is a delicate issue and the main criticism to Bayesian inference. Two people analyzing the same data but using different priors could obtain different answers. When carry out a Bayesian analysis it is important to examine the prior sensitivity, that is, determine how sensitive are the results to the choice of the prior distribution.
+  The likelihood functions under both models are proportional (both involve <math>\theta^9 (1-\theta)^3</math> up to a constant), yet the frequentist p-values differ: approximately 0.073 for the binomial model and 0.033 for the negative binomial model. At a 5% significance level, the negative binomial model would reject <math>H_0</math>, while the binomial model would not—despite the data being the same.
 ===Bayes theorem for density functions===

@@ Line 57: / Line 57: @@
 * ''Likelihood Principle'': Bayesian inference obeys the likelihood principle. The likelihood principle states that if two sampling models yield proportional likelihoods for <math>\theta</math>, then inference about <math>\theta</math> should be identical under the two models. Frequentist inference does not always obey the likelihood principle! For example: suppose we observe the clinical outcomes in experimental treatment applied on 12 patients and at the end of the trial we observe 3 patients cured and 9 still sick. We want to test the hypothesis that the probability of successful cure is better than 50/50, i.e., <math>H_o: \theta=1/2</math> vs. <math>H_a: \theta>1/2</math>. Suppose that we set up two sampling models for these data:
-** Model 1: <math>Y</math>, number of successful treatments, as a [[SMHS_ProbabilityDistributions#Binomial_distribution|Binomial(n=12, <math>\theta</math>)]];
+** Model 1: <math>Y_1</math>, number of successful treatments, as a [[SMHS_ProbabilityDistributions#Binomial_distribution|Binomial(n=12, <math>\theta</math>)]];
-** Model 2: <math>Y</math>, number of successfully treated patients to be observed before terminating the clinical trial (in our example, getting 3 patients cured), which is a [[SMHS_ProbabilityDistributions#Negative_binomial_distribution|Negative Binomial(r=3, <math>1-\theta</math>)]].
+** Model 2: <math>Y_2</math>, number of successfully treated patients to be observed before terminating the clinical trial (in our example, getting 3 patients cured), which is a [[SMHS_ProbabilityDistributions#Negative_binomial_distribution|Negative Binomial(r=3, <math>1-\theta</math>)]].
 : The two likelihoods of the 2 models are proportional but they lead to two different p-values.

← Older revision		Revision as of 20:46, 21 July 2017
Line 232:		Line 232:
	===References===		===References===
	* [http://en.wikipedia.org/wiki/Bayesian_inference Wikipedia Bayesian Inference]		* [http://en.wikipedia.org/wiki/Bayesian_inference Wikipedia Bayesian Inference]
		+	* [http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0019178 Christou N, Dinov, ID (2011) Confidence Interval Based Parameter Estimation—A New SOCR Applet and Activity. PLoS ONE 6(5): e19178. doi:10.1371/journal.pone.0019178].

@@ Line 14: / Line 14: @@
 ====Multiplication rule====
-For any two events, $A$ and $B$, $ P(A\cap B)=P(A│B)P(B) $. In general, for $n$ events $A_1, ..., A_n$: $ P(A_1 \cap A_2 \cap A_3 \cap … \cap A_n ) =$ $ P(A_1 )P(A_1│A_2 )P(A_3│A_1\cap A_2 ) … P(A_n│A_1\cap A_1\cap A_2\cap A_3\cap … \cap A_{n-1}) $.
+For any two events, $A$ and $B$, $ P(A\cap B)=P(A│B)P(B) $. In general, for $n$ events $A_1, ..., A_n$: $ P(A_1 \cap A_2 \cap A_3 \cap … \cap A_n ) =$ $ P(A_1 )P(A_2│A_1)P(A_3│A_1\cap A_2) … P(A_n│A_1\cap A_1\cap A_2\cap A_3\cap … \cap A_{n-1})$.
 ====Law of total probability====