Science Fair Project Encyclopedia

Science Fair Project Encyclopedia Contents Page

Bayes factor

In statistics, the use of Bayes factors is a Bayesian alternative to classical hypothesis testing.

Given a model selection problem in which we have to choose between two models M_{1_{and M_{2_{, on the basis of a data vector x. The Bayes factor K is given by}}}}

$K = \frac{p(x|M_1)}{p(x|M_2)}.$

This is similar to a likelihood-ratio test, but instead of maximisimg the likelihood Bayesians average it over the parameters. Generally, the models M_{1_{and M_{2_{will be parametrised by vectors of parameters θ₁ and θ₂; thus K is given by}}}}

$K = \frac{p(x|M_1)}{p(x|M_2)} = \frac{\int \,p(\theta_1|M_1)p(x|\theta_1, M_1)d\theta_1}{\int \,p(\theta_2|M_2)p(x|\theta_2, M_2)d\theta_2}.$

A value of K > 1 means that the data indicate that M₁ is more likely than M₂ and vice versa. Note that classical hypothesis testing gives one hypothesis (or model) preferred status (the 'null hypothesis'), and only considers evidence against it. Harold Jeffreys gave a scale for interpretation of K:

K	Strength of evidence
< 1	Negative (supports M₂)
1 to 3	Barely worth mentioning
3 to 12	Positive
12 to 150	Strong
> 150	Very strong

Many Bayesian statisticians would use a Bayes factor as part of making a choice, but would also combine it with their estimates of the prior probability of each of the models and the loss functions associated with making the wrong choice.

Example

Suppose we have a random variable which produces either a success or a failure. We want to consider a model M₁ where the probability of success is q=½, and another model M₂ where q is completely unknown and we take a prior distribution for q which is uniform on [0,1]. We take a sample of 200, and find 115 success and 85 failures. The likelihood is:

${200 \choose 115}q^{115}(1-q)^{85}$

So we have

$P(X=115|M_1)={200 \choose 115}\left({1 \over 2}\right)^{200}=0.00595...$

but

$P(X=115|M_2)=\int_{q=0}^1 1{200 \choose 115}q^{115}(1-q)^{85}dq = {1 \over 201} = 0.00497...$

The ratio is then 1.197..., which is "barely worth mentioning" even if it points very slightly towards M₁.

This is not the same as a classical likelihood ratio test, which would have found the maximum likelihood estimate for q, namely ¹¹⁵⁄₂₀₀=0.575, and from that get a ratio of 0.1045..., and so pointing towards M₂. A frequentist hypothesis test would have produced an even more dramatic result, saying that that M₁ could be rejected at the 5% confidence level, since the probability of getting 115 or more successes from a sample of 200 if q=½ is 0.0200..., and as a two-tailed test of getting a figure as extreme as or more extreme than 115 is 0.0400... Note that 115 is more than two standard deviations away from 100.

M₂ is a more complex model than M₁ because it has a free parameter which allows it to model the data more closely. The ability of Bayes factors to take this into account is a reason why Bayesian inference has been put forward as a theoretical justification for and generalisation of Occam's razor, reducing Type I errors.

All Science Fair Projects