Lindleys Paradox

Lindley's paradox

• Lindley's paradox: Situation where bayesian and frequentists to testing a hypothesis give different results for specific prior distributions.
• Bayesian inference: Statistical inference method that uses Baye's theorem to find a probability for the hypothesis, based on prior observations/evidence. Thus using prior distribution to estimate posterior probabilities.
• Bayes' Theorem: Mathematical rule that inverts conditional probabilities, that allows finding a cause from the effect, or more updates the probability of a hypothesis after observing the evidence found. $P(A|B) = \frac{P(B|A)P(A)}{P(B)}$
• Conditional Probability: The probability that an event will occur, based on another event (correlation) having already occurred. $P(A|B)=\frac{P(A\cap B))}{P(B)}$ , or the probability of A given B.
• Prior Probability: or prior probability distribution, is the assumed probability distribution before evidence is considered.
• Posterior Probabilities: A conditional probability that arises from updating the prior probability.
• Frequentist Inference: A type of statistical inference that is based on frequentist probability and treats probability as frequency to draw conclusions form sample-data.
• Frequentist Probability: or frequentism, represents an interpretation of probability that defines an event's probability as the limit of the event's relative frequency in an infinite number of trials.
• relative frequency or empirical probability, is the estimate of probability that an event will occur.
• binomial distribution: A series of $n$ experiments all asking boolean outcomes (yes/no) with a probability $p$.
• p-value: the probability based on the assumption the null hypothesis is correct, of getting results at least as extreme as the observed result.

Description of the paradox

The results of an experiment, $x$, has two possible explanations, and it is uncertain which hypothesis is more accurate given the prior distribution.

Numerical example

Given an example of boys vs girls born over a period of time in a city, the frequentist and the bayesian approaches deliver different predictions for $x$ the observed births that are boys, with a null hypothesis of $H_0:\theta = 0.5$ and a hypothesis that $H_1:\theta \ne 0.5$. With a very large sample, even a tiny difference from 0.5 can produce a statistically significant p-value. The frequentist test may reject H₀; while the bayesian test still favors $H_0$. So paradoxically a null hypothesis is rejected by one test and accepted by another.