Bayes' formula

Bayes' formula

Bayes' formula, also known as Bayes' theorem, is a fundamental theorem in probability theory and statistics that describes how to update the probabilities of hypotheses when given evidence. It is named after Thomas Bayes, an English statistician, philosopher, and Presbyterian minister, who first provided an equation that allows new evidence to update beliefs in his posthumously published work in 1763. Bayes' formula has applications in a wide range of fields, from medicine to machine learning.

Definition

Bayes' formula is stated mathematically as:

\[ P(A|B) = \frac{P(B|A) \cdot P(A)}{P(B)} \]

where:

• \(P(A|B)\) is the conditional probability of \(A\) given \(B\), i.e., the probability of \(A\) if \(B\) is true.
• \(P(B|A)\) is the conditional probability of \(B\) given \(A\).
• \(P(A)\) and \(P(B)\) are the probabilities of \(A\) and \(B\) independently of each other.

In words, Bayes' formula tells us how to update our prior belief about \(A\) (the prior probability \(P(A)\)) in light of new evidence \(B\) to obtain a revised belief (the posterior probability \(P(A|B)\)).

Applications

Bayes' formula has numerous applications across various disciplines:

• In medicine, it is used for medical diagnosis, helping to determine the probability of a disease given the presence of certain symptoms.
• In machine learning and artificial intelligence, Bayes' theorem is the foundation of the naive Bayes classifier, a simple probabilistic classifier based on applying Bayes' theorem with strong (naive) independence assumptions between the features.
• In finance, it can be used to update the probability of a market movement given new economic indicators or other information.

Example

Consider a medical test for a certain disease. Let \(A\) represent having the disease, and \(B\) represent testing positive for the disease. If the probability of having the disease (\(P(A)\)) is 0.01 (1%), the probability of testing positive if you have the disease (\(P(B|A)\)) is 0.95 (95%), and the probability of testing positive regardless (\(P(B)\)) is 0.10 (10%), then the probability of having the disease given that you tested positive (\(P(A|B)\)) can be calculated using Bayes' formula as follows:

\[ P(A|B) = \frac{0.95 \cdot 0.01}{0.10} = 0.095 \]

This means that even if a person tests positive, there is only a 9.5% chance they actually have the disease, highlighting the importance of considering base rates and test accuracy in medical diagnosis.

See Also

• Probability theory
• Conditional probability
• Thomas Bayes
• Naive Bayes classifier