Log probability

Log probability is a measure of the likelihood of an event expressed in logarithmic terms. It is commonly used in probability theory and statistics to simplify the multiplication of probabilities and to avoid numerical underflow in computations involving very small probabilities.

Definition

The log probability of an event is defined as the logarithm of the probability of that event. If \( P(A) \) is the probability of event \( A \), then the log probability is given by: \[ \log P(A) \]

The base of the logarithm can vary, but common choices include the natural logarithm (base \( e \)) and the binary logarithm (base 2).

Applications

Log probabilities are particularly useful in fields such as machine learning, natural language processing, and bioinformatics. They are used to:

• Simplify the computation of the product of probabilities, as the sum of log probabilities is equal to the log of the product of the original probabilities.
• Prevent numerical underflow when dealing with very small probabilities, which can occur in applications like hidden Markov models and Bayesian networks.

Advantages

• Numerical Stability: Log probabilities help in maintaining numerical stability in computations involving very small probabilities.
• Simplification: They convert the multiplication of probabilities into addition, which is computationally simpler and more efficient.

Related Concepts

• Log-likelihood
• Logarithm
• Probability distribution
• Bayesian inference
• Entropy (information theory)

See Also

• Probability theory
• Statistics
• Machine learning
• Natural language processing
• Bioinformatics