Berry–Esseen theorem

Berry–Esseen theorem is a fundamental result in probability theory and statistics that quantifies the rate of convergence of the distribution of a sum of independent, identically distributed (i.i.d.) random variables to the normal distribution (also known as the Gaussian distribution). This theorem is a cornerstone in the theory of the central limit theorem, providing explicit bounds on the difference between the distribution function of a normalized sum of i.i.d. random variables and the standard normal distribution function. The theorem is named after the mathematicians Andrew C. Berry and Carl-Gustav Esseen who independently derived this result in the 1940s.

Statement of the Theorem

The Berry–Esseen theorem states that if \(X_1, X_2, ..., X_n\) are i.i.d. random variables with expected value \(E[X_i] = 0\), variance \(\sigma^2 = Var(X_i) > 0\), and third absolute moment \(\rho = E[|X_i|^3] < \infty\), then there exists a constant \(C\) such that for all \(n\) and all real \(x\),

\[ \left| F_n(x) - \Phi(x) \right| \leq \frac{C\rho}{\sigma^3\sqrt{n}} \]

where \(F_n(x)\) is the cumulative distribution function (CDF) of the standardized sum \(\frac{1}{\sigma\sqrt{n}}\sum_{i=1}^n X_i\), and \(\Phi(x)\) is the CDF of the standard normal distribution. The best known value of \(C\) is approximately 0.4748, as shown in later refinements.

Importance and Applications

The Berry–Esseen theorem is significant for several reasons. Firstly, it provides a quantitative measure of the convergence rate in the central limit theorem, which is crucial for finite sample sizes. This has practical implications in various fields such as econometrics, psychometrics, and machine learning, where the approximation of distributions of sums of random variables is frequently required.

Secondly, the theorem highlights the role of the third moment (skewness) in the convergence rate, indicating that distributions with higher skewness may converge more slowly to the normal distribution. This insight is valuable in the analysis of data that may not be symmetrically distributed.

Proof and Refinements

The original proofs by Berry and Esseen utilized complex analysis techniques, specifically contour integration and the properties of characteristic functions. Subsequent refinements have focused on optimizing the constant \(C\) and extending the theorem to more general conditions, including non-identical distributions and dependent variables under certain conditions.

Limitations

While the Berry–Esseen theorem is powerful, it has limitations. The requirement for a finite third moment excludes certain heavy-tailed distributions. Additionally, the theorem does not provide tight bounds for all distributions, particularly those with significant skewness or kurtosis.

See Also

• Central limit theorem
• Normal distribution
• Law of large numbers
• Skewness
• Kurtosis