Glivenko–Cantelli theorem

Glivenko–Cantelli Theorem

The Glivenko–Cantelli theorem, also known as the fundamental theorem of statistics, is a foundational result in the field of probability theory and statistics. It establishes that the empirical distribution function converges uniformly to the cumulative distribution function of the population from which the sample is drawn, as the sample size increases to infinity. This theorem is named after Valery Glivenko and Francesco Paolo Cantelli, who independently discovered this property in the early 20th century.

Statement of the Theorem

Formally, let \(F(x)\) be the cumulative distribution function (CDF) of a random variable \(X\), and let \(F_n(x)\) be the empirical distribution function (EDF) based on a sample \(X_1, X_2, \ldots, X_n\), defined as:

\[F_n(x) = \frac{1}{n} \sum_{i=1}^n I(X_i \leq x)\]

where \(I\) is the indicator function, which is 1 if \(X_i \leq x\) and 0 otherwise. The Glivenko–Cantelli theorem states that:

\[\lim_{n \to \infty} \sup_{x \in \mathbb{R}} |F_n(x) - F(x)| = 0\]

almost surely, where \(\sup\) denotes the supremum.

Implications

The Glivenko–Cantelli theorem has profound implications in both theoretical and applied statistics. It justifies the use of the empirical distribution function as a consistent estimator for the true cumulative distribution function. This is crucial in many statistical procedures, including hypothesis testing, non-parametric statistics, and goodness-of-fit tests.

Proof and Extensions

The original proof of the Glivenko–Cantelli theorem relies on properties of the empirical distribution function and the cumulative distribution function. Since its initial formulation, several extensions and variations of the theorem have been developed, including versions for dependent and identically distributed random variables.

Applications

In practice, the Glivenko–Cantelli theorem is applied in various statistical methodologies, such as the Kolmogorov-Smirnov test, which is a non-parametric test for the equality of continuous, one-dimensional probability distributions. It also underpins many techniques in machine learning and data science, where understanding the convergence of empirical measures to true distributions is essential.

See Also

• Law of large numbers
• Central limit theorem
• Kolmogorov-Smirnov test
• Empirical distribution function
• Cumulative distribution function

References

• Billingsley, Patrick (1995). Probability and Measure. Wiley-Interscience.
• Van der Vaart, A. W. (1998). Asymptotic Statistics. Cambridge University Press.