Sufficient statistic

A sufficient statistic is a concept in statistics that describes a type of statistic that captures all relevant information from a sample about a given parameter of a probability distribution. The formal definition of a sufficient statistic is based on the factorization theorem, which provides a method to determine whether a statistic is sufficient for a parameter of interest.

Definition

According to the factorization theorem, a statistic T(X) is sufficient for parameter θ if and only if the probability density function (pdf) or probability mass function (pmf) of the sample X can be expressed as:

\[ f(x; \theta) = g(T(x); \theta) h(x) \]

where:

• f(x; \theta) is the joint pdf or pmf of the sample X,
• g is a function that depends on the sample only through T(X),
• h is a function that does not depend on the parameter θ.

This theorem implies that the statistic T(X) contains all the information needed to estimate θ, making the rest of the data irrelevant for this purpose once T(X) is known.

Examples

Discrete Distribution

For a sample X = (X_1, X_2, ..., X_n) from a Bernoulli distribution with parameter p, the sum T(X) = \sum_{i=1}^n X_i is a sufficient statistic for p. This is because the joint pmf of X can be factored into a function of T(X) and p, and another function that does not depend on p.

Continuous Distribution

In the case of a sample from a normal distribution with known variance, the sample mean is a sufficient statistic for the mean of the distribution. The joint pdf of the sample can be expressed in a form that depends on the sample mean and the mean parameter, and another part that does not depend on the mean parameter.

Properties

Sufficient statistics are particularly valuable in statistical inference because they reduce the data needed to perform estimations, simplifying analysis without losing information about the parameter of interest. They are closely related to other statistical concepts such as completeness, minimal sufficiency, and ancillary statistics.

Applications

Sufficient statistics are used in various statistical methodologies, including estimation theory, where they help in deriving efficient estimators like the maximum likelihood estimator (MLE). They are also crucial in the development of statistical models and in the simplification of complex data into a manageable form without losing critical information.

See Also

• Estimator
• Statistics
• Probability theory
• Data reduction