Score test

Score Test
Purpose	Statistical hypothesis testing

The Score Test, also known as the Lagrange Multiplier Test, is a statistical test used to determine whether a simpler model is sufficient or if a more complex model is necessary. It is particularly useful in the context of maximum likelihood estimation and is one of the three classical approaches to hypothesis testing, alongside the Likelihood Ratio Test and the Wald Test.

Overview

The Score Test is used to test the null hypothesis that a parameter of interest is equal to a specific value, typically zero, without having to estimate the parameter under the alternative hypothesis. This makes it computationally attractive in situations where estimating the full model is complex or resource-intensive.

Mathematical Formulation

The Score Test is based on the score function, which is the gradient (or derivative) of the log-likelihood function with respect to the parameter of interest. The test statistic is derived from the score function and the Fisher information matrix.

Let \( \theta \) be the parameter of interest, and let \( L(\theta) \) be the likelihood function. The score function \( U(\theta) \) is given by:

\[ U(\theta) = \frac{\partial \log L(\theta)}{\partial \theta} \]

The Fisher information \( I(\theta) \) is given by:

\[ I(\theta) = -E\left[ \frac{\partial^2 \log L(\theta)}{\partial \theta^2} \right] \]

The Score Test statistic \( S \) is then calculated as:

\[ S = \frac{U(\theta_0)^2}{I(\theta_0)} \]

where \( \theta_0 \) is the value of the parameter under the null hypothesis. Under the null hypothesis, the test statistic \( S \) follows a chi-squared distribution with degrees of freedom equal to the number of parameters being tested.

Applications

The Score Test is widely used in econometrics, biostatistics, and other fields where model selection and hypothesis testing are crucial. It is particularly useful in generalized linear models, time series analysis, and survival analysis.

Advantages and Limitations

Advantages

• Computational Efficiency: The Score Test does not require estimation of the parameters under the alternative hypothesis, making it computationally efficient.
• Robustness: It is robust to certain model misspecifications.

Limitations

• Sensitivity to Initial Estimates: The test can be sensitive to the choice of initial parameter estimates.
• Assumptions: Like other tests, it relies on certain assumptions about the distribution of the data and the model.

Also see

• Likelihood Ratio Test
• Wald Test
• Maximum Likelihood Estimation
• Generalized Linear Model
• Hypothesis Testing

References

• Buse, A. (1982). "The Likelihood Ratio, Wald, and Lagrange Multiplier Tests: An Expository Note." The American Statistician.
• Rao, C. R. (1948). "Large Sample Tests of Statistical Hypotheses Concerning Several Parameters with Applications to Problems of Estimation." Proceedings of the Cambridge Philosophical Society.