Statistical significance

NormalDist1.96.png

Statistical significance is a determination by an analyst that the results in the data are not explainable by chance alone. It is a fundamental concept in statistics and is used to determine whether the null hypothesis can be rejected in favor of the alternative hypothesis.

Overview

Statistical significance is often assessed using a p-value, which is the probability of obtaining test results at least as extreme as the results actually observed, under the assumption that the null hypothesis is correct. A result is considered statistically significant if the p-value is less than the chosen significance level, often denoted by α (alpha). Common significance levels are 0.05, 0.01, and 0.001.

Hypothesis Testing

In hypothesis testing, the null hypothesis (H0) represents a default position that there is no relationship between two measured phenomena or no association among groups. The alternative hypothesis (H1) represents the position that there is some relationship or association. Statistical significance is used to determine whether the null hypothesis can be rejected.

P-Value

The p-value is a measure of the strength of the evidence against the null hypothesis. A smaller p-value indicates stronger evidence in favor of the alternative hypothesis. If the p-value is less than the chosen significance level (α), the null hypothesis is rejected.

Significance Levels

The significance level, denoted by α, is the threshold at which the null hypothesis is rejected. Commonly used significance levels are:

• 0.05 (5%)
• 0.01 (1%)
• 0.001 (0.1%)

Type I and Type II Errors

In the context of statistical significance, two types of errors can occur:

• Type I error: Rejecting the null hypothesis when it is actually true (false positive).
• Type II error: Failing to reject the null hypothesis when it is actually false (false negative).

Confidence Intervals

A confidence interval is a range of values, derived from the sample data, that is believed to contain the true value of an unknown population parameter. Confidence intervals are related to statistical significance in that if a confidence interval does not include the null hypothesis value, the result is statistically significant.

Applications

Statistical significance is widely used in various fields such as medicine, psychology, economics, and social sciences to test hypotheses and make inferences about populations based on sample data.

Criticism

Despite its widespread use, the concept of statistical significance has been criticized for various reasons, including the potential for misuse and misinterpretation. Critics argue that statistical significance does not measure the size or importance of an effect and that reliance on p-values can lead to misleading conclusions.

See Also

• Null hypothesis
• Alternative hypothesis
• P-value
• Type I and Type II errors
• Confidence interval
• Effect size