Intraclass correlation coefficient

Statistical measure of reliability

The intraclass correlation coefficient (ICC) is a descriptive statistic that can be used when quantitative measurements are made on units that are organized into groups. It describes how strongly units in the same group resemble each other. The ICC is commonly used to assess the reliability of measurements or ratings in the fields of psychology, education, and medicine.

Definition

The ICC is a measure of the proportion of variance in a set of measurements that is attributable to the grouping structure of the data. It is defined as the ratio of the variance between groups to the total variance (which is the sum of the variance between groups and the variance within groups). Mathematically, it can be expressed as:

\[ \text{ICC} = \frac{\sigma^2_{\text{between}}}{\sigma^2_{\text{total}}} = \frac{\sigma^2_{\text{between}}}{\sigma^2_{\text{between}} + \sigma^2_{\text{within}}} \]

where:

• \( \sigma^2_{\text{between}} \) is the variance between groups,
• \( \sigma^2_{\text{within}} \) is the variance within groups,
• \( \sigma^2_{\text{total}} \) is the total variance.

Applications

The ICC is widely used in various fields:

• In psychometrics, it is used to assess the reliability of psychological tests.
• In medicine, it is used to evaluate the reliability of diagnostic tests and measurements.
• In education, it is used to assess the consistency of grading or scoring by different raters.

Types of ICC

There are several types of ICC, each suitable for different study designs and purposes:

• ICC(1,1): Used when each subject is rated by a different set of raters.
• ICC(2,1): Used when each subject is rated by the same set of raters, and the raters are considered to be a random sample from a larger population.
• ICC(3,1): Used when each subject is rated by the same set of raters, and the raters are considered to be the only raters of interest.

Interpretation

The ICC value ranges from 0 to 1, where:

• 0 indicates no reliability (all variance is within groups),
• 1 indicates perfect reliability (all variance is between groups).

General guidelines for interpreting ICC values are:

• Less than 0.5: Poor reliability
• Between 0.5 and 0.75: Moderate reliability
• Between 0.75 and 0.9: Good reliability
• Greater than 0.9: Excellent reliability

Calculation

The ICC can be calculated using various statistical software packages, such as R, SPSS, and SAS. The choice of model (one-way, two-way, or mixed) and type (consistency or absolute agreement) depends on the study design and the research question.

Limitations

The ICC assumes that the data are normally distributed and that the variances are homogeneous across groups. Violations of these assumptions can lead to biased estimates of the ICC. Additionally, the ICC does not account for systematic differences between raters or measurements.

Also see

• Reliability (statistics)
• Variance (statistics)
• Coefficient of determination
• Cronbach's alpha