Ordinary least squares

Error creating thumbnail:

Okuns law quarterly differences

Error creating thumbnail:

OLS geometric interpretation

OLS example weight vs height scatterplot

OLS example weight vs height fitted line

OLS example weight vs height residuals

Error creating thumbnail:

HeightWeightResiduals

Ordinary Least Squares (OLS) is a type of linear regression method used in statistics and econometrics for estimating the unknown parameters in a linear regression model. OLS selects the parameters of a linear function of a set of explanatory variables by minimizing the sum of the squares of the differences between the observed dependent variable in the given dataset and those predicted by the linear function.

Overview

The primary goal of OLS is to closely fit a line through a scatter plot of data points where one axis represents the independent variable(s) and the other the dependent variable. The best fit line, according to OLS, is the one that minimizes the sum of the squared differences (residuals) between the observed values and the values predicted by the linear model.

Mathematical Formulation

Given a dataset containing n observations, n pairs {(yᵢ, xᵢ)}, where i = 1, 2, ..., n, and yᵢ is the dependent variable and xᵢ is the independent variable (or vector of variables in multiple regression), the OLS method seeks to minimize the sum of squared residuals:

\[ S = \sum_{i=1}^{n} (y_i - \beta_0 - \sum_{j=1}^{k} \beta_j x_{ij})^2 \]

where β₀ is the intercept term, βⱼ (for j = 1, 2, ..., k) are the slope coefficients, and k is the number of independent variables.

Assumptions

The OLS method is based on several key assumptions, including:

• Linearity: The relationship between the dependent and independent variables is linear.
• Independence: Observations are independent of each other.
• Homoscedasticity: The variance of error terms is constant across all levels of the independent variables.
• No perfect multicollinearity: The independent variables are not perfectly correlated with each other.
• Normality: The error terms are normally distributed (this assumption is necessary for hypothesis testing but not for the estimation of coefficients).

Estimation

The OLS estimates of the coefficients (β) are calculated using the formula:

\[ \hat{\beta} = (X'X)^{-1}X'Y \]

where X is the matrix of independent variables (including a column of ones for the intercept), Y is the vector of the dependent variable, and X'X is the matrix multiplication of X transposed and X. The inverse of X'X is then multiplied by X'Y to give the estimate of β.

Properties

Under the OLS assumptions, the OLS estimator has several desirable properties:

• Unbiasedness: The expected value of the OLS estimator is equal to the true parameter value.
• Efficiency: Among all linear unbiased estimators, OLS estimators have the smallest variance.
• Consistency: As the sample size increases, the OLS estimators converge in probability to the true parameter values.

Applications

OLS is widely used in various fields such as economics, finance, political science, and biology, among others, for predictive modeling and inference about relationships between variables.

Limitations

While OLS is a powerful and widely used method, it has limitations, especially when its assumptions are violated. For example, if there is heteroscedasticity or autocorrelation in the residuals, the standard errors of the OLS estimates can be biased, leading to incorrect inferences. In such cases, other estimation techniques or adjustments to the standard errors may be necessary.

This article is a stub. You can help WikiMD by registering to expand it. Editing is available only to registered and verified users. WikiMD is a comprehensive, free health & wellness encyclopedia.