Simple linear regression

Okuns law quarterly differences

Okuns law with confidence bands

OLS example weight vs height fitted linear

Fitting a straight line to a data with outliers

Simple linear regression is a statistical method that allows us to summarize and study relationships between two continuous (quantitative) variables. This method is termed "simple" because it examines the relationship between two variables only: one independent variable (or predictor variable) and one dependent variable (or outcome variable). The primary goal of simple linear regression is to create a linear model that can describe the relationship between these two variables.

Overview

Simple linear regression fits a linear equation to the observed data. The equation of the straight line model is:

Y = β0 + β1X + ε

where:

• Y is the dependent variable,
• X is the independent variable,
• β0 is the y-intercept of the regression line,
• β1 is the slope of the regression line, which represents the change in the dependent variable for a one-unit change in the independent variable,
• ε represents the error term (also known as the residual errors), depicting the difference between the observed values and the values predicted by the model.

Assumptions

The application of simple linear regression is based on several key assumptions:

• Linearity: The relationship between the independent and dependent variable is linear.
• Independence: Observations are independent of each other.
• Homoscedasticity: The variance of residual is the same for any value of the independent variable.
• Normality: For any fixed value of X, Y is normally distributed.

Estimation Techniques

The parameters of the simple linear regression model (β0 and β1) are typically estimated using the least squares approach. This method minimizes the sum of the squared differences between the observed values and the values predicted by the model.

Goodness-of-Fit

The goodness-of-fit of a linear regression model is typically evaluated using the R-squared statistic, which measures the proportion of the variance in the dependent variable that is predictable from the independent variable.

Applications

Simple linear regression analysis is widely used in various fields, including economics, finance, biology, and engineering, to predict the value of an outcome based on the value of one predictor variable.

Limitations

While simple linear regression is a powerful tool for prediction and analysis, it has limitations. It assumes a linear relationship between the variables, can be affected by outliers, and does not account for the complexity of many real-world situations where the relationship between variables may be influenced by multiple factors or non-linear patterns.