Statistical regression

Statistical regression is a statistical process for estimating the relationships among variables. It includes many techniques for modeling and analyzing several variables, when the focus is on the relationship between a dependent variable and one or more independent variables (or 'predictors').

History

The term "regression" was coined by Francis Galton in the 19th century to describe a biological phenomenon. The phenomenon was that the heights of descendants of tall ancestors tend to regress down towards a normal average (a phenomenon also known as regression toward the mean).

Types of regression

There are several types of regression analysis. The main types include:

• Linear regression: This is the most basic type of regression. It assumes a linear relationship between the input variables (x) and the single output variable (y).
• Logistic regression: This is used when the dependent variable is binary in nature.
• Polynomial regression: This is a form of regression analysis in which the relationship between the independent variable x and the dependent variable y is modeled as an nth degree polynomial.
• Ridge regression: This is a method of estimating the coefficients of multiple-regression models in scenarios where independent variables are highly correlated.
• Lasso regression: This is a regression analysis method that performs both variable selection and regularization.

Applications

Statistical regression is widely used for prediction and forecasting, where its use has substantial overlap with the field of machine learning. Regression analysis is also used to understand which among the independent variables are related to the dependent variable, and to explore the forms of these relationships.

See also

• Correlation and dependence
• Curve fitting
• Least squares
• Machine learning
• Prediction interval
• Regression toward the mean
• Regression validation