Ridge regression

1. What is ridge regression in simple terms?
2. What does a ridge regression do?
3. What is ridge regression vs linear regression?
4. Why it is called ridge regression?

What is ridge regression in simple terms?

Ridge regression is a method of estimating the coefficients of multiple-regression models in scenarios where the independent variables are highly correlated. It has been used in many fields including econometrics, chemistry, and engineering.

What does a ridge regression do?

Ridge regression aims at reducing the standard error by adding some bias in the estimates of the regression. The reduction of the standard error in regression estimates significantly increases the reliability of the estimates.

What is ridge regression vs linear regression?

Linear Regression establishes a relationship between dependent variable (Y) and one or more independent variables (X) using a best fit straight line (also known as regression line). Ridge Regression is a technique used when the data suffers from multicollinearity ( independent variables are highly correlated).

Why it is called ridge regression?

Ridge regression adds a ridge parameter (k), of the identity matrix to the cross product matrix, forming a new matrix (X`X + kI). It's called ridge regression because the diagonal of ones in the correlation matrix can be described as a ridge.