December 24, 2020
In lasso regression the loss function in addition to RSS (or other error function) has lambda (a constant) times summation of coefficients of independent variables. By doing this we make loss proportional to magnitude of coefficients of independent variables and using the gradient descent we get the new value of coefficients which are lower than their previous values. Thus reducing overfitting.
It is generally used when the multicollinearity is relatively high. Unlike ridge regression, lasso does reduce the value of some coefficients to zero, thus performing feature selection and treating multicollinearity. It is especially useful in descriptive models. The value of lambda is a hyperparameter and is calculated during cross validation i.e. to arrive at the optimal value hyperparameter tuning is to be performed.
by : Monis Khan
Quick Summary:
In lasso regression the loss function in addition to RSS (or other error function) has lambda (a constant) times summation of coefficients of independent variables. By doing this we make loss proportional to magnitude of coefficients of independent variables and using the gradient descent we get the new value of coefficients which are lower than […]