Linear Model Selection and Regularization

This programming assignment will use the Tidy Models platform. It will take a look at regularization models and hyperparameter tuning. These models contain a regularization term. This assignment will use parsnip for model fitting and recipes and workflows to perform the transformations, and tune and dials to tune the hyperparameters of the model.

You will be using the Hitters data set from the ISLR package. You wish to predict the baseball players Salary based on several different characteristics which are included in the data set.

Since you wish to predict Salary, then you need to remove any missing data from that column. Otherwise, you won't be able to run the models.

Set output as Hitters

library$($tidymodels$)$

library$($ISLR$2)$

$$

# Your code here

# Hitters $<-$

Hitters $<-$ ISLR$2$::Hitters

Hitters $<-$ Hitters $\%>\%$ drop$\_$na$($Salary$)$

# your code here

$$

$$ Attaching packages $$ tidymodels $1\mathrm{.}0\mathrm{.}0$

$$ broom $1\mathrm{.}0\mathrm{.}4$ recipes $1\mathrm{.}0\mathrm{.}5$

$$ dials $1\mathrm{.}1\mathrm{.}0$ rsample $1\mathrm{.}1\mathrm{.}1$

$$ dplyr $1\mathrm{.}1\mathrm{.}0$ tibble $3\mathrm{.}2\mathrm{.}0$

$$ ggplot$23\mathrm{.}4\mathrm{.}1$ tidyr $1\mathrm{.}3\mathrm{.}0$

$$ infer $1\mathrm{.}0\mathrm{.}4$ tune $1\mathrm{.}0\mathrm{.}1$

$$ modeldata $1\mathrm{.}1\mathrm{.}0$ workflows $1\mathrm{.}1\mathrm{.}3$

$$ parsnip $1\mathrm{.}0\mathrm{.}4$ workflowsets $1\mathrm{.}0\mathrm{.}0$

$$ purrr $1\mathrm{.}0\mathrm{.}1$ yardstick $1\mathrm{.}1\mathrm{.}0$

$$ Conflicts $$ tidymodels$\_$conflicts$()$

$$ purrr::discard$()$ masks scales::discard$()$

$$ dplyr::filter$()$ masks stats::filter$()$

$$ dplyr::lag$()$ masks stats::lag$()$

$$ recipes::step$()$ masks stats::step$()$

$$ Use suppressPackageStartupMessages$()$ to eliminate package startup messages

# Hidden Tests

You will use the glmnet package to perform ridge regression. parsnip does not have a dedicated function to create a ridge regression model specification. You need to use linear$\_$reg$()$ and set mixture $=0$ to specify a ridge model. The mixture argument specifies the amount of different types of regularization, mixture $=0$ specifies only ridge regularization and mixture $=1$ specifies only lasso regularization.

Setting mixture to a value between $0$ and $1$ lets us use both. When using the glmnet engine you also need to set a penalty to be able to fit the model. You will set this value to $0$ for now, it is not the best value, but you will look at how to select the best value in a little bit.

ridge$\_$spec $<-$ linear$\_$reg$($mixture $=0,$ penalty $=0)\%>\%$

set$\_$mode$("$regression$")\%>\%$

set$\_$engine$("$glmnet$")$

Once the specification is created you can fit it to you data. You will use all the predictors. Use the fit function here.

ridge$\_$fit $<-$ fit$($ridge$\_$spec, Salary ~ $.,$ data $=$ Hitters$)$

The glmnet package will fit the model for all values of penalty at once, so you can now see see what the parameter estimate for the model is now that you have penalty $=0.$ You can use the tidy function to accomplish this specific task.

tidy$($ridge$\_$fit$)$

Loading required package: Matrix

Attaching package: $$Matrix$$

The following objects are masked from $$package:tidyr$$:

expand, pack, unpack

Loaded glmnet $4\mathrm{.}1-6$

A tibble: $20\backslash$times $3$

term estimate penalty

$($Intercept$)8\mathrm{.}112693$e$+010$

AtBat $-6\mathrm{.}815959$e$-010$

Hits $2\mathrm{.}772312$e$+000$

HmRun $-1\mathrm{.}365680$e$+000$

Runs $1\mathrm{.}014826$e$+000$

RBI $7\mathrm{.}130224$e$-010$

Walks $3\mathrm{.}378558$e$+000$

Years $-9\mathrm{.}066800$e$+000$

CAtBat $-1\mathrm{.}199478$e$-030$

CHits $1\mathrm{.}361029$e$-010$

CHmRun $6\mathrm{.}979958$e$-010$

CRuns $2\mathrm{.}958896$e$-010$

CRBI $2\mathrm{.}570711$e$-010$

CWalks $-2\mathrm{.}789666$e$-010$

LeagueN $5\mathrm{.}321272$e$+010$

DivisionW $-1\mathrm{.}228345$e$+020$

PutOuts $2\mathrm{.}638876$e$-010$

Assists $1\mathrm{.}698796$e$-010$

Errors $-3\mathrm{.}685645$e$+000$

NewLeagueN $-1\mathrm{.}810510$e$+010$

Let us instead see what the estimates would be if the penalty was $11498.$ Store your output to tidy$2.$ What do you notice?

# Your code here

$$

# tidy$2<-$

tidy$2<-$ tidy$($

linear$\_$reg$($penalty $=11498,$ mixture $=0)\%>\%$

set$\_$mode$("$regression$")\%>\%$

set$\_$engine$("$glmnet$")\%>\%$

fit$($Salary ~ $.,$ data $=$ Hitters$)$

$)$

$$

# Print the parameter estimates for penalty $=11498$

tidy$2$

# your code here

$$

A tibble: $20\backslash$times $3$

term estimate penalty

$($Intercept$)407\mathrm{.}20593677411498$

AtBat $0\mathrm{.}03700308311498$

Hits $0\mathrm{.}13835755211498$

HmRun $0\mathrm{.}52519550811498$

Runs $0\mathrm{.}23097829011498$

RBI $0\mathrm{.}24011477511498$

Walks $0\mathrm{.}28997155511498$

Years $1\mathrm{.}10883239911498$

CAtBat $0\mathrm{.}00313521511498$

CHits $0\mathrm{.}01166668411498$

CHmRun $0\mathrm{.}08764278911498$

CRuns $0\mathrm{.}02340625811498$

CRBI $0\mathrm{.}02416572311498$

CWalks $0\mathrm{.}02504211711498$

LeagueN $0\mathrm{.}08662923411498$

DivisionW $-6\mathrm{.}22543133211498$

PutOuts $0\mathrm{.}01650659611498$

Assists $0\mathrm{.}00261633511498$

Errors $-0\mathrm{.}02056415811498$

NewLeagueN $0\mathrm{.}30292289911498$

# Hidden Tests

Look below at the parameter estimates for penalty $=705.$ Store your output to tidy$3.$ Once again, use the tidy function to accomplish this task.

# Your code here

$$

# tidy$3<-$

tidy$3<-$ tidy$($

linear$\_$reg$($penalty $=705,$ mixture $=0)\%>\%$

set$\_$mode$("$regression$")\%>\%$

set$\_$engine$("$glmnet$")\%>\%$

fit$($Salary ~ $.,$ data $=$ Hitters$)$

$)$

$$

# Print the parameter estimates for penalty $=705$

tidy$3$

# your code here

$$

A tibble: $20\backslash$times $3$

term estimat