Introduction to SuperML

Manish Saraswat

2019-11-28

SuperML R package is designed to unify the model training process in R like Python. Generally, it’s seen that people spend lot of time in searching for packages, figuring out the syntax for training machine learning models in R. This behaviour is highly apparent in users who frequently switch between R and Python. This package provides a python´s scikit-learn interface (fit, predict) to train models faster.

In addition to building machine learning models, there are handy functionalities to do feature engineering

This ambitious package is my ongoing effort to help the r-community build ML models easily and faster in R.

Install

You can install latest cran version using (recommended):

install.packages("superml")

You can install the developmemt version directly from github using:

devtools::install_github("saraswatmks/superml")

Examples - Machine Learning Models

This package uses existing r-packages to build machine learning model. In this tutorial, we’ll use data.table R package to do all tasks related to data manipulation.

Regression Data

We’ll quickly prepare the data set to be ready to served for model training.

load("../data/reg_train.rda")
# if the above doesn't work, you can try: load("reg_train.rda")

library(data.table)
library(caret)
#> Loading required package: lattice
#> Loading required package: ggplot2
library(superml)
#> Loading required package: R6

library(Metrics)
#> 
#> Attaching package: 'Metrics'
#> The following objects are masked from 'package:caret':
#> 
#>     precision, recall

head(reg_train)
#>    Id MSSubClass MSZoning LotFrontage LotArea Street Alley LotShape LandContour
#> 1:  1         60       RL          65    8450   Pave  <NA>      Reg         Lvl
#> 2:  2         20       RL          80    9600   Pave  <NA>      Reg         Lvl
#> 3:  3         60       RL          68   11250   Pave  <NA>      IR1         Lvl
#> 4:  4         70       RL          60    9550   Pave  <NA>      IR1         Lvl
#> 5:  5         60       RL          84   14260   Pave  <NA>      IR1         Lvl
#> 6:  6         50       RL          85   14115   Pave  <NA>      IR1         Lvl
#>    Utilities LotConfig LandSlope Neighborhood Condition1 Condition2 BldgType
#> 1:    AllPub    Inside       Gtl      CollgCr       Norm       Norm     1Fam
#> 2:    AllPub       FR2       Gtl      Veenker      Feedr       Norm     1Fam
#> 3:    AllPub    Inside       Gtl      CollgCr       Norm       Norm     1Fam
#> 4:    AllPub    Corner       Gtl      Crawfor       Norm       Norm     1Fam
#> 5:    AllPub       FR2       Gtl      NoRidge       Norm       Norm     1Fam
#> 6:    AllPub    Inside       Gtl      Mitchel       Norm       Norm     1Fam
#>    HouseStyle OverallQual OverallCond YearBuilt YearRemodAdd RoofStyle RoofMatl
#> 1:     2Story           7           5      2003         2003     Gable  CompShg
#> 2:     1Story           6           8      1976         1976     Gable  CompShg
#> 3:     2Story           7           5      2001         2002     Gable  CompShg
#> 4:     2Story           7           5      1915         1970     Gable  CompShg
#> 5:     2Story           8           5      2000         2000     Gable  CompShg
#> 6:     1.5Fin           5           5      1993         1995     Gable  CompShg
#>    Exterior1st Exterior2nd MasVnrType MasVnrArea ExterQual ExterCond Foundation
#> 1:     VinylSd     VinylSd    BrkFace        196        Gd        TA      PConc
#> 2:     MetalSd     MetalSd       None          0        TA        TA     CBlock
#> 3:     VinylSd     VinylSd    BrkFace        162        Gd        TA      PConc
#> 4:     Wd Sdng     Wd Shng       None          0        TA        TA     BrkTil
#> 5:     VinylSd     VinylSd    BrkFace        350        Gd        TA      PConc
#> 6:     VinylSd     VinylSd       None          0        TA        TA       Wood
#>    BsmtQual BsmtCond BsmtExposure BsmtFinType1 BsmtFinSF1 BsmtFinType2
#> 1:       Gd       TA           No          GLQ        706          Unf
#> 2:       Gd       TA           Gd          ALQ        978          Unf
#> 3:       Gd       TA           Mn          GLQ        486          Unf
#> 4:       TA       Gd           No          ALQ        216          Unf
#> 5:       Gd       TA           Av          GLQ        655          Unf
#> 6:       Gd       TA           No          GLQ        732          Unf
#>    BsmtFinSF2 BsmtUnfSF TotalBsmtSF Heating HeatingQC CentralAir Electrical
#> 1:          0       150         856    GasA        Ex          Y      SBrkr
#> 2:          0       284        1262    GasA        Ex          Y      SBrkr
#> 3:          0       434         920    GasA        Ex          Y      SBrkr
#> 4:          0       540         756    GasA        Gd          Y      SBrkr
#> 5:          0       490        1145    GasA        Ex          Y      SBrkr
#> 6:          0        64         796    GasA        Ex          Y      SBrkr
#>    1stFlrSF 2ndFlrSF LowQualFinSF GrLivArea BsmtFullBath BsmtHalfBath FullBath
#> 1:      856      854            0      1710            1            0        2
#> 2:     1262        0            0      1262            0            1        2
#> 3:      920      866            0      1786            1            0        2
#> 4:      961      756            0      1717            1            0        1
#> 5:     1145     1053            0      2198            1            0        2
#> 6:      796      566            0      1362            1            0        1
#>    HalfBath BedroomAbvGr KitchenAbvGr KitchenQual TotRmsAbvGrd Functional
#> 1:        1            3            1          Gd            8        Typ
#> 2:        0            3            1          TA            6        Typ
#> 3:        1            3            1          Gd            6        Typ
#> 4:        0            3            1          Gd            7        Typ
#> 5:        1            4            1          Gd            9        Typ
#> 6:        1            1            1          TA            5        Typ
#>    Fireplaces FireplaceQu GarageType GarageYrBlt GarageFinish GarageCars
#> 1:          0        <NA>     Attchd        2003          RFn          2
#> 2:          1          TA     Attchd        1976          RFn          2
#> 3:          1          TA     Attchd        2001          RFn          2
#> 4:          1          Gd     Detchd        1998          Unf          3
#> 5:          1          TA     Attchd        2000          RFn          3
#> 6:          0        <NA>     Attchd        1993          Unf          2
#>    GarageArea GarageQual GarageCond PavedDrive WoodDeckSF OpenPorchSF
#> 1:        548         TA         TA          Y          0          61
#> 2:        460         TA         TA          Y        298           0
#> 3:        608         TA         TA          Y          0          42
#> 4:        642         TA         TA          Y          0          35
#> 5:        836         TA         TA          Y        192          84
#> 6:        480         TA         TA          Y         40          30
#>    EnclosedPorch 3SsnPorch ScreenPorch PoolArea PoolQC Fence MiscFeature
#> 1:             0         0           0        0   <NA>  <NA>        <NA>
#> 2:             0         0           0        0   <NA>  <NA>        <NA>
#> 3:             0         0           0        0   <NA>  <NA>        <NA>
#> 4:           272         0           0        0   <NA>  <NA>        <NA>
#> 5:             0         0           0        0   <NA>  <NA>        <NA>
#> 6:             0       320           0        0   <NA> MnPrv        Shed
#>    MiscVal MoSold YrSold SaleType SaleCondition SalePrice
#> 1:       0      2   2008       WD        Normal    208500
#> 2:       0      5   2007       WD        Normal    181500
#> 3:       0      9   2008       WD        Normal    223500
#> 4:       0      2   2006       WD       Abnorml    140000
#> 5:       0     12   2008       WD        Normal    250000
#> 6:     700     10   2009       WD        Normal    143000

split <- createDataPartition(y = reg_train$SalePrice, p = 0.7)
xtrain <- reg_train[split$Resample1]
xtest <- reg_train[!split$Resample1]
# remove features with 90% or more missing values
# we will also remove the Id column because it doesn't contain
# any useful information
na_cols <- colSums(is.na(xtrain)) / nrow(xtrain)
na_cols <- names(na_cols[which(na_cols > 0.9)])

xtrain[, c(na_cols, "Id") := NULL]
xtest[, c(na_cols, "Id") := NULL]

# encode categorical variables
cat_cols <- names(xtrain)[sapply(xtrain, is.character)]

for(c in cat_cols){
    lbl <- LabelEncoder$new()
    lbl$fit(c(xtrain[[c]], xtest[[c]]))
    xtrain[[c]] <- lbl$transform(xtrain[[c]])
    xtest[[c]] <- lbl$transform(xtest[[c]])
}
#> The data contains NA values. Imputing NA with 'NA' 
#> The data contains NA values. Imputing NA with 'NA' 
#> The data contains NA values. Imputing NA with 'NA' 
#> The data contains NA values. Imputing NA with 'NA' 
#> The data contains NA values. Imputing NA with 'NA' 
#> The data contains NA values. Imputing NA with 'NA' 
#> The data contains NA values. Imputing NA with 'NA' 
#> The data contains NA values. Imputing NA with 'NA' 
#> The data contains NA values. Imputing NA with 'NA' 
#> The data contains NA values. Imputing NA with 'NA' 
#> The data contains NA values. Imputing NA with 'NA' 
#> The data contains NA values. Imputing NA with 'NA' 
#> The data contains NA values. Imputing NA with 'NA' 
#> The data contains NA values. Imputing NA with 'NA' 
#> The data contains NA values. Imputing NA with 'NA' 
#> The data contains NA values. Imputing NA with 'NA' 
#> The data contains NA values. Imputing NA with 'NA' 
#> The data contains NA values. Imputing NA with 'NA' 
#> The data contains NA values. Imputing NA with 'NA' 
#> The data contains NA values. Imputing NA with 'NA' 
#> The data contains NA values. Imputing NA with 'NA' 
#> The data contains NA values. Imputing NA with 'NA' 
#> The data contains NA values. Imputing NA with 'NA' 
#> The data contains NA values. Imputing NA with 'NA' 
#> The data contains NA values. Imputing NA with 'NA' 
#> The data contains NA values. Imputing NA with 'NA' 
#> The data contains NA values. Imputing NA with 'NA' 
#> The data contains NA values. Imputing NA with 'NA' 
#> The data contains NA values. Imputing NA with 'NA' 
#> The data contains NA values. Imputing NA with 'NA' 
#> The data contains NA values. Imputing NA with 'NA' 
#> The data contains NA values. Imputing NA with 'NA' 
#> The data contains NA values. Imputing NA with 'NA' 
#> The data contains NA values. Imputing NA with 'NA' 
#> The data contains NA values. Imputing NA with 'NA' 
#> The data contains NA values. Imputing NA with 'NA' 
#> The data contains NA values. Imputing NA with 'NA' 
#> The data contains NA values. Imputing NA with 'NA'

# removing noise column
noise <- c('GrLivArea','TotalBsmtSF')

xtrain[, c(noise) := NULL]
xtest[, c(noise) := NULL]

# fill missing value with  -1
xtrain[is.na(xtrain)] <- -1
xtest[is.na(xtest)] <- -1

KNN Regression

SVM Regression

Simple Regresison

lf <- LMTrainer$new(family="gaussian")
lf$fit(X = xtrain, y = "SalePrice")
summary(lf$model)
#> 
#> Call:
#> stats::glm(formula = f, family = self$family, data = X, weights = self$weights)
#> 
#> Deviance Residuals: 
#>     Min       1Q   Median       3Q      Max  
#> -339890   -15323    -1318    12807   250963  
#> 
#> Coefficients:
#>                 Estimate Std. Error t value Pr(>|t|)    
#> (Intercept)   -1.195e+06  1.638e+06  -0.729 0.465962    
#> MSSubClass    -2.242e+02  6.628e+01  -3.382 0.000749 ***
#> MSZoning       1.092e+03  1.540e+03   0.709 0.478447    
#> LotFrontage   -4.871e+01  3.475e+01  -1.402 0.161268    
#> LotArea        4.661e-01  1.298e-01   3.591 0.000346 ***
#> Street        -3.448e+04  1.740e+04  -1.981 0.047849 *  
#> LotShape       4.938e+02  2.187e+03   0.226 0.821389    
#> LandContour   -7.851e+02  2.197e+03  -0.357 0.720844    
#> Utilities     -6.306e+04  3.504e+04  -1.800 0.072220 .  
#> LotConfig      5.514e+02  1.429e+03   0.386 0.699782    
#> LandSlope      5.230e+03  5.165e+03   1.013 0.311535    
#> Neighborhood  -5.642e+01  1.876e+02  -0.301 0.763710    
#> Condition1    -2.654e+03  9.116e+02  -2.911 0.003681 ** 
#> Condition2    -1.060e+04  3.385e+03  -3.133 0.001783 ** 
#> BldgType       2.757e+03  2.818e+03   0.978 0.328252    
#> HouseStyle     1.497e+02  9.619e+02   0.156 0.876401    
#> OverallQual    1.377e+04  1.442e+03   9.547  < 2e-16 ***
#> OverallCond    5.256e+03  1.279e+03   4.109 4.31e-05 ***
#> YearBuilt      3.357e+02  8.213e+01   4.088 4.72e-05 ***
#> YearRemodAdd   1.312e+02  8.670e+01   1.513 0.130639    
#> RoofStyle      4.687e+03  2.225e+03   2.106 0.035423 *  
#> RoofMatl      -1.515e+04  3.200e+03  -4.735 2.53e-06 ***
#> Exterior1st    7.662e+01  5.791e+02   0.132 0.894780    
#> Exterior2nd    1.645e+02  6.367e+02   0.258 0.796203    
#> MasVnrType     2.443e+03  1.736e+03   1.407 0.159808    
#> MasVnrArea     3.306e+01  7.277e+00   4.543 6.25e-06 ***
#> ExterQual      1.017e+03  2.427e+03   0.419 0.675372    
#> ExterCond      3.279e+03  2.450e+03   1.338 0.181078    
#> Foundation    -3.947e+03  1.548e+03  -2.550 0.010929 *  
#> BsmtQual       6.917e+03  1.580e+03   4.378 1.33e-05 ***
#> BsmtCond      -6.143e+03  2.023e+03  -3.037 0.002457 ** 
#> BsmtExposure   4.922e+03  1.048e+03   4.697 3.03e-06 ***
#> BsmtFinType1  -1.187e+03  7.239e+02  -1.639 0.101545    
#> BsmtFinSF1     3.679e+00  6.208e+00   0.593 0.553574    
#> BsmtFinType2  -3.781e+02  1.264e+03  -0.299 0.764917    
#> BsmtFinSF2     8.339e+00  1.095e+01   0.761 0.446649    
#> BsmtUnfSF      1.463e+00  5.833e+00   0.251 0.801986    
#> Heating       -1.677e+03  3.957e+03  -0.424 0.671797    
#> HeatingQC     -1.338e+03  1.484e+03  -0.902 0.367528    
#> CentralAir     6.649e+03  5.696e+03   1.167 0.243434    
#> Electrical     7.839e+01  2.282e+03   0.034 0.972597    
#> `1stFlrSF`     5.272e+01  7.348e+00   7.175 1.46e-12 ***
#> `2ndFlrSF`     5.011e+01  6.185e+00   8.103 1.65e-15 ***
#> LowQualFinSF   4.548e+01  2.161e+01   2.104 0.035630 *  
#> BsmtFullBath   1.067e+04  3.127e+03   3.413 0.000670 ***
#> BsmtHalfBath   4.019e+03  4.636e+03   0.867 0.386155    
#> FullBath       8.986e+03  3.344e+03   2.687 0.007335 ** 
#> HalfBath       9.154e+02  3.151e+03   0.291 0.771453    
#> BedroomAbvGr  -5.486e+03  2.011e+03  -2.728 0.006491 ** 
#> KitchenAbvGr  -2.481e+04  5.959e+03  -4.164 3.42e-05 ***
#> KitchenQual    7.359e+03  1.867e+03   3.942 8.68e-05 ***
#> TotRmsAbvGrd   4.127e+03  1.430e+03   2.886 0.003995 ** 
#> Functional    -5.056e+03  1.692e+03  -2.988 0.002876 ** 
#> Fireplaces    -3.645e+03  2.747e+03  -1.327 0.184893    
#> FireplaceQu    5.295e+03  1.461e+03   3.623 0.000306 ***
#> GarageType    -1.595e+03  1.290e+03  -1.237 0.216431    
#> GarageYrBlt   -6.246e+00  4.742e+00  -1.317 0.188070    
#> GarageFinish   1.039e+03  1.530e+03   0.679 0.497258    
#> GarageCars     1.247e+04  3.390e+03   3.677 0.000249 ***
#> GarageArea     8.377e+00  1.114e+01   0.752 0.452356    
#> GarageQual     5.224e+03  2.257e+03   2.315 0.020829 *  
#> GarageCond    -1.755e+03  2.877e+03  -0.610 0.542112    
#> PavedDrive    -2.110e+03  3.438e+03  -0.614 0.539589    
#> WoodDeckSF     2.171e+01  9.324e+00   2.328 0.020118 *  
#> OpenPorchSF   -1.085e+01  1.681e+01  -0.645 0.518909    
#> EnclosedPorch  1.356e+01  1.956e+01   0.693 0.488349    
#> `3SsnPorch`    4.777e+01  3.952e+01   1.209 0.227103    
#> ScreenPorch    8.152e+01  1.886e+01   4.323 1.70e-05 ***
#> PoolArea      -1.398e+02  3.339e+01  -4.187 3.10e-05 ***
#> Fence         -7.729e+02  1.124e+03  -0.688 0.491915    
#> MiscVal        5.601e-01  1.990e+00   0.281 0.778420    
#> MoSold        -1.505e+02  3.995e+02  -0.377 0.706492    
#> YrSold         1.151e+02  8.142e+02   0.141 0.887568    
#> SaleType       2.828e+03  1.357e+03   2.084 0.037447 *  
#> SaleCondition  1.312e+03  1.301e+03   1.008 0.313565    
#> ---
#> Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
#> 
#> (Dispersion parameter for gaussian family taken to be 1062927255)
#> 
#>     Null deviance: 6.6562e+12  on 1023  degrees of freedom
#> Residual deviance: 1.0087e+12  on  949  degrees of freedom
#> AIC: 24263
#> 
#> Number of Fisher Scoring iterations: 2
predictions <- lf$predict(df = xtest)
rmse(actual = xtest$SalePrice, predicted = predictions)
#> [1] 34112.1

Lasso Regression

Ridge Regression

Logistic Regression with CV

Random Forest

rf <- RFTrainer$new(n_estimators = 500,classification = 0)
rf$fit(X = xtrain, y = "SalePrice")
pred <- rf$predict(df = xtest)
rf$get_importance()
#>               tmp.order.tmp..decreasing...TRUE..
#> OverallQual                         876335663148
#> 1stFlrSF                            496824285769
#> GarageCars                          496428723445
#> GarageArea                          375950891452
#> YearBuilt                           372595809715
#> FullBath                            334535870077
#> GarageYrBlt                         271502481223
#> 2ndFlrSF                            238506672041
#> TotRmsAbvGrd                        234508183975
#> BsmtFinSF1                          231560790441
#> ExterQual                           229571841020
#> MasVnrArea                          188253292094
#> LotArea                             187929069935
#> YearRemodAdd                        178232672872
#> FireplaceQu                         147653809502
#> KitchenQual                         141720155457
#> Fireplaces                          141232109658
#> BsmtQual                            116273512210
#> Foundation                           98363852614
#> OpenPorchSF                          87715869624
#> LotFrontage                          85282098599
#> BsmtUnfSF                            77180208968
#> BsmtFinType1                         73008736307
#> Neighborhood                         69394532266
#> WoodDeckSF                           60691270268
#> HeatingQC                            56498062396
#> BedroomAbvGr                         55035448679
#> MoSold                               43046764506
#> GarageType                           42130030414
#> Exterior2nd                          41065067463
#> MSSubClass                           39169061611
#> BsmtExposure                         38887073076
#> OverallCond                          37349932072
#> Exterior1st                          32908955958
#> HouseStyle                           30700922286
#> HalfBath                             29944784145
#> LotShape                             25612111728
#> BsmtFullBath                         23577174837
#> RoofStyle                            22592851206
#> GarageFinish                         21792129589
#> YrSold                               21745080963
#> MSZoning                             18527787130
#> LandContour                          17362367795
#> BsmtHalfBath                         15640597540
#> SaleType                             13879412169
#> SaleCondition                        13859845514
#> ScreenPorch                          13214213782
#> LotConfig                            13009691424
#> BldgType                             12984733852
#> MasVnrType                           12927751811
#> Condition1                           12860107386
#> CentralAir                           12443210183
#> RoofMatl                             12301992390
#> GarageQual                           11828494238
#> EnclosedPorch                        10704554730
#> BsmtCond                             10699731310
#> LandSlope                            10683526316
#> KitchenAbvGr                         10306427352
#> GarageCond                            9216481926
#> BsmtFinSF2                            8581199639
#> ExterCond                             7926070704
#> Functional                            6060073754
#> BsmtFinType2                          6004231202
#> PavedDrive                            5306595470
#> Fence                                 5134494920
#> Electrical                            5071077592
#> LowQualFinSF                          4776684987
#> Condition2                            3624570125
#> Heating                               3039478135
#> MiscVal                               2706596509
#> 3SsnPorch                             2160379168
#> PoolArea                              2054262842
#> Street                                 462732232
#> Utilities                               35919859
rmse(actual = xtest$SalePrice, predicted = pred)
#> [1] 28487.59

Xgboost

Grid Search

xgb <- XGBTrainer$new(objective ="reg:linear")

gst <-GridSearchCV$new(trainer = xgb,
                             parameters = list(n_estimators = c(10,50), max_depth = c(5,2)),
                             n_folds = 3,
                             scoring = c('accuracy','auc'))
gst$fit(xtrain, "SalePrice")
#> [1] "entering grid search"
#> [1] "In total, 4 models will be trained"
#> converting the data into xgboost format..
#> starting with training...
#> [1]  train-rmse:143770.906250 
#> Will train until train_rmse hasn't improved in 50 rounds.
#> 
#> [10] train-rmse:16830.447266
#> converting the data into xgboost format..
#> starting with training...
#> [1]  train-rmse:143539.765625 
#> Will train until train_rmse hasn't improved in 50 rounds.
#> 
#> [10] train-rmse:15880.406250
#> converting the data into xgboost format..
#> starting with training...
#> [1]  train-rmse:141585.953125 
#> Will train until train_rmse hasn't improved in 50 rounds.
#> 
#> [10] train-rmse:15536.412109
#> converting the data into xgboost format..
#> starting with training...
#> [1]  train-rmse:143770.906250 
#> Will train until train_rmse hasn't improved in 50 rounds.
#> 
#> [50] train-rmse:3607.689697
#> converting the data into xgboost format..
#> starting with training...
#> [1]  train-rmse:143539.765625 
#> Will train until train_rmse hasn't improved in 50 rounds.
#> 
#> [50] train-rmse:3777.051514
#> converting the data into xgboost format..
#> starting with training...
#> [1]  train-rmse:141585.953125 
#> Will train until train_rmse hasn't improved in 50 rounds.
#> 
#> [50] train-rmse:3666.455811
#> converting the data into xgboost format..
#> starting with training...
#> [1]  train-rmse:144819.453125 
#> Will train until train_rmse hasn't improved in 50 rounds.
#> 
#> [10] train-rmse:32198.896484
#> converting the data into xgboost format..
#> starting with training...
#> [1]  train-rmse:144439.859375 
#> Will train until train_rmse hasn't improved in 50 rounds.
#> 
#> [10] train-rmse:31735.691406
#> converting the data into xgboost format..
#> starting with training...
#> [1]  train-rmse:142559.171875 
#> Will train until train_rmse hasn't improved in 50 rounds.
#> 
#> [10] train-rmse:30720.810547
#> converting the data into xgboost format..
#> starting with training...
#> [1]  train-rmse:144819.453125 
#> Will train until train_rmse hasn't improved in 50 rounds.
#> 
#> [50] train-rmse:18957.185547
#> converting the data into xgboost format..
#> starting with training...
#> [1]  train-rmse:144439.859375 
#> Will train until train_rmse hasn't improved in 50 rounds.
#> 
#> [50] train-rmse:17248.701172
#> converting the data into xgboost format..
#> starting with training...
#> [1]  train-rmse:142559.171875 
#> Will train until train_rmse hasn't improved in 50 rounds.
#> 
#> [50] train-rmse:16948.439453
gst$best_iteration()
#> $n_estimators
#> [1] 10
#> 
#> $max_depth
#> [1] 5
#> 
#> $accuracy_avg
#> [1] 0
#> 
#> $accuracy_sd
#> [1] 0
#> 
#> $auc_avg
#> [1] NaN
#> 
#> $auc_sd
#> [1] NA

Random Search

Binary Classification Data

Here, we will solve a simple binary classification problem (predict people who survived on titanic ship). The idea here is to demonstrate how to use this package to solve classification problems.

Data Preparation

# load class
load('../data/cla_train.rda')
# if the above doesn't work, you can try: load("cla_train.rda")

head(cla_train)
#>    PassengerId Survived Pclass
#> 1:           1        0      3
#> 2:           2        1      1
#> 3:           3        1      3
#> 4:           4        1      1
#> 5:           5        0      3
#> 6:           6        0      3
#>                                                   Name    Sex Age SibSp Parch
#> 1:                             Braund, Mr. Owen Harris   male  22     1     0
#> 2: Cumings, Mrs. John Bradley (Florence Briggs Thayer) female  38     1     0
#> 3:                              Heikkinen, Miss. Laina female  26     0     0
#> 4:        Futrelle, Mrs. Jacques Heath (Lily May Peel) female  35     1     0
#> 5:                            Allen, Mr. William Henry   male  35     0     0
#> 6:                                    Moran, Mr. James   male  NA     0     0
#>              Ticket    Fare Cabin Embarked
#> 1:        A/5 21171  7.2500              S
#> 2:         PC 17599 71.2833   C85        C
#> 3: STON/O2. 3101282  7.9250              S
#> 4:           113803 53.1000  C123        S
#> 5:           373450  8.0500              S
#> 6:           330877  8.4583              Q

# split the data
split <- createDataPartition(y = cla_train$Survived,p = 0.7)
xtrain <- cla_train[split$Resample1]
xtest <- cla_train[!split$Resample1]

# encode categorical variables - shorter way
for(c in c('Embarked','Sex','Cabin')){
    lbl <- LabelEncoder$new()
    lbl$fit(c(xtrain[[c]], xtest[[c]]))
    xtrain[[c]] <- lbl$transform(xtrain[[c]])
    xtest[[c]] <- lbl$transform(xtest[[c]])
}
#> The data contains blank values. Imputing them with 'NA' 
#> The data contains blank values. Imputing them with 'NA' 
#> The data contains blank values. Imputing them with 'NA' 
#> The data contains blank values. Imputing them with 'NA' 
#> The data contains blank values. Imputing them with 'NA'

# impute missing values
xtrain[, Age := replace(Age, is.na(Age), median(Age, na.rm = T))]
xtest[, Age := replace(Age, is.na(Age), median(Age, na.rm = T))]

# drop these features
to_drop <- c('PassengerId','Ticket','Name')

xtrain <- xtrain[,-c(to_drop), with=F]
xtest <- xtest[,-c(to_drop), with=F]

Now, our data is ready to be served for model training. Let’s do it.

KNN Classification

Naive Bayes Classification

SVM Classification

Logistic Regression

Lasso Logistic Regression

Ridge Logistic Regression

Random Forest

Xgboost

Grid Search

xgb <- XGBTrainer$new(objective="binary:logistic")
gst <-GridSearchCV$new(trainer = xgb,
                             parameters = list(n_estimators = c(10,50),
                             max_depth = c(5,2)),
                             n_folds = 3,
                             scoring = c('accuracy','auc'))
gst$fit(xtrain, "Survived")
#> [1] "entering grid search"
#> [1] "In total, 4 models will be trained"
#> converting the data into xgboost format..
#> starting with training...
#> [1]  train-error:0.144231 
#> Will train until train_error hasn't improved in 50 rounds.
#> 
#> [10] train-error:0.108173
#> converting the data into xgboost format..
#> starting with training...
#> [1]  train-error:0.134615 
#> Will train until train_error hasn't improved in 50 rounds.
#> 
#> [10] train-error:0.112981
#> converting the data into xgboost format..
#> starting with training...
#> [1]  train-error:0.115385 
#> Will train until train_error hasn't improved in 50 rounds.
#> 
#> [10] train-error:0.084135
#> converting the data into xgboost format..
#> starting with training...
#> [1]  train-error:0.144231 
#> Will train until train_error hasn't improved in 50 rounds.
#> 
#> [50] train-error:0.045673
#> converting the data into xgboost format..
#> starting with training...
#> [1]  train-error:0.134615 
#> Will train until train_error hasn't improved in 50 rounds.
#> 
#> [50] train-error:0.045673
#> converting the data into xgboost format..
#> starting with training...
#> [1]  train-error:0.115385 
#> Will train until train_error hasn't improved in 50 rounds.
#> 
#> [50] train-error:0.038462
#> converting the data into xgboost format..
#> starting with training...
#> [1]  train-error:0.211538 
#> Will train until train_error hasn't improved in 50 rounds.
#> 
#> [10] train-error:0.158654
#> converting the data into xgboost format..
#> starting with training...
#> [1]  train-error:0.201923 
#> Will train until train_error hasn't improved in 50 rounds.
#> 
#> [10] train-error:0.168269
#> converting the data into xgboost format..
#> starting with training...
#> [1]  train-error:0.206731 
#> Will train until train_error hasn't improved in 50 rounds.
#> 
#> [10] train-error:0.141827
#> converting the data into xgboost format..
#> starting with training...
#> [1]  train-error:0.211538 
#> Will train until train_error hasn't improved in 50 rounds.
#> 
#> [50] train-error:0.127404
#> converting the data into xgboost format..
#> starting with training...
#> [1]  train-error:0.201923 
#> Will train until train_error hasn't improved in 50 rounds.
#> 
#> [50] train-error:0.132212
#> converting the data into xgboost format..
#> starting with training...
#> [1]  train-error:0.206731 
#> Will train until train_error hasn't improved in 50 rounds.
#> 
#> [50] train-error:0.108173
gst$best_iteration()
#> $n_estimators
#> [1] 10
#> 
#> $max_depth
#> [1] 5
#> 
#> $accuracy_avg
#> [1] 0
#> 
#> $accuracy_sd
#> [1] 0
#> 
#> $auc_avg
#> [1] 0.8619512
#> 
#> $auc_sd
#> [1] 0.02280628

Random Search

Let’s create some new feature based on target variable using target encoding and test a model.