Decision Trees and Ensemble Methods

A dataset is given and you must fit a decision tree using rpart() and then prune it.

Use rpart(y ~ ., data = df, method = "anova" or "class") with control parameters cp (complexity) and minsplit. Examine the cp table with printcp() and select the cp at the minimum xerror (or one-SE rule). Prune with prune(tree, cp = chosen_cp). Visualize with rpart.plot() and interpret each split path.

rpart(formula, method, control); printcp() then prune(tree, cp).

You must fit a random forest and a gradient boosting model and tune key hyperparameters.

Random forest with randomForest(): tune mtry (number of variables tried at each split) and ntree. Use OOB error to select. GBM with gbm(): tune n.trees, interaction.depth, shrinkage. Smaller shrinkage with more trees usually generalizes better. Use a validation set or CV to select all hyperparameters; report final OOB or CV error.

RF: tune mtry, ntree. GBM: tune n.trees, depth, shrinkage. Validate.

You have fitted both a GLM and a tree-ensemble model and must choose one for production.

Compare on holdout metrics (RMSE, AUC, log loss) using the same validation set. Consider interpretability — GLMs are easier to explain to underwriters and regulators. Consider data size and feature interactions — trees handle non-linearities and high-order interactions automatically. State the tradeoff explicitly in the recommendation.

Choose based on (predictive performance, interpretability, regulatory acceptance, complexity).