Building a Regression Tree

1. Use Recursive Binary Splitting to grow the Tree
2. Apply Cost Complexity Pruning. We obtain trees as a function of $\alpha$ by minimizing the equation in Cost Complexity Pruning.
3. Use K-fold Cross Validation to find the best $\alpha$. Average the results for $\alpha$ and pick some $\alpha$ to minimize the MSE.

K-Fold Cross Validation

For each fold $i$ for $i\in 1,2,...k$ we will have scores $S_{{\alpha }_1}^i,S_{{\alpha }_2}^i,S_{{\alpha }_3}^i$. Then for some ${\alpha }_p$ and score $S_{{\alpha }_p}^i$ we will average those scores take $\alpha$ which produced the smallest scores.