AdaBoost and gradient boosting are the two foundational algorithms behind boosted decision trees. This post explains how each works, what makes them different, and why an ensemble of weak trees outperforms a single deep one.
Analysis review committees see the same boosted decision tree mistakes repeatedly. Here are the five most common ones: overtraining, data-MC disagreement, train-measure overlap, biased cut optimisation, and unpropagated systematics.
Decision trees split nodes by maximising purity. Gini impurity and information gain (entropy) are the two standard criteria. Here is what they measure, how they differ, and why the choice almost never matters in practice.
Overtraining is the single most common failure mode in HEP classifiers. Learn the standard diagnostics: train-test overlay plots, the Kolmogorov-Smirnov test, and what to do when they fail.
Rectangular cuts force axis-aligned selection boundaries. When discriminating variables are correlated, signal lives along diagonals that boxes cannot follow. Here is why multivariate methods beat cuts — and exactly when they matter most.
A short, runnable example: on a two-variable signal-vs-background problem, a boosted decision tree recovers signal that rectangular cuts throw away. Copy the code, run it, see the ROC gap yourself.
Machine learning has its own vocabulary, but most of it maps one-to-one onto concepts a particle physicist already owns. Features are observables, overtraining is overfitting a χ², ROC is efficiency vs purity. A translation table.