The Orange Book of Machine Learning The essentials of making predictions using supervised regression and classification for… (Carl McBride Ellis)（Z-Library）

【One-Line Pitch】 A hands-on guide to supervised machine learning on tabular data, walking you from data cleaning and cross-validation through regression, classification, and modern gradient-boosting and foundation models. Best for practitioners who already know some Python and want a rigorous, workflow-oriented reference for building and evaluating predictive models. 【Book Arc】 - **Opening (~0%–15%)**: Frames core vocabulary and statistical foundations — estimators vs. approximators, errors vs. residuals, aleatoric vs. epistemic uncertainty, and descriptive statistics/visualization (histograms, box/violin plots, correlation coefficients). - **Early (~15%–30%)**: Covers the unglamorous but decisive preprocessing work: missing-value handling (MCAR/MAR/MNAR, imputation strategies), outlier detection, encoding, scaling, and cross-validation design including data leakage and covariate shift. - **Middle (~30%–55%)**: Builds the two core supervised tasks — regression (linear models, trees, regularization, bias-variance, quantile and conformal prediction intervals) and classification (logistic regression, decision trees, probabilistic metrics, calibration, multiclass and imbalanced data). - **Late (~55%–75%)**: Extends into generalized linear/additive models, ensemble estimators (Random Forest, boosting, stacking), and hyperparameter optimization, including the Optimizer's Curse and automated search routines. - **Ending (~75%–100%)**: Moves into feature engineering and selection, then introduces tabular foundation models and in-context learning (TabPFN, TabICL) as the frontier of structured-data prediction. 【Key Takeaways】 - **Supervised tabular ML is a workflow, not a single model choice** (Opening): the book sequences exploration → cleaning → validation → modeling → evaluation → optimization, so you learn where each decision fits. - **Uncertainty deserves explicit treatment** (Early–Middle): the text distinguishes aleatoric from epistemic uncertainty and teaches conformal prediction intervals and quantile regression for honest point-prediction ranges. - **Classification is fundamentally probabilistic** (Middle): calibration, reliability diagrams, strictly proper scoring rules, and Venn-ABERS calibration are treated as first-class concerns, not afterthoughts. - **Cross-validation design prevents self-deception** (Early): nested CV, fold-count bias, permutation invariance, and data leakage are covered to keep out-of-sample estimates trustworthy. - **Hyperparameter optimization has its own failure mode** (Late): the Optimizer's Curse and optimistic bias are named explicitly, with guidance on early stopping and automated search. - **Ensembles and boosting are the practical workhorses** (Late): Random Forest, gradient-boosted decision trees, stacking, and convex combinations of predictions are presented as the default strong baselines for tabular data. - **Feature engineering and selection remain decisive** (Late): interaction terms, selection methods, and preprocessing choices are shown to materially affect performance before any exotic model is tried. - **Tabular foundation models are emerging** (Ending): TabPFN and TabICL with in-context learning are introduced as a newer alternative paradigm for structured data. 【Reading Tips】 - **Deep-read the cross-validation and data-cleaning chapters** (Early): these underpin every later result; skimming them will make the regression and classification chapters harder to trust. - **Skim the statistics refresher if you're already comfortable** (Opening): use it as a vocabulary check rather than a full study pass. - **Treat the regression and classification chapters as parallel tracks** (Middle): read one fully, then the other, comparing how uncertainty and evaluation are handled differently. - **Don't skip the hyperparameter optimization and feature chapters** (Late): they contain the practical judgment calls that separate a working model from a well-tuned one. - **Run the accompanying notebooks** (throughout): the book is code-driven, and the GitHub repository is the intended companion for hands-on practice. 【Coverage Limits】 This guide is synthesized from stratified excerpts covering the table of contents and early chapters; detailed content of later chapters (ensembles, HPO, feature engineering, foundation models) is inferred from section headings rather than full text. Specific code examples, datasets, and numerical results are not covered here.