- Binary classification: did the driver fail to finish (DNF)?
- Multiclass classification: DNF / P11+ / P4–P10 / Podium / Winner
- Random Forest and ANN (MLP) for both tasks, tuned with
GridSearchCV - Per-hyperparameter validation curves, learning curves, and confusion matrices
- Stratified 5-fold CV with fixed seeds for full reproducibility
Overview
A machine learning pipeline that predicts Formula 1 race outcomes from the Kaggle / Ergast F1 dataset (1950–2024). Two classification tasks, four tuned models, full hyperparameter sweeps with validation and learning curves. Final project for CSC 272 at Hofstra, completed with my teammate Michael Campos. This writeup covers the Random Forest and ANN half of the project (main.py); KNN, SVM, and Decision Tree results live in the accompanying report.
Tasks
| Task | Target | Era | Samples | Primary metric |
|---|---|---|---|---|
| Binary | dnf (did the driver fail to finish?) | 1958–2024 | 24,618 | Weighted F1 + AUC-ROC |
| Multiclass | positionTier (DNF / P11+ / P4–P10 / Podium / Winner) | 2012–2024 | 5,082 | Macro F1 |
The multiclass task is restricted to 2012+ because bestQualiTime and pitStopCount are only reliably populated from then on.
Headline Results
| Task | Model | Train F1 | CV Val F1 | Test F1 | Test AUC |
|---|---|---|---|---|---|
| Binary | Random Forest | 0.956 | 0.812 | 0.804 | 0.877 |
| Binary | ANN (MLP) | 0.821 | 0.812 | 0.802 | 0.874 |
| Multiclass | Random Forest | 1.000 | 0.491 | 0.492 | N/A |
| Multiclass | ANN (MLP) | 0.503 | 0.450 | 0.436 | N/A |
The binary task hits a shared ceiling near F1 ≈ 0.80; the multiclass task is data-limited (5,082 samples) and dominated by adjacent-tier confusion (P2–P3 ↔ P1, P4–P10 ↔ P11+).
Features
Binary (six pre-race features, no post-race leakage):
grid, driverAge, driverStandingPos, constructorPoints, year, round.
Multiclass (six pace and strategy features):
grid, driverPoints, constructorPoints, bestQualiTime, pitStopCount, year.
Hyperparameter Grids
| Model | Hyperparameters |
|---|---|
| Binary RF | n_estimators ∈ 300, max_depth ∈ 25 |
| Binary ANN | hidden_layer_sizes ∈ 5 archs from (50,) to (50,50,50), learning_rate_init ∈ 0.1 |
| Multiclass RF | n_estimators ∈ 300, max_features ∈ 6 |
| Multiclass ANN | hidden_layer_sizes ∈ 5 archs from (50,) to (50,50,50), alpha ∈ 0.1 |
All experiments use StratifiedKFold(n_splits=5, shuffle=True, random_state=42). Binary scoring is f1_weighted; multiclass scoring is f1_macro. RF uses class_weight="balanced"; ANN uses StandardScaler plus early_stopping=True.
What the Numbers Actually Say
- Binary task plateaus near F1 ≈ 0.80 for both models. Random Forest is the very narrow winner on test F1 and AUC, but ANN is within noise. The ceiling is the feature set, not the model.
- Multiclass is data-limited. 5,082 samples across 5 classes, with most error coming from adjacent tiers (podium vs winner, midfield vs lower midfield). Random Forest doubles ANN's training F1 but only beats it by ~5 points on the test set.
- Reproducibility is exact. Every seed is fixed (
random_state=42) across train/test splitting,StratifiedKFold,RandomForestClassifier, andMLPClassifier. Re-runningmain.pyagainst the same CSVs produces identical numbers and identical figures.
Tech Stack
- Python 3.11, scikit-learn (
RandomForestClassifier,MLPClassifier,GridSearchCV,StratifiedKFold) - pandas, NumPy for the data merge across 8 Ergast CSVs
- Matplotlib for the 16 generated figures (validation curves, learning curves, confusion matrices)
- conda environment pinned via
environment.yml