- Multi-layer PyTorch networks for one regression task and one classification task
- Four-architecture sweep on each, varying depth, optimizer (SGD vs Adam), and learning rate
- Shared pipeline: feature engineering, correlation-based feature selection, standardization, training, evaluation
- Train/validation loss curves and per-architecture comparisons
Overview
Two PyTorch notebooks built around a shared pipeline. One predicts worldwide box-office revenue from movie metadata (TMDB Box Office Prediction). The other predicts passenger survival on the Titanic. Course project for CSC 158 at Hofstra, completed with my teammate Yash who handled the classification side.
Shared Pipeline
Both notebooks follow the same six steps:
- Load the CSV with pandas.
- Feature engineer. Turn JSON/string columns into numeric ones (cast/crew counts and
has_collectionfor TMDB;Gender,Has_cabin,Family_size,alone,Embarked_*for Titanic). - Feature selection by the assignment's correlation rules: keep features with
|corr|≥ 0.2 with the target, drop one of any pair with|corr|> 0.8. - Standardize with
sklearn.preprocessing.StandardScalerfit on the training split only (80/20 train/val). - Define a
Modelsubclass oftorch.nn.Modulewith configurable hidden layers. - Train four configurations and compare train vs validation curves.
Regression: TMDB Box-Office Revenue
7 engineered numeric features, linear output, MSE loss, 2,000 epochs per configuration.
| Model | Hidden layers | Optimizer | Learning rate | Best val MSE |
|---|---|---|---|---|
| A | [32] | SGD | 1e-3 | 0.51 |
| B | [64, 32] | Adam | 1e-3 | 0.59 |
| C | [64, 32, 16] | Adam | 1e-3 | 0.62 |
| D | [64, 32, 16] | Adam | 1e-4 | 0.48 |
The deep Adam network with lr=1e-4 wins. Larger Adam models with lr=1e-3 overfit badly past ~500 epochs, and the smaller SGD baseline (0.51) ends up beating them.
Classification: Titanic Survival
5 engineered numeric features, 2-class softmax output, cross-entropy loss, 500 epochs per configuration with mini-batch SGD/Adam.
| Model | Hidden layers | Optimizer | Learning rate | Val accuracy |
|---|---|---|---|---|
| A | [11] | SGD | 1e-3 | 81.6% |
| B | [22, 11] | Adam | 1e-3 | 82.7% |
| C | [22, 11] | SGD | 1e-3 | did not converge |
| D | [22, 11, 5] | Adam | 1e-4 | 80.4% |
The two-layer Adam network with lr=1e-3 wins. A shallow SGD network is close behind; the deeper SGD network at the same learning rate fails to converge in 500 epochs.
What the Numbers Actually Say
- Adam dominates on small tabular data with the right learning rate. Both winners are Adam; the SGD baselines are close but never beat Adam when Adam is tuned.
- Depth without a matching learning rate hurts. The deepest models converge only at the smaller learning rate. At
lr=1e-3they either overfit (regression) or stall (classification). - Feature engineering does most of the work. On the Titanic side, the intercorrelation check caught a 0.90 overlap between
Has_cabinandDeck_num, and dropping one was worth more than any architectural choice.
Tech Stack
- Python 3, PyTorch, pandas, NumPy, scikit-learn, Matplotlib
- Jupyter notebooks for the full training and evaluation pipeline
- Kaggle datasets: TMDB Box Office Prediction, Titanic