Training FootNet
We trained the FootNet models on 500,000 examples across the entire Contiguous United States. The training data consists of more than 20 TB of space, and as such is not feasible with traditional deep learning model training methods.
We used PyTorch's Distributed Data Parallel (DDP) library to conduct distributed training over 10-15 machines synchronously.
Below are the scripts that we used to train Surface FootNet and Column FootNet models using distributed learning.
These scripts can be used as a reference to fine-tune FootNet models or train a new FootNet model from scratch.
Fine-tuning/Training checklist
- Gather footprints
- Gather corresponding meteorology data
- Prepare feature-target pairs for training
- Preprocess/transform the features
- Train model and validate
- Save model
For more information regarding training or fine-tuning, feel free to reach out to turneraj@uw.edu or nd349@uw.edu.