File size: 408 Bytes
b56ea75 fd559aa b56ea75 |
1 2 3 4 5 6 7 8 9 10 11 12 13 |
---
datasets:
- loganengstrom/dsdm-candidate-c4
---
*Pythia-410M models pre-trained by MATES.*
The training step is the iteration divided by 4, i.e., iter-040000-ckpt.pth corresponds to the model checkpoint in step 10000.
Paper: [MATES: Model-Aware Data Selection for Efficient Pretraining with Data Influence Models](https://arxiv.org/pdf/2406.06046)
Official codebase: https://github.com/cxcscmu/MATES |