File size: 505 Bytes
16972a2 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 |
---
datasets:
- JeanKaddour/minipile
language:
- en
license: mpl-2.0
---
# RWKV Minipile
## Model Specifications
- **Architecture**: RWKV
- **Vocabulary Size**: 65,536
- **Embedding Size**: 768
- **Number of Layers**: 12
- **Context Length**: 512
- **Data Type**: bfloat16
- **Dataset**: Minipile
- **Tokens**: 20,643,840 (20 Million)
The model underwent a rigorous training regimen, completing 30 epochs to optimize performance.
## Inference
```bash
pip install torch numpy
python inference.py
``` |