| pipeline_tag: text-generation | |
| license: apache-2.0 | |
| Artifacts for the GoldFinch paper | |
| Code available at https://github.com/recursal/GoldFinch-paper | |
| Contains checkpoints for various small scale ablations and 1.5B scale models, all trained on minipile | |
| ## Bibtex citation | |
| ```bibtex | |
| @misc{goldstein2024goldfinchhighperformancerwkvtransformer, | |
| title={GoldFinch: High Performance RWKV/Transformer Hybrid with Linear Pre-Fill and Extreme KV-Cache Compression}, | |
| author={Daniel Goldstein and Fares Obeid and Eric Alcaide and Guangyu Song and Eugene Cheah}, | |
| year={2024}, | |
| eprint={2407.12077}, | |
| archivePrefix={arXiv}, | |
| primaryClass={cs.CL}, | |
| url={https://arxiv.org/abs/2407.12077}, | |
| } | |
| ``` |