LLammas-base π
Llama-2-7B with continued pre-training of 5B tokens of CulturaX (75% Estonian, 25% English documents).
This model is also instruction-tuned resulting in Llammas.
More details in our paper.
Citation
@misc{kuulmets2024teaching,
title={Teaching Llama a New Language Through Cross-Lingual Knowledge Transfer},
author={Hele-Andra Kuulmets and Taido Purason and Agnes Luhtaru and Mark Fishel},
year={2024},
eprint={2404.04042},
archivePrefix={arXiv},
primaryClass={cs.CL}
}
- Downloads last month
- 27
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social
visibility and check back later, or deploy to Inference Endpoints (dedicated)
instead.