|
--- |
|
license: llama3 |
|
base_model: catallama/CataLlama-v0.2-Instruct-SFT-DPO-Merged |
|
tags: |
|
- llama |
|
- llama-3 |
|
- catalan |
|
model-index: |
|
- name: CataLlama-v0.2-Instruct-SFT-DPO-Merged-GGUF |
|
results: [] |
|
datasets: |
|
- catallama/Catalan-DPO-V2 |
|
- catallama/Catalan-Instruct-V2 |
|
language: |
|
- ca |
|
- en |
|
pipeline_tag: text-generation |
|
--- |
|
|
|
 |
|
|
|
# CataLlama-v0.2-Instruct-SFT-DPO-Merged-GGUF |
|
|
|
**CataLlama-v0.2-Instruct-SFT-DPO-Merged-GGUF** is a quantisation of [catallama/CataLlama-v0.2-Instruct-SFT-DPO-Merged](https://huggingface.co/catallama/CataLlama-v0.2-Instruct-SFT-DPO-Merged) |
|
|
|
**This is an instruction fine-tuned model, optimised with DPO, proficient on the following tasks in Catalan** |
|
|
|
- *Information extraction (suitable for RAG)* |
|
- *Named Entity Recognition (NER)* |
|
- *Translation from English to Catalan and Catalan to English* |
|
- *Summarization - both short form and long form* |
|
- *Sentiment analysis* |
|
- *Chat* |
|
|
|
**Model developers** [Laurentiu Petrea](https://www.linkedin.com/in/laurentiupetrea/) based on Llama-3 from Meta. |
|
|
|
**Model Architecture** CataLlama is an auto-regressive language model that uses an optimized transformer architecture. The tuned versions use supervised fine-tuning (SFT) and direct preference optimisation (DPO) to align with human preferences for helpfulness and safety. |
|
|
|
**License** The model uses the llama-3 license available at: [https://llama.meta.com/llama3/license](https://llama.meta.com/llama3/license) |
|
|
|
|
|
## Benchmarks (for the bf16 model) |
|
|
|
| Model | CataLlama-v0.2-Instruct-DPO | CataLlama-v0.2-Instruct-SFT | CataLlama-v0.2-Instruct-SFT-DPO-Merged | |
|
| ------------------ | --------------------------- | ------------------------------- | ------------------------------------------ | |
|
| MMLU 5 shot | 58.89 | 59.35 | **60.53** | |
|
| GSM8K CoT 8 shot | 60.05 | 76.04 | **77.26** | |
|
|
|
|
|
**Please see the original model card for more details** |
|
|
|
## Intended Use |
|
|
|
**Note:** This model is not intended to beat benchmarks, but to demonstrate techniques for augmenting LLMs on new languages and preserve rare languages as part of our world heritage. |
|
|
|
**Intended Use Cases** Llama 3 is intended for commercial and research use in English. Instruction tuned models are intended for assistant-like chat, whereas pretrained models can be adapted for a variety of natural language generation tasks. |
|
|
|
**Out-of-scope** Use in any manner that violates applicable laws or regulations (including trade compliance laws). Use in any other way that is prohibited by the Acceptable Use Policy and Llama 3 Community License. Use in languages other than English**. |
|
|
|
**Note: Developers may fine-tune Llama 3 models for languages beyond English provided they comply with the Llama 3 Community License and the Acceptable Use Policy. |
|
|
