File size: 2,574 Bytes
d449243 d42a314 3c8ddd3 6b2a58c d449243 b98815b a3f98ab b98815b a91e4e7 d449243 7a35c80 d449243 c1c93b9 7a35c80 873c72b d449243 d42a314 d449243 d42a314 d449243 d42a314 d449243 d42a314 d449243 d42a314 d449243 b98815b d449243 d42a314 d449243 d42a314 d449243 d42a314 d449243 d42a314 e5323f0 658cb4f |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 |
---
library_name: transformers
base_model:
- unsloth/Llama-3.2-1B-Instruct
license: llama3.2
language:
- en
- it
tags:
- translation
---
# LlaMaestra - A tiny Llama model tuned for text translation
```html
_ _ ___ ___ _
| | | | | \/ | | |
| | | | __ _| . . | __ _ ___ ___| |_ _ __ __ _
| | | |/ _` | |\/| |/ _` |/ _ \/ __| __| '__/ _` |
| |___| | (_| | | | | (_| | __/\__ \ |_| | | (_| |
\_____/_|\__,_\_| |_/\__,_|\___||___/\__|_| \__,_|
```
## Model Card
This model was finetuned with roughly 300.000 examples of translations from English to Italian and Italian to English. The model was finetuned in a way to more directly provide a translation without much explanation.
Finetuning took about 10 hours on an A10G Nvidia GPU.
Due to its size, the model runs very well on CPUs.
## Usage
```python
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig
from peft import PeftModel
base_model_id = "unsloth/Llama-3.2-1B-Instruct"
bnb_config = BitsAndBytesConfig(
load_in_4bit=True,
bnb_4bit_use_double_quant=True,
bnb_4bit_quant_type="nf4",
bnb_4bit_compute_dtype=torch.bfloat16
)
base_model = AutoModelForCausalLM.from_pretrained(
base_model_id, # Mistral, same as before
quantization_config=bnb_config, # Same quantization config as before
device_map="auto",
trust_remote_code=True,
)
tokenizer = AutoTokenizer.from_pretrained(base_model_id, add_bos_token=True, trust_remote_code=True)
ft_model = PeftModel.from_pretrained(base_model, "LeonardPuettmann/LlaMaestra-3.2-1B-Instruct-v0.1-4bit")
row_json = [
{"role": "system", "content": "Your job is to return translations for sentences or words from either Italian to English or English to Italian."},
{"role": "user", "content": "Scontri a Bologna, la destra lancia l'offensiva contro i centri sociali."}
]
prompt = tokenizer.apply_chat_template(row_json, tokenize=False)
model_input = tokenizer(prompt, return_tensors="pt").to("cuda")
with torch.no_grad():
print(tokenizer.decode(ft_model.generate(**model_input, max_new_tokens=1024)[0]))
```
## Data used
The source for the data were sentence pairs from tatoeba.com. The data can be downloaded from here: https://tatoeba.org/downloads
## Credits
Base model: `unsloth/Llama-3.2-1B-Instruct` derived from `meta-llama/Llama-3.2-1B-Instruct`
Finetuned by: Leonard Püttmann https://www.linkedin.com/in/leonard-p%C3%BCttmann-4648231a9/ |