File size: 520 Bytes
f28d01d c65f616 9e433c4 339c7f5 9e433c4 8933331 9e433c4 a3dc6a1 9e433c4 c65f616 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 |
---
language:
- it
pipeline_tag: fill-mask
---
This model (based on [bert-tiny](https://huggingface.co/google/bert_uncased_L-2_H-128_A-2)) has been trained from scratch on 20k italian Wikipedia articles and on a wide collection of italian words and dictionary definitions.
The project is still a work in progress, new versions will come with time.
# Training
- epochs: 200
- lr: 1e-5
- optim: AdamW
- weight_decay: 1e-3
# Eval
- perplexity: 50 (it's a 12MB model, don't expect this to be ChatGPT anytime soon :)
|