metadata
license: apache-2.0
Ask2Democracy project
What's baizemocracy-lora-7B-cfqa model?
This model is an open-source chat model fine-tuned with LoRA inspired by Baize project. It was trained with the Baize datasets and the ask2democracy-cfqa-salud-pension dataset, wich contains almost 4k instructions to answers questions based on a context relevant to citizen concerns and public debate in spanish.
- Developed by:
- 🇨🇴 Jorge Henao
- 🇨🇴 David Torres
Training Parameters
- Base Model: LLaMA-7B
- Training Epoch: 1
- Batch Size: 16
- Maximum Input Length: 512
- Learning Rate: 2e-4
- LoRA Rank: 8
- Updated Modules: All Linears
Training Dataset
- Ask2Democracy-cfqa-salud-pension (3,806)
- Standford Alpaca (51,942)
- Quora Dialogs (54,456):
- StackOverflow Dialogs (57,046)
- Alpacaca chat Dialogs
- Medical chat Dialogs
More details can be found in the Ask2Democracy GitHub