metadata
license: apache-2.0
language:
- en
library_name: transformers
pipeline_tag: text-generation
Falcon 7b model trained on FAQ from an ecommerce website.
Tutorials
- Text tutorial: https://www.mlexpert.io/prompt-engineering/fine-tuning-llm-on-custom-dataset-with-qlora
- YouTube video: https://www.youtube.com/watch?v=DcBC4yGHV4Q
Training
The model is trained using QLoRA with the help of bitsandbytes, peft and transformers. Full reproduction is available in the tutorials.