---
license: apache-2.0
language:
- en
library_name: transformers
pipeline_tag: text-generation
---

[Falcon 7b](https://huggingface.co/tiiuae/falcon-7b) model trained on [FAQ from an ecommerce website](https://www.kaggle.com/datasets/saadmakhdoom/ecommerce-faq-chatbot-dataset).

## Tutorials

- Text tutorial: https://www.mlexpert.io/prompt-engineering/fine-tuning-llm-on-custom-dataset-with-qlora
- YouTube video: https://www.youtube.com/watch?v=DcBC4yGHV4Q

## Training

Fine-tuned using the QLoRA technique (only adapter uploaded). With the help of bitsandbytes, peft and transformers. Full reproduction is available in the tutorials.