README.md

---
license: apache-2.0
base_model: Qwen/Qwen2-0.5B-Instruct
tags:
- trl
- sft
- text-to-SQL
- generated_from_trainer
model-index:
- name: Qwen2-0.5B-Instruct-SQL-query-generator
  results: []
---

# Qwen2-0.5B-Instruct-SQL-query-generator

This model is a fine-tuned version of [Qwen/Qwen2-0.5B-Instruct](https://huggingface.co/Qwen/Qwen2-0.5B-Instruct) on the [motherduckdb/duckdb-text2sql-25k](https://huggingface.co/datasets/motherduckdb/duckdb-text2sql-25k) dataset (first 10k rows).

## Model Description

The Qwen2-0.5B-Instruct-SQL-query-generator is a specialized model fine-tuned to generate SQL queries from natural language text prompts. This fine-tuning allows the model to better understand and convert text inputs into corresponding SQL queries, facilitating tasks such as data retrieval and database querying through natural language interfaces.

## Intended Uses & Limitations

### Intended Uses

- Convert natural language questions to SQL queries.
- Facilitate data retrieval from databases using natural language.
- Assist in building natural language interfaces for databases.

### Limitations

- The model is fine-tuned on a specific subset of data and may not generalize well to all SQL query formats or databases.
- It is recommended to review the generated SQL queries for accuracy and security, especially before executing them on live databases.

## Training and Evaluation Data

### Training Data

The model was fine-tuned on the [motherduckdb/duckdb-text2sql-25k](https://huggingface.co/datasets/motherduckdb/duckdb-text2sql-25k) dataset, specifically using the first 10,000 rows. This dataset includes natural language questions and their corresponding SQL queries, providing a robust foundation for training a text-to-SQL model.

### Evaluation Data

The evaluation data used for fine-tuning was a subset of the same dataset, ensuring consistency in training and evaluation metrics.

## Training Procedure

Github Code: https://github.com/omaratef3221/SQL_Query_Generator_llm/
### Training Hyperparameters

The following hyperparameters were used during training:
- `learning_rate`: 1e-4
- `train_batch_size`: 8
- `save_steps`: 1
- `logging_steps`: 500
- `num_epochs`: 5

### Training Frameworks

- Transformers: 4.39.0
- PyTorch: 2.2.0
- Datasets: 2.20.0
- Tokenizers: 0.15.2

### Training Results

During the training process, the model was periodically evaluated to ensure it was learning effectively. The specific training metrics and results were logged for further analysis.

## Model Performance

### Evaluation Metrics

- Evaluation metrics such as accuracy, precision, recall, and F1-score were used to assess the model's performance. (Specific values can be added here if available.)

## Usage

To use this model, simply load it from the Hugging Face Model Hub and provide natural language text prompts. The model will generate the corresponding SQL queries.

```python
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

tokenizer = AutoTokenizer.from_pretrained("omaratef3221/Qwen2-0.5B-Instruct-SQL-query-generator")
model = AutoModelForSeq2SeqLM.from_pretrained("omaratef3221/Qwen2-0.5B-Instruct-SQL-query-generator")

inputs = tokenizer("Show me all employees with a salary greater than $100,000", return_tensors="pt")
outputs = model.generate(**inputs)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))