|
--- |
|
license: apache-2.0 |
|
base_model: Qwen/Qwen2-0.5B-Instruct |
|
tags: |
|
- trl |
|
- sft |
|
- text-to-SQL |
|
- generated_from_trainer |
|
model-index: |
|
- name: Qwen2-0.5B-Instruct-SQL-query-generator |
|
results: [] |
|
--- |
|
|
|
# Qwen2-0.5B-Instruct-SQL-query-generator |
|
|
|
This model is a fine-tuned version of [Qwen/Qwen2-0.5B-Instruct](https://huggingface.co/Qwen/Qwen2-0.5B-Instruct) on the [motherduckdb/duckdb-text2sql-25k](https://huggingface.co/datasets/motherduckdb/duckdb-text2sql-25k) dataset (first 10k rows). |
|
|
|
## Model Description |
|
|
|
The Qwen2-0.5B-Instruct-SQL-query-generator is a specialized model fine-tuned to generate SQL queries from natural language text prompts. This fine-tuning allows the model to better understand and convert text inputs into corresponding SQL queries, facilitating tasks such as data retrieval and database querying through natural language interfaces. |
|
|
|
## Intended Uses & Limitations |
|
|
|
### Intended Uses |
|
|
|
- Convert natural language questions to SQL queries. |
|
- Facilitate data retrieval from databases using natural language. |
|
- Assist in building natural language interfaces for databases. |
|
|
|
### Limitations |
|
|
|
- The model is fine-tuned on a specific subset of data and may not generalize well to all SQL query formats or databases. |
|
- It is recommended to review the generated SQL queries for accuracy and security, especially before executing them on live databases. |
|
|
|
## Training and Evaluation Data |
|
|
|
### Training Data |
|
|
|
The model was fine-tuned on the [motherduckdb/duckdb-text2sql-25k](https://huggingface.co/datasets/motherduckdb/duckdb-text2sql-25k) dataset, specifically using the first 10,000 rows. This dataset includes natural language questions and their corresponding SQL queries, providing a robust foundation for training a text-to-SQL model. |
|
|
|
### Evaluation Data |
|
|
|
The evaluation data used for fine-tuning was a subset of the same dataset, ensuring consistency in training and evaluation metrics. |
|
|
|
## Training Procedure |
|
|
|
Github Code: https://github.com/omaratef3221/SQL_Query_Generator_llm/ |
|
### Training Hyperparameters |
|
|
|
The following hyperparameters were used during training: |
|
- `learning_rate`: 1e-4 |
|
- `train_batch_size`: 8 |
|
- `save_steps`: 1 |
|
- `logging_steps`: 500 |
|
- `num_epochs`: 5 |
|
|
|
### Training Frameworks |
|
|
|
- Transformers: 4.39.0 |
|
- PyTorch: 2.2.0 |
|
- Datasets: 2.20.0 |
|
- Tokenizers: 0.15.2 |
|
|
|
### Training Results |
|
|
|
During the training process, the model was periodically evaluated to ensure it was learning effectively. The specific training metrics and results were logged for further analysis. |
|
|
|
## Model Performance |
|
|
|
### Evaluation Metrics |
|
|
|
- Evaluation metrics such as accuracy, precision, recall, and F1-score were used to assess the model's performance. (Specific values can be added here if available.) |
|
|
|
## Usage |
|
|
|
To use this model, simply load it from the Hugging Face Model Hub and provide natural language text prompts. The model will generate the corresponding SQL queries. |
|
|
|
```python |
|
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM |
|
|
|
tokenizer = AutoTokenizer.from_pretrained("omaratef3221/Qwen2-0.5B-Instruct-SQL-query-generator") |
|
model = AutoModelForSeq2SeqLM.from_pretrained("omaratef3221/Qwen2-0.5B-Instruct-SQL-query-generator") |
|
|
|
inputs = tokenizer("Show me all employees with a salary greater than $100,000", return_tensors="pt") |
|
outputs = model.generate(**inputs) |
|
print(tokenizer.decode(outputs[0], skip_special_tokens=True)) |
|
|
