---
language:
- en
- it
license: apache-2.0
tags:
- text-generation-inference
- transformers
- unsloth
- llama
- trl
- sft
---

# Meta LLaMA 3.1 8B 4-bit Finetuned Model

This model is a fine-tuned version of `Meta-Llama-3.1-8B`, developed by **ruslanmv** for text generation tasks. It leverages 4-bit quantization, making it more efficient for inference while maintaining strong performance in natural language generation.

---

## Model Details

- **Base Model**: `unsloth/meta-llama-3.1-8b-bnb-4bit`
- **Finetuned by**: ruslanmv
- **Language**: English
- **License**: [Apache 2.0](https://www.apache.org/licenses/LICENSE-2.0)
- **Tags**: 
  - text-generation-inference
  - transformers
  - unsloth
  - llama
  - trl
  - sft

---

## Model Usage

### Installation

To use this model, you will need to install the necessary libraries:

```bash
pip install transformers accelerate bitsandbytes
```

### Loading the Model in Python

Here’s an example of how to load this fine-tuned model using Hugging Face's `transformers` library:

```python
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

# Load the model and tokenizer
model_name = "Meta-Llama-3.1-8B-Text-to-SQL-4bit"

# Ensure you have the right device setup
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")

# Load the model and tokenizer from the Hugging Face Hub
model = AutoModelForCausalLM.from_pretrained(model_name, device_map="auto", torch_dtype=torch.float16)
tokenizer = AutoTokenizer.from_pretrained(model_name)

# Example usage
input_text = "Recupera il conteggio di tutte le righe nella tabella table1"
inputs = tokenizer(input_text, return_tensors="pt").to(device)

# Generate output text
outputs = model.generate(**inputs, max_length=50)

# Decode and print the generated text
generated_text = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(generated_text)
```

### Model Features

- **Text Generation**: This model is fine-tuned to generate coherent and contextually accurate text based on the provided input. 
- **Efficiency**: Using 4-bit quantization with the `bitsandbytes` library, it optimizes memory and inference performance.

### License

This model is licensed under the [Apache 2.0 License](https://www.apache.org/licenses/LICENSE-2.0). You are free to use, modify, and distribute this model, provided that you comply with the license terms.

### Acknowledgments

This model was fine-tuned by **ruslanmv** based on the original work of `unsloth` and the `meta-llama-3.1-8b-bnb-4bit` model.