Hugging Face's logo

ishitas2365
/
llama-3.2-3b-instruct-finetunedToPersona

Text Generation
Transformers
Safetensors
llama
conversational
text-generation-inference
Inference Endpoints
4-bit precision
bitsandbytes
Model card Files Community
llama-3.2-3b-instruct-finetunedToPersona
File size: 2,164 Bytes 
9ba219a
 
 
9b02ab3
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
 
---
license: apache-2.0
library_name: transformers
datasets:
- Cynaptics/persona-chat
base_model:
- meta-llama/Llama-3.2-3B-Instruct
---


## Model Details
This is a fine tuned version of Meta Llama-3.2-3B-Instruct model on persona chat based dataset. The LLM inherits the persona B and responds in a humane way.



### Training Procedure

Fine tuning method used : QLoRA




## How to Get Started with the Model

Use the code below to get started with the model.

```python
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("ishitas2365/llama-3.2-3b-instruct-finetunedToPersona")
model = AutoModelForCausalLM.from_pretrained("ishitas2365/llama-3.2-3b-instruct-finetunedToPersona")

tokenizer.pad_token_id = tokenizer.eos_token_id

# Enter the characteristics of persona in system prompt and the initial dialogue of the user in user prompt
messages = [
    {
        "role": "system",
        "content": "Persona B's characteristics: My name is David, and I'm a 35-year-old math teacher. "
                   "I like to hike and spend time in nature. I'm married with two kids."
    },
    {
        "role": "user",
        "content": "Morning! I think I saw you at the parent meeting, what's your name?"
    }
]

prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(prompt, return_tensors='pt', padding=True, truncation=True).to("cuda")
with torch.no_grad():
    outputs = model.generate(
        input_ids=inputs.input_ids,
        attention_mask=inputs.attention_mask,
        max_length=200,
        num_return_sequences=1,
        temperature=0.8,  
        top_p=0.9        
    )
decoded_text = tokenizer.decode(outputs[0], skip_special_tokens=True)
if "assistant" in decoded_text:
    response = decoded_text.split("assistant", 1)[1].strip()
else:
    response = decoded_text.strip()

print("Assistant's Reply:", response)
```






###  Sample Result

Assistant's Reply: Good morning! Yeah, I was at the parent meeting. My name's David, nice to meet you. I'm a math teacher here at the school. How about you, do you have kids in the school?