Update README.md

README.md

@@ -11,12 +11,57 @@ tags:
 - trl
 ---
 
-# Uploaded  model
+<div class="alert alert-block alert-danger">  
+<h2><center><strong>Mental Health Chatbot using 1B finetuned Llama 3.2 Model</strong></center></h2>
+        
+</div>
 
-- **Developed by:** ImranzamanML
-- **License:** apache-2.0
-- **Finetuned from model :** unsloth/Llama-3.2-1B-bnb-4bit
+## Inference
 
-This llama model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library.
+```python
+from unsloth import FastLanguageModel
+model, tokenizer = FastLanguageModel.from_pretrained(
+model_name = "ImranzamanML/1B_finetuned_llama3.2",
+max_seq_length = 5020,
+dtype = None,
+load_in_4bit = True)
+```
+
+
+## Using this text to feed into model for getting the response 
+
+text="I'm going through some things with my feelings and myself. I barely sleep and I do nothing but think about how I'm worthless and how I shouldn't be here. I've never tried or contemplated suicide. I've always wanted to fix my issues, but I never get around to it. How can I change my feeling of being worthless to everyone?"
+
+<div style="background-color: #f2f2f2; border-left: 5px solid #4CAF50; padding: 15px; margin: 20px 0;">
+    <strong>Note:</strong> Lets use the fine-tuned model for inference in order to generate responses based on mental health-related prompts !
+</div>
+
+<h3 style="color: #388e3c; font-family: Arial, sans-serif;">Here is some keys to note:</h3>
+
+<ol style="margin-left: 20px;">
+        <p>The <code>model = FastLanguageModel.for_inference(model)</code> configures the model specifically for inference, optimizing its performance for generating responses.</p>
+    </li>
+        <p>The input text is tokenized using the <code>tokenizer</code>, it convert the text into a format that model can process. We are using <code>data_prompt</code> to format the input text, while the response placeholder is left empty to get response from model. The <code>return_tensors = "pt"</code> parameter specifies that the output should be in PyTorch tensors, which are then moved to the GPU using <code>.to("cuda")</code> for faster processing.</p>
+    </li>
+        <p>The <code>model.generate</code> method generating response based on the tokenized inputs. The parameters <code>max_new_tokens = 5020</code> and <code>use_cache = True</code> ensure that the model can produce long and coherent responses efficiently by utilizing cached computation from previous layers.</p>
+    </li>
+</ol>
+
+```python
+model = FastLanguageModel.for_inference(model)
+inputs = tokenizer(
+[
+    data_prompt.format(
+        #instructions
+        text,
+        #answer
+        "",
+    )
+], return_tensors = "pt").to("cuda")
+
+outputs = model.generate(**inputs, max_new_tokens = 5020, use_cache = True)
+answer=tokenizer.batch_decode(outputs)
+answer = answer[0].split("### Response:")[-1]
+print("Answer of the question is:", answer)
+```
 
-[<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)