|
--- |
|
datasets: |
|
- heliosbrahma/mental_health_chatbot_dataset |
|
- mpingale/mental-health-chat-dataset |
|
library_name: peft |
|
pipeline_tag: text-generation |
|
tags: |
|
- SFT |
|
- PEFT |
|
- Mental Health |
|
- Psychotherapy |
|
- Fine-tuning |
|
- Text Generation |
|
- Chatbot |
|
--- |
|
|
|
# Model Card for Model ID |
|
|
|
<!-- Provide a quick summary of what the model is/does. --> |
|
A LORA fine tuned version of Llama 3 8B instruct that is meant to serve you as a outlet to your negative thoughts |
|
|
|
- **Developed by:** John4Blues (Alt account for 9Skies) |
|
- **Finetuned from model:** https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct |
|
- **Demo:** https://huggingface.co/spaces/John4Blues/Therapy_Llama_3_8B |
|
|
|
|
|
|
|
## Risks and Limitations |
|
|
|
<!-- This section is meant to convey both technical and sociotechnical limitations. --> |
|
|
|
- **By no means is this suppose to replace a therapy counselor, please seek professional help if you believe you need it.** |
|
|
|
- **The responses from the model may not be factually accurate, please double check with other sources when you believe you need to.** |
|
|
|
|
|
|
|
## How to Get Started with the Model |
|
|
|
Use the code below to get started with the model. |
|
|
|
The LORA/PEFT has already been merged with the uploaded model. |
|
|
|
``` |
|
from transformers import AutoModelForCausalLM |
|
model_id = "John4Blues/Llama-3-8B-Therapy" |
|
model = AutoModelForCausalLM.from_pretrained(model_id) |
|
|
|
``` |
|
|
|
|
|
|
|
## Training Details |
|
|
|
### Training Data |
|
|
|
<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. --> |
|
|
|
1. [Amod/mental_health_counseling_conversations](https://huggingface.co/datasets/Amod/mental_health_counseling_conversations) |
|
2. [mpingale/mental-health-chat-dataset](https://huggingface.co/datasets/mpingale/mental-health-chat-dataset) (processed) |
|
|
|
|
|
|
|
### Training Procedure |
|
|
|
<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. --> |
|
|
|
[google colab notebook](https://huggingface.co/John4Blues/Llama-3-8B-Therapy/blob/main/Therapy_LORA_Fined_Tuned_Llama3_8B.ipynb) |
|
|
|
|
|
|
|
#### Training Hyperparameters |
|
|
|
<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. --> |
|
|
|
|
|
- batch_size: 2 |
|
- gradient_accumulation_steps: 2 |
|
- epochs: 3 |
|
- learning_rate: 2e-4 |
|
- warmup_ratio: 0.03 |
|
- dtype: fp16 |
