File size: 8,263 Bytes
19d9abc dca99f4 8143c42 dca99f4 8143c42 dca99f4 313f14f dca99f4 072ee0e dca99f4 072ee0e 0f13af6 dca99f4 313f14f 0f13af6 dca99f4 0f13af6 dca99f4 0f13af6 dca99f4 4640be7 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 |
---
datasets:
- nbertagnolli/counsel-chat
language:
- en
base_model:
- google/gemma-2-2b-it
pipeline_tag: text-generation
tags:
- therapy
- chat-bot
library_name: transformers
---
# TADBot
## Problem Statement
- Mental health issues are a significant problem that affects millions of people worldwide, yet many people struggle to seek help due to stigma, lack of resources, or other barriers.
- Existing mental health resources often lack personalization and empathy, making it difficult for individuals to connect with them and receive the help they need.
- There is a need for a more accessible, affordable, and personalized mental health resource that can provide support and advice to individuals who may be struggling with mental health issues.
## Overview
TADBot is a small language model that is fine-tuned on the nbertagnolli/counsel-chat dataset to assist people deal with mental problems and offer them advice based on the context of the conversation. It is designed to be a supportive and empathetic resource for those who may be struggling with mental health issues, providing personalized and tailored support and advice. TADBot is still in development and is not yet available for public use, but it has the potential to significantly improve access to mental health resources and support for individuals who may be struggling with mental health issues.
## Technology used
- Gemma 2 2B: A small language model with 2 billion parameters that TADBot is fine-tuned on.
- nbertagnolli/counsel-chat: The dataset used to train TADBot on mental health and advice-giving tasks.
- Hugging Face Transformers: A library used to fine-tune the Gemma 2 2B model on the nbertagnolli/counsel-chat dataset.
- PyTorch: A library used for training and fine-tuning the language model.
- Flask: A library used to create a server for TADBot.
- Raspberry Pi: A small, low-cost computer used to host Test to Speech and Speech To Text models and TADBot server.
- FER: A deep learning model used to detect emotions from faces in real-time using a webcam.
- S2T and T2S: Speech to Text and Text to Speech models used to convert speech to text and text to speech respectively.
# Features
## FER Model:
- TADBot uses a deep learning model to detect emotions from faces in real-time using a webcam. This allows TADBot to better understand the emotional context of a conversation and provide more appropriate and empathetic responses.
- The Data from the FER model is sent to the TADBot server and is used to identify the emotion from the image sent by the client. This information is then used to generate a more appropriate response from the model.
- The data is also logged seperatly in a text file which can be accessed by the client to track the change in emotion during the conversation. This can be used to provide insights into the conversation.
- The Data is not collected and erased after every conversation adhereing to the doctor-client privacy
> HLD for the FER model
```mermaid
flowchart TD
%% User Interface Layer
A[Raspberry PI] -->|Sends Image| B[detecfaces.py]
B --->|Returns processed data| A
%%Server
subgraph Server
%% Processing Layer
B --> |Captured Image| T1[prediction.py]
M1[RAFDB trained model] --> |epoch with best acc 92%|B
T1-->|Top 3 emotions predicted| B
%%Model Layer
M1
%% Processing
subgraph Processing
T1 --> |Send Image|T2[detec_faces]
T2 --> |Returns a 224x224 face|T1
end
end
```
## S2T Model and T2S Model:
- The Text to Speech (T2S) and Speech to Text are facilitaed by the pyttx library that captures the audio from the system microphone and then uses the google backend to process the audio into text format
- This text format is then passed onto the Language Model to generate a response based on the text provided by the pyttx library
- The response generated by the Language Model is then again converted to Speech and output via a speaker
> LLD for the S2T and T2S model
```mermaid
sequenceDiagram
%% User level
User->>+Raspberry PI: Speak into microphone
Raspberry PI->>+main.py: Capture audio
main.py->>+pyttx: Convert audio to text
pyttx->>+Google Backend: Send audio for transcription
Google Backend-->>-pyttx: Transcribed text
pyttx-->>-main.py: Transcribed text
main.py->>+LLM: Send text for response generation
LLM-->>-main.py: Generated response text
main.py->>+pyttx: Convert response text to audio
pyttx->>+Google Backend: Send text for synthesis
Google Backend-->>-pyttx: Synthesized audio
pyttx-->>-main.py: Synthesized audio
main.py->>+Raspberry PI: Output audio through speaker
Raspberry PI-->>-User: Speaker output
```
# How It Works
## Model
TADBot uses a fine-tuned version of the Gemma 2 2B language model to generate responses. The model is trained on the nbertagnolli/counsel-chat dataset from hugging face, which contains conversations between mental health professionals and clients. The model is fine-tuned using the Hugging Face Transformers library and PyTorch.
### Dataset
The raw version of the dataset consists of 2275 conversation taken from an online mental health platform.
- The data consists of 'questionID', 'questionTitle', 'questionText', 'questionLink', 'topic', 'therapistInfo', 'therapistURL', 'answerText', 'upvotes', 'views' as features
- The dataset is cleaned and preprocessed to remove any irrelevant or sensitive information only retaining
- The data is then mapped to a custom prompt that allows for system level role instruction to be given to the model allow for much better responses.
- The data is then split into training, validation
- The model is trained on the training set and evaluated on the validation set.
### Training
- All the training data configuration is stored on a YAML file for easy access and modification. The YAML file contains the following information:
- model name
- new model name
- lora configs
- peft configs
- training arguments
- sft arguments
- The general model is trained on the following parameters
-learning rate: 2e-4
-batch size: 2
-gradient accumulation steps: 2
-num train epochs: 1
-weight decay: 0.01
-optimizer: paged_adamw_32bit
- The fine tuned lora adapters are then merged with the base model to give the final model
- The final model is then saved and pushed to the Hugging Face model hub for easy access and deployment.
```mermaid
sequenceDiagram
participant hyperparameter
participant trainer
participant model
participant tokenizer
participant dataset
participant custom prompt
hyperparameter->>trainer: Initialize PEFT config
hyperparameter->>model: Initialize bitsandbytes config
hyperparameter->>trainer: Initialize training arguments
hyperparameter->>trainer: Initialize SFT arguments
dataset->>custom prompt: Custome prompt
custom prompt-->>dataset: Mapped dataset
dataset-->>trainer: Dataset for training
tokenizer->>trainer: Tokenize input text
model-->>trainer: Model for training
trainer->>model: Train model
model-->>trainer: Trained model
trainer->>model: Save model
trainer->>tokenizer: Save tokenizer
model-->>Hugging Face: Push model to Hub
tokenizer-->>Hugging Face: Push tokenizer to Hub
```
### Inference
- Since the model size is generally quite large and difficult to run on commerical hardwares, the model is quantized using
llama.cpp to reduce the model size from ~5GB to > 2GB. This allows the model to be run on a Raspberry Pi 4 with 4GB of RAM.
- The model is then deployed on a Raspberry Pi 4 and inferenced using the ollama rest api.
- The conversations are also stored in a vector embedding to futher improve the response generated by the model
- at the end of the conversation the model creates a log file that stores conversation between the user and the model that can be usefull during diagnosis by a therapist
# Deployment Instructions
To deploy TADBot locally, you will need to follow these steps:
- create a virtual environment(preferrablly python 3.12) with pip-tools or uv installed and install the required dependencies
```
pip sync requirements.txt #if u are using pip-tools
pip install -r requirements.txt #if u are using pip
uv sync #if u are using uv
``` |