margotwagner
commited on
Commit
•
b71fc84
1
Parent(s):
b3f6055
Update README.md
Browse files
README.md
CHANGED
@@ -37,78 +37,39 @@ Given a sentence, this model provides a binary classification as either symptoma
|
|
37 |
## Uses
|
38 |
|
39 |
<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
|
40 |
-
|
41 |
-
### Direct Use
|
42 |
-
|
43 |
-
<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
|
44 |
-
|
45 |
-
[More Information Needed]
|
46 |
|
47 |
## How to Get Started with the Model
|
48 |
|
49 |
Use the code below to get started with the model.
|
|
|
|
|
|
|
|
|
50 |
|
51 |
-
[
|
|
|
|
|
|
|
|
|
|
|
52 |
|
53 |
## Training Details
|
54 |
|
55 |
### Training Data
|
56 |
|
57 |
-
|
58 |
-
|
59 |
-
[More Information Needed]
|
60 |
|
61 |
### Training Procedure
|
62 |
|
63 |
-
|
64 |
-
|
65 |
-
#### Preprocessing [optional]
|
66 |
-
|
67 |
-
[More Information Needed]
|
68 |
-
|
69 |
-
|
70 |
-
#### Training Hyperparameters
|
71 |
-
|
72 |
-
- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
|
73 |
-
|
74 |
-
#### Speeds, Sizes, Times [optional]
|
75 |
-
|
76 |
-
<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
|
77 |
-
|
78 |
-
[More Information Needed]
|
79 |
-
|
80 |
-
## Evaluation
|
81 |
|
82 |
-
|
83 |
-
|
84 |
-
### Testing Data, Factors & Metrics
|
85 |
|
86 |
#### Testing Data
|
87 |
|
88 |
-
|
89 |
-
|
90 |
-
[More Information Needed]
|
91 |
-
|
92 |
-
#### Factors
|
93 |
-
|
94 |
-
<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
|
95 |
-
|
96 |
-
[More Information Needed]
|
97 |
|
98 |
#### Metrics
|
99 |
|
100 |
-
|
101 |
-
|
102 |
-
[More Information Needed]
|
103 |
-
|
104 |
-
### Results
|
105 |
-
|
106 |
-
[More Information Needed]
|
107 |
-
|
108 |
-
#### Summary
|
109 |
-
|
110 |
-
|
111 |
-
|
112 |
-
|
113 |
-
|
114 |
-
|
|
|
37 |
## Uses
|
38 |
|
39 |
<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
|
40 |
+
This model is intended to be used to assess the mental health status using sentence-level text data. Specifically, it looks for symptoms related to anxiety and depression.
|
|
|
|
|
|
|
|
|
|
|
41 |
|
42 |
## How to Get Started with the Model
|
43 |
|
44 |
Use the code below to get started with the model.
|
45 |
+
```python
|
46 |
+
from transformers import pipeline
|
47 |
+
|
48 |
+
classifier = pipeline(task="text-classification", model="margotwagner/roberta-psychotherapy-eval")
|
49 |
|
50 |
+
sentences = ["I am not having a great day"]
|
51 |
+
|
52 |
+
model_outputs = classifier(sentences)
|
53 |
+
print(model_outputs[0])
|
54 |
+
# produces a list of dicts for each of the labels
|
55 |
+
```
|
56 |
|
57 |
## Training Details
|
58 |
|
59 |
### Training Data
|
60 |
|
61 |
+
This model was fine-tuned using English sentence-level data in a supervised manner where symptomatic labels were obtained from expert clinicians. Sentences were required to be independent in nature. Back-translation was utilized to increase the size of the training dataset.
|
|
|
|
|
62 |
|
63 |
### Training Procedure
|
64 |
|
65 |
+
Weighted cross-entropy loss function was employed to address class imbalance. Model accuracy in the form of F1 was used for model selection.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
66 |
|
67 |
+
### Testing Data & Metrics
|
|
|
|
|
68 |
|
69 |
#### Testing Data
|
70 |
|
71 |
+
The testing data used was clinical data from a board-reviewed and ethically-compliant online psychotherapy clinical trial conducted at Queen’s University between 2020 and 2021. The study underwent a thorough review process by the Queen’s University Health Sciences and Affiliated Teaching Hospitals Research Ethics Board to ensure adherence to ethical standards (File #: 6020045).
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
72 |
|
73 |
#### Metrics
|
74 |
|
75 |
+
F1 score was used as the model accuracy metric, as it maintains a balance between precision and recall with particular importance given to positive examples.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|