This model is a fine-tuned version the cardiffnlp/twitter-roberta-base model. It has been trained using a recently published corpus: Shared task on Detecting Signs of Depression from Social Media Text at LT-EDI 2022-ACL 2022.
The obtained macro f1-score is 0.54, on the development set of the competition.
Intended uses
This model is trained to classify the given text into one of the following classes: moderate, severe, or not depressed. It corresponds to a multiclass classification task.
Training and evaluation data
The train dataset characteristics are:
| Class | Nº sentences | Avg. document length (in sentences) | Nº words | Avg. sentence length (in words) |
|---|---|---|---|---|
| not depression | 7,884 | 4 | 153,738 | 78 |
| moderate | 36,114 | 6 | 601,900 | 100 |
| severe | 9,911 | 11 | 126,140 | 140 |
Similarly, the evaluation dataset characteristics are:
| Class | Nº sentences | Avg. document length (in sentences) | Nº words | Avg. sentence length (in words) |
|---|---|---|---|---|
| not depression | 3,660 | 2 | 10,980 | 6 |
| moderate | 66,874 | 29 | 804,794 | 349 |
| severe | 2,880 | 8 | 75,240 | 209 |
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 2e-05
- evaluation_strategy: epoch
- save_strategy: epoch
- per_device_train_batch_size: 8
- per_device_eval_batch_size: 8
- num_train_epochs: 5
- seed: 10
- weight_decay: 0.01
- metric_for_best_model: macro-f1