t5-base-DreamBank-Generation-Act-Char

This model is a fine-tuned version of DReAMy-lib/t5-base-DreamBank-Generation-NER-Char on the DreamBank dataset. The uploaded model contains the weights of the best-performing model (see table below), tune to annotate a given dream report according to Hall and Van de Castle the Activity feature

Model description

The model is trained end-to-end using a text2text solution to annotate dream reports following the Activity feature from the Hall and Van de Castle scoring framework. Given a report, the model generates texts of the form (initialiser : activity type : receiver). For those cases where initialiser and receiver are the same entity, the output will follow the (initialiser : alone activity type : none) setting.

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.001
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum
No log	1.0	49	0.3674	0.4008	0.3122	0.3821	0.3812
No log	2.0	98	0.3200	0.4240	0.3433	0.4130	0.4121
No log	3.0	147	0.2845	0.4591	0.3883	0.4459	0.4455
No log	4.0	196	0.2508	0.4614	0.3930	0.4504	0.4497
No log	5.0	245	0.2632	0.4614	0.3929	0.4467	0.4459
No log	6.0	294	0.2688	0.4706	0.4036	0.4537	0.4534
No log	7.0	343	0.2790	0.4682	0.4043	0.4559	0.4556
No log	8.0	392	0.2895	0.4670	0.3972	0.4529	0.4534
No log	9.0	441	0.3058	0.4708	0.4040	0.4576	0.4572
No log	10.0	490	0.3169	0.4690	0.4001	0.4547	0.4544

Framework versions

• Transformers 4.25.1
• Pytorch 1.12.1
• Datasets 2.5.1
• Tokenizers 0.12.1

Cite

Should use our models in your work, please consider citing us as:

@article{BERTOLINI2024406,
title = {DReAMy: a library for the automatic analysis and annotation of dream reports with multilingual large language models},
journal = {Sleep Medicine},
volume = {115},
pages = {406-407},
year = {2024},
note = {Abstracts from the 17th World Sleep Congress},
issn = {1389-9457},
doi = {https://doi.org/10.1016/j.sleep.2023.11.1092},
url = {https://www.sciencedirect.com/science/article/pii/S1389945723015186},
author = {L. Bertolini and A. Michalak and J. Weeds}
}