Update README.md

README.md

@@ -1 +1,15 @@
-cp /home/gdpr/Desktop/Experiments/2021-06/19-01-roberta-lm-mimic/mimic128-4gpu/checkpoint-36000/* ./
+* Continue pre-training RoBERTa-base using discharge summaries from MIMIC-III datasets.
+
+* Details can be found in the following paper
+
+> Xiang Dai and Ilias Chalkidis and Sune Darkner and Desmond Elliott. 2022. Revisiting Transformer-based Models for Long Document Classification. (https://arxiv.org/abs/2204.06683)
+
+* Important hyper-parameters 
+
+| | |
+|---|---|
+| Max sequence | 128 |
+| Batch size | 128 |
+| Learning rate | 5e-5 |
+| Training epochs | 15 |
+| Training time | 40 GPU-hours |