Update README.md
Browse files
README.md
CHANGED
@@ -11,7 +11,7 @@ tags:
|
|
11 |
|
12 |
# German Medical BERT
|
13 |
|
14 |
-
This is a fine-tuned model on Medical domain for German language and based on German BERT. This model has only been trained to improve on
|
15 |
|
16 |
## Overview
|
17 |
**Language model:** bert-base-german-cased
|
@@ -22,26 +22,26 @@ This is a fine-tuned model on Medical domain for German language and based on Ge
|
|
22 |
|
23 |
**Eval data:** NTS-ICD-10 dataset (Classification)
|
24 |
|
25 |
-
**Infrastructure:**
|
26 |
|
27 |
|
28 |
## Details
|
29 |
- We fine-tuned using Pytorch with Huggingface library on Colab GPU.
|
30 |
-
- With standard parameter settings for fine-tuning as mentioned in original BERT
|
31 |
-
- Although had to train for
|
32 |
|
33 |
## Performance (Micro precision, recall and f1 score for multilabel code classification)
|
34 |
|
35 |
-
|Models
|
36 |
-
|
37 |
-
|German BERT
|
38 |
-
|German MedBERT-256
|
39 |
-
|German MedBERT-512
|
40 |
|
41 |
## Author
|
42 |
Manjil Shrestha: `shresthamanjil21 [at] gmail.com`
|
43 |
|
44 |
-
## Related Paper
|
45 |
[Report](https://opus4.kobv.de/opus4-rhein-waal/frontdoor/index/index/searchtype/collection/id/16225/start/0/rows/10/doctypefq/masterthesis/docId/740)
|
46 |
|
47 |
Get in touch:
|
|
|
11 |
|
12 |
# German Medical BERT
|
13 |
|
14 |
+
This is a fine-tuned model on the Medical domain for the German language and based on German BERT. This model has only been trained to improve on-target task (Masked Language Model). It can later be used to perform a downstream task of your needs, while I performed it for the NTS-ICD-10 text classification task.
|
15 |
|
16 |
## Overview
|
17 |
**Language model:** bert-base-german-cased
|
|
|
22 |
|
23 |
**Eval data:** NTS-ICD-10 dataset (Classification)
|
24 |
|
25 |
+
**Infrastructure:** Google Colab
|
26 |
|
27 |
|
28 |
## Details
|
29 |
- We fine-tuned using Pytorch with Huggingface library on Colab GPU.
|
30 |
+
- With standard parameter settings for fine-tuning as mentioned in the original BERT paper.
|
31 |
+
- Although had to train for up to 25 epochs for classification.
|
32 |
|
33 |
## Performance (Micro precision, recall and f1 score for multilabel code classification)
|
34 |
|
35 |
+
|Models\\\\\\\\t\\\\\\\\t\\\\\\\\t|P\\\\\\\\t|R\\\\\\\\t|F1\\\\\\\\t|
|
36 |
+
|:--------------\\\\\\\\t|:------|:------|:------|
|
37 |
+
|German BERT\\\\\\\\t\\\\\\\\t|86.04\\\\\\\\t|75.82\\\\\\\\t|80.60\\\\\\\\t|
|
38 |
+
|German MedBERT-256\\\\\\\\t|87.41\\\\\\\\t|77.97\\\\\\\\t|82.42\\\\\\\\t|
|
39 |
+
|German MedBERT-512\\\\\\\\t|87.75\\\\\\\\t|78.26\\\\\\\\t|82.73\\\\\\\\t|
|
40 |
|
41 |
## Author
|
42 |
Manjil Shrestha: `shresthamanjil21 [at] gmail.com`
|
43 |
|
44 |
+
## Related Paper:
|
45 |
[Report](https://opus4.kobv.de/opus4-rhein-waal/frontdoor/index/index/searchtype/collection/id/16225/start/0/rows/10/doctypefq/masterthesis/docId/740)
|
46 |
|
47 |
Get in touch:
|