File size: 1,517 Bytes
38be2b2 91dd9d9 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 |
---
title: README
emoji: 📉
colorFrom: green
colorTo: yellow
sdk: static
pinned: false
---
# German Wikipedia LMs (GWLMs)
We present Language Models ([BERT](https://huggingface.co/gwlms/bert-base-dewiki-v1), [BERT with Token Dropping](https://huggingface.co/gwlms/bert-base-token-dropping-dewiki-v1), [TEAMS](https://huggingface.co/gwlms/teams-base-dewiki-v1-discriminator), [T5](https://huggingface.co/gwlms/t5-efficient-large-dewiki-v1)) pretrained on German Wikipedia.
This is an ongoing project!
# German Wikipedia Corpus
We use a recent Wikipedia Dump, that can can be accessed [here](https://huggingface.co/datasets/gwlms/dewiki-20230701). Additionally, a sentence-segmented (using NLTK) is available [here](https://huggingface.co/datasets/gwlms/dewiki-20230701-nltk-corpus).
# Fine-tuned Models
We fine-tuned NER models using [SpanMarker](https://github.com/tomaarsen/SpanMarkerNER) library on GermEval 2014 NER dataset and upload the best models:
* [GermEval 2014 NER Model with BERT as backbone LM](https://huggingface.co/gwlms/span-marker-bert-germeval14)
* [GermEval 2014 NER with BERT + Token Dropping as backbone LM](https://huggingface.co/gwlms/span-marker-token-dropping-bert-germeval14)
* [GermEval 2014 NER Model with TEAMS as backbone LM](https://huggingface.co/gwlms/span-marker-teams-germeval14)
# Acknowledgements
Research supported with Cloud TPUs from Google's [TPU Research Cloud](https://sites.research.google/trc/about/) (TRC).
Many Thanks for providing access to the TPUs ❤️ |