robinq commited on
Commit
0dc7756
1 Parent(s): 488040e

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +2 -3
README.md CHANGED
@@ -4,16 +4,15 @@ language:
4
 
5
  ---
6
 
7
- # megatron.bert-base.bpe-64k-no_pretok.25k-steps
8
 
9
  This BERT model was trained using the NeMo library.
10
- The size of the model is a regular bert-base.
11
  The model was trained on more than 245GB of data, consisting mostly of web-data and Swedish newspaper text curated by the National Library of Sweden.
12
 
13
  Training was done for 25k training steps using a batch size of 8k.
14
 
15
  The model has multiple sibling models trained on the same dataset using different tokenizers or more/less parameters:
16
-
17
  - [megatron.bert-base.bpe-32k-no_pretok.25k-steps](https://huggingface.co/KBLab/megatron.bert-base.bpe-32k-no_pretok.25k-steps)
18
  - [megatron.bert-base.bpe-64k-no_pretok.25k-steps](https://huggingface.co/KBLab/megatron.bert-base.bpe-64k-no_pretok.25k-steps)
19
  - [megatron.bert-base.spe-bpe-32k-no_pretok.25k-steps](https://huggingface.co/KBLab/megatron.bert-base.spe-bpe-32k-no_pretok.25k-steps)
 
4
 
5
  ---
6
 
7
+ # megatron.bert-base.bpe-32k-no_pretok.25k-steps
8
 
9
  This BERT model was trained using the NeMo library.
10
+ The size of the model is a regular bert-large.
11
  The model was trained on more than 245GB of data, consisting mostly of web-data and Swedish newspaper text curated by the National Library of Sweden.
12
 
13
  Training was done for 25k training steps using a batch size of 8k.
14
 
15
  The model has multiple sibling models trained on the same dataset using different tokenizers or more/less parameters:
 
16
  - [megatron.bert-base.bpe-32k-no_pretok.25k-steps](https://huggingface.co/KBLab/megatron.bert-base.bpe-32k-no_pretok.25k-steps)
17
  - [megatron.bert-base.bpe-64k-no_pretok.25k-steps](https://huggingface.co/KBLab/megatron.bert-base.bpe-64k-no_pretok.25k-steps)
18
  - [megatron.bert-base.spe-bpe-32k-no_pretok.25k-steps](https://huggingface.co/KBLab/megatron.bert-base.spe-bpe-32k-no_pretok.25k-steps)