jacobfulano
commited on
Commit
•
6fccab2
1
Parent(s):
82a6d46
Update README.md
Browse files
README.md
CHANGED
@@ -7,7 +7,7 @@ language:
|
|
7 |
inference: false
|
8 |
---
|
9 |
|
10 |
-
# MosaicBERT: mosaic-bert-base-seqlen-
|
11 |
|
12 |
MosaicBERT-Base is a new BERT architecture and training recipe optimized for fast pretraining.
|
13 |
MosaicBERT trains faster and achieves higher pretraining and finetuning accuracy when benchmarked against
|
@@ -42,7 +42,7 @@ April 2023
|
|
42 |
|
43 |
```python
|
44 |
from transformers import AutoModelForMaskedLM
|
45 |
-
mlm = AutoModelForMaskedLM.from_pretrained('mosaicml/mosaic-bert-base', trust_remote_code=True)
|
46 |
```
|
47 |
|
48 |
The tokenizer for this model is simply the Hugging Face `bert-base-uncased` tokenizer.
|
@@ -58,7 +58,7 @@ To use this model directly for masked language modeling, use `pipeline`:
|
|
58 |
from transformers import AutoModelForMaskedLM, BertTokenizer, pipeline
|
59 |
|
60 |
tokenizer = BertTokenizer.from_pretrained('bert-base-uncased')
|
61 |
-
mlm = AutoModelForMaskedLM.from_pretrained('mosaicml/mosaic-bert-base', trust_remote_code=True)
|
62 |
|
63 |
classifier = pipeline('fill-mask', model=mlm, tokenizer=tokenizer)
|
64 |
|
@@ -75,7 +75,7 @@ This model requires that `trust_remote_code=True` be passed to the `from_pretrai
|
|
75 |
|
76 |
```python
|
77 |
mlm = AutoModelForMaskedLM.from_pretrained(
|
78 |
-
'mosaicml/mosaic-bert-base',
|
79 |
trust_remote_code=True,
|
80 |
revision='24512df',
|
81 |
)
|
|
|
7 |
inference: false
|
8 |
---
|
9 |
|
10 |
+
# MosaicBERT: mosaic-bert-base-seqlen-256 Pretrained Model
|
11 |
|
12 |
MosaicBERT-Base is a new BERT architecture and training recipe optimized for fast pretraining.
|
13 |
MosaicBERT trains faster and achieves higher pretraining and finetuning accuracy when benchmarked against
|
|
|
42 |
|
43 |
```python
|
44 |
from transformers import AutoModelForMaskedLM
|
45 |
+
mlm = AutoModelForMaskedLM.from_pretrained('mosaicml/mosaic-bert-base-seqlen-256', trust_remote_code=True)
|
46 |
```
|
47 |
|
48 |
The tokenizer for this model is simply the Hugging Face `bert-base-uncased` tokenizer.
|
|
|
58 |
from transformers import AutoModelForMaskedLM, BertTokenizer, pipeline
|
59 |
|
60 |
tokenizer = BertTokenizer.from_pretrained('bert-base-uncased')
|
61 |
+
mlm = AutoModelForMaskedLM.from_pretrained('mosaicml/mosaic-bert-base-seqlen-256', trust_remote_code=True)
|
62 |
|
63 |
classifier = pipeline('fill-mask', model=mlm, tokenizer=tokenizer)
|
64 |
|
|
|
75 |
|
76 |
```python
|
77 |
mlm = AutoModelForMaskedLM.from_pretrained(
|
78 |
+
'mosaicml/mosaic-bert-base-seqlen-256',
|
79 |
trust_remote_code=True,
|
80 |
revision='24512df',
|
81 |
)
|