expand usage instructions in README (#2)

- expand usage instructions in README (58795918e6dc0d4b8fdca281a229a1a5cf09edc2)


Co-authored-by: Sam <sam-mosaic@users.noreply.huggingface.co>

README.md

@@ -8,26 +8,27 @@ inference: false
 ---
 
 # MosaicBERT-Base model
+
 MosaicBERT-Base is a new BERT architecture and training recipe optimized for fast pretraining. 
 MosaicBERT trains faster and achieves higher pretraining and finetuning accuracy when benchmarked against 
 Hugging Face's [bert-base-uncased](https://huggingface.co/bert-base-uncased).
 
-### Model Date
+## Model Date
 
 March 2023
 
 ## Documentation
+
 * Blog post
 * [Github (mosaicml/examples/bert repo)](https://github.com/mosaicml/examples/tree/main/examples/bert)
 
-# How to use
-
-We recommend using the code in the [mosaicml/examples/bert repo](https://github.com/mosaicml/examples/tree/main/examples/bert) for pretraining and finetuning this model.
+## How to use
 
 ```python
 from transformers import AutoModelForMaskedLM
 mlm = AutoModelForMaskedLM.from_pretrained('mosaicml/mosaic-bert-base', trust_remote_code=True)
 ```
+
 The tokenizer for this model is simply the Hugging Face `bert-base-uncased` tokenizer.
 
 ```python
@@ -35,6 +36,37 @@ from transformers import BertTokenizer
 tokenizer = BertTokenizer.from_pretrained('bert-base-uncased')
 ```
 
+To use this model directly for masked language modeling, use `pipeline`:
+
+```python
+from transformers import AutoModelForMaskedLM, BertTokenizer, pipeline
+
+tokenizer = BertTokenizer.from_pretrained('bert-base-uncased')
+mlm = AutoModelForMaskedLM.from_pretrained('mosaicml/mosaic-bert-base', trust_remote_code=True)
+
+classifier = pipeline('fill-mask', model=mlm, tokenizer=tokenizer)
+
+classifier("I [MASK] to the store yesterday.")
+```
+
+**To continue MLM pretraining**, follow the [MLM pre-training section of the mosaicml/examples/bert repo](https://github.com/mosaicml/examples/tree/main/examples/bert#mlm-pre-training).
+
+**To fine-tune this model for classification**, follow the [Single-task fine-tuning section of the mosaicml/examples/bert repo](https://github.com/mosaicml/examples/tree/main/examples/bert#single-task-fine-tuning).
+
+### Remote Code
+
+This model requires that `trust_remote_code=True` be passed to the `from_pretrained` method. This is because we train using [FlashAttention (Dao et al. 2022)](https://arxiv.org/pdf/2205.14135.pdf), which is not part of the `transformers` library and depends on [Triton](https://github.com/openai/triton) and some custom PyTorch code. Since this involves executing arbitrary code, you should consider passing a git `revision` argument that specifies the exact commit of the code, for example:
+
+```python
+mlm = AutoModelForMaskedLM.from_pretrained(
+   'mosaicml/mosaic-bert-base',
+   trust_remote_code=True,
+   revision='24512df',
+)
+```
+
+However, if there are updates to this model or code and you specify a revision, you will need to manually check for them and update the commit hash accordingly.
+
 ## Model description
 
 In order to build MosaicBERT, we adopted architectural choices from the recent transformer literature.