|
--- |
|
license: apache-2.0 |
|
library_name: mlx-llm |
|
language: |
|
- en |
|
tags: |
|
- mlx |
|
- exbert |
|
datasets: |
|
- bookcorpus |
|
- wikipedia |
|
--- |
|
|
|
|
|
# BERT large model (uncased) - MLX |
|
|
|
Pretrained model on English language using a masked language modeling (MLM) objective. It was introduced in |
|
[this paper](https://arxiv.org/abs/1810.04805) and first released in |
|
[this repository](https://github.com/google-research/bert). This model is uncased: it does not make a difference |
|
between english and English. |
|
|
|
|
|
## Model description |
|
|
|
Please, refer to the [original model card](https://huggingface.co/bert-large-uncased) for more details on bert-base-uncased. |
|
|
|
## Use it with mlx-llm |
|
|
|
Install `mlx-llm` from GitHub. |
|
```bash |
|
git clone https://github.com/riccardomusmeci/mlx-llm |
|
cd mlx-llm |
|
pip install . |
|
``` |
|
|
|
Run |
|
```python |
|
from mlx_llm.model import create_model |
|
from transformers import BertTokenizer |
|
import mlx.core as mx |
|
|
|
model = create_model("bert-large-uncased") # it will download weights from this repository |
|
tokenizer = BertTokenizer.from_pretrained("bert-large-uncased") |
|
|
|
batch = ["This is an example of BERT working on MLX."] |
|
tokens = tokenizer(batch, return_tensors="np", padding=True) |
|
tokens = {key: mx.array(v) for key, v in tokens.items()} |
|
|
|
output, pooled = model(**tokens) |
|
``` |
