Edit model card
YAML Metadata Error: "tags" must be an array

Arabic BERT Model

AraBERTMo is an Arabic pre-trained language model based on Google's BERT architechture. AraBERTMo_base uses the same BERT-Base config. AraBERTMo_base now comes in 10 new variants All models are available on the HuggingFace model page under the Ebtihal name. Checkpoints are available in PyTorch formats.

Pretraining Corpus

`AraBertMo_base_V5' model was pre-trained on ~3 million words:

  • OSCAR - Arabic version "unshuffled_deduplicated_ar".

Training results

this model achieves the following results:

Task Num examples Num Epochs Batch Size steps Wall time training loss
Fill-Mask 50046 5 64 3910 6h 49m 59s 7.4599

Load Pretrained Model

You can use this model by installing torch or tensorflow and Huggingface library transformers. And you can use it directly by initializing it like this:

from transformers import AutoTokenizer, AutoModel
tokenizer = AutoTokenizer.from_pretrained("Ebtihal/AraBertMo_base_V5")
model = AutoModelForMaskedLM.from_pretrained("Ebtihal/AraBertMo_base_V5")

This model was built for master's degree research in an organization:

Downloads last month
10
Inference Examples
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.