---
model-index:
- name: bert-base-dv
  results: []
datasets:
- alakxender/haveeru-articles
language:
- dv
pipeline_tag: fill-mask
library_name: transformers
---
# BERT base for Dhivehi

Pretrained model on Dhivehi language using masked language modeling (MLM).

## Tokenizer

The *WordPiece* tokenizer uses several components:

* **Normalization**: lowercase and then NFKD unicode normalization.
* **Pretokenization**: splits by whitespace and punctuation.
* **Postprocessing**: single sentences are output in format `[CLS] sentence A [SEP]` and pair sentences in format `[CLS] sentence A [SEP] sentence B [SEP]`.

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 32
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 2

### Framework versions

- Transformers 4.41.2
- Pytorch 2.3.1+cu121
- Datasets 2.19.2
- Tokenizers 0.19.1