README.md

metadata

language:
  - 'no'
  - nb
  - nn
inference: false
tags:
  - BERT
  - NorBERT
  - Norwegian
  - encoder
license: cc-by-4.0

NorBERT 3 small

Other sizes:

• NorBERT 3 xs (15M)
• NorBERT 3 small (40M)
• NorBERT 3 base (123M)
• NorBERT 3 large (323M)

Example usage

This model currently needs a custom wrapper from modeling_norbert.py. Then you can use it like this:

import torch
from transformers import AutoTokenizer
from modeling_norbert import NorbertForMaskedLM

tokenizer = AutoTokenizer.from_pretrained("path/to/folder")
bert = NorbertForMaskedLM.from_pretrained("path/to/folder")

mask_id = tokenizer.convert_tokens_to_ids("[MASK]")
input_text = tokenizer("Nå ønsker de seg en[MASK] bolig.", return_tensors="pt")
output_p = bert(**input_text)
output_text = torch.where(input_text.input_ids == mask_id, output_p.logits.argmax(-1), input_text.input_ids)

# should output: '[CLS] Nå ønsker de seg en ny bolig.[SEP]'
print(tokenizer.decode(output_text[0].tolist()))

The following classes are currently implemented: NorbertForMaskedLM, NorbertForSequenceClassification, NorbertForTokenClassification, NorbertForQuestionAnswering and NorbertForMultipleChoice.