sinhala-roberta-sentence-transformer / tokenizer_config.json
Ransaka's picture
Pushing sample model
bf874e5
raw
history blame
254 Bytes
{
"clean_up_tokenization_spaces": true,
"max_length": 512,
"model_max_length": 1000000000000000019884624838656,
"stride": 0,
"tokenizer_class": "PreTrainedTokenizerFast",
"truncation_side": "right",
"truncation_strategy": "longest_first"
}