jinyong-gpt2 / tokenizer_config.json
supermy's picture
Upload tokenizer
822f766
{
"add_special_token": false,
"lowercase": true,
"name_or_path": "jinyong-gpt2-finetuning",
"pad_token": "[PAD]",
"special_tokens_map_file": "vocab/jinyong-bpe/special_tokens_map.json",
"tokenizer_class": "PreTrainedTokenizerFast"
}