timaeus
/

GPT2-tokenizer-8k-ctx

Model card Files Files and versions Community

GPT2-tokenizer-8k-ctx / added_tokens.json

svwingerden's picture

Duplicate from georgeyw/GPT2-tokenizer-8k-ctx

1f5ef36 verified 4 months ago

history blame contribute delete

29 Bytes

	{
	"<\|endoftext\|>": 50256
	}