license: apache-2.0 datasets: - oscar language: - ko
oscar ํ๊ตญ์ด ๋ฐ์ดํฐ์์ ํ์ต๋ SentencePieceUnigramTokenizer ๋ฐ t5 v1.1 ๋ชจ๋ธ์ ๋๋ค.
from transformers import AutoTokenizer, T5ForConditionalGeneration tokenizer = AutoTokenizer.from_pretrained('sangmin6600/t5-v1_1-base-ko') model = T5ForConditionalGeneration.from_pretrained('sangmin6600/t5-v1_1-base-ko')