metadata
language: ru
tags:
- normalization
- denoising autoencoder
- russian
widget:
- text: меня тобой не понимать
license: mit
This is a small Russian denoising autoencoder. It can be used for restoring corrupted sentences.
This model was produced by fine-tuning the rut5-small model on the task of reconstructing a sentence:
- restoring word positions (after slightly shuffling them)
- restoring dropped words and punctuation marks (after dropping some of them randomly)
- restoring inflection of words (after changing their inflection randomly using natasha and pymorphy2 packages)
The fine-tuning was performed on a Leipzig web corpus of Russian sentences.
The model can be applied as follows:
# !pip install transformers sentencepiece
import torch
from transformers import T5ForConditionalGeneration, T5Tokenizer
tokenizer = T5Tokenizer.from_pretrained("cointegrated/rut5-small-normalizer")
model = T5ForConditionalGeneration.from_pretrained("cointegrated/rut5-small-normalizer")
text = 'меня тобой не понимать'
inputs = tokenizer(text, return_tensors='pt')
with torch.no_grad():
hypotheses = model.generate(
**inputs,
do_sample=True, top_p=0.95,
num_return_sequences=5,
repetition_penalty=2.5,
max_length=32,
)
for h in hypotheses:
print(tokenizer.decode(h, skip_special_tokens=True))
A possible output is:
# Мне тебя не понимать.
# Если бы ты понимаешь меня?
# Я с тобой не понимаю.
# Я тебя не понимаю.
# Я не понимаю о чем ты.