Model Description

This is a sharded version of the T5-11B-SSM-NQ model, fine-tuned on the Natural Questions dataset for text-to-text generation tasks. The model is stored and processed in multiple shards to facilitate easier handling of its large size (11 billion parameters).

Usage

This model can be used for text-to-text generation tasks like question answering and text summarization.

from transformers import AutoModelForSeq2SeqLM, AutoTokenizer
import torch

tokenizer = AutoTokenizer.from_pretrained('iarroyof/t5-11b-ssm-nq-sharded')
model = AutoModelForSeq2SeqLM.from_pretrained(
    'iarroyof/t5-11b-ssm-nq-sharded',
    device_map='auto',
    low_cpu_mem_usage=True,
    torch_dtype=torch.float16,
)

inputs = tokenizer('What is and how to deal with insomnia?', return_tensors='pt').input_ids.to('cuda')
outputs = model.generate(inputs)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
---
Downloads last month
16
Safetensors
Model size
11.3B params
Tensor type
F32
·
FP16
·
Inference Providers NEW
This model is not currently available via any of the supported Inference Providers.

Model tree for iarroyof/t5-11b-ssm-nq-sharded

Finetuned
(1)
this model

Dataset used to train iarroyof/t5-11b-ssm-nq-sharded