SteveTran/T5-small-query-expansion-INT8

Query Generation

The T5-base model was trained on the MS MARCO Passage Dataset, which consists of about 500k real search queries from Bing together with the relevant passage.

The model can be used for query expansion to learn semantic search models without requiring annotated training data: Synthetic Query Generation.

Usage

from optimum.intel import OVModelForSeq2SeqLM
from transformers import AutoTokenizer, pipeline

model_id = "SteveTran/T5-small-query-expansion-Q4"
model = OVModelForSeq2SeqLM.from_pretrained(model_id, use_cache=True, use_io_binding=False)
tokenizer = AutoTokenizer.from_pretrained(model_id)

instruction = "rewrite: "
prompt = "Who lived longer, Nikola Tesla or Milutin Milankovic?"
inputs = tokenizer(
    ["{} {}".format(instruction, prompt)],
    padding=False,
    return_tensors="pt",
)

outputs = model.generate(**inputs, max_new_tokens=24, use_cache=False, temperature=0.6, do_sample=True, top_p=0.95)
print("Answer: ", tokenizer.batch_decode(outputs, skip_special_tokens=True))
# Nikola Tesla vs Milutin Milankovic lifespan