cross-encoder/ms-marco-TinyBERT-L-2-v2 not returning list of scores

#2
by shivdesh - opened

Hello, When invoking this model for a list of ('Query', 'Paragraph1') I am expecting to get a list of scores as output:

For example,

from sentence_transformers import CrossEncoder

model = CrossEncoder("cross-encoder/ms-marco-TinyBERT-L-2-v2", max_length=512)
scores = model.predict(
[('How many people live in Berlin?',
'Berlin had a population of 3,520,031 registered inhabitants in an area of 891.82 square kilometers.'),
('How many people live in Berlin?', 'Berlin is well known for its museums.')]
)

print("scores:", scores)

scores: [ 7.1523647 -6.2870436]

This is what I get while running in Python code.

But while running thru Inference API on the Model Card page, as well as in Sagemaker deployment I just get one score, as an output:
Sagemaker deploy params used:
HF_MODEL_ID: cross-encoder/ms-marco-TinyBERT-L-2-v2
HF_TASK: text-classification
Output:
[
[
{
"label": "LABEL_0",
"score": 0.00012489620712585747
}
]
]

Anything I am missing? How can I get a list of scores as an output (In Sagemaker deployment), in above example?

Sign up or log in to comment