Binary classification
I am wondering how to use these zero shot classification models for a binary problem. Should I only input one label, assuming the negative entailment is automatically included? E.g., classifying whether the text is about dogs or not about dogs, should I only include one label “dogs” or two labels “dogs”/“not dogs”?
good question. If you only pass a single label like "dog", the probability output will be the "entailment" score for this single lable/hypothesis (f"This text is about {label}"). If you pass two labels like "dog" and "not dog" and you set "multi_label=False" as an argument for the zeroshotpipeline (default), then the output is the softmax over the two entailment scores for "dog" and "not dog", i.e. you get two scores that sum to 1. If you use multi_label=True with two labels/hypotheses, then probabilities are the pure entailment scores for both labels/hypotheses separately. So in this case, the score for "dog" will be the same if you pass "dog" as the only label or any number of additional labels.
What is best depends on your use-case. If you know that all texts must always be about either "dog" or "not dog", then I would pass these two labels and set multi_label=False. If texts can be about many other things and you only want those that mention dogs, I'd just pass the one label.
Very clear, thanks!