Update README.md
Browse files
README.md
CHANGED
@@ -9,13 +9,13 @@ pipeline_tag: zero-shot-classification
|
|
9 |
---
|
10 |
|
11 |
# Presentation
|
12 |
-
We introduce the Bloomz-560m-NLI model, fine-tuned
|
13 |
This model is trained on a Natural Language Inference (NLI) task in a language-agnostic manner. The NLI task involves determining the semantic relationship
|
14 |
between a hypothesis and a set of premises, often expressed as pairs of sentences.
|
15 |
|
16 |
-
The goal is to predict textual entailment (does sentence A imply/contradict/neither sentence B?) and is a classification task (given two sentences, predict one of
|
17 |
three labels).
|
18 |
-
|
19 |
$$P(premise=c\in\{contradiction, entailment, neutral\}\vert hypothesis)$$
|
20 |
|
21 |
### Language-agnostic approach
|
@@ -32,7 +32,7 @@ It should be noted that hypotheses and premises are randomly chosen between Engl
|
|
32 |
|
33 |
### Benchmark
|
34 |
|
35 |
-
Here are the performances for the hypothesis and premise in French:
|
36 |
|
37 |
| **model** | **accuracy (%)** | **MCC (x100)** |
|
38 |
| :--------------: | :--------------: | :------------: |
|
@@ -55,14 +55,14 @@ And now the hypothesis in French and the premise in English (cross-language cont
|
|
55 |
| [cmarkea/bloomz-7b1-mt-nli](https://huggingface.co/cmarkea/bloomz-7b1-mt-nli) | 85.43 | 78.25 |
|
56 |
|
57 |
# Zero-shot Classification
|
58 |
-
The primary
|
59 |
-
without specific training. What sets the Bloomz-560m-NLI LLMs apart in this
|
60 |
and lengthy test structures compared to models like BERT, RoBERTa, or CamemBERT.
|
61 |
|
62 |
The zero-shot classification task can be summarized by:
|
63 |
$$P(hypothesis=i\in\mathcal{C}|premise)=\frac{e^{P(premise=entailment\vert hypothesis=i)}}{\sum_{j\in\mathcal{C}}e^{P(premise=entailment\vert hypothesis=j)}}$$
|
64 |
With *i* representing a hypothesis composed of a template (for example, "This text is about {}.") and *#C* candidate labels ("cinema", "politics", etc.), the set
|
65 |
-
of hypotheses
|
66 |
is the sentence we aim to classify.
|
67 |
|
68 |
### Performance
|
|
|
9 |
---
|
10 |
|
11 |
# Presentation
|
12 |
+
We introduce the Bloomz-560m-NLI model, fine-tuned from the [Bloomz-560m-chat-dpo](https://huggingface.co/cmarkea/bloomz-560m-dpo-chat) foundation model.
|
13 |
This model is trained on a Natural Language Inference (NLI) task in a language-agnostic manner. The NLI task involves determining the semantic relationship
|
14 |
between a hypothesis and a set of premises, often expressed as pairs of sentences.
|
15 |
|
16 |
+
The goal is to predict textual entailment (does sentence A imply/contradict/neither sentence B?) and is a classification task (given two sentences, predict one of the
|
17 |
three labels).
|
18 |
+
If sentence A is called *premise*, and sentence B is called *hypothesis*, then the goal of the modelization is to estimate the following:
|
19 |
$$P(premise=c\in\{contradiction, entailment, neutral\}\vert hypothesis)$$
|
20 |
|
21 |
### Language-agnostic approach
|
|
|
32 |
|
33 |
### Benchmark
|
34 |
|
35 |
+
Here are the performances for both the hypothesis and premise in French:
|
36 |
|
37 |
| **model** | **accuracy (%)** | **MCC (x100)** |
|
38 |
| :--------------: | :--------------: | :------------: |
|
|
|
55 |
| [cmarkea/bloomz-7b1-mt-nli](https://huggingface.co/cmarkea/bloomz-7b1-mt-nli) | 85.43 | 78.25 |
|
56 |
|
57 |
# Zero-shot Classification
|
58 |
+
The primary interest of training such models lies in their zero-shot classification performance. This means that the model is able to classify any text with any label
|
59 |
+
without a specific training. What sets the Bloomz-560m-NLI LLMs apart in this domain is their ability to model and extract information from significantly more complex
|
60 |
and lengthy test structures compared to models like BERT, RoBERTa, or CamemBERT.
|
61 |
|
62 |
The zero-shot classification task can be summarized by:
|
63 |
$$P(hypothesis=i\in\mathcal{C}|premise)=\frac{e^{P(premise=entailment\vert hypothesis=i)}}{\sum_{j\in\mathcal{C}}e^{P(premise=entailment\vert hypothesis=j)}}$$
|
64 |
With *i* representing a hypothesis composed of a template (for example, "This text is about {}.") and *#C* candidate labels ("cinema", "politics", etc.), the set
|
65 |
+
of hypotheses is composed of {"This text is about cinema.", "This text is about politics.", ...}. It is these hypotheses that we will measure against the premise, which
|
66 |
is the sentence we aim to classify.
|
67 |
|
68 |
### Performance
|