Update README.md
Browse files
README.md
CHANGED
@@ -21,9 +21,13 @@ We present DistilCamemBERT-NLI which is [DistilCamemBERT](https://huggingface.co
|
|
21 |
This modelization is close to [BaptisteDoyen/camembert-base-xnli](https://huggingface.co/BaptisteDoyen/camembert-base-xnli) based on [CamemBERT](https://huggingface.co/camembert-base) model. The problem of the modelizations based on CamemBERT is at the scaling moment, for the production phase for example. Indeed, inference cost can be a technological issue. To counteract this effect, we propose this modelization which divides the inference time by 2 with the same consumption power thanks to DistilCamemBERT.
|
22 |
|
23 |
Dataset
|
24 |
-
|
25 |
|
26 |
-
The dataset
|
|
|
|
|
|
|
|
|
27 |
|
28 |
| **class** | **precision (%)** | **f1-score (%)** | **support** |
|
29 |
| :----------------: | :---------------: | :--------------: | :---------: |
|
@@ -32,6 +36,8 @@ The dataset (XNLI)[https://huggingface.co/datasets/xnli] is composed of 392,702
|
|
32 |
| **entailment** | 82.90 | 78.87 | 1,670 |
|
33 |
| **neutral** | 72.18 | 74.04 | 1,670 |
|
34 |
|
|
|
|
|
35 |
|
36 |
| **NLI** | **time (ms)** | **MCC (x100)** |
|
37 |
| :--------------: | :-----------: | :------------: |
|
|
|
21 |
This modelization is close to [BaptisteDoyen/camembert-base-xnli](https://huggingface.co/BaptisteDoyen/camembert-base-xnli) based on [CamemBERT](https://huggingface.co/camembert-base) model. The problem of the modelizations based on CamemBERT is at the scaling moment, for the production phase for example. Indeed, inference cost can be a technological issue. To counteract this effect, we propose this modelization which divides the inference time by 2 with the same consumption power thanks to DistilCamemBERT.
|
22 |
|
23 |
Dataset
|
24 |
+
-------
|
25 |
|
26 |
+
The dataset XNLI from [FLUE](https://huggingface.co/datasets/flue) is composed of 392,702 premises with their hypothesis for the train and 5,010 couples for the test. The goal is to predict textual entailment (does sentence A imply/contradict/neither sentence B) and is a classification task (given two sentences, predict one of three labels). The sentence A is called *premise* and sentence B is called *hypothesis*, then the goal of modelization is determined :
|
27 |
+
$$P(premise\in\{contradiction, entailment, neutral\}\vert hypothesis)$$
|
28 |
+
|
29 |
+
Evaluation results
|
30 |
+
------------------
|
31 |
|
32 |
| **class** | **precision (%)** | **f1-score (%)** | **support** |
|
33 |
| :----------------: | :---------------: | :--------------: | :---------: |
|
|
|
36 |
| **entailment** | 82.90 | 78.87 | 1,670 |
|
37 |
| **neutral** | 72.18 | 74.04 | 1,670 |
|
38 |
|
39 |
+
Benchmark
|
40 |
+
---------
|
41 |
|
42 |
| **NLI** | **time (ms)** | **MCC (x100)** |
|
43 |
| :--------------: | :-----------: | :------------: |
|