Cyrile commited on
Commit
f287bfe
1 Parent(s): e502d25

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +8 -2
README.md CHANGED
@@ -21,9 +21,13 @@ We present DistilCamemBERT-NLI which is [DistilCamemBERT](https://huggingface.co
21
  This modelization is close to [BaptisteDoyen/camembert-base-xnli](https://huggingface.co/BaptisteDoyen/camembert-base-xnli) based on [CamemBERT](https://huggingface.co/camembert-base) model. The problem of the modelizations based on CamemBERT is at the scaling moment, for the production phase for example. Indeed, inference cost can be a technological issue. To counteract this effect, we propose this modelization which divides the inference time by 2 with the same consumption power thanks to DistilCamemBERT.
22
 
23
  Dataset
24
- =======
25
 
26
- The dataset (XNLI)[https://huggingface.co/datasets/xnli] is composed of 392,702 premises with their hypothesis for the train and 5,010 couples for the test.
 
 
 
 
27
 
28
  | **class** | **precision (%)** | **f1-score (%)** | **support** |
29
  | :----------------: | :---------------: | :--------------: | :---------: |
@@ -32,6 +36,8 @@ The dataset (XNLI)[https://huggingface.co/datasets/xnli] is composed of 392,702
32
  | **entailment** | 82.90 | 78.87 | 1,670 |
33
  | **neutral** | 72.18 | 74.04 | 1,670 |
34
 
 
 
35
 
36
  | **NLI** | **time (ms)** | **MCC (x100)** |
37
  | :--------------: | :-----------: | :------------: |
 
21
  This modelization is close to [BaptisteDoyen/camembert-base-xnli](https://huggingface.co/BaptisteDoyen/camembert-base-xnli) based on [CamemBERT](https://huggingface.co/camembert-base) model. The problem of the modelizations based on CamemBERT is at the scaling moment, for the production phase for example. Indeed, inference cost can be a technological issue. To counteract this effect, we propose this modelization which divides the inference time by 2 with the same consumption power thanks to DistilCamemBERT.
22
 
23
  Dataset
24
+ -------
25
 
26
+ The dataset XNLI from [FLUE](https://huggingface.co/datasets/flue) is composed of 392,702 premises with their hypothesis for the train and 5,010 couples for the test. The goal is to predict textual entailment (does sentence A imply/contradict/neither sentence B) and is a classification task (given two sentences, predict one of three labels). The sentence A is called *premise* and sentence B is called *hypothesis*, then the goal of modelization is determined :
27
+ $$P(premise\in\{contradiction, entailment, neutral\}\vert hypothesis)$$
28
+
29
+ Evaluation results
30
+ ------------------
31
 
32
  | **class** | **precision (%)** | **f1-score (%)** | **support** |
33
  | :----------------: | :---------------: | :--------------: | :---------: |
 
36
  | **entailment** | 82.90 | 78.87 | 1,670 |
37
  | **neutral** | 72.18 | 74.04 | 1,670 |
38
 
39
+ Benchmark
40
+ ---------
41
 
42
  | **NLI** | **time (ms)** | **MCC (x100)** |
43
  | :--------------: | :-----------: | :------------: |