asahi417 commited on
Commit
555220b
1 Parent(s): 45fe1c6

model update

Browse files
Files changed (1) hide show
  1. README.md +25 -40
README.md CHANGED
@@ -33,44 +33,25 @@ model-index:
33
  metrics:
34
  - name: BLEU4
35
  type: bleu4
36
- value: 0.2266718077651334
37
  - name: ROUGE-L
38
  type: rouge-l
39
- value: 0.4953920093132268
40
  - name: METEOR
41
  type: meteor
42
- value: 0.24675728793398077
43
  - name: BERTScore
44
  type: bertscore
45
- value: 0.9016671782685416
46
  - name: MoverScore
47
  type: moverscore
48
- value: 0.630633217445346
49
  ---
50
 
51
  # Model Card of `lmqg/t5-small-squad-default`
52
- This model is fine-tuned version of [t5-small](https://huggingface.co/t5-small) for question generation task on the
53
- [lmqg/qg_squad](https://huggingface.co/datasets/lmqg/qg_squad) (dataset_name: default) via [`lmqg`](https://github.com/asahi417/lm-question-generation).
54
  This model is fine-tuned without parameter search (default configuration is taken from [ERNIE-GEN](https://arxiv.org/abs/2001.11314)).
55
 
56
- Please cite our paper if you use the model ([https://arxiv.org/abs/2210.03992](https://arxiv.org/abs/2210.03992)).
57
-
58
- ```
59
-
60
- @inproceedings{ushio-etal-2022-generative,
61
- title = "{G}enerative {L}anguage {M}odels for {P}aragraph-{L}evel {Q}uestion {G}eneration",
62
- author = "Ushio, Asahi and
63
- Alva-Manchego, Fernando and
64
- Camacho-Collados, Jose",
65
- booktitle = "Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing",
66
- month = dec,
67
- year = "2022",
68
- address = "Abu Dhabi, U.A.E.",
69
- publisher = "Association for Computational Linguistics",
70
- }
71
-
72
- ```
73
-
74
  ### Overview
75
  - **Language model:** [t5-small](https://huggingface.co/t5-small)
76
  - **Language:** en
@@ -82,35 +63,40 @@ Please cite our paper if you use the model ([https://arxiv.org/abs/2210.03992](h
82
  ### Usage
83
  - With [`lmqg`](https://github.com/asahi417/lm-question-generation#lmqg-language-model-for-question-generation-)
84
  ```python
85
-
86
  from lmqg import TransformersQG
 
87
  # initialize model
88
- model = TransformersQG(language='en', model='lmqg/t5-small-squad-default')
 
89
  # model prediction
90
- question = model.generate_q(list_context=["William Turner was an English painter who specialised in watercolour landscapes"], list_answer=["William Turner"])
91
 
92
  ```
93
 
94
  - With `transformers`
95
  ```python
96
-
97
  from transformers import pipeline
98
- # initialize model
99
- pipe = pipeline("text2text-generation", 'lmqg/t5-small-squad-default')
100
- # question generation
101
- question = pipe('generate question: <hl> Beyonce <hl> further expanded her acting career, starring as blues singer Etta James in the 2008 musical biopic, Cadillac Records.')
102
 
103
- ```
 
104
 
105
- ## Evaluation Metrics
106
 
 
107
 
108
- ### Metrics
109
 
110
- | Dataset | Type | BLEU4 | ROUGE-L | METEOR | BERTScore | MoverScore | Link |
111
- |:--------|:-----|------:|--------:|-------:|----------:|-----------:|-----:|
112
- | [lmqg/qg_squad](https://huggingface.co/datasets/lmqg/qg_squad) | default | 0.227 | 0.495 | 0.247 | 0.902 | 0.631 | [link](https://huggingface.co/lmqg/t5-small-squad-default/raw/main/eval/metric.first.sentence.paragraph_answer.question.lmqg_qg_squad.default.json) |
113
 
 
 
 
 
 
 
 
 
 
 
114
 
115
 
116
 
@@ -137,7 +123,6 @@ The full configuration can be found at [fine-tuning config file](https://hugging
137
 
138
  ## Citation
139
  ```
140
-
141
  @inproceedings{ushio-etal-2022-generative,
142
  title = "{G}enerative {L}anguage {M}odels for {P}aragraph-{L}evel {Q}uestion {G}eneration",
143
  author = "Ushio, Asahi and
 
33
  metrics:
34
  - name: BLEU4
35
  type: bleu4
36
+ value: 22.67
37
  - name: ROUGE-L
38
  type: rouge-l
39
+ value: 49.54
40
  - name: METEOR
41
  type: meteor
42
+ value: 24.68
43
  - name: BERTScore
44
  type: bertscore
45
+ value: 90.17
46
  - name: MoverScore
47
  type: moverscore
48
+ value: 63.06
49
  ---
50
 
51
  # Model Card of `lmqg/t5-small-squad-default`
52
+ This model is fine-tuned version of [t5-small](https://huggingface.co/t5-small) for question generation task on the [lmqg/qg_squad](https://huggingface.co/datasets/lmqg/qg_squad) (dataset_name: default) via [`lmqg`](https://github.com/asahi417/lm-question-generation).
 
53
  This model is fine-tuned without parameter search (default configuration is taken from [ERNIE-GEN](https://arxiv.org/abs/2001.11314)).
54
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
55
  ### Overview
56
  - **Language model:** [t5-small](https://huggingface.co/t5-small)
57
  - **Language:** en
 
63
  ### Usage
64
  - With [`lmqg`](https://github.com/asahi417/lm-question-generation#lmqg-language-model-for-question-generation-)
65
  ```python
 
66
  from lmqg import TransformersQG
67
+
68
  # initialize model
69
+ model = TransformersQG(language="en", model="lmqg/t5-small-squad-default")
70
+
71
  # model prediction
72
+ questions = model.generate_q(list_context="William Turner was an English painter who specialised in watercolour landscapes", list_answer="William Turner")
73
 
74
  ```
75
 
76
  - With `transformers`
77
  ```python
 
78
  from transformers import pipeline
 
 
 
 
79
 
80
+ pipe = pipeline("text2text-generation", "lmqg/t5-small-squad-default")
81
+ output = pipe("generate question: <hl> Beyonce <hl> further expanded her acting career, starring as blues singer Etta James in the 2008 musical biopic, Cadillac Records.")
82
 
83
+ ```
84
 
85
+ ## Evaluation
86
 
 
87
 
88
+ - ***Metric (Question Generation)***: [raw metric file](https://huggingface.co/lmqg/t5-small-squad-default/raw/main/eval/metric.first.sentence.paragraph_answer.question.lmqg_qg_squad.default.json)
 
 
89
 
90
+ | | Score | Type | Dataset |
91
+ |:-----------|--------:|:--------|:---------------------------------------------------------------|
92
+ | BERTScore | 90.17 | default | [lmqg/qg_squad](https://huggingface.co/datasets/lmqg/qg_squad) |
93
+ | Bleu_1 | 54.85 | default | [lmqg/qg_squad](https://huggingface.co/datasets/lmqg/qg_squad) |
94
+ | Bleu_2 | 38.46 | default | [lmqg/qg_squad](https://huggingface.co/datasets/lmqg/qg_squad) |
95
+ | Bleu_3 | 29.07 | default | [lmqg/qg_squad](https://huggingface.co/datasets/lmqg/qg_squad) |
96
+ | Bleu_4 | 22.67 | default | [lmqg/qg_squad](https://huggingface.co/datasets/lmqg/qg_squad) |
97
+ | METEOR | 24.68 | default | [lmqg/qg_squad](https://huggingface.co/datasets/lmqg/qg_squad) |
98
+ | MoverScore | 63.06 | default | [lmqg/qg_squad](https://huggingface.co/datasets/lmqg/qg_squad) |
99
+ | ROUGE_L | 49.54 | default | [lmqg/qg_squad](https://huggingface.co/datasets/lmqg/qg_squad) |
100
 
101
 
102
 
 
123
 
124
  ## Citation
125
  ```
 
126
  @inproceedings{ushio-etal-2022-generative,
127
  title = "{G}enerative {L}anguage {M}odels for {P}aragraph-{L}evel {Q}uestion {G}eneration",
128
  author = "Ushio, Asahi and