model update
Browse files
README.md
CHANGED
@@ -33,44 +33,25 @@ model-index:
|
|
33 |
metrics:
|
34 |
- name: BLEU4
|
35 |
type: bleu4
|
36 |
-
value:
|
37 |
- name: ROUGE-L
|
38 |
type: rouge-l
|
39 |
-
value:
|
40 |
- name: METEOR
|
41 |
type: meteor
|
42 |
-
value:
|
43 |
- name: BERTScore
|
44 |
type: bertscore
|
45 |
-
value:
|
46 |
- name: MoverScore
|
47 |
type: moverscore
|
48 |
-
value:
|
49 |
---
|
50 |
|
51 |
# Model Card of `lmqg/t5-base-squadshifts-new_wiki`
|
52 |
-
This model is fine-tuned version of [lmqg/t5-base-squad](https://huggingface.co/lmqg/t5-base-squad) for question generation task on the
|
53 |
-
[lmqg/qg_squadshifts](https://huggingface.co/datasets/lmqg/qg_squadshifts) (dataset_name: new_wiki) via [`lmqg`](https://github.com/asahi417/lm-question-generation).
|
54 |
This model is continuously fine-tuned with [lmqg/t5-base-squad](https://huggingface.co/lmqg/t5-base-squad).
|
55 |
|
56 |
-
Please cite our paper if you use the model ([https://arxiv.org/abs/2210.03992](https://arxiv.org/abs/2210.03992)).
|
57 |
-
|
58 |
-
```
|
59 |
-
|
60 |
-
@inproceedings{ushio-etal-2022-generative,
|
61 |
-
title = "{G}enerative {L}anguage {M}odels for {P}aragraph-{L}evel {Q}uestion {G}eneration",
|
62 |
-
author = "Ushio, Asahi and
|
63 |
-
Alva-Manchego, Fernando and
|
64 |
-
Camacho-Collados, Jose",
|
65 |
-
booktitle = "Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing",
|
66 |
-
month = dec,
|
67 |
-
year = "2022",
|
68 |
-
address = "Abu Dhabi, U.A.E.",
|
69 |
-
publisher = "Association for Computational Linguistics",
|
70 |
-
}
|
71 |
-
|
72 |
-
```
|
73 |
-
|
74 |
### Overview
|
75 |
- **Language model:** [lmqg/t5-base-squad](https://huggingface.co/lmqg/t5-base-squad)
|
76 |
- **Language:** en
|
@@ -82,35 +63,40 @@ Please cite our paper if you use the model ([https://arxiv.org/abs/2210.03992](h
|
|
82 |
### Usage
|
83 |
- With [`lmqg`](https://github.com/asahi417/lm-question-generation#lmqg-language-model-for-question-generation-)
|
84 |
```python
|
85 |
-
|
86 |
from lmqg import TransformersQG
|
|
|
87 |
# initialize model
|
88 |
-
model = TransformersQG(language=
|
|
|
89 |
# model prediction
|
90 |
-
|
91 |
|
92 |
```
|
93 |
|
94 |
- With `transformers`
|
95 |
```python
|
96 |
-
|
97 |
from transformers import pipeline
|
98 |
-
# initialize model
|
99 |
-
pipe = pipeline("text2text-generation", 'lmqg/t5-base-squadshifts-new_wiki')
|
100 |
-
# question generation
|
101 |
-
question = pipe('generate question: <hl> Beyonce <hl> further expanded her acting career, starring as blues singer Etta James in the 2008 musical biopic, Cadillac Records.')
|
102 |
|
103 |
-
|
|
|
104 |
|
105 |
-
|
106 |
|
|
|
107 |
|
108 |
-
### Metrics
|
109 |
|
110 |
-
|
111 |
-
|:--------|:-----|------:|--------:|-------:|----------:|-----------:|-----:|
|
112 |
-
| [lmqg/qg_squadshifts](https://huggingface.co/datasets/lmqg/qg_squadshifts) | new_wiki | 0.117 | 0.295 | 0.27 | 0.931 | 0.659 | [link](https://huggingface.co/lmqg/t5-base-squadshifts-new_wiki/raw/main/eval/metric.first.sentence.paragraph_answer.question.lmqg_qg_squadshifts.new_wiki.json) |
|
113 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
114 |
|
115 |
|
116 |
|
@@ -137,7 +123,6 @@ The full configuration can be found at [fine-tuning config file](https://hugging
|
|
137 |
|
138 |
## Citation
|
139 |
```
|
140 |
-
|
141 |
@inproceedings{ushio-etal-2022-generative,
|
142 |
title = "{G}enerative {L}anguage {M}odels for {P}aragraph-{L}evel {Q}uestion {G}eneration",
|
143 |
author = "Ushio, Asahi and
|
|
|
33 |
metrics:
|
34 |
- name: BLEU4
|
35 |
type: bleu4
|
36 |
+
value: 11.67
|
37 |
- name: ROUGE-L
|
38 |
type: rouge-l
|
39 |
+
value: 29.49
|
40 |
- name: METEOR
|
41 |
type: meteor
|
42 |
+
value: 27.04
|
43 |
- name: BERTScore
|
44 |
type: bertscore
|
45 |
+
value: 93.07
|
46 |
- name: MoverScore
|
47 |
type: moverscore
|
48 |
+
value: 65.94
|
49 |
---
|
50 |
|
51 |
# Model Card of `lmqg/t5-base-squadshifts-new_wiki`
|
52 |
+
This model is fine-tuned version of [lmqg/t5-base-squad](https://huggingface.co/lmqg/t5-base-squad) for question generation task on the [lmqg/qg_squadshifts](https://huggingface.co/datasets/lmqg/qg_squadshifts) (dataset_name: new_wiki) via [`lmqg`](https://github.com/asahi417/lm-question-generation).
|
|
|
53 |
This model is continuously fine-tuned with [lmqg/t5-base-squad](https://huggingface.co/lmqg/t5-base-squad).
|
54 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
55 |
### Overview
|
56 |
- **Language model:** [lmqg/t5-base-squad](https://huggingface.co/lmqg/t5-base-squad)
|
57 |
- **Language:** en
|
|
|
63 |
### Usage
|
64 |
- With [`lmqg`](https://github.com/asahi417/lm-question-generation#lmqg-language-model-for-question-generation-)
|
65 |
```python
|
|
|
66 |
from lmqg import TransformersQG
|
67 |
+
|
68 |
# initialize model
|
69 |
+
model = TransformersQG(language="en", model="lmqg/t5-base-squadshifts-new_wiki")
|
70 |
+
|
71 |
# model prediction
|
72 |
+
questions = model.generate_q(list_context="William Turner was an English painter who specialised in watercolour landscapes", list_answer="William Turner")
|
73 |
|
74 |
```
|
75 |
|
76 |
- With `transformers`
|
77 |
```python
|
|
|
78 |
from transformers import pipeline
|
|
|
|
|
|
|
|
|
79 |
|
80 |
+
pipe = pipeline("text2text-generation", "lmqg/t5-base-squadshifts-new_wiki")
|
81 |
+
output = pipe("generate question: <hl> Beyonce <hl> further expanded her acting career, starring as blues singer Etta James in the 2008 musical biopic, Cadillac Records.")
|
82 |
|
83 |
+
```
|
84 |
|
85 |
+
## Evaluation
|
86 |
|
|
|
87 |
|
88 |
+
- ***Metric (Question Generation)***: [raw metric file](https://huggingface.co/lmqg/t5-base-squadshifts-new_wiki/raw/main/eval/metric.first.sentence.paragraph_answer.question.lmqg_qg_squadshifts.new_wiki.json)
|
|
|
|
|
89 |
|
90 |
+
| | Score | Type | Dataset |
|
91 |
+
|:-----------|--------:|:---------|:---------------------------------------------------------------------------|
|
92 |
+
| BERTScore | 93.07 | new_wiki | [lmqg/qg_squadshifts](https://huggingface.co/datasets/lmqg/qg_squadshifts) |
|
93 |
+
| Bleu_1 | 30.1 | new_wiki | [lmqg/qg_squadshifts](https://huggingface.co/datasets/lmqg/qg_squadshifts) |
|
94 |
+
| Bleu_2 | 20.86 | new_wiki | [lmqg/qg_squadshifts](https://huggingface.co/datasets/lmqg/qg_squadshifts) |
|
95 |
+
| Bleu_3 | 15.29 | new_wiki | [lmqg/qg_squadshifts](https://huggingface.co/datasets/lmqg/qg_squadshifts) |
|
96 |
+
| Bleu_4 | 11.67 | new_wiki | [lmqg/qg_squadshifts](https://huggingface.co/datasets/lmqg/qg_squadshifts) |
|
97 |
+
| METEOR | 27.04 | new_wiki | [lmqg/qg_squadshifts](https://huggingface.co/datasets/lmqg/qg_squadshifts) |
|
98 |
+
| MoverScore | 65.94 | new_wiki | [lmqg/qg_squadshifts](https://huggingface.co/datasets/lmqg/qg_squadshifts) |
|
99 |
+
| ROUGE_L | 29.49 | new_wiki | [lmqg/qg_squadshifts](https://huggingface.co/datasets/lmqg/qg_squadshifts) |
|
100 |
|
101 |
|
102 |
|
|
|
123 |
|
124 |
## Citation
|
125 |
```
|
|
|
126 |
@inproceedings{ushio-etal-2022-generative,
|
127 |
title = "{G}enerative {L}anguage {M}odels for {P}aragraph-{L}evel {Q}uestion {G}eneration",
|
128 |
author = "Ushio, Asahi and
|