asahi417 commited on
Commit
3e32d12
1 Parent(s): 598a086

model update

Browse files
Files changed (1) hide show
  1. README.md +25 -40
README.md CHANGED
@@ -33,44 +33,25 @@ model-index:
33
  metrics:
34
  - name: BLEU4
35
  type: bleu4
36
- value: 0.21121852916203582
37
  - name: ROUGE-L
38
  type: rouge-l
39
- value: 0.4746967055057577
40
  - name: METEOR
41
  type: meteor
42
- value: 0.23384596803152297
43
  - name: BERTScore
44
  type: bertscore
45
- value: 0.8964476409584947
46
  - name: MoverScore
47
  type: moverscore
48
- value: 0.6207232474685432
49
  ---
50
 
51
  # Model Card of `lmqg/t5-small-squad-no-answer`
52
- This model is fine-tuned version of [t5-small](https://huggingface.co/t5-small) for question generation task on the
53
- [lmqg/qg_squad](https://huggingface.co/datasets/lmqg/qg_squad) (dataset_name: default) via [`lmqg`](https://github.com/asahi417/lm-question-generation).
54
  This model is fine-tuned without answer information, i.e. generate a question only given a paragraph (note that normal model is fine-tuned to generate a question given a pargraph and an associated answer in the paragraph).
55
 
56
- Please cite our paper if you use the model ([https://arxiv.org/abs/2210.03992](https://arxiv.org/abs/2210.03992)).
57
-
58
- ```
59
-
60
- @inproceedings{ushio-etal-2022-generative,
61
- title = "{G}enerative {L}anguage {M}odels for {P}aragraph-{L}evel {Q}uestion {G}eneration",
62
- author = "Ushio, Asahi and
63
- Alva-Manchego, Fernando and
64
- Camacho-Collados, Jose",
65
- booktitle = "Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing",
66
- month = dec,
67
- year = "2022",
68
- address = "Abu Dhabi, U.A.E.",
69
- publisher = "Association for Computational Linguistics",
70
- }
71
-
72
- ```
73
-
74
  ### Overview
75
  - **Language model:** [t5-small](https://huggingface.co/t5-small)
76
  - **Language:** en
@@ -82,35 +63,40 @@ Please cite our paper if you use the model ([https://arxiv.org/abs/2210.03992](h
82
  ### Usage
83
  - With [`lmqg`](https://github.com/asahi417/lm-question-generation#lmqg-language-model-for-question-generation-)
84
  ```python
85
-
86
  from lmqg import TransformersQG
 
87
  # initialize model
88
- model = TransformersQG(language='en', model='lmqg/t5-small-squad-no-answer')
 
89
  # model prediction
90
- question = model.generate_q(list_context=["William Turner was an English painter who specialised in watercolour landscapes"], list_answer=["William Turner"])
91
 
92
  ```
93
 
94
  - With `transformers`
95
  ```python
96
-
97
  from transformers import pipeline
98
- # initialize model
99
- pipe = pipeline("text2text-generation", 'lmqg/t5-small-squad-no-answer')
100
- # question generation
101
- question = pipe('generate question: <hl> Beyonce further expanded her acting career, starring as blues singer Etta James in the 2008 musical biopic, Cadillac Records. <hl>')
102
 
103
- ```
 
104
 
105
- ## Evaluation Metrics
106
 
 
107
 
108
- ### Metrics
109
 
110
- | Dataset | Type | BLEU4 | ROUGE-L | METEOR | BERTScore | MoverScore | Link |
111
- |:--------|:-----|------:|--------:|-------:|----------:|-----------:|-----:|
112
- | [lmqg/qg_squad](https://huggingface.co/datasets/lmqg/qg_squad) | default | 0.211 | 0.475 | 0.234 | 0.896 | 0.621 | [link](https://huggingface.co/lmqg/t5-small-squad-no-answer/raw/main/eval/metric.first.sentence.paragraph_sentence.question.lmqg_qg_squad.default.json) |
113
 
 
 
 
 
 
 
 
 
 
 
114
 
115
 
116
 
@@ -137,7 +123,6 @@ The full configuration can be found at [fine-tuning config file](https://hugging
137
 
138
  ## Citation
139
  ```
140
-
141
  @inproceedings{ushio-etal-2022-generative,
142
  title = "{G}enerative {L}anguage {M}odels for {P}aragraph-{L}evel {Q}uestion {G}eneration",
143
  author = "Ushio, Asahi and
 
33
  metrics:
34
  - name: BLEU4
35
  type: bleu4
36
+ value: 21.12
37
  - name: ROUGE-L
38
  type: rouge-l
39
+ value: 47.47
40
  - name: METEOR
41
  type: meteor
42
+ value: 23.38
43
  - name: BERTScore
44
  type: bertscore
45
+ value: 89.64
46
  - name: MoverScore
47
  type: moverscore
48
+ value: 62.07
49
  ---
50
 
51
  # Model Card of `lmqg/t5-small-squad-no-answer`
52
+ This model is fine-tuned version of [t5-small](https://huggingface.co/t5-small) for question generation task on the [lmqg/qg_squad](https://huggingface.co/datasets/lmqg/qg_squad) (dataset_name: default) via [`lmqg`](https://github.com/asahi417/lm-question-generation).
 
53
  This model is fine-tuned without answer information, i.e. generate a question only given a paragraph (note that normal model is fine-tuned to generate a question given a pargraph and an associated answer in the paragraph).
54
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
55
  ### Overview
56
  - **Language model:** [t5-small](https://huggingface.co/t5-small)
57
  - **Language:** en
 
63
  ### Usage
64
  - With [`lmqg`](https://github.com/asahi417/lm-question-generation#lmqg-language-model-for-question-generation-)
65
  ```python
 
66
  from lmqg import TransformersQG
67
+
68
  # initialize model
69
+ model = TransformersQG(language="en", model="lmqg/t5-small-squad-no-answer")
70
+
71
  # model prediction
72
+ questions = model.generate_q(list_context="William Turner was an English painter who specialised in watercolour landscapes", list_answer="William Turner")
73
 
74
  ```
75
 
76
  - With `transformers`
77
  ```python
 
78
  from transformers import pipeline
 
 
 
 
79
 
80
+ pipe = pipeline("text2text-generation", "lmqg/t5-small-squad-no-answer")
81
+ output = pipe("generate question: <hl> Beyonce further expanded her acting career, starring as blues singer Etta James in the 2008 musical biopic, Cadillac Records. <hl>")
82
 
83
+ ```
84
 
85
+ ## Evaluation
86
 
 
87
 
88
+ - ***Metric (Question Generation)***: [raw metric file](https://huggingface.co/lmqg/t5-small-squad-no-answer/raw/main/eval/metric.first.sentence.paragraph_sentence.question.lmqg_qg_squad.default.json)
 
 
89
 
90
+ | | Score | Type | Dataset |
91
+ |:-----------|--------:|:--------|:---------------------------------------------------------------|
92
+ | BERTScore | 89.64 | default | [lmqg/qg_squad](https://huggingface.co/datasets/lmqg/qg_squad) |
93
+ | Bleu_1 | 53.37 | default | [lmqg/qg_squad](https://huggingface.co/datasets/lmqg/qg_squad) |
94
+ | Bleu_2 | 36.67 | default | [lmqg/qg_squad](https://huggingface.co/datasets/lmqg/qg_squad) |
95
+ | Bleu_3 | 27.4 | default | [lmqg/qg_squad](https://huggingface.co/datasets/lmqg/qg_squad) |
96
+ | Bleu_4 | 21.12 | default | [lmqg/qg_squad](https://huggingface.co/datasets/lmqg/qg_squad) |
97
+ | METEOR | 23.38 | default | [lmqg/qg_squad](https://huggingface.co/datasets/lmqg/qg_squad) |
98
+ | MoverScore | 62.07 | default | [lmqg/qg_squad](https://huggingface.co/datasets/lmqg/qg_squad) |
99
+ | ROUGE_L | 47.47 | default | [lmqg/qg_squad](https://huggingface.co/datasets/lmqg/qg_squad) |
100
 
101
 
102
 
 
123
 
124
  ## Citation
125
  ```
 
126
  @inproceedings{ushio-etal-2022-generative,
127
  title = "{G}enerative {L}anguage {M}odels for {P}aragraph-{L}evel {Q}uestion {G}eneration",
128
  author = "Ushio, Asahi and