UNIST-Eunchan commited on
Commit
591097e
1 Parent(s): 0a8efd8

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +46 -17
README.md CHANGED
@@ -6,7 +6,32 @@ datasets:
6
  - arxiv-summarization
7
  model-index:
8
  - name: Paper-Summarization-ArXiv
9
- results: []
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
10
  ---
11
 
12
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
@@ -15,9 +40,22 @@ should probably proofread and complete it, then remove this comment. -->
15
  # Paper-Summarization-ArXiv
16
 
17
  This model is a fine-tuned version of [google/pegasus-x-base](https://huggingface.co/google/pegasus-x-base) on the arxiv-summarization dataset.
 
18
  It achieves the following results on the evaluation set:
19
  - Loss: 2.0127
20
 
 
 
 
 
 
 
 
 
 
 
 
 
21
  ## Model description
22
 
23
  More information needed
@@ -28,13 +66,14 @@ Paper Summarization
28
 
29
  ## Compare to Baseline
30
  - **Pegasus-X-base Zero-shot Performance:**
31
- - 'ROUGE-1': 0.062269
32
- - 'ROUGE-2': 0.007894
33
- - 'ROUGE-L': 0.046905
34
- - 'ROUGE-Lsum': 0.054591
35
  - **Our Model (Generated with length_penalty=1, num_beams=2, max_length=128*4,min_length=150, no_repeat_ngram_size= 3, top_k=25,top_p=0.95**)
36
- - {'rouge1': 0.43230484397753544, 'rouge2': 0.16657053834996538, 'rougeL': 0.2443145883290619, 'rougeLsum': 0.3393990490342812}
37
- - **Our Model**
 
 
 
 
38
  ## Training and evaluation data
39
 
40
  More information needed
@@ -55,16 +94,6 @@ The following hyperparameters were used during training:
55
  - lr_scheduler_warmup_steps: 1586
56
  - num_epochs: 5
57
 
58
- ### Training results
59
-
60
- | Training Loss | Epoch | Step | Validation Loss |
61
- |:-------------:|:-----:|:-----:|:---------------:|
62
- | 2.6153 | 1.0 | 3172 | 2.1045 |
63
- | 2.202 | 2.0 | 6344 | 2.0511 |
64
- | 2.1547 | 3.0 | 9516 | 2.0282 |
65
- | 2.132 | 4.0 | 12688 | 2.0164 |
66
- | 2.1222 | 5.0 | 15860 | 2.0127 |
67
-
68
 
69
  ### Framework versions
70
 
 
6
  - arxiv-summarization
7
  model-index:
8
  - name: Paper-Summarization-ArXiv
9
+ results:
10
+ - task:
11
+ name: Summarization
12
+ type: summarization
13
+ dataset:
14
+ name: ccdv/arxiv-summarization
15
+ type: ccdv/arxiv-summarization
16
+ config: section
17
+ split: test
18
+ args: section
19
+ metrics:
20
+ - name: ROUGE-1
21
+ type: rouge
22
+ value: 43.2305
23
+ - name: ROUGE-2
24
+ type: rouge
25
+ value: 16.6571
26
+ - name: ROUGE-L
27
+ type: rouge
28
+ value: 24.4315
29
+ - name: ROUGE-LSum
30
+ type: rouge
31
+ value: 33.9399
32
+
33
+
34
+
35
  ---
36
 
37
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
 
40
  # Paper-Summarization-ArXiv
41
 
42
  This model is a fine-tuned version of [google/pegasus-x-base](https://huggingface.co/google/pegasus-x-base) on the arxiv-summarization dataset.
43
+
44
  It achieves the following results on the evaluation set:
45
  - Loss: 2.0127
46
 
47
+ ### Training results
48
+
49
+ | Training Loss | Epoch | Step | Validation Loss |
50
+ |:-------------:|:-----:|:-----:|:---------------:|
51
+ | 2.6153 | 1.0 | 3172 | 2.1045 |
52
+ | 2.202 | 2.0 | 6344 | 2.0511 |
53
+ | 2.1547 | 3.0 | 9516 | 2.0282 |
54
+ | 2.132 | 4.0 | 12688 | 2.0164 |
55
+ | 2.1222 | 5.0 | 15860 | 2.0127 |
56
+
57
+
58
+
59
  ## Model description
60
 
61
  More information needed
 
66
 
67
  ## Compare to Baseline
68
  - **Pegasus-X-base Zero-shot Performance:**
69
+ - ROUGE-1 | ROUGE-2 | ROUGE-L | ROUGE-LSUM : 6.2269 | 0.7894 | 4.6905 | 5.4591
 
 
 
70
  - **Our Model (Generated with length_penalty=1, num_beams=2, max_length=128*4,min_length=150, no_repeat_ngram_size= 3, top_k=25,top_p=0.95**)
71
+ - ROUGE-1 | ROUGE-2 | ROUGE-L | ROUGE-LSUM : 43.2305 | 16.6571 | 24.4315 | 33.9399
72
+
73
+
74
+
75
+
76
+
77
  ## Training and evaluation data
78
 
79
  More information needed
 
94
  - lr_scheduler_warmup_steps: 1586
95
  - num_epochs: 5
96
 
 
 
 
 
 
 
 
 
 
 
97
 
98
  ### Framework versions
99