Thomas De Decker commited on
Commit
306e18b
1 Parent(s): 0697c55
README.md CHANGED
@@ -1,3 +1,259 @@
1
  ---
 
 
2
  license: mit
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
+
3
+ language: en
4
  license: mit
5
+ tags:
6
+ - keyphrase-extraction
7
+ datasets:
8
+ - midas/kptimes
9
+ metrics:
10
+ - seqeval
11
+ widget:
12
+ - text: "Keyphrase extraction is a technique in text analysis where you extract the important keyphrases from a document.
13
+ Thanks to these keyphrases humans can understand the content of a text very quickly and easily without reading
14
+ it completely. Keyphrase extraction was first done primarily by human annotators, who read the text in detail
15
+ and then wrote down the most important keyphrases. The disadvantage is that if you work with a lot of documents,
16
+ this process can take a lot of time.
17
+
18
+ Here is where Artificial Intelligence comes in. Currently, classical machine learning methods, that use statistical
19
+ and linguistic features, are widely used for the extraction process. Now with deep learning, it is possible to capture
20
+ the semantic meaning of a text even better than these classical methods. Classical methods look at the frequency,
21
+ occurrence and order of words in the text, whereas these neural approaches can capture long-term semantic dependencies
22
+ and context of words in a text."
23
+ example_title: "Example 1"
24
+ - text: "FoodEx is the largest trade exhibition for food and drinks in Asia, with about 70,000 visitors checking out the products presented by hundreds of participating companies. I was lucky to enter as press; otherwise, visitors must be affiliated with the food industry— and pay ¥5,000 — to enter. The FoodEx menu is global, including everything from cherry beer from Germany and premium Mexican tequila to top-class French and Chinese dumplings. The event was a rare chance to try out both well-known and exotic foods and even see professionals making them. In addition to booths offering traditional Japanese favorites such as udon and maguro sashimi, there were plenty of innovative twists, such as dorayaki , a sweet snack made of two pancakes and a red-bean filling, that came in coffee and tomato flavors. While I was there I was lucky to catch the World Sushi Cup Japan 2013, where top chefs from around the world were competing … and presenting a wide range of styles that you would not normally see in Japan, like the flower makizushi above."
25
+ example_title: "Example 2"
26
+ model-index:
27
+ - name: ml6team/keyphrase-extraction-distilbert-kptimes
28
+ results:
29
+ - task:
30
+ type: keyphrase-extraction
31
+ name: Keyphrase Extraction
32
+ dataset:
33
+ type: midas/kptimes
34
+ name: kptimes
35
+ metrics:
36
+ - type: F1 (Seqeval)
37
+ value: 0.000
38
+ name: F1 (Seqeval)
39
+ - type: F1@M
40
+ value: 0.331
41
+ name: F1@M
42
  ---
43
+ # 🔑 Keyphrase Extraction Model: KBIR-KPTimes
44
+ Keyphrase extraction is a technique in text analysis where you extract the important keyphrases from a document. Thanks to these keyphrases humans can understand the content of a text very quickly and easily without reading it completely. Keyphrase extraction was first done primarily by human annotators, who read the text in detail and then wrote down the most important keyphrases. The disadvantage is that if you work with a lot of documents, this process can take a lot of time ⏳.
45
+
46
+ Here is where Artificial Intelligence 🤖 comes in. Currently, classical machine learning methods, that use statistical and linguistic features, are widely used for the extraction process. Now with deep learning, it is possible to capture the semantic meaning of a text even better than these classical methods. Classical methods look at the frequency, occurrence and order of words in the text, whereas these neural approaches can capture long-term semantic dependencies and context of words in a text.
47
+
48
+
49
+ ## 📓 Model Description
50
+ This model uses [KBIR](https://huggingface.co/distilbert-base-uncased) as its base model and fine-tunes it on the [KPTimes dataset](https://huggingface.co/datasets/midas/kptimes). KBIR or Keyphrase Boundary Infilling with Replacement is a pre-trained model which utilizes a multi-task learning setup for optimizing a combined loss of Masked Language Modeling (MLM), Keyphrase Boundary Infilling (KBI) and Keyphrase Replacement Classification (KRC).
51
+ You can find more information about the architecture in this [paper](https://arxiv.org/abs/2112.08547).
52
+
53
+ Keyphrase extraction models are transformer models fine-tuned as a token classification problem where each word in the document is classified as being part of a keyphrase or not.
54
+
55
+ | Label | Description |
56
+ | ----- | ------------------------------- |
57
+ | B-KEY | At the beginning of a keyphrase |
58
+ | I-KEY | Inside a keyphrase |
59
+ | O | Outside a keyphrase |
60
+
61
+ ## ✋ Intended Uses & Limitations
62
+ ### 🛑 Limitations
63
+ * This keyphrase extraction model is very domain-specific and will perform very well on news articles from NY Times. It's not recommended to use this model for other domains, but you are free to test it out.
64
+ * Limited amount of predicted keyphrases.
65
+ * Only works for English documents.
66
+ * For a custom model, please consult the [training notebook]() for more information.
67
+
68
+ ### ❓ How To Use
69
+ ```python
70
+ from transformers import (
71
+ TokenClassificationPipeline,
72
+ AutoModelForTokenClassification,
73
+ AutoTokenizer,
74
+ )
75
+ from transformers.pipelines import AggregationStrategy
76
+ import numpy as np
77
+
78
+ # Define keyphrase extraction pipeline
79
+ class KeyphraseExtractionPipeline(TokenClassificationPipeline):
80
+ def __init__(self, model, *args, **kwargs):
81
+ super().__init__(
82
+ model=AutoModelForTokenClassification.from_pretrained(model),
83
+ tokenizer=AutoTokenizer.from_pretrained(model),
84
+ *args,
85
+ **kwargs
86
+ )
87
+
88
+ def postprocess(self, model_outputs):
89
+ results = super().postprocess(
90
+ model_outputs=model_outputs,
91
+ aggregation_strategy=AggregationStrategy.SIMPLE,
92
+ )
93
+ return np.unique([result.get("word").strip() for result in results])
94
+
95
+ ```
96
+
97
+ ```python
98
+ # Load pipeline
99
+ model_name = "ml6team/keyphrase-extraction-kbir-kptimes"
100
+ extractor = KeyphraseExtractionPipeline(model=model_name)
101
+ ```
102
+ ```python
103
+ # Inference
104
+ text = """
105
+ Keyphrase extraction is a technique in text analysis where you extract the
106
+ important keyphrases from a document. Thanks to these keyphrases humans can
107
+ understand the content of a text very quickly and easily without reading it
108
+ completely. Keyphrase extraction was first done primarily by human annotators,
109
+ who read the text in detail and then wrote down the most important keyphrases.
110
+ The disadvantage is that if you work with a lot of documents, this process
111
+ can take a lot of time.
112
+
113
+ Here is where Artificial Intelligence comes in. Currently, classical machine
114
+ learning methods, that use statistical and linguistic features, are widely used
115
+ for the extraction process. Now with deep learning, it is possible to capture
116
+ the semantic meaning of a text even better than these classical methods.
117
+ Classical methods look at the frequency, occurrence and order of words
118
+ in the text, whereas these neural approaches can capture long-term
119
+ semantic dependencies and context of words in a text.
120
+ """.replace("\n", " ")
121
+
122
+ keyphrases = extractor(text)
123
+
124
+ print(keyphrases)
125
+
126
+ ```
127
+
128
+ ```
129
+ # Output
130
+ ['artificial intelligence']
131
+ ```
132
+
133
+ ## 📚 Training Dataset
134
+ [KPTimes](https://huggingface.co/datasets/midas/kptimes) is a keyphrase extraction/generation dataset consisting of 279,923 news articles from NY Times and 10K from JPTimes and annotated by professional indexers or editors.
135
+
136
+ You can find more information in the [paper](https://arxiv.org/abs/1911.12559).
137
+
138
+ ## 👷‍♂️ Training procedure
139
+ For more in detail information, you can take a look at the [training notebook]().
140
+
141
+ ### Training parameters
142
+
143
+ | Parameter | Value |
144
+ | --------- | ------|
145
+ | Learning Rate | 1e-4 |
146
+ | Epochs | 50 |
147
+ | Early Stopping Patience | 3 |
148
+
149
+ ### Preprocessing
150
+ The documents in the dataset are already preprocessed into list of words with the corresponding labels. The only thing that must be done is tokenization and the realignment of the labels so that they correspond with the right subword tokens.
151
+
152
+ ```python
153
+ from datasets import load_dataset
154
+ from transformers import AutoTokenizer
155
+
156
+ # Labels
157
+ label_list = ["B", "I", "O"]
158
+ lbl2idx = {"B": 0, "I": 1, "O": 2}
159
+ idx2label = {0: "B", 1: "I", 2: "O"}
160
+
161
+ # Tokenizer
162
+ tokenizer = AutoTokenizer.from_pretrained("bloomberg/KBIR")
163
+ max_length = 512
164
+
165
+ # Dataset parameters
166
+ dataset_full_name = "midas/kptimes"
167
+ dataset_subset = "raw"
168
+ dataset_document_column = "document"
169
+ dataset_biotags_column = "doc_bio_tags"
170
+
171
+ def preprocess_fuction(all_samples_per_split):
172
+ tokenized_samples = tokenizer.batch_encode_plus(
173
+ all_samples_per_split[dataset_document_column],
174
+ padding="max_length",
175
+ truncation=True,
176
+ is_split_into_words=True,
177
+ max_length=max_length,
178
+ )
179
+ total_adjusted_labels = []
180
+ for k in range(0, len(tokenized_samples["input_ids"])):
181
+ prev_wid = -1
182
+ word_ids_list = tokenized_samples.word_ids(batch_index=k)
183
+ existing_label_ids = all_samples_per_split[dataset_biotags_column][k]
184
+ i = -1
185
+ adjusted_label_ids = []
186
+
187
+ for wid in word_ids_list:
188
+ if wid is None:
189
+ adjusted_label_ids.append(lbl2idx["O"])
190
+ elif wid != prev_wid:
191
+ i = i + 1
192
+ adjusted_label_ids.append(lbl2idx[existing_label_ids[i]])
193
+ prev_wid = wid
194
+ else:
195
+ adjusted_label_ids.append(
196
+ lbl2idx[
197
+ f"{'I' if existing_label_ids[i] == 'B' else existing_label_ids[i]}"
198
+ ]
199
+ )
200
+
201
+ total_adjusted_labels.append(adjusted_label_ids)
202
+ tokenized_samples["labels"] = total_adjusted_labels
203
+ return tokenized_samples
204
+
205
+ # Load dataset
206
+ dataset = load_dataset(dataset_full_name, dataset_subset)
207
+
208
+ # Preprocess dataset
209
+ tokenized_dataset = dataset.map(preprocess_fuction, batched=True)
210
+
211
+ ```
212
+
213
+ ### Postprocessing (Without Pipeline Function)
214
+ If you do not use the pipeline function, you must filter out the B and I labeled tokens. Each B and I will then be merged into a keyphrase. Finally, you need to strip the keyphrases to make sure all unnecessary spaces have been removed.
215
+ ```python
216
+ # Define post_process functions
217
+ def concat_tokens_by_tag(keyphrases):
218
+ keyphrase_tokens = []
219
+ for id, label in keyphrases:
220
+ if label == "B":
221
+ keyphrase_tokens.append([id])
222
+ elif label == "I":
223
+ if len(keyphrase_tokens) > 0:
224
+ keyphrase_tokens[len(keyphrase_tokens) - 1].append(id)
225
+ return keyphrase_tokens
226
+
227
+
228
+ def extract_keyphrases(example, predictions, tokenizer, index=0):
229
+ keyphrases_list = [
230
+ (id, idx2label[label])
231
+ for id, label in zip(
232
+ np.array(example["input_ids"]).squeeze().tolist(), predictions[index]
233
+ )
234
+ if idx2label[label] in ["B", "I"]
235
+ ]
236
+
237
+ processed_keyphrases = concat_tokens_by_tag(keyphrases_list)
238
+ extracted_kps = tokenizer.batch_decode(
239
+ processed_keyphrases,
240
+ skip_special_tokens=True,
241
+ clean_up_tokenization_spaces=True,
242
+ )
243
+ return np.unique([kp.strip() for kp in extracted_kps])
244
+
245
+ ```
246
+
247
+ ## 📝 Evaluation Results
248
+
249
+ Traditional evaluation methods are the precision, recall and F1-score @k,m where k is the number that stands for the first k predicted keyphrases and m for the average amount of predicted keyphrases.
250
+ The model achieves the following results on the KPTimes test set:
251
+
252
+ | Dataset | P@5 | R@5 | F1@5 | P@10 | R@10 | F1@10 | P@M | R@M | F1@M |
253
+ |:-----------------:|:----:|:----:|:----:|:----:|:----:|:-----:|:----:|:----:|:----:|
254
+ | KPTimes Test Set | 0.19 | 0.35 | 0.23 | 0.10 | 0.36 | 0.15 | 0.36 | 0.36 | 0.33 |
255
+
256
+ For more information on the evaluation process, you can take a look at the keyphrase extraction [evaluation notebook]().
257
+
258
+ ## 🚨 Issues
259
+ Please feel free to start discussions in the Community Tab.
config.json ADDED
@@ -0,0 +1,38 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_name_or_path": "bloomberg/KBIR",
3
+ "architectures": [
4
+ "RobertaForTokenClassification"
5
+ ],
6
+ "attention_probs_dropout_prob": 0.1,
7
+ "bos_token_id": 0,
8
+ "classifier_dropout": null,
9
+ "eos_token_id": 2,
10
+ "gradient_checkpointing": false,
11
+ "hidden_act": "gelu",
12
+ "hidden_dropout_prob": 0.1,
13
+ "hidden_size": 1024,
14
+ "id2label": {
15
+ "0": "B-KEY",
16
+ "1": "I-KEY",
17
+ "2": "O"
18
+ },
19
+ "initializer_range": 0.02,
20
+ "intermediate_size": 4096,
21
+ "label2id": {
22
+ "B-KEY": 0,
23
+ "I-KEY": 1,
24
+ "O": 2
25
+ },
26
+ "layer_norm_eps": 1e-05,
27
+ "max_position_embeddings": 514,
28
+ "model_type": "roberta",
29
+ "num_attention_heads": 16,
30
+ "num_hidden_layers": 24,
31
+ "pad_token_id": 1,
32
+ "position_embedding_type": "absolute",
33
+ "torch_dtype": "float32",
34
+ "transformers_version": "4.18.0",
35
+ "type_vocab_size": 1,
36
+ "use_cache": true,
37
+ "vocab_size": 50265
38
+ }
merges.txt ADDED
The diff for this file is too large to render. See raw diff
 
pytorch_model.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:27dff89525c5760ed85b726024e2566f5f775ff0b8eed72389152bb93a71a3ab
3
+ size 1417389425
special_tokens_map.json ADDED
@@ -0,0 +1 @@
 
 
1
+ {"bos_token": {"content": "<s>", "single_word": false, "lstrip": false, "rstrip": false, "normalized": true}, "eos_token": {"content": "</s>", "single_word": false, "lstrip": false, "rstrip": false, "normalized": true}, "unk_token": {"content": "<unk>", "single_word": false, "lstrip": false, "rstrip": false, "normalized": true}, "sep_token": {"content": "</s>", "single_word": false, "lstrip": false, "rstrip": false, "normalized": true}, "pad_token": {"content": "<pad>", "single_word": false, "lstrip": false, "rstrip": false, "normalized": true}, "cls_token": {"content": "<s>", "single_word": false, "lstrip": false, "rstrip": false, "normalized": true}, "mask_token": {"content": "<mask>", "single_word": false, "lstrip": true, "rstrip": false, "normalized": true}}
tokenizer.json ADDED
The diff for this file is too large to render. See raw diff
 
tokenizer_config.json ADDED
@@ -0,0 +1 @@
 
 
1
+ {"errors": "replace", "bos_token": {"content": "<s>", "single_word": false, "lstrip": false, "rstrip": false, "normalized": true, "__type": "AddedToken"}, "eos_token": {"content": "</s>", "single_word": false, "lstrip": false, "rstrip": false, "normalized": true, "__type": "AddedToken"}, "sep_token": {"content": "</s>", "single_word": false, "lstrip": false, "rstrip": false, "normalized": true, "__type": "AddedToken"}, "cls_token": {"content": "<s>", "single_word": false, "lstrip": false, "rstrip": false, "normalized": true, "__type": "AddedToken"}, "unk_token": {"content": "<unk>", "single_word": false, "lstrip": false, "rstrip": false, "normalized": true, "__type": "AddedToken"}, "pad_token": {"content": "<pad>", "single_word": false, "lstrip": false, "rstrip": false, "normalized": true, "__type": "AddedToken"}, "mask_token": {"content": "<mask>", "single_word": false, "lstrip": true, "rstrip": false, "normalized": true, "__type": "AddedToken"}, "add_prefix_space": true, "trim_offsets": true, "use_fast": true, "max_length": 512, "special_tokens_map_file": null, "name_or_path": "bloomberg/KBIR", "tokenizer_class": "RobertaTokenizer"}
training_args.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:33b9936b9af770f36469bcaa654f94d8ff192f8e1b6582e32bde8468b1db9dc4
3
+ size 3055
vocab.json ADDED
The diff for this file is too large to render. See raw diff