julien-c HF staff commited on
Commit
3054f43
1 Parent(s): 0f3d983

Migrate model card from transformers-repo

Browse files

Read announcement at https://discuss.huggingface.co/t/announcement-all-model-cards-will-be-migrated-to-hf-co-model-repos/2755
Original file history: https://github.com/huggingface/transformers/commits/master/model_cards/abhilash1910/financial_roberta/README.md

Files changed (1) hide show
  1. README.md +132 -0
README.md ADDED
@@ -0,0 +1,132 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ tags:
3
+ - finance
4
+ ---
5
+ # Roberta Masked Language Model Trained On Financial Phrasebank Corpus
6
+
7
+
8
+ This is a Masked Language Model trained with [Roberta](https://huggingface.co/transformers/model_doc/roberta.html) on a Financial Phrasebank Corpus.
9
+ The model is built using Huggingface transformers.
10
+ The model can be found at :[Financial_Roberta](https://huggingface.co/abhilash1910/financial_roberta)
11
+
12
+
13
+ ## Specifications
14
+
15
+
16
+ The corpus for training is taken from the Financial Phrasebank (Malo et al)[https://www.researchgate.net/publication/251231107_Good_Debt_or_Bad_Debt_Detecting_Semantic_Orientations_in_Economic_Texts].
17
+
18
+
19
+ ## Model Specification
20
+
21
+
22
+ The model chosen for training is [Roberta](https://arxiv.org/abs/1907.11692) with the following specifications:
23
+ 1. vocab_size=56000
24
+ 2. max_position_embeddings=514
25
+ 3. num_attention_heads=12
26
+ 4. num_hidden_layers=6
27
+ 5. type_vocab_size=1
28
+
29
+
30
+ This is trained by using RobertaConfig from transformers package.
31
+ The model is trained for 10 epochs with a gpu batch size of 64 units.
32
+
33
+
34
+
35
+ ## Usage Specifications
36
+
37
+
38
+ For using this model, we have to first import AutoTokenizer and AutoModelWithLMHead Modules from transformers
39
+ After that we have to specify, the pre-trained model,which in this case is 'abhilash1910/financial_roberta' for the tokenizers and the model.
40
+
41
+
42
+ ```python
43
+ from transformers import AutoTokenizer, AutoModelWithLMHead
44
+
45
+ tokenizer = AutoTokenizer.from_pretrained("abhilash1910/financial_roberta")
46
+
47
+ model = AutoModelWithLMHead.from_pretrained("abhilash1910/financial_roberta")
48
+ ```
49
+
50
+
51
+ After this the model will be downloaded, it will take some time to download all the model files.
52
+ For testing the model, we have to import pipeline module from transformers and create a masked output model for inference as follows:
53
+
54
+
55
+ ```python
56
+ from transformers import pipeline
57
+ model_mask = pipeline('fill-mask', model='abhilash1910/inancial_roberta')
58
+ model_mask("The company had a <mask> of 20% in 2020.")
59
+ ```
60
+
61
+
62
+ Some of the examples are also provided with generic financial statements:
63
+
64
+ Example 1:
65
+
66
+
67
+ ```python
68
+ model_mask("The company had a <mask> of 20% in 2020.")
69
+ ```
70
+
71
+
72
+ Output:
73
+
74
+
75
+ ```bash
76
+ [{'sequence': '<s>The company had a profit of 20% in 2020.</s>',
77
+ 'score': 0.023112965747714043,
78
+ 'token': 421,
79
+ 'token_str': 'Ġprofit'},
80
+ {'sequence': '<s>The company had a loss of 20% in 2020.</s>',
81
+ 'score': 0.021379893645644188,
82
+ 'token': 616,
83
+ 'token_str': 'Ġloss'},
84
+ {'sequence': '<s>The company had a year of 20% in 2020.</s>',
85
+ 'score': 0.0185744296759367,
86
+ 'token': 443,
87
+ 'token_str': 'Ġyear'},
88
+ {'sequence': '<s>The company had a sales of 20% in 2020.</s>',
89
+ 'score': 0.018143286928534508,
90
+ 'token': 428,
91
+ 'token_str': 'Ġsales'},
92
+ {'sequence': '<s>The company had a value of 20% in 2020.</s>',
93
+ 'score': 0.015319528989493847,
94
+ 'token': 776,
95
+ 'token_str': 'Ġvalue'}]
96
+ ```
97
+
98
+ Example 2:
99
+
100
+ ```python
101
+ model_mask("The <mask> is listed under NYSE")
102
+ ```
103
+
104
+ Output:
105
+
106
+ ```bash
107
+ [{'sequence': '<s>The company is listed under NYSE</s>',
108
+ 'score': 0.1566661298274994,
109
+ 'token': 359,
110
+ 'token_str': 'Ġcompany'},
111
+ {'sequence': '<s>The total is listed under NYSE</s>',
112
+ 'score': 0.05542507395148277,
113
+ 'token': 522,
114
+ 'token_str': 'Ġtotal'},
115
+ {'sequence': '<s>The value is listed under NYSE</s>',
116
+ 'score': 0.04729423299431801,
117
+ 'token': 776,
118
+ 'token_str': 'Ġvalue'},
119
+ {'sequence': '<s>The order is listed under NYSE</s>',
120
+ 'score': 0.02533523552119732,
121
+ 'token': 798,
122
+ 'token_str': 'Ġorder'},
123
+ {'sequence': '<s>The contract is listed under NYSE</s>',
124
+ 'score': 0.02087237872183323,
125
+ 'token': 635,
126
+ 'token_str': 'Ġcontract'}]
127
+ ```
128
+
129
+
130
+ ## Resources
131
+
132
+ For all resources , please look into the [HuggingFace](https://huggingface.co/) Site and the [Repositories](https://github.com/huggingface).