w11wo commited on
Commit
65d123a
1 Parent(s): aedbc26

Created README; included essential model information

Browse files
Files changed (1) hide show
  1. README.md +51 -0
README.md ADDED
@@ -0,0 +1,51 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language: jv
3
+ tags:
4
+ - javanese-bert-small-imdb-classifier
5
+ license: mit
6
+ datasets:
7
+ - w11wo/imdb-javanese
8
+ widget:
9
+ - text: "Dhuh Gusti, film iki elek banget. Aku getun ndelok !!!"
10
+ ---
11
+
12
+ ## Javanese BERT Small IMDB Classifier
13
+ Javanese BERT Small IMDB Classifier is a movie-classification model based on the [BERT model](https://arxiv.org/abs/1810.04805). It was trained on Javanese IMDB movie reviews.
14
+
15
+ The model was originally [`w11wo/javanese-bert-small-imdb`](https://huggingface.co/w11wo/javanese-bert-small-imdb) which is then fine-tuned on the [`w11wo/imdb-javanese`](https://huggingface.co/datasets/w11wo/imdb-javanese) dataset consisting of Javanese IMDB movie reviews. It achieved an accuracy of 76.37% on the validation dataset. Many of the techniques used are based on a Hugging Face tutorial [notebook](https://github.com/huggingface/notebooks/blob/master/examples/text_classification.ipynb) written by [Sylvain Gugger](https://github.com/sgugger).
16
+
17
+ Hugging Face's `Trainer` class from the [Transformers]((https://huggingface.co/transformers)) library was used to train the models. PyTorch was used as the backend framework during training, but the model remains compatible with TensorFlow nonetheless.
18
+
19
+ ## Model
20
+ | Model | #params | Arch. | Training/Validation data (text) |
21
+ |---------------------------------------|----------|----------------|---------------------------------|
22
+ | `javanese-bert-small-imdb-classifier` | 110M | BERT Small | Javanese IMDB (47.5 MB of text) |
23
+
24
+ ## Evaluation Results
25
+ The model was trained for 5 epochs and the following is the final result once the training ended.
26
+
27
+ | train loss | valid loss | accuracy | total time |
28
+ |------------|------------|------------|------------|
29
+ | 0.131 | 1.113 | 0.763 | 59:16 |
30
+
31
+ ## How to Use
32
+ ### As Text Classifier
33
+ ```python
34
+ from transformers import pipeline
35
+
36
+ pretrained_name = "w11wo/javanese-bert-small-imdb-classifier"
37
+
38
+ nlp = pipeline(
39
+ "sentiment-analysis",
40
+ model=pretrained_name,
41
+ tokenizer=pretrained_name
42
+ )
43
+
44
+ nlp("Film sing apik banget!")
45
+ ```
46
+
47
+ ## Disclaimer
48
+ Do consider the biases which came from the IMDB review that may be carried over into the results of this model.
49
+
50
+ ## Author
51
+ Javanese BERT Small IMDB Classifier was trained and evaluated by [Wilson Wongso](https://w11wo.github.io/). All computation and development are done on Google Colaboratory using their free GPU access.