End of training

README.md

@@ -0,0 +1,79 @@
+---
+license: mit
+base_model: rinna/japanese-roberta-base
+tags:
+- generated_from_trainer
+datasets:
+- ja_qu_ad
+model-index:
+- name: rinna-roberta-qa-ja
+  results: []
+---
+
+<!-- This model card has been generated automatically according to the information the Trainer had access to. You
+should probably proofread and complete it, then remove this comment. -->
+
+# rinna-roberta-qa-ja
+
+This model is a fine-tuned version of [rinna/japanese-roberta-base](https://huggingface.co/rinna/japanese-roberta-base) on the ja_qu_ad dataset.
+It achieves the following results on the evaluation set:
+- Loss: 1.6226
+
+## Model description
+
+More information needed
+
+## Intended uses & limitations
+
+More information needed
+
+## Training and evaluation data
+
+More information needed
+
+## Training procedure
+
+### Training hyperparameters
+
+The following hyperparameters were used during training:
+- learning_rate: 7e-05
+- train_batch_size: 2
+- eval_batch_size: 1
+- seed: 42
+- gradient_accumulation_steps: 16
+- total_train_batch_size: 32
+- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
+- lr_scheduler_type: linear
+- lr_scheduler_warmup_steps: 100
+- num_epochs: 9
+
+### Training results
+
+| Training Loss | Epoch | Step | Validation Loss |
+|:-------------:|:-----:|:----:|:---------------:|
+| 2.1602        | 0.47  | 150  | 0.9559          |
+| 1.1212        | 0.95  | 300  | 0.7738          |
+| 0.6935        | 1.42  | 450  | 0.8850          |
+| 0.6329        | 1.9   | 600  | 0.5839          |
+| 0.3988        | 2.37  | 750  | 1.0007          |
+| 0.3905        | 2.85  | 900  | 0.9119          |
+| 0.2245        | 3.32  | 1050 | 1.2299          |
+| 0.2331        | 3.79  | 1200 | 1.2494          |
+| 0.1556        | 4.27  | 1350 | 1.3691          |
+| 0.1537        | 4.74  | 1500 | 1.2580          |
+| 0.0811        | 5.22  | 1650 | 1.3939          |
+| 0.1139        | 5.69  | 1800 | 1.1321          |
+| 0.0623        | 6.17  | 1950 | 1.3723          |
+| 0.1096        | 6.64  | 2100 | 1.1039          |
+| 0.0622        | 7.11  | 2250 | 1.4657          |
+| 0.0731        | 7.59  | 2400 | 1.5731          |
+| 0.0482        | 8.06  | 2550 | 1.7190          |
+| 0.0518        | 8.54  | 2700 | 1.6226          |
+
+
+### Framework versions
+
+- Transformers 4.33.0
+- Pytorch 2.0.1+cu118
+- Datasets 2.14.4
+- Tokenizers 0.13.3

config.json

@@ -0,0 +1,28 @@
+{
+  "_name_or_path": "rinna/japanese-roberta-base",
+  "architectures": [
+    "RobertaForQuestionAnswering"
+  ],
+  "attention_probs_dropout_prob": 0.1,
+  "bos_token_id": 1,
+  "classifier_dropout": null,
+  "eos_token_id": 2,
+  "gradient_checkpointing": false,
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.1,
+  "hidden_size": 768,
+  "initializer_range": 0.02,
+  "intermediate_size": 3072,
+  "layer_norm_eps": 1e-05,
+  "max_position_embeddings": 514,
+  "model_type": "roberta",
+  "num_attention_heads": 12,
+  "num_hidden_layers": 12,
+  "pad_token_id": 3,
+  "position_embedding_type": "absolute",
+  "torch_dtype": "float32",
+  "transformers_version": "4.33.0",
+  "type_vocab_size": 2,
+  "use_cache": true,
+  "vocab_size": 32000
+}

pytorch_model.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:634ec32a48f86bd78f376013fcfa51e64c9d533f10f9149429b61c2065ebcfb3
+size 440187625

special_tokens_map.json

@@ -0,0 +1,9 @@
+{
+  "bos_token": "<s>",
+  "cls_token": "[CLS]",
+  "eos_token": "</s>",
+  "mask_token": "[MASK]",
+  "pad_token": "[PAD]",
+  "sep_token": "[SEP]",
+  "unk_token": "<unk>"
+}

spiece.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b5cbdfa8aa7c54c8c5af85b78c309c54a5f2749a20468bf6f60eee007fe6fec1
+size 805634

tokenizer_config.json

@@ -0,0 +1,17 @@
+{
+  "additional_special_tokens": [],
+  "bos_token": "<s>",
+  "clean_up_tokenization_spaces": true,
+  "cls_token": "[CLS]",
+  "do_lower_case": true,
+  "eos_token": "</s>",
+  "extra_ids": 0,
+  "legacy": true,
+  "mask_token": "[MASK]",
+  "model_max_length": 1000000000000000019884624838656,
+  "pad_token": "[PAD]",
+  "sep_token": "[SEP]",
+  "sp_model_kwargs": {},
+  "tokenizer_class": "T5Tokenizer",
+  "unk_token": "<unk>"
+}

training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7d430a3589284eb7d9e31f3b7177ac459edfb1986336d968d3299044558abf17
+size 4027