|
--- |
|
language: |
|
- ja |
|
tags: |
|
- japanese-stablelm |
|
- causal-lm |
|
pipeline_tag: text-generation |
|
base_model: stabilityai/japanese-stablelm-base-gamma-7b |
|
license: apache-2.0 |
|
extra_gated_fields: |
|
Name: text |
|
Email: text |
|
Country: text |
|
Organization or Affiliation: text |
|
I allow Stability AI to contact me about information related to its models and research: checkbox |
|
--- |
|
|
|
|
|
# Reproduced Japanese Stable LM Instruct Gamma 7B |
|
|
|
## Model Description |
|
|
|
This is a reproduction of 7B-parameter decoder-only Japanese language model fine-tuned on instruction-following datasets, built on top of the base model [Japanese Stable LM Base Gamma 7B](https://huggingface.co/stabilityai/japanese-stablelm-base-gamma-7b). |
|
|
|
This model is trained with [notus](https://github.com/argilla-io/notus) code base. |
|
|
|
*If you are in search of the official model, please check [Japanese Stable LM Instruct Gamma 7B](https://huggingface.co/stabilityai/japanese-stablelm-instruct-gamma-7b).* |
|
|
|
|
|
|
|
## Model Details |
|
|
|
### Training Datasets |
|
|
|
- [Japanese translation of the Databricks Dolly-15k dataset](https://huggingface.co/datasets/kunishou/databricks-dolly-15k-ja) |
|
- [Japanese translation of the subset of the Anthropic HH dataset](https://huggingface.co/datasets/fujiki/japanese_hh-rlhf-49k) |
|
- [Wikinews](https://ja.wikinews.org/wi) [subset](https://huggingface.co/datasets/fujiki/llm-japanese-dataset_wikinews) of the [izumi-lab/llm-japanese-dataset](https://huggingface.co/datasets/izumi-lab/llm-japanese-dataset) |
|
|
|
|
|
### Benchmarks |
|
|
|
The result is evaluated by [Nejumi-leaderboard Neo](https://github.com/wandb/llm-leaderboard/tree/b2723944d4955768cb93c18ffe162a8ff4e88955). |
|
|
|
- llm-jp-eval: |
|
|
|
|AVG |EL |FA |MC |MR |NLI |QA |RC |chabsa|jamp |janli|jcommonsenseqa|jemhopqa|jnli |jsem |jsick|jsquad |mawps |niilc |wiki_coreference|wiki_dependency|wiki_ner|wiki_pas|wiki_reading| |
|
|------|---|----|-----|----|------|-------|-------|------|-----|-----|--------------|--------|-----|-----|-----|-------|------|------|----------------|---------------|--------|--------|------------| |
|
|0.26 |0 |0.14|0.27 |0.1 |0.302 |0.2619 |0.7464 |0.0 |0.15 |0.5 |0.27 |0.2528 |0.04 |0.67 |0.15 |0.7464 |0.1 |0.271 |0.0 |0.0 |0.0 |0.0 |0.7 | |
|
|
|
|
|
- Japanese Mt-Bench: |
|
|
|
|coding|extraction|humanities|math|reasoning|roleplay|stem|writing| |
|
|------|----------|----------|----|---------|--------|----|-------| |
|
|1.3 |1.75 |2.35 |1.45|3.4 |5.8 |4.3 |3.1 | |
|
|
|
|
|
- Overall Average: 0.283125 |
|
|
|
|
|
## Credits |
|
The training was carried out by [Hwigeon Oh](https://huggingface.co/ohwi) and [Fujiki Nakamura](https://huggingface.co/fujiki). |
